Limitations of ICMP Echo for Network Measurement

Many people assume that ICMP Echo, more familiar as "ping", is a valid tool for measuring the performance and behavior of the internet.

That assumption is incorrect. Ping is a tool of value mainly for determining whether connectivity exists or not. Ping is a weak tool for measuring delay, variation in delay (jitter), and throughput.

What are ICMP and ICMP Echo?

ICMP is the Internet Control Message Protocol. It is described in RFC 792. It is a full internet standard.

ICMP defines a number of packets. These packets are most frequently used to indicate operational issues at the Internet Protocol (RFC 791) layer. ICMP messages indicate, for example, that a packet should take another path (ICMP Redirect) or that a service is not available or that IP fragmentation would be necessary (ICMP Destination Unreachable). Other ICMP messages indicate that an IP packet was discarded because its time to live has been exceed (ICMP Time Exceeded).

Two of these ICMP messages are ICMP Echo Request and ICMP Echo Reply. As their names indicate these packets form a stimulus-response pair. These packets are Ping.

How does Ping work?

Ping is very simple:

The client computer brews up an ICMP Echo Request packet and sends it to the target computer.
The target machine, upon receipt of the ICMP Echo Request builds a corresponding ICMP Echo Reply packet and sends it back to the client computer.

The client can build an Echo Request packet of any size that is valid for an IP packet - up to about 64K bytes. However, most Ping programs default to 56 bytes of data which, when ICMP and IP headers are added, results in an IP packet of 64 bytes.

Most clients, by default, do not set any of the IP type-of-service bits.

The amount of processing done by the target computer to process an Echo Request and turn it into an Echo Reply is very small and it is often, but not always done, in the kernel. Kernel processing is usually very fast and does not incur the overhead of scheduling a non-kernel process to handle the received Echo Request.

The target computer may impose a limit on how many ICMP Echo Requests it will process per second. This is a self-protective mechanism to ward off the effects of denial of service attacks. Those Echo Requests that exceed the limits are simply and silently discarded. Similarly, routers on the network paths between the client and target may impose rate limits on how many ICMP packets they will carry per second. Echo Request and Echo Reply packets that exceed those limits are simply and silently discarded.

What does Ping do well?

Ping is very good at doing a basic reachability test; ping is a lightweight test of whether an IP packet can go from a client. Even when used this way, ping can fail - if there are ICMP Echo filters along the path or if there are rate limiters clamping the number of ICMP packets then ping might fail.

What does Ping do poorly?

Ping receives lower priority treatment, causing delays.
Ping packets differ in size from TCP or UDP packets, leading to varying network experiences.
Ping is a poor indicator of how networks respond to different protocols.
Sophisticated router mechanisms can treat Ping packets differently, causing misleading results.
Ping's performance isn't affected by self-limiting behaviors of certain flows, presenting a flawed network condition.
ICMP Echo packets may follow different paths than actual data, leading to errors.
Ping reflects round trip time, not the path's bandwidth, making it unsuitable for throughput estimation.
Network characterization based on Ping is likely to be inaccurate.
Emulating a network using Ping often reinforces measurement errors.
Over-reliance on Ping can lead to the "Garbage In, Garbage Out" (GIGO) scenario.

ICMP echo packets are often given second-class treatment by routers and target hosts. This means that ICMP Echo Requests and Responses may be backed up behind traffic that is considered more important. And if resources are really at the point of starvation, then a router or target host is often more likely to dispose of a Ping than other kinds of network traffic.

This means that the round-trip time reflected by Ping can easily be rather different than what would be experienced by other, higher priority, kinds of traffic.

In addition, because Pings are often small packets their experience as they wend their way through the net can be quite different than that experienced by fully laden TCP or UDP packets.

An easy way to think of this is to consider the experiences of a person riding a bicycle and a person driving an 18-wheel truck as they try to go through a major city. Each will experience a different set of delays. And what is a fast pathway for one (e.g. a narrow alley for a bicycle or an interstate highway for the truck) may not be useful for the other.

Data packets of various sizes, various priorities, and even of various protocols tend to experience the net differently. Simple ICMP Echo Request/Reply exchanges often are very inaccurate indicators of how the net will appear to other protocols.

Many routers have sophisticated mechanisms in which different packet flows are identified and the packets from each flow are scheduled or discarded according to "fair queuing" criteria. Other router mechanisms, such as RED, actually will discard packets from flows that appear to be using more than their fair share of network resources.

Host computers and routers have "traffic shapers" that control the pacing of outbound packets. For example, a typical traffic shaper will limit individual flows or some aggregation of flows so that they send their packets no faster than some given rate. Traffic shaping doesn't even kick-in until enough packets are flowing. This means that isolated packets, such as Pings, may not experience the traffic pacing that is applied to real application packets.

Certain types of packet flows may engage in self-limiting behavior. For example, TCP connections, because good TCP implementations obey "good citizen" rules of slow-start and congestion avoidance, can easily have seemingly poor throughput as the result of self-imposed slowdowns while Pings show no problems at all.

Network carriers, often under pressure to provide low-delay/low-jitter pathways for Voice over IP (VOIP), are beginning to engineer their networks so that different kinds of traffic take different routes through the net. In such situations, ICMP Echo packets may not take the same path as the kinds of application data one is concerned with. That, in turn, can lead to erroneous results.

Streaming protocols such as TCP can be strongly affected by the bandwidth of a network path. Ping tends to reflect merely the round trip time of a path and not the potential bandwidth of that path. Thus ping tends to be a poor indicator of the kind of throughput that can be obtained on a TCP connection between the client and the target.

A network characterization or baseline based on ICMP Echo (Ping) is likely to be inaccurate. A more accurate characterization or baseline requires measurements of actual traffic flows or of synthetic flows of the same protocols, nature, and dynamics as the actual traffic.

Moreover, any attempt to emulate a network based on ICMP Echo (Ping) measurements will tend to reinforce the measurement errors that come from using ICMP Echo.

Ping is a useful tool in its own right, but like any tool it has its limits. And blind dependence of Ping is likely to make one remember the old computer cliché and acronym: Garbage In, Garbage Out, GIGO.

What are the alternatives?

PCHAR: One of the most interesting tools for evaluating the status of a pathway between here and there is called "pchar."

Pchar is an open-source tool that is available at kitchenlab.org/www/bmah/Software/pchar

Note: Pchar appears to have been neglected for the past few years and it may not compile on all system, however work is underway to publish an updated version.

Pchar is not fast - it sends approximately 1500 packets for each hop of the pathway and does so over an extended period of time that can easily exceed an hour.

TTCP: Another alternative is "ttcp" which, despite its name is for both TCP and UDP. Ttcp usually has to be running at both ends of the connection. Ttcp comes in in various forms. It is often included with an operating system distribution (e.g. Fedora or Suse Linux). The following pointers may be useful:

LEARN MORE