Delay, throughput and bandwidth-delay product

Two numbers describe how a network performs for you: how long a packet takes to arrive (delay) and how many bits a second actually get through (throughput). At every hop a packet meets four delays: processing in the router, queueing behind other packets, transmission, the time to put its bits onto the link, and propagation, the time the signal needs to travel the length of the link. Throughput is limited by the slowest link on the path, the bottleneck, and often by the sender too. Their meeting point is the bandwidth-delay product: the bottleneck rate times the round-trip time, which is how much data must be on its way at any moment to keep the path busy. This lesson works out each delay with numbers, shows with a simulation why queueing delay explodes as a link fills up, and sizes the data in flight for real paths.

The four delays of one hop

• Processing delay is the router’s own work: checking the header and looking up the next hop. It is usually measured in microseconds.
• Queueing delay is the time a packet waits behind earlier packets for the same outgoing link. It is zero on an idle link and can reach seconds on an overloaded one, so it is the delay that varies.
• Transmission delay is the packet’s length divided by the link rate: L / R. A 1,500-byte packet is 12,000 bits, so it needs 120 µs at 100 Mb/s and 1.2 µs at 10 Gb/s.
• Propagation delay is the link’s length divided by the speed of the signal. Light in fibre covers a kilometre in about 5 µs; that is the planning value ITU-T G.114 gives for optical fibre systems, roughly two-thirds of the speed of light in a vacuum.

Transmission and propagation are easy to mix up. Transmission depends on the packet and the rate but not on the distance: it is how long the router takes to push the packet out. Propagation depends on the distance but not on the packet: it is how long each bit then spends on the wire. The delay of one hop is the sum of the four, and a path adds up its hops.

Bits, bytes and prefixes

Link rates are counted in bits a second with decimal prefixes: 1 Mb/s is 1,000,000 bits a second. Memory and many file managers count bytes with binary prefixes, which NIST spells MiB and GiB: 1 MiB is 2²⁰ = 1,048,576 bytes. A byte is 8 bits. Mixing the three is the most common way to get a delay question wrong:

# Link rates use decimal prefixes (1 Mb/s = 1,000,000 bits a second); memory and many file
# managers use binary ones (1 MiB = 1,048,576 bytes). A byte is 8 bits.

RATE = 100e6  # a "100 Mb/s" link

print(f"100 Mb/s = {RATE / 8 / 1e6:.2f} MB/s = {RATE / 8 / 2**20:.2f} MiB/s")

for size, label in ((100e6, "100 MB"), (100 * 2**20, "100 MiB")):
    print(f"{label:>7} = {size:>13,.0f} bytes: {size * 8 / RATE:.2f} s at 100 Mb/s")

# The classic slip: dividing bytes by a rate in bits a second.
print(f"forgetting the factor 8 gives {100e6 / RATE:.2f} s for 100 MB")

100 Mb/s = 12.50 MB/s = 11.92 MiB/s
 100 MB =   100,000,000 bytes: 8.00 s at 100 Mb/s
100 MiB =   104,857,600 bytes: 8.39 s at 100 Mb/s
forgetting the factor 8 gives 1.00 s for 100 MB

A 100 MiB file is 4.9 % bigger than a 100 MB one, and dividing a size in bytes by a rate in bits a second makes the answer eight times too small.

A path, delay by delay

The example follows one 1,500-byte packet from a home in Pune to a server in Mumbai, and on to Singapore. The route lengths are assumptions for the example (cables follow roads and coasts, so they are longer than a straight line), and each router is assumed to need 20 µs and to have an empty queue:

# The delays of one 1,500-byte packet along two paths that start in Pune. The route lengths
# are assumptions for the example (cables do not run in straight lines), and the routers are
# assumed idle, so there is no queueing. Light in fibre: 5 microseconds a kilometre (ITU-T G.114).

PACKET_BITS = 1_500 * 8
FIBRE_S_PER_KM = 5e-6
PROCESSING = 20e-6  # assumed 20 microseconds in each router, for the example

paths = {
    "Pune to Mumbai": [
        ("home fibre to the ISP", 100e6, 2),
        ("ISP to Mumbai", 10e9, 150),
    ],
    "Pune to Singapore": [
        ("home fibre to the ISP", 100e6, 2),
        ("ISP to Mumbai", 10e9, 150),
        ("submarine cable to Singapore", 100e9, 5_000),
    ],
}

for name, links in paths.items():
    print(name)
    one_way = 0.0
    for label, rate, km in links:
        transmit = PACKET_BITS / rate
        propagate = km * FIBRE_S_PER_KM
        one_way += transmit + propagate + PROCESSING
        print(f"  {label:29} {rate / 1e6:>7,.0f} Mb/s {km:>5,} km  transmit {transmit * 1e6:8.2f} us  propagate {propagate * 1e6:9.1f} us")
    print(f"  one way {one_way * 1e3:.3f} ms, round trip about {2 * one_way * 1e3:.3f} ms")

Pune to Mumbai
  home fibre to the ISP             100 Mb/s     2 km  transmit   120.00 us  propagate      10.0 us
  ISP to Mumbai                  10,000 Mb/s   150 km  transmit     1.20 us  propagate     750.0 us
  one way 0.921 ms, round trip about 1.842 ms
Pune to Singapore
  home fibre to the ISP             100 Mb/s     2 km  transmit   120.00 us  propagate      10.0 us
  ISP to Mumbai                  10,000 Mb/s   150 km  transmit     1.20 us  propagate     750.0 us
  submarine cable to Singapore  100,000 Mb/s 5,000 km  transmit     0.12 us  propagate   25000.0 us
  one way 25.941 ms, round trip about 51.883 ms

Two patterns stand out:

• On the slow access link, transmission dominates: 120 µs, against 10 µs of propagation over 2 km. Upgrading that link is what makes the packet leave the house sooner.
• On long links, propagation dominates: the 5,000 km to Singapore take 25 ms, and at 100 Gb/s transmission costs a tenth of a microsecond. No upgrade of the link rate shortens that; only a shorter route or a closer server does. That is why content is cached near its readers.

Queueing delay and traffic intensity

Queueing delay depends on how busy the link is. If packets of L bits arrive at an average rate of a packets a second at a link of R bits a second, the traffic intensity is the share of the link’s time they need:

Above 1, packets arrive faster than the link can send them, so the queue grows until the buffer is full and packets are dropped. Below 1, the queue empties from time to time, but random arrivals still bunch up. The example simulates one router queue in front of a 10 Mb/s link: 1,500-byte packets, each needing 1.2 ms, arriving at random:

# One router queue in front of a 10 Mb/s link. Packets are 1,500 bytes, so each takes
# 1.2 ms to send; they arrive at random (exponential gaps), at an average rate a.
# Traffic intensity I = L * a / R is the share of the link's time the packets need.
import random
from collections import deque

SERVICE = 1_500 * 8 / 10e6  # 1.2 ms per packet
PACKETS = 300_000


def simulate(intensity, seed):
    rnd = random.Random(seed)
    rate = intensity / SERVICE  # packets a second
    clock = link_free = waited = 0.0
    starts = deque()  # when the packets still in the queue will start being sent
    seen = 0  # packets waiting, summed over what each arrival finds
    for _ in range(PACKETS):
        clock += rnd.expovariate(rate)
        while starts and starts[0] <= clock:
            starts.popleft()
        seen += len(starts)
        start = max(clock, link_free)
        waited += start - clock
        link_free = start + SERVICE
        starts.append(start)
    return rate, waited / PACKETS, seen / PACKETS, len(starts)


def formula(intensity):
    """Average wait for random arrivals and fixed-size packets (Pollaczek-Khinchine)."""
    return intensity * SERVICE / (2 * (1 - intensity))


print("   I  wait (ms)  formula (ms)  packets waiting  rate x wait")
for seed, intensity in enumerate((0.5, 0.8, 0.9, 0.95)):
    rate, wait, waiting, _ = simulate(intensity, seed)
    print(f"{intensity:4}  {wait * 1e3:9.2f}  {formula(intensity) * 1e3:12.2f}  {waiting:15.2f}  {rate * wait:11.2f}")

runs = [simulate(0.99, seed)[1] * 1e3 for seed in range(4)]
print(f"I = 0.99, four runs: {', '.join(f'{w:.1f}' for w in runs)} ms (formula {formula(0.99) * 1e3:.1f} ms)")

_, wait, _, left = simulate(1.1, seed=9)
print(f"I = 1.1: average wait {wait:.1f} s, and {left:,} packets are still queued at the end")

   I  wait (ms)  formula (ms)  packets waiting  rate x wait
 0.5       0.60          0.60             0.25         0.25
 0.8       2.40          2.40             1.60         1.60
 0.9       5.41          5.40             4.06         4.05
0.95      11.12         11.40             8.79         8.80
I = 0.99, four runs: 51.3, 62.7, 63.8, 122.8 ms (formula 59.4 ms)
I = 1.1: average wait 16.7 s, and 27,593 packets are still queued at the end

At half load a packet waits 0.6 ms on average, half a packet time. At 90 % load it waits 5.4 ms, four and a half packet times, and at 95 % about 11 ms. The formula column is the Pollaczek-Khinchine result for random arrivals and fixed-size packets, I × (L/R) / (2(1 − I)), and the simulation follows it closely up to 95 %. Close to full load the average is not only large but unstable: four runs of 300,000 packets at 99 % give 51 to 123 ms around a long-run average of 59 ms. At 110 % the queue never stops growing: the average wait reaches 16.7 s and 27,593 packets are still queued when the run ends.

The last two columns check Little’s law: in any stable queue, the average number of packets in it equals their arrival rate times the average time each spends there, N = λT. The column “packets waiting” counts what each new arrival finds in the queue; with random arrivals that equals the queue’s average over time (the “Poisson arrivals see time averages” property), and it stays within 0.01 of rate × wait. Little’s law needs no assumption about the arrival pattern, so you can use it on real measurements: a router that receives 4,000 packets a second and holds each for 2 ms has 8 packets inside on average.

Throughput: the bottleneck link

Throughput is the rate at which data arrives, end to end. For one transfer with the path to itself, it is the rate of the slowest link: a laptop on 100 Mb/s Wi-Fi gets at most 100 Mb/s from a server on a 10 Gb/s link. The bottleneck need not be the first or the last link, and throughput can fall below even the bottleneck rate:

• Sharing. Flows that cross the same link share it; ten busy downloads over one 100 Mb/s link get about 10 Mb/s each.
• The sender’s window. A protocol that may send at most W bytes before an acknowledgement comes back can deliver at most W bytes per round trip, whatever the links allow: throughput ≤ W / RTT.
• Overheads. Headers, acknowledgements and retransmitted packets use link capacity without carrying new data.

The window limit is where delay and throughput meet.

The bandwidth-delay product

Think of the path as a pipe: its width is the bottleneck rate R and its length is the round-trip time. The data that fills it is the bandwidth-delay product:

A sender that wants the full rate must have a BDP’s worth of data sent but not yet acknowledged at every moment. At 100 Mb/s and a 50 ms round trip that is 5,000,000 bits, or 625,000 bytes. TCP’s window field is 16 bits, so a receiver can offer at most 65,535 bytes without the window scale option of RFC 7323, which multiplies the window by 2 to the power of a shift count of at most 14 (windows up to about 1 GiB). The example sizes a few paths:

# Bandwidth-delay product: the bits a sender must have in flight to keep a path busy.
# Without window scaling a TCP receiver can offer at most 65,535 bytes (a 16-bit field);
# RFC 7323 multiplies that by 2**shift, with a shift of at most 14.

MAX_WINDOW = 65_535

paths = [  # (path, bottleneck in bit/s, round-trip time in s); example figures
    ("Wi-Fi to a nearby server", 100e6, 0.010),
    ("100 Mb/s at 50 ms", 100e6, 0.050),
    ("1 Gb/s, India to Singapore", 1e9, 0.060),
    ("geostationary satellite hop", 50e6, 0.520),
]

print(f"{'path':28} {'BDP bytes':>12}  {'65,535-byte window':>18}  shift needed")
for name, rate, rtt in paths:
    bdp = rate * rtt / 8
    capped = min(rate, MAX_WINDOW * 8 / rtt)  # throughput <= window / RTT
    shift = 0
    while MAX_WINDOW * 2**shift < bdp:
        shift += 1
    print(f"{name:28} {bdp:12,.0f}  {capped / 1e6:13.1f} Mb/s  {shift:12}")

path                            BDP bytes  65,535-byte window  shift needed
Wi-Fi to a nearby server          125,000           52.4 Mb/s             1
100 Mb/s at 50 ms                 625,000           10.5 Mb/s             4
1 Gb/s, India to Singapore      7,500,000            8.7 Mb/s             7
geostationary satellite hop     3,250,000            1.0 Mb/s             6

The round-trip times are example figures, except the satellite hop: ITU-T G.114 gives 260 ms one way between earth stations for a satellite 36,000 km up, so at least 520 ms for a round trip. A 65,535-byte window over 50 ms delivers only 10.5 Mb/s on a 100 Mb/s path, and over the satellite hop 1.0 Mb/s of 50. The last column is the smallest shift that opens the window wide enough.

Bufferbloat: when a full queue becomes the delay

Router buffers absorb bursts, but a buffer that is too big and stays full adds delay without adding throughput. A 1 MB buffer in front of a 10 Mb/s uplink holds 8,000,000 bits, which take 0.8 s to drain, so every packet that joins the back of a full queue waits most of a second. A large upload can keep such a queue full for as long as it runs, and every video call and game packet that shares the uplink waits behind it. RFC 7567, the IETF’s best current practice on queue management, calls keeping the steady-state queue small “perhaps the most important goal for queue management”, and recommends that network devices drop or mark packets before the queue is full (active queue management) so that senders slow down early.

You can see bufferbloat on your own connection by measuring latency twice: once on an idle line and once while a large download or upload is running. A jump from tens to hundreds of milliseconds is a full queue.

Exercise: three numbers for a path

FIBRE_S_PER_KM = 5e-6  # propagation delay in fibre: 5 microseconds a kilometre


def path_metrics(links, packet_bytes):
    """Return {"delay": seconds, "throughput": bits a second, "bdp": bytes} for a path of (rate, km) links."""
    delay = 0.0
    throughput = 0.0
    bdp = 0.0
    # Work out the three numbers here.
    return {"delay": delay, "throughput": throughput, "bdp": bdp}

import math

from metrics import path_metrics


def close(a, b):
    return math.isclose(a, b, rel_tol=1e-9, abs_tol=1e-12)


def test_one_link():
    """a 2 km link of 100 Mb/s and a 1,500-byte packet"""
    m = path_metrics([(100e6, 2)], 1_500)
    assert close(m["delay"], 0.00013)
    assert close(m["throughput"], 100e6)
    assert close(m["bdp"], 250)


def test_pune_to_mumbai():
    """the lesson's first path, without the routers' processing time"""
    m = path_metrics([(100e6, 2), (10e9, 150)], 1_500)
    assert close(m["delay"], 0.0008812)
    assert close(m["throughput"], 100e6)
    assert close(m["bdp"], 19_000)


def test_bottleneck_in_the_middle():
    """the slowest link sets the throughput wherever it is"""
    m = path_metrics([(1e9, 10), (50e6, 100), (1e9, 10)], 1_000)
    assert close(m["delay"], 0.000776)
    assert close(m["throughput"], 50e6)
    assert close(m["bdp"], 7_500)


def test_long_haul():
    """6,000 km at 1 Gb/s: propagation dominates and the pipe holds megabytes"""
    m = path_metrics([(1e9, 6_000)], 1_500)
    assert close(m["delay"], 0.030012)
    assert close(m["bdp"], 7_500_000)


def test_keys():
    """returns exactly the three numbers"""
    assert set(path_metrics([(1e6, 1)], 100)) == {"delay", "throughput", "bdp"}

Interview questions

What is the bandwidth-delay product, and why does it matter? (warm-up)

It is the bottleneck rate of a path times its round-trip time: the amount of data that is in flight, sent but not yet acknowledged, when the path is full. A sender that keeps less than that in flight leaves the path idle part of every round trip, so its throughput is at most its window divided by the round-trip time. For example, 100 Mb/s with a 50 ms round trip gives 625,000 bytes, ten times the 65,535 bytes TCP can offer without window scaling, so an unscaled connection would reach only about 10.5 Mb/s. The product therefore tells you the window and the buffers a fast, long path needs.

Key takeaways

• Each hop adds processing, queueing, transmission (L / R) and propagation (distance / speed) delay; light in fibre needs about 5 µs a kilometre.
• Transmission depends on packet size and rate, propagation on distance; slow access links are dominated by the first, long-haul links by the second.
• Traffic intensity I = La / R: queueing delay grows without bound as I approaches 1, is unstable close to it, and never settles above it.
• Little’s law, N = λT, links queue length, arrival rate and time in the system for any stable queue.
• Throughput is at most the bottleneck rate, and at most window / RTT; sharing and overheads lower it further.
• The bandwidth-delay product R × RTT is the data in flight a path needs; above 65,535 bytes, TCP needs window scaling.
• Oversized, full buffers add delay without adding throughput (bufferbloat); active queue management keeps queues short.