Inside the IPv4 header

Every IPv4 datagram begins with a header of at least 20 bytes. It tells each router and the receiving host which version of IP this is, how long the header and the whole datagram are, how to rebuild the datagram if it was split into fragments, how many more routers may forward it, which protocol its data belongs to, and where it comes from and goes to, with a checksum that protects the header itself. Routers read the header at every hop and change two of its fields. Everything else about the datagram, from TCP and UDP to the web page inside, rides in the data that follows.

This lesson reads one complete header byte by byte in Python, explains each field and the mistakes people make with it, and follows a datagram’s time to live across a chain of routers.

The header at a glance

The diagram draws the header 16 bits wide rather than the usual 32. Each row is then one 16-bit word, the unit the header checksum adds up, and the byte offset of every field can be read down the left side.

Reading a header byte by byte

Here are the first 20 bytes of a small UDP datagram, a name lookup sent from 192.0.2.10 to 198.51.100.53. Both addresses come from the blocks that RFC 5737 sets aside for documentation, so they never belong to a real host. Python’s struct module unpacks the bytes in network byte order (the ! in the format string, big-endian), and shifts and masks split the bytes that hold two fields:

"""Read the 20-byte IPv4 header of one UDP datagram, field by field, and check it."""
import ipaddress
import struct

# The first 20 bytes of a small UDP datagram (a name lookup), two hex digits per byte.
HEADER = bytes.fromhex("45 00 00 40 8c 1f 40 00 40 11 c2 1a c0 00 02 0a c6 33 64 35")

PROTOCOLS = {1: "ICMP", 6: "TCP", 17: "UDP"}  # a few entries of the IANA protocol numbers registry

# ! = network byte order (big-endian). B = 1 byte, H = 2 bytes, 4s = 4 raw bytes.
(ver_ihl, tos, total_length, ident, flags_frag,
 ttl, protocol, checksum, src, dst) = struct.unpack("!BBHHHBBH4s4s", HEADER)

version = ver_ihl >> 4          # high 4 bits
ihl = ver_ihl & 0x0F            # low 4 bits: the header length in 32-bit words
dscp = tos >> 2                 # high 6 bits of the second byte
ecn = tos & 0b11                # low 2 bits
df = (flags_frag >> 14) & 1     # the 3 flag bits: reserved, DF, MF
mf = (flags_frag >> 13) & 1
offset = flags_frag & 0x1FFF    # low 13 bits, counted in 8-byte units

print(f"version          {version}")
print(f"IHL              {ihl} words = {ihl * 4} bytes")
print(f"DSCP, ECN        {dscp}, {ecn:02b}")
print(f"total length     {total_length} bytes ({total_length - ihl * 4} bytes after the header)")
print(f"identification   0x{ident:04x}")
print(f"flags            DF={df} MF={mf}")
print(f"fragment offset  {offset}")
print(f"time to live     {ttl}")
print(f"protocol         {protocol} ({PROTOCOLS.get(protocol, 'other')})")
print(f"header checksum  0x{checksum:04x}")
print(f"source           {ipaddress.IPv4Address(src)}")
print(f"destination      {ipaddress.IPv4Address(dst)}")

# The receiver's check: the ones' complement sum of all ten words, checksum included.
total = 0
for (word,) in struct.iter_unpack("!H", HEADER[: ihl * 4]):
    total += word
    total = (total & 0xFFFF) + (total >> 16)   # add the carry back in at the bottom
print(f"check            sum of the ten words = 0x{total:04x} ({'intact' if total == 0xFFFF else 'damaged'})")

version          4
IHL              5 words = 20 bytes
DSCP, ECN        0, 00
total length     64 bytes (44 bytes after the header)
identification   0x8c1f
flags            DF=1 MF=0
fragment offset  0
time to live     64
protocol         17 (UDP)
header checksum  0xc21a
source           192.0.2.10
destination      198.51.100.53
check            sum of the ten words = 0xffff (intact)

Read the output against the table. The first byte, 0x45, holds two fields at once: the version in its high four bits and the IHL in its low four. An IHL of 5 means five 32-bit words, so 20 bytes, and with a total length of 64 the remaining 44 bytes are the UDP header and the lookup itself. DF is set and MF is not, the usual combination for a datagram that was never split. The last line repeats what a receiver does: it adds up every 16-bit word of the header, checksum included, and an undamaged header always sums to 0xffff.

The fields, group by group

Version and IHL

The version is 4 for every IPv4 datagram; IPv6 puts 6 in the same four bits, so a receiver can tell the two apart from the first byte. The IHL counts the header in 32-bit words, and RFC 791 makes 5 the smallest correct value because the fixed part alone is five words. The largest four-bit value, 15, allows 60 bytes, so options can add at most 40 bytes. The two length fields use different units, which is the classic trap: IHL is in words, total length is in bytes.

DSCP and ECN

The second byte began life in RFC 791 as “type of service”. RFC 2474 redefined its high six bits as the Differentiated Services Code Point (DSCP), which selects how routers that support differentiated services treat the datagram, and RFC 3168 gave the low two bits to Explicit Congestion Notification (ECN). Splitting the byte shows both:

"""Split the second header byte into its two fields: DSCP (6 bits) and ECN (2 bits)."""

ECN = {0b00: "Not-ECT (sender cannot use ECN)",
       0b01: "ECT(1) (ECN-capable)",
       0b10: "ECT(0) (ECN-capable)",
       0b11: "CE (congestion experienced)"}


def dscp_name(dscp):
    if dscp == 0:
        return "default forwarding"
    if dscp == 46:
        return "Expedited Forwarding (low delay)"
    if dscp & 0b111 == 0:
        return f"class selector {dscp >> 3}"     # xxx000: the old IP precedence values
    return "other"


print("byte  binary     DSCP  meaning                           ECN")
for tos in (0x00, 0xB8, 0xC0, 0x02, 0x03):
    dscp, ecn = tos >> 2, tos & 0b11
    print(f"0x{tos:02x}  {tos >> 2:06b} {ecn:02b}  {dscp:>4}  {dscp_name(dscp):<32}  {ECN[ecn]}")

# A router that sees congestion on an ECN-capable packet marks it instead of dropping it:
marked = 0x02 | 0b11
print(f"0x02 marked by a congested router becomes 0x{marked:02x}: {ECN[marked & 0b11]}")

byte  binary     DSCP  meaning                           ECN
0x00  000000 00     0  default forwarding                Not-ECT (sender cannot use ECN)
0xb8  101110 00    46  Expedited Forwarding (low delay)  Not-ECT (sender cannot use ECN)
0xc0  110000 00    48  class selector 6                  Not-ECT (sender cannot use ECN)
0x02  000000 10     0  default forwarding                ECT(0) (ECN-capable)
0x03  000000 11     0  default forwarding                CE (congestion experienced)
0x02 marked by a congested router becomes 0x03: CE (congestion experienced)

A DSCP of 0 is ordinary best-effort forwarding, the default that RFC 2474 recommends. 46 is the code point that RFC 3246 recommends for Expedited Forwarding, the class for traffic such as voice that needs low loss, low delay and low jitter. Values ending in three zero bits are class selectors, kept so that the old precedence bits still mean something. The ECN values tell routers that the sender can react to congestion signals: rather than dropping such a packet when a queue builds up, a router can change its ECN bits to CE, and the receiver passes the signal back to the sender, which slows down.

Total length

Total length counts the whole datagram in bytes, header included, so the largest IPv4 datagram is 65,535 bytes. Real datagrams are much smaller, because each must fit the links it crosses or be split into fragments. RFC 791 asks every host to accept datagrams of at least 576 bytes, whether they arrive whole or in pieces.

Identification, flags and fragment offset

These three fields exist for fragmentation, the subject of the next lesson. When a datagram is too big for the next link, it can be cut into fragments that share its identification value; MF is set on every fragment but the last, and the 13-bit fragment offset says where each piece’s data starts, counted in 8-byte units. DF asks routers not to fragment at all. RFC 6864 tightened the rules: the identification field may be used for nothing but fragmentation, and a datagram that is not a fragment and may not be fragmented (DF set, MF clear, offset 0) may carry any value in it, so never treat it as a unique packet number.

Time to live

TTL was defined in seconds, but every router must lower it by at least one, so in practice it counts hops. A router that would lower it to zero discards the datagram and sends an ICMP Time Exceeded message back to the source. That rule stops a datagram from circling forever when a routing loop sends it back and forth, and it is the mechanism traceroute is built on. IANA’s recommended default starting value is 64, which is also the Linux default (ip_default_ttl):

"""Follow the TTL of one datagram across a chain of routers."""


def send(ttl, routers):
    """Each router lowers the TTL by one and drops the datagram when the TTL would reach zero."""
    for hop, name in enumerate(routers, start=1):
        if ttl <= 1:
            return f"dropped by {name} (router {hop}), which sends ICMP Time Exceeded to the source"
        ttl -= 1
    return f"delivered with TTL {ttl}"


path = [f"r{n}" for n in range(1, 13)]   # twelve routers between the two hosts

print("sent with TTL 64:", send(64, path))
print("sent with TTL 3: ", send(3, path))
print("sent with TTL 13:", send(13, path))
print("sent with TTL 12:", send(12, path))

# A receiver that knows the sender started at 64 can count the routers on the way.
received = 52
print(f"received TTL {received}: {64 - received} routers, if the sender used the default 64")

# A loop between two routers that point at each other: the TTL is what stops it.
loop = ["r1", "r2"] + ["r3", "r4"] * 40
print("caught in a loop with TTL 64:", send(64, loop))

sent with TTL 64: delivered with TTL 52
sent with TTL 3:  dropped by r3 (router 3), which sends ICMP Time Exceeded to the source
sent with TTL 13: delivered with TTL 1
sent with TTL 12: dropped by r12 (router 12), which sends ICMP Time Exceeded to the source
received TTL 52: 12 routers, if the sender used the default 64
caught in a loop with TTL 64: dropped by r4 (router 64), which sends ICMP Time Exceeded to the source

Two lines deserve a second look. A datagram sent with TTL 13 crosses all twelve routers and arrives with TTL 1, while one sent with TTL 12 dies at the twelfth router: the router that receives TTL 1 is the one that drops it. And because the TTL falls by one per router, a receiver that knows the sender’s starting value can count the hops, which is why a TTL of 52 on a reply suggests twelve routers from a sender that started at 64. The looping datagram shows the safety net at work: after 63 forwarding steps the 64th router to receive it drops it, and the source hears about it.

Protocol

The protocol field says which protocol the data portion belongs to, so the receiving host knows whether to hand the data to TCP, UDP, ICMP or something else. The numbers come from IANA’s protocol numbers registry: 1 is ICMP, 6 is TCP, 17 is UDP, 41 is an IPv6 packet carried inside IPv4, 47 is GRE and 50 is ESP, the encrypted payload of IPsec.

Header checksum

The checksum covers the header only, never the data, which TCP, UDP and ICMP protect with checksums of their own. RFC 791 defines it this way:

The checksum field is the 16 bit one’s complement of the one’s complement sum of all 16 bit words in the header. For purposes of computing the checksum, the value of the checksum field is zero.

The lesson on parity and checksums computes this sum word by word and shows how a router patches it after a change instead of adding the whole header up again. Two consequences matter here. Every router that lowers the TTL must update the checksum too, because the TTL is one of the words it covers. And a router or host that finds a header whose words do not sum to 0xffff discards the datagram at once, without telling anyone.

Source and destination addresses

The two 32-bit addresses fill the last eight bytes of the fixed header, in the same byte order as the rest. Ordinary routers never change them; a device doing network address translation (NAT) rewrites one of them, and must then fix the IP checksum and the TCP or UDP checksum, which covers the addresses too.

Options

Options such as Record Route, Timestamp and the source-route options let the sender ask routers for special treatment. They are rare: RFC 7126 recommends that filtering devices drop datagrams carrying the source-route options and Record Route by default, so a datagram with options often does not arrive at all. When options are present, IHL grows to cover them, and padding fills the header to a whole number of 32-bit words.

Mistakes to avoid

• Reading IHL as bytes. An IHL of 5 is 20 bytes, and an IHL of 6 means 4 bytes of options before the data.
• Forgetting that total length includes the header. The data length is total length minus IHL × 4.
• Reading the fragment offset as bytes. Multiply it by 8 to get the byte position in the original data.
• Checksumming the data. The IPv4 checksum stops at the end of the header; the data has its own protection.
• Reading multi-byte fields in the wrong byte order. Headers are big-endian; on a little-endian computer, unpack with ! or > in struct, or with int.from_bytes(…, "big").
• Using the identification field as a packet counter. Since RFC 6864 a sender may put any value there for datagrams that will never be fragmented, and many do.

Exercise

import ipaddress
import struct


def parse_ipv4(header):
    """Return the fields of an IPv4 header as a dict; raise ValueError for an invalid header."""
    # Replace this line with your code.
    return {}

from ipv4 import parse_ipv4

UDP = bytes.fromhex("45 00 00 40 8c 1f 40 00 40 11 c2 1a c0 00 02 0a c6 33 64 35")
VOICE = bytes.fromhex("45 b8 00 c8 00 00 40 00 3f 11 d4 1b cb 00 71 14 c6 33 64 09")
TCP_ECT0 = bytes.fromhex("45 02 05 dc 3e 51 40 00 34 06 16 82 c6 33 64 09 c0 00 02 0a")
WITH_OPTIONS = bytes.fromhex("46 00 00 54 0d 2c 00 00 01 01 ab 2d c0 00 02 4d cb 00 71 01 01 01 01 00")
FRAGMENT = bytes.fromhex("45 00 05 dc be ef 20 b9 1e 11 ca 4b c6 33 64 c8 c0 00 02 21")


def test_udp_header():
    """reads every field of a plain 20-byte header"""
    h = parse_ipv4(UDP)
    assert h["version"] == 4
    assert h["header_length"] == 20
    assert h["total_length"] == 64
    assert h["identification"] == 0x8C1F
    assert h["df"] is True and h["mf"] is False
    assert h["fragment_offset"] == 0
    assert h["ttl"] == 64 and h["protocol"] == 17
    assert h["source"] == "192.0.2.10" and h["destination"] == "198.51.100.53"
    assert h["options"] == b""
    assert h["valid_checksum"] is True


def test_dscp_and_ecn():
    """splits the second byte into DSCP (6 bits) and ECN (2 bits)"""
    voice = parse_ipv4(VOICE)
    assert (voice["dscp"], voice["ecn"]) == (46, 0)
    tcp = parse_ipv4(TCP_ECT0)
    assert (tcp["dscp"], tcp["ecn"]) == (0, 2)
    assert tcp["protocol"] == 6 and tcp["total_length"] == 1500


def test_checksum_detects_damage():
    """notices a header changed in transit"""
    assert parse_ipv4(VOICE)["valid_checksum"] is True
    damaged = bytearray(VOICE)
    damaged[8] -= 1  # a TTL lowered without patching the checksum
    assert parse_ipv4(bytes(damaged))["valid_checksum"] is False


def test_options():
    """uses IHL to find where the header ends"""
    h = parse_ipv4(WITH_OPTIONS)
    assert h["header_length"] == 24
    assert h["options"] == bytes.fromhex("01010100")
    assert h["valid_checksum"] is True
    assert h["ttl"] == 1 and h["protocol"] == 1


def test_fragment_fields():
    """reads MF and the offset of a fragment"""
    h = parse_ipv4(FRAGMENT)
    assert h["df"] is False and h["mf"] is True
    assert h["fragment_offset"] == 185
    assert h["valid_checksum"] is True


def test_rejects_invalid_headers():
    """raises ValueError for input that cannot be an IPv4 header"""
    for bad in (UDP[:19], b"\x65" + UDP[1:], b"\x44" + UDP[1:], b"\x46" + UDP[1:]):
        try:
            parse_ipv4(bad)
        except ValueError:
            continue
        assert False, f"no ValueError for {bad[:2].hex(' ')} … ({len(bad)} bytes)"

Check yourself

Interview questions

What does the IHL field of an IPv4 header measure, and what is its smallest valid value?

IHL is the length of the header in 32-bit words, not in bytes. The fixed part of the header is 20 bytes, five words, so the smallest valid value is 5, and anything lower means the bytes are not a well-formed IPv4 header. The field has four bits, so the largest value is 15, which allows a 60-byte header with up to 40 bytes of options. A receiver uses IHL × 4 to find where the data starts, and total length minus IHL × 4 to find how much data there is.

Key takeaways

• The IPv4 header is 20 bytes without options and at most 60 bytes with them; IHL counts it in 32-bit words.
• The second byte holds DSCP (how the network should treat the datagram) and ECN (congestion signals).
• Identification, the flags and the fragment offset exist for fragmentation; the offset counts 8-byte units.
• TTL counts hops in practice: the router that receives a TTL of 1 drops the datagram and reports ICMP Time Exceeded.
• The protocol field names the data’s protocol by IANA number: 1 ICMP, 6 TCP, 17 UDP.
• The header checksum covers only the header, so every router that lowers the TTL must update it.