Reading RFCs and network standards

Network protocols are defined in public documents. Internet protocols such as IP, TCP, DNS, TLS and HTTP are specified in the IETF’s RFCs; Ethernet and Wi-Fi in IEEE 802 standards; mobile networks in 3GPP specifications; and much of what browsers do, such as how fetch() follows a redirect, in WHATWG standards. To read them well you need three habits. Find the current document, because a published RFC never changes and newer RFCs replace or amend it. Read the requirement words MUST, SHOULD and MAY as defined terms. And check what kind of document you have, because not every RFC is a standard and an Internet-Draft is not one at all.

Finding the current document

Once an RFC is published, its text never changes. Mistakes are recorded as errata, and a new RFC is published to replace or amend an old one (RFC Editor). Every RFC’s information page therefore lists two relations at the top, and they are the first thing to read:

• Obsoleted by: a newer RFC replaces this one as the reference. Read and cite the newer one.
• Updated by: a newer RFC changes or adds to parts of this one. Read both.

The RFC Editor also publishes these relations as data, at https://www.rfc-editor.org/rfc/rfcNNNN.json. The file below holds them for twenty RFCs, trimmed to four fields, and the program reads them:

"""Reads saved RFC Editor metadata and prints what replaced or changed four RFCs.

rfc_index.json holds, for each RFC, its status and the lists the RFC Editor publishes as "Obsoleted by" and
"Updated by" (trimmed from https://www.rfc-editor.org/rfc/rfcNNNN.json to these fields).
"""
import json

with open("rfc_index.json", encoding="utf-8") as f:
    index = json.load(f)


def names(ids):
    return ", ".join(i.replace("RFC", "RFC ") for i in ids) or "-"


for doc in ("RFC8446", "RFC2616", "RFC7540", "RFC2119"):
    rfc = index[doc]
    print(f"{doc.replace('RFC', 'RFC ')}: {rfc['title']}")
    print(f"  status:       {rfc['status']}")
    print(f"  obsoleted by: {names(rfc['obsoleted_by'])}")
    print(f"  updated by:   {names(rfc['updated_by'])}")

current = [doc for doc, rfc in index.items() if not rfc["obsoleted_by"]]
print(f"\nNot obsoleted (each is the reference for its topic): {names(current)}")

RFC 8446: The Transport Layer Security (TLS) Protocol Version 1.3
  status:       Proposed Standard
  obsoleted by: RFC 9846
  updated by:   -
RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1
  status:       Draft Standard
  obsoleted by: RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235
  updated by:   RFC 2817, RFC 5785, RFC 6266, RFC 6585
RFC 7540: Hypertext Transfer Protocol Version 2 (HTTP/2)
  status:       Proposed Standard
  obsoleted by: RFC 9113
  updated by:   RFC 8740
RFC 2119: Key words for use in RFCs to Indicate Requirement Levels
  status:       Best Current Practice
  obsoleted by: -
  updated by:   RFC 8174

Not obsoleted (each is the reference for its topic): RFC 9110, RFC 9111, RFC 9112, RFC 9113, RFC 9846, RFC 2119, RFC 8174, RFC 9915

{
  "RFC2068": {"title": "Hypertext Transfer Protocol -- HTTP/1.1", "status": "Proposed Standard", "obsoleted_by": ["RFC2616"], "updated_by": []},
  "RFC2616": {"title": "Hypertext Transfer Protocol -- HTTP/1.1", "status": "Draft Standard", "obsoleted_by": ["RFC7230", "RFC7231", "RFC7232", "RFC7233", "RFC7234", "RFC7235"], "updated_by": ["RFC2817", "RFC5785", "RFC6266", "RFC6585"]},
  "RFC7230": {"title": "Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing", "status": "Proposed Standard", "obsoleted_by": ["RFC9110", "RFC9112"], "updated_by": ["RFC8615"]},
  "RFC7231": {"title": "Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content", "status": "Proposed Standard", "obsoleted_by": ["RFC9110"], "updated_by": []},
  "RFC7232": {"title": "Hypertext Transfer Protocol (HTTP/1.1): Conditional Requests", "status": "Proposed Standard", "obsoleted_by": ["RFC9110"], "updated_by": []},
  "RFC7233": {"title": "Hypertext Transfer Protocol (HTTP/1.1): Range Requests", "status": "Proposed Standard", "obsoleted_by": ["RFC9110"], "updated_by": []},
  "RFC7234": {"title": "Hypertext Transfer Protocol (HTTP/1.1): Caching", "status": "Proposed Standard", "obsoleted_by": ["RFC9111"], "updated_by": []},
  "RFC7235": {"title": "Hypertext Transfer Protocol (HTTP/1.1): Authentication", "status": "Proposed Standard", "obsoleted_by": ["RFC9110"], "updated_by": []},
  "RFC9110": {"title": "HTTP Semantics", "status": "Internet Standard", "obsoleted_by": [], "updated_by": []},
  "RFC9111": {"title": "HTTP Caching", "status": "Internet Standard", "obsoleted_by": [], "updated_by": []},
  "RFC9112": {"title": "HTTP/1.1", "status": "Internet Standard", "obsoleted_by": [], "updated_by": ["RFC9931"]},
  "RFC7540": {"title": "Hypertext Transfer Protocol Version 2 (HTTP/2)", "status": "Proposed Standard", "obsoleted_by": ["RFC9113"], "updated_by": ["RFC8740"]},
  "RFC9113": {"title": "HTTP/2", "status": "Proposed Standard", "obsoleted_by": [], "updated_by": []},
  "RFC5246": {"title": "The Transport Layer Security (TLS) Protocol Version 1.2", "status": "Proposed Standard", "obsoleted_by": ["RFC8446", "RFC9846"], "updated_by": ["RFC5746", "RFC5878", "RFC6176", "RFC7465", "RFC7507", "RFC7568", "RFC7627", "RFC7685", "RFC7905", "RFC7919", "RFC8447", "RFC9155", "RFC10015"]},
  "RFC8446": {"title": "The Transport Layer Security (TLS) Protocol Version 1.3", "status": "Proposed Standard", "obsoleted_by": ["RFC9846"], "updated_by": []},
  "RFC9846": {"title": "The Transport Layer Security (TLS) Protocol Version 1.3", "status": "Proposed Standard", "obsoleted_by": [], "updated_by": []},
  "RFC2119": {"title": "Key words for use in RFCs to Indicate Requirement Levels", "status": "Best Current Practice", "obsoleted_by": [], "updated_by": ["RFC8174"]},
  "RFC8174": {"title": "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", "status": "Best Current Practice", "obsoleted_by": [], "updated_by": []},
  "RFC8415": {"title": "Dynamic Host Configuration Protocol for IPv6 (DHCPv6)", "status": "Proposed Standard", "obsoleted_by": ["RFC9915"], "updated_by": []},
  "RFC9915": {"title": "Dynamic Host Configuration Protocol for IPv6 (DHCPv6)", "status": "Internet Standard", "obsoleted_by": [], "updated_by": []}
}

Some replacements are worth knowing by heart:

• TLS 1.3 moved from RFC 8446 to RFC 9846. The protocol version is still 1.3, so existing connections are unaffected; the new RFC also obsoletes RFC 5246, the TLS 1.2 specification, and adds new requirements for TLS 1.2 implementations.
• HTTP/1.1 was one document, RFC 2616. It was replaced by six RFCs, and those were replaced by RFC 9110 (the semantics shared by every HTTP version), RFC 9111 (caching) and RFC 9112 (the HTTP/1.1 message format). Code comments that still cite RFC 2616 point to text that has been rewritten twice.
• DHCPv6 moved from RFC 8415 to RFC 9915, which is an Internet Standard.
• DMARC, the email authentication policy, moved from the Informational RFC 7489 to three Proposed Standards, RFCs 9989 to 9991 (RFC 9989).

“Obsoleted” describes documents, not deployments: TLS 1.2 is still in use, with RFC 9846 setting rules for it. Before you build on a passage, also open the RFC’s errata. An erratum is technical or editorial, and its status is reported, verified, held for a document update or rejected; errata are not merged into the RFC’s text or PDF.

Reading MUST, SHOULD and MAY

Most RFCs say near the start that they use the key words of BCP 14, the pair RFC 2119 and RFC 8174. Those words then have precise meanings:

• MUST, REQUIRED and SHALL: an absolute requirement.
• MUST NOT and SHALL NOT: an absolute prohibition.
• SHOULD and RECOMMENDED: do it, unless you have a valid reason not to and understand the full implications.
• SHOULD NOT and NOT RECOMMENDED: do not, unless you have a valid reason and understand the full implications.
• MAY and OPTIONAL: truly optional; implementations that do it and those that do not must still work together.

RFC 8174 adds the rule people forget: the words have this meaning only in capitals. “The client may retry” is ordinary English. This program finds the key words in four sentences written like a specification:

"""Finds the requirement keywords of BCP 14 (RFC 2119 and RFC 8174) in the text of a specification.

RFC 8174 says the words carry their special meaning only when they are written in capitals, so "may" in lower
case is ordinary English. Longer phrases come first, so that "MUST NOT" is not read as "MUST".
"""
import re

FORBIDDEN = "absolute prohibition"
AVOID = "avoid it, unless you understand the consequences"
REQUIRED = "absolute requirement"
ADVISED = "do it, unless there is a good reason not to"
OPTIONAL = "truly optional"
LEVELS = {
    "MUST NOT": FORBIDDEN, "SHALL NOT": FORBIDDEN, "SHOULD NOT": AVOID, "NOT RECOMMENDED": AVOID,
    "MUST": REQUIRED, "REQUIRED": REQUIRED, "SHALL": REQUIRED,
    "SHOULD": ADVISED, "RECOMMENDED": ADVISED, "MAY": OPTIONAL, "OPTIONAL": OPTIONAL,
}
KEYWORD = re.compile(r"\b(" + "|".join(LEVELS) + r")\b")  # case-sensitive on purpose
SPEC = """\
A sender MUST NOT retransmit a request more than three times.
A receiver SHOULD log every frame that it cannot parse.
Implementations may keep an answer for as long as its time to live allows.
A client MAY offer compression, and a server is REQUIRED to accept an uncompressed reply."""

for n, sentence in enumerate(SPEC.splitlines(), 1):
    found = KEYWORD.findall(sentence)
    print(f"{n}. {sentence}")
    for word in found:
        print(f"   {word}: {LEVELS[word]}")
    if not found:
        lower = [w for w in LEVELS if re.search(rf"\b{w.lower()}\b", sentence)]
        print(f"   no keyword; {', '.join(repr(w.lower()) for w in lower)} in lower case is ordinary English")

1. A sender MUST NOT retransmit a request more than three times.
   MUST NOT: absolute prohibition
2. A receiver SHOULD log every frame that it cannot parse.
   SHOULD: do it, unless there is a good reason not to
3. Implementations may keep an answer for as long as its time to live allows.
   no keyword; 'may' in lower case is ordinary English
4. A client MAY offer compression, and a server is REQUIRED to accept an uncompressed reply.
   MAY: truly optional
   REQUIRED: absolute requirement

SHOULD is where implementations and documents part ways, and that is allowed. RFC 6298 says TCP’s retransmission timeout SHOULD be rounded up to at least 1 second, while Linux uses a minimum of 200 ms (TCP_RTO_MIN in its source). Linux does not break the standard, but a calculation that assumes the RFC’s value will be wrong for Linux servers, so always say which one you describe.

What kind of document is it?

The RFC Editor shows each RFC’s status, and only some of the statuses are standards. The diagram shows how a document gets there:

• Internet Standard: the top of the standards track, mature and widely implemented. Examples: RFC 9293 (TCP), RFC 9110 (HTTP semantics), RFC 9915 (DHCPv6).
• Proposed Standard: on the standards track and stable; many protocols in daily use stay at this level. Examples: RFC 9846 (TLS 1.3), RFC 9000 (QUIC), RFC 9113 (HTTP/2).
• Draft Standard: an older middle level. Since RFC 6410 the track has only the two levels above, but old documents keep the label, as RFC 4271 (BGP-4) does.
• Best Current Practice (BCP): guidance on operations or process. Examples: RFC 2119 and RFC 8174, together BCP 14, and RFC 1918 on private addresses.
• Informational: published for information, not a standard. Examples: RFC 3022 (NAT) and RFC 7489, the old DMARC.
• Experimental: a specification for experiments, which may become a standard later, such as RFC 7413 (TCP Fast Open).
• Historic: retired and kept for the record, such as RFC 1058, the first version of RIP.
• Internet-Draft: work in progress and not an RFC at all, such as draft-ietf-ccwg-bbr (BBR congestion control).

The maturity levels come from RFC 2026, reduced to two by RFC 6410. RFC 2026 is blunt about drafts: an Internet-Draft has no formal status, is removed after six months unless a new version replaces it, and must never be cited as a standard or claimed for compliance. Drafts are still worth reading, because they show where a protocol is going. Each draft has a page in the IETF’s Datatracker with its versions, the comments on it and, once it is published, its RFC number; draft-ietf-ccwg-bbr (BBR congestion control) and draft-ietf-happy-happyeyeballs-v3 (Happy Eyeballs version 3) are good ones to look up.

Standards beyond the IETF

The RFCs stop at the edges of the Internet layers, and other bodies take over:

• IEEE 802 defines Ethernet (802.3), bridging and VLANs (802.1Q) and Wi-Fi (802.11). The Get IEEE 802 program makes individual IEEE 802 standards available to the public at no cost six months after publication; drafts are not included (Get IEEE 802 guidelines, IEEE GET Program).
• 3GPP writes the specifications of 4G and 5G mobile networks and groups its work in numbered Releases. Its releases page lists Release 18 as frozen and Releases 19 and 20 as open, so a 5G feature is always described by the Release that adds it.
• The WHATWG maintains the HTML, DOM, URL and Fetch standards as Living Standards, which are updated continuously rather than in numbered versions (WHATWG FAQ). Browser behaviour such as how fetch() handles redirects and CORS is defined there, not in an RFC.
• The ITU publishes Recommendations such as ITU-T X.200, the OSI reference model, and the G-series standards for fibre and DSL access that the previous lessons use.

Interview questions

Fresher · What do MUST, SHOULD and MAY mean in an RFC?

They are the requirement levels of BCP 14, RFC 2119 and RFC 8174, and they carry that meaning only when written in capitals. MUST (also REQUIRED or SHALL) is an absolute requirement and MUST NOT an absolute prohibition. SHOULD means you do it unless you have a valid reason not to and understand the full implications; SHOULD NOT is the reverse. MAY (OPTIONAL) is truly optional, and implementations that do it must still work with those that do not. A good answer gives an example of a SHOULD that implementations depart from, such as Linux’s 200 ms minimum retransmission timeout against RFC 6298’s 1 second.

Key takeaways

• A published RFC never changes: check “Obsoleted by” and “Updated by” before you rely on one, and read its errata.
• MUST, SHOULD and MAY are defined terms only in capitals; SHOULD leaves room for implementations to differ.
• Internet Standards and Proposed Standards are on the standards track; BCPs give practice, Informational RFCs give information, and an Internet-Draft is never a standard.
• Ethernet and Wi-Fi are IEEE 802 standards, mobile networks are 3GPP Releases and browser behaviour is in WHATWG’s Living Standards.