Signals, line coding and channel capacity

Every link turns bits into something physical: a voltage on copper, a pulse of light in fibre, a radio wave. Three numbers describe that signal and are easy to confuse. Bandwidth in hertz is the range of frequencies the channel lets through. The symbol rate, in baud, is how many signal elements (symbols) are sent each second. The bit rate is the symbol rate times the bits each symbol carries. Two limits bound the bit rate: Nyquist’s, for a channel without noise, caps the symbol rate at twice the bandwidth; Shannon’s, for a channel with noise, caps the bit rate itself at B log2(1 + S/N). How the bits become symbols is the job of the line code, and each code in use exists to fix a problem: keeping the receiver’s clock in step, leaving room for control signals, or squeezing more bits into each symbol. This lesson works through all three with numbers you can run.

Hertz, baud and bits per second

A symbol is one signal element: a voltage level held for a moment, one pulse, one point of a radio constellation. If a symbol can take M distinct values, it carries log2 M bits: 2 levels carry 1 bit, 4 levels 2 bits, and 256-QAM, a radio constellation of 256 points, 8 bits. The bit rate is therefore

The channel’s width in hertz limits the symbol rate, and the noise in it limits how many levels a receiver can tell apart. The diagram puts the pieces together.

Nyquist: the limit without noise

Nyquist showed that a channel B hertz wide can carry at most 2B independent symbols a second; sending them faster makes neighbouring symbols blur into each other. With M levels per symbol, the most a noiseless channel can carry is

For a channel 3 kHz wide, about what an analogue telephone line passes, that is 6,000 bit/s with 2 levels and 24,000 bit/s with 16. On paper the rate grows without end as you add levels. In practice noise stops it: levels closer together than the noise can no longer be told apart.

Shannon: the limit with noise

Shannon gave the limit for a channel with random (Gaussian) noise: no code, however clever, carries more than

bits a second without errors, where S/N is the ratio of signal power to noise power. Ratios are usually quoted in decibels: ITU-R V.574 defines the decibel value of a power ratio as 10 times its base-10 logarithm, so 10 dB is a ratio of 10, 20 dB of 100 and 30 dB of 1,000. Shannon’s formula needs the plain ratio, never the decibel number.

# The two limits of a channel. Nyquist (no noise): a channel B hertz wide carries at most
# 2B symbols a second, so with M signal levels C = 2B log2(M). Shannon (with noise):
# C = B log2(1 + S/N), where S/N is a plain power ratio, not decibels.
from math import log2, log10, sqrt


def db_to_ratio(db):
    return 10 ** (db / 10)


def ratio_to_db(ratio):
    return 10 * log10(ratio)


for db in (0, 3, 10, 20, 30, 40):
    print(f"{db:2} dB = power ratio {db_to_ratio(db):>8,.1f}")

B = 3_000  # a channel 3 kHz wide, about what an analogue telephone line passes
print(f"Nyquist, B = {B:,} Hz:")
for levels in (2, 4, 16, 64):
    print(f"  {levels:2} levels: {2 * B * log2(levels):>7,.0f} bit/s")

snr = db_to_ratio(30)
print(f"Shannon at 30 dB: {B * log2(1 + snr):,.0f} bit/s")
# The two formulas meet where 2B log2(M) = B log2(1 + S/N), that is M = sqrt(1 + S/N).
print(f"levels worth using at 30 dB: about {sqrt(1 + snr):.1f}")
print(f"check: S/N of 1,001 is {ratio_to_db(1 + snr):.2f} dB")

 0 dB = power ratio      1.0
 3 dB = power ratio      2.0
10 dB = power ratio     10.0
20 dB = power ratio    100.0
30 dB = power ratio  1,000.0
40 dB = power ratio 10,000.0
Nyquist, B = 3,000 Hz:
   2 levels:   6,000 bit/s
   4 levels:  12,000 bit/s
  16 levels:  24,000 bit/s
  64 levels:  36,000 bit/s
Shannon at 30 dB: 29,902 bit/s
levels worth using at 30 dB: about 31.6
check: S/N of 1,001 is 30.00 dB

At 30 dB the 3 kHz channel tops out at 29,902 bit/s. The two formulas also tell you how many levels are worth using: setting 2B log2 M equal to B log2(1 + S/N) gives M = √(1 + S/N), about 31.6 here, so a code with more than about 32 levels gains nothing at this noise. Note that 3 dB is a factor of 2.0, a handy number for quick estimates: every 3 dB doubles or halves a power.

Wide channels and clean signals

Shannon’s formula says capacity grows in proportion to bandwidth but only with the logarithm of the signal-to-noise ratio. The example works it out for the channel widths Wi-Fi uses, from 20 MHz to the 320 MHz channels that Wi-Fi 7 added in the 6 GHz band:

# Shannon's limit for the channel widths Wi-Fi uses, at three signal-to-noise ratios. One
# spatial stream; MIMO radios send several streams at once, each with its own limit.
from math import log2

WIDTHS_MHZ = (20, 40, 80, 160, 320)
SNRS_DB = (10, 20, 30)


def shannon_mbps(mhz, db):
    return mhz * 1e6 * log2(1 + 10 ** (db / 10)) / 1e6


print("width   " + "".join(f"{f'at {db} dB':>12}" for db in SNRS_DB))
for mhz in WIDTHS_MHZ:
    cells = "".join(f"{f'{shannon_mbps(mhz, db):,.0f} Mb/s':>12}" for db in SNRS_DB)
    print(f"{mhz:>3} MHz {cells}")

# Doubling the width doubles the limit; 10 dB more signal adds about 3.3 bits a second per hertz.
gain = log2(1 + 10 ** 3) - log2(1 + 10 ** 2)
print(f"20 dB -> 30 dB adds {gain:.2f} bit/s per hertz")

width       at 10 dB    at 20 dB    at 30 dB
 20 MHz      69 Mb/s    133 Mb/s    199 Mb/s
 40 MHz     138 Mb/s    266 Mb/s    399 Mb/s
 80 MHz     277 Mb/s    533 Mb/s    797 Mb/s
160 MHz     554 Mb/s  1,065 Mb/s  1,595 Mb/s
320 MHz   1,107 Mb/s  2,131 Mb/s  3,190 Mb/s
20 dB -> 30 dB adds 3.31 bit/s per hertz

Doubling the width doubles the limit, while going from 20 dB to 30 dB, ten times the signal power, adds only about 3.3 bit/s per hertz. That is why Wi-Fi 7 doubled its widest channel to 320 MHz, which the Wi-Fi Alliance credits with twice the throughput of Wi-Fi 6, and why denser constellations pay off only with a strong, clean signal: the Wi-Fi Alliance rates 4K-QAM, 12 bits a symbol, at 20 % more than the 10 bits of 1024-QAM. The table is for one stream; MIMO radios send several spatial streams at once, each with its own limit. Wide channels, dense constellations and several streams together are how IEEE 802.11be targets a maximum throughput of at least 30 Gbit/s.

Line codes: keeping the receiver’s clock in step

A receiver does not get a separate clock wire. It must recover the sender’s timing from the transitions in the signal itself, so a long stretch without a change leaves it guessing where one bit ends and the next begins. Three simple codes show the problem and the classic fix:

• NRZ-L (non-return to zero, level): a 1 is high and a 0 is low for the whole bit. Simple and compact, but a run of identical bits is a flat line.
• NRZI (non-return to zero, inverted): a 1 flips the level and a 0 keeps it. Runs of ones now have transitions; runs of zeros are still flat.
• Manchester: every bit changes level in its middle. In the IEEE 802.3 convention, which 10 Mb/s Ethernet uses, a 1 goes from low to high and a 0 from high to low. The opposite convention also appears in some books and exam papers, so an answer should say which one it uses.

# Three ways to put bits on a wire, drawn as text: "‾" is a high level, "_" a low level,
# two characters per bit so that a change in the middle of a bit shows.

def nrz_l(bits):
    """1 is high, 0 is low, for the whole bit."""
    return "".join("‾‾" if b == "1" else "__" for b in bits)


def nrzi(bits, level="_"):
    """A 1 flips the level at the start of the bit; a 0 keeps it."""
    out = []
    for b in bits:
        if b == "1":
            level = "‾" if level == "_" else "_"
        out.append(level * 2)
    return "".join(out)


def manchester(bits):
    """IEEE 802.3: a 1 goes low to high in the middle of the bit, a 0 high to low."""
    return "".join("_‾" if b == "1" else "‾_" for b in bits)


def longest_flat(wave):
    """The longest stretch without a level change, in bit times."""
    longest = run = 1
    for a, b in zip(wave, wave[1:]):
        run = run + 1 if a == b else 1
        longest = max(longest, run)
    return longest / 2


letter = "A"
bits = format(ord(letter), "08b")
print(f"bits of {letter!r}:  " + " ".join(bits))
for name, code in (("NRZ-L", nrz_l), ("NRZI", nrzi), ("Manchester", manchester)):
    print(f"{name:11} {code(bits)}")

print("longest stretch without a change, in bit times:")
for label, stream in (("64 zeros", "0" * 64), ("64 ones", "1" * 64), (f"{letter!r}", bits)):
    flats = [longest_flat(code(stream)) for code in (nrz_l, nrzi, manchester)]
    print(f"  {label:9} NRZ-L {flats[0]:4g}   NRZI {flats[1]:4g}   Manchester {flats[2]:g}")

# Manchester needs two signal elements per bit, so 10 Mb/s Ethernet signals at 20 Mbaud.
print(f"10 Mb/s with Manchester: {10e6 * 2 / 1e6:g} million signal elements a second")

bits of 'A':  0 1 0 0 0 0 0 1
NRZ-L       __‾‾__________‾‾
NRZI        __‾‾‾‾‾‾‾‾‾‾‾‾__
Manchester  ‾__‾‾_‾_‾_‾_‾__‾
longest stretch without a change, in bit times:
  64 zeros  NRZ-L   64   NRZI   64   Manchester 0.5
  64 ones   NRZ-L   64   NRZI    1   Manchester 0.5
  'A'       NRZ-L    5   NRZI    6   Manchester 1
10 Mb/s with Manchester: 20 million signal elements a second

Manchester never goes more than one bit time without a change, but it pays for that with two signal elements per bit: 10 Mb/s Ethernet signals at 20 million elements a second. At higher speeds that doubling is too expensive, so faster links use block codes instead.

Block codes: 4B/5B, 8B/10B and 64B/66B

A block code maps every group of data bits to a slightly longer code group, chosen so that the stream always has enough transitions and some code groups are left over for control signals such as “start of frame” and “idle”.

• 4B/5B turns 4 bits into 5. 100BASE-TX Ethernet sends its 4B/5B code groups at 125 million code bits a second for 100 Mb/s of data, scrambled, NRZI-encoded and carried as a three-level MLT-3 signal; Texas Instruments’ data sheet for its DP83848C transceiver lists the code groups and explains that the scrambler spreads the signal’s energy over a wide range of frequencies to keep radiated emissions down.
• 8B/10B turns 8 bits into 10. The Linux kernel’s documentation describes 1000BASE-X as running at a fixed 1.25 Gbaud for 1 Gb/s of data.
• 64B/66B adds a 2-bit header to every 64 bits, which also marks whether the block holds data or control. The kernel’s documentation gives 25GBASE-R, which uses the same coding as 10GBASE-R running 2.5 times as fast, a rate of 25.78125 Gbaud; 10GBASE-R itself runs at 10.3125 Gbaud.

# Block codes trade a little bandwidth for guaranteed transitions and spare control codes.
from itertools import product
from math import log10

# The 16 data code groups of 4B/5B (100BASE-X): 4 data bits become 5 code bits.
FOUR_B_FIVE_B = ["11110", "01001", "10100", "10101", "01010", "01011", "01110", "01111",
                 "10010", "10011", "10110", "10111", "11010", "11011", "11100", "11101"]

# However two code groups follow each other, the longest run of zeros stays short. With NRZI
# (a 1 is a level change) that bounds the time without a transition.
longest = max(max(len(run) for run in (a + b).split("1")) for a, b in product(FOUR_B_FIVE_B, repeat=2))
print(f"4B/5B: longest run of zeros across any two data code groups: {longest} bits")
print(f"4B/5B: {32 - 16} of the 32 five-bit patterns are left for control codes and errors")


def si(value, unit):
    """100000000, 'b/s' -> '100 Mb/s'; 25781250000, 'baud' -> '25.78125 Gbaud'."""
    for factor, prefix in ((1e9, "G"), (1e6, "M")):
        if value >= factor:
            return f"{value / factor:.5f}".rstrip("0").rstrip(".") + f" {prefix}{unit}"
    return f"{value:g} {unit}"


for name, data, code, rate in (("4B/5B", 4, 5, 100e6), ("8B/10B", 8, 10, 1e9), ("64B/66B", 64, 66, 25e9)):
    overhead = (code - data) / data
    print(f"{name:7} overhead {overhead:7.3%}: {si(rate, 'b/s')} needs {si(rate * code / data, 'baud')}")

# PAM4 sends 2 bits per symbol with four levels; NRZ sends 1 bit with two.
for name, bits in (("NRZ", 1), ("PAM4", 2)):
    per_symbol = f"{bits} bit{'s' if bits > 1 else ''} a symbol"
    print(f"{name:4} {2 ** bits} levels, {per_symbol}: 58 Gb/s needs {58 / bits:g} Gsym/s")
# On the same -1..+1 swing, PAM4's levels are 2/3 apart instead of 2: a third of the eye.
print(f"PAM4 eye height against NRZ: 1/3, a penalty of {20 * log10(3):.2f} dB")

4B/5B: longest run of zeros across any two data code groups: 3 bits
4B/5B: 16 of the 32 five-bit patterns are left for control codes and errors
4B/5B   overhead 25.000%: 100 Mb/s needs 125 Mbaud
8B/10B  overhead 25.000%: 1 Gb/s needs 1.25 Gbaud
64B/66B overhead  3.125%: 25 Gb/s needs 25.78125 Gbaud
NRZ  2 levels, 1 bit a symbol: 58 Gb/s needs 58 Gsym/s
PAM4 4 levels, 2 bits a symbol: 58 Gb/s needs 29 Gsym/s
PAM4 eye height against NRZ: 1/3, a penalty of 9.54 dB

On its own, before any scrambling, 4B/5B keeps the longest run of zeros across any two data code groups to 3 bits, and with NRZI each 1 is a transition, so the receiver sees a change at least every 4 bit times. Only 16 of the 32 five-bit patterns carry data; the rest supply the control codes and let the receiver spot invalid ones. The cost is 25 % more bits on the line, the same as 8B/10B, against 3.125 % for 64B/66B, which keeps its transitions by scrambling the data instead of mapping it through a table.

PAM4: two bits in every symbol

At tens of gigabits a second the channel’s loss grows quickly with frequency, so links send more bits per symbol instead of more symbols. PAM4 (four-level pulse amplitude modulation) uses four signal levels, so each symbol carries 2 bits and the symbol rate halves. The Optical Internetworking Forum’s Common Electrical I/O agreement (CEI 4.0) defines 56 Gb/s-class electrical lanes both ways: NRZ lanes at up to 58 gigasymbols a second for very short reaches, and PAM4 lanes at up to 29, with the levels at +1, +1/3, −1/3 and −1. The price is in the gaps: on the same voltage swing PAM4’s levels are a third as far apart as NRZ’s two, which costs about 9.5 dB of signal-to-noise ratio. The CEI agreement accepts that: its very-short-reach PAM4 lane needs a raw bit error ratio of only 10⁻⁶ and assumes forward error correction brings it to 10⁻¹⁵.

Exercise: Manchester code and Shannon’s limit

import math


def manchester_encode(bits):
    """'10' -> 'LHHL' (IEEE 802.3: a 1 goes low to high, a 0 high to low, in the middle of the bit)."""
    # Replace this line with your code: it sends NRZ, one level per bit.
    return "".join("H" if b == "1" else "L" for b in bits)


def manchester_decode(levels):
    """'LHHL' -> '10'; raise ValueError when the levels are not Manchester code."""
    # Replace this line with your code.
    return ""


def capacity(bandwidth_hz, snr_db):
    """Shannon's limit in bits a second for a channel of bandwidth_hz with a signal-to-noise ratio of snr_db decibels."""
    # Replace this line with your code: it forgets to turn decibels into a ratio.
    return bandwidth_hz * math.log2(1 + snr_db)

import math

from signals import capacity, manchester_decode, manchester_encode


def test_encode_one_and_zero():
    """a 1 is low then high, a 0 is high then low"""
    assert manchester_encode("1") == "LH"
    assert manchester_encode("0") == "HL"
    assert manchester_encode("10") == "LHHL"


def test_encode_a_byte():
    """the letter A, 01000001"""
    assert manchester_encode("01000001") == "HLLHHLHLHLHLHLLH"


def test_decode_round_trip():
    """decoding what was encoded gives the bits back"""
    for bits in ("", "0", "1", "1100101", "0" * 16):
        assert manchester_decode(manchester_encode(bits)) == bits


def test_decode_rejects_bad_levels():
    """a bit without its mid-bit transition, or half a bit, is not Manchester code"""
    for bad in ("HH", "LHLL", "LHH"):
        try:
            manchester_decode(bad)
        except ValueError:
            continue
        raise AssertionError(f"{bad!r} should raise ValueError")


def test_capacity_telephone_channel():
    """3 kHz at 30 dB: about 29.9 kb/s"""
    assert math.isclose(capacity(3_000, 30), 3_000 * math.log2(1_001), rel_tol=1e-9)


def test_capacity_scales_with_width():
    """doubling the bandwidth doubles the limit; 0 dB means S/N = 1, one bit a second per hertz"""
    assert math.isclose(capacity(40e6, 20), 2 * capacity(20e6, 20), rel_tol=1e-9)
    assert math.isclose(capacity(1e6, 0), 1e6, rel_tol=1e-9)

Interview questions

What is the difference between baud rate and bit rate? (warm-up)

Baud rate is the number of symbols, or signal changes, sent each second; bit rate is the number of bits sent each second. They are equal only when each symbol carries one bit. A symbol with M possible values carries log2 M bits, so the bit rate is the baud rate times log2 M: PAM4 at 29 gigasymbols a second carries 58 Gb/s, and 256-QAM carries 8 bits per symbol. Line codes can make the baud rate higher than the bit rate instead: Manchester sends two signal elements per bit, so 10 Mb/s Ethernet signals at 20 Mbaud. Bandwidth in hertz is a third quantity: it limits the baud rate (Nyquist: at most twice the bandwidth), while noise limits the bits per symbol.

Key takeaways

• Bandwidth (Hz), symbol rate (baud) and bit rate (bit/s) are different quantities: bit rate = baud × log2 M.
• Nyquist: a channel B hertz wide carries at most 2B symbols a second, so at most 2B log2 M bit/s without noise.
• Shannon: with noise, at most B log2(1 + S/N) bit/s, with S/N as a plain ratio (dB = 10 log10 of the power ratio).
• Capacity grows in proportion to bandwidth but only logarithmically with signal-to-noise ratio, which is why radios widen their channels.
• Line codes keep the receiver’s clock in step: Manchester has a transition in every bit (IEEE 802.3: 1 is low to high), at twice the symbol rate.
• Block codes add transitions and control codes for less: 4B/5B and 8B/10B cost 25 %, 64B/66B 3.125 %.
• PAM4 carries 2 bits per symbol and halves the symbol rate, at the cost of levels a third as far apart.