Media pipelines: transcoding and adaptive bitrate

One minute of video becomes 192 MB once it is encoded into the nine renditions of Apple’s example ladder, and a viewer on its default rung pulls 900 MB an hour from the CDN. A video pipeline turns one upload into those files: it validates the upload, transcodesDecoding a video and encoding it again in another format, resolution or bit rate, as the pipeline does once for every rung of the ladder. it into a ladder of renditions, each a resolution and a bit rate, cuts every rendition into short segments listed in playlists, and publishes them through a CDN.

The player then measures how fast each segment downloads and picks the rung for the next one, so a viewer on a weak phone connection and one on home fibre watch the same video at different qualities without stalling. Each stage is a queued job that can fail and be retried, and images follow the same idea with a few sizes and modern formats.

A pipeline turns one upload into a ladder of renditions

Read the diagram as a chain of independent jobs. Probing rejects files that are not what they claim before any expensive work starts. Each rung transcodes separately, so a slow 1080p job never delays the 360p rendition that weak connections need first. And the packager writes segments that never change, which the CDN can cache for as long as the video exists. Follow one upload down the chain:

The ladderbitrate ladder: The set of renditions a video is encoded into, each a resolution and a bit rate, from which a player picks the rung its network can sustain. decides most of the cost. Apple’s HLS authoring specification gives an example H.264 ladder of nine rungs, from 416×234 at 145 kbit/s to 1920×1080 at 7,800 kbit/s, and recommends the 2,000 kbit/s rung as the default variant (HLS authoring specification). Every rung is a full copy of the video, so storage adds up across the ladder:

# What a bitrate ladder costs to store, using the example H.264 ladder of Apple's HLS authoring specification (16:9,
# average bit rates in kbit/s). Every rung is a full copy of the video, so the storage adds up across the ladder.
LADDER = [
    ("416x234", 145), ("640x360", 365), ("768x432", 730), ("768x432", 1100), ("960x540", 2000),
    ("1280x720", 3000), ("1280x720", 4500), ("1920x1080", 6000), ("1920x1080", 7800),
]


def mb_per_minute(kbps):
    return kbps * 1000 * 60 / 8 / 10**6  # kilobits a second -> megabytes a minute


print(f"{'rung':<12}{'kbit/s':>8}{'MB per minute':>15}")
for size, kbps in LADDER:
    print(f"{size:<12}{kbps:>8,}{mb_per_minute(kbps):>15.1f}")
total = sum(mb_per_minute(k) for _, k in LADDER)
print(f"{'all rungs':<12}{sum(k for _, k in LADDER):>8,}{total:>15.1f}")

print(f"\nA 10-minute upload stored in every rung: {total * 10 / 1000:.2f} GB")
print(f"A library of 100,000 hours: {total * 60 * 100_000 / 10**6:,.0f} TB")
print(f"One viewer on the 2,000 kbit/s rung uses {mb_per_minute(2000) * 60:,.0f} MB an hour; on the top rung {mb_per_minute(7800) * 60:,.0f} MB")

rung          kbit/s  MB per minute
416x234          145            1.1
640x360          365            2.7
768x432          730            5.5
768x432        1,100            8.2
960x540        2,000           15.0
1280x720       3,000           22.5
1280x720       4,500           33.8
1920x1080      6,000           45.0
1920x1080      7,800           58.5
all rungs     25,640          192.3

A 10-minute upload stored in every rung: 1.92 GB
A library of 100,000 hours: 1,154 TB
One viewer on the 2,000 kbit/s rung uses 900 MB an hour; on the top rung 3,510 MB

One minute of video becomes 192 MB in nine renditions, and a viewer of the default rung pulls 900 MB an hour from the CDN. Those two numbers, storage per minute and egress per viewing hour, are what the ladder trades against quality.

A few more terms complete the picture, each in one sentence. HLS, defined in RFC 8216, lists media segments in playlists that players fetch over plain HTTP (RFC 8216). MPEG-DASH does the same with an XML manifest and is standardised as ISO/IEC 23009-1. CMAF is a segment format that HLS and DASH can share, which DASH-IF’s guidelines build on (DASH-IF). The segment duration sets how often a player can switch rungs and how far a live stream lags. Adaptive bitrateadaptive bitrate: Streaming in which the player picks the rendition of each next segment from what it measures, such as recent download rates or the seconds already buffered. logic in the player chooses the rung for each segment, from recent download rates, from the seconds of video already buffered, or from both. A ladder is fitted to the audience’s networks, and a job that keeps failing goes to a dead-letter queue. Images get the same treatment with a few sizes in formats such as WebP.

Designing a ladder for an audience, with numbers Pro: opens with a Pro, Premium or Ultimate pass