Design live cricket streaming like JioHotstar

Live cricket streaming sends one match to tens of millions of screens at once, each a few seconds behind the ball. The pipeline runs in one direction: feeds from the ground are encoded into a ladder of bit rates, cut into segments of a few seconds and listed in playlists, and a shielded origin hands each new segment to the edges of several CDNs, which serve every player.

Cricket adds two hard parts to ordinary video. The audience arrives in waves that follow the game, a toss, a wicket, a tight last over, so capacity must be in place before the peak rather than added during it. And anything that would touch a central service once per viewer per segment, such as checking a subscription or choosing an ad, must happen once per viewing or once per group of viewers instead.

80 million screens at once, fed by shared segments

Ask these first, and note the answer you assume:

• How many at once? JioStar reported 65.2 million peak concurrent viewers for a T20 World Cup semi-final between India and England (JioStar) and 72.5 million concurrent streams for the final between India and New Zealand (JioStar). Plan for 80 million.
• On what? Mostly phones on mobile data, then connected TVs and browsers.
• How far behind live? Under 20 seconds by default.
• In which languages? JioStar describes its IPL coverage as more than 25 simultaneous feeds in 12 languages, so commentary comes in many languages over the same pictures.
• Who may watch? Subscribers, by plan and region, on a limited number of devices.

The functional requirements: watch a match live on a phone, a TV or a browser, choose the commentary language, follow the score, and see ads in the breaks. The non-functional ones follow from the answers above: no stall at the moment everyone is watching, a start within a couple of seconds, a match that keeps playing when an encoder, a region or a whole CDN fails, and central services that never see one request per viewer per segment.

# The estimate sheet of live cricket streaming at its peak (lesson system-design-hld/design-live-cricket-streaming).
# Every number below is an assumption of this lesson; the target sits a little above the largest concurrency a
# streaming service in India has reported, 72.5 million streams. Bit rates are megabits a second (10**6 bits).
PEAK_STREAMS = 80_000_000
SEGMENT_S = 4  # each media segment holds 4 seconds of video
REQUESTS_PER_SEGMENT = 2  # one segment and one playlist reload per segment for each viewer
EDGE_CACHES = 2_000  # cache servers, across CDNs and caches inside networks, that each fetch on their own
RENDITIONS = 6  # rungs of the bit-rate ladder
SHIELDS = 20  # the mid tier that edge caches fetch through
TOKEN_MINUTES = 10  # how long a playback token lasts before the player renews it

# The ladder and the share of viewers on each rung at the peak (a phone-heavy audience).
LADDER = {"180p": (0.25, 0.20), "360p": (0.6, 0.35), "480p": (1.0, 0.25), "720p": (2.0, 0.12),
          "1080p": (4.0, 0.06), "2160p": (12.0, 0.02)}
assert abs(sum(share for _, share in LADDER.values()) - 1) < 1e-9

avg = sum(mbit * share for mbit, share in LADDER.values())
edge = PEAK_STREAMS * REQUESTS_PER_SEGMENT / SEGMENT_S
no_shield = EDGE_CACHES * RENDITIONS * REQUESTS_PER_SEGMENT / SEGMENT_S
shielded = SHIELDS * RENDITIONS * REQUESTS_PER_SEGMENT / SEGMENT_S

print("Live cricket streaming: the estimate sheet at the peak")
print(f"  Streams      {PEAK_STREAMS / 10**6:,.0f} million at once")
print(f"  Bit rate     {avg:.2f} Mbit/s on average over the ladder's mix")
print(f"  Egress       {PEAK_STREAMS * avg / 10**6:,.1f} Tbit/s from the edges")
print(f"  Edge         {edge / 10**6:,.0f} million requests/s: a segment and a playlist every {SEGMENT_S} s per viewer")
print(f"  Origin       {no_shield:,.0f} requests/s if all {EDGE_CACHES:,} edge caches fetch directly, {shielded:,.0f} through {SHIELDS} shields")
print(f"  Tokens       {PEAK_STREAMS / (TOKEN_MINUTES * 60):,.0f} renewals/s for {TOKEN_MINUTES}-minute playback tokens,")
print(f"               against {PEAK_STREAMS / SEGMENT_S / 10**6:,.0f} million/s if every segment asked the entitlement service")

Live cricket streaming: the estimate sheet at the peak
  Streams      80 million at once
  Bit rate     1.23 Mbit/s on average over the ladder's mix
  Egress       98.4 Tbit/s from the edges
  Edge         40 million requests/s: a segment and a playlist every 4 s per viewer
  Origin       6,000 requests/s if all 2,000 edge caches fetch directly, 60 through 20 shields
  Tokens       133,333 renewals/s for 10-minute playback tokens,
               against 20 million/s if every segment asked the entitlement service

At the peak the edges send 98.4 Tbit/s and take 40 million requests a second, far more than one provider should carry for one event, so delivery is spread over several CDNs and caches inside internet providers’ networks. The origin needs only 60 requests a second once the edges fetch through 20 shieldsshield tier: An extra cache layer between a CDN's edges and the origin. Every edge sends its misses there, so a file wanted at many edges costs the origin about one request instead of one per edge. More in “CDNs and edge caching”. And checking the subscription on every segment would cost 20 million calls a second, against 133,333 renewals of ten-minute playback tokens.

COMPONENTS = {
    "player",
    "CDN edge",
    "shield",
    "origin packager",
    "live encoder",
    "entitlement service",
    "subscription database",
    "ad decision service",
}


def match_sheet(streams, avg_mbit, segment_s, edge_caches, renditions, shields):
    """Four figures of a live stream's estimate sheet at its peak, as a dictionary."""
    # Replace this line with your code.
    return {}


def cached_segment_path():
    """The components that work on one segment request when the edge already holds the segment."""
    # Replace this line with your code.
    return set()

import math

from match_sheet import COMPONENTS, cached_segment_path, match_sheet

LESSON = dict(streams=80_000_000, avg_mbit=1.23, segment_s=4, edge_caches=2_000, renditions=6, shields=20)
SMALL = dict(streams=5_000_000, avg_mbit=2.0, segment_s=6, edge_caches=300, renditions=5, shields=4)


def near(actual, expected):
    """Within 10%, as an estimate should be."""
    return math.isclose(actual, expected, rel_tol=0.10)


def test_egress():
    """turns streams and their average bit rate into terabits a second"""
    assert near(match_sheet(**LESSON)["egress_tbit"], 98.4)
    assert near(match_sheet(**SMALL)["egress_tbit"], 10.0)


def test_edge():
    """counts a segment and a playlist reload per viewer per segment duration"""
    assert near(match_sheet(**LESSON)["edge_requests"], 40_000_000)
    assert near(match_sheet(**SMALL)["edge_requests"], 1_666_667)


def test_origin():
    """counts the caches that fetch from the origin, each for every rendition"""
    lesson = match_sheet(**LESSON)
    assert near(lesson["origin_direct"], 6_000)
    assert near(lesson["origin_shielded"], 60)
    small = match_sheet(**SMALL)
    assert near(small["origin_direct"], 500)
    assert near(small["origin_shielded"], 6.67)


def test_cached_segment_path():
    """keeps every service behind the edge out of a cache hit"""
    path = cached_segment_path()
    assert path <= COMPONENTS, f"unknown names: {sorted(path - COMPONENTS)}"
    assert path == {"player", "CDN edge"}

The first design: one live path, and one call per viewing

Follow the ball. The contribution feedsThe video and sound sent from the ground to the encoders, usually over two separate paths, so that one failed link does not stop the match. reach ingest in two regions, encoders make every rung of the ladderbitrate ladder: The set of renditions a video is encoded into, each a resolution and a bit rate, from which a player picks the rung its network can sustain. More in “Media pipelines: transcoding and adaptive bitrate”, and the packager cuts segments and updates playlists; only misses travel from the origin through the shields to the edges. A player presses play once at the playback service, which checks the subscription, signs a token and picks a CDN, and from then on it talks only to the edges. Step through it:

The questions this design has to answer:

• Where do the seconds of delay go, and what does a shorter segment cost?
• How many requests reach the origin, and what can make that number explode?
• How are a dozen commentary languages delivered without multiplying the cache?
• How does an edge check every request without calling anyone?
• What must be ready before the toss, and what is switched off when the peak outgrows the plan?
• How are ads chosen for 80 million viewers who reach a break in the same seconds?
• What happens when an encoder, a region or a CDN fails during the match?

Each segment of delay costs requests: 15 seconds and 40 million a second at 4 s Pro: opens with a Pro, Premium or Ultimate pass