Case studies: links, files and media interview questions

A public paste can be read by anyone and may appear in the service's listings and in search. An unlisted paste can also be read by anyone, but only by someone who has its link: it is left out of every listing, so the link is the only thing that protects it. A private paste can be read only by its owner, signed in, so knowing the link is not enough. The difference shapes the design: public and unlisted pastes may be cached by a CDN, private ones never, and an unlisted paste's id must be long and random enough that nobody finds it by guessing.

Because of the numbers. Sixty million listeners at about 100 minutes a day keep about 9 million streams playing at once at the evening peak, and at an average of about 170 kbit/s that is more than 1.5 Tbit/s. A CDN serves it from edges close to the listeners, where the popular tracks are already cached, so the service's own servers only decide what may be played and hand out signed addresses, and the origin sends just the misses. Listeners also get faster starts, because the first bytes travel a shorter distance.

Because the bytes would crowd out everything else the servers do. At a peak of 2,315 photos a second of 2 MB each, originals arrive at about 37 Gbit/s; an API server that relays them holds a connection and a buffer for every slow phone upload, while it should be answering small requests about posts and feeds. With a presigned address, the API server only checks that the user may upload and signs a short-lived address for one object key; the phone sends the bytes to the object store, which is built to take them, and retries go to the same key. The finished upload then raises a job for the processing pipeline.

Because segments can be cached and shared. The live encoder writes each few seconds of every rendition once, as a file listed in a playlist; every viewer of that rendition fetches the same files, so a CDN edge fetches each segment once and serves it to everyone nearby. A continuous stream per viewer would need a server session for each of tens of millions of viewers. Segments also let a player change bit rate at every segment boundary, recover from a dropped connection by asking for the next file, and use the same HTTP caches as everything else. The price is delay: a player starts a few segments behind the newest one.

A follower feed shows recent posts from the accounts a viewer follows, so its hard part is fan-out: copying or merging posts from many accounts. A short-video feed is mostly recommended. For each page it gathers a few hundred candidates from sources such as the viewer's interests, followed accounts, what is popular in their region and language, and new videos; a ranking model predicts how the viewer will respond to each, and rules for variety and safety set the order. It also plays one video at a time, so the next one must start the moment the viewer swipes, and the viewer's last swipes change the next page.