Design a Q&A forum like Quora or Reddit

A popular Q&A forum serves about 7,000 thread pages a second, 21,000 at the evening peak, while people add only 116 answers and comments and cast 2,315 votes a second. To design a Q&A forum like Quora or Reddit, get three things right: threads, votes and moderation. Every thread is a tree of answers and replies that should load in one read; each person casts at most one vote per item, and the counts must stay cheap to show; and a removal must reach every copy of a page within a minute.

The design caches pages hard for readers who are not signed in, keeps each thread on one shard, stores every vote as a row keyed by voter and item, ranks answers with a confidence bound and posts with a score that favours new ones, and sends every change through one change log. This case study sizes the forum, then follows its threads, votes, ranking and moderation.

Thread pages outnumber new comments sixty to one

Ask before designing: questions that gather many answers, or links and posts with comment threads? Whether you are asked to design a Q&A forum like Quora or a discussion site like Reddit, the data model is the same, a post with a tree of comments, so plan for both. Are there communities with their own moderators? How deep may replies nest? May people change their votes? What must disappear quickly when removed?

Functionally, people post questions or links, answer and reply at any depth, vote up or down once per item and change their vote, read threads sorted by best, top or new and community pages sorted by hot, search, and report content. Moderators remove, lock and ban. The non-functional requirements: a thread page within 300 ms at the 99th percentilep99 (99th percentile): The latency within which 99 % of requests finish, so the slowest 1 % take longer. Tail latency is read from such high percentiles, because an average hides the slowest requests. More in “Latency, throughput and percentiles”, vote counts that may trail by a few seconds but never count one person twice, a removal that is gone everywhere within a minute, and a site that stays readable when writes fail.

Quantity	Assumption	Result
Thread pages	40 million people a day × 15 pages	600 million a day: 6,944 a second, about 21,000 at peak
Answers and comments	10 million a day	116 a second
Votes	200 million a day	2,315 a second; one viral post can draw thousands a second alone
Text	10 million × 1 KB with metadata	10 GB a day, about 3.65 TB a year
Votes kept	200 million × 24 bytes	4.8 GB a day, about 1.75 TB a year

Reads dominate: 6,944 thread pages a second against 116 new comments, so most pages should come from a CDN or a cache, and the store should serve only misses and signed-in readers. Votes are the busiest writes, and they never touch a thread’s text, so they get their own service and counters. In the first design, reads flow down through the CDN and the thread cache to the store, and every change, whether a comment, a vote or a removal, goes to a change log that feeds the ranking jobs, the search index and cache invalidation.

Follow one read by someone who is not signed in, then one vote by someone who is, through that design:

The drawing shows the whole scene; its 9 steps are listed below.

Every step as text

1. A reader opens a thread. Someone who is not signed in opens a popular thread. Every such reader sees the same page, so the request goes first to the CDN edge nearest to them.
2. Answered at the edge. The edge holds a recent copy of the page and answers at once. Thread pages outnumber new comments about sixty to one, so most reads end here.
3. A miss goes on. When the edge's copy has expired, the request passes the API gateway to the thread service, which looks in the thread cache first.
4. One shard, one read. On a cache miss the thread service reads the store. A thread's comments all sit on the shard of its post, so the page is one read, not one per comment.
5. A signed-in reader votes. A reader who is signed in sends an upvote. Votes never touch a thread's text, so the gateway sends them to their own service.
6. One vote per person. The vote is stored as a row keyed by the voter and the item, so a second vote by the same person replaces the first instead of counting twice. The counters take the change.
7. Every change goes to the log. New comments, vote changes and removals all go to one change log, in order for each item. Nothing downstream is written by the services directly.
8. The log feeds everything else. Ranking jobs re-sort the community's lists, the search index takes new and removed comments, and the thread cache drops pages that changed.
9. Reads and writes apart. Reads end at the edge, the cache or one shard; writes end in the log. A burst of votes on a viral post never slows the thread page down.

A round then goes deeper. These are the questions it usually opens, each with the idea behind its answer:

• Threads as trees: each comment stores the path of its ancestors, so a whole subtree is one range of an index.
• One vote per person: a vote is a row keyed by voter and item, and counts live in sharded counters.
• Ranking answers: “best” sorts by the lower bound of a confidence interval for the share of upvotes.
• Ranking posts: “hot” adds the time a post was made to the logarithm of its votes.
• Moderation: a removal travels as an event to the cache, the CDN, the search index and the ranked lists.
• Hot threads: a viral thread is cached briefly at the CDN, and its counters are split.

An answer is judged on its requirements and estimates, the data model for threads and votes, ranking, moderation, caching and failure handling.

Every record is keyed for the read it serves Pro: opens with a Pro, Premium or Ultimate pass

In this part:

• A materialised path reads a subtree as one range
• One row per voter and item counts every vote once
• A confidence bound ranks answers; posting time ranks posts
• A removal is a state change that reaches every copy
• A 30-second edge lifetime absorbs a viral thread
• At ten times the load, vote rows and counters give out first
  • Bottlenecks at 10× and 100×
  • Failure drills
  • Designs this one rejects
• What each level covers, and a 45-minute self-mock
• Practise: order a thread best first
  • Order a thread best first, to a fixed depth
• Interview questions
  • A post goes viral: 8,000 votes a second land on it and 60,000 people a second open its thread. Votes time out and the page is slow. How do you record the votes, show the counts and serve the thread without overloading one row?
  • Answers are sorted by upvotes minus downvotes, and people who write new answers say nobody ever sees them. Propose a ranking for the best answers and one for a community's front page, and show with numbers why each is fairer.
  • A moderator removes a harmful answer, but it stays visible for an hour: in the cached thread page, in search results and in the apps of people who loaded it earlier. Design removal so that it takes effect everywhere within a minute.
• Key takeaways

2 diagrams · 3 runnable examples (SQL) · 1 exercise with sample tests · 3 interview answers · about 57 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

• Pro pass · 1 week · ₹79 · $2.99
• 30 days · ₹179 · $5.99
• 1 year · ₹1,699 · $45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a passI have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

Read a complete sample first: Design a news feed