How design rounds are assessed, level by level

A design round has no answer key. Two candidates can draw different architectures for the same prompt and both do well, or draw the same one and be judged at different levels. What is assessed is how you work: while you talk and draw, the interviewer collects evidence, and afterwards weighs it against what the level being hired for expects. This track scores that evidence on four dimensions, each from 1 to 4 against written descriptions:

• Problem navigation: finding the problem worth solving, with its numbers.
• Solution design: a whole that works, from the operations to where the data lives.
• Technical depth: the deep dives, the trade-offs and what happens when parts fail.
• Communication: how you run the discussion and work with the interviewer.

The levels ask for more of the same four things, and above all for more independence. A fresher reaches a working design with help; a senior engineer drives the round and finds the hard parts without being asked; a staff engineer also weighs cost, operations and how the system will have to change.

Four dimensions, scored from evidence

An interviewer can score only what the round shows. A trade-off you weighed in silence, or a failure you thought of and never mentioned, leaves nothing to score, which is why this track keeps asking you to say your reasons aloud. Each dimension below says what it covers and what its evidence sounds like.

Problem navigation is finding the problem worth solving: the questions that change the design, the assumptions said out loud, the non-goals, and numbers that decide something. Its evidence sounds like “can an appointment be moved after it is booked?” or “that is about 23 reminders a second, so one queue is plenty”.

Solution design is a design that works end to end. Every functional requirement maps to an operation, each kind of data lives where it is read, and each main path can be walked along the diagram. The notation advice of the C4 model gives a quick test for the diagram: every line should point one way and carry a label more specific than “uses”.

Technical depth is going deep where the design is most likely to break: choosing that part, working it through with numbers, naming the alternatives and why they lose, and saying what fails, how you would notice and how the system recovers. A trade-off said well has the shape of an architectural decision record, which keeps one decision together with its rationale, its trade-offs and its consequences.

Communication is how the discussion is run: saying which step you are on, keeping the diagram in step with your words, checking in at decisions, taking hints and summarising what was decided. Unlike the other three, it is scored in every minute of the round.

The phases in the figure follow the time boxes of a 45-minute round, and they explain a common surprise: a dimension can only score as high as the evidence the round leaves behind. Start the deep dives in minute 3 and the interviewer may see excellent depth and almost no navigation. Spend 25 minutes on requirements and the depth score has nothing to stand on.

What each score looks like

Every score is tied to a description of what the round shows, so that two people who score the same round reach the same number, and so that you can score yourself. Guidance on structured interviews from the U.S. Office of Personnel Management describes the most structured interviews as rated against benchmarks of proficiency, and reports that more structure goes with higher reliability and agreement between raters. Read each list from score 1 up, and stop at the highest score the evidence supports.

Problem navigation

• Score 1: Drawing starts before any question; the requirements stay vague and no number is said.
• Score 2: Some questions, but not the one that changes the design; numbers are said and decide nothing.
• Score 3: The questions that change the design, the assumptions and non-goals said aloud, and at least one decision made from a number.
• Score 4: The requirement that makes the problem hard is found, the scope is agreed with the interviewer, and it is revisited when something new comes up.

Solution design

• Score 1: Boxes without flows: the main operations cannot be traced.
• Score 2: The main path works, with gaps: a missing store, a requirement no operation serves, arrows that say nothing.
• Score 3: Complete end to end: every requirement maps to an operation, data lives where it is read, and each main path can be walked.
• Score 4: Complete and simple: no part without a reason, alternatives named with why they lost, and room left for the likely next requirement.

Technical depth

• Score 1: Technologies are named without reasons; nothing is said about failures.
• Score 2: Deep dives happen only when the interviewer leads them; trade-offs stay general (“it scales better”).
• Score 3: The riskiest part is chosen without being asked and worked through with numbers, including what fails and how it recovers.
• Score 4: Several deep dives at the level of mechanisms, such as races, ordering and capacity at ten times the load, and a clear view of where the design breaks first.

Communication

• Score 1: Long silences or a monologue; the diagram and the words disagree.
• Score 2: Easy to follow but unstructured: the interviewer has to ask for the plan.
• Score 3: The steps are signposted, the diagram stays in step, decisions are checked with the interviewer and hints are taken up.
• Score 4: The discussion runs like a design review: decisions are summarised, challenge is invited, and a changed problem is handled smoothly.

What changes from fresher to staff

Each level has a bar: the lowest score it expects on every dimension, given here in the order navigation, design, depth and communication. As a rough guide, fresher means up to about 2 years of experience, mid 2 to 5, senior 5 to 10 and staff 10 or more, but a level is about the scope someone can handle, not the time they have worked.

• Fresher: 2, 2, 2, 2. The bar is a design that works for the main path, reached with prompts from the interviewer along the way. The interviewer expects to lead the deep dives; what counts is following them sensibly.
• Mid: 3, 3, 2, 2. The design is complete without help, and its numbers drive it. Depth still comes when the interviewer points at a part, and a 2 in communication is enough.
• Senior: 3, 3, 3, 3. The difference from mid is who leads: a senior answer picks the risky parts itself, puts numbers on them and says how they fail, while the interviewer mostly follows.
• Staff: 4, 4, 4, 3. A staff answer has everything a senior one has, judged in a wider frame: what the system costs to run, how it is operated and changed, which parts other teams would own, and which decisions can wait. It must be clear, but the extra weight of the level is in that judgement, so the communication bar stays at 3.

Reading a level from four scores

The rubric reads a level as the highest one whose bar is met on every dimension. It never averages, because an average lets a strong dimension hide a weak one, and the weak one is exactly what the next practice round should target. The program below reads four answers:

"""Read the level an answer reaches from its four rubric scores.

This track's rubric scores a design round on four dimensions, each from 1 to 4. A level's
bar is the lowest score it expects on every dimension, and an answer reaches the highest
level whose bar it meets on all four: a strong dimension never makes up for a weak one.
"""

DIMENSIONS = ("navigation", "design", "depth", "communication")

# The lowest score each level expects, dimension by dimension (in the order of DIMENSIONS).
BARS = {
    "fresher": (2, 2, 2, 2),
    "mid": (3, 3, 2, 2),
    "senior": (3, 3, 3, 3),
    "staff": (4, 4, 4, 3),
}


def meets(scores, level):
    """True when every score reaches the level's bar."""
    return all(score >= bar for score, bar in zip(scores, BARS[level]))


def level_reached(scores):
    """The highest level whose bar every score meets, or None below the lowest bar."""
    reached = None
    for level in BARS:  # from the lowest bar to the highest
        if not meets(scores, level):
            break
        reached = level
    return reached


def show(name, scores, target):
    """One answer: its scores, their average, the level reached and what it is short of."""
    print(name)
    print("  scores:", ", ".join(f"{d} {s}" for d, s in zip(DIMENSIONS, scores)))
    level = level_reached(scores) or "below every bar"
    print(f"  average {sum(scores) / len(scores):.2f}, level reached: {level}")
    gaps = [f"{d} by {bar - s}" for d, s, bar in zip(DIMENSIONS, scores, BARS[target]) if s < bar]
    print(f"  short of {target}: {', '.join(gaps) or 'nothing, the bar is met'}")


show("Answer A, the reminder service", (3, 3, 2, 3), "senior")
print()
show("Deep and clear, but scoped late", (2, 4, 4, 3), "senior")
print()
show("Staff-quality design, said too quietly", (4, 4, 4, 2), "staff")
print()
show("A first practice round", (1, 2, 2, 2), "fresher")

Answer A, the reminder service
  scores: navigation 3, design 3, depth 2, communication 3
  average 2.75, level reached: mid
  short of senior: depth by 1

Deep and clear, but scoped late
  scores: navigation 2, design 4, depth 4, communication 3
  average 3.25, level reached: fresher
  short of senior: navigation by 1

Staff-quality design, said too quietly
  scores: navigation 4, design 4, depth 4, communication 2
  average 3.50, level reached: mid
  short of staff: communication by 1

A first practice round
  scores: navigation 1, design 2, depth 2, communication 2
  average 1.75, level reached: below every bar
  short of fresher: navigation by 1

• Answer A, scored in full below, averages 2.75 and reaches mid. One more point of depth would meet the senior bar.
• The second answer averages 3.25 but reaches only the fresher bar. Its navigation score of 2 misses the mid bar of 3: a fine design for a problem that was never pinned down.
• The third averages 3.50 and reaches mid. Its three design dimensions are at the staff bar, but a communication score of 2 misses the senior one, so a single point of communication stands between it and staff.
• The last is the edge case: one score of 1 puts an answer below every bar, whatever the other three are.

Press Edit and put your own scores into a show(...) line to read your level.

Scoring a recorded practice answer

Score from a recording, not from memory: memory keeps the good moments and drops the silences.

1. Record the round. Audio is enough. Keep the diagram you drew beside it.
2. Score it after a break. A few hours later you hear what you said, not what you meant.
3. Log the evidence with times. Play the recording once and write a line for each moment that shows a dimension: the minute, the words and the dimension. “Minute 4: ‘about 23 reminders a second, so one queue is plenty’ (navigation).”
4. Score each dimension from the log. Read its list from score 1 up and stop at the highest score your logged moments support. A behaviour you cannot point to in the log did not happen.
5. Compare with a second scorer. Swap recordings with a friend, or first score an answer whose reference scores you know, like the two below. The OPM guidance describes panels whose members rate on their own and then discuss large differences; do the same. Two points apart on a dimension means the two of you read its descriptions differently, so find the moments each of you counted.
6. Choose one thing to practise. Take the dimension furthest below your target bar, practise it, and use a prompt you have not met before in the next round. The exercise of this lesson writes that choice as code.

Two answers to one prompt, scored

The prompt: design a service that reminds people of their appointments 24 hours before they start, by email or app notification. Below are moments from two practice rounds of 45 minutes. Score Answer A yourself before you read its scores.

Answer A

• Minute 1. “Before I draw anything: can an appointment be moved or cancelled after it is booked, and does a moved appointment get a new reminder?”
• Minute 2. “I’ll assume 2 million appointments a day, one reminder each, and that a reminder may arrive a few minutes late. Two-way replies are out of scope.”
• Minute 4. “That’s about 23 reminders a second on average. Most appointments are in the morning, so I’ll plan for ten times that, around 230 a second: one queue and a few workers will do.”
• Minute 9. “Three operations: create an appointment, move it, cancel it. Each one sets or clears the time its reminder is due.”
• Minute 15. “One table of appointments, with the time each reminder is due, indexed, and a status: pending, sent or cancelled.”
• Minute 21. Draws a scheduler that collects due reminders every minute and puts them on a queue, and workers that send them through the email and notification providers. Every arrow is labelled.
• Minute 31. The interviewer asks what happens if a worker crashes after sending but before it marks the reminder sent. “Then it could go out twice. I’d add a lock so that only one worker handles each reminder.”
• Minute 42. “The scheduler is the bottleneck, so we can run more of them.”
• Throughout. Says which step comes next, and checks before the data model: “Does this cover cancellations for you?”

The scores: 3, 3, 2 and 3.

• Navigation 3. The questions of minutes 1 and 2 change the design, because a moved appointment moves its reminder, and a number decides the queue. Not 4: nothing found what makes the problem hard, such as the morning peak meeting the sending limits of the providers.
• Design 3. Every requirement maps to an operation, the due time lives where the scheduler reads it, and each path can be walked on the diagram. Not 4: no alternative was named, such as one delayed queue message per reminder, and nothing said why it lost.
• Depth 2. The only deep dive came when the interviewer asked, and “a lock” names a tool, not a mechanism: after the crash the lock expires and the next worker sends again. A 3 would have chosen the risky part unprompted and worked it through. A missed reminder costs more than a repeated one, so the design sends at least once and keeps repeats rare: each reminder carries a key made of its appointment and due time, which a provider that supports such keys uses to drop a repeat, and the reminder is marked sent before its queue message is acknowledged. A last check of its status just before sending lets a cancellation that arrived in the meantime win.
• Communication 3. Signposted steps and a check at a decision. Not 4: no summary of the decisions, and a one-sentence wrap-up.

Answer A meets the mid bar, and the program above names the gap to senior: one point of depth. The drill follows from it: in the next round, choose the risky part before the interviewer does.

Answer B

• Minute 1. “So it’s basically a cron job. I’ll use a queue and a pool of workers.” Asked whether there are questions about the requirements: “Email or text message? Both, okay.”
• Minute 5. “There will be millions of appointments, so it has to scale.”
• Minute 8. Draws an API, a database, a scheduler, a queue and the senders, joined by arrows without labels. Moved and cancelled appointments are not mentioned.
• Minute 18. Without being asked: “The dangerous part is sending twice or not at all. Each reminder gets a key, its appointment and due time, sent with the request so the provider drops a repeat. The reminder is marked sent before the queue message is acknowledged, so a crash means a retry, not a loss. A provider timeout is retried three times over ten minutes; after that the reminder is logged as failed for the clinic to see.”
• Minutes 8 to 30. Long stretches of drawing in silence; at minute 25 the interviewer asks where things stand.
• Minute 43. “I think that covers it.”

Score Answer B on the four dimensions now. The first four questions below ask for its scores, and a score within one point of the reference counts as right; the explanations give the evidence for each reference score.

Mistakes that cost points

• Scoring what you know instead of what you showed. Knowing about idempotency keys scores nothing if the round never mentions them.
• Averaging. Four scores that average 3 can still miss a bar by a point.
• Rehearsing one prompt until it scores well. A score on a prompt you have practised measures how well you remember that answer, not how you handle a new problem.
• Treating the rubric as a script. The interviewer steers, and how you take the steer is part of the evidence.

Interview questions

What does an interviewer assess in a system design round, if not the final architecture?

How you work, from what the round shows: whether you find the real problem and its numbers, whether you reach a design that works end to end, whether you go deep where it is most likely to break and say what fails, and how clearly you run the discussion. Two different architectures can both do well when each choice follows from the requirements and the reasons are said aloud.

Why do written score descriptions help two interviewers agree?

Each score is tied to something both of them can observe, such as asking the questions that change the design, rather than to an impression. They compare the same evidence with the same descriptions, so they disagree less, and when they do disagree, the difference points to particular moments of the round that they can discuss.

Key takeaways

• A design round is scored from evidence on four dimensions: problem navigation, solution design, technical depth and communication.
• Each score from 1 to 4 has a written description; score your own rounds from a recording and a timed log of evidence, never from memory.
• A level is the highest bar met on every dimension, so an average can hide the gap that matters.
• From fresher to staff the dimensions stay the same; what grows is independence and the width of the judgement.
• After each practice round, work on the dimension furthest below your target, then use a prompt you have not met.