UI/UX Design Module 11 – Usability testing and UX measurement
Heuristic evaluation and cognitive walkthroughs
Inspect a design before users see it, with several evaluators and Nielsen's ten heuristics, a step-by-step walk of a new task, and findings a team can fix.
What you will learn
- Run a heuristic evaluation with three evaluators and merge their findings
- Rank merged findings by severity and write each with evidence, heuristic and fix
- Run a cognitive walkthrough of a task someone does for the first time
- Choose between a heuristic evaluation, a cognitive walkthrough and a usability test
- Explain what inspection methods miss and what to run after them
Before you start
What you’ll learn · Before you start
What you will learn
- Run a heuristic evaluation with three evaluators and merge their findings
- Rank merged findings by severity and write each with evidence, heuristic and fix
- Run a cognitive walkthrough of a task someone does for the first time
- Choose between a heuristic evaluation, a cognitive walkthrough and a usability test
- Explain what inspection methods miss and what to run after them
Before you start
On this page
TiffinNow, a lunch-delivery app drawn for this lesson, has five screens and twelve usability problems planted in them. Its first usability test with customers is three weeks away, and every problem fixed before then leaves the participants’ time for the problems only they can show.
A usability inspection: Judging an interface by the knowledge of evaluators rather than by watching users try it; heuristic evaluations and cognitive walkthroughs are two kinds. finds problems without participants, through people who know what to look for. A heuristic evaluation: An inspection in which a few evaluators each check an interface alone against a short list of usability heuristics, then pool the problems they found. has a few evaluators each check every screen alone against a short list of rules of thumb, then pool what they found. cognitive walkthrough: An inspection in which a small group follows one task step by step from the point of view of a first-time user, asking the same questions about every step. follow one task step by step and ask, at each step, whether someone using the app for the first time would get through it. Both are quick and cheap; neither replaces watching real users.
A heuristic evaluation checks each screen against ten rules
On TiffinNow’s slot screen, nothing changes for eight seconds after Pay is pressed. A customer cannot tell whether the payment is going through, and some will press again. The screen breaks a known rule of thumb: a design should always show people what is going on.
Jakob Nielsen and Rolf Molich described heuristic evaluation at the CHI conference in 1990, and Nielsen’s ten usability heuristic: A rule of thumb used to judge an existing interface, applied by walking through it against a known list of common problems. More in “What UX design is and how teams practise it” each name a promise an interface should keep, from showing its status to offering help where it is needed. The program prints TiffinNow’s twelve drawn problems as a grid of screens and heuristics. Run it, then add a problem you would expect, with its heuristic:
"""TiffinNow, an app drawn for this lesson: the twelve usability problems planted in its five screens.
Each problem is kept as (screen, heuristic number, what a person meets there). The program prints a grid of the
screens against Nielsen's ten heuristics, then every problem under the heuristic it breaks. Add a problem you would
expect to PROBLEMS, with the heuristic you think it breaks, and run the program again.
"""
HEURISTICS = {
1: "Visibility of system status",
2: "Match between the system and the real world",
3: "User control and freedom",
4: "Consistency and standards",
5: "Error prevention",
6: "Recognition rather than recall",
7: "Flexibility and efficiency of use",
8: "Aesthetic and minimalist design",
9: "Help users recognize, diagnose, and recover from errors",
10: "Help and documentation",
}
SCREENS = ["home", "meal", "cart", "slot", "track"]
PROBLEMS = [
("home", 2, "kitchens are listed by code (K-14), not by the cook's name"),
("home", 7, "no way to order yesterday's tiffin again"),
("meal", 4, "the add button says Proceed here and Add on home"),
("meal", 6, "allergens are on the home card, not on the meal page"),
("cart", 3, "a tiffin can be added but never removed"),
("cart", 10, "the 10 a.m. order cut-off is explained nowhere"),
("slot", 5, "a full delivery slot can still be picked"),
("slot", 1, "nothing changes for 8 seconds after Pay is pressed"),
("slot", 9, "a failed payment says only Error 402"),
("track", 1, "the page never updates after it opens"),
("track", 2, "the stage reads DISPATCH_PENDING"),
("track", 8, "a promo carousel pushes the delivery time off screen"),
]
print(f"TiffinNow: {len(SCREENS)} screens, {len(PROBLEMS)} drawn problems")
print("screen " + " ".join(f"H{n:<2}" for n in HEURISTICS))
for screen in SCREENS:
cells = []
for n in HEURISTICS:
count = sum(1 for s, h, _ in PROBLEMS if s == screen and h == n)
cells.append(f"{count or '.':<3}")
print((f"{screen:<7}" + " ".join(cells)).rstrip())
for n, name in HEURISTICS.items():
print(f"\nH{n} {name}")
for s, h, what in PROBLEMS:
if h == n:
print(f" {s}: {what}")
broken = len({h for _, h, _ in PROBLEMS})
per_screen = [f"{s} {sum(1 for p in PROBLEMS if p[0] == s)}" for s in SCREENS]
print(f"\nHeuristics broken at least once: {broken} of {len(HEURISTICS)}")
print("Problems per screen: " + ", ".join(per_screen))Output
TiffinNow: 5 screens, 12 drawn problems screen H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 home . 1 . . . . 1 . . . meal . . . 1 . 1 . . . . cart . . 1 . . . . . . 1 slot 1 . . . 1 . . . 1 . track 1 1 . . . . . 1 . . H1 Visibility of system status slot: nothing changes for 8 seconds after Pay is pressed track: the page never updates after it opens H2 Match between the system and the real world home: kitchens are listed by code (K-14), not by the cook's name track: the stage reads DISPATCH_PENDING H3 User control and freedom cart: a tiffin can be added but never removed H4 Consistency and standards meal: the add button says Proceed here and Add on home H5 Error prevention slot: a full delivery slot can still be picked H6 Recognition rather than recall meal: allergens are on the home card, not on the meal page H7 Flexibility and efficiency of use home: no way to order yesterday's tiffin again H8 Aesthetic and minimalist design track: a promo carousel pushes the delivery time off screen H9 Help users recognize, diagnose, and recover from errors slot: a failed payment says only Error 402 H10 Help and documentation cart: the 10 a.m. order cut-off is explained nowhere Heuristics broken at least once: 10 of 10 Problems per screen: home 2, meal 2, cart 2, slot 3, track 3
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 drawn_problems.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
Every heuristic is broken at least once, and the slot and track screens hold three problems each. A real evaluation has no such answer key, which is why Nielsen Norman Group (NN/g) asks for three to five evaluators working independently: each person misses some problems, and different people miss different ones. Shneiderman’s eight golden rules and the interaction principles of ISO 9241-110:2020 are lists of the same kind, and Nielsen suggests extra rules for a product’s own category.
A heuristic number is not a finding
“Slot: breaks heuristic 1” names a rule but not what goes wrong, so nobody can judge it or test a fix. Write what a person meets, then the rule: “Slot: nothing changes for 8 seconds after Pay is pressed, so people may pay twice (visibility of system status).”
Checkpoint
Check: Heuristics and findings
Cognitive walkthroughs follow one task as a newcomer would
A heuristic evaluation looks at every screen; a walkthrough follows one task through a newcomer’s eyes. Clayton Lewis, Peter Polson, Cathleen Wharton and John Rieman presented the method in 1990 for walk-up-and-use machines such as kiosks, and teams today use a shorter practitioner’s version by Wharton and colleagues.
Cognitive walkthroughs suit learnability: How easily people manage a task the first time they meet an interface, without training or help from someone who already knows it. questions: a flow that is new to the people who will use it, or a task they do too rarely to remember. The group agrees on the user, often a persona: A written description of a behaviour pattern, with its goals and context, shared by several real people, in which every line can be traced to the research notes it rests on. More in “Personas, jobs to be done and scenarios”, and on the correct steps of the task, then questions each step in turn. A checkout that follows the pattern every shopper knows needs no walkthrough; a first order that sets up a weekly plan may.
Three facts about your design settle most of the choice; give the table your answers to see which methods still fit:
Answer the questions above: the table then marks what fits your case and says why the rest does not.
| Method | What it covers | Whose view it takes | What it needs | What it cannot see |
|---|---|---|---|---|
| Heuristic evaluation Who does the work: 3 to 5 evaluators, alone | Good every screen in scope, each judged against the whole list of heuristics | Mixed the evaluators' trained eye, rather than any one user's goal | Good a prototype or even paper screens, and an hour or two from each evaluator | Poor problems only real users show, and it flags some that nobody would ever meet |
| Cognitive walkthrough Who does the work: a small group together | Mixed one task, step by step, and nothing off its path | Good a first-time user of a named kind, at every step of the task | Mixed the correct steps written down first, then a session with the whole group | Poor screens off the chosen path, and what users really know or want |
| Usability test Who does the work: participants and a moderator | Mixed the tasks you set, as real people attempt them | Good the people the product is for, in their own words | Poor recruiting, consent and a moderator: days of work rather than hours | Good little within its tasks, though a small round can miss rare problems |
How the answers decide
- “Is the task new to the people who will do it, or one they do too rarely to remember?” No, they know it well: adds a caveat to Cognitive walkthrough. A walkthrough of a pattern people already know mostly confirms what everyone expected; keep the effort for steps that are new to them.
- “Is the task new to the people who will do it, or one they do too rarely to remember?” Yes, new or rare: adds a caveat to Heuristic evaluation. Heuristics judge each screen on its own, so they can pass a step that looks fine but that a newcomer would never think to take.
- “Do you need every screen checked, or one task followed closely?” Every screen: adds a caveat to Cognitive walkthrough. A walkthrough looks only at the steps of the task it follows; screens off that path are never examined.
- “Do you need every screen checked, or one task followed closely?” One task: adds a caveat to Heuristic evaluation. A heuristic evaluation can be scoped to the task's screens, but it judges each screen rather than the order the task takes through them.
- “Can you reach people from the target group before the decision is due?” No, not in time: rules out Usability test. Without people from the target group there is no test to run before the decision; inspect now, and test as soon as they can be reached.
- “Can you reach people from the target group before the decision is due?” Yes, in time: adds a caveat to Heuristic evaluation and Cognitive walkthrough. Inspect first to clear the obvious problems, then test, because only the test shows what real users actually meet.
The three are not rivals: a heuristic evaluation covers many screens, a walkthrough goes deep on one new task, and a usability test checks both with real users.
Checkpoint
Check: Choosing an inspection
Running a heuristic evaluation with three evaluators and merging the findings Pro: opens with a Pro, Premium or Ultimate pass
One evaluator finds only some of an interface’s problems, and which ones depends on who it is. This part runs TiffinNow’s evaluation with three evaluators working alone, merges their lists and rates what remains on the 0 to 4 scale.
In this part:
- Setting up the evaluation: scope, a task and the shape of a finding
- Merging three lists into one, with the coverage they reach
- Rating severity on the 0 to 4 scale, with four worked problems
1 diagram · 1 runnable example (Python) · about 8 minutes
Opening this part with your pass…
This part opens with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; nothing renews unless you choose to. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so this part can’t be opened at the moment. The free part above stands on its own. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
Inspections come before a usability test, never instead of it
Neither inspection watches a user, so both judge only what the team can imagine. They cannot show which words customers expect, what they are really trying to do, or how a busy kitchen or a weak network changes the task, and some of what they flag no real customer ever meets. Step through where both fit in TiffinNow’s three weeks:
The drawing shows the whole scene; its 5 steps are listed below.
Every step as text
- Screens before users. TiffinNow's five screens can be clicked through, and its first usability test with customers is three weeks away. Two inspections can start this week, before a single participant is booked.
- Every screen, several eyes. A heuristic evaluation covers all five screens: a few evaluators check each one on their own against the ten heuristics, then pool their lists.
- One task, as a newcomer. A cognitive walkthrough follows one task that is new to customers, the group order for an office, step by step and from the point of view of someone doing it for the first time.
- Fix what both turned up. The team fixes the worst problems the two inspections found, so the test does not spend its sessions on problems the team could have seen for itself.
- Then watch real customers. The usability test shows what no inspection can: the words customers expect, the goals they bring, and which of the predicted problems never happen at all.
Each inspection costs only the team’s hours, so the test’s few participants are kept for what only customers can show. NN/g gives one more reason to inspect first: a heuristic evaluation’s findings can show which parts of the design the test should target.
Checkpoint
Check: Inspecting before testing
Running a cognitive walkthrough and writing the findings up Pro: opens with a Pro, Premium or Ultimate pass
A cognitive walkthrough gives up breadth for depth: one task, one kind of newcomer, every step questioned. This part walks MealBox’s first order, sets out what both inspections miss, and turns findings into a list a team can fix.
In this part:
- The four questions, asked at every step of MealBox’s first order
- What both inspections miss, and what finds it
- Findings a team can act on: evidence, heuristic, severity and fix
- Practise: merge and rank Ward Desk’s findings
- Merge three evaluators' findings and rank them by severity
- Results of the sample tests
1 runnable example (Python) · 1 exercise with sample tests · about 10 minutes
Opening this part with your pass…
This part opens with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; nothing renews unless you choose to. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so this part can’t be opened at the moment. The free part above stands on its own. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
Practise
The questions ask which heuristic a problem breaks, which method a design question needs, and why inspection comes before testing.
Practice that adapts to you
Practise this lesson
11 questions about 8 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.
Interview questions
Warm-up (fresher to 2–5 years): which comes first, a heuristic evaluation or a usability test, and how do they differ?
In a usability test, people from the intended audience try real tasks while someone watches, so every problem it finds is one a real user met. In a heuristic evaluation, a few evaluators judge the interface against a list of heuristics with no users present: it is faster and cheaper, but it rests on their judgement and flags some problems no user would notice. Inspect first when a design is new or rough, so the test’s participants spend their time on what only they can show.
Each question below sets a harder case; note your first two moves, then open its row:
-
Mid to senior
MealBox's only designer is asked to run a heuristic evaluation of the new gift-plan flow alone on Saturday and fix whatever they find before Monday's launch. What do you advise, and what changes if nobody else can help?
-
Mid to staff
Ward Desk officers will move complaints between departments in a new three-screen flow they will use a few times a month. A heuristic evaluation of the three screens found two cosmetic problems, and an engineer says that settles it. How do you respond, and what do you run next?
-
Mid to senior
An agency's heuristic evaluation of MealBox arrives as 40 findings, ranked by the agency's single evaluator. The product manager wants the top ten fixed this sprint, and two of your engineers dispute several of the severities. What do you do with the report?
Flashcards · 7 cards
Recall it, then check
Say the answer to yourself before you look: recalling it is what makes it stick.
-
What is a usability inspection?
A way to find usability problems without participants, through people who know what to look for. Heuristic evaluations and cognitive walkthroughs are both quick and cheap, and neither replaces watching real users.
-
How does a heuristic evaluation work?
A few evaluators each check every screen alone against a short list of rules of thumb, such as Nielsen's ten usability heuristics, and then pool what they found.
-
Why does Nielsen Norman Group ask for three to five evaluators working independently?
Each person misses some problems, and different people miss different ones, so the pooled list holds what each evaluator alone would miss.
-
What turns a broken heuristic into a finding someone can act on?
Writing what a person meets before the rule it breaks: Slot: nothing changes for 8 seconds after Pay is pressed, so people may pay twice (visibility of system status). A heuristic number alone names a rule, not what goes wrong.
-
Which lists besides Nielsen's ten heuristics can an evaluation use?
Shneiderman's eight golden rules and the interaction principles of ISO 9241-110, plus extra rules for the product's own category.
-
What is a cognitive walkthrough, and when does it suit?
A group agrees on the user, often a persona, and the correct steps of one task, then asks at each step whether someone using the product for the first time would get through it. It suits learnability questions: a flow new to its users, or a task done too rarely to remember.
-
Why do inspections come before a usability test, never instead of it?
Neither watches a user, so both judge only what the team can imagine: they cannot show which words customers expect or how a busy kitchen or a weak network changes the task, and some of what they flag no customer meets. Inspecting first keeps the test's few participants for what only they can show.
Key takeaways
- A heuristic evaluation checks every screen against a short list of rules of thumb, and each finding names what a person meets before the heuristic it breaks.
- Several evaluators work alone and then pool their lists, because each one misses problems the others catch.
- Cognitive walkthroughs follow one task step by step from a newcomer’s point of view, and they suit flows that are new or rarely used.
- Both inspections judge only what the team can imagine, so they come before a usability test, never instead of it.
Terms in this lesson
- cognitive walkthrough
- An inspection in which a small group follows one task step by step from the point of view of a first-time user, asking the same questions about every step.
- heuristic evaluation
- An inspection in which a few evaluators each check an interface alone against a short list of usability heuristics, then pool the problems they found.
- learnability
- How easily people manage a task the first time they meet an interface, without training or help from someone who already knows it.
- usability inspection
- Judging an interface by the knowledge of evaluators rather than by watching users try it; heuristic evaluations and cognitive walkthroughs are two kinds.
References
- Heuristic evaluation of user interfaces (ACM (Proceedings of CHI '90))
- 10 Usability Heuristics for User Interface Design (Nielsen Norman Group)
- How to Conduct a Heuristic Evaluation (Nielsen Norman Group)
- The Theory Behind Heuristic Evaluations (Nielsen Norman Group)
- Severity Ratings for Usability Problems (Nielsen Norman Group)
- A mathematical model of the finding of usability problems (ACM (Proceedings of INTERCHI '93))
- Evaluate Interface Learnability with Cognitive Walkthroughs (Nielsen Norman Group)
- Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces (ACM (Proceedings of CHI '90))
- Cognitive walkthroughs: a method for theory-based evaluation of user interfaces (International Journal of Man-Machine Studies (Elsevier))
- ISO 9241-110:2020 Ergonomics of human-system interaction, Part 110: Interaction principles (International Organization for Standardization (ISO))
- The Eight Golden Rules of Interface Design (University of Maryland (Ben Shneiderman))
Related tools
Report a problem with this lesson
Kept only in this browser. Your Learn progress