Interviews, contextual inquiry and diaries

The slowest sentence in a user-research session is not the one anyone speaks. It is the one that a researcher rushes to fill when a participant pauses for three seconds, breaking the moment the real detail was about to arrive. The interview and the field session are the same craft performed on different scales: hold the plan lightly, ask about things that already happened, listen for the specific words people use, and let the silences do their work.

This lesson sets out the shape of the interview and the three methods it belongs beside in field research: contextual inquiry when the task lives in a place, diary studies when the behaviour spans days, and remote sessions when travel keeps people out. Every example here uses MealBox or Ward Desk, both reserved under RFC 2606, and every quoted number is drawn for the lesson.

Ask about what happened, not what might happen

Verbal reports are not all equal. In their 1980 paper “Verbal reports as data” in Psychological Review, Ericsson and Simon drew the line that has guided interviewing ever since: people describe what they did and noticed, in specific past episodes, far more reliably than they describe why they did it or what they would do in general. A useful question starts there. “Walk me through the last time you ordered lunch at work” produces a story anchored to one Tuesday; “how do you usually choose lunch?” produces an average the participant has not kept and will often reconstruct from the most recent or most dramatic time.

Four shapes of question tend to appear in a draft guide, and three of them are usually mistakes. The example below reads a 15-question MealBox draft, flags each one, and prints three rewrites that move the whole guide forward:

"""A 15-question MealBox interview guide, scored by a short lint.

Every question below is invented for this lesson. The rules check for the
four problems the GOV.UK Service Manual's page on using in-depth interviews
warns about: leading, hypothetical, double-barrelled and jargon questions.
A fifth rule asks that each section contains at least one past-behaviour
question, following Ericsson and Simon's 1980 note that verbal reports of
specific episodes are far more reliable than reports about generalities.
"""

GUIDE = [
    # section, question
    ("Opening", "Tell me about yourself and what you do."),
    ("Opening", "Would you say you like trying new food?"),                           # leading
    ("Opening", "What makes a good lunch for you?"),

    ("Lunch today", "What did you have for lunch yesterday, and how did you get it?"),
    ("Lunch today", "If MealBox did not exist, what would you be doing instead?"),    # hypothetical
    ("Lunch today", "How often do you order delivery and do you cook on weekends?"),  # double-barrelled

    ("Signing up", "Would you ever use a subscription lunch service?"),               # hypothetical + leading
    ("Signing up", "Walk me through the last time you set up a subscription for anything."),
    ("Signing up", "What did you expect would happen after your first order?"),

    ("The first week", "Tell me about your first week with MealBox."),
    ("The first week", "What was the moment you first thought about cancelling, if any?"),
    ("The first week", "Did you find the UX of the pause flow intuitive?"),           # jargon + leading

    ("Pausing a week", "The last time you wanted to skip a day of your plan, what did you do?"),
    ("Pausing a week", "Was the pause button clear and easy to find?"),               # double-barrelled + leading
    ("Pausing a week", "Would a daily reminder before the kitchen cut-off time help?"),  # hypothetical
]

LEADING = ["would you say", "do you agree", "clear and easy", "intuitive", "wouldn't you", "isn't it"]
HYPOTHETICAL_PHRASES = ["would you ever", "would you use", "if it did not", "if it existed", "what would you be",
                        "would it help", "would a daily", "would a weekly", "if you had", "if the service",
                        "if we built"]
DOUBLE = [" and ", " or "]
JARGON = ["ux", "ui", "cta", "mvp", "funnel", "nps", "sprint", "sop"]


def lint_question(q):
    """Return the list of problems with q, or [] if it is clean."""
    text = q.lower()
    problems = []
    if any(phrase in text for phrase in LEADING):
        problems.append("leading")
    if any(phrase in text for phrase in HYPOTHETICAL_PHRASES):
        problems.append("hypothetical")
    # A conjunction is only double-barrelled when it joins two different asks ("do X and do Y?"), not when it
    # chains a single flow ("the last time X, and what happened next?").
    for conj in DOUBLE:
        parts = text.split(conj)
        if len(parts) >= 2:
            # Count halves that each start a new question verb
            new_question_starts = sum(
                1 for p in parts if p.strip().startswith(("do you", "did you", "how often", "how much"))
            )
            if new_question_starts >= 2:
                problems.append("double-barrelled")
                break
    for term in JARGON:
        if any(text[i:i + len(term) + 1] == f" {term}" for i in range(len(text) - len(term))):
            problems.append(f"jargon: {term}")
            break
    return sorted(set(problems))


def lint_guide(guide):
    """Return a per-question report and a section-level 'past-behaviour present' check."""
    report = []
    sections = {}
    for section, q in guide:
        problems = lint_question(q)
        report.append((section, q, problems))
        sections.setdefault(section, []).append((q, "the last time" in q.lower() or "walk me through" in q.lower()))
    section_findings = {s: not any(past for _, past in rows) for s, rows in sections.items()}
    return report, section_findings


report, missing_past = lint_guide(GUIDE)

print("Interview-guide lint on a 15-question MealBox draft")
print(f"{sum(1 for _, _, p in report if not p):>2} of {len(report)} questions pass as written")
print()

issues = [(s, q, p) for s, q, p in report if p]
for s, q, p in issues:
    print(f"  [{s:<13}] {', '.join(p)}")
    print(f"    - {q}")

print()
missing = [s for s, ok in missing_past.items() if ok]
if missing:
    print(f"Sections without a past-behaviour question: {', '.join(missing)}")
else:
    print("Every section has at least one past-behaviour question.")

print()
print("Three rewrites that move the whole guide forward")
print("  'Would you say you like trying new food?' ->")
print("    'The last time you tried a new food, what was it and what made you pick it?'")
print("  'Did you find the UX of the pause flow intuitive?' ->")
print("    'The last time you paused a plan, what did you do, step by step?'")
print("  'If MealBox did not exist, what would you be doing instead?' ->")
print("    'How did you handle lunch at work before you signed up?'")

Interview-guide lint on a 15-question MealBox draft
 8 of 15 questions pass as written

  [Opening      ] leading
    - Would you say you like trying new food?
  [Lunch today  ] hypothetical
    - If MealBox did not exist, what would you be doing instead?
  [Lunch today  ] double-barrelled
    - How often do you order delivery and do you cook on weekends?
  [Signing up   ] hypothetical
    - Would you ever use a subscription lunch service?
  [The first week] jargon: ux, leading
    - Did you find the UX of the pause flow intuitive?
  [Pausing a week] leading
    - Was the pause button clear and easy to find?
  [Pausing a week] hypothetical
    - Would a daily reminder before the kitchen cut-off time help?

Sections without a past-behaviour question: Opening, Lunch today, The first week

Three rewrites that move the whole guide forward
  'Would you say you like trying new food?' ->
    'The last time you tried a new food, what was it and what made you pick it?'
  'Did you find the UX of the pause flow intuitive?' ->
    'The last time you paused a plan, what did you do, step by step?'
  'If MealBox did not exist, what would you be doing instead?' ->
    'How did you handle lunch at work before you signed up?'

• Leading questions suggest the answer: “do you agree that the pause flow is clear?” or “would you say the food is good?”. The participant tends to agree, especially under the social pressure of an interview, and the data goes with the lead.
• Hypothetical questions ask about a design or situation that does not yet exist: “would you use a weekly digest email?”. People answer agreeably, and the answer does not survive the real thing.
• Double-barrelled questions ask two things at once: “do you order delivery and do you cook at home on weekends?”. The split answers do not line up with each other and are hard to analyse.
• Jargon questions use team words that live only in meeting rooms: “did you find the UX of the pause flow intuitive?”. Participants either guess, or politely nod along to a word they do not recognise.

Rewrite each one as a question about a specific past episode: “the last time you paused a plan on any service, what did you do, step by step?” is one of the strongest shapes in the whole field, and it works for interviews, contextual visits and diary prompts alike.

Moderation is the small things done the right way

A session is a conversation with a shape. Rapport, grand tour, probes, themes and debrief, which the Service Manual’s page on in-depth interviews calls the arc of a discussion guide, each do a specific job. The scene below walks the five moments, from the opening minute’s rapport to the debrief at the end:

Three habits make the small moments do what they should. The first is neutral probes. “What happened next?”, “what did you expect?”, “you said X — can you tell me more?” each keep the participant’s own words rather than substituting the researcher’s; “did you find that annoying?” does the opposite, and the answer that follows comes coloured by the word. The second habit is silence. The researcher’s job is to try to stay quiet, as the Service Manual puts it; the moment a participant pauses to think is where the detail that was not in the first answer arrives. The third is the plan held lightly. When a participant goes off topic, the Service Manual recommends waiting for a natural break and then gently bringing them back, rather than cutting the detour short: most of what the plan did not expect is more valuable than what it did.

The language of the session matters as much as its shape. Interview in the participant’s own language where possible. Interpreters tend to summarise, so ask for literal translation of key phrases, and record code-switching — for example, between Hindi and English in the same sentence — as it happens, rather than “correcting” it; the code-switch itself is often the data. Confirm numbers and names by reading them back (“so you said eighteen hundred rupees, is that right?”) rather than paraphrasing them away.

Contextual inquiry, when the task lives in its room

Interviews can describe a place; contextual inquiry carries it with you. The method comes from Beyer and Holtzblatt’s apprenticeship model, which the Service Manual’s page on contextual research and observation sets out as visits to the real environment where a task happens, so you can see the paper on the desk, the phone that rings, and the person at the next chair who the participant turns to. For MealBox, that is the office kitchenette at 1 p.m.; for Ward Desk’s complaint handling, it is the contact centre and the ward engineer’s van.

The Service Manual gives three ways to observe during a visit, each with a trade-off. Silent observation keeps the task natural, and leaves the researcher guessing why. Occasional questions interrupt the flow but give reasons for what was observed. Think-aloud gives the richest reasons and distorts the task the most. Pick one mode per visit and say which, and keep in mind that the choice itself changes the data.

Visits usually run 1 to 3 hours, with an upper limit of 4 one-hour or 2 three-hour visits in a day, because observation is tiring. The Service Manual recommends working in pairs for safety, note-taking and so more of the team meets participants in person. Four field methods differ in what they will show, which the table below summarises:

In practice, contextual visits for a service like Ward Desk pay off even when the participant number is small. Two visits to the complaints contact centre, with permission from the staff and a short information sheet for anyone who passes the recording, usually turn up what a dozen interviews with citizens cannot — the paper form that nobody shows the citizen, the shared screen that the engineer trusts and the SMS that is sent only sometimes.

Diary studies for what happens between sessions

Some things are not describable in an interview because they happened on four different days. Diary studies capture behaviour as it happens: short, timed prompts over 7 to 14 days ask the participant to send a note, a photo or a short answer, so when the follow-up interview comes it is grounded in a specific moment rather than a reconstructed average.

Three practical questions decide the shape of a diary study. How often? Daily prompts work for a two-week plan; longer studies drop off quickly. Which channel? In India, messaging apps are the honest answer for the largest reach, with an email or an in-app form for people who prefer them. How much work per prompt? Short wins: one message, one line, one photo if useful; a form with five fields is a prompt people miss on busy days.

Messaging apps need special care, because they move personal data to a third party. The Service Manual’s rules on managing research data cover this: get consent for the specific channel, collect only what the study needs, and export and delete the messages from the service when the study ends. Say so plainly on the information sheet, and keep a short retention period in sight.

Remote sessions and the three columns of good notes

Remote sessions by phone or video call include people who cannot travel and keep research moving on a budget. The Service Manual’s page on remote user research lists the practical differences from a face-to-face session: you lose some non-verbal cues, you need the participant to say what they are doing on their screen if you cannot see it, and you need an audio-only fallback for a connection that drops. Plan the fallback in advance (a shared phone number, a quick way to restart the recording) so the moment is not improvised on the day.

Note-taking is where remote and in-person sessions converge. The habit the lesson comes back to is three columns. In Observation, write what happened: what the participant said, in your short words, with the timestamp. In Quote, write the words the participant used, in quotation marks, with the timestamp. In Interpretation, write what the note-taker thinks it means, which is a hypothesis for analysis to check later. Keeping the three visibly apart stops interpretations from being treated as findings, and stops the quote from losing its own voice in a summary.

Allow one hour of analysis for every two hours of research, as the GOV.UK Service Manual asks, and do it the same day where possible. Themes written as statements (“new subscribers do not know the kitchen cut-off time”) carry further than themes written as topics (“kitchen cut-off”).

The next lesson of this module, surveys and behavioural data, picks up the counting part of a round: the margin that goes beside the pattern a round of interviews finds, and the funnel that shows how common the behaviour is.

Practise

Ten MealBox interview questions, each with a planted problem; rewrite them so a lint identical to the lesson’s example passes, and keep one past-behaviour question in every section:

python -m pytest test_guide.py

"""Starter: rewrite each question so the lint in test_guide.py passes."""

GUIDE = [
    ("Opening", "Would you say you like trying new food?"),
    ("Opening", "Tell me about yourself."),
    ("Lunch today", "If MealBox did not exist, what would you be doing instead?"),
    ("Lunch today", "How often do you order delivery and do you cook on weekends?"),
    ("Signing up", "Would you ever use a subscription lunch service?"),
    ("Signing up", "What did you expect would happen after your first order?"),
    ("The first week", "Did you find the UX of the pause flow intuitive?"),
    ("Pausing a week", "Was the pause button clear and easy to find?"),
    ("Pausing a week", "Would a daily reminder before the kitchen cut-off time help?"),
    ("Pausing a week", "How often do you want to pause and would it be useful to skip single days?"),
]

from collections import defaultdict

from guide import GUIDE

LEADING = ["would you say", "do you agree", "clear and easy", "intuitive", "wouldn't you", "isn't it"]
HYPOTHETICAL = ["would you ever", "would you use", "if it did not", "if it existed", "what would you be",
                "would it help", "would a daily", "would a weekly", "if you had", "if the service",
                "if we built"]
JARGON = ["ux", "ui", "cta", "mvp", "funnel", "nps", "sprint", "sop"]


def is_double_barrelled(text):
    """Does this ask two things joined by a conjunction?"""
    for conj in (" and ", " or "):
        parts = text.split(conj)
        if len(parts) >= 2 and sum(
            1 for p in parts if p.strip().startswith(("do you", "did you", "how often", "how much"))
        ) >= 2:
            return True
    return False


def check(q):
    """Return a list of lint problems with q, or [] if it is clean."""
    text = q.lower()
    problems = []
    if any(phrase in text for phrase in LEADING):
        problems.append("leading")
    if any(phrase in text for phrase in HYPOTHETICAL):
        problems.append("hypothetical")
    if is_double_barrelled(text):
        problems.append("double-barrelled")
    for term in JARGON:
        if any(text[i:i + len(term) + 1] == f" {term}" for i in range(len(text) - len(term))):
            problems.append(f"jargon: {term}")
            break
    return sorted(set(problems))


def is_past_episode(q):
    text = q.lower()
    return "the last time" in text or "walk me through" in text


def test_every_question_passes_the_lint():
    """None of the ten rewritten questions is leading, hypothetical, double-barrelled or jargon-heavy."""
    for section, q in GUIDE:
        problems = check(q)
        assert not problems, f"{section!r} question {q!r}: {problems}"


def test_each_section_has_a_past_behaviour_question():
    """Every section still carries at least one question that asks about a specific past episode."""
    sections = defaultdict(list)
    for section, q in GUIDE:
        sections[section].append(q)
    for section, qs in sections.items():
        assert any(is_past_episode(q) for q in qs), f"{section!r} has no 'the last time' or 'walk me through' question"


def test_number_of_questions_unchanged():
    """Keep the guide the same length as the starter (ten questions)."""
    assert len(GUIDE) == 10, "rewrite the ten questions, do not add or remove any"


def test_every_question_is_a_sentence():
    """Every rewritten question is a non-trivial, question-shaped sentence."""
    for section, q in GUIDE:
        assert isinstance(q, str) and len(q) >= 20, f"{section!r}: question too short"
        assert q.strip().endswith(("?", ".")) or "walk me through" in q.lower(), \
            f"{section!r}: end the question with ? or '.'"

The self-check below moves from question shapes to the note-taking split:

Interview questions

Warm-up (fresher to 2–5 years): why do interview questions ask about specific past episodes?

People describe a specific past episode far more reliably than they describe a general practice. Ericsson and Simon’s 1980 paper, “Verbal reports as data” in Psychological Review, drew the line clearly: reports of what somebody actually did, in a particular case, carry more weight than reports of what they “usually” do or what they “would” do. A grand-tour question anchors the session to one real day — “walk me through the last time you ordered lunch at work” — and the probes travel through what happened step by step, in the participant’s own words. General and hypothetical questions, by contrast, invite answers that average memories the participant never kept, often weighted by what came most to mind.

Two harder questions follow: one deals with a participant who answers in three words, the other plans a contextual visit inside a contact centre. Try your own approach first, then look at the opening step each question publishes:

Key takeaways

• Interview questions start from specific past episodes and keep the participant’s own words through neutral probes, with silence as a probe of its own.
• A discussion guide is a list of topics with starter questions and probes, not a script; the moderator holds the plan lightly and comes back from a side road at the next natural break.
• Contextual inquiry carries the real place with it, which an interview room cannot do, and pays off even with two or three visits when the task lives among paper, people and other tools.
• Diary studies capture behaviour as it happens over days or weeks, with short prompts, the right consent for the channel and a prompt deletion schedule.
• Session notes keep observation, quotes and interpretation visibly apart, so interpretations stay hypotheses until analysis checks them.