Customer focus and quality questions

“Tell me about a time you put a customer first” is often answered with “I always do”, which gives the interviewer a claim and nothing to write down. A story with 900 support tickets, one change and a measured fall in repeat contacts gives them evidence they can score.

Customer focus and quality questions both look for that evidence: how you found out what users needed, what you changed, what it cost and what users noticed. Below are a customer story told part by part, what makes a quality story count, and a checker for your own.

Customer questions ask how you found out what users needed

Employers that publish their values put the customer near the top, and each describes an action rather than a feeling. Amazon’s Customer Obsession says that leaders “start with the customer and work backwards”, and its preparation page for senior engineers asks candidates to highlight how their decisions or projects affected customers and stakeholders. Microsoft’s interviews look for the ability to “learn about customers’ perspectives” and to show empathy for their experiences. Atlassian puts the customer’s perspective first, “collectively, not just a handful”, Salesforce counts customer success among its core values, and Stripe says that everyone at Stripe talks to users.

So customer focus is shown by how you found out. A customer story has five parts, a MySmartCoPilot pattern: where you found out (tickets, calls, data, using the product yourself), what users needed, what you changed, what it cost and what users noticed. Step through Ananya’s:

For an engineer the customer is often inside the company: the support staff who answer the tickets, the on-call engineer woken by a noisy alert, the team that calls your service. They are internal customersinternal customer: A person or team inside your own organisation who uses what you build, such as the support staff, the on-call engineers or another team that calls your service., and their evidence counts in the same way.

Quality questions ask what standard you held, and what it cost

Quality questions ask about a standard you held when holding it cost something. Amazon’s Insist on the Highest Standards asks leaders to “ensure that defects do not get sent down the line”, and Stripe’s principle Create with craft and beauty says that “well-crafted work indicates care for the user”. Four kinds of story answer quality questions:

Each has a trade-off, because quality costs time, and the story should say how much. Vikram, an invented backend engineer, proposed contract testscontract test: A test that checks that two services still agree on the messages they exchange, such as the fields one sends and the other expects, so that a change to one cannot silently break the other. after two releases in a row broke checkout: they cost about two days of a sprint and a build four minutes slower, and in the six releases since, no change of format has reached customers. Judgement also includes lowering a standard on purpose, such as a throwaway prototype built to answer one question, and saying why.

Connect the work to what users noticed

Technical work reaches customers through a chain, and a strong answer names each link. Vikram’s tests stopped changes of format; those changes had broken checkout; broken checkout meant customers who could not pay. When the work is far from users, follow the chain until it reaches someone:

Stop at the first person who noticed and say what changed for them, with a number when you have one. Claim a link further along only if you measured it.

The payment-failure story, with its evidence worked out Pro: opens with a Pro, Premium or Ultimate pass

Raising the bar, lowering it on purpose, and the numbers behind both Pro: opens with a Pro, Premium or Ultimate pass

Practise: check your two stories

Write one customer story and one quality story, put them in the checker in place of Ananya’s and Vikram’s, and fix what it marks missing:

// Customer and quality stories, checked for the three things that make them evidence: a source that shows what users
// experienced (tickets, calls, data, an incident), a trade-off you accepted to make the change, and a result users could
// notice, with a number. Ananya, Vikram and their stories are invented. Put your own stories in STORIES, one string
// each, and run it again.

const STORIES = [
  {
    name: 'Ananya, customer story, first draft',
    text: 'I am passionate about customers and always put them first. I improved the payment error messages, and users were much happier.',
  },
  {
    name: 'Ananya, customer story, rewritten',
    text: 'Support was getting about 900 tickets a week about failed payments, and in a sample of 200 that I read, most users could not tell whether their money had left their account. I rewrote the failure messages so that each one said whether the money had moved and what to do next, and grouped our 31 bank error codes into 6 messages users could act on. The trade-off was a week of delay to the wallet feature I was due to start, which I agreed with my product manager. Repeat contacts about failed payments fell from 38 % to 12 % of those tickets.',
  },
  {
    name: 'Vikram, quality story, first draft',
    text: 'I always insist on high quality. I added a lot of tests to our services, and our code became much more reliable.',
  },
  {
    name: 'Vikram, quality story, rewritten',
    text: 'Two releases in a row broke checkout because the orders service changed a field the payments service still expected, and both times customers found it before we did. I proposed contract tests between the two services and wrote the first eight myself; the cost was about two days of the sprint and a build four minutes slower. In the six releases since, no change of format has reached customers, and the tests stopped three such changes before release.',
  },
];

// What users experienced, and where you learned it.
const SOURCE = /\b(?:tickets?|support|calls?|complaints?|reviews?|surveys?|conversations?|interviews?|logs?|analytics|dashboards?|recordings?|incidents?|customers found|users reported|reported)\b/i;
// What it cost: a trade-off you accepted, said as one.
const TRADE_OFF = /\b(?:trade-?off|the cost|cost (?:us|me|was)|delay(?:ed)?|instead of|rather than|gave up|accepted|slower|at the expense of)\b/i;
// A result users could notice: a sentence that names them, with a number in it.
const USERS = /\b(?:users?|customers?|support|tickets?|contacts?|complaints?|checkout|sign-ups?|on-call|pages)\b/i;
const NUMBER = /\d|\b(?:one|two|three|four|five|six|seven|eight|nine|ten|half|twice|a third|a quarter|a fifth|no)\b/i;

const sentencesOf = (text) => text.split(/(?<=[.!?])\s+/);
// The first words of a sentence, cut at a space, so that each line of the report stays short.
const shorten = (s) => (s.length <= 64 ? s : `${s.slice(0, s.lastIndexOf(' ', 60))} …`);
let complete = 0;
for (const story of STORIES) {
  const sentences = sentencesOf(story.text);
  const source = SOURCE.exec(story.text);
  const tradeOff = TRADE_OFF.exec(story.text);
  // The result usually closes the story: the last sentence (never the first) that names users and gives a number.
  const result = sentences.slice(1).reverse().find((s) => USERS.test(s) && NUMBER.test(s));
  console.log(story.name);
  console.log(source ? `  ok       a source of evidence: "${source[0]}"` : '  missing  a source of evidence: tickets, calls, data or an incident that showed what users experienced');
  console.log(tradeOff ? `  ok       a trade-off: "${tradeOff[0]}"` : '  missing  a trade-off: what the change cost, and why you accepted it');
  console.log(result ? `  ok       a result users noticed: "${shorten(result)}"` : '  missing  a result users noticed: a sentence about them, with a number');
  console.log('');
  if (source && tradeOff && result) complete++;
}
console.log(`${complete} of ${STORIES.length} stories have all three parts.`);

Ananya, customer story, first draft
  missing  a source of evidence: tickets, calls, data or an incident that showed what users experienced
  missing  a trade-off: what the change cost, and why you accepted it
  missing  a result users noticed: a sentence about them, with a number

Ananya, customer story, rewritten
  ok       a source of evidence: "Support"
  ok       a trade-off: "trade-off"
  ok       a result users noticed: "Repeat contacts about failed payments fell from 38 % to 12 % …"

Vikram, quality story, first draft
  missing  a source of evidence: tickets, calls, data or an incident that showed what users experienced
  missing  a trade-off: what the change cost, and why you accepted it
  missing  a result users noticed: a sentence about them, with a number

Vikram, quality story, rewritten
  ok       a source of evidence: "customers found"
  ok       a trade-off: "the cost"
  ok       a result users noticed: "In the six releases since, no change of format has reached …"

2 of 4 stories have all three parts.

Both first drafts claim a virtue and show none of the three parts. The checker reads words, not truth, so a story that passes still has to be true and yours.

Interview questions

Start with the warm-up and its answer. Then tell each of the other three to an empty room, with a real story of yours, and only then read its first step.

Key takeaways

• A customer story shows where you found out, what users needed, what you changed, what it cost and what users noticed.
• Internal users count: support staff, on-call engineers and the teams that call your service.
• A quality story is a standard you held, what it cost and what it prevented, and lowering a standard on purpose is judgement when you can say why.
• Follow the chain from the work to the first person who noticed, and claim further links only when you measured them.