This page needs a newer browser to run its tools. You can still read everything here.

Data Analysis with Python (pandas, polars, DuckDB) Module 15

Data-quality checks you can automate interview questions

Interview questions on data-quality checks you can automate: warm-ups, then what each harder question tests and how a good answer starts.

11 questions: 3 warm-ups and 8 with model answers · Levels: Fresher to staff

Warm-up questions

Short questions with short answers, each answered in full.

Fresher

What is the difference between profiling a dataset and checking it?

Answered in the lesson Data-quality dimensions and checks in code
Fresher to mid

What does lazy=True change when you validate a DataFrame with pandera?

Answered in the lesson Schema validation and tests: pandera and pytest
Fresher

What is a reconciliation table, and what goes in each row?

Answered in the lesson Reconciliation, audit trails and reproducibility

Questions with model answers

What each question tests, the first step of a good answer and the follow-ups to expect.

Mid to senior

Every data-quality check on yesterday's orders passed, yet finance says the day's revenue in your report is 18 % too low. How can both be true, and which checks were missing?
What it tests

Telling row-level checks from the completeness of a period and from accuracy, reconciling with an outside total, and adding checks with the right severity.

First step

Compare like with like before blaming the data: the same day boundary and time zone, the same statuses and the same treatment of refunds as finance. Then count the day's orders hour by hour. A feed that stopped early passes every row-level check and leaves its last hours empty.
Follow-ups
- The hourly counts look normal, but finance subtracts refunds and you do not. What now?
- How would you check accuracy without access to finance's system?
- Where does each new check run, and what is its severity?
From the lesson Data-quality dimensions and checks in code
Model answer, Pro. Model answers open with any pass; prices are in the notice on this page.
Opening the model answers with your pass…
Model answers open with any MySmartCoPilot pass.
Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45
One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so the model answers can’t be opened at the moment. If you have a pass, enter your licence key.
Choose a pass I have a licence key
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 4 minutes
Senior to staff

Your team runs 40 data-quality checks on a daily pipeline. They raise about 25 warnings a week, people have stopped reading them, and a real problem went unnoticed for nine days. How do you redesign the checks and their alerts?
What it tests

Designing severities, setting thresholds from the history of results, routing alerts to owners, and measuring whether a check is worth keeping.

First step

Start from the saved results of past runs: for each check, count how often it warned or failed and how often that led to a fix. Checks that fire often and never lead to action are the noise; the nine-day problem shows which signal was missing or drowned.
Follow-ups
- How do you set a threshold for a check that has no history yet?
- Which checks would you delete, and how do you justify it?
- How do you show that the redesign works?
From the lesson Data-quality dimensions and checks in code
Model answer, Pro.
In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 3 minutes
Mid to senior

A cleaning function has 30 passing unit tests, yet last week it turned 2 % of amounts into missing values in production, and nobody noticed for four days. What did the tests miss, and what would you add: more fixtures, a schema check at run time, or property-based tests?
What it tests

The gap between example-based tests and production data, run-time schema validation, property-based tests, and where each kind of check belongs.

First step

Find the inputs that produced the missing amounts and compare them with the fixtures. Usually they hold a format the fixtures never contained, such as a thousands separator or a currency sign. That names the missing test; the four days name a missing run-time check.
Follow-ups
- Which property would have caught it before release?
- Where in the pipeline does the schema check run, and what happens when it fails?
- How do you keep fixtures realistic as the source changes?
From the lesson Schema validation and tests: pandera and pytest
Model answer, Pro.
In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 3 minutes
Mid to senior

A partner sends 2 million rows a day, and about 0.3 % fail your pandera schema. Product wants the load to go on. Design what happens to the failing rows, who sees them, and when the load should stop instead.
What it tests

Lazy validation at scale, quarantine design, rules for stopping a load, and bringing corrected rows back without counting them twice.

First step

Agree on what a failing row costs before designing anything: is a missing payment worse than a late one? Then split each run into rows that pass, rows set aside with their failure cases, and a stop rule, such as more than 1 % failing or any failure in a key column.
Follow-ups
- How do you store the failure cases so that the partner can fix them?
- One day the failure rate jumps to 4 %. What happens?
- How do corrected rows get back into the table without being counted twice?
From the lesson Schema validation and tests: pandera and pytest
Model answer, Pro.
In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 3 minutes
Senior to staff

Your analysts maintain pandera schemas for tables that also live in a database. A senior engineer says the rules belong in the database as constraints instead. When is each right, and what do you do with rules that fit neither?
What it tests

What constraints and schemas enforce and report, what each costs, rules across tables and over time, and keeping one source of truth for the rules.

First step

Sort the rules by kind: keys and required values, allowed values and ranges, rules across a row, rules across tables, and rules about shares, trends or freshness. Each kind has a natural home, and the disagreement is usually about the middle kinds only.
Follow-ups
- Which rules can a CHECK constraint not express?
- What does a constraint do to a bulk load with one bad row?
- How do you keep the allowed values in both places from drifting apart?
From the lesson Schema validation and tests: pandera and pytest
Model answer, Pro.
In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 3 minutes
Mid to senior

An auditor picks one order from the source system and asks why it is not in your quarterly revenue report. What must you have kept to answer within minutes, and how do you answer?
What it tests

Audit trail design: removed and changed rows with reasons, versioned inputs and rules, and answering one row's history without re-running the pipeline.

First step

Look the order up in three places: the inputs of the quarter's run (was it in the extract at all?), the trail of removed rows (which step and rule removed it?) and the trail of changes (did its date or key change, so that it moved?). The run log says which inputs and code that run used.
Follow-ups
- The rule that removed it has changed since. What do you show?
- The auditor now asks for every order that rule removed.
- How long do you keep the trail, and what about the personal data in it?
From the lesson Reconciliation, audit trails and reproducibility
Model answer, Pro.
In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 3 minutes
Mid to senior

A scheduled job's output file gets a different SHA-256 hash every night, even when its input files are byte for byte the same. How do you tell whether the results really changed, and how do you make the run reproducible?
What it tests

The sources of difference between runs (timestamps, row order, unseeded sampling, set order, floating-point order, versions), comparing content instead of bytes, and run logs.

First step

Compare the content, not the bytes: load two nights' outputs, sort both by the key and compare them with assert_frame_equal. If they match, the difference is in the file itself, such as a timestamp or the row order; if they do not, look for randomness and for changed versions.
Follow-ups
- The two tables differ only in their row order.
- One column differs in the 15th significant digit.
- How do you rebuild the file a run produced six months ago?
From the lesson Reconciliation, audit trails and reproducibility
Model answer, Pro.
In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 3 minutes
Senior to staff

For the same month, the CRM reports 48,210 active customers, billing reports 46,955 and your analysis 47,388. Leadership wants one number by the end of the week. Plan the reconciliation: what do you compare, in what order, and what do you report?
What it tests

Definitions before data, grain and keys across systems, time windows and test accounts, a reconciliation table between systems, and reporting explained and unexplained differences.

First step

Write down each system's definition of an active customer: the event that counts, the time window and its time zone, and what one customer is (a person, an account or a login). Most of the gap usually comes from definitions, and measuring each one in turn gives the first lines of the reconciliation.
Follow-ups
- Billing counts a customer with two accounts twice.
- After every known cause, 61 customers remain unexplained.
- How do you stop the three numbers drifting apart again?
From the lesson Reconciliation, audit trails and reproducibility
Model answer, Pro.
In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 3 minutes

All Data Analysis with Python (pandas, polars, DuckDB) interview questions · The Data Analysis with Python (pandas, polars, DuckDB) track

Warm-up questions

What is the difference between profiling a dataset and checking it?

What does lazy=True change when you validate a DataFrame with pandera?

What is a reconciliation table, and what goes in each row?

Questions with model answers

Every data-quality check on yesterday's orders passed, yet finance says the day's revenue in your report is 18 % too low. How can both be true, and which checks were missing?

Your team runs 40 data-quality checks on a daily pipeline. They raise about 25 warnings a week, people have stopped reading them, and a real problem went unnoticed for nine days. How do you redesign the checks and their alerts?

A cleaning function has 30 passing unit tests, yet last week it turned 2 % of amounts into missing values in production, and nobody noticed for four days. What did the tests miss, and what would you add: more fixtures, a schema check at run time, or property-based tests?

A partner sends 2 million rows a day, and about 0.3 % fail your pandera schema. Product wants the load to go on. Design what happens to the failing rows, who sees them, and when the load should stop instead.

Your analysts maintain pandera schemas for tables that also live in a database. A senior engineer says the rules belong in the database as constraints instead. When is each right, and what do you do with rules that fit neither?

An auditor picks one order from the source system and asks why it is not in your quarterly revenue report. What must you have kept to answer within minutes, and how do you answer?

A scheduled job's output file gets a different SHA-256 hash every night, even when its input files are byte for byte the same. How do you tell whether the results really changed, and how do you make the run reproducible?

For the same month, the CRM reports 48,210 active customers, billing reports 46,955 and your analysis 47,388. Leadership wants one number by the end of the week. Plan the reconciliation: what do you compare, in what order, and what do you report?