Data Engineering Module 1
Foundations: what data engineers build interview questions
Interview questions on “Foundations: what data engineers build”: warm-ups, then what each harder question tests and how a good answer starts.
Warm-up questions
Short questions with short answers, each answered in full.
-
Fresher
What does a data engineer build, and how does the role differ from a data analyst?
Answered in the lesson What data engineering is, and what it is not
-
Fresher to mid
How do a data warehouse, a data lake and a lakehouse differ?
Answered in the lesson The data lifecycle from source to serving
-
Fresher
What does it mean to pin a version, and why does a data lab pin one?
-
Fresher
How do an SLI, an SLO and an SLA fit together for a data pipeline?
Answered in the lesson Pipeline requirements: SLAs, freshness, sizing
Questions with model answers
What each question tests, the first step of a good answer and the follow-ups to expect.
-
Fresher to mid
A team lead asks you to explain the difference between a data pipeline and a SQL query. What do you say, and what example would you give to make it concrete?
- What it tests
- Distinguishing a one-time query from a repeatable, scheduled, tested pipeline; naming the components that make a pipeline trustworthy (schema checks, retries, idempotency, monitoring).
- First step
- Start with what they have in common: both answer questions about data with SQL or code. Then say what a pipeline adds: it runs on a schedule, it retries on failure, it checks the data it produces and it can be backfilled without duplicating rows.
- Follow-ups
-
- What happens when a query is promoted to a pipeline?
- How would you test a pipeline before it runs in production?
- What changes when the pipeline must run every five minutes instead of every day?
From the lesson What data engineering is, and what it is not
Model answer, Pro. Model answers open with any pass; prices are in the notice on this page.Opening the model answers with your pass…
Model answers open with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so the model answers can’t be opened at the moment. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
In the answer:
- Model answer
- Common mistakes
- Rubric
- Variants
- about 5 minutes
-
Mid to senior
An ML engineer asks you to build a feature table for a fraud-detection model. What questions do you ask before you start, and what must the pipeline guarantee?
- What it tests
- Understanding feature requirements (point-in-time correctness, freshness, backfill), asking the right questions (schema, update frequency, historical depth), and knowing what can go wrong (data leakage, training-serving skew).
- First step
- Ask what features they need (columns, types, meaning), how fresh the features must be at prediction time, how far back training data should go, and whether they need features at the event time (point-in-time correct) or at the current time.
- Follow-ups
-
- How do you prevent data leakage in a feature table?
- The model uses a "30-day rolling average" feature. What does the pipeline need to store?
- How would you test that the pipeline produces the same features offline and online?
From the lesson What data engineering is, and what it is not
Model answer, Pro.In the answer:
- Model answer
- Common mistakes
- Rubric
- Variants
- about 5 minutes
-
Mid to senior
Your CDC connector crashes and loses its place in the source database's transaction log. What do you do, and how do you prevent duplicates from reaching downstream tables?
- What it tests
- Understanding at-least-once delivery, offset management, deduplication patterns and the trade-off between data loss and data duplication.
- First step
- Check the last committed offset the connector stored. Replay from that offset, accepting that you may re-read some events, and deduplicate on the target side by merging on the event's primary key.
- Follow-ups
-
- The connector stored no offset at all and the log has been truncated. Now what?
- How would you detect that duplication happened after the fact?
- What if the source table has no natural key?
From the lesson The data lifecycle from source to serving
Model answer, Pro.In the answer:
- Model answer
- Common mistakes
- Rubric
- Variants
- about 5 minutes
-
Mid to senior
A data-minimisation rule says personal data must be dropped before it leaves the source region. Your team uses ELT everywhere. What changes, and how do you implement the transform before the load?
- What it tests
- Knowing when ELT cannot apply, how to add an ETL step for compliance, and the operational cost of maintaining two patterns.
- First step
- ELT loads raw data first, which would include the personal data. You need an ETL step: transform at the edge, inside the source region, before the data is sent elsewhere. That transform drops or masks the personal columns.
- Follow-ups
-
- How do you test that the transform never lets personal data through?
- What if a downstream model later needs a field you dropped?
- How do you handle late-arriving corrections to the dropped field?
From the lesson The data lifecycle from source to serving
Model answer, Pro.In the answer:
- Model answer
- Common mistakes
- Rubric
- Variants
- about 4 minutes
-
Fresher to mid
A daily report printed 4,120 last month and prints 4,118 today, with no change in version control. A teammate cannot reproduce either number. How do you find the cause and stop it recurring?
- What it tests
- Treating version drift as the first suspect when code did not change, finding it by comparing the resolved toolchain, and preventing it by pinning and recording versions rather than by patching the number.
- First step
- Do not touch the query yet. Capture the versions each run actually resolved, yours, last month's and your teammate's, and compare them. A tool or library that moved while the code stood still is the usual reason the same code gives a different number, so the drift is what you look for first.
- Follow-ups
-
- Which versions would you compare first, and how would you capture them?
- Once you find the drifting package, how do you make the two numbers agree again?
- How do you stop this from happening to the next report?
- The teammate still cannot reproduce it after pinning. What now?
From the lesson A local data lab you can trust
Model answer, Pro.In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 4 minutes
-
Mid to senior
You run DuckDB, Spark, Kafka, Flink and Airflow on a laptop you take to shared workspaces and cafes. What makes this lab unsafe by default, and how do you make it safe without a cloud account or a paid tool?
- What it tests
- Knowing that these engines are not secure by default, that some bind every interface, and that the fix is to bind loopback, verify from the sockets, and keep credentials out of files, all locally and at no cost.
- First step
- Name the exposure: these engines ship with authentication off and some listen on every interface, so on an untrusted network anyone who can reach the laptop can drive them. The goal is that nothing listens beyond loopback and no secret sits in a file, checked from the live sockets rather than assumed.
- Follow-ups
-
- Which services listen beyond loopback out of the box, and what do you change for each?
- How do you confirm the lab is actually closed, not just configured to be?
- Where do credentials go if not in a config file?
- A teammate needs to reach your Airflow UI for an hour. What is the safe way?
From the lesson A local data lab you can trust
Model answer, Pro.In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 4 minutes
-
Fresher to senior
A product team wants to keep every tap and screen view of its mobile app for two years, to study how features are used. The app has about 3 million daily users. Estimate the storage and the peak event rate, say which inputs you would measure first, and how you would check the estimate.
- What it tests
- Turning a vague volume into an estimate with stated inputs, measuring bytes per event and the compression ratio instead of guessing them, sizing throughput for the busiest minute, and checking the estimate against real days.
- First step
- Write the formula before any number: events a day times bytes per event, divided by the compression ratio, times the days kept, for every copy. Then mark which inputs are guesses (events per user, bytes per event, the ratio, the peak factor) and say how one day of real traffic would measure each of them.
- Follow-ups
-
- The analysts also want a cleaned copy beside the raw events. What changes in the estimate?
- How would you size the collector for the evening of a product launch?
- A year later the store is 40 % above the estimate. Where do you look first?
From the lesson Pipeline requirements: SLAs, freshness, sizing
Model answer, Pro.In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 4 minutes
-
Mid to staff
A finance lead asks that the pipeline loading card and wallet settlements into the warehouse should never lose a single row and never go down. How do you turn that into recovery objectives, what would each choice cost, and how would you show that the objectives are met?
- What it tests
- Translating an absolute demand into an RPO and an RTO tied to business deadlines, knowing what zero would cost, using kept input and idempotent replay, and showing recovery with drills instead of documents.
- First step
- Ask what a lost row and an hour without the data would each cost, and by when finance needs the data each day. Then split the demand: never losing a row is a recovery point objective that kept input and replay can meet; never going down is a recovery time objective, and no system offers zero, so agree a time tied to that deadline.
- Follow-ups
-
- The settlement provider deletes its files after 3 days. Does that change your answer?
- What would an RPO of zero cost for the warehouse tables themselves?
- How do you show finance that the RTO is real before an incident happens?
From the lesson Pipeline requirements: SLAs, freshness, sizing
Model answer, Pro.In the answer:
- Model answer
- Follow-up answers
- Common mistakes
- Rubric
- Variants
- about 4 minutes
All Data Engineering interview questions · The Data Engineering track