Data Engineering Interview Guide: Present a GCS to BigQuery Load Runbook
Explain a batch ingestion project using contracts, checks and a recovery walkthrough.
What should a data-engineering project interview cover?
Show that you understand what happens before, during and after ingestion. A useful portfolio includes a data contract, sample files, a loading plan, validation queries and a recovery runbook. Describe the system you actually built; a diagram of a proposed architecture is not proof that you operated it in production.
Start with an explicit contract
For an invented sales exercise, define columns order_id, order_date and amount. State the CSV delimiter, header convention, date format and whether a file is a full replacement or an increment. Explain how you would handle an empty file, a malformed record and an unexpected column. Use synthetic identifiers rather than real customer details in your public repository.
Walk through a load command
The following is a reviewed BigQuery command template, not a cloud execution performed for this article. Replace all placeholders and check project, bucket location, access and destination before running it. A real load can incur costs and requires suitable permissions.
bq --project_id=YOUR_PROJECT load \
--source_format=CSV --skip_leading_rows=1 \
YOUR_PROJECT:practice.orders_stage \
gs://YOUR_BUCKET/practice/orders.csv \
order_id:INTEGER,order_date:DATE,amount:NUMERICFor a fixture containing order 1 for 100 and order 2 for 50, your validation target is two rows and total amount 150. This is an expected fixture result, not a claim that this placeholder command ran successfully. Show the query you would use to confirm both values and where you would record the job outcome.
Rehearse a failed-run recovery
Ask yourself what to do if a load fails halfway through your wider workflow or a scheduler retries a task. Identify the file or batch being processed and inspect the actual job state before starting another attempt. Explain how staging and an explicit promotion step would help you validate data before downstream use. Describe your chosen write behavior and its consequences instead of assuming every command replaces the destination.
What to include in the portfolio
Include sample input, contract, command template, validation SQL and a failure scenario. Add a runbook explaining how an operator finds the affected batch, checks the result and decides whether recovery is safe. If you use an orchestrator, distinguish dependency ordering from the load's own guarantees. Keep secrets out of sample files and never embed service-account private keys in code.
Practice and common mistakes
Add a second file containing the same two orders. Explain how your design prevents accidental double counting and how that claim would be tested. Do not simply say exactly once without describing identity and retry behavior. Rehearse a schema-change discussion: who approves the new field, what rejects invalid input and how consumers learn about the change? These are interview preparation prompts, not promises about hiring outcomes.
BigQuery CSV loading reference.
Prepared by AI Leaf Global Learn for classroom learners in Mudichur/Tambaram, Chennai, and online learners across India. Confirm current batch details with the training team. Tamil Nadu, Chennai and Tambaram are audience tags, not additional branch claims.