Q1
Design a data pipeline that ingests data from multiple sources (APIs, databases, files) into a data warehouse. Walk us through your architecture choices, including how you'd handle schema evolution, data quality checks, and incremental vs. full loads.
Why they ask this:* This tests your ability to design scalable, production-ready pipelines and your understanding of real-world data engineering challenges like schema management and data quality.
Q2
Explain the differences between Apache Spark and Apache Flink. When would you choose one over the other, and what are the trade-offs in terms of latency, throughput, and state management?
Why they ask this:* This assesses your knowledge of distributed processing frameworks and your ability to evaluate tools based on use cases—a critical skill for mid-level engineers.
Q3
You have a 10TB table in a data warehouse that's being queried slowly. Walk us through your approach to diagnosing the performance issue and the optimization techniques you'd consider (indexing, partitioning, materialized views, etc.).
Why they ask this:* This evaluates your practical troubleshooting skills and understanding of database optimization—essential for maintaining performant data systems.
Q4
Describe how you would implement a slowly changing dimension (SCD) in your data warehouse. What are the different types, and which would you use for customer demographic data that changes infrequently?