Data Lake Interview Questions: 44 Architecture & Scenario Answers

A data lake interview is rarely just a test of whether you can define S3, Parquet, or schema-on-read. The harder questions ask whether you can keep data discoverable, safe, queryable, and recoverable as sources change and workloads grow.

This guide gives you 44 data lake interview questions and answers, from core definitions to senior-level scenarios. It focuses on the distinctions interviewers often probe: storage versus table format, governance versus transactions, raw retention versus curated data, and low storage cost versus total operating cost.

How to answer data lake interview questions

Use this five-part structure when a question asks you to design or troubleshoot a lake:

1. Clarify the workload. Ask about sources, volume pattern, freshness, query filters, consumers, retention, and security needs.
2. Name the layer. Is the question about object storage, ingestion, file format, table format, catalog, processing, query, or access control?
3. Make a scoped choice. Recommend a pattern or tool only after stating the relevant assumptions.
4. Explain the trade-off. Mention what your choice improves and what it costs in complexity, latency, compute, or maintenance.
5. Close with operations. Say how you will observe failures, replay data, manage schema change, enforce access, and recover.

What real deployments can, and cannot tell you

AWS has published several customer examples for data lakes built on Amazon S3. These are useful for seeing how workload requirements shape an architecture, but they are vendor-published examples, not independent benchmarks.

AWS reports that Siemens Cyber Defense Center built a lake capable of collecting 6 TB of log data per day and used its analytics platform to process 60,000 potentially critical events per second. The reported setup combines S3 with Glue, Lambda, and machine-learning services.

The interview lesson is to connect the storage design to the operational outcome and team constraints; the public write-up does not provide a controlled comparison against another architecture.

In another AWS-published example, Sysco combined S3 and S3 Glacier and reported a 40% reduction in storage costs. Yulu used an S3-based lake and a prediction model; AWS reports a 30–35% improvement in service efficiency after the company spent its first six months collecting usage data. These vendor-reported outcomes are not guarantees for another workload. The Sysco page identifies S3 and S3 Glacier, but does not establish that a particular lifecycle policy caused the reported savings; the Yulu account likewise does not isolate one technical change as the sole cause. See AWS’s customer examples.

Data lake interview questions: fundamentals and architecture

1. What is a data lake?

A data lake is a data platform architecture that stores data from multiple sources in a shared repository and makes it available for processing, analytics, and other uses. It commonly preserves some data close to its source form, while separate processing and governance layers make it usable and safe.

It is not just a storage bucket. A production lake also needs metadata, ownership, access controls, quality checks, processing, and a way for people or systems to find and query data. Microsoft’s data lake architecture guidance highlights governance, discoverability, integrity, and quality as challenges that mature designs must address.

2. How does a data lake differ from a data warehouse and a lakehouse?

A traditional warehouse is usually organized for structured, repeatable analytical queries. A lake commonly holds a broader mix of raw and curated data. 

A lakehouse adds table-management and analytical capabilities over data stored in lake-oriented storage. Those are common patterns, not hard product boundaries: platforms increasingly combine features.

A good answer avoids saying that lakes only contain unstructured data or that warehouses can only contain structured data. State the workload and implementation you are comparing. For a deeper comparison of architecture and modeling trade-offs, see data warehouse interview guide.

3. What do schema-on-read and schema-on-write mean?

Schema-on-read means that some interpretation or structure is applied when data is read or processed. Schema-on-write means data is checked or transformed against a structure as it is written into a target system. These describe where some schema decisions happen; they do not mean that a lake has no schema or that a warehouse never changes schema.

Teams often mix the patterns. They may land source data with minimal changes, then validate and publish curated tables under explicit contracts. This keeps source fidelity without asking every analyst to interpret inconsistent files independently. Microsoft’s data lake guidance describes schema-on-read, while Azure Databricks lakehouse guidance documents schema-on-write in refined Delta tables.

4. What are the main components of a production data lake?

A strong answer names the layers and their jobs:

• Storage: keeps source and curated data.
• Ingestion: moves batch, event, database, and file data into the platform.
• Processing: validates, transforms, and publishes datasets.
• Metadata and catalog: records schemas, locations, owners, and descriptions.
• Table management: tracks snapshots, schema changes, and file membership where a table format is used.
• Query and consumption: serves analysts, applications, models, or downstream systems.
• Governance and operations: handles identity, policy, quality, lineage, monitoring, retention, and recovery.

A cloud product can combine several roles, but the roles remain useful when debugging. A catalog permission issue is not the same as a storage permission issue, and neither is the same as a failed table commit.

5. Is a data lake the same thing as object storage?

No. Object storage is a common way to store lake data, but a data lake also includes the conventions and services that make data discoverable, governed, processed, and useful. A folder layout on its own does not supply data contracts, lineage, quality checks, table transactions, or access policy.

A concise interview answer might say: “Object storage is the substrate; the lake is the data architecture and operating model built around it.” Then add the relevant caveat for the cloud or platform being discussed.

6. What changes when an organization adds lakehouse capabilities?

It usually adds managed table metadata and operations over files in lake storage. Depending on the format and engine, these may include snapshots, schema handling, updates or deletes, and concurrent reads or writes. The exact behavior is not guaranteed by the word “lakehouse.”

For example, Delta Lake describes ACID transactions on Spark in its official documentation, while Apache Iceberg describes table state through metadata files and snapshots in its specification. Support still depends on the engine, catalog, locking or commit implementation, and version. Ask which operations the actual stack supports before promising cross-engine behavior.

7. What is a data swamp?

A data swamp is a lake where data exists but users cannot reliably discover, interpret, trust, or access it. Common causes include missing ownership, unclear schemas, poor documentation, broken ingestion, duplicate versions, inconsistent naming, and weak quality controls.

The fix is not simply “add a catalog.” A catalog helps only if metadata is maintained, definitions are understandable, access works, and owners respond when data changes. Translating ‘quality’ into measurable dimensions — completeness, uniqueness, validity, accuracy, consistency, and timeliness, is what turns a vague control into one a team can actually enforce.

8. Are raw, curated, and serving layers, or Bronze, Silver, and Gold, required?

No. They are useful design patterns, not a universal standard. A raw layer can preserve source fidelity; a cleansed layer can apply validation and standardization; a curated layer can serve business definitions or downstream use. Microsoft describes a Bronze/Silver/Gold pattern in its Azure architecture guidance, and Databricks explicitly says its medallion architecture is recommended, not required, in its medallion guidance. The labels do not guarantee quality by themselves.

Name layers according to their contract and purpose. If a team cannot explain who owns a layer and what quality guarantee it offers, renaming it “Silver” will not fix the design.

Ingestion, schema change, and data contracts

9. How would you design an end-to-end data lake for relational, event, log, and file sources?

Start with requirements: freshness, source volume, change semantics, query patterns, sensitive fields, recovery needs, and consumers. Then choose ingestion paths. A database may need incremental extraction or CDC; events may need a streaming path; files may need batch validation and quarantine.

Land data with source identity and ingestion metadata. Preserve enough information to replay or explain changes. Catalog the dataset, apply quality checks, and publish curated tables for the intended engine. Finally, define ownership, access, monitoring, retention, and recovery. Do not choose the table format before confirming the target engines and catalog can support the required operations.

10. When would you choose batch ingestion versus streaming?

Choose batch when the business can tolerate scheduled updates, the source arrives in files or periodic extracts, or simplicity matters more than low latency. Choose streaming when consumers need events continuously and the system can support the operational demands of continuous processing.

Many platforms use both: streaming for fresh changes and batch for reconciliation, historical loads, or backfills. Compare latency requirements, late data, ordering, retries, cost, and the team’s ability to monitor and recover the pipeline. “Real time” is not a requirement until the required delay is defined.

11. How should change data capture (CDC) be used with a data lake?

CDC captures changes from a source database’s change stream or log and forwards them downstream. The lake design must preserve enough ordering and source metadata to apply inserts, updates, and deletes correctly. A consumer then has to decide whether to keep an append-only change history, materialize current state, or support both.

Do not equate CDC with exactly-once processing. Delivery semantics, checkpointing, deduplication, and sink behavior depend on the connector and platform. Replay also depends on retaining the source log or offset long enough; after it expires, recovery may require a fresh snapshot or reconciliation. A stable key alone does not make downstream writes idempotent, the sink must handle retries and operation ordering. For broader pipeline interview practice, see ETL interview questions and production CDC guide.

12. How do you handle late-arriving or out-of-order events?

Keep event time and ingestion time separate. Define how long the system will accept late events, whether a late event updates a prior result or creates a correction, and how downstream consumers learn about that correction. The timestamps help explain arrival and business chronology; reconstructing a prior state also requires retained snapshots or versioned history.

For replayable systems, retain a stable event identifier or source offset where available. Test duplicates and reordering explicitly. Avoid using a folder path as the only proof that a record is unique or current.

13. What is the difference between event time and ingestion time?

Event time is when the source says an event occurred. Ingestion time is when the event reaches a defined pipeline stage, such as landing or durable-table commit; name the stage consistently. The timestamps differ when a source is offline, data is delayed, or a historical backfill is loaded.

Preserving both helps measure pipeline delay and separate business chronology from arrival chronology. Reconstructing what was known at a past time also requires retained immutable or versioned records, snapshots, and a reliable ordering or commit marker; the two timestamps alone are not enough. Choose the timestamp used for partitioning or reporting based on query and correctness requirements, not convenience alone.

14. How do you handle source schema drift?

Detect the change before it silently damages downstream output. Compare incoming fields with the expected contract; classify additions, removals, type changes, and semantic changes; then decide whether to accept, quarantine, or reject the batch. Notify the dataset owner and assess affected consumers.

A practitioner described a manufacturing pipeline repeatedly breaking as SAP extracts changed columns across years and manual data entry varied. That is an individual field report, not a universal pattern, but it shows why schema tools cannot replace upstream ownership and data-quality checks. The relevant schema-drift discussion can be used as a scenario prompt, not as evidence that a particular format fixes the problem.

15. How do you evolve a table schema without breaking consumers?

Treat source changes, table schema changes, and consumer contracts as separate steps. Assess whether a field is optional, renamed, removed, or changing type. Add compatibility tests, update documentation, and coordinate rollout so readers do not assume a new meaning for an old field.

Formats such as Iceberg and Delta have schema-management features, but support and behavior depend on the writer, reader, catalog, and version. A safe answer describes the compatibility test and rollback plan, not only the format name. Vertex Frontier has a detailed Apache Iceberg architecture guide for the deeper format-specific discussion.

16. How should retries, idempotency, replay, backfills, and deduplication work together?

Retries should not create unintended duplicate business records. Use a stable event/change identifier or source position/version, not only an entity key, along with deterministic transformations where possible and a commit/checkpoint strategy that distinguishes completed work from work to replay. Preserve operation type so inserts, updates, deletes, and corrections can be applied in a defined order.

For backfills, specify the affected range, how results reconcile with existing data and ongoing ingestion, and how overlapping writes are resolved. Test rerunning the same batch, receiving an older event late, and receiving a correction for an existing key. A single practitioner discussion about delayed files and duplicates illustrates the scenario; it is anecdotal, not representative evidence.

File formats, table formats, catalogs, and snapshots

17. Which file formats are common in a data lake?

JSON, CSV, and Avro are common for source exchange or event data; Parquet and ORC are common columnar formats for analytical reads. The right choice depends on source compatibility, schema needs, query engines, compression, and downstream use. This is not a ranking of “best” formats.

Avro is a schema-based serialization format; Parquet is columnar and commonly used for analytical scans. Neither is categorically better, the choice follows the exchange, schema, and query requirements.

When you mention Parquet, distinguish it from the table format or catalog that manages a table. The Apache Parquet documentation describes Parquet as a column-oriented file format; it does not by itself provide a lake-wide catalog or transaction protocol.

18. Why use Parquet instead of querying raw JSON or CSV for analytics?

Parquet stores values by column, so a query engine can often read only the columns it needs and use file metadata to skip irrelevant data. Compression can also reduce the bytes read. AWS explains these mechanisms in its Athena data-optimization guidance.

But conversion is not automatically worthwhile for every feed. Tiny datasets, infrequent queries, complex nested data, or extra pipeline maintenance may change the choice. A useful design keeps raw input when it supports replay, then creates a query-oriented representation when measurement shows a need.

19. What is the difference between a file format and a table format?

A file format defines how records are encoded in files. A table format tracks which files make up a table and can add metadata for snapshots, schema changes, deletes, or commits. A catalog helps readers and writers locate the table and its metadata. A query engine reads or changes the data.

The distinction matters in interviews because a format may support a feature that a particular engine or catalog does not implement. For a focused comparison, read our guide Apache Iceberg vs. Parquet guide.

20. How would you choose among Apache Iceberg, Delta Lake, and Apache Hudi?

Start with your write pattern, engines, catalog, update/delete needs, concurrency, streaming model, and operating skills. Then verify the exact features and versions supported by each component. Avoid choosing from a feature checklist without validating the full stack.

These are starting points, not universal product rankings. Apache Hudi’s official overview and compaction guide explain its table services and Merge-on-Read trade-offs. Our guide Iceberg, Delta Lake, and Hudi comparison goes deeper into workload-based selection.

21. What does a metadata catalog do, and how is it different from a schema registry?

A catalog commonly records table names, locations, schemas, and other metadata so tools can find and interpret datasets. A schema registry typically manages schema definitions for data exchanged through particular event or serialization systems. Products can overlap, so explain the function rather than relying on the product label.

A catalog does not automatically provide data quality, clear ownership, business definitions, or correct permissions. Those need their own process and enforcement.

22. How do table formats provide transactional or snapshot semantics over object storage?

A table format tracks table state in metadata and publishes a new state through a commit mechanism. For example, the Iceberg specification describes metadata files, snapshots, manifests, and optimistic concurrency. Delta Lake documents ACID transactions on Spark in its project documentation.

These semantics belong to the format and supported implementation, not to raw object storage alone. Be precise about the engine, catalog, locking or commit protocol, and operation. “The files are on S3, so they are transactional” is not a complete answer.

23. What are snapshots and time travel?

A snapshot represents a table state at a point in time. A supported engine can read an earlier state while its metadata and referenced data files remain available. This history can support comparison, rollback, and reproducible analysis, but it is not a compliance/security audit trail or a backup; retain independent logs and backups for those needs.

They also create a retention decision. Expiring snapshots can remove the ability to query those historical states. Apache Iceberg’s maintenance guide explains that expiration removes old snapshot history and can delete files no longer needed by retained snapshots. Retention should match recovery and compliance requirements.

24. How should you choose partition columns?

Choose partitions from common query filters and data volume, not just from the source folder structure. A date partition can help if users regularly filter by date. Adding highly granular or high-cardinality keys can create many small partitions and files.

AWS Athena’s partitioning guidance recommends designing around common queries and warns that too many keys can fragment datasets. The exact trade-off depends on the engine, table format, storage layout, and read pattern.

25. What is partition evolution?

Partition evolution lets a table’s partition specification change as requirements change, without treating every old file as if it had been written under the new layout. In Iceberg, old files retain their earlier spec, new writes use the evolved spec, and query planning evaluates files under the applicable layout. Iceberg’s hidden partitioning can prune files without requiring queries to target a specific spec partition-evolution documentation.

The interview point is that a partition scheme should not be frozen forever. But changing metadata is not the same as rewriting all existing data into a new physical layout. Test how your engine handles old and new files together.

Query performance, maintenance, and operations

26. Why do many small files hurt some lake queries?

Each file can add listing, planning, metadata, open, and request overhead. Columnar formats also have internal structures that may not deliver their benefits on tiny files. AWS documents these effects for Athena, and Iceberg notes the metadata and file-open costs of many small files Athena optimization guide and Iceberg maintenance guide.

The answer is not “compact everything.” Compaction consumes compute and rewrites data. Measure file counts, query planning time, bytes scanned, and the workload’s update/read pattern first. Do not claim one ideal file size for every engine.

With Apache Iceberg configured as a Spark catalog, you can inspect the current file inventory through the table’s files metadata table. That inventory can contain both data files and delete files; content = 0 filters for data files in this example. This is a diagnostic query, not a universal small-file threshold; compare the observed distribution with your engine and workload. See the Iceberg Spark query documentation for metadata-table naming and content values.

Spark SQL · Apache Iceberg: inspect the smallest current files:

SELECT file_path,
       file_size_in_bytes,
       record_count
FROM prod.db.events.files
WHERE content = 0
ORDER BY file_size_in_bytes ASC
LIMIT 20;

Here, prod is the configured Iceberg catalog and db.events is the table. Replace them with your own identifiers. File size alone does not prove a performance problem; confirm the query plan and read pattern before rewriting data.

27. How are compaction, snapshot expiration, VACUUM, and orphan-file cleanup different?

They solve different problems, and the exact command or behavior depends on the table format and engine.

For example, Iceberg warns against using an orphan-file retention interval shorter than the longest expected write because an active file could be deleted. Delta’s utility documentation describes its own VACUUM behavior and retention. These operations are not interchangeable; never run cleanup from a generic interview answer without checking the platform and recovery requirements.

For an Iceberg table accessed through Spark, the documented rewrite_data_files procedure uses bin-pack by default. Configure an Iceberg Spark catalog first: Iceberg procedures require the Iceberg SQL extensions in Spark 3.x; Spark 4.0 supports procedures natively, but procedure names are case-sensitive. Confirm the version, catalog, permissions, and procedure support in the deployed stack. The following example scopes the rewrite to files that may contain the selected records; the predicate and identifiers are illustrative. See the official procedure reference before running it.

Spark SQL · Apache Iceberg: rewrite candidate data files:

CALL prod.system.rewrite_data_files(
  table => 'db.events',
  where => 'customer_id = 42'
);

This is a table-writing maintenance operation, not a harmless read query. It can consume significant compute and create a new table commit. Test the scope, expected rewrite volume, permissions, and concurrent-writer behavior first; do not run an unrestricted rewrite on a large production table simply because small files exist.

28. How do you detect over-partitioning?

Look for excessive partition counts, many partitions with very little data, a large number of small files, slow planning, or queries that still read too much data. Compare the partition scheme with actual filter patterns. A design can be under-partitioned too, forcing common queries to scan unnecessary data.

Before changing it, review query plans and table metadata. The right fix may be fewer partition keys, sorting or clustering, compaction, a new table layout, or a different serving system. Avoid treating partition count as a universal pass/fail number.

29. How would you investigate a query that scans more data than expected?

Use a repeatable diagnostic sequence:

1. Confirm the query filters match the table’s partition columns and expected types.
2. Inspect the query plan or engine’s explain output to see which partitions and files are selected.
3. Check file format, file count, row-group or stripe metadata, and whether the data is laid out for the filters.
4. Compare estimated bytes scanned with actual bytes and isolate whether the issue is planning, pruning, decoding, or a broad query.
5. Change one layout or query choice at a time, then measure again.

AWS’s Athena optimization documentation recommends using query and data layout together; do not assume that adding more partitions will solve every scan problem.

30. When can sorting, statistics, partition pruning, or columnar layout reduce read work?

They help when the engine can use them to avoid reading irrelevant data. Partition pruning skips partitions; columnar formats can avoid unused columns; min/max statistics and sorted data may help an engine skip blocks or row groups. The benefit depends on the query predicates, file organization, metadata, and engine support.

If the interview asks for an optimization, describe how you would confirm it in the plan and measure bytes read. Avoid promising a speedup without a workload and test.

31. How do concurrent readers and writers affect correctness?

They depend on the table format’s commit model and the engine/catalog implementation. Iceberg describes optimistic concurrency around metadata commits; a writer may need to retry if the base table state has changed Iceberg specification. Delta Lake documents concurrency behavior for its supported operations in its concurrency-control guide.

A strong answer identifies the writer engines, conflict handling, retry behavior, isolation needs, and what happens if two jobs update overlapping data. Do not assume every file-based table is safe for concurrent writes.

32. What should you test before letting multiple engines share a table?

Test the actual combination, not just the format’s marketing feature list. Verify catalog access, supported table versions, file formats, schema changes, partition changes, deletes, merges, time travel, concurrent writes, permissions, metadata-table access, and cleanup operations. Also test failure and retry behavior.

A feature that works in one engine may be unsupported or differently implemented in another. For example, AWS documents specific Iceberg-version, locking, and Lake Formation limitations for Athena in its Iceberg query guide. Re-check the matrix for the engines you intend to run.

Governance, quality, security, and cost

33. How do you validate data quality while keeping raw data available?

Keep an immutable or replayable source copy when policy and cost allow. Validate schema, required fields, keys, ranges, freshness, and cross-field rules before publishing trusted datasets. Send invalid records to a quarantine path or error table with enough context to diagnose them. Define which failures block publication versus alert, who owns quarantined records, and how they are corrected or replayed. Track quality results over time.

Raw retention is not a substitute for quality. Nor should every raw copy be kept forever: privacy, legal, cost, and recovery rules may limit retention.

34. How do catalogs, ownership, and lineage make a lake easier to use?

The catalog helps users find tables and understand basic technical metadata. Ownership identifies who can explain or change the data. Business definitions explain what fields mean. Lineage shows how the data was produced. Together, they let consumers assess whether a dataset fits a use case and whom to contact when it changes.

Lineage is only as complete as its enabled integrations and emitted events. State which sources, jobs, and consumers are covered, and instrument gaps rather than assuming that a catalog or crawler captures end-to-end lineage. AWS documents the scope and integration requirements for DataZone lineage.

A crawler can infer structure, but it cannot reliably supply every business definition or quality guarantee. Discovery works only when metadata has an owner and a maintenance path.

35. How should access control span storage, catalog, and query engines?

Map identity and permissions across each layer. A user may need permission to discover a table, query it, and access its underlying data, depending on the platform. Apply least privilege, protect sensitive columns or rows where supported, and audit both policy changes and data access.

Keep vendor claims scoped. AWS Lake Formation can govern access to registered Amazon S3 locations through integrated services, alongside IAM; it does not prevent direct S3 API or console access already permitted by IAM or S3 policies, so review those paths separately Lake Formation underlying-data access. 

Azure Data Lake Storage Gen2 supports RBAC and ACLs, but ACLs cannot restrict access already granted by a sufficient RBAC role. Shared Key, account SAS, and service SAS also follow separate authorization paths outside identity-based RBAC/ACL evaluation; user-delegation SAS has different ACL behavior Azure access-control model. These are provider-specific implementations, not features automatically supplied by every lake.

36. How do you handle sensitive data, deletion requests, and historical snapshots?

Start with classification and policy: identify sensitive fields, define who may read them, determine how long raw and curated data must remain, and understand whether historical snapshots retain copies. Then test deletion or redaction behavior across the table format, storage, backups, extracts, and downstream systems.

A logical delete from a current table may not immediately remove older retained files. Do not promise compliance from a single table operation. Confirm the organization’s legal and retention requirements with the appropriate owners, then implement and audit the approved process.

37. What should be audited in a data lake?

Audit access to sensitive datasets, permission changes, table writes, schema changes, failed jobs, administrative actions, and deletion or retention operations. Logs need enough context, principal, time, dataset, action, and result, to investigate an incident. Define retention and protection for the logs themselves.

The exact event coverage depends on the cloud provider, query engine, catalog, access path, and logging configuration. In AWS, CloudTrail records Lake Formation API calls, but those management events are not the same as a complete record of every table read. If object-level read auditing is required, configure and test the relevant S3 data events and review direct S3 access paths separately; AWS notes that S3 data events incur additional charges Lake Formation CloudTrail logging and underlying-data access.

38. Why does low-cost storage not guarantee a low-cost data lake?

Storage is only one part of total cost. Query scans, compute clusters, object requests, data transfer, duplicate copies, table maintenance, catalog operations, security controls, monitoring, and engineering time can all matter. A design that stores cheaply but requires expensive repeated transformations may not be cheap overall.

Ask what workload is being priced: ingestion, storage duration, query frequency, freshness, concurrency, and recovery. Do not quote a cost saving without a workload model and a stated comparison.

39. What changes when a small lake grows to support more users and sources?

Metadata quality, access policy, ownership, workload isolation, schema contracts, and operating routines become more important. Very high file or partition counts can increase planning, metadata, or file-open overhead for some workloads; figures such as “thousands of small files” are illustrative, not universal thresholds. The effect depends on the engine, catalog, table format, and query pattern.

Revisit design using actual metrics: file counts, partition counts, query patterns, failure rate, freshness, cost, and number of independent writers. Scale the controls as well as the storage.

40. When is a data lake unnecessary?

A lake may be unnecessary when the data is small, well-structured, serves a narrow set of SQL reports, and can be managed more simply in a database or warehouse. It may also be a poor first step if no team can own quality, security, cataloging, and operations.

That is a decision question, not a size threshold. Practitioner discussions raise simpler tools such as PostgreSQL, DuckDB, or a managed warehouse for small projects, but those are anecdotes rather than comparative benchmarks. Ask what requirements justify the extra platform. For adjacent comparison practice, use the site’s data warehouse interview questions.

Scenario-based data lake interview questions

41. One small JSON file arrives each day, and analysts query it with Trino. What would you do?

First ask about the file count, retention, query frequency, latency, schema stability, and expected growth. I would usually preserve the source files if replay matters, then evaluate a curated table or batched Parquet output if repeated queries make the raw layout costly. I would not rewrite or append into one mutable object every day without checking the storage and table design.

A single archived practitioner discussion debates this choice and suggests avoiding overengineering a simple feed; treat it as anecdotal, not a measured recommendation daily JSON and Trino discussion. Official Athena guidance confirms that many small files can add overhead in that engine, but it does not set a universal threshold for Trino or every workload.

42. A manufacturing source changes columns across years. How do you prevent repeated pipeline rewrites?

Separate the response into source contract, landing, schema handling, quality rules, and consumer compatibility. Capture the source version or extraction context. Detect additions, removals, and type changes; route breaking or ambiguous data for review; evolve the table only when the change is understood; and notify downstream owners.

A format’s schema-evolution feature can help represent structural changes. It cannot decide whether a changed field still means the same thing or repair inconsistent manual data entry. Assign an owner for those decisions. This scenario is informed by one archived practitioner report about a manufacturing/SAP pipeline; it is illustrative, not representative evidence schema-drift discussion.

43. A backfill creates duplicate or revised files. How do you find the authoritative record and replay safely?

Ask whether the source provides stable IDs, update timestamps, sequence numbers, or change logs. Define a precedence rule and preserve ingestion metadata. Make the transformation idempotent where possible, scope the backfill, and compare affected outputs before publishing. If the source cannot identify revisions, surface that as a source-contract limitation rather than silently guessing.

The test should include rerunning the same batch, receiving an older event late, and receiving a corrected event for an existing key. This scenario reflects one archived practitioner discussion, not a representative study backfill and file-layout discussion.

44. A streaming table is queried by several engines while writers continue to update it. What do you verify?

Confirm the table format, catalog, writer engines, reader engines, supported table version, commit/locking model, operations used, and permission paths. Test concurrent writes, schema change, retry after conflict, snapshot reads, deletes, compaction, and historical retention. Check that every engine interprets the same table metadata and that operations are supported in the chosen combination.

Do not answer only “use Iceberg” or “use Delta.” The format is one layer; compatibility and operational behavior belong to the full stack.

Common mistakes in data lake interview answers

• Calling a lake “unstructured storage.” Lakes can contain structured, semi-structured, and unstructured data.
• Treating schema-on-read as no schema. Validation and contracts still matter.
• Confusing Parquet with Iceberg, Delta Lake, or Hudi. One encodes files; the others manage table state and operations.
• Assuming all table formats support the same operations in every engine. Check the version and compatibility matrix.
• Partitioning by every available field. More partitions can fragment the data and create small files.
• Compacting automatically without measuring. Rewrites consume resources and may compete with ingestion.
• Treating VACUUM and orphan cleanup as harmless housekeeping. Retention affects recovery and historical reads; unsafe cleanup can delete needed files.
• Promising “exactly once,” compliance, or a performance gain without conditions. State the mechanism and the environment that provides it.
• Recommending a lake before clarifying the problem. A smaller database or warehouse may be easier to operate.

Before and after: improving an interview answer

Before: “I’d use S3, Parquet, and partitions by customer ID. Then Spark will make it fast.”

After: “I’d first ask how analysts filter the data and how many customers there are. If most queries filter by date, I would test a time-oriented layout rather than partitioning on every customer ID, which could create many small partitions. I’d keep a replayable raw copy if policy allows, publish a curated table using a format supported by our readers, and compare query plans and bytes scanned before changing the design.”

The second answer is stronger because it names assumptions, identifies the relevant layer, explains a risk, and proposes a measurement. It does not promise an outcome the candidate has not tested.

Final thought

The best interview answers do not start with a product name. They start with the requirement, identify the layer that owns the behavior, and explain how the design will be tested and operated. That is what separates a list of data lake terms from an engineering answer.

Frequently asked questions

About The Author

A Gadallh

Ahmed Gadallah is the Founder and Editor of Vertex Frontier, where he publishes research-driven articles on AI, data science, cloud computing, cybersecurity, software engineering, and emerging technologies, with a focus on technical accuracy, clarity, and practical insights.

View all articles by A Gadallh →