Free Practice Questions for Databricks Databricks-Certified-Data-Engineer-Associate Exam

A data lakehouse is a data management architecture that combines the flexibility, cost-efficiency, and scale of data lakes with the data management and ACID transactions of data warehouses, enabling business intelligence (BI) and machine learning (ML) on all data12. One of the key features of a data lakehouse is that it supports ACID-compliant transactions, which means that it ensures data integrity, consistency, and isolation across concurrent read and write operations3. This feature results in improved data quality over a traditional data lake, which does not support transactions and may suffer from data corruption, duplication, or inconsistency due to concurrent or streaming data ingestion and processing . References: 1: What is a Data Lakehouse? - Databricks 2: What is a Data Lakehouse? Definition, features & benefits. - Qlik 3: ACID Transactions - Databricks : [Data Lake vs Data Warehouse: Key Differences] : [Data Lakehouse: The Future of Data Engineering]

 In Databricks, you can use different languages within the same notebook by using magic commands. Magic commands are special commands that start with a percentage sign (%) and allow you to change the behavior of the cell. To use SQL within a cell of a Python notebook, you can add %sql to the first line of the cell. This will tell Databricks to interpret the rest of the cell as SQL code and execute it against the default database. You can also specify a different database by using the USE statement. The result of the SQL query will be displayed as a table or a chart, depending on the output mode. You can also assign the result to a Python variable by using the -o option. For example, %sql -o df SELECT * FROM my_table will run the SQL query and store the result as a pandas DataFrame in the Python variable df. Option A is incorrect, as it is possible to use SQL in a Python notebook using magic commands. Option B is incorrect, as attaching the cell to a SQL endpoint is not necessary and will not change the language of the cell. Option C is incorrect, as simply writing SQL syntax in the cell will result in a syntax error, as the cell will still be interpreted as Python code. Option E is incorrect, as changing the default language of the notebook to SQL will affect all the cells, not just one. References: Use SQL in Notebooks - Knowledge Base - Noteable, [SQL magic commands - Databricks], [Databricks SQL Guide - Databricks]

The engineer should inspect the job’s maximum concurrent-runs configuration and its queueing setting. max_concurrent_runs controls how many active runs of the same job can execute simultaneously. When that limit has already been reached, Databricks can skip the newly triggered run if queueing is disabled. When queueing is enabled, the additional run waits until capacity becomes available instead of being skipped. Spot-instance availability might delay or prevent compute startup, but it would not normally produce a job-level concurrency message. SQL warehouse permissions affect SQL task execution after a run begins, while timeout_seconds determines how long a task may run before timing out. Therefore, option C directly addresses the reported run state. This candidate was also compared against the existing Word question bank.

================

Option A uses the supported ALTER TABLE ... SET MANAGED conversion for a Unity Catalog external Delta table. Databricks documents that this conversion retains the table name, permissions, dependent objects, settings, and table history while minimizing downtime for readers and writers. After conversion, Databricks manages the table’s storage lifecycle, satisfying the requirement for platform-controlled cleanup when the managed table is dropped. Dropping the original table with PURGE destroys the object instead of migrating it. SET LOCATION changes the storage path referenced by an external table but does not convert the table into a managed table. Creating a replacement table with CREATE TABLE AS SELECT introduces a new object and requires manual migration of permissions, dependencies, and workloads. Therefore, ALTER TABLE catalog.schema.sales SET MANAGED is the correct action.

================

A SQL Serverless Warehouse is the most suitable option for running SQL-based exploratory data analysis at scale while minimizing operational and cost overhead. Serverless SQL warehouses are fully managed by Databricks , meaning Databricks automatically provisions and scales compute resources, applies Photon acceleration , and optimizes query execution without requiring the user to manage infrastructure. This makes them ideal for workloads involving large numbers of ad hoc or interactive SQL queries , which is common during EDA. Because the data resides in a custom-defined cloud network , serverless SQL warehouses can still securely access governed data when Unity Catalog and networking requirements are satisfied. All-purpose clusters (option A) are designed for collaborative development and mixed workloads, not cost-efficient high-concurrency SQL querying. Classic SQL warehouses (option D) are legacy and lack the performance and cost optimizations of newer serverless offerings. Pro SQL warehouses (option B) require user-managed sizing decisions, whereas serverless warehouses optimize both performance and cost automatically. Databricks documentation recommends SQL Serverless Warehouses for fast, scalable, and cost-effective SQL analytics.

=========

Option D satisfies the requirements because a Lakeflow Connect managed SaaS connector provides source-specific authentication, incremental ingestion, schema evolution, and automated retries as managed capabilities. Managed connectors use serverless infrastructure and publish governed destination tables that can be processed downstream with Lakeflow Spark Declarative Pipelines. This avoids custom code for OAuth token handling, API pagination, CDC state management, retry behavior, and schema-change recovery. Option A still requires the engineering team to implement and maintain CDC and operational logic. Options B and C place most authentication, schema evolution, change tracking, and failure recovery responsibilities in custom notebooks or jobs, conflicting with the minimal-maintenance requirement. Therefore, the fully managed connector described in option D is the only solution that delivers the requested ingestion and operational behavior as an integrated platform capability.

================

The expected behavior when a batch of data containing data that violates the expectation is processed is that the job will fail. This is because the expectation clause has the ON VIOLATION FAIL UPDATE option, which means that if any record in the batch does not meet the expectation, the entire batch will be rejected and the job will fail. This option is useful for enforcing strict data quality rules and preventing invalid data from entering the target dataset.

Option A is not correct, as the ON VIOLATION FAIL UPDATE option does not drop the records that violate the expectation, but fails the entire batch. To drop the records that violate the expectation and record them as invalid in the event log, the ON VIOLATION DROP RECORD option should be used.

Option C is not correct, as the ON VIOLATION FAIL UPDATE option does not drop the records that violate the expectation, but fails the entire batch. To drop the records that violate the expectation and load them into a quarantine table, the ON VIOLATION QUARANTINE RECORD option should be used.

Option D is not correct, as the ON VIOLATION FAIL UPDATE option does not add the records that violate the expectation, but fails the entire batch. To add the records that violate the expectation and record them as invalid in the event log, the ON VIOLATION LOG RECORD option should be used.

Option E is not correct, as the ON VIOLATION FAIL UPDATE option does not add the records that violate the expectation, but fails the entire batch. To add the records that violate the expectation and flag them as invalid in a field added to the target dataset, the ON VIOLATION FLAG RECORD option should be used.

Delta Live Tables Expectations

[Databricks Data Engineer Professional Exam Guide]

One of the features of DLT is that it provides data quality metrics for each dataset in the pipeline, such as the number of records that pass or fail expectations, the number of records that are dropped, and the number of records that are written to the target. These metrics can be accessed from the DLT pipeline page, where the data engineer can click on each table and view the data quality statistics for the latest update or any previous update. This way, they can identify which table is dropping the records and why. References:

Monitor Delta Live Tables pipelines

Manage data quality with Delta Live Tables

According to the medallion lakehouse architecture, gold tables are the final layer of data that powers analytics, machine learning, and production applications. They are often highly refined and aggregated, containing data that has been transformed into knowledge, rather than just information. Silver tables, on the other hand, are the intermediate layer of data that represents a validated, enriched version of the raw data from the bronze layer. They provide an enterprise view of all its key business entities, concepts and transactions, but they may not have all the aggregations and calculations that are required for specific use cases. Therefore, gold tables are more likely to contain aggregations than silver tables. References:

What is the medallion lakehouse architecture?

What is a Medallion Architecture?