Free Practice Questions for Databricks Databricks-Certified-Data-Engineer-Associate Exam

Delta Sharing is an open protocol developed by Databricks that enables secure, real-time sharing of data across organizations without requiring recipients to have a Databricks account. It is specifically designed for scenarios where external partners need read-only access to data stored in Delta format. With Delta Sharing, data providers create a share and grant access to recipients via a secure mechanism (such as a profile file or URL-based access), ensuring that the data cannot be modified by the consumer. This approach eliminates the need for data duplication or manual file transfers, maintaining data freshness and governance. Option B is incorrect because Unity Catalog governs internal access within an organization, not external sharing without accounts. Option C is inefficient and breaks data lineage and freshness. Option D violates security best practices by granting excessive permissions. Delta Sharing ensures fine- grained access control, auditability, and secure external collaboration , making it the correct solution for this requirement.

=========

Structured Streaming uses checkpointing and write-ahead logs to record the offset range of the data being processed in each trigger. This ensures that the engine can reliably track the exact progress of the processing and handle any kind of failure by restarting and/or reprocessing. Checkpointing is the mechanism of saving the state of a streaming query to fault-tolerant storage (such as HDFS) so that it can be recovered after a failure. Write-ahead logs are files that record the offset range of the data being processed in each trigger and are written to the checkpoint location before the processing starts. These logs are used to recover the query state and resume processing from the last processed offset range in case of a failure. References: Structured Streaming Programming Guide, Fault Tolerance Semantics

Databricks recommends managed tables when you want the platform to control both the table metadata and the underlying storage lifecycle. For Unity Catalog managed tables, Databricks manages storage layout and applies platform-managed optimization features over time. Databricks also documents that when you drop a managed table , the underlying cloud data is deleted automatically after the managed retention period, which simplifies cleanup and reduces storage administration. That behavior matches the requirement exactly. External tables, by contrast, are used when you want to retain direct control over the storage path and lifecycle. Databricks explicitly states that dropping an external table removes only the metadata and does not delete the underlying files . Foreign tables are for federation scenarios against external systems, not for Bronze ingestion storage managed by Databricks. Temporary views also do not provide the desired governed lifecycle. Because the team wants Databricks-managed storage, managed retention, and simplified lifecycle handling, managed tables are the correct choice for these Bronze ingestion tables.

=========

 Option E is the correct answer because it is the only mandatory requirement when creating a new Delta Live Tables pipeline. A pipeline is a data processing workflow that contains materialized views and streaming tables declared in Python or SQL source files. Delta Live Tables infers the dependencies between these tables and ensures updates occur in the correct order. To create a pipeline, you need to specify at least one notebook library to be executed, which contains the Delta Live Tables syntax. You can also specify multiple libraries of different languages within your pipeline. The other options are optional or not applicable for creating a pipeline. Option A is not required, but you can optionally provide a key-value pair configuration to customize the pipeline settings, such as the storage location, the target schema, the notifications, and the pipeline mode. Option B is not applicable, as the DBU/hour cost is determined by the cluster configuration, not the pipeline creation. Option C is not required, but you can optionally specify a storage location for the output data from the pipeline. If you leave it empty, the system uses a default location. Option D is not required, but you can optionally specify a location of a target database for the written data, either in the Hive metastore or the Unity Catalog.

 Tutorial: Run your first Delta Live Tables pipeline, What is Delta Live Tables?, Create a pipeline, Pipeline configuration.

A SQL Serverless Warehouse is the most suitable option for running SQL-based exploratory data analysis at scale while minimizing operational and cost overhead. Serverless SQL warehouses are fully managed by Databricks , meaning Databricks automatically provisions and scales compute resources, applies Photon acceleration , and optimizes query execution without requiring the user to manage infrastructure. This makes them ideal for workloads involving large numbers of ad hoc or interactive SQL queries , which is common during EDA. Because the data resides in a custom-defined cloud network , serverless SQL warehouses can still securely access governed data when Unity Catalog and networking requirements are satisfied. All-purpose clusters (option A) are designed for collaborative development and mixed workloads, not cost-efficient high-concurrency SQL querying. Classic SQL warehouses (option D) are legacy and lack the performance and cost optimizations of newer serverless offerings. Pro SQL warehouses (option B) require user-managed sizing decisions, whereas serverless warehouses optimize both performance and cost automatically. Databricks documentation recommends SQL Serverless Warehouses for fast, scalable, and cost-effective SQL analytics.

=========

The data engineering team can use the spark.table method to access the Delta table sales in PySpark. This method returns a DataFrame representation of the Delta table, which can be used for further processing or testing. The spark.table method works for any table that is registered in the Hive metastore or the Spark catalog, regardless of the file format1. Alternatively, the data engineering team can also use the DeltaTable.forPath method to load the Delta table from its path2. References: 1: SparkSession | PySpark 3.2.0 documentation 2: Welcome to Delta Lake’s Python documentation page — delta-spark 2.4.0 documentation

In a Databricks customer ' s cloud account, the primary elements stored include:

Data: This is the central type of content stored in the customer ' s cloud account. Data might include various datasets, tables, and files that are used and managed through Databricks platforms.

Notebooks: These are also stored within a customer ' s cloud account. Notebooks include scripts, notes, and other information necessary for data analysis and processing tasks.

Cluster management metadata is indeed managed through the cloud, but it ' s primarily handled by Databricks rather than stored directly in the customer ' s account. The Databricks web application itself is not stored within the customer ' s cloud account; rather, it ' s a service provided by Databricks.

The SQL keyword that can be used to convert a table from a long format to a wide format is PIVOT. The PIVOT clause is used to rotate the rows of a table into columns of a new table1. The PIVOT clause can aggregate the values of a column based on the distinct values of another column, and use those values as the column names of the new table1. The PIVOT clause can be useful for transforming data from a long format, where each row represents an observation with multiple attributes, to a wide format, where each row represents an observation with a single attribute and multiple values2. For example, the PIVOT clause can be used to convert a table that contains the sales of different products by different regions into a table that contains the sales of each product by each region as separate columns1.

The other options are not suitable for converting a table from a long format to a wide format. CONVERT is a function that can be used to change the data type of an expression3. WHERE is a clause that can be used to filter the rows of a table based on a condition4. TRANSFORM is a keyword that can be used to apply a user-defined function to a group of rows in a table5. SUM is a function that can be used to calculate the total of a numeric column.

1: PIVOT | Databricks on AWS

2: Reshaping Data - Long vs Wide Format | Databricks on AWS

3: CONVERT | Databricks on AWS

4: WHERE | Databricks on AWS

5: TRANSFORM | Databricks on AWS

[SUM | Databricks on AWS]