Free Practice Questions for Databricks Databricks-Certified-Data-Engineer-Associate Exam

The STREAM function is used to process data from a streaming live table or view, which is a table or view that contains data that has been added only since the last pipeline update. Streaming live tables and views are stateful, meaning that they retain the state of the previous pipeline run and only process new data based on the current query. This is useful for incremental processing of streaming or batch data sources. The customers table in the query is a streaming live table, which means that it contains the latest data from the source. The STREAM function enables the query to read the data from the customers table incrementally and create another streaming live table named loyal_customers, which contains the customer IDs of the customers with high loyalty level. References: Difference between LIVE TABLE and STREAMING LIVE TABLE, CREATE STREAMING TABLE, Load data using streaming tables in Databricks SQL.

JSON data is a text-based format that represents data as a collection of name-value pairs. By default, when Auto Loader infers the schema of JSON data, it treats all columns as strings. This is because JSON data can have varying data types for the same column across different files or records, and Auto Loader does not attempt to reconcile these differences. For example, a column named “age” may have integer values in some files, but string values in others. To avoid data loss or errors, Auto Loader infers the column as a string type. However, Auto Loader also provides an option to infer more precise column types based on the sample data. This option is called cloudFiles.inferColumnTypes and it can be set to true or false. When set to true, Auto Loader tries to infer the exact data types of the columns, such as integers, floats, booleans, or nested structures. When set to false, Auto Loader infers all columns as strings. The default value of this option is false. References: Configure schema inference and evolution in Auto Loader, Schema inference with auto loader (non-DLT and DLT), Using and Abusing Auto Loader’s Inferred Schema, Explicit path to data or a defined schema required for Auto loader.

DataFrame.dropna() removes rows containing NULL or NaN values. Its default how= " any " behavior removes a row when any column included in the specified subset is NULL. By passing [ " customer_email " , " customer_phone " ] as the subset, the operation removes a record when either required field is missing, while ignoring NULL values in unrelated columns. Option B can produce the same filtering result, but it uses two chained where() calls and therefore does not meet the single-method-call requirement. Option C uses how= " all " and would remove a row only when all evaluated columns are NULL. Option D contains one filter() operation but requires multiple column-function calls. Consequently, option A is the direct PySpark method designed for this requirement.

================

 Data Explorer is a graphical interface that allows you to browse, create, and manage data objects such as databases, tables, and views in your workspace. You can also review and modify the permissions on these data objects using Data Explorer. To access Data Explorer, you can click on the Data icon in the sidebar, or use the %sql magic command in a notebook. You can then select a database and a table, and click on the Permissions tab to view and edit the access control lists (ACLs) for the table. You can also use SQL commands such as SHOW GRANT and GRANT to query and modify the permissions on a Delta table. References:

Data Explorer

Access control for Delta tables

SHOW GRANT

[GRANT]

 In Databricks SQL, you can use scheduled query executions to update your dashboards or enable routine alerts. By default, your queries do not have a schedule. To set the schedule, you can use the dropdown pickers to specify the frequency, period, starting time, and time zone. You can also choose to end the schedule on a certain date by selecting the End date checkbox and picking a date from the calendar. This way, you can ensure that the query does not run beyond the first week of the project’s release and does not incur any additional cost. Option A is incorrect, as setting a limit to the number of DBUs does not stop the query from running. Option B is incorrect, as there is no option to end the schedule after a certain number of refreshes. Option C is incorrect, as there is a way to ensure the query does not cost the organization money beyond the first week of the project’s release. Option D is incorrect, as setting a limit to the number of individuals who can manage the query’s refresh schedule does not affect the query’s execution or cost. References: Schedule a query, Schedule a query - Azure Databricks - Databricks SQL

In Unity Catalog, an external table is defined by referencing data that already exists in cloud storage, and Databricks documentation states that this is done by using a LOCATION clause that points to the storage path. Databricks also notes that the external table’s path should be contained within an existing external location that has been configured with the appropriate storage credential and access permissions. This makes option A the correct choice. Option B is incorrect because simply calling something “unmanaged” without specifying a location does not create a proper Unity Catalog external table. Option C is also incorrect because Databricks documentation explicitly requires a storage location for Unity Catalog external tables. Option D is wrong because managed tables are intended for Databricks-managed storage and lifecycle handling, not for referencing existing external files. So, for ADLS-based data that must remain in place while still being queryable through Databricks, the right pattern is to configure the external location and then create the table with a CREATE TABLE ... LOCATION statement.

=========