DP-300 practice question 17 of 35

You have an Azure Data Lake Storage account that contains a staging zone. You need to design a daily process to ingest incremental data from the…

Domain Configure and manage automation

Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution. After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear in the review screen. You have an Azure Data Lake Storage account that contains a staging zone. You need to design a daily process to ingest incremental data from the staging zone, transform the data by executing an R script, and then insert the transformed data into a data warehouse in Azure Synapse Analytics. Solution: You schedule an Azure Databricks job that executes an R notebook, and then inserts the data into the data warehouse. Does this meet the goal?

Answer options

A: Yes Correct answer

B: No

Correct Answer:

A. Yes

Explanation

The proposed solution meets all the stated goals. Azure Databricks is an Apache Spark-based analytics platform designed for large-scale data engineering and collaborative data science. It natively supports R notebooks for data transformation. It can efficiently read data from Azure Data Lake Storage and write data to Azure Synapse Analytics using the optimized Azure Synapse connector. Furthermore, Azure Databricks has a built-in job scheduler that can be configured to run the R notebook on a daily basis, fulfilling the scheduling requirement. This makes it a comprehensive and appropriate solution for the described ETL (Extract, Transform, Load) process.

Why Incorrect

B. No: This option is incorrect. The solution is entirely feasible as Azure Databricks provides all the necessary components: R language support, connectivity to both the source (Azure Data Lake Storage) and destination (Azure Synapse Analytics), and a job scheduler to automate the daily execution.

References

1. Azure Databricks R Support: Microsoft Corporation. (2023). R language. Azure Databricks Documentation. Retrieved from https://docs.microsoft.com/en-us/azure/databricks/languages/r/. This document confirms that R is a supported language in Azure Databricks, allowing for the execution of R scripts and notebooks.

2. Connecting Databricks to Synapse Analytics: Microsoft Corporation. (2023). Azure Synapse Analytics connector. Azure Databricks Documentation. Retrieved from https://docs.microsoft.com/en-us/azure/databricks/data/data-sources/azure/synapse-analytics. This source details the connector used to read from and write data to Azure Synapse Analytics from Databricks.

3. Connecting Databricks to Azure Data Lake Storage: Microsoft Corporation. (2023). Access Azure Data Lake Storage Gen2 and Blob Storage. Azure Databricks Documentation. Retrieved from https://docs.microsoft.com/en-us/azure/databricks/data/data-sources/azure/adls-gen2/. This guide explains how to configure access to Azure Data Lake Storage from Databricks notebooks.

4. Scheduling Databricks Jobs: Microsoft Corporation. (2023). Create, run, and manage Azure Databricks Jobs. Azure Databricks Documentation. Retrieved from https://docs.microsoft.com/en-us/azure/databricks/jobs. This documentation describes the job scheduling feature, which allows for the automation of notebooks on a recurring basis, such as daily.

Keep practicing

Take the free DP-300 practice test

Ten exam-style questions with answers and explanations, plus the exam facts and study guides.

Start the free practice test