DEA-C01 practice question 2 of 25

A data engineer needs to build an enterprise data catalog based on the company's Amazon S3 buckets and Amazon RDS databases. The data catalog must include storage format metadata for the data in the catalog. Which solution will meet these requirements with the LEAST effort?

Full set: 287 questions
Answer options
A: Use an AWS Glue crawler to scan the S3 buckets and RDS databases and build a data catalog. Use data stewards to inspect the data and update the data catalog with the data format.
B: Use an AWS Glue crawler to build a data catalog. Use AWS Glue crawler classifiers to recognize the format of data and store the format in the catalog. Correct answer
C: Use Amazon Macie to build a data catalog and to identify sensitive data elements. Collect the data format information from Macie.
D: Use scripts to scan data elements and to assign data classifications based on the format of the data.
Correct Answer:
B. Use an AWS Glue crawler to build a data catalog. Use AWS Glue crawler classifiers to recognize the format of data and store the format in the catalog.
Explanation

The most efficient way to build a data catalog from S3 and RDS with minimal effort is by using an AWS Glue crawler. A crawler can connect to various data stores, automatically infer schemas, and populate the AWS Glue Data Catalog. To specifically identify the storage format (e.g., Parquet, CSV, JSON), the crawler uses classifiers. AWS Glue provides built-in classifiers for common formats and allows for custom classifiers for more complex or non-standard data formats. This automated discovery and classification process is the core purpose of Glue crawlers and requires the least manual intervention.

Why Incorrect

A. Requiring data stewards to manually inspect and update the catalog contradicts the "LEAST effort" requirement, as this is a manual process.
C. Amazon Macie is a data security service for discovering sensitive data, not a tool for building a general-purpose data catalog with storage format metadata.
D. Writing custom scripts to scan data and assign classifications is a high-effort, manual approach that AWS Glue crawlers are designed to replace.

References

1. AWS Glue Developer Guide. (2023). Cataloging tables with a crawler. AWS Documentation. Retrieved from https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html

2. AWS Glue Developer Guide. (2023). Working with classifiers in AWS Glue. AWS Documentation. Retrieved from https://docs.aws.amazon.com/glue/latest/dg/classifiers.html (Section: "Built-in classifiers in AWS Glue")

Scroll to Top