A data engineer needs to build an enterprise data catalog based on the company's Amazon S3 buckets and Amazon RDS databases. The data catalog must…

An AWS Glue crawler is the purpose-built service for automatically discovering and cataloging metadata from various data sources, including Amazon S3 and Amazon RDS. Crawlers use a prioritized list of classifiers (both built-in and custom) to automatically determine the data's schema, including its storage format (e.g., Parquet, JSON, CSV). This process populates the AWS Glue Data Catalog with the required metadata tables. This automated approach is the most direct and efficient method, fulfilling the "LEAST effort" requirement by eliminating the need for manual scripting or metadata entry.

1. AWS Glue Developer Guide - Cataloging tables with a crawler: "A crawler connects to a data store

progresses through a prioritized list of classifiers to determine the schema for your data

and then creates metadata tables in your Data Catalog." This confirms the crawler's primary function.

2. AWS Glue Developer Guide - Working with classifiers on the AWS Glue console: "AWS Glue uses classifiers to infer the schema of your data. [...] AWS Glue provides built-in classifiers for various formats

including JSON

CSV

Parquet

ORC

Avro

and Grok." This explains the mechanism for automatically identifying the storage format.

3. AWS Glue Developer Guide - Crawler source types: This section documents that crawlers can use "Amazon S3 paths" and "JDBC data stores" (which includes Amazon RDS) as sources

confirming its applicability to the scenario. (See section: "Defining a crawler").