You have a large 5-TB AVRO file stored in a Cloud Storage bucket. Your analysts are proficient only in SQL and need access to the data stored in this…

Creating a BigQuery external table allows you to query data directly in Cloud Storage without ingesting it into BigQuery's managed storage. This approach is the fastest because it only involves a quick metadata definition, eliminating the time-consuming step of loading a 5-TB file. It is also the most cost-effective for this ad-hoc request, as you only pay for the query execution (bytes scanned) and not for storing the 5 TB of data within BigQuery. This solution provides a standard SQL interface directly meeting the analysts' needs.

A. Cloud Datastore is a NoSQL database and does not offer a standard SQL interface. It is not designed for large-scale analytical queries.

B. Loading 5 TB of data into BigQuery incurs storage costs and takes significant time, contradicting the "as soon as possible" and "cost-effective" requirements.

D. Creating and managing a Hadoop cluster (e.g., with Dataproc) is operationally complex, slow to set up, and more expensive than using a serverless tool like BigQuery.

1. Google Cloud Documentation - Introduction to external tables: "External tables are useful for querying data without loading it into BigQuery managed storage. For example

you might have a frequent ETL job that loads data from Cloud Storage into BigQuery. Instead of loading the data before running queries

you can create an external table that points to the data in Cloud Storage." This directly supports the speed and cost-effectiveness of option C.

2. Google Cloud Documentation - Querying Cloud Storage data: "You can query data in Cloud Storage directly from BigQuery... Querying external data sources is useful for... Performing ad hoc queries on your data without loading it into BigQuery." This confirms the use case for ad-hoc analysis on data residing in Cloud Storage.

3. Google Cloud Documentation - BigQuery pricing - Storage pricing: "Storage pricing is the cost to store data that you load into BigQuery." By using an external table (Option C)

this storage cost is avoided

making it more cost-effective than loading the data (Option B).

4. Google Cloud Documentation - What is Datastore?: "Datastore is a highly scalable NoSQL database for your applications." This confirms that Datastore is not a SQL database and is intended for application backends

not large-scale analytics.