PROFESSIONAL-DATA-ENGINEER practice question 4 of 10
You need to deploy additional dependencies to all of a Cloud Dataproc cluster at startup using an existing initialization action. Company security policies require that Cloud Dataproc nodes do not have access to the Internet so public initialization actions cannot fetch resources. What should you do?
Full set: 401 questionsThe primary constraint is that Cloud Dataproc nodes cannot access the public internet, which prevents initialization actions from downloading dependencies from public repositories. The correct and secure approach is to pre-stage all required dependencies in a location accessible from within the Google Cloud private network. A Cloud Storage bucket is the ideal service for this. By copying dependencies to a bucket and ensuring the cluster's subnet has Private Google Access enabled, the initialization action script can use gsutil to securely copy the files from the bucket to the cluster nodes for installation, fully complying with the security policy.
A. The Cloud SQL Proxy is a specific tool for connecting securely to Cloud SQL instances; it is not used for managing or deploying general software dependencies on a cluster.
B. Creating an SSH tunnel to the internet directly contravenes the explicit security requirement that nodes must not have internet access. This is a policy violation, not a solution.
D. The Network User role (roles/compute.networkUser) grants permissions to use network resources, such as subnets in a Shared VPC. It does not provide a mechanism for accessing software dependencies.
1. Google Cloud Documentation - Dataproc Initialization Actions: "We recommend that you store initialization actions in a Cloud Storage bucket... When you create a cluster, Dataproc copies the scripts from the bucket to the master and worker nodes in the cluster and then runs the scripts." This establishes Cloud Storage as the standard location for initialization action resources. (Source: Google Cloud, "Initialization actions," Section: "Where to store initialization actions").
2. Google Cloud Documentation - Configuring Private Google Access: "With Private Google Access, VMs that only have internal IP addresses (no external IP addresses) can reach the external IP addresses of Google APIs and services," including Cloud Storage. This is the underlying network configuration that enables VMs without internet access to reach the Cloud Storage API. (Source: Google Cloud, "Private Google Access," Section: "Overview").
3. Google Cloud Documentation - Dataproc Cluster Networking: In scenarios without external IP addresses, "To access Google Cloud services, including Cloud Storage where the Dataproc staging bucket is located, you must enable Private Google Access on the cluster subnet." This explicitly links the need for Private Google Access to allow Dataproc to access Cloud Storage in isolated networks. (Source: Google Cloud, "Dataproc Cluster Network Configuration," Section: "Cluster network requirements and considerations").
Keep practicing
Take the free PROFESSIONAL-DATA-ENGINEER practice test
Ten exam-style questions with answers and explanations, plus the exam facts and study guides.
More questions
Other PROFESSIONAL-DATA-ENGINEER practice questions
- Question 1You orchestrate ETL pipelines by using Cloud Composer One of the tasks in the Apache Airf…
- Question 2You want to rebuild your batch pipeline for structured data on Google Cloud You are using…
- Question 3You are designing an Apache Beam pipeline to enrich data from Cloud Pub/Sub with static r…
- Question 5A shipping company has live package-tracking data that is sent to an Apache Kafka stream …
- Question 6You use a dataset in BigQuery for analysis. You want to provide third-party companies wit…
- Question 7How would you query specific partitions in a BigQuery table?
