Free PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice Test

The most efficient and integrated method to query data from BigQuery and load it into a pandas DataFrame within a Vertex AI (formerly AI Platform) Notebook is to use the built-in %%bigquery cell magic. This command allows you to execute a SQL query directly against BigQuery and store the results in a specified pandas DataFrame variable in a single step. This approach avoids the overhead of exporting data to intermediate files in Cloud Storage or a local disk, streamlining the workflow significantly.

B. Exporting to Google Drive and using its API is an unnecessarily complex, multi-step process not designed for this workflow.
C. Downloading to a local machine and then uploading introduces manual steps and inefficient data transfer through an intermediary machine.
D. This is a viable but less efficient method. It requires three separate commands (bq extract, gsutil cp, pandas.readcsv) and creates unnecessary intermediate files in Cloud Storage and on the notebook's local disk.

1. Google Cloud Documentation for Vertex AI Workbench: The official documentation explicitly describes using the %%bigquery magic command to query data and load it into a pandas DataFrame. It states, "To run a BigQuery query and store the results in a pandas DataFrame, add the --dest argument to the %%bigquery magic."

Source: Google Cloud, Vertex AI Documentation, "Query and explore data in a user-managed notebook".

2. Google Cloud BigQuery Client Libraries for Python: The %%bigquery magic is a convenient wrapper around the BigQuery client library. The library's todataframe() method is the underlying function that converts a BigQuery query result set directly into a pandas DataFrame, demonstrating the intended direct integration.

Source: Google Cloud, Python Client for Google BigQuery Documentation, google.cloud.bigquery.job.QueryJob.todataframe().

The scenario requires monitoring for two types of changes: "changes in the feature distributions" (data drift) and "changes in the correlations between the features" (concept drift). Vertex AI Model Monitoring detects data drift by default. To detect concept drift, feature attribution monitoring must be enabled. Given the large volume of prediction requests and the need to minimize cost, the most effective control is the prediction sampling rate. Setting a low prediction-sampling-rate (a value closer to 0) reduces the amount of data logged and analyzed, directly lowering monitoring costs while still providing a representative sample to detect drift.

A. A higher monitoring frequency increases the number of analysis jobs run, which increases, not minimizes, cost.
B. A prediction sampling rate closer to 1 would log and analyze almost all requests, maximizing costs, which is unsuitable for a high-volume scenario.
C. While lowering monitoring frequency reduces cost, reducing the sampling rate is the most direct and impactful method for managing costs with high prediction traffic.

1. Google Cloud Documentation, "Introduction to Vertex AI Model Monitoring": This document explains the types of drift detection. It states, "Feature attribution skew and drift shows you how the attribution of your features has changed between your training data and recent production data. A change in feature attributions might mean that the model is relying on a feature more or less to make its predictions, which can be a signal of concept drift." This supports the need for feature attribution monitoring to detect changes in feature correlations.

2. Google Cloud Documentation, "Configure model monitoring": This page details the configuration parameters for a monitoring job. It defines prediction-sampling-rate as "The fraction of incoming prediction requests that are logged... For example, if you set this parameter to 0.1, then 10% of your requests are logged." This confirms that a low sampling rate reduces the volume of data analyzed.

3. Google Cloud Documentation, "Vertex AI pricing", Model Monitoring section: The pricing model is explicitly stated as being based on the volume of data analyzed. "Model monitoring is priced by the number of predictions analyzed per month." This directly links cost to the number of predictions analyzed, reinforcing that reducing the sampling rate is the primary mechanism for cost control.

The question requires building a model that recommends products based on a user's purchase behavior and their similarity to other users. This is the definition of collaborative filtering. This method operates on user-item interaction data (e.g., purchases, ratings) to find users with similar tastes. It then recommends items that these similar users have liked but the target user has not yet interacted with. This approach does not require knowledge of item features, relying instead on the "wisdom of the crowd" to make relevant suggestions.

A. A classification model predicts a discrete label (e.g., will buy/won't buy) for a single item, rather than generating a ranked list of recommendations based on user similarity.
B. A knowledge-based filtering model relies on explicit user requirements and domain knowledge (e.g., "show me blue running shoes"), not implicit behavioral data like purchase history.
D. A regression model predicts a continuous value (e.g., a potential rating), which can be a component of a recommender, but it doesn't inherently capture the user-to-user similarity aspect.

1. Google Cloud Documentation: "Choose a model type for recommendations". Google's Recommendations AI documentation describes the "Others you may like" model type, which is a form of collaborative filtering that recommends items frequently purchased or viewed by users who also interacted with the same items as the current user. This directly aligns with the prompt's requirement.

Source: Google Cloud, Recommendations AI Documentation, "Choose a model type", Section: "Others you may like".

2. University Courseware: Ng, A. (2018). CS229 Lecture Notes: Recommender Systems. Stanford University.

Section 1, "The recommender system problem," defines the problem of predicting user preferences. Section 2, "Content-based filtering," is contrasted with collaborative filtering, which is described as leveraging ratings from other users to make recommendations, matching the scenario.

3. Academic Publication: Aggarwal, C. C. (2016). Recommender Systems: The Textbook. Springer.

Chapter 1, "An Introduction to Recommender Systems," Section 1.2, "Basic Models of Recommender Systems," explicitly defines collaborative filtering as a method that "uses the known preferences of a group of users to make recommendations to other users" (p. 5). This is the core principle described in the question.

DOI: https://doi.org/10.1007/978-3-319-29659-3

This scenario requires a scalable data processing pipeline and a powerful analytics data warehouse. Dataflow is a fully managed service designed for large-scale batch and stream processing, making it ideal for handling over a million daily files from Cloud Storage. It can perform the necessary transformations, such as using the Cloud Data Loss Prevention (DLP) API to redact PII. For the analytics component, BigQuery is a serverless, petabyte-scale data warehouse that provides a standard SQL ANSI-2011 compliant interface. It also supports regional datasets, satisfying the data residency requirement. This combination provides a robust, scalable, and compliant solution.

B. 1 Pub/Sub, 2 Datastore: Pub/Sub is a messaging service for data ingestion, not a processing engine. Datastore is a NoSQL database and does not offer a SQL interface.
C. 1 Dataflow, 2 Cloud SQL: While Dataflow is correct, Cloud SQL is a relational database (OLTP) not optimized for the large-scale analytical queries required for this volume of data, unlike BigQuery (OLAP).
D. 1 Cloud Function, 2 Cloud SQL: Cloud Functions are not suitable for large-scale, potentially long-running processing jobs due to execution time limits. Cloud SQL is not the optimal choice for analytics at this scale.

1. Google Cloud Documentation, "Dataflow overview": "Dataflow is a managed service for executing a wide variety of data processing patterns... Use cases include... ETL pipelines to move data, for example, from Cloud Storage to BigQuery." This supports using Dataflow for the processing step (Component 1).

2. Google Cloud Documentation, "What is BigQuery?": "BigQuery is a fully managed enterprise data warehouse that helps you manage and analyze your data with built-in features like machine learning, geospatial analysis, and business intelligence... BigQuery supports a standard SQL dialect that is ANSI:2011 compliant." This confirms BigQuery's suitability as the analytics warehouse (Component 2) with the required SQL interface.

3. Google Cloud Documentation, "Dataset locations": "You can specify a location to store your BigQuery data when you create a dataset... This setting controls the geographic location where your data is stored." This addresses the data residency requirement.

4. Google Cloud Architecture Center, "Store your data": This document contrasts different storage options. It recommends BigQuery for "Analytical querying (OLAP) in a data warehouse" and Cloud SQL for "Web frameworks and OLTP workloads," clarifying why BigQuery is the superior choice over Cloud SQL for this analytics scenario.

Oscillation in the loss function during neural network training is a classic indicator that the learning rate is too high. The learning rate determines the step size the optimization algorithm takes to minimize the loss. If the steps are too large, the optimizer will repeatedly overshoot the optimal minimum point in the loss landscape, causing the loss to bounce back and forth, or oscillate. By decreasing the learning rate, the step size is reduced, allowing for a more stable and gradual descent towards the minimum. This prevents overshooting and helps the model to converge smoothly.

A. Increase the size of the training batch: This provides a more accurate gradient estimate, which can smooth the loss curve, but it does not fix the fundamental problem of overshooting caused by a large learning rate.
B. Decrease the size of the training batch: This introduces more variance (noise) into the gradient updates, which would likely exacerbate the loss oscillation rather than reduce it.
C. Increase the learning rate hyperparameter: This would make the optimization steps even larger, worsening the overshooting problem and likely causing the loss to diverge completely.

1. Stanford University CS231n Course Notes: In the section on training dynamics, it is explicitly stated and visually demonstrated that "A learning rate that is too high will cause the loss to diverge... you may see the loss oscillating." This directly supports decreasing the learning rate to fix oscillation.

Source: Stanford CS231n: Convolutional Neural Networks for Visual Recognition, Module 1, "Setting up the data and the model," subsection "Learning rate."

2. Google Cloud Documentation: The official documentation for Vertex AI hyperparameter tuning identifies the learning rate as a critical parameter that controls the training process. It explains that finding an optimal value is key to convergence, implicitly supporting the need to adjust it (in this case, decrease it) when observing undesirable behavior like oscillation.

Source: Google Cloud, Vertex AI Documentation, "Overview of hyperparameter tuning," section "Hyperparameters."

3. Academic Publication (Foundational Textbook): The "Deep Learning" book, a standard academic text, explains the challenges in neural network optimization. It notes that if the learning rate is too high, "the cost may actually increase after a parameter update," leading to oscillations or divergence.

Source: Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press. Chapter 8, "Optimization for Training Deep Models," Section 8.3.2, "Challenges in Neural Network Optimization," pp. 299-300.

The optimal solution is to leverage BigQuery, a fully managed, serverless, petabyte-scale data warehouse designed for high-speed analytics using SQL. This approach directly meets all requirements:
1. Serverless: BigQuery is serverless, eliminating infrastructure management.
2. SQL Syntax: PySpark transformation logic can be converted to standard SQL for use in BigQuery.
3. Speed: BigQuery's massively parallel processing engine is designed to execute complex queries on large datasets far faster than a typical PySpark job on a long-running cluster. Loading data from Cloud Storage into BigQuery is a standard, efficient pattern, and materializing the transformed data into a new table is best practice for subsequent ML model training.

A. Data Fusion is primarily a GUI-based data integration service, which does not meet the specific requirement to use and convert existing logic to SQL syntax.
B. Dataproc is not serverless; it requires provisioning and managing a Spark cluster, which contradicts a core requirement of the desired solution.
C. Cloud SQL is a relational database designed for transactional (OLTP) workloads, not large-scale analytical transformations. It would be a performance bottleneck.

1. Google Cloud Documentation, "What is BigQuery?": "BigQuery is a serverless, highly scalable, and cost-effective cloud data warehouse... BigQuery provides a SQL interface for querying and managing data." This source confirms that BigQuery is serverless, scalable, and uses SQL, directly addressing the core requirements.

2. Google Cloud Documentation, "Introduction to loading data from Cloud Storage": "You can load data into BigQuery from Cloud Storage in batch." This supports the first step of the proposed pipeline in the correct answer (D).

3. Google Cloud Documentation, "What is Dataproc?": "Dataproc is a fully managed and highly scalable service for running Apache Spark... You can easily create and delete Dataproc clusters..." This reference confirms that Dataproc is cluster-based, making it an incorrect choice for the serverless requirement.

4. Google Cloud Documentation, "Choosing a database option": This documentation page positions Cloud SQL for "OLTP workloads" and BigQuery for "OLAP workloads" and "Big data analytics," clarifying why Cloud SQL (Option C) is inappropriate for large-scale data transformation.

5. Stanford University, CS229: Machine Learning, "Lecture Notes 13: Recommender Systems & Large-Scale ML": Course materials often discuss the architectural pattern of using a scalable data warehouse like BigQuery for pre-processing and feature engineering at scale before ML model training, which aligns with the logic in answer D. The notes emphasize using scalable, parallel systems for large data transformations.

The primary goal is to accelerate the training of a Convolutional Neural Network (CNN) with minimal initial effort for experimentation. A Deep Learning VM is ideal as it comes pre-configured with necessary drivers (NVIDIA drivers, CUDA) and ML frameworks, eliminating manual setup time. Using a single GPU provides a significant performance boost for CNNs over CPUs without requiring code modifications for distribution strategies. TensorFlow and PyTorch automatically prioritize an available GPU for operations, fitting the "no manual device placement" constraint. This option represents the fastest and most direct path to achieving the desired training speedup for an initial experiment.

A. AVM on Compute Engine and 1 TPU with all dependencies installed manually. Training on TPUs requires significant, specific code changes (e.g., using tf.distribute.TPUStrategy), which contradicts the desire for a quick experiment with the existing codebase.
B. AVM on Compute Engine and 8 GPUs with all dependencies installed manually. Effectively using multiple GPUs requires code modifications to implement a distribution strategy (e.g., tf.distribute.MirroredStrategy). Without this, only one GPU would be used.
D. A Deep Learning VM with more powerful CPU e2-highcpu-16 machines with all libraries pre-installed. While a faster CPU helps, the performance gain for CNN training is marginal compared to the orders-of-magnitude speedup provided by a GPU, which is designed for parallel computations.

1. Google Cloud Documentation, "Deep Learning VM overview": "Deep Learning VM Image makes it easy and fast to instantiate a virtual machine image containing the most popular AI frameworks... It includes NVIDIA® drivers and the CUDA-X AI stack for GPU instances to accelerate your tasks." This supports the choice of a pre-configured environment to save time.

2. TensorFlow Official Documentation, "Use a GPU": Under the section "Manual device placement," it states: "If a TensorFlow operation has both CPU and GPU implementations, the GPU device is prioritized when the operation is assigned." This confirms that with a single GPU instance and no explicit placement, the code will automatically leverage the GPU, making option C viable without code changes.

3. TensorFlow Official Documentation, "Distributed training with TensorFlow": This guide details the necessity of using tf.distribute.Strategy subclasses like TPUStrategy (for TPUs) and MirroredStrategy (for multi-GPU on one machine). This confirms that options A and B would require code modifications, which the user is trying to avoid in this initial experimental phase.

4. Stanford University, CS231n Convolutional Neural Networks for Visual Recognition, "Hardware and Software": The course notes explain: "Training Neural Networks is a very computationally expensive task... The fact that GPUs are very good at matrix/vector operations makes them ideal for training Neural Networks." This academic source validates prioritizing a GPU (Option C) over a more powerful CPU (Option D) for CNNs.

With less than 1% positive examples, the dataset is severely imbalanced. This often prevents model convergence because the loss function is dominated by the majority (negative) class, providing little incentive for the model to learn the minority class patterns. Downsampling the majority class creates a more balanced training set, which helps the model learn from both classes. However, this alters the class distribution. Upweighting (a form of cost-sensitive learning) is applied to give more importance to the minority class examples, ensuring the model's predictions are calibrated to the original, real-world probability of failures and preventing the loss of information from the downsampled class. This combined approach is a robust strategy to handle severe imbalance.

A. "Generating" examples can mean naive oversampling (duplication), which risks overfitting. This option is less specific and robust than the combined downsampling and weighting strategy in C.
B. Simply changing the model architecture to a CNN does not address the fundamental data imbalance problem. The CNN would likely suffer from the same convergence failure.
D. This is extreme downsampling. Removing over 99% of the negative examples would cause a massive loss of information, likely leading to a model that generalizes poorly.

1. Google Cloud Documentation - Best practices for creating training data: In the context of tabular workflows, Google's documentation recommends resampling for imbalanced datasets: "If you have a highly imbalanced dataset, you should consider the following... Undersampling the majority class." This directly supports the "downsample" part of the correct answer.

Source: Google Cloud. (n.d.). Best practices for creating training data. Vertex AI Documentation. Retrieved from https://cloud.google.com/vertex-ai/docs/tabular-data/best-practices. See section: "Work with imbalanced data".

2. Google Cloud Documentation - Data splits and weights for tabular data: Google's documentation for AutoML Tables explains the use of a weight column to handle imbalance, which is conceptually equivalent to upweighting: "For classification models, class weights are useful when you have an imbalanced dataset... A higher weight tells the model that examples of that class are more important during the training process."

Source: Google Cloud. (n.d.). Data splits and weights for tabular data. AutoML Tables Documentation. Retrieved from https://cloud.google.com/automl-tables/docs/data-splits-weights. See section: "Weight column".

3. Stanford University Courseware - CS229 Machine Learning: Course materials frequently discuss handling class imbalance. The lecture notes on "Evaluation Metrics and Imbalanced Datasets" describe modifying the loss function by weighting examples, which is a core principle behind upweighting, and also discuss the trade-offs of undersampling.

Source: Ng, A., & Katanforoosh, K. (2018). CS229 - Machine Learning: Autumn 2018. Stanford University. See lecture notes on "Practical advice for applying machine learning," which covers skewed datasets and the use of modified loss functions (weighting).

4. Academic Publication - Learning from Imbalanced Data: This highly-cited survey paper details various approaches to the class imbalance problem. It describes undersampling and cost-sensitive learning (which includes upweighting) as two primary and effective methods. The paper notes that hybrid approaches combining sampling with cost-sensitive methods are often superior.

Source: He, H., & Garcia, E. A. (2009). Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284. See Sections 3.2 "Under-sampling" and 4.1 "Cost-sensitive learning". DOI: https://doi.org/10.1109/TKDE.2008.239

Data Catalog is Google Cloud's fully managed data discovery and metadata management service. It is specifically designed for enterprise-scale scenarios involving numerous datasets. It automatically ingests metadata from sources like BigQuery, including table descriptions, and provides a powerful search interface. This allows users to efficiently find relevant data assets by searching for keywords in their descriptions, making it the ideal tool for the described task of locating the correct table among thousands.

B. Tagging AI Platform models is a best practice for tracking data lineage after a model is trained, not for the initial discovery of the training data itself.
C. Maintaining a custom lookup table is inefficient and redundant. It requires manual effort to build and maintain, whereas Data Catalog provides this functionality automatically as a managed service.
D. Querying INFORMATIONSCHEMA is a valid way to get metadata, but it is less scalable and user-friendly than Data Catalog for enterprise-wide discovery, especially across multiple projects.

1. Google Cloud, Data Catalog documentation, "Introduction to Data Catalog": "Data Catalog is a fully managed and scalable data discovery and metadata management service. [...] Data Catalog provides a centralized place for organizations to find, understand, and manage data." This establishes Data Catalog as the purpose-built tool for this scenario.

2. Google Cloud, Data Catalog documentation, "Search with Data Catalog": "Data Catalog provides a powerful predicate-based search experience for finding data assets to which you have access. You can search on the metadata of an asset." This section explicitly details how to search using metadata like descriptions, which directly addresses the user's need.

3. Google Cloud, BigQuery documentation, "Introduction to BigQuery INFORMATIONSCHEMA": "BigQuery INFORMATIONSCHEMA is a series of views that provide access to metadata about datasets, routines, tables, views, jobs, reservations, and streaming data." This confirms that option D is technically possible, but the existence of the more specialized Data Catalog service makes it a less optimal choice for enterprise-scale discovery.

This combination provides an integrated, serverless MLOps platform. Vertex AI Pipelines is Google Cloud's managed orchestration service designed for building and scheduling ML workflows, including model retraining. It natively supports executing steps within custom Docker containers. Vertex AI Prediction (formerly AI Platform Prediction) is a managed service specifically for deploying models for online prediction. It offers built-in autoscaling to handle fluctuating request loads and integrates with Cloud Monitoring to provide visibility into model performance, directly fulfilling all requirements of the scenario.

A. Vertex AI Pipelines and App Engine: App Engine is a general-purpose platform for applications, not a specialized, optimized service for ML model serving like Vertex AI Prediction.
C. Cloud Composer, BigQuery ML, and AI Platform Prediction: BigQuery ML does not support training with custom Docker containers; it is designed for training models on data within BigQuery using SQL syntax.
D. Cloud Composer, AI Platform Training with custom containers, and App Engine: While Cloud Composer can schedule workflows, Vertex AI Pipelines is more tightly integrated with the ML ecosystem. App Engine is not the optimal choice for model serving.

1. Vertex AI Pipelines: "Vertex AI Pipelines helps you to automate, monitor, and govern your ML systems by orchestrating your ML workflow in a serverless manner... You can use the Google Cloud Pipeline Components SDK to build components that run your code in a Docker container."

Source: Google Cloud Documentation, "Introduction to Vertex AI Pipelines".

2. Vertex AI Prediction: "When you deploy a custom-trained model or an AutoML model to an endpoint on Vertex AI, you create a DeployedModel resource. The DeployedModel resource serves online predictions from your model... For online prediction, Vertex AI scales the number of prediction nodes to handle the traffic to your models."

Source: Google Cloud Documentation, "Deploy a model to an endpoint".

3. Scheduling with Vertex AI Pipelines: "You can schedule a pipeline to run on a recurring basis by using the PipelineJobSchedule service."

Source: Google Cloud Documentation, "Schedule pipeline runs".

4. BigQuery ML Limitations: "BigQuery ML lets you create and execute machine learning models in BigQuery by using standard SQL queries... It supports a specific set of model types." This contrasts with the requirement for custom Docker containers, which allow for arbitrary model architectures and training code.

Source: Google Cloud Documentation, "Introduction to BigQuery ML".