PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice Questions

AI Platform Notebooks is a service that provides managed Jupyter notebooks for data science and

machine learning. You can use AI Platform Notebooks to create, run, and share your code and

analysis in a collaborative and interactive environment1. BigQuery is a service that allows you to

analyze large-scale and complex data using SQL queries. You can use BigQuery to stream, store, and

query your data in a fast and cost-effective way2. Pandas is a popular Python library that provides

data structures and tools for data analysis and manipulation. You can use pandas to create,

manipulate, and visualize dataframes, which are tabular data structures with rows and columns3.

AI Platform Notebooks provides a cell magic, %%bigquery, that allows you to run SQL queries on

BigQuery data and ingest the results as a pandas dataframe. A cell magic is a special command that

applies to the whole cell in a Jupyter notebook. The %%bigquery cell magic can take various

arguments, such as the name of the destination dataframe, the name of the destination table in

BigQuery, the project ID, and the query parameters4. By using the %%bigquery cell magic, you can

query the data in BigQuery with minimal code and manipulate the results with pandas in AI Platform

Notebooks. This is the most convenient and efficient way to achieve your goal.

The other options are not as good as option A, because they involve more steps, more code, and

more manual effort. Option B requires you to export your table as a CSV file from BigQuery to Google

Drive, and then use the Google Drive API to ingest the file into your notebook instance. This option is

cumbersome and time-consuming, as it involves moving the data across different services and

formats. Option C requires you to download your table from BigQuery as a local CSV file, and then

upload it to your AI Platform notebook instance. This option is also inefficient and impractical, as it

involves downloading and uploading large files, which can take a long time and consume a lot of

bandwidth. Option D requires you to use a bash cell in your AI Platform notebook to export the table

as a CSV file to Cloud Storage, and then copy the data into the notebook. This option is also complex

and unnecessary, as it involves using different commands and tools to move the data around.

Therefore, option A is the best option for this use case.

Reference:

AI Platform Notebooks documentation

BigQuery documentation

pandas documentation

Using Jupyter magics to query BigQuery data

The best option for using Vertex AI Model Monitoring for drift detection and minimizing the cost is to

use the features and the feature attributions for monitoring, and set a prediction-sampling-rate value

that is closer to 0 than 1. This option allows you to leverage the power and flexibility of Google Cloud

to detect feature drift in the input predict requests for custom models, and reduce the storage and

computation costs of the model monitoring job. Vertex AI Model Monitoring is a service that can

track and compare the results of multiple machine learning runs. Vertex AI Model Monitoring can

monitor the model’s prediction input data for feature skew and drift. Feature drift occurs when the

feature data distribution in production changes over time. If the original training data is not available,

you can enable drift detection to monitor your models for feature drift. Vertex AI Model Monitoring

uses TensorFlow Data Validation (TFDV) to calculate the distributions and distance scores for each

feature, and compares them with a baseline distribution. The baseline distribution is the statistical

distribution of the feature’s values in the training data. If the training data is not available, the

baseline distribution is calculated from the first 1000 prediction requests that the model receives. If

the distance score for a feature exceeds an alerting threshold that you set, Vertex AI Model

Monitoring sends you an email alert. However, if you use a custom model, you can also enable

feature attribution monitoring, which can provide more insights into the feature drift. Feature

attribution monitoring analyzes the feature attributions, which are the contributions of each feature

to the prediction output. Feature attribution monitoring can help you identify the features that have

the most impact on the model performance, and the features that have the most significant drift

over time. Feature attribution monitoring can also help you understand the relationship between the

features and the prediction output, and the correlation between the features1. The prediction-

sampling-rate is a parameter that determines the percentage of prediction requests that are logged

and analyzed by the model monitoring job. Using a lower prediction-sampling-rate can reduce the

storage and computation costs of the model monitoring job, but also the quality and validity of the

data. Using a lower prediction-sampling-rate can introduce sampling bias and noise into the data,

and make the model monitoring job miss some important features or patterns of the data. However,

using a higher prediction-sampling-rate can increase the storage and computation costs of the model

monitoring job, and also the amount of data that needs to be processed and analyzed. Therefore,

there is a trade-off between the prediction-sampling-rate and the cost and accuracy of the model

monitoring job, and the optimal prediction-sampling-rate depends on the business objective and the

data characteristics2. By using the features and the feature attributions for monitoring, and setting a

prediction-sampling-rate value that is closer to 0 than 1, you can use Vertex AI Model Monitoring for

drift detection and minimize the cost.

The other options are not as good as option D, for the following reasons:

Option A: Using the features for monitoring and setting a monitoring-frequency value that is higher

than the default would not enable feature attribution monitoring, and could increase the cost of the

model monitoring job. The monitoring-frequency is a parameter that determines how often the

model monitoring job analyzes the logged prediction requests and calculates the distributions and

distance scores for each feature. Using a higher monitoring-frequency can increase the frequency

and timeliness of the model monitoring job, but also the computation costs of the model monitoring

job. Moreover, using the features for monitoring would not enable feature attribution monitoring,

which can provide more insights into the feature drift and the model performance1.

Option B: Using the features for monitoring and setting a prediction-sampling-rate value that is

closer to 1 than 0 would not enable feature attribution monitoring, and could increase the cost of the

model monitoring job. The prediction-sampling-rate is a parameter that determines the percentage

of prediction requests that are logged and analyzed by the model monitoring job. Using a higher

prediction-sampling-rate can increase the quality and validity of the data, but also the storage and

computation costs of the model monitoring job. Moreover, using the features for monitoring would

not enable feature attribution monitoring, which can provide more insights into the feature drift and

the model performance12.

Option C: Using the features and the feature attributions for monitoring and setting a monitoring-

frequency value that is lower than the default would enable feature attribution monitoring, but

could reduce the frequency and timeliness of the model monitoring job. The monitoring-frequency is

a parameter that determines how often the model monitoring job analyzes the logged prediction

requests and calculates the distributions and distance scores for each feature. Using a lower

monitoring-frequency can reduce the computation costs of the model monitoring job, but also the

frequency and timeliness of the model monitoring job. This can make the model monitoring job less

responsive and effective in detecting and alerting the feature drift1.

Reference:

Preparing for Google Cloud Certification: Machine Learning Engineer, Course 3: Production ML

Systems, Week 4: Evaluation

Google Cloud Professional Machine Learning Engineer Exam Guide, Section 3: Scaling ML models in

production, 3.3 Monitoring ML models in production

Official Google Cloud Certified Professional Machine Learning Engineer Study Guide, Chapter 6:

Production ML Systems, Section 6.3: Monitoring ML Models

Using Model Monitoring

Understanding the score threshold slider

A recommender system is a type of machine learning system that suggests relevant items to users

based on their p

and behavior. Recommender systems are widely used in e-commerce,

media, and entertainment industries to enhance user experience and increase revenue1

There are different types of recommender systems that use different filtering methods to generate

recommendations. The most common types are:

Content-based filtering: This method uses the features of the items and the users to find the

similarity between them. For example, a content-based recommender system for movies may use

the genre, director, cast, and ratings of the movies, and the preferences, demographics, and history

of the users, to recommend movies that are similar to the ones the user liked before2

Collaborative filtering: This method uses the feedback and ratings of the users to find the similarity

between them and the items. For example, a collaborative filtering recommender system for books

may use the ratings of the users for different books, and recommend books that are liked by other

users who have similar ratings to the target user3

Hybrid method: This method combines content-based and collaborative filtering methods to

overcome the limitations of each method and improve the accuracy and diversity of the

recommendations. For example, a hybrid recommender system for music may use both the features

of the songs and the artists, and the ratings and listening habits of the users, to recommend songs

that match the user’s taste and preferences4

Deep learning-based: This method uses deep neural networks to learn complex and non-linear

patterns from the data and generate recommendations. Deep learning-based recommender systems

can handle large-scale and high-dimensional data, and incorporate various types of information, such

as text, images, audio, and video. For example, a deep learning-based recommender system for

fashion may use the images and descriptions of the products, and the profiles and feedback of the

users, to recommend products that suit the user’s style and preferences.

For the use case of building a model that will recommend new products to the user based on their

purchase behavior and similarity with other users, the best option is to build a collaborative-based

filtering model. This is because collaborative filtering can leverage the implicit feedback and ratings

of the users to find the items that are most likely to interest them. Collaborative filtering can also

help discover new products that the user may not be aware of, and increase the diversity and

serendipity of the recommendations3

The other options are not as suitable for this use case. Building a classification model or a regression

model using the features as predictors is not a good idea, as these models are not designed for

recommendation tasks, and may not capture the preferences and behavior of the users. Building a

knowledge-based filtering model is not relevant, as this method uses the explicit knowledge and

requirements of the users to find the items that meet their criteria, and does not rely on the

purchase behavior or similarity with other users.

Reference: 1: Recommender system 2: Content-based filtering 3: Collaborative filtering 4: Hybrid

recommender system : [Deep learning for recommender systems] : [Knowledge-based recommender

system]

A data pipeline is a set of steps or processes that move data from one or more sources to one or

more destinations, usually for the purpose of analysis, transformation, or storage. A data pipeline

can be designed using various components, such as data sources, data processing tools, data storage

systems, and data analytics tools1

To design a data pipeline for analyzing customer sentiments in each call, one should consider the

following requirements and constraints:

The call center receives over one million calls daily, and data is stored in Cloud Storage. This implies

that the data is large, unstructured, and distributed, and requires a scalable and efficient data

processing tool that can handle various types of data formats, such as audio, text, or image.

The data collected must not leave the region in which the call originated, and no Personally

Identifiable Information (Pll) can be stored or analyzed. This implies that the data is sensitive and

subject to data privacy and compliance regulations, and requires a secure and reliable data storage

system that can enforce data encryption, access control, and regional policies.

The data science team has a third-party tool for visualization and access which requires a SQL ANSI-

2011 compliant interface. This implies that the data analytics tool is external and independent of the

data pipeline, and requires a standard and compatible data interface that can support SQL queries

and operations.

One of the best options for selecting components for data processing and for analytics is to use

Dataflow for data processing and BigQuery for analytics. Dataflow is a fully managed service for

executing Apache Beam pipelines for data processing, such as batch or stream processing, extract-

transform-load (ETL), or data integration. BigQuery is a serverless, scalable, and cost-effective data

warehouse that allows you to run fast and complex queries on large-scale data23

Using Dataflow and BigQuery has several advantages for this use case:

Dataflow can process large and unstructured data from Cloud Storage in a parallel and distributed

manner, and apply various transformations, such as converting audio to text, extracting sentiment

scores, or anonymizing PII. Dataflow can also handle both batch and stream processing, which can

enable real-time or near-real-time analysis of the call data.

BigQuery can store and analyze the processed data from Dataflow in a secure and reliable way, and

enforce data encryption, access control, and regional policies. BigQuery can also support SQL ANSI-

2011 compliant interface, which can enable the data science team to use their third-party tool for

visualization and access. BigQuery can also integrate with various Google Cloud services and tools,

such as AI Platform, Data Studio, or Looker.

Dataflow and BigQuery can work seamlessly together, as they are both part of the Google Cloud

ecosystem, and support various data formats, such as CSV, JSON, Avro, or Parquet. Dataflow and

BigQuery can also leverage the benefits of Google Cloud infrastructure, such as scalability,

performance, and cost-effectiveness.

The other options are not as suitable or feasible. Using Pub/Sub for data processing and Datastore for

analytics is not ideal, as Pub/Sub is mainly designed for event-driven and asynchronous messaging,

not data processing, and Datastore is mainly designed for low-latency and high-throughput key-value

operations, not analytics. Using Cloud Function for data processing and Cloud SQL for analytics is not

optimal, as Cloud Function has limitations on the memory, CPU, and execution time, and does not

support complex data processing, and Cloud SQL is a relational database service that may not scale

well for large-scale data. Using Cloud Composer for data processing and Cloud SQL for analytics is not

relevant, as Cloud Composer is mainly designed for orchestrating complex workflows across multiple

systems, not data processing, and Cloud SQL is a relational database service that may not scale well

for large-scale data.

Reference: 1: Data pipeline 2: Dataflow overview 3: BigQuery overview : [Dataflow documentation] :

[BigQuery documentation]

Oscillation in the loss during batch training of a neural network means that the model is

overshooting the optimal point of the loss function and bouncing back and forth. This can prevent

the model from converging to the minimum loss value. One of the main reasons for this

phenomenon is that the learning rate hyperparameter, which controls the size of the steps that the

model takes along the gradient, is too high. Therefore, decreasing the learning rate hyperparameter

can help the model take smaller and more precise steps and avoid oscillation. This is a common

technique to improve the stability and performance of neural network training12.

Reference:

Interpreting Loss Curves

Is learning rate the only reason for training loss oscillation after few epochs?

BigQuery is a serverless, scalable, and cost-effective data warehouse that allows users to run SQL

queries on large volumes of data. BigQuery Load is a tool that can ingest data from Cloud Storage

into BigQuery tables. BigQuery SQL is a dialect of SQL that supports many of the same functions and

operations as PySpark, such as window functions, aggregate functions, joins, and subqueries. By

using BigQuery Load and BigQuery SQL, you can rebuild your ML pipeline for structured data on

Google Cloud without having to manage any servers or clusters, and with faster performance and

lower cost than using PySpark on Dataproc. You can also use BigQuery ML to create and evaluate ML

models using SQL commands. Reference:

BigQuery documentation

BigQuery Load documentation

BigQuery SQL reference

BigQuery ML documentation

In this scenario, the goal is to speed up model training for a CNN-based architecture on Google

Cloud. The code does not include any manual device placement and has not been wrapped in

Estimator model-level abstraction. Given these constraints, the best environment to train the model

on would be a Deep Learning VM with an n1-standard-2 machine and 1 GPU with all libraries pre-

installed. Option C is the correct answer.

Option C: A Deep Learning VM with an n1-standard-2 machine and 1 GPU with all libraries pre-

installed. This option is the most suitable for the scenario because it provides a ready-to-use

environment for deep learning on Google Cloud. A Deep Learning VM is a specialized VM image that

is pre-installed with popular deep learning frameworks such as TensorFlow, PyTorch, Keras, and

more. A Deep Learning VM also comes with NVIDIA GPU drivers and CUDA libraries that enable GPU

acceleration for model training. A Deep Learning VM can be easily configured and launched from the

Google Cloud Console or the Cloud SDK. An n1-standard-2 machine is a general-purpose machine

type that provides 2 vCPUs and 7.5 GB of memory. This machine type can be sufficient for running a

CNN-based architecture. A GPU is a specialized hardware accelerator that can speed up the

computation of matrix operations and convolutions, which are common in CNN-based architectures.

By using a Deep Learning VM with an n1-standard-2 machine and 1 GPU, the model training can be

significantly faster than on an on-premises CPU-only infrastructure.

Option A: A VM on Compute Engine and 1 TPU with all dependencies installed manually. This option

is not suitable for the scenario because it requires manual installation of dependencies and device

placement. A TPU is a custom-designed ASIC that can provide high performance and efficiency for

TensorFlow models. However, to use a TPU, the code needs to include manual device placement and

be wrapped in Estimator model-level abstraction. Moreover, to use a TPU, the dependencies such as

TensorFlow, Cloud TPU Client, and Cloud Storage need to be installed manually on the VM. This

option can be complex and time-consuming to set up and may not be compatible with the existing

code.

Option B: A VM on Compute Engine and 8 GPUs with all dependencies installed manually. This option

is not suitable for the scenario because it requires manual installation of dependencies and may not

be cost-effective. While using 8 GPUs can provide high parallelism and speed for model training, it

also increases the cost and complexity of the environment. Moreover, to use GPUs, the

dependencies such as NVIDIA GPU drivers, CUDA libraries, and deep learning frameworks need to be

installed manually on the VM. This option can be tedious and error-prone to set up and may not be

necessary for the scenario.

Option D: A Deep Learning VM with more powerful CPU e2-highcpu-16 machines with all libraries

pre-installed. This option is not suitable for the scenario because it does not leverage GPU

acceleration for model training. While using more powerful CPU machines can provide more

compute resources and memory for model training, it may not be as fast and efficient as using GPU

machines. CPU machines are not optimized for matrix operations and convolutions, which are

common in CNN-based architectures. Moreover, using more powerful CPU machines can also

increase the cost of the environment. This option can be suboptimal and wasteful for the scenario.

Reference:

Deep Learning VM Image documentation

Compute Engine documentation

Cloud TPU documentation

Machine types documentation

GPUs on Compute Engine documentation

The class imbalance problem is a common challenge in machine learning, especially in classification

tasks. It occurs when the distribution of the target classes is highly skewed, such that one class (the

majority class) has much more examples than the other class (the minority class). The minority class

is often the more interesting or important class, such as failure incidents, fraud cases, or rare

diseases. However, most machine learning algorithms are designed to optimize the overall accuracy,

which can be biased towards the majority class and ignore the minority class. This can result in poor

predictive performance, especially for the minority class.

There are different techniques to deal with the class imbalance problem, such as data-level methods,

algorithm-level methods, and evaluation-level methods1. Data-level methods involve resampling the

original dataset to create a more balanced class distribution. There are two main types of data-level

methods: oversampling and undersampling. Oversampling methods increase the number of

examples in the minority class, either by duplicating existing examples or by generating synthetic

examples. Undersampling methods reduce the number of examples in the majority class, either by

randomly removing examples or by using clustering or other criteria to select representative

examples. Both oversampling and undersampling methods can be combined with upweighting or

downweighting, which assign different weights to the examples according to their class frequency, to

further balance the dataset.

For the use case of investigating failures of a production line component based on sensor readings,

the best option is to downsample the data with upweighting to create a sample with 10% positive

examples. This option involves randomly removing some of the negative examples (the majority

class) until the ratio of positive to negative examples is 1:9, and then assigning higher weights to the

positive examples to compensate for their low frequency. This option can create a more balanced

dataset that can improve the performance of the classification models, while preserving the diversity

and representativeness of the original data. This option can also reduce the computation time and

memory usage, as the size of the dataset is reduced. Therefore, downsampling the data with

upweighting to create a sample with 10% positive examples is the best option for this use case.

Reference:

A Systematic Study of the Class Imbalance Problem in Convolutional Neural Networks

Data Catalog is a fully managed and scalable metadata management service that allows you to

quickly discover, manage, and understand your data in Google Cloud. You can use Data Catalog to

search the BigQuery datasets by using keywords in the table description, as well as other metadata

attributes such as table name, column name, labels, tags, and more. Data Catalog also provides a rich

browsing experience that lets you explore the schema, preview the data, and access the BigQuery

console directly from the Data Catalog UI. Data Catalog helps you find the data that you need for your

model building on AI Platform without writing any code or queries.

Reference:

[Data Catalog documentation]

[Data Catalog overview]

[Searching for data assets]

Vertex AI Pipelines and AI Platform Prediction are the platform components that best suit the

requirements of the data science team. Vertex AI Pipelines is a service that allows you to orchestrate

and automate your machine learning workflows using pipelines. Pipelines are portable and scalable

ML workflows that are based on containers. You can use Vertex AI Pipelines to schedule model

retraining, use custom containers, and integrate with other Google Cloud services. AI Platform

Prediction is a service that allows you to host your trained models and serve online predictions. You

can use AI Platform Prediction to deploy models trained on Vertex AI or elsewhere, and benefit from

features such as autoscaling, monitoring, logging, and explainability. Reference:

Vertex AI Pipelines

AI Platform Prediction