MLA-C01 Practice Questions

The primary goal is to minimize false positives, which are instances incorrectly predicted as positive (e.g., predicting a weed is present when it is not). This is equivalent to maximizing the precision metric, defined as True Positives / (True Positives + False Positives). To increase the precision score, the model must reduce the number of false positives. The Amazon SageMaker Linear Learner algorithm provides hyperparameters to optimize the model's classification threshold for specific metrics. By increasing the value of a precision-oriented hyperparameter, the training process is explicitly guided to favor a model that makes fewer false positive errors.

A. Set the value of the weight decay hyperparameter to zero: This removes L2 regularization, which can increase the risk of overfitting. It is a general model complexity control, not a direct tool for minimizing false positives.

B. Increase the number of training epochs: Increasing epochs can lead to better model convergence but also risks overfitting. It does not specifically target the trade-off between precision and recall to reduce false positives.

D. Change the value of the predictortype hyperparameter to regressor: This changes the problem from classification (predicting a category) to regression (predicting a continuous value), making the concept of false positives inapplicable.

1. Amazon SageMaker Developer Guide: The Linear Learner documentation details hyperparameters for optimizing the model. For classification

the model can be optimized for metrics like precision. The guide states

"For the binaryclassifier predictor type

you can also optimize the model for a specific metric (precision

recall

fbeta) by setting the binaryclassifiermodelselectioncriteria hyperparameter." While the question specifies multiclassclassifier

the principle of optimizing for precision to reduce false positives is a core

intended mechanism presented in the algorithm's capabilities.

Source: Amazon SageMaker Developer Guide

"Linear Learner Algorithm

" section on "Optimization Hyperparameters."

2. Stanford University CS229 Course Notes: Course materials on machine learning evaluation metrics define precision and its relationship to false positives. "Precision measures

of all the examples the classifier labeled as positive

what fraction were actually positive... A model with high precision will have a low number of false positives."

Source: Stanford University

CS229 Machine Learning

"Evaluation Metrics" by Andrew Ng. (Specifically

discussions on Precision-Recall).

3. Peer-Reviewed Publication: In "The Elements of Statistical Learning

" the concepts of Type I (false positive) and Type II (false negative) errors are fundamental. Optimizing for precision is a standard strategy to control the rate of Type I errors.

Source: Hastie

T.

Tibshirani

R.

& Friedman

J. (2009). The Elements of Statistical Learning: Data Mining

Inference

and Prediction. Springer. (See Chapter 7

"Model Assessment and Selection").

The problem describes a supervised classification task (fraud detection) using tabular data with significant feature interdependencies and class imbalance. The LightGBM algorithm, a gradient boosting framework using tree-based models, is the most appropriate choice. Tree-based ensembles are highly effective at capturing complex, non-linear interactions between features, which is a key requirement ("many of the features have interdependencies"). The Amazon SageMaker implementation of LightGBM also provides specific hyperparameters, such as isunbalance or scaleposweight, to directly address the class imbalance problem, making it superior to other options for this specific scenario.

B. Linear learner: This algorithm assumes a linear relationship between features and the target and does not inherently capture complex feature interdependencies, which is a stated problem in the scenario.

C. К-means clustering: This is an unsupervised learning algorithm used for grouping data points. It is not suitable for a supervised classification task like fraud detection where labeled data is available.

D. Neural Topic Model (NTM): This is an unsupervised algorithm designed for topic modeling on text documents. It is completely irrelevant for a classification task on structured, tabular financial data.

1. LightGBM Algorithm Documentation: "LightGBM is a gradient boosting framework that uses tree-based learning algorithms... It is often used for regression and classification problems with tabular datasets... To handle an imbalanced dataset

use the isunbalance or scaleposweight parameter."

Source: AWS. (2023). Use the SageMaker LightGBM Algorithm. Amazon SageMaker Developer Guide. https://docs.aws.amazon.com/sagemaker/latest/dg/lightgbm.html

2. Linear Learner Algorithm Documentation: "The Amazon SageMaker Linear Learner algorithm provides a solution for both classification and regression problems." While it can handle classification

it does not inherently model the complex feature interactions that tree-based models like LightGBM excel at.

Source: AWS. (2023). Linear Learner Algorithm. Amazon SageMaker Developer Guide. https://docs.aws.amazon.com/sagemaker/latest/dg/linear-learner.html

3. Tree-based Models for Feature Interactions: Academic literature confirms that tree-based methods are adept at modeling interactions. "Decision trees can capture non-linear interaction effects between features... an advantage over parametric models like linear regression."

Source: Hastie

T.

Tibshirani

R.

& Friedman

J. (2009). The Elements of Statistical Learning: Data Mining

Inference

and Prediction. Springer. (Chapter 9.2.4

"Trees and Rules").

The task of predicting apartment prices is a regression problem, as the target variable (price) is a continuous numerical value. Mean Absolute Error (MAE) is a standard metric used to evaluate the performance of regression models. MAE calculates the average of the absolute differences between the predicted values and the actual values. This provides a straightforward, interpretable measure of the model's average prediction error in the same units as the target variable (e.g., dollars), making it highly suitable for this scenario.

A. Accuracy: This metric is used for classification tasks to measure the proportion of correct predictions. It is unsuitable for evaluating continuous value predictions like price.

B. Area Under the ROC Curve (AUC): AUC is a performance metric for binary classification models. It evaluates a model's ability to distinguish between positive and negative classes.

C. F1 score: The F1 score is the harmonic mean of precision and recall. It is a metric used to evaluate classification models, particularly in cases of class imbalance.

1. Amazon SageMaker Developer Guide: "For regression tasks

Amazon SageMaker automatically reports the mean squared error (MSE)

root mean square error (RMSE)

mean absolute error (MAE)

mean absolute percentage error (MAPE)

and the R-squared (R2) score." This source explicitly lists MAE as a primary metric for regression tasks within the AWS ecosystem.

Source: Amazon Web Services. (2024). Metrics for regression. Amazon SageMaker Developer Guide. Retrieved from https://docs.aws.amazon.com/sagemaker/latest/dg/notebooks-auto-ml-v2-metrics.html#notebooks-auto-ml-v2-regression

2. Stanford University Courseware (CS229): In the context of linear regression

evaluation involves minimizing a cost function. While Mean Squared Error (MSE) is often used

Mean Absolute Error (MAE) is a common alternative

particularly when robustness to outliers is desired. Both are fundamental regression loss functions and evaluation metrics. Classification metrics like Accuracy

AUC

and F1 are discussed separately for logistic regression and other classification algorithms.

Source: Ng

A. (2023). CS229: Machine Learning - Lecture Notes 1. Stanford University. pp. 4-5. Retrieved from https://cs229.stanford.edu/mainnotes.pdf

Amazon SageMaker Data Wrangler is specifically designed to simplify and accelerate data preparation for machine learning using a visual, low-code interface. For the task of dropping unnecessary columns from a 50 MB file, Data Wrangler offers the most direct and lowest-effort solution. An ML engineer can import the dataset into a Data Wrangler flow and apply the built-in "Drop column" transform through a point-and-click user interface without writing any code. This visual approach is significantly faster and requires less technical overhead than scripting a solution or configuring a separate processing cluster.

A: Downloading the file for local processing is inefficient, involves manual data movement, and does not leverage the integrated, scalable AWS environment.

B: Using Amazon EMR is intended for large-scale big data processing. Configuring an EMR cluster for a single 50 MB file is excessive and represents the highest effort.

C: A SageMaker processing job is a valid method but requires writing, testing, and managing a custom Python script, which constitutes more effort than using a pre-built visual transform in Data Wrangler.

1. SageMaker Data Wrangler: "Amazon SageMaker Data Wrangler reduces the time it takes to aggregate and prepare data for machine learning (ML) from weeks to minutes... You can simplify your data preparation and feature engineering process and complete each step of the data preparation workflow

including data selection

cleansing

exploration

and visualization from a single visual interface."

Source: AWS Documentation

"Prepare ML Data with Amazon SageMaker Data Wrangler

" Introduction. https://docs.aws.amazon.com/sagemaker/latest/dg/data-wrangler.html

2. SageMaker Processing: "With Amazon SageMaker Processing

you can use a simplified

managed experience to run your data processing workloads... You provide a script that SageMaker runs as a processing job."

Source: AWS Documentation

"Process Data with Amazon SageMaker Processing

" Introduction. https://docs.aws.amazon.com/sagemaker/latest/dg/processing-job.html

3. Amazon EMR: "Amazon EMR is the industry-leading cloud big data solution for petabyte-scale data processing

interactive analytics

and machine learning using open-source frameworks such as Apache Spark

Apache Hive

and Presto."

Source: AWS Documentation

"What is Amazon EMR?

" Introduction. https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-what-is-emr.html

AWS Secrets Manager is the purpose-built AWS service for managing the lifecycle of secrets, including API tokens. It provides a native framework for automatic rotation. For secrets that AWS cannot rotate natively, such as third-party API tokens, Secrets Manager integrates directly with AWS Lambda. You can configure Secrets Manager to invoke a specific Lambda function on a schedule (e.g., every 90 days) to handle the custom logic required to connect to the third-party API, generate a new token, and update the secret value in Secrets Manager.

B: While AWS Systems Manager Parameter Store can store secrets, it lacks the built-in, managed rotation framework that Secrets Manager offers, making it a less direct and less capable solution for this specific requirement.

C: AWS Key Management Service (AWS KMS) is used to create and manage cryptographic keys for encryption, not to store or rotate application-level secrets like API tokens.

D: Similar to option C, AWS KMS manages encryption keys. The key type (AWS owned vs. AWS managed) is irrelevant as the service's fundamental purpose is not secret storage or rotation.

1. AWS Secrets Manager Documentation: "For other types of secrets

including API keys... you can create a Lambda function to define how Secrets Manager rotates the secret. This is called a custom rotation strategy."

Source: AWS Secrets Manager User Guide

"Rotate AWS Secrets Manager secrets".

2. AWS Lambda and Secrets Manager Integration: "Secrets Manager uses a Lambda function to rotate a secret... Secrets Manager calls your Lambda function at least twice during a rotation."

Source: AWS Secrets Manager User Guide

"How rotation works".

3. AWS KMS Service Definition: "AWS Key Management Service (AWS KMS) is a managed service that makes it easy for you to create and control the cryptographic keys that you use to protect your data."

Source: AWS Key Management Service Developer Guide

"What is AWS Key Management Service?".

Apache Parquet is a columnar storage file format optimized for performance in large-scale data processing. For complex data structures, Parquet's columnar layout significantly minimizes processing time by allowing Amazon SageMaker to read only the specific columns needed for analysis, rather than scanning entire rows. This drastically reduces I/O operations and accelerates data retrieval. Its support for efficient, splittable compression further enhances performance in distributed environments like those used by AWS services.

A. CSV files compressed with Snappy: CSV is a row-based format and is inefficient for querying specific columns from complex data, which increases I/O and processing time despite the fast Snappy compression.

B. JSON objects in JSONL format: JSONL is also row-based. While it handles complex structures, processing requires reading and parsing entire JSON objects, making it slower than Parquet's selective column reads.

C. JSON files compressed with gzip: A single, large gzip-compressed file is not splittable. This prevents parallel processing, creating a bottleneck and significantly increasing processing time for large datasets.

1. AWS Big Data Blog: In a post on performance tuning

AWS states

"We recommend you use Apache Parquet or Apache ORC... Parquet is a columnar data format that is optimized for fast retrieval of data." This directly supports using Parquet to minimize processing time.

Source: AWS Big Data Blog

"Top 10 performance tuning tips for Amazon Athena

" Tip #2. (URL: https://aws.amazon.com/blogs/big-data/top-10-performance-tuning-tips-for-amazon-athena/)

2. Apache Parquet Official Documentation: The official documentation describes Parquet as "a columnar storage format... designed for efficient as well as performant flat columnar storage format of data compared to row based files like CSV or TSV files."

Source: The Apache Software Foundation

"Apache Parquet Documentation." (URL: https://parquet.apache.org/docs/)

3. Amazon SageMaker Developer Guide: The guide explains that for efficient data processing

especially with services like AWS Glue

converting data to Apache Parquet is a best practice. This reduces the amount of data scanned and improves performance.

Source: Amazon SageMaker Developer Guide

"Input and Output Data." (URL: https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-training.html - Note: While this link discusses data formats for training

the principle of Parquet's performance advantage is a core concept across AWS analytics and ML services).

The requirement is to create an on-demand workflow to monitor bias drift for a deployed model. Amazon SageMaker Clarify is the specific service designed to detect bias in models and data, both before training and after deployment (bias drift). A SageMaker Clarify job can be run as a processing job. To create an "on-demand workflow" that can be triggered by a web application, an AWS Lambda function is the ideal serverless compute service. The Lambda function can be invoked via an API call from the application, and it will, in turn, initiate the SageMaker Clarify processing job to perform the bias drift analysis. This architecture directly and precisely meets all stated requirements.

B: This is imprecise. A Lambda function would start a SageMaker job that uses the image; it does not "pull" the image itself. More importantly, it names a generic monitor container, whereas SageMaker Clarify is the specific tool for bias.

C: AWS Glue Data Quality is used to monitor the quality of data within data pipelines and data lakes (e.g., null counts, uniqueness), not for detecting statistical bias in machine learning models.

D: SageMaker notebooks are for interactive development and experimentation. They are not a suitable component for an automated, on-demand production workflow triggered by an application.

1. Amazon SageMaker Clarify Documentation: "After a model is deployed

you can use SageMaker Clarify to detect bias drift and feature attribution drift... You can run a SageMaker Clarify job on demand by using the Amazon SageMaker Python SDK

or you can set up a scheduled job with SageMaker Model Monitor." This confirms Clarify is the correct tool and can be run on-demand.

Source: AWS Documentation

"Detect Post-Training Bias Drift with Amazon SageMaker Clarify".

2. Amazon SageMaker Model Monitor Documentation: "SageMaker Clarify is integrated with SageMaker Model Monitor to help you monitor your models for bias drift over time... You can start a baselining job to generate constraints and then set up a scheduled monitoring job. You can also run a single (on-demand) monitoring job." This shows the integration and the on-demand capability.

Source: AWS Documentation

"Bias Drift".

3. AWS Architecture for MLOps: Architectural patterns for MLOps frequently use AWS Lambda functions to orchestrate various components of the ML lifecycle

including triggering SageMaker Processing Jobs

in response to events or API calls. This validates the use of Lambda as the trigger for the on-demand workflow.

Source: AWS Whitepaper

"MLOps: Continuous delivery for machine learning on AWS". (See architectural diagrams showing Lambda functions orchestrating SageMaker jobs).

The most operationally efficient solution is to use AWS Lake Formation's native capabilities for fine-grained access control. Lake Formation Tag-Based Access Control (LF-TBAC) is designed for this exact scenario. By assigning LF-Tags to Data Catalog resources (which represent the data in Amazon S3) based on campaign, and then granting permissions on those tags to the corresponding ML engineers' roles, the company can manage access centrally. This single policy framework is enforced for users accessing data through both Amazon Athena and directly in S3 (via Lake Formation's credential vending), meeting all requirements in a unified and scalable manner.

A: This approach is incomplete as it only addresses access through Athena and the AWS Glue Data Catalog, but not direct S3 bucket access.

B: This is an overly complex, custom-built solution that is not operationally efficient. It also only solves for S3 access, ignoring the Athena requirement.

D: This method only controls direct S3 bucket access and does not manage permissions for queries run through Athena. It also becomes difficult to manage at scale.

1. AWS Lake Formation Developer Guide

"Tag-based access control with AWS Lake Formation": This guide explicitly describes using LF-Tags to manage permissions at scale. It states

"You can define LF-tags and assign them to Data Catalog resources... You then grant permissions on those LF-tags to principals. Lake Formation allows operations on the resources if the principal has permissions on the LF-tags for that resource." This directly supports the methodology in option C.

URL: https://docs.aws.amazon.com/lake-formation/latest/dg/tag-based-access-control.html

2. AWS Lake Formation Developer Guide

"How data access control works in Lake Formation": This document explains that Lake Formation provides a single place to manage access controls for data in an S3 data lake

which are then enforced across multiple services like Amazon Athena. This highlights the operational efficiency of a centralized model.

URL: https://docs.aws.amazon.com/lake-formation/latest/dg/how-it-works-access-control.html

3. AWS Big Data Blog

"Govern your data lake centrally with AWS Lake Formation tag-based access control": This official blog post details the exact use case of using tags to grant permissions to different groups (like ML engineers for specific campaigns)

confirming it as the intended and most efficient pattern.

URL: https://aws.amazon.com/blogs/big-data/govern-your-data-lake-centrally-with-aws-lake-formation-tag-based-access-control/

The selection of each technique aligns with standard machine learning data preprocessing practices.

City (name) is a nominal categorical feature. One-hot encoding is the appropriate method to convert these non-ordinal categories into a numerical format that an ML model can process, creating a separate binary feature for each city.

Type_year is a composite feature containing two distinct pieces of information: the home type and its construction year. Feature splitting is necessary to separate these into two independent, more useful features (type_of_home and year_built).

Size of the building is a continuous numerical feature. In real estate, features like size and price are often right-skewed. A Logarithmic transformation is used to handle this skewness, making the distribution more symmetric and helping to meet the assumptions of certain models like linear regression.

One-hot encoding: Géron, A. (2022). Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow (3rd ed.). O'Reilly Media, Inc. In Chapter 2, "Handling Text and Categorical Attributes" (pp. 71-73), the author explains and demonstrates the use of OneHotEncoder for nominal categorical features, which is the exact case for the City (name) feature.

Feature splitting: Stanford University. (n.d.). CS229: Machine Learning - Unsupervised Learning, K-means clustering. In the context of feature engineering, it's a common practice to decompose features. While not a formal named method, splitting a feature like Type_year into Type and Year is a fundamental feature engineering step to create more granular and meaningful inputs for the model. This is conceptually similar to extracting the month or day from a date feature.

Logarithmic transformation: University of California, Berkeley. (n.d.). Data 8: Foundations of Data Science, Chapter 15.2: Transformations. The course material explains that applying transformations like the logarithm is useful when dealing with skewed distributions or non-linear relationships, which is highly characteristic of housing size and price data. Applying a log transform can help linearize the relationship and stabilize variance.

• Embedding – converts words or passages into high-dimensional vectors that capture semantic meaning, letting the model compare related concepts numerically.

• Retrieval Augmented Generation (RAG) – a pattern that first retrieves relevant external documents and then supplies them as additional context to the LLM, enriching its generated answer with up-to-date or domain-specific knowledge.

• Temperature – a decoding parameter that scales the soft-max distribution during generation; higher values increase randomness/creativity while lower values make output more deterministic.

1. AWS

“Use Amazon Titan Embeddings to create vector embeddings for text

” Amazon Bedrock Developer Guide

§ Model overview. https://docs.aws.amazon.com/bedrock/latest/userguide/embeddings-titan.html

2. AWS

“Build a Retrieval-Augmented Generation (RAG) system with Amazon Bedrock

” AWS GenAI Workshop

Steps 1-2 (concept definition). https://catalog.workshops.aws/generative-ai/en-US/rag-bedrock

3. Stanford University

CS224N: Natural Language Processing with Deep Learning

Lecture 10 “Language Modeling & Generation

” slide 32 (temperature parameter in softmax sampling). https://web.stanford.edu/class/cs224n/slides/cs224n-2023-lecture10.pdf