AIF-C01 Practice Questions

Partial dependence plots (PDPs) are a primary tool for machine learning explainability. They illustrate the marginal effect of one or two features on the predicted outcome of a model, holding other features constant. For stakeholders, this provides a clear, visual representation of how key business drivers (e.g., price, marketing spend) influence the model's demand forecasts. This directly addresses the need for transparency into the model's logic and explainability of its predictions in an accessible format, fostering trust and better decision-making.

1. Molnar

C. (2022). Interpretable Machine Learning: A Guide for Making Black Box Models Explainable (2nd ed.). In Chapter 5.2

"Partial Dependence Plot (PDP)

" the author states

"The partial dependence plot (PDP or PD plot) shows the marginal effect one or two features have on the predicted outcome of a machine learning model... A partial dependence plot can show whether the relationship between the target and a feature is linear

monotonic or more complex." This directly supports using PDPs for explainability.

2. Friedman

J. H. (2001). Greedy Function Approximation: A Gradient Boosting Machine. The Annals of Statistics

29(5)

1189–1232. In Section 8.2

"Interpretation

" Friedman introduces partial dependence plots as a method to understand the effect of specific variables on the model's predictions

stating they are "useful for gaining an understanding of the dependence of the function f(x) on the joint values of the corresponding subset of variables." (p. 1217). DOI: https://doi.org/10.1214/aos/1013203451

3. University of California

Berkeley. (n.d.). CS 189/289A: Introduction to Machine Learning

Lecture 18: Interpretability. Course materials discuss PDPs as a model-agnostic method for understanding feature effects

highlighting their role in explaining what a model has learned to a human.

Object detection is a computer vision technology that deals with identifying and locating objects within an image or video. Specifically, it draws bounding boxes around detected objects (identification/localization) and assigns a class label to them (categorization). This directly addresses the requirement to automatically find where the animals are in the photos and determine what kind of animals they are. The process is automated, fulfilling the "without manual human effort" constraint once the model is trained.

1. Stanford University CS231n Course Notes. In the "Object Detection" section

it is defined as the task of classification and localization of objects. The notes state

"we want to classify each object and localize it using a bounding box."

Source: Stanford CS231n: Convolutional Neural Networks for Visual Recognition

Spring 2022

Lecture 11: Detection and Segmentation

Slides 4-6.

2. Official Vendor Documentation (Amazon Web Services). The Amazon Rekognition documentation describes its "Object and Scene Detection" feature as being able to "identify thousands of objects such as cars

furniture

or animals

and scenes such as a beach or city."

Source: AWS Documentation

Amazon Rekognition Developer Guide

"Detecting objects in images

" Section: "Objects and scenes detected in images."

3. Peer-Reviewed Academic Publication. A comprehensive survey on object detection defines the task as determining "where objects are located in a given image (object localization) and which category each object belongs to (object classification)."

Source: Zou

Z.

Shi

Z.

Guo

Y.

& Ye

J. (2019). Object Detection in 20 Years: A Survey. arXiv preprint arXiv:1905.05055

Section 1

Paragraph 1. (Note: While an arXiv preprint

this survey is widely cited and foundational in the field

reflecting academic consensus).

Building a responsible AI solution, especially in a sensitive domain like lending, hinges on the core principles of fairness and transparency. Detecting imbalances or disparities in the training data is a critical first step to mitigate bias, as models learn and amplify patterns present in the data. This directly addresses the goal of minimizing negative effects on customers.

Furthermore, evaluating the model's behavior to provide transparency is essential for accountability. Stakeholders, including customers, regulators, and internal auditors, must be able to understand how decisions are made. This fosters trust and allows the company to demonstrate that its criteria are being applied fairly and responsibly, in line with ethical and legal standards.

1. Official Vendor Documentation: Amazon SageMaker Developer Guide

"Fairness and Explainability with Amazon SageMaker Clarify." This guide details how to "measure imbalances in the training data before training a model" (supporting option A) and how to "explain the behavior of a model as a whole" using techniques like SHAP to provide transparency (supporting option C). See the sections on "Pre-training Bias Metrics" and "Explainability for Models."

2. University Courseware: Stanford University

Human-Centered Artificial Intelligence (HAI)

"AI Index Report 2023." Chapter 4

"Technical AI Ethics

" discusses the growing importance and implementation of fairness and transparency evaluations. It notes that "fairness... is concerned with ensuring that ML systems do not disproportionately harm (or benefit) different groups of people

" which begins with data analysis (supporting option A).

3. Peer-reviewed Academic Publication: Barocas

S.

Hardt

M.

& Narayanan

A. (2019). Fairness and Machine Learning: Limitations and Opportunities. Chapter 1

"Introduction

" discusses how biased datasets lead to discriminatory outcomes and emphasizes the need for transparency to audit and understand model behavior. This foundational text establishes the direct link between data inspection (A) and model evaluation for transparency (C) as central to fairness. (Available via fairmlbook.org

an online resource associated with the authors' work).

In generative AI, particularly in Large Language Models (LLMs), a token is the fundamental atomic unit of text that the model processes. Raw text input is first segmented into a sequence of these tokens through a process called tokenization. A token can represent a whole word (e.g., "apple"), a subword (e.g., the "token" and "ization" in "tokenization"), or even a single character or punctuation mark. This approach allows the model to handle a vast and open vocabulary efficiently, including rare words, misspellings, and compound words, by breaking them down into known smaller pieces. The model's input and output are fundamentally sequences of these tokens.

1. OpenAI Documentation: In the "Platform" documentation

under the section "Introduction to tokens

" it is stated: "Large language models process text by breaking it down into smaller units

called tokens. Tokens can be words

chunks of words

or single characters." This directly supports that tokens are the basic units of input/output.

Source: OpenAI. (n.d.). What are tokens?. OpenAI Platform Documentation. Retrieved from the official OpenAI website's section on tokens.

2. Stanford University Courseware (CS224N): The lecture notes for "CS224N: Natural Language Processing with Deep Learning" describe the initial step of any NLP pipeline as tokenization. It defines this process as "segmenting text into units such as words

punctuation

etc." These units are referred to as tokens.

Source: Manning

C. D.

& Jurafsky

D. (2023). Lecture 2: Word Vectors and Word Senses. Stanford CS224N: NLP with Deep Learning

Winter 2023.

3. Google Cloud AI Documentation: In the introduction to Large Language Models

the documentation explains: "LLMs break down text into smaller units called tokens for processing. A token can be a word

a subword

a character

or a group of characters."

Source: Google Cloud. (n.d.). Large language models introduction. Vertex AI Documentation.

4. Hugging Face Courseware: The official Hugging Face NLP course explains tokenization in detail: "Splitting the input into words

subwords

or symbols that are called tokens is called tokenization... The tokenizer has to convert our text inputs into numerical data the model can make sense of."

Source: Hugging Face. (n.d.). Chapter 2: Behind the pipeline - What is a Tokenizer?. Hugging Face NLP Course.

The issue described is a direct result of sampling bias. This type of bias occurs when the data used to train a model is not a representative sample of the real-world population or the environment where the model will be deployed. In this case, the security camera footage used for training likely contained a disproportionately high number of images of people from a specific ethnic group labeled as committing theft. Consequently, the model learned a spurious correlation between that group's features and the act of theft, leading to biased and unfair predictions.

1. Google. (n.d.). Responsible AI Practices. In the "Identify types of bias" section

Sampling bias is defined: "occurs when a dataset doesn't reflect the realities of the environment in which a model will run. This can happen if data is collected in a way that over-represents or under-represents certain groups or characteristics." This directly aligns with the scenario.

2. Barocas

S.

Hardt

M.

& Narayanan

A. (2019). Fairness and machine learning: Limitations and opportunities. In Chapter 2

"Sources of Bias

" the authors detail how the composition of a dataset

if not representative of the broader population (i.e.

sampling bias)

is a primary source of allocative and quality-of-service harms

where a system unfairly disadvantages certain groups.

3. Amazon Web Services. (2023). Amazon SageMaker Developer Guide. In the section "Fairness and Explainability with SageMaker Clarify

" the documentation discusses pre-training bias metrics. It explains how imbalances in the training data

where certain facets or groups are under- or over-represented

must be identified and addressed to prevent biased model outcomes. This describes the core problem of sampling bias.

The context window, also referred to as context length, is the specific parameter that defines the maximum number of tokens (a combination of the input prompt and the model's generated output) that a model can process at one time. It directly determines how much information can be fed into a single prompt. For a task like sentiment analysis on a large document, a model with a larger context window is necessary to ensure the entire text can be analyzed in one go, preserving the full context for an accurate assessment.

1. Amazon Web Services (AWS) Documentation: The Amazon Bedrock User Guide explicitly lists the "Max context length" or "Max tokens" for each foundation model. This value represents the context window. For example

for Anthropic's Claude models

it states

"Maximum context length (input + output): 200

000 tokens."

Source: Amazon Bedrock User Guide

"Base models in Amazon Bedrock

" section on "Model attributes."

2. University Courseware: Stanford University's course CS324: Large Language Models defines context length as a core property of a language model. Lecture slides and notes explain that the "context length" (or window) is the finite number of tokens that a model can take as input.

Source: Stanford University

CS324: Large Language Models

Winter 2023

Lecture 2: "Language Model Properties."

3. Academic Publication: The foundational paper on the Transformer architecture

which underpins most modern LLMs

establishes the concept of processing sequences of a fixed length

which is the basis for the context window.

Source: Vaswani

A.

et al. (2017). "Attention Is All You Need." Advances in Neural Information Processing Systems 30 (NIPS 2017). Section 3.1

"Encoder and Decoder Stacks

" describes processing input sequences of symbol representations. (DOI: https://doi.org/10.48550/arXiv.1706.03762)

Ongoing pre-training, also known as continued pre-training or domain-adaptive pre-training, involves further training a general foundation model on a large corpus of domain-specific, unlabeled data. This process adapts the model's internal representations, vocabulary, and understanding to the nuances of the target domain (e.g., legal, medical, or financial text). By aligning the model with the specific data distribution of the target domain before task-specific fine-tuning, it achieves a better starting point, which consistently leads to improved performance and accuracy on downstream tasks within that domain.

1. Gururangan

S.

Marasović

A.

Swayamdipta

S.

Lo

K.

Beltagy

I.

Downey

D.

& Smith

N. A. (2020). Don't Stop Pretraining: Adapt Language Models to Domains and Tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 8342–8356). The paper's central thesis

summarized in the abstract and demonstrated in Section 4 ("Results")

is that "pretraining on data from the target domain (domain-adaptive pretraining) leads to performance gains." (DOI: https://doi.org/10.18653/v1/2020.acl-main.740)

2. Bommasani

R.

et al. (2021). On the Opportunities and Risks of Foundation Models. Stanford University Center for Research on Foundation Models (CRFM). In Section 4.2.1 ("Adaptation")

the report discusses methods for adapting FMs

stating

"The goal of adaptation is to steer the behavior of a foundation model to better perform a desired downstream task." Continued pre-training is a key method to achieve this performance improvement. (Page 63).

3. Stanford University. (2023). CS224N: NLP with Deep Learning

Winter 2023 Lecture 12: Pretraining and Transfer Learning. The lecture notes explain that continued pretraining on a domain-specific corpus before fine-tuning helps the model learn the specific statistics and vocabulary of that domain

which improves final task performance.

The core requirements are high accuracy and minimizing incorrect annotations for an image dataset. Amazon SageMaker Ground Truth Plus is a fully managed data labeling service designed specifically for this purpose. It provides an expert workforce and employs multi-step validation workflows, including human-in-the-loop (HITL) reviews, to ensure the creation of high-quality, accurately labeled training datasets. This directly addresses the need to minimize annotation errors and achieve high accuracy, making it the most suitable solution.

1. Amazon SageMaker Developer Guide: "Amazon SageMaker Ground Truth Plus is a turnkey data labeling service that uses an expert workforce and machine learning (ML) to create high-quality training datasets. ... Ground Truth Plus uses a multi-step labeling workflow that includes pre-labeling using ML and human verification to increase the quality of the output data labels." (See the "Amazon SageMaker Ground Truth Plus" section).

2. AWS Official Documentation - Amazon SageMaker Ground Truth Plus: "Amazon SageMaker Ground Truth Plus is a fully managed data labeling service that enables you to create high-quality training datasets without having to build labeling applications or manage the labeling workforce on your own." (Source: Amazon SageMaker Ground Truth Plus product page).

3. AWS Official Documentation - Amazon Rekognition: "Amazon Rekognition makes it easy to add image and video analysis to your applications using proven

highly scalable

deep learning technology that requires no machine learning expertise to use." (Source: Amazon Rekognition product page

which describes its function as analysis

not guaranteed-accuracy labeling).

Few-shot learning for intent detection involves providing a Large Language Model (LLM) with a small number of examples (the "shots") within the prompt to guide its predictions. The specific task is to identify the user's intent from their message. Therefore, the examples must demonstrate this exact mapping. Each example should consist of a sample user message (the input) and its corresponding correct user intent (the desired output label). This allows the model to understand the classification task and apply the pattern to new, unseen user messages.

1. Amazon Bedrock User Guide: In the section on "Prompt engineering guidelines

" the concept of few-shot prompting is explained. It states that for classification tasks

providing examples of input and their corresponding labels improves accuracy. For intent detection

the input is the user message and the label is the intent. (Source: Amazon Bedrock User Guide

"Prompting

" "Prompt engineering guidelines").

2. Jurafsky

D.

& Martin

J. H. (2023). Speech and Language Processing (3rd ed. draft). In the chapter on "Dialogue Systems and Chatbots

" Section 16.2 discusses the components of a task-oriented dialogue system. The Natural Language Understanding (NLU) component's primary task is intent determination

which is framed as a classification problem. The training data for this classifier consists of user utterances

each labeled with an intent.

3. University of Washington

CSE 517: Natural Language Processing (Winter 2023)

Lecture 19: Dialogue. The lecture notes describe the NLU module in dialogue systems

where the goal is to map a user utterance to a semantic representation

such as a dialogue act or intent. The training data for this module is explicitly defined as pairs of utterances and their corresponding intents (e.g.

(utterance

intent)).

Amazon QuickSight is the AWS business intelligence (BI) service designed for creating interactive dashboards and data visualizations. The integration of Amazon Q within QuickSight specifically enables users to ask questions in natural language (e.g., "Show total sales for top products in the last 12 months") and have the service automatically generate the relevant graphs and visuals. This capability directly addresses the company's requirement to automate the generation of graphs from their sales data without needing to manually build reports.

1. Amazon QuickSight User Guide: "With Amazon Q in Amazon QuickSight

business users can ask questions about their data in natural language and receive accurate answers with relevant visualizations

helping them to understand their data better."

Source: AWS Documentation

Amazon QuickSight User Guide

"Building visuals with Amazon Q in Amazon QuickSight".

2. Amazon Q Developer Documentation: "Amazon Q Developer is a generative AI-powered assistant that can help you with a wide variety of tasks throughout your software development lifecycle (SDLC)."

Source: AWS Documentation

Amazon Q Developer User Guide

"What is Amazon Q Developer?".

3. Amazon Q in AWS Chatbot: "Amazon Q in AWS Chatbot helps you get answers to your questions about AWS services

architecture

and best practices... You can use Amazon Q to troubleshoot issues

get guidance on how to configure your AWS resources

and more."

Source: AWS Documentation

AWS Chatbot Administrator Guide

"Using Amazon Q in AWS Chatbot".

4. Carnegie Mellon University

Cloud Computing Courseware: Course materials on cloud-based data analytics often distinguish between developer tools and BI platforms. They categorize services like QuickSight as BI tools for end-user reporting and visualization

separate from developer-centric AI assistants.

Source: Carnegie Mellon University

15-319/619 Cloud Computing

Fall 2023

Lecture 19: Data Analytics & Big Data. (Illustrates the distinct purpose of BI services).