CSPAI Practice Questions

The defining characteristic of a Retrieval-Augmented Generation (RAG) system is its two-stage process. The initial and most crucial step is "Retrieval," where the system queries a specialized knowledge base, typically a vector database, to find information chunks most relevant to the user's prompt. This retrieved context is then passed to the "Generation" stage. By grounding the language model with this specific, pre-vetted information, the RAG architecture ensures the final response is contextually accurate, relevant, and less prone to hallucination. The retrieval step is the fundamental mechanism that provides the factual basis for the subsequent generation.

A. Leveraging diverse data sources is a prerequisite for building a robust knowledge base, not the active step that ensures relevance for a specific query.

B. Integrating advanced search algorithms describes how retrieval is performed, but the act of retrieval itself (Option D) is the fundamental step in the process.

C. Utilizing feedback mechanisms is a method for long-term system improvement, not the core, real-time process for ensuring a single response's accuracy.

1. Lewis

P.

Perez

E.

Piktus

A.

Petroni

F.

Karpukhin

V.

Goyal

N.

Küttler

H.

Lewis

M.

Yih

W.

Rocktäschel

T.

Riedel

S.

& Kiela

D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Section 2

"Methods

" describes the core architecture where a retriever first fetches relevant documents that are then conditioned upon by a generator.

2. Gao

Y.

Xiong

Y.

Gao

X.

Jia

K.

Pan

J.

Bi

Y.

Dai

Y.

Sun

J.

& Wang

H. (2023). Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv preprint arXiv:2312.10997. Section 2.1

"The General RAG Paradigm

" explicitly states

"The process begins with the retrieval phase

where the system... retrieves a set of relevant documents from an external knowledge source."

3. Manning

C. D. (2022). CS224N: Natural Language Processing with Deep Learning - Lecture 12: Retrieval and Question Answering. Stanford University. The lecture notes detail how retrieval-based models first use a retriever component to find relevant passages from a corpus before a reader/generator model processes them to formulate an answer.

ISO/IEC 42001:2023 mandates a structured and continuous AI risk management process. Clause 8 requires an organization to establish and implement an AI risk assessment and treatment process. This involves systematically identifying risks associated with the AI system's lifecycle, analyzing their potential consequences and likelihood, and evaluating them against established risk criteria. Based on this evaluation, the organization must then select and implement appropriate risk treatment plans to modify these risks. This comprehensive approach ensures that AI-related risks are managed proactively and effectively within the organization's AI Management System (AIMS).

B. Risk acceptance is a formal decision based on defined criteria; simply "ignoring" risks, even low-level ones, is not compliant with the standard's systematic process.

C. The organization's management is ultimately accountable for risk management; this core function cannot be fully delegated to external auditors, who serve an assessment role.

D. The standard requires risk management across the entire AI system lifecycle, from conception and design through deployment and decommissioning, not just post-deployment.

1. ISO/IEC 42001:2023

"Information technology — Artificial intelligence — Management system":

Clause 8.2 (AI risk assessment) specifies the requirements to "identify

analyse and evaluate AI risks."

Clause 8.3 (AI risk treatment) details the process for selecting and implementing options for addressing the assessed AI risks

which forms the basis of the treatment plan.

2. British Standards Institution (BSI)

"AI Management System - A guide to ISO/IEC 42001" (2023):

Section on "Clause 8: Operation" (Page 12) explains that the standard requires organizations to plan

implement

and control the processes needed to meet AIMS requirements

which centrally involves the AI risk assessment and AI risk treatment processes described in Clauses 8.2 and 8.3.

3. Ali

M. Y.

& Toumi

N. (2024). "A Comprehensive Overview of the ISO/IEC 42001:2023 Standard for Artificial Intelligence Management Systems." Journal of ICT Standardization

12(1)

1-24.

Section 4.3

"Risk Management in AIMS

" analyzes Clause 8

stating

"The core operational aspect of ISO 42001 is the risk management framework... It mandates a cyclical process of risk identification

analysis

evaluation

and treatment

ensuring that potential negative impacts of AI systems are systematically addressed." (DOI will be available upon final publication; citation reflects typical academic analysis of new standards).

The term 'As-is' in software engineering and AI model deployment refers to the integration of a component without any alterations to its core structure or pre-trained state. In the context of an LLM, this means embedding the model directly into an application and utilizing its existing, general-purpose capabilities through API calls. This approach, often facilitated by zero-shot or few-shot prompting, leverages the model's out-of-the-box functionality without undergoing task-specific fine-tuning or customization. The application's frame essentially acts as a user interface for the unmodified, pre-trained LLM.

B. This describes replacing the model entirely, which is the opposite of embedding the current one 'as-is'.

C. This involves customization and fine-tuning, which fundamentally alters the model from its 'as-is' state to suit specific needs.

D. This defines an architectural placement (backend processing) but does not specify the state of the LLM; a customized or an 'as-is' model could be used this way.

1. Zhao

W. X.

et al. (2023). A Survey of Large Language Models. arXiv preprint arXiv:2303.18223. Section 4.2

"Tuning-free Prompting

" describes methods like zero-shot and few-shot learning where the pre-trained model's parameters are not updated

which is synonymous with using the LLM 'as-is'. (https://doi.org/10.48550/arXiv.2303.18223)

2. Yang

J.

et al. (2023). Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond. arXiv preprint arXiv:2304.13712. Section 3

"Paradigms of LLM-based Applications

" contrasts approaches that modify the LLM (e.g.

fine-tuning) with those that treat the LLM as a "frozen" or unchangeable component

which corresponds to an 'as-is' integration. (https://doi.org/10.48550/arXiv.2304.13712)

3. Stanford University. (2023). CS224N: Natural Language Processing with Deep Learning. Lecture 10

"Prompting

Instruction Finetuning

and RLHF." The lecture distinguishes between using a base pre-trained model directly via prompting (the 'as-is' scenario) and adapting the model's weights through fine-tuning for specific tasks.

The primary responsibility of a Chief Experience Officer (CXO) is to manage and enhance the overall customer journey and interaction with an organization. Generative AI-powered chatbots are a direct application tailored for this role. They enable the creation of sophisticated, human-like conversational agents that can provide 24/7 support, answer complex queries, and personalize interactions at scale. This directly impacts key CXO metrics like customer satisfaction, engagement, and support efficiency. The technology allows CXOs to strategically design and deploy improved customer service channels, which is a core function of their position.

A. Developing autonomous vehicles is an engineering and R&D function, typically overseen by a CTO or Chief Product Officer, not a CXO.

B. Automating financial transactions falls under the purview of a CFO or CTO, focusing on financial operations and technology infrastructure, not direct customer experience.

C. Genetic sequencing is a highly specialized scientific and medical application, relevant to a Chief Science Officer or Chief Medical Officer in the healthcare sector.

1. University Courseware/Publication: Stanford University Human-Centered AI (HAI). (2023). How Generative AI Can Augment Human Creativity. Stanford HAI. In the section "Customer Service

" the article discusses how generative AI is used to power chatbots that "can answer customer questions

provide information

and even troubleshoot problems

" which is a direct tool for enhancing customer experience. (Available at: hai.stanford.edu)

2. Academic Publication: Davenport

T. H.

& Mittal

N. (2022). How Generative AI Is Changing Creative Work. MIT Sloan Management Review

64(1). The article details how generative AI can create various forms of content

including "conversational text for chatbots

" directly linking the technology to customer-facing applications that are central to a CXO's role.

3. Academic Publication: Dwivedi

Y. K.

et al. (2023). "So what if ChatGPT wrote it?" Multidisciplinary perspectives on opportunities

challenges and implications of generative conversational AI for research

practice and policy. International Journal of Information Management

71

102642. Section 4.2

"Customer service and support

" explicitly states that generative AI models "can be used to power chatbots and virtual assistants that can provide 24/7 customer support." https://doi.org/10.1016/j.ijinfomgt.2023.102642

Adversarial testing is a security evaluation method specifically designed for AI/ML models. It involves systematically generating and applying "adversarial examples"—inputs intentionally crafted to deceive or disrupt the model. This process simulates various attack scenarios to analyze the model's behavior, identify vulnerabilities, and measure its robustness against malicious manipulations. The core purpose is to understand how the model performs under conditions it was not explicitly trained on, particularly those mimicking hostile actions, which directly aligns with the question's description.

A. input sanitation: This is a defensive measure to clean or validate data before it is processed by the model, not a method for simulating attacks to test behavior.

B. Model firewall: This is a security control that inspects and filters requests to a model, acting as a protective barrier rather than a testing methodology.

C. Prompt injections: This is a specific type of attack on large language models, representing one scenario within the broader practice of adversarial testing.

1. National Institute of Standards and Technology (NIST). (2023). Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2). U.S. Department of Commerce. Section 2.1

"Adversarial Machine Learning

" describes the process of using crafted inputs to cause a model to misbehave

which is the foundation of adversarial testing.

2. Goodfellow

I. J.

Shlens

J.

& Szegedy

C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. Section 1

Paragraph 1

introduces the concept of adversarial examples as a tool to analyze model properties and blind spots. https://doi.org/10.48550/arXiv.1412.6572

3. MIT OpenCourseWare. (2020). 6.S191: Introduction to Deep Learning. Lecture on Deep Learning Security and Privacy. The lecture materials discuss adversarial testing as the primary method for evaluating a model's resilience by simulating attacks with adversarial examples.

AI enhances the retail customer experience through a multi-faceted approach. It leverages data analytics to create highly customized shopping journeys and personalized interactions (A). It also optimizes direct customer touchpoints through automated systems like chatbots for instant support and tailored recommendation engines for relevant product discovery (B). Furthermore, AI improves backend processes by automating tasks and providing data-driven insights, which leads to better service, optimized inventory, and a more seamless overall customer experience (D). These elements work together to create a more efficient, personalized, and responsive retail environment.

C: This describes a poor customer experience. The goal of AI is personalization and contextual awareness, not providing the same generic response to every customer.

1. Guha

A.

Grewal

D.

Kopalle

P. K.

Haenlein

M.

Schneider

M. J.

& Warmath

D. (2021). How artificial intelligence is changing the retail industry. Journal of Retailing

97(1)

1-10. (Discusses personalization

customer service automation

and operational efficiencies as key AI applications in retail). DOI: https://doi.org/10.1016/j.jretai.2021.01.005

2. Davenport

T.

Guha

A.

Grewal

D.

& Bressgott

T. (2020). How artificial intelligence will change the future of marketing. Journal of the Academy of Marketing Science

48(1)

24-42. (See Section: "Personalization and Customization

" pp. 30-32

which details how AI uses data to tailor experiences

supporting options A and B). DOI: https://doi.org/10.1007/s11747-019-00696-0

3. Wisskirchen

G.

et al. (2017). Artificial Intelligence and Robotics and Their Impact on the Workplace. IBA Global Employment Institute. (See Chapter 4

"Retail

" which describes how AI automates repetitive tasks and provides data insights to improve both operations and customer-facing services

supporting option D). Available via research portals like ResearchGate.

Single-task fine-tuning optimizes a model for a narrow data distribution and a specific objective. This high degree of specialization often leads to overfitting on the target task, causing the model to lose some of the general-purpose capabilities acquired during pre-training. Consequently, its ability to generalize to new, unseen tasks is diminished. In contrast, multi-task fine-tuning exposes the model to diverse tasks simultaneously. This process acts as a form of regularization, encouraging the model to learn more robust and transferable representations that are useful across different domains, thereby enhancing its generalization performance.

A. Model version management is an operational (MLOps) challenge applicable to both single and multi-task fine-tuning, not a primary challenge inherent to the single-task method itself.

C. This is generally incorrect. Fine-tuning is valued for its data efficiency. Single-task fine-tuning does not inherently require more data than multi-task fine-tuning to achieve high performance on its specific task.

D. This describes "catastrophic forgetting," which is a valid concern. However, the primary comparative advantage of multi-task learning is its superior generalization to new tasks, making the lack thereof the main challenge for single-task tuning.

1. Ruder

S. (2017). An Overview of Multi-Task Learning in Deep Neural Networks. arXiv:1706.05098. In Section 2

"How multi-task learning works

" the paper states

"MTL improves generalization by leveraging the domain-specific information contained in the training signals of related tasks... By learning tasks in parallel

the model can learn a representation that is shared among them... For this reason

MTL is often considered a form of regularization." This supports the idea that single-task learning lacks this generalization benefit.

2. Raffel

C.

Shazeer

N.

Roberts

A.

Lee

K.

Narang

S.

Matena

M.

... & Liu

P. J. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research

21(140)

1-67. Section 3.3 discusses multi-task pre-training

noting that combining diverse tasks helps the model achieve better generalization

a benefit not fully realized when fine-tuning on a single

narrow downstream task.

3. Stanford University. (2023). CS224N: Natural Language Processing with Deep Learning

Winter 2023 Lecture 12: Pretraining and Transfer Learning. In the discussion on fine-tuning

the lecture notes explain that fine-tuning on a small

single-task dataset risks overfitting

whereas multi-task learning helps the model learn more general features

improving performance on out-of-domain or unseen tasks.

GenAI risk models evaluate bias by using quantitative fairness metrics. These metrics are specifically designed to measure disparities in model performance across different demographic groups (e.g., race, gender). Metrics like demographic parity (or statistical parity) ensure that the proportion of positive outcomes is similar across groups, while equalized odds ensures that the true positive rate and false positive rate are equal across groups. These are standard, technical measures for identifying and quantifying systemic bias in a model's predictions, which is a critical component of responsible AI governance and risk management.

A. Accuracy rate without considering demographics is incorrect because high overall accuracy can mask severe underperformance and bias against specific minority subgroups.

C. Computational efficiency during training measures resource consumption (time, memory) and is unrelated to the fairness or ethical implications of the model's outputs.

D. The number of parameters in the model relates to its complexity and capacity, not its fairness. A model can be biased regardless of its size.

1. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). NIST AI 100-1. In the discussion of the "MAP" function

the framework emphasizes measuring for bias: "Approaches to test

evaluate

verify

and validate (TEVV) AI systems for bias include... statistical and conditional statistical parity..." (p. 29). This directly supports the use of fairness metrics.

2. Verma

S.

& Rubin

J. (2018). Fairness Definitions Explained. In Proceedings of the 2018 ACM/IEEE International Workshop on Software Fairness (FairWare). This paper provides a formal overview of fairness metrics

stating

"Statistical Parity (a.k.a. Demographic Parity)... requires that the prediction... be independent of the protected attribute" (Section 3.1

p. 2). It also defines Equalized Odds (Section 3.2). DOI: https://doi.org/10.1145/3194770.3194776

3. Stanford University. (2021). CS224N: Natural Language Processing with Deep Learning - Lecture 18: Fairness

Accountability

Transparency

and Ethics in NLP. The lecture notes explicitly discuss fairness metrics for bias evaluation

covering "Group Fairness: Equalized Odds" and "Group Fairness: Demographic Parity" as key methods to assess model bias across different populations (Slides 35-40). Available at: https://web.stanford.edu/class/cs224n/slides/cs224n-2021-lecture-18-FATE.pdf

ISO/IEC TR 27563:2023 is a Technical Report titled "Information security, cybersecurity and privacy protection — Artificial intelligence — Privacy-enhancing technologies." Its primary purpose is to provide an overview and guidance on the selection and implementation of various Privacy-Enhancing Technologies (PETs) for Artificial Intelligence systems that process Personally Identifiable Information (PII). The document describes different PETs, such as differential privacy and homomorphic encryption, from a technical perspective, helping organizations mitigate privacy risks associated with AI applications without mandating specific tools.

B. As a Technical Report (TR), this standard provides informative guidance and recommendations; it does not mandate the use of specific encryption algorithms, which would be overly prescriptive.

C. The standard is explicitly designed to provide guidance for AI systems that do process personally identifiable information (PII), rather than limiting their use to non-personal data.

D. The core focus of ISO 27563 is to enhance privacy protection in AI systems, which is contrary to prioritizing performance metrics at the expense of privacy.

1. International Organization for Standardization (ISO). (2023). ISO/IEC TR 27563:2023: Information security

cybersecurity and privacy protection — Artificial intelligence — Privacy-enhancing technologies. Geneva

Switzerland: ISO.

Reference: The scope statement in Section 1 explicitly states

"This document provides an overview of privacy-enhancing technologies (PETs) that can be applied to artificial intelligence (AI) systems which process personally identifiable information (PII)... This document provides guidance on selecting and implementing PETs for AI systems."

2. Stübing

H.

& Kuper

P. (2023). Standardization Landscape for Secure and Trustworthy AI. 2023 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)

310-315.

DOI: https://doi.org/10.1109/EuroSPW59978.2023.00041

Reference: In Section IV

"STANDARDS FOR AI SECURITY AND PRIVACY

" the paper notes: "ISO/IEC TR 27563 provides an overview of privacy-enhancing technologies (PETs) that can be applied to AI systems which process PII." This confirms the standard's role in guiding the use of PETs.

In a tiered or hybrid conversational AI architecture, a Large Language Model (LLM) is commonly used as a secondary system. The primary system, often a more specialized or rule-based bot, handles frequent and straightforward queries efficiently. When the primary bot encounters a query that is too complex, ambiguous, or outside its predefined scope, it escalates the interaction to the secondary LLM. This LLM then uses its broader knowledge and generative capabilities to provide a more nuanced or comprehensive answer, thus serving as a fallback or supplementary assistant to ensure a robust user experience and prevent conversation failure.

B. The term "secondary" explicitly defines a supporting or subordinate role, not a replacement for the primary system.

C. A secondary chatbot's function is to support the primary application's objectives, not to divert to unrelated tasks, which would degrade the user experience.

D. Managing user credentials is a highly sensitive security function that requires specialized, secure modules, not a general-purpose generative AI.

1. Gao

J.

Galley

M.

& Li

L. (Eds.). (2022). Neural Approaches to Conversational AI. Now Foundations and Trends. This text discusses hybrid systems where a task-oriented dialogue system (primary) can hand off to an open-domain chit-chat model like an LLM (secondary) for out-of-scope queries. (See Chapter 1

Section 1.2: "Task-Oriented Dialogue Systems vs. Open-Domain Chatbots").

2. Jurafsky

D.

& Martin

J. H. (2023). Speech and Language Processing (3rd ed. draft). Stanford University. Chapter 25

"Chatbots & Dialogue Systems

" describes architectures where simpler retrieval-based or frame-based systems can be augmented with generative models (like LLMs) to handle the "long tail" of more complex user utterances. This establishes the principle of a secondary

more powerful system.

3. Adamopoulou

E.

& Moussiades

L. (2020). An Overview of Chatbot Technology. In IFIP International Conference on Artificial Intelligence Applications and Innovations (pp. 373-383). Springer. DOI: https://doi.org/10.1007/978-3-030-49186-431. This paper discusses hybrid models (p. 378) that combine the strengths of different approaches

where a generative model can supplement a retrieval-based or rule-based one

aligning with the secondary/fallback use case.