AAISM Practice Questions

Automation bias is the tendency for humans to over-rely on automated systems, trusting their outputs even when there is contradictory information from other sources. In this scenario, the driver excessively trusted the AI-enabled mapping software's recommendation and, as a result, ignored clear, real-world evidence—the numerous signs indicating a one-way street. The driver's failure to apply critical judgment and override the incorrect automated instruction is a classic manifestation of automation bias, where the human cedes decision-making authority to the machine, leading to an error.

A. Selection: Selection bias occurs during data collection when the data used to train a model is not a representative sample of the real world, which is not what the driver's action demonstrates.

B. Reporting: Reporting bias arises when the frequency of events or outcomes in the data does not reflect their real-world frequency. This is a data-level issue, not a human-computer interaction bias.

C. Confirmation: Confirmation bias is the tendency to favor information that confirms one's pre-existing beliefs. The driver was following the AI's instructions, not confirming a personal belief about the route.

1. ISACA. (2023). Artificial Intelligence Audit Framework. Section 3

"AI Risks and Controls

" identifies "Human biases (e.g.

automation bias

confirmation bias)" as a key risk area. It highlights the danger of individuals placing undue trust in AI system outputs

which directly aligns with the scenario.

2. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). In the "AI RMF Playbook

" under the Govern function (GOV-3)

it discusses human-AI interaction and the risk of automation bias

where "individuals may over-rely on AI system outputs without sufficient critical assessment."

3. Parasuraman

R.

& Manzey

D. H. (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors: The Journal of the Human Factors and Ergonomics Society

52(3)

381–410. Page 382 defines automation bias as "the use of automation as a heuristic replacement for vigilant information seeking and processing

" which describes the driver's failure to process the information from the road signs. https://doi.org/10.1177/0018720810376055

4. Goddard

K.

Roudsari

A.

& Wyatt

J. C. (2012). Automation bias: a systematic review of frequency

effect mediators

and mitigators. Journal of the American Medical Informatics Association

19(1)

121–127. The article defines automation bias as "the tendency to over-rely on automated aids and decision support systems

" which is precisely what the driver exhibited by following the AI route into a dangerous situation. https://doi.org/10.1136/amiajnl-2011-000099

The most effective method to prevent large language models (LLMs) from exposing sensitive data in their output is to ensure they are never exposed to it in the first place. Data sanitization techniques, such as masking, redaction, or tokenization, are applied to the training data and user prompts to remove or anonymize personally identifiable information (PII) and other sensitive data before the LLM processes it. By cleaning the input, the model does not learn the sensitive information and therefore cannot inadvertently reproduce or "regurgitate" it in its responses. This is a direct, preventative control that addresses the root cause of the risk.

A. Encrypting data in transit and at rest protects data during storage and transmission but not during processing. The LLM requires decrypted data to function, at which point it can be exposed.

B. Conducting adversarial testing is a validation technique used to identify if an LLM is vulnerable to leaking sensitive data, rather than a preventative control that reduces the risk itself.

D. Enforcing least privilege access restricts who can interact with the LLM or its underlying data, but it does not prevent an authorized user or process from using sensitive data that the model can then leak.

1. ISACA

Artificial Intelligence Audit Toolkit

2023. In the "AI Governance and Risk Management" domain

Control Objective A&G-05: "Data Privacy and Protection" emphasizes implementing controls to protect sensitive data throughout the AI lifecycle. This includes the data preprocessing stage

where sanitization is a key activity to prevent PII from being ingested by the model.

2. ISACA

Auditing Artificial Intelligence White Paper

2023. Page 12

under the "Data Acquisition and Preparation" section

states that a key audit step is to "verify that controls are in place to ensure that PII and other sensitive data are removed or anonymized from data sets prior to their use in AI/ML models." This directly supports sanitization as the primary control.

3. Mee

T.

et al. A Survey on Privacy in Large Language Models: Attacks

Defenses

and Future Directions. arXiv

2023. Section 4.1

"Data Pre-processing

" identifies data sanitization and anonymization as primary defense mechanisms against privacy attacks. It states

"The most straightforward defense is to remove sensitive information from the pre-training data." (DOI: https://doi.org/10.48550/arXiv.2312.06717).

The primary purpose of implementing moderation controls in generative AI is to serve as a safety mechanism. These controls are designed to analyze content generated by the model against a predefined set of policies to detect and filter outputs that are harmful, inappropriate, or otherwise undesirable. This includes categories such as hate speech, harassment, self-harm, and explicit content. By filtering such content, organizations can mitigate risks, protect users, and ensure the AI application operates within acceptable ethical and safety boundaries.

A. Moderation controls are inherently restrictive, designed to limit certain types of output, which is the opposite of increasing diversity or creativity.

B. Adding a moderation layer requires an additional processing step, which introduces latency and therefore increases, rather than optimizes, the model's response time.

C. While moderation can help identify some privacy-violating content (e.g., PII), its primary scope is broader safety. Comprehensive privacy compliance requires dedicated data governance and legal frameworks.

1. Official Vendor Documentation: OpenAI. (n.d.). Moderation. OpenAI API Documentation. Retrieved from openai.com/docs/guides/moderation.

Reference Point: The documentation explicitly states

"The moderation endpoint is a tool you can use to check whether content complies with OpenAI's usage policies. Developers can thus identify content that our usage policies prohibit and take action

for instance by filtering it." This directly supports that the primary function is filtering harmful/inappropriate content.

2. Peer-Reviewed Academic Publication: Weidinger

L.

et al. (2021). Taxonomy of Risks posed by Language Models. Proceedings of the 2022 ACM Conference on Fairness

Accountability

and Transparency (FAccT '22)

214–229. https://doi.org/10.1145/3531146.3533088

Reference Point: Section 5

"Mitigation

" discusses various strategies to address risks. Sub-section 5.2

"Filtering

" describes methods for "filtering training data or model outputs to remove undesirable content

" which is the core function of moderation controls for mitigating risks like generating harmful content (discussed in Section 3.2).

3. University Courseware: Stanford University. (2023). CS224N: Natural Language Processing with Deep Learning

Winter 2023. Lecture 18: "Toxicity and Bias".

Reference Point: The lecture slides and associated readings discuss the problem of models generating toxic or biased language. It covers techniques for "content moderation" and "toxicity detection" as primary methods to prevent the model from outputting harmful content

reinforcing that filtering is the key benefit.

Embedding unique identifiers into AI models, a technique known as digital watermarking or fingerprinting, is a primary method for protecting the intellectual property (IP) of the model. This unique marker serves as a proof of origin, allowing the owner to track the model's distribution and assert ownership if it is stolen, leaked, or used without authorization. It directly links the model artifact back to its creator or legal owner, which is crucial for provenance, accountability, and IP rights enforcement. This technique does not inherently provide security against unauthorized access, eliminate bias, or detect adversarial attacks but focuses specifically on ownership verification.

A. Preventing unauthorized access is managed by access control systems (e.g., authentication, authorization), not by an embedded identifier.

C. Eliminating AI system biases is addressed through data balancing, algorithmic fairness techniques, and careful model design, not by a unique ID.

D. Detecting adversarial attacks requires specific defense mechanisms designed to identify and mitigate malicious inputs, which is unrelated to a static identifier.

1. Zhang

J.

Gu

D.

Jang

J.

Wu

H.

Stoecklin

M. P.

& Chen

G. (2018). Protecting intellectual property of deep neural networks with watermarking. In Proceedings of the 2018 on Asia Conference on Computer and Communications Security (ASIA CCS '18). The abstract states

"we propose a new methodology to watermark a deep neural network for the purpose of intellectual property protection... to verify the ownership and track the pirated copies." (DOI: https://doi.org/10.1145/3196494.3196550)

2. Adi

Y.

Baum

C.

Cisse

M.

Pinkas

B.

& Keshet

J. (2018). Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring. In 27th USENIX Security Symposium (USENIX Security 18). Section 1

Introduction

paragraph 2

explains that watermarking allows a model owner "to prove their ownership over the model" if it is stolen and deployed by an adversary.

3. Carnegie Mellon University

ECE Course 18-739: Security and Fairness of Deep Learning. Lecture 15

"Model Stealing and Watermarking

" describes watermarking as a technique to "Embed a 'signature' in a model to identify the owner" and "prove ownership of a stolen model."

The primary ethical concern of generative AI is its potential to cause significant information integrity issues. By its nature, generative AI creates new content that can be indistinguishable from human-created content but may be factually incorrect, misleading, or entirely fabricated (i.e., "hallucinations"). This capability directly undermines the reliability, trustworthiness, and authenticity of information. The potential for mass generation of disinformation and deepfakes poses a fundamental threat to societal trust and the integrity of the information ecosystem, making it the most central ethical challenge.

A. Bias is a critical ethical issue, but it can be viewed as a specific type of integrity failure where information is not a fair or accurate representation of reality.

C. Generative AI's core function is to create, not restrict, information. Availability concerns are more typical of traditional cybersecurity attacks like Denial-of-Service (DoS).

D. Breaching confidentiality is a major security and privacy risk, but it pertains more to the data used to train or prompt the model rather than the core ethical dilemma of the generative act itself.

---

1. Isaca

Artificial Intelligence for Auditing White Paper

2023: This document highlights key risks associated with generative AI. In the section "Key Risks and Challenges of Generative AI

" it explicitly lists "Hallucinations and Misinformation

" stating

"Generative AI models can sometimes produce outputs that are factually incorrect

nonsensical or disconnected from the input context. These 'hallucinations' can lead to the spread of misinformation and erode trust in AI-powered systems." This directly supports information integrity as a primary concern.

2. Isaca

AI Governance: A Primer for Audit Professionals White Paper

2024: This guide discusses the governance of AI systems and identifies "Inaccurate or Misleading Outputs (Hallucinations)" as a key risk area. It emphasizes that "The potential for generative AI to produce plausible but incorrect or nonsensical information... poses significant risks to decision-making

reputation

and trust

" framing the integrity of the output as a central governance challenge.

3. Stanford University

Center for Research on Foundation Models (CRFM)

"On the Opportunities and Risks of Foundation Models

" 2021: This foundational academic paper discusses the capabilities and societal impacts of large-scale models. Section 4.2

"Misinformation and disinformation

" details how these models can be used to generate "high-quality

targeted

and inexpensive synthetic text

" which fundamentally threatens the integrity of information online. (Available at: https://arxiv.org/abs/2108.07258)

The most significant legal risk associated with generative AI using internet data is intellectual property (IP) and copyright infringement. Activating filtering logic to exclude data with IP flags is a direct, preventative technical control. This measure proactively reduces the likelihood of the AI model ingesting, training on, or reproducing protected content without authorization. By addressing the risk at the data ingestion stage, the service provider directly mitigates the root cause of potential infringement lawsuits, making it the most effective recommendation among the choices.

B. Disclosing policies is a transparency measure. While important for compliance, it does not prevent the underlying act of IP infringement from occurring.

C. Appointing a data steward is a governance control. It is an indirect measure; the specific technical controls the steward implements, such as filtering, provide the direct mitigation.

D. Reviewing logs is a detective control performed after data has already been collected. It does not prevent the initial collection of infringing data.

1. ISACA

Artificial Intelligence Audit and Security Management (AAISM) Study Guide

1st Edition. Domain 2: AI Data and Model Management. This domain emphasizes the criticality of data sourcing

provenance

and quality for AI systems. It outlines that organizations must implement controls to ensure training data complies with legal and regulatory requirements

including copyright and IP laws. Filtering data sources is a primary technical control to achieve this.

2. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). Section 4.3

"Govern" Function. The framework highlights the need for policies and controls to manage risks associated with AI training data. It states

"Data-related controls can include checking for and filtering out copyrighted material

personally identifiable information (PII)

or biased content from training datasets to mitigate legal and ethical risks." This directly supports filtering as a key risk mitigation strategy.

3. Liang

W.

et al. (2023). Holistic Evaluation of Language Models. Stanford University

Center for Research on Foundation Models (CRFM). This and similar research from Stanford's HAI (Human-Centered AI) institute frequently discuss the challenges of "data contamination

" which includes copyrighted and other problematic data from web scrapes. The research underscores the importance of data curation and filtering techniques to improve model safety

legality

and alignment. (See discussions on data curation in sections related to model training and evaluation).

Weak controls for access to the AI model represent a fundamental cybersecurity vulnerability. Exploiting poor access controls (e.g., default credentials, lack of multi-factor authentication, insecure APIs) typically requires less specialized knowledge and fewer resources than executing AI-specific attacks. An adversary can leverage common hacking tools and techniques to gain unauthorized access, which can lead to model theft, data poisoning, or system manipulation. This makes it the most accessible and, therefore, most easily exploited vulnerability compared to the other, more technically complex options.

A. Exploiting inaccurate generalizations requires feeding the model specifically crafted data over time to influence its learning, which is a complex and often slow process.

C. While common, launching a denial-of-service (DoS) attack that successfully overwhelms modern, resilient infrastructure often requires significant computational resources and coordination.

D. Creating subtle input modifications (adversarial examples) to cause inappropriate outputs is technically sophisticated and typically requires significant knowledge of the model's architecture and parameters.

---

1. ISACA

"Auditing Artificial Intelligence

" 2019. This guide emphasizes that foundational IT controls are critical for AI systems. It states

"Traditional IT general controls (ITGCs) are still relevant and applicable to AI systems. These controls

such as access management

change management and IT operations

are the foundation on which AI-specific controls are built." (Page 31

Section: AI-Specific Controls). This supports the premise that a failure in a foundational control like access management is a primary and exploitable risk.

2. ENISA (European Union Agency for Cybersecurity)

"AI Cybersecurity Challenges: A Taxonomy of Threats and Vulnerabilities

" December 2020. This report provides a threat taxonomy for AI systems. It lists "Unauthorized Access" as a key threat

which can be caused by "insufficient access control." This is presented as a foundational threat that enables other attacks

distinct from more complex

AI-specific attacks like "Evasion" (adversarial examples). (Page 16

Table 1).

3. NIST

"AI Risk Management Framework (AI RMF 1.0)

" NIST AI 100-1

January 2023. The framework's core practices for managing AI risks include the sub-category "Govern 4: Policy and Procedures

" which explicitly mentions "access control policies and procedures." The document stresses that securing AI systems involves robust cybersecurity hygiene

where access control is a cornerstone

indicating its critical and primary nature as a vulnerability. (Page 23

Section 4.1).

Hidden triggers, also known as backdoors or trojans, are vulnerabilities intentionally embedded in an AI model during training. They cause the model to produce a specific, often malicious, output when a specific input (the trigger) is present. Adversarial training is the most direct and effective mitigation technique. This process involves intentionally generating adversarial examples, which can include potential triggers, and using them to retrain the model. By exposing the model to these inputs and teaching it the correct, non-triggered response, the training process effectively neutralizes the backdoor's influence and hardens the model against such manipulations. This is a proactive defense that directly addresses the vulnerability within the model itself.

A. Regularly retraining the model using a diverse data set is a general best practice but may not specifically find and overwrite a carefully crafted trigger.

B. Applying differential privacy is primarily a technique to protect the privacy of individuals in the training data, not to mitigate security backdoors.

D. Monitoring model outputs is a detective control that identifies potential trigger activations after they occur, rather than a mitigative control that reduces the vulnerability itself.

1. ISACA

Auditing Artificial Intelligence

2023: In the section on "AI Security Risks

" the document discusses backdoor attacks (data poisoning) as a significant threat. It explicitly lists "adversarial training to make the model more robust" as a key defense mechanism against such attacks.

2. Li

Y.

et al. (2022). A Survey of Backdoor Attacks and Defenses in Deep Learning. ACM Computing Surveys

55(1)

1-37: Section 4

"Backdoor Defenses

" categorizes defense strategies. It places techniques like adversarial training under "Training-Phase Defenses

" describing them as methods to enhance model robustness by augmenting the training process with examples designed to expose and nullify the effects of potential backdoors. (DOI: https://doi.org/10.1145/3508019)

3. Madry

A.

et al. (2018). Towards Deep Learning Models Resistant to Adversarial Attacks. ICLR 2018: This foundational academic paper

often cited in university courseware (e.g.

MIT 6.S191 Introduction to Deep Learning)

establishes adversarial training as a principled and effective defense for making neural networks robust against worst-case perturbations

the same principle that applies to neutralizing hidden triggers.

In high-stakes applications like credit risk assessment, ensuring ethical decision-making is paramount. The "human-in-the-loop" (HITL) approach, where the AI system provides advisory outputs and a human expert makes the final decision, is the highest priority. This design embeds accountability and allows for contextual judgment that an AI model cannot replicate. A human expert can override potentially biased or unfair automated recommendations, providing a critical safeguard against algorithmic discrimination and ensuring that decisions are both explainable and ethically sound. This approach directly addresses the core requirement of preventing bias at the point of decision.

A. Regularly updating the model is a standard practice for maintaining performance, but it does not inherently address ethical concerns or bias. New data can perpetuate or even introduce new biases.

B. An appeal mechanism is a crucial reactive control for redress after a decision is made, but it is not a preventative measure integrated into the initial decision-making process itself.

D. Restricting the model to "objective" financial metrics is a flawed strategy, as these metrics can act as proxies for protected characteristics (e.g., zip code correlating with race), leading to systemic bias.

1. ISACA

Artificial Intelligence Audit Framework

2023. In the section on AI Governance and Risk Management

the framework emphasizes the need for human oversight

especially for high-risk AI systems. It states

"For high-risk AI systems

human oversight should be in place to allow for human intervention to prevent or minimize harm... The final decision is made by a human." (Domain 1: AI Governance

Section 1.3 Human Oversight and Involvement).

2. High-Level Expert Group on Artificial Intelligence

Ethics Guidelines for Trustworthy AI

European Commission

2019. This foundational document lists "Human agency and oversight" as one of the seven key requirements for trustworthy AI. It specifies that for critical decisions

the system should support human autonomy

with the final determination left to a human. It recommends "human-in-the-loop (HITL)" approaches for high-stakes domains. (Chapter II

Page 14).

3. Barocas

S.

& Selbst

A. D.

"Big Data's Disparate Impact

" California Law Review

Vol. 104

2016. This academic paper explains how seemingly neutral or "objective" criteria can function as proxies for protected classes

leading to discriminatory outcomes. It demonstrates why simply restricting data inputs (as suggested in option D) is insufficient to prevent bias. (Section II.A. The Problem of Proxies

pp. 679-685). DOI: https://doi.org/10.15779/Z38G655.

An accountability model establishes a clear governance structure, defining roles, responsibilities, and oversight mechanisms for AI systems. This framework is fundamental to ensuring that an AI product is used ethically and responsibly throughout its lifecycle. It addresses the organization's concerns by creating clear lines of authority for managing risks related to fairness, bias, transparency, and societal impact. This model provides the foundation upon which specific tools and practices, such as model cards and security protocols, are implemented and enforced.

A. Model cards are a tool for transparency, which is a component of a responsible AI strategy, but they do not constitute the entire governance framework needed for accountability.

B. Vendor monitoring is relevant only when using third-party AI systems and does not address the organization's internal responsibility for a product it has developed.

D. Security by design is a critical practice focused on protecting the AI system from threats, but it does not cover the full spectrum of ethical considerations like fairness and bias.

1. ISACA

"Artificial Intelligence: An Audit and Assurance Framework

" 2023. Page 14

under the "AI Governance Framework" section

states

"A governance framework establishes accountability

roles and responsibilities

and decision rights... It also provides a structure for oversight to ensure that AI systems are aligned with the organization’s ethical principles."

2. National Institute of Standards and Technology (NIST)

"AI Risk Management Framework (AI RMF 1.0)

" January 2023. Section 3.1

"The GOVERN Function

" emphasizes that this function is central to responsible AI. It states

"Governance processes should be in place to ensure accountability for AI risks and their management... This includes assigning roles and responsibilities for all stages of the AI lifecycle."

3. Mittelstadt

B. (2019). "Principles alone cannot guarantee ethical AI." Nature Machine Intelligence

1(11)

501-507. This academic paper argues that high-level ethical principles are insufficient without practical implementation through "mechanisms of accountability" and robust governance structures to translate principles into practice. (DOI: https://doi.org/10.1038/s42256-019-0114-4)