CY0-001 Practice Questions

The root cause of the reputational loss is the model producing biased and untrustworthy outputs. The most direct and effective long-term mitigation is to improve the model's alignment with desired ethical guidelines and user expectations. Reinforcement Learning from Human Feedback (RLHF) is a state-of-the-art technique specifically designed for this purpose. It uses human-provided feedback to fine-tune the model, rewarding helpful, harmless, and unbiased responses, thereby directly addressing the source of the brand damage. This aligns with AI TRiSM principles of model reliability and trust.

B. Migrating a vector database to on-premises is an infrastructure decision for data control and security, but it does not change the model's biased behavior.

C. Deploying additional containers improves system availability and scalability, not the quality or safety of the AI-generated content.

D. Increasing GPU capacity speeds up model inference but has no impact on the underlying logic or data that leads to biased outputs.

1. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). NIST AI 100-1. The framework's "Govern" function emphasizes creating a culture of risk management, and the "Map" function involves addressing impacts on individuals and society (Section 2.2), which directly relates to mitigating bias and reputational harm.

2. Ziegler, D. M., et al. (2019). Fine-Tuning Language Models from Human Preferences. This paper details an early implementation of RLHF, showing its effectiveness in aligning models to produce outputs preferred by humans, which includes reducing harmful or biased content (Section 1: Introduction).

3. Stanford University Human-Centered Artificial Intelligence (HAI). (2023). On the Opportunities and Risks of Foundation Models. The report discusses alignment techniques like RLHF as crucial for ensuring models behave safely and in accordance with human values (Chapter 4: Risks).

The primary defensive benefit of integrating Generative AI into high-interaction honeypots is enhancing their realism to deceive attackers. GenAI can dynamically adjust persona-based responses, creating believable, non-repetitive interactions that mimic legitimate users or systems, making the trap harder to identify. It also excels at generating large volumes of realistic synthetic data (e.g., files, logs, network traffic) to populate the honeypot environment. This increases the attacker's dwell time, as they are more convinced of the system's legitimacy, allowing defenders to gather more extensive intelligence on their tactics, techniques, and procedures (TTPs).

A. Automatic application of Model Watermarking to all outbound traeic: Watermarking is for tracing AI-generated content origin, not a primary defensive benefit of a honeypot's interaction with an attacker.

D. Reduction of Vector Database latency for faster model retraining: This is a backend performance optimization. It does not directly contribute to the honeypot's defensive capability of deceiving an attacker.

1. Weber, S., et al. (2023). HoneyGen: A generative-AI-based honeypot framework. 2023 IEEE International Conference on Cyber Security and Resilience (CSR), pp. 465-470. DOI: 10.1109/CSR57506.2023.10224999. (Describes using GenAI to create dynamic and believable honeypot content to increase attacker engagement).

2. Fang, Y., et al. (2023). LLM-based high-interaction honeypot for industrial control systems. Computers & Security, 135, 103491. DOI: 10.1016/j.cose.2023.103491. (Highlights the use of LLMs to generate realistic responses and simulate complex interactions, thereby increasing attacker dwell time).

3. Brown, T., et al. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems 33. (The underlying capabilities of LLMs described in this foundational paper enable the generation of realistic, context-aware text and data, which is the basis for options B and C).

Transparency in Responsible AI is built on principles that ensure a system's operations are understandable and auditable. Interpretability (I) allows stakeholders to understand the model's decision-making logic. Traceability (II) provides a clear audit trail of the data, code, and processes used, ensuring provenance and accountability. Disclosure (IV) is the ethical obligation to inform users they are interacting with an AI, which is a fundamental aspect of transparent operation. Obfuscation (III) is the direct opposite of transparency, as it involves intentionally hiding a model's internal workings, primarily for IP protection, not for responsible disclosure.

A. II, III, and IV only: This is incorrect because it includes Obfuscation (III), which is antithetical to transparency, and omits the core component of Interpretability (I).

B. I and II only: This is incorrect because it omits Disclosure (IV), a crucial component of transparently communicating the nature of the system to end-users.

C. I, II, III, and IV: This is incorrect because Obfuscation (III) is a practice that reduces transparency, it is not a component of it.

1. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). NIST AI 100-1, p. 21, Section 4.3.2, defines the "Transparent" characteristic as encompassing explainability, interpretability, and disclosure about the system.

2. High-Level Expert Group on AI. (2019). Ethics Guidelines for Trustworthy AI. European Commission, p. 14, Section "The requirement of Transparency". The guidelines explicitly list traceability, explainability, and communication (disclosure) as key elements of transparency.

3. Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1, 389–399. https://doi.org/10.1038/s42256-019-0088-2. This paper's analysis of global AI ethics principles consistently identifies transparency as a core value, encompassing concepts of explainability and disclosure.

The question asks for priority actions to secure the MLOps pipeline against supply chain attacks.

A. Enforce Multi-Factor Authentication for model registry access: A model registry is a critical component of the MLOps supply chain, storing trained models. A supply chain attack could involve an attacker gaining unauthorized access to this registry to poison a model or steal intellectual property. Enforcing MFA is a fundamental access control measure that significantly hardens this component against unauthorized access.

D. Implement Signed Container Images for ML training environments: ML development and training environments, such as Jupyter notebooks, are often deployed as container images. A supply chain attack can compromise these base images, injecting malicious code that could steal data or compromise the training process. Signing container images provides a cryptographic guarantee of their authenticity and integrity, ensuring that only trusted, verified images are used in the CI/CD pipeline.

B. Perform Feature Engineering to improve Model F1-scores: This is a data science task focused on improving model performance and accuracy, not securing the underlying infrastructure from supply chain attacks.

C. Label Large Data Sets for supervised learning tasks: This is a data preparation step. While the integrity of the data labeling process is important, the act of labeling itself is not a direct security control against attacks on the CI/CD pipeline or software dependencies.

E. Manage Hyperparameter Tuning via automated grid search: This is a model optimization technique used to find the best model configuration. It is related to model performance, not the security of the MLOps infrastructure.

1. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). NIST AI 100-1. The framework emphasizes securing all components of the AI lifecycle, stating that "AI systems are subject to the same vulnerabilities as other software systems, as well as new, AI-specific vulnerabilities." Securing artifacts (models, data, code) and the development environment is a core tenet. (Section 4.2, "Govern").

2. Google Cloud Documentation. Securing your software supply chain. This documentation outlines best practices for securing CI/CD pipelines, including the use of artifact verification. "Software supply chain security is about ensuring the integrity of the software you use... Binary Authorization for Borg (BAB) enforces that production software is properly reviewed and authorized." Signing images is a key part of this verification. (Section: "Verifying artifacts").

3. OWASP. Machine Learning Security Top 10. M08-2023: AI Supply Chain Compromise. This risk category explicitly covers attacks on components used to build the ML system, including development environments and third-party dependencies. Recommended mitigations include "Verify software signatures and use lock files for all third-party libraries" and "Implement strong access controls for all components of the ML pipeline."

The MITRE ATLAS framework is a knowledge base of adversarial tactics and techniques based on real-world observations and is intentionally modeled after the structure of the well-known MITRE ATT&CK framework. This common structure facilitates integration into existing security operations and allows for cross-domain analysis. ATLAS includes AI-specific tactics that describe an adversary's goals, such as 'ML Model Access' and 'Exfiltration' of model components or data. However, the framework is not limited to Generative AI or LLMs; it covers a wide range of AI/ML systems, including computer vision and other predictive models, making statement III incorrect.

B. This option is incorrect because statement III is false. The ATLAS framework's scope extends beyond just Generative AI and LLMs to encompass the broader AI/ML landscape.

C. This option is incorrect because statement I is true. The structural similarity between ATLAS and ATT&CK is a fundamental design choice to promote adoption and integration.

D. This option is incorrect because statement II is also true. ATLAS defines specific tactics like 'ML Model Access' and 'Exfiltration' that are unique to AI system attacks.

1. MITRE. (2023). MITRE ATLAS Overview. MITRE ATLAS™. Retrieved from https://atlas.mitre.org/ "ATLAS is a knowledge base of adversarial tactics, techniques, and case studies for artificial intelligence (AI) systems based on real-world observations... ATLAS is modeled after the MITRE ATT&CK® framework".

2. MITRE. (2023). ATLAS Tactics. MITRE ATLAS™. Retrieved from https://atlas.mitre.org/tactics. The official list of tactics includes TA0003 - ML Model Access and TA0010 - Exfiltration.

3. Strohm, B., et al. (2020). Adversarial Threat Landscape for Artificial-Intelligence Systems (ATLAS) (Technical Report MTR20-0328). MITRE Corporation. Section 2.1, "ATLAS is modeled after the MITRE ATT&CK framework...".

AI-enabled IDE plugins are a core component of the "Shift Left" security paradigm, which focuses on integrating security practices early in the software development lifecycle (SDLC). These tools analyze code in real-time as it is written, identifying potential vulnerabilities (D). By leveraging AI, they can understand the context of the entire local codebase, allowing them to provide highly relevant and actionable remediation suggestions, unlike generic static analyzers (A). This immediate feedback loop helps developers write more secure code from the outset, reducing the cost and complexity of fixing issues later.

B. These plugins perform static analysis (SAST) on code at rest; they do not replace DAST, which tests the application in its running state and is still a crucial security practice.

C. No security tool can guarantee 100% protection, especially against novel or unknown (Zero-Day) attacks. This is an unrealistic and absolute claim.

1. O'Neill, M., et al. (2023). Engineering Secure Software: A Guide for Developers. MITRE Corporation. Section 2.3, "Shifting Security Left," discusses integrating security tools into the developer workflow.

2. Parnin, C. (2023). CS 510: Software Engineering, Lecture on DevSecOps. North Carolina State University, Department of Computer Science. The lecture covers the role of IDE-integrated security tools in enabling the Shift Left approach.

3. Snyk. (2022). Snyk IDE plugin documentation. Snyk.io. (Vendor Documentation). Describes how the tool provides "contextual security issues and remediation advice" directly within the IDE.

The Monitoring and Maintenance phase of the AI security life cycle involves activities performed after a model is deployed. This includes continuously monitoring for performance degradation like model drift (I), performing scheduled retraining to update the model and maintain its accuracy (II), and patching the underlying infrastructure, such as virtual containers, to protect against new vulnerabilities (III). Data sanitization on the initial training set (IV) is a data preparation activity that occurs during the development phase, before the model is deployed and monitored.

A. This option is incomplete as it omits the critical security maintenance task of patching the operational environment (III), which is essential for ongoing security.

B. This option is incorrect because data sanitization (IV) is a pre-deployment activity performed during the data preparation stage of the AI development life cycle.

D. This option incorrectly includes data sanitization (IV), which is part of the initial data preparation and development phase, not the post-deployment monitoring and maintenance phase.

1. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). The "Govern" function is mapped across the lifecycle, and the "Operate & Monitor" stage explicitly covers post-deployment activities, including performance monitoring (related to I) and managing system security (related to III).

2. Microsoft. (2020). Machine learning operations (MLOps) framework. Azure Architecture Center. The "Model deployment & monitoring" phase of the MLOps cycle includes monitoring for drift (I) and triggering retraining (II). The overall operational management includes infrastructure security (III).

3. Kumar, S., et al. (2022). Engineering Trustworthy AI Systems: A Development and Operations (DevOps) Perspective. arXiv:2206.00535. Section 4.3, "Monitoring and Maintenance," details the need for continuous monitoring for concept drift (I), model retraining (II), and infrastructure security management (III).

The system prompt is a set of high-level instructions provided to an LLM before any user input. It is the standard and most effective method for defining the model's persona, operational boundaries, and safety guardrails. By configuring the system prompt to state that the model is a "security analyst" and must "never reveal administrative credentials," the team directly enforces the required behavioral constraints at the core of the model's operation for that session.

A. Homomorphic Encryption protects data privacy during processing but does not influence or control the behavioral output of the LLM.

C. Adjusting the temperature parameter controls the randomness of the output, not the adherence to fundamental operational rules or safety policies.

D. RBAC controls user access to the model, not the internal behavior of the model itself during an inference session.

1. Wei, A., et al. (2023). "Jailbroken: How Does LLM Safety Training Fail?". arXiv preprint arXiv:2307.02483. This paper discusses how system prompts are the primary mechanism for safety alignment and how attackers try to override them. (Section 2.1, "System Prompts").

2. OpenAI. (2023). "GPTs - Custom instructions". OpenAI Documentation. The official documentation for custom GPTs explains that the "Instructions" field (a system prompt) is used to define the model's personality and what it should or should not do. (Accessed via OpenAI's developer platform documentation on GPT customization).

3. Perez, F., & Ribeiro, I. (2022). "Ignore Previous Prompt: Attack Techniques For Language Models". Stanford University HAI. This research highlights the critical role of the initial prompt (system prompt) in defining model behavior and how attacks focus on escaping these constraints. (Section: "Prompt Injection").

The core problem is a lack of centralized governance, standardized policies, and technical guidance for AI security. An AI Center of Excellence (CoE) is an organizational body specifically designed to address these issues. It centralizes AI expertise, establishes best practices and security standards, and provides oversight for all AI initiatives within an organization. This structure directly resolves the described challenges of inconsistent protocols and decentralized risk management by creating a single, authoritative source for AI governance.

A. Implement a Shadow AI monitoring and blocking policy: This is a specific security policy, not a comprehensive organizational governance structure that provides leadership and guidance.

B. Deploy a decentralized MLOps pipeline for each department: This reinforces the existing decentralized approach that caused the problem, rather than creating the required centralized governance.

D. Outsource all Model Inference to a third-party provider: This is a tactical decision for one part of the ML lifecycle and does not establish an internal governance structure.

1. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). (NIST AI 100-1). The "Govern" function of the framework (Section 3.1) emphasizes establishing a culture of risk management, which is a primary role of a CoE.

2. Davenport, T. H., & Ronanki, R. (2018). Artificial Intelligence for the Real World. Harvard Business Review. The article discusses organizational structures for AI, highlighting the role of a Center of Excellence in centralizing capabilities and spreading best practices.

3. Stanford University. (Course CS329S). Machine Learning Systems Design. Lecture notes on MLOps and organizational structure frequently discuss the CoE model as a method for scaling AI capabilities and governance effectively across an enterprise.

In safety-critical systems like autonomous robots, reliability and safety are paramount. Adversarial Training (C) directly enhances model robustness by exposing it to a wide range of unexpected or intentionally crafted malicious inputs during the training phase. This makes the model less likely to fail when encountering out-of-distribution data in the real world. A hard-coded fail-safe mechanism (D) is a fundamental safety engineering principle. It provides a deterministic, reliable override that can halt or place the system in a safe state if the complex AI logic fails or behaves unpredictably, directly ensuring physical safety.

A. Using open-source models is a sourcing strategy focused on cost or development speed; it does not inherently guarantee or improve reliability or safety.

B. Rotating API keys is a standard access control measure for cloud services. It secures the development environment, not the operational safety of the deployed robot.

1. Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014). Explaining and Harnessing Adversarial Examples. arXiv:1412.6572. This foundational paper introduces adversarial training as a method to improve model robustness against adversarial inputs, a key aspect of reliability.

2. Koopman, P. (2022). The UL 4600 Standard for Autonomous System Safety. Carnegie Mellon University. Section 7, "Safety Case," emphasizes the need for robust fault detection, fail-safe responses, and risk mitigation for hazards, including AI model failures. Hard-coded fail-safes are a primary example of such a response.

3. International Organization for Standardization (ISO). (2021). ISO/IEC TR 5469:2024 Artificial intelligence — Functional safety and AI systems. This technical report discusses the relationship between AI and functional safety standards (like IEC 61508), highlighting the need for both AI-specific robustness measures and traditional safety mechanisms like fail-safes.