312-41 practice question 30 of 35

Within a high-hazard industrial environment, an AI system is assessed for use in controlling pressure valves connected to volatile chemical…

Full set: 100 questions
Within a high-hazard industrial environment, an AI system is assessed for use in controlling pressure valves connected to volatile chemical processes. Although the system demonstrates the technical ability to make real-time adjustments, any incorrect action could initiate an uncontrolled reaction with severe safety consequences. As a result, the organization restricts the system’s role to monitoring and reporting sensor data, while all valve adjustments remain exclusively under human control. On the Collaboration Spectrum, which factor most directly explains why the AI’s autonomy is limited in this manner?
Answer options
A: Regulatory Request
B: Team Readiness
C: AI Maturity
D: Risk Level Correct answer
Correct Answer:
D. Risk Level
Explanation

The decision to limit the AI's autonomy is a direct consequence of the high-risk level associated with its task. In this high-hazard environment, the potential for a catastrophic outcome (an uncontrolled chemical reaction) from an incorrect AI action outweighs the benefits of full automation. Even though the AI is technically capable, the severity of potential failure mandates that a human remains in control for safety-critical decisions. This principle of risk mitigation is fundamental in deploying AI in high-stakes applications.

Why Incorrect

A. Regulatory Request: The scenario describes an internal organizational decision based on safety, not an external mandate from a regulatory body.
B. Team Readiness: The limitation is not due to the team's lack of skill but the inherent danger of the process itself.
C. AI Maturity: The system is described as technically capable, so its performance or maturity is not the limiting factor; the consequence of its failure is.

References

1. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). (NIST AI 100-1). Page 4, Section 2.1, states that AI risk is a function of the likelihood of an event and the magnitude of its impact. The "severe safety consequences" in the scenario represent a high-magnitude impact, justifying the risk-mitigation strategy of limiting autonomy.

2. Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 30(3), 286–297. https://doi.org/10.1109/3468.844354. This foundational paper discusses how the allocation of functions between humans and automation depends on factors including the costs and consequences of actions, which aligns with the risk-based decision in the scenario.

Scroll to Top