CASE STUDY Please use the following answer the next question: A mid-size US healthcare network has decided to develop an Al solution to detect a type…

An impact assessment is the most effective proactive measure to mitigate discrimination prior to model training and testing. This process systematically identifies and analyzes potential risks, including fairness and bias, associated with the AI system's intended use, data sources, and operational context. By conducting this assessment early, the healthcare network can foresee how the system might disproportionately affect certain demographic groups, evaluate the representativeness of its training data, and design mitigation strategies before significant resources are invested in development. This aligns with the core principles of responsible AI development, which emphasize foresight and risk management from the project's inception.

A. Procure more data from clinical research partners: Simply acquiring more data does not inherently mitigate bias; if the new data shares the same demographic imbalances as the original data, it could amplify discrimination.
B. Engage a third party to perform an audit: Audits are typically performed on a system that has already been built or deployed to validate its performance and compliance, making it a reactive, not a proactive, measure.
D. Create a bias bounty program: This is a post-development activity where a model is exposed to testers to find biases. It is a useful tool for identifying issues in a trained model, not for prevention beforehand.

1. NIST. (2023). AI Risk Management Framework (AI RMF 1.0). National Institute of Standards and Technology.

Reference: The "MAP" function of the framework (Section 3.3

Page 18) emphasizes understanding the context and mapping risks before an AI system is designed or deployed. It states

"MAP is intended to be applied at the outset of the AI lifecycle... to frame risks in relation to an AI system’s context." This includes identifying risks of harmful bias and discrimination

which is the core purpose of an impact assessment in this scenario.

2. Reisman

D.

Schultz

J.

Crawford

K.

& Whittaker

M. (2018). Algorithmic Impact Assessments: A Practical Framework for Public Agency Accountability. AI Now Institute.

Reference: Page 6 of this foundational report states

"An AIA is a tool for assessing the potential impacts of an algorithmic system before it is deployed... It provides a repeatable

structured process to assess the potential societal impacts of an automated system and to determine whether mitigation strategies are required." This directly supports performing an assessment prior to development to mitigate harm.

3. Selbst

A. D.

boyd

d.

Crawford

K.

& Whittaker

M. (2019). Fairness and Abstraction in Sociotechnical Systems. Proceedings of the 2019 ACM Conference on Fairness

Accountability

and Transparency

59–68.

Reference: Section 3

"Problem Formulation

" argues that critical decisions impacting fairness are made at the very beginning of a project. The paper notes

"The act of problem formulation itself—the translation of a real-world problem into the formal

constrained language of a technical system—is a locus of normative decision-making." An impact assessment forces the team to confront these decisions and their potential discriminatory consequences at the earliest stage.

DOI: https://doi.org/10.1145/3287560.3287598