Isabella, a Lead Data Scientist, is auditing a credit-scoring model that shows a statistically significant disparity in approval rates for shift…

Bias infiltrated the system during feature selection. The prompt explicitly states the engineering team "decided to include telemetry data related to hardware brand and application timestamp." This decision, made to find new indicators, inadvertently introduced features that act as proxies for socioeconomic status. The algorithm itself is sound, and the training data is technically accurate. The critical error was the choice of which attributes to use for model training, which is the essence of feature selection. This introduced a systemic bias that penalizes a protected group based on correlation rather than causation.

A. The prompt explicitly states the algorithm is "mathematically sound and functions exactly as designed," ruling out algorithmic bias as the source.

B. The issue is rooted in the model's design and the data it was trained on, not from feedback loops or interactions with users.

C. While the training data contains the problematic features, the bias was introduced by the act of selecting those features for inclusion, not by the data's inherent inaccuracy.

1. Mehrabi, N., et al. (2021). A Survey on Bias and Fairness in Machine Learning. ACM Computing Surveys, 54(6), Article 115. Page 6, Section 3.1, "Bias can be introduced... by selecting features that are proxies for sensitive attributes (e.g., using zip code as a proxy for race)." DOI: https://doi.org/10.1145/3457607.

2. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). NIST.AI.100-1. Page 20, Section 4.2.2, "Systemic and statistical biases can be encoded in the data... Biases can also be introduced in the feature engineering and selection process."

3. Stanford University. (2023). CS224N: NLP with Deep Learning, Lecture on AI Ethics. The courseware discusses how feature selection can introduce societal biases, such as using seemingly neutral data points that are highly correlated with protected attributes.