A project manager is preparing for an AI model evaluation. The model has shown an overall 70% accuracy rate, but the project key performance…

A significant gap between a model's performance (70% accuracy) and the required KPI (89%) most often points to a fundamental problem with the data. The principle of "Garbage In, Garbage Out" is paramount in AI projects. If the data used to train the model does not accurately reflect the real-world conditions in which the model will operate, the model cannot generalize effectively, leading to poor performance. Therefore, the first and most critical step in diagnosing such a performance issue is to investigate whether the training data is a representative sample of the production environment. This is a more foundational issue than algorithm choice or computational resources.

B. Inadequate computational power being used typically slows down or prevents the training of complex models; it does not directly cause low accuracy in an already trained model.

C. Failure to split datasets usually results in overfitting, where the model performs exceptionally well on data it has seen, leading to an artificially high and misleading accuracy score.

D. Incorrect selection of model algorithms is a valid reason for poor performance, but it should be investigated only after confirming the data's quality and representativeness are sound.

1. Ng, A. (2018). Machine Learning Yearning. Chapter 18: Your dev and test sets should come from the same distribution. This chapter emphasizes that a mismatch between the training data distribution and the real-world (dev/test set) distribution is a primary cause for a model performing poorly on the metrics that matter.

2. Sambasivan, N., Kapania, S., Highfill, H., et al. (2021). "Everyone wants to do the model work, not the data work": Data Cascades in High-Stakes AI. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. This paper highlights that data issues, including unrepresentative samples, are often the root cause of failures in AI systems. (DOI: https://doi.org/10.1145/3411764.3445518)

3. Zliobaite, I. (2010). Learning under concept drift: an overview. ITIM. This academic work provides a formal overview of concept drift, a situation where the statistical properties of the target variable change over time, making the initial training data unrepresentative and causing model accuracy to degrade. (ISSN: 1822-880X, Section 2).