Which use case for an AI model to be used by a food delivery service would pose ethical risk to the organization?

This use case poses a significant ethical risk due to "algorithmic management," where an AI makes high-stakes decisions about human employment. Basing termination solely on a metric like "delivered orders per total hours worked" is inherently problematic. The model is likely to be biased, as it cannot account for contextual factors beyond a driver's control, such as restaurant delays, traffic conditions, or order distance. This can lead to unfair and discriminatory outcomes, directly impacting individuals' livelihoods without transparency or a clear process for appeal. Such automated decisions raise profound ethical questions about fairness, accountability, and the dehumanization of labor.

A. This is a customer remediation strategy. Using AI to identify and compensate customers for poor service is generally considered an ethically positive or neutral business practice.

C. This involves performance management of business entities (restaurants), not individuals. While it may create pressure, it does not carry the same direct ethical weight as terminating an individual's employment.

D. This is a standard business analytics function for strategic forecasting. It does not involve making automated decisions that directly and negatively impact the rights or livelihood of individuals.

1. Kellogg

K. C.

Valentine

M. A.

& Christin

A. (2020). Algorithms and the Organization of Work: A Review and Research Agenda. The Academy of Management Annals

14(2)

575-610. (Section: "Algorithms and the Evaluation of Work

" discusses how algorithmic performance metrics can be biased and lead to unfair worker evaluations and terminations

which is the core ethical risk in option B). https://doi.org/10.5465/annals.2018.0074

2. Stanford University. (2021). CS224N: Natural Language Processing with Deep Learning

Winter 2021. Lecture: "Ethics in NLP". (This lecture

and similar AI ethics courses

covers the concept of algorithmic bias and its societal impact. It explains how models trained on narrow or incomplete data can perpetuate and amplify unfairness

directly applicable to using a single metric for termination decisions).

3. European Commission's High-Level Expert Group on AI. (2019). Ethics Guidelines for Trustworthy AI. (Section I

"Foundations of Trustworthy AI

" pp. 12-14. The guideline on "Fairness" explicitly addresses the need to avoid unfair bias and ensure that algorithmic decisions do not lead to discriminatory or unjust outcomes

a principle directly violated by the scenario in option B).