An organization utilizes AI-enabled mapping software to plan routes for delivery drivers. A driver following the AI route drives the wrong way down a…

Automation bias is the tendency for humans to over-rely on automated systems, trusting their outputs even when there is contradictory information from other sources. In this scenario, the driver excessively trusted the AI-enabled mapping software's recommendation and, as a result, ignored clear, real-world evidence—the numerous signs indicating a one-way street. The driver's failure to apply critical judgment and override the incorrect automated instruction is a classic manifestation of automation bias, where the human cedes decision-making authority to the machine, leading to an error.

A. Selection: Selection bias occurs during data collection when the data used to train a model is not a representative sample of the real world, which is not what the driver's action demonstrates.

B. Reporting: Reporting bias arises when the frequency of events or outcomes in the data does not reflect their real-world frequency. This is a data-level issue, not a human-computer interaction bias.

C. Confirmation: Confirmation bias is the tendency to favor information that confirms one's pre-existing beliefs. The driver was following the AI's instructions, not confirming a personal belief about the route.

1. ISACA. (2023). Artificial Intelligence Audit Framework. Section 3

"AI Risks and Controls

" identifies "Human biases (e.g.

automation bias

confirmation bias)" as a key risk area. It highlights the danger of individuals placing undue trust in AI system outputs

which directly aligns with the scenario.

2. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). In the "AI RMF Playbook

" under the Govern function (GOV-3)

it discusses human-AI interaction and the risk of automation bias

where "individuals may over-rely on AI system outputs without sufficient critical assessment."

3. Parasuraman

R.

& Manzey

D. H. (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors: The Journal of the Human Factors and Ergonomics Society

52(3)

381–410. Page 382 defines automation bias as "the use of automation as a heuristic replacement for vigilant information seeking and processing

" which describes the driver's failure to process the information from the road signs. https://doi.org/10.1177/0018720810376055

4. Goddard

K.

Roudsari

A.

& Wyatt

J. C. (2012). Automation bias: a systematic review of frequency

effect mediators

and mitigators. Journal of the American Medical Informatics Association

19(1)

121–127. The article defines automation bias as "the tendency to over-rely on automated aids and decision support systems

" which is precisely what the driver exhibited by following the AI route into a dangerous situation. https://doi.org/10.1136/amiajnl-2011-000099