When a broker goes down, what will the Controller do?

The Controller is a designated broker in a Kafka cluster responsible for managing the state of partitions and replicas. When a broker fails, the Controller detects this failure (historically via ZooKeeper, now via the KRaft consensus protocol). For every partition whose leader was on the failed broker, the Controller initiates a leader election. It selects a new leader from the set of in-sync replicas (ISRs) for that partition. This action ensures that leadership for the affected partitions is redistributed among the remaining healthy brokers, maintaining data availability.

A. Wait for a follower to take the lead.

This is incorrect because the process is not passive or follower-initiated. The Controller actively manages and triggers the election process.

C. Become the leader for the topic/partition that needs a leader, pending the broker return in the cluster.

The Controller broker does not centralize leadership for all failed partitions onto itself; its role is to manage and delegate leadership to appropriate replicas.

D. Automatically elect the least loaded broker to become the leader for every orphan's partitions.

Leader election during failure is not based on broker load. The new leader is chosen from the partition's specific list of in-sync replicas (ISRs) to guarantee no data loss.

1. Apache Kafka Documentation, Section 5.2 Replication: "One of the brokers in the cluster also functions as the controller. The controller is responsible for managing the state of partitions and replicas and for performing administrative tasks like reassigning partitions. ... When the controller notices that a broker has shut down, it performs a leader election for all the partitions for which that broker was the leader."

2. Confluent Documentation, "Apache Kafka Internals: Understand the Guts of Kafka": "The controller is a broker that is responsible for maintaining the leader/follower relationship for all the partitions in a Kafka cluster. If a node by some reason fails, the controller will ensure that another replica will take the role of a partition leader."

3. Kreps, J., Narkhede, N., & Rao, J. (2011). Kafka: A Distributed Messaging System for Log Processing. Proceedings of the NetDB, 1-7. In Section 3.2, "Replication," the paper describes the role of the controller in managing broker failures: "The controller detects failures at the broker level and is responsible for electing a new leader for all affected partitions on a failed broker." (DOI not available for this specific workshop paper, but it is the foundational academic source).