39 of 55. A Spark developer is developing a Spark application to monitor task performance across a cluster. One requirement is to track the maximum…

An RDD (or Dataset) action such as reduce() executes on every partition, applies a commutative/associative function (e.g., max), and returns the single aggregated result to the driver. This exactly satisfies “track the maximum processing time … and consolidate this information on the driver.”

A. Broadcast variables are read-only values sent from the driver to executors; they cannot collect or update data back to the driver.

B. The Spark UI passively displays metrics; it is not a programmatic technique for collecting values inside an application.

D. Standard accumulators are intended for sums/counters and are write-only from tasks; although custom AccumulatorV2 can implement max, Spark docs discourage relying on accumulators to retrieve computation results.

1. Apache Spark 3.4.2 Programming Guide – Actions: reduce (returns aggregated value to driver) (docs/latest/rdd-programming-guide.html#actions).

2. Apache Spark 3.4.2 Programming Guide – Broadcast Variables (read-only

not for returning results) (docs/latest/rdd-programming-guide.html#broadcast-variables).

3. Apache Spark 3.4.2 Programming Guide – Accumulators (write-only from tasks; recommended only for debugging) (docs/latest/rdd-programming-guide.html#accumulators).