CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice question 19 of 25
39 of 55. A Spark developer is developing a Spark application to monitor task performance across a cluster. One requirement is to track the maximum processing time for tasks on each worker node and consolidate this information on the driver for further analysis. Which technique should the developer use?
Full set: 135 questionsAn RDD (or Dataset) action such as reduce() executes on every partition, applies a commutative/associative function (e.g., max), and returns the single aggregated result to the driver. This exactly satisfies “track the maximum processing time … and consolidate this information on the driver.”
A. Broadcast variables are read-only values sent from the driver to executors; they cannot collect or update data back to the driver.
B. The Spark UI passively displays metrics; it is not a programmatic technique for collecting values inside an application.
D. Standard accumulators are intended for sums/counters and are write-only from tasks; although custom AccumulatorV2 can implement max, Spark docs discourage relying on accumulators to retrieve computation results.
1. Apache Spark 3.4.2 Programming Guide – Actions: reduce (returns aggregated value to driver) (docs/latest/rdd-programming-guide.html#actions).
2. Apache Spark 3.4.2 Programming Guide – Broadcast Variables (read-only, not for returning results) (docs/latest/rdd-programming-guide.html#broadcast-variables).
3. Apache Spark 3.4.2 Programming Guide – Accumulators (write-only from tasks; recommended only for debugging) (docs/latest/rdd-programming-guide.html#accumulators).
Keep practicing
Take the free CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice test
Ten exam-style questions with answers and explanations, plus the exam facts and study guides.
More questions
Other CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice questions
- Question 1You have: DataFrame A: 128 GB of transactions DataFrame B: 1 GB user lookup table Which s…
- Question 2A Spark DataFrame df is cached using the MEMORY_AND_DISK storage level, but the DataFrame…
- Question 344 of 55. A data engineer is working on a real-time analytics pipeline using Spark Struct…
- Question 7A developer wants to test Spark Connect with an existing Spark application. What are the …
- Question 8A data engineer is running a Spark job to process a dataset of 1 TB stored in distributed…
- Question 10A data engineer is building a Structured Streaming pipeline and wants the pipeline to rec…
