H13-711 practice question 11 of 30

Spark's Executor can run multiple tasks.

Full set: 201 questions
Answer options
A: is correct Correct answer
B: error
Correct Answer:
A. is correct
Explanation

An Apache Spark Executor is a process launched on a worker node responsible for executing tasks. Each Executor is allocated a specific number of CPU cores (configurable via spark.executor.cores). A single Executor can run multiple tasks concurrently, with each task typically running in its own thread on a separate core. This parallel execution within a single Executor process is a fundamental aspect of Spark's architecture, enabling efficient utilization of resources on each worker node. For example, if an Executor is configured with 4 cores, it can run up to 4 tasks simultaneously.

Why Incorrect

B. error: This is incorrect. The ability of an Executor to run multiple tasks in parallel is a core design feature of Spark, essential for achieving high performance in distributed computing.

References

1. Apache Spark Official Documentation, "Configuration": The parameter spark.executor.cores is defined as "The number of cores to use on each executor." This explicitly states that an executor can be configured with multiple cores, and since each core runs a task, an executor can run multiple tasks. (See section: "Executor Cores").

2. Apache Spark Official Documentation, "Cluster Mode Overview": This document describes the Spark architecture, stating, "Each application gets its own executor processes, which stay up for the duration of the whole application and run tasks in multiple threads." This confirms that executors are multi-threaded to run tasks. (See section: "Components").

3. Zaharia, M., Chowdhury, M., Franklin, M. J., Shenker, S., & Stoica, I. (2012). Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. USENIX. In Section 3.1, "Spark Runtime," the paper describes the architecture where the driver launches tasks on worker nodes. The implementation detail of multiple tasks per executor process is a direct consequence of this architecture for resource utilization.

Scroll to Top