H13-711 practice question 27 of 30

The driver R in Spark on YARN mode can run on the client.

Full set: 201 questions
Answer options
A: True
B: False
Correct Answer:
TRUE
Explanation

The statement is correct. Apache Spark when running on a YARN (Yet Another Resource Negotiator) cluster supports two distinct deployment modes: client mode and cluster mode. In client mode, the Spark Driver process is launched and runs on the client machine from which the spark-submit command was executed. The Application Master's primary role in this mode is to negotiate resources (containers) for the Spark Executors from the YARN ResourceManager. The driver on the client then communicates directly with the executors running within the YARN cluster. Since client mode is a standard and valid configuration, the driver can indeed run on the client.

Why Incorrect

The alternative, 'False', would be an incorrect answer. This is because it completely disregards the existence and functionality of the client deployment mode. While it is true that in cluster mode the driver runs within a YARN-managed container inside the cluster, the question asks if the driver can run on the client, which is the specific purpose of client mode.

References

1. Huawei FusionInsight HD 6.5.1, Spark Application Development Guide, Section 4.1 "Submitting an Application": This official Huawei documentation specifies the --deploy-mode parameter for the spark-submit command. It explicitly states: "In client mode, the driver runs on the client. In cluster mode, the driver runs on one of the cluster nodes." This directly confirms that running the driver on the client is a supported mode.

2. Apache Spark 3.1.2 Documentation, "Running Spark on YARN", Section: "Launching Spark on YARN": As the foundational technology for Huawei's Spark distribution, the official Apache Spark documentation provides the definitive explanation. It states, "When running Spark on YARN, there are two deploy modes that can be used... In client mode, the driver runs in the client process, and the application master is only used for requesting resources from YARN."

3. UC Berkeley, EECS C100/C200A, "Lecture 19: Spark", Fall 2018, Slide 29, "Spark on a Cluster (YARN)": University courseware from UC Berkeley illustrates the Spark on YARN architecture. The diagram and accompanying text for "YARN Client Mode" clearly show the "Driver Program" located on the "Client Machine," separate from the "Cluster Nodes" where the YARN ResourceManager, NodeManagers, and Spark Executors reside. This validates the concept from an academic perspective.

Scroll to Top