CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice question 15 of 25

23 of 55. A data scientist is working with a massive dataset that exceeds the memory capacity of a single machine. The data scientist is considering using Apache Spark™ instead of traditional single-machine languages like standard Python scripts. Which two advantages does Apache Spark™ offer over a normal single-machine language in this scenario? (Choose 2 answers)

Full set: 135 questions
Answer options
A: It can distribute data processing tasks across a cluster of machines, enabling horizontal scalability. Correct answer
B: It requires specialized hardware to run, making it unsuitable for commodity hardware clusters.
C: It processes data solely on disk storage, reducing the need for memory resources.
D: It eliminates the need to write any code, automatically handling all data processing.
E: It has built-in fault tolerance, allowing it to recover seamlessly from node failures during computation. Correct answer
Correct Answer:
A. It can distribute data processing tasks across a cluster of machines, enabling horizontal scalability.; E. It has built-in fault tolerance, allowing it to recover seamlessly from node failures during computation.
Explanation

Apache Spark is a distributed computing system designed for large-scale data processing. Its primary advantage is the ability to partition data and distribute computational tasks across a cluster of machines, enabling horizontal scalability (A). This allows it to process datasets that are far too large to fit into the memory of a single machine. Furthermore, Spark is inherently fault-tolerant. It achieves this by tracking the lineage of transformations used to build a dataset (specifically, a Resilient Distributed Dataset or RDD). If a node fails and a data partition is lost, Spark can automatically recompute that partition on another available node, ensuring the job completes successfully (E).

Why Incorrect

B. It requires specialized hardware to run, making it unsuitable for commodity hardware clusters. This is incorrect. A key design principle of Spark is its ability to run efficiently on clusters of inexpensive, commodity hardware.
C. It processes data solely on disk storage, reducing the need for memory resources. This is incorrect. Spark's performance advantage comes from its ability to perform computations in-memory, though it can spill to disk if necessary. It does not operate solely on disk.
D. It eliminates the need to write any code, automatically handling all data processing. This is incorrect. Spark is a framework that requires developers to write code using its APIs (in Python, Scala, Java, R, or SQL) to define data processing logic.

References

1. Zaharia

M.

Chowdhury

M.

Franklin

M. J.

Shenker

S.

& Stoica

I. (2012). Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. In Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation (NSDI'12). USENIX Association.

Section 1 (Introduction): "We present Resilient Distributed Datasets (RDDs)

a distributed memory abstraction that lets programmers perform in-memory computations on large clusters in a fault-tolerant manner." This statement directly supports both the distributed nature (A) and fault-tolerance (E) of Spark.

DOI: https://doi.org/10.5555/2228298.2228301

2. Databricks Official Documentation

What is Apache Spark?

Section: "How does Apache Spark work?": "Apache Spark is a distributed processing system... Spark distributes data for processing and processes it in parallel across multiple machines." This directly supports the concept of distributed processing and horizontal scalability (A). The same section also states

"Spark’s primary abstraction is a distributed collection of items called a Resilient Distributed Dataset (RDD). RDDs... are fault-tolerant." This supports option (E).

3. Apache Spark™ Official Documentation

RDD Programming Guide.

Section: "Resilient Distributed Datasets (RDDs)": "The main abstraction Spark provides is a resilient distributed dataset (RDD)... RDDs are fault-tolerant

meaning that if a partition of an RDD is lost

it can be recomputed from its lineage." This explicitly confirms the built-in fault tolerance mechanism (E).

Scroll to Top