H13-711 practice question 20 of 30

Neither Spark nor Hadoop is suitable for iterative computing scenarios.

Full set: 201 questions
Answer options
A: True
B: False
Correct Answer:
FALSE
Explanation

The statement is false because Apache Spark is explicitly designed and optimized for iterative computing scenarios. Spark's core architectural feature, the Resilient Distributed Dataset (RDD), enables data to be cached in memory across multiple operations. This in-memory processing capability allows iterative algorithms, such as those used in machine learning (e.g., K-Means, Logistic Regression) and graph analysis (e.g., PageRank), to execute with high performance by avoiding the significant disk I/O overhead that occurs between iterations. While it is true that Hadoop's MapReduce framework is inefficient for iterative computing—as it writes intermediate results to the Hadoop Distributed File System (HDFS) after each pass, incurring high latency—Spark was created specifically to overcome this limitation. Therefore, the claim that neither platform is suitable is incorrect.

Why Incorrect

The statement would be considered "True" only if both Hadoop and Spark were unsuitable for iterative computing. However, Spark's fundamental design principle of in-memory data persistence between stages makes it exceptionally well-suited for such tasks, directly contradicting the premise of the question.

References

1. Huawei HCIA-Big Data V2.0 Training Material (H13-711). In the module covering Apache Spark, the documentation explicitly contrasts Spark with MapReduce, highlighting Spark's superior performance for iterative algorithms. It states: "A key advantage of Spark is its support for in-memory computing, which makes it ideal for iterative algorithms... Unlike MapReduce, which writes to HDFS after each job, Spark can keep data in memory between iterations, significantly speeding up performance." (Refer to: HCIA-Big Data V2.0 Training Material, Chapter on "Spark - A New-Generation Big Data Computing Engine", Section on "Spark vs. MapReduce").

2. Zaharia, M., Chowdhury, M., Franklin, M. J., Shenker, S., & Stoica, I. (2010). Spark: Cluster Computing with Working Sets. In Proceedings of the 2nd USENIX conference on Hot topics in cloud computing (HotCloud'10). This foundational academic paper on Spark introduces the framework as a solution to the inefficiency of MapReduce for a specific class of applications. The abstract states: "We find that current cluster computing frameworks are inefficient for two important classes of applications: iterative algorithms and interactive data mining tools... We propose a new framework called Spark that supports these applications while retaining the scalability and fault tolerance of MapReduce." (Page 1, Abstract). DOI: Not available for this specific workshop paper, but it is a foundational and widely cited source in the field.

3. University of California, Berkeley. CS 186/286, Introduction to Database Systems, Lecture 21: Spark. Course materials explain the limitations of MapReduce for iterative jobs and how Spark's RDDs solve the problem. The lecture notes state: "MapReduce is ill-suited for iterative jobs... Each iteration is a separate MapReduce job. This means that data has to be read from HDFS and written back to HDFS at every iteration... Spark avoids this by keeping data in memory." (Refer to: UC Berkeley EECS, CS 186/286 Lecture Slides on Spark).

Scroll to Top