CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice question 1 of 25
You have: DataFrame A: 128 GB of transactions DataFrame B: 1 GB user lookup table Which strategy is correct for broadcasting?
Full set: 135 questionsThe most effective join optimization strategy in this scenario is the broadcast hash join. This strategy involves sending the smaller DataFrame (DataFrame B, 1 GB) to every executor node in the cluster. By doing this, each executor has a complete local copy of the user lookup table. The join can then be performed on each partition of the large DataFrame (DataFrame A, 128 GB) locally, without requiring the costly network operation of shuffling the massive 128 GB DataFrame. This significantly improves performance by minimizing data movement across the network.
A. The primary performance gain from a broadcast join comes from avoiding the shuffle of the large DataFrame, not the small one being broadcasted.
C. Attempting to broadcast the large DataFrame (128 GB) is incorrect. This would likely cause an OutOfMemoryError on the driver node as it tries to collect the entire large DataFrame.
D. This statement is factually incorrect; DataFrame A is the larger of the two DataFrames and should not be broadcasted. ---
1. Apache Spark Official Documentation
SQL Tuning Guide: In the section on Join Optimization
the documentation states: "Spark uses broadcast join instead of hash join when the size of one side of the join data is smaller than the spark.sql.autoBroadcastJoinThreshold... Spark broadcasts the smaller table to all executors... This avoids a shuffle of the data in the larger table." This directly supports broadcasting the smaller table (B) to avoid shuffling the larger one (A).
Source: Apache Spark 3.5.0 Documentation
Performance Tuning > SQL Tuning Guide > Join Optimization.
2. Databricks Official Documentation
Join optimization: The Databricks documentation on optimizations explains: "When you join a large table to a small table
a broadcast join can give you better performance than a merge join. A broadcast join broadcasts the small table to all nodes in the cluster and performs a hash join locally on each node
which reduces network I/O." This confirms that the smaller table is the candidate for broadcasting.
Source: Databricks Documentation > Workloads > Data engineering > Delta Lake > Optimizations > Join optimization.
3. Karau
H.
Konwinski
A.
Wendell
P.
& Zaharia
M. (2015). Learning Spark: Lightning-Fast Big Data Analysis. O'Reilly Media
Inc. In Chapter 5
"Working with Key/Value Data
" the authors explain the broadcast join mechanism: "If one of the RDDs is much smaller than the other... we can do a broadcast hash join... This avoids the cost of the shuffle." This principle is fundamental to Spark's architecture and directly applies to DataFrames.
Source: Chapter 5
section on "Joins".
Question breakdown
Key idea: A broadcast hash join ships the small table to every executor, so the large table can be joined in place without being shuffled.
How to reach the answer
- Compare the sizes: A is 128 GB and B is 1 GB, so only B is a realistic broadcast candidate.
- Broadcasting copies the whole table to the driver and to every executor, so the broadcast side must fit in memory.
- Once each executor holds B, it joins its local partitions of A directly.
- The benefit is that the 128 GB side is never shuffled, which is what option B states.
Each option
Common trap: Option A names the right table, so it is easy to accept without noticing that the stated benefit (B avoiding its own shuffle) is not the real reason broadcasting helps.
In practice: Fact-to-dimension joins are the classic case for broadcast(). Note that 1 GB is well above the default 10 MB spark.sql.autoBroadcastJoinThreshold, so you would normally need an explicit hint or a higher threshold, and you should confirm executors have the memory headroom.
Remember: Broadcast the small side so the big side never has to be shuffled.
Official documentation: Hints (Databricks SQL)
Drafted with AI assistance and checked against the answer key.
Keep practicing
Take the free CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice test
Ten exam-style questions with answers and explanations, plus the exam facts and study guides.
More questions
Other CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice questions
- Question 2A Spark DataFrame df is cached using the MEMORY_AND_DISK storage level, but the DataFrame…
- Question 344 of 55. A data engineer is working on a real-time analytics pipeline using Spark Struct…
- Question 7A developer wants to test Spark Connect with an existing Spark application. What are the …
- Question 8A data engineer is running a Spark job to process a dataset of 1 TB stored in distributed…
- Question 10A data engineer is building a Structured Streaming pipeline and wants the pipeline to rec…
- Question 11Given a DataFrame df that has 10 partitions, after running the code: result = df.coalesce…
