CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice question 1 of 25

You have: DataFrame A: 128 GB of transactions DataFrame B: 1 GB user lookup table Which strategy is correct for broadcasting?

Choose your answer, then check it against the explanation and the full breakdown below.

You have: DataFrame A: 128 GB of transactions DataFrame B: 1 GB user lookup table Which strategy is correct for broadcasting?

Answer options

Correct Answer:

B

Explanation

The most effective join optimization strategy in this scenario is the broadcast hash join. This strategy involves sending the smaller DataFrame (DataFrame B, 1 GB) to every executor node in the cluster. By doing this, each executor has a complete local copy of the user lookup table. The join can then be performed on each partition of the large DataFrame (DataFrame A, 128 GB) locally, without requiring the costly network operation of shuffling the massive 128 GB DataFrame. This significantly improves performance by minimizing data movement across the network.

Why Incorrect

A. The primary performance gain from a broadcast join comes from avoiding the shuffle of the large DataFrame, not the small one being broadcasted.

C. Attempting to broadcast the large DataFrame (128 GB) is incorrect. This would likely cause an OutOfMemoryError on the driver node as it tries to collect the entire large DataFrame.

D. This statement is factually incorrect; DataFrame A is the larger of the two DataFrames and should not be broadcasted.

---

References

1. Apache Spark Official Documentation

SQL Tuning Guide: In the section on Join Optimization

the documentation states: "Spark uses broadcast join instead of hash join when the size of one side of the join data is smaller than the spark.sql.autoBroadcastJoinThreshold... Spark broadcasts the smaller table to all executors... This avoids a shuffle of the data in the larger table." This directly supports broadcasting the smaller table (B) to avoid shuffling the larger one (A).

Source: Apache Spark 3.5.0 Documentation

Performance Tuning > SQL Tuning Guide > Join Optimization.

2. Databricks Official Documentation

Join optimization: The Databricks documentation on optimizations explains: "When you join a large table to a small table

a broadcast join can give you better performance than a merge join. A broadcast join broadcasts the small table to all nodes in the cluster and performs a hash join locally on each node

which reduces network I/O." This confirms that the smaller table is the candidate for broadcasting.

Source: Databricks Documentation > Workloads > Data engineering > Delta Lake > Optimizations > Join optimization.

3. Karau

H.

Konwinski

A.

Wendell

P.

& Zaharia

M. (2015). Learning Spark: Lightning-Fast Big Data Analysis. O'Reilly Media

Inc. In Chapter 5

"Working with Key/Value Data

" the authors explain the broadcast join mechanism: "If one of the RDDs is much smaller than the other... we can do a broadcast hash join... This avoids the cost of the shuffle." This principle is fundamental to Spark's architecture and directly applies to DataFrames.

Source: Chapter 5

section on "Joins".

Question 1 of 25

Question breakdown

Tests: Choose the correct side of a join to broadcast
Difficulty: Easy

Key idea: A broadcast hash join ships the small table to every executor, so the large table can be joined in place without being shuffled.

How to reach the answer

Compare the sizes: A is 128 GB and B is 1 GB, so only B is a realistic broadcast candidate.
Broadcasting copies the whole table to the driver and to every executor, so the broadcast side must fit in memory.
Once each executor holds B, it joins its local partitions of A directly.
The benefit is that the 128 GB side is never shuffled, which is what option B states.

Each option

AIncorrect. It picks the right table but gives the wrong reason; the main gain is that the large DataFrame A is not shuffled, not anything about B itself.
BCorrect. Broadcasting the 1 GB lookup table lets each executor join its local partitions of the 128 GB table, so the large side is never shuffled.
CIncorrect. Broadcasting 128 GB would overwhelm the driver and executor memory, and avoiding a shuffle of the small side gains almost nothing.
DIncorrect. DataFrame A is the larger table, not the smaller one, so the premise is false.

Common trap: Option A names the right table, so it is easy to accept without noticing that the stated benefit (B avoiding its own shuffle) is not the real reason broadcasting helps.

In practice: Fact-to-dimension joins are the classic case for broadcast(). Note that 1 GB is well above the default 10 MB spark.sql.autoBroadcastJoinThreshold, so you would normally need an explicit hint or a higher threshold, and you should confirm executors have the memory headroom.

Remember: Broadcast the small side so the big side never has to be shuffled.

Official documentation: Hints (Databricks SQL)

Drafted with AI assistance and checked against the answer key.

Keep practicing

Take the free CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice test

Ten exam-style questions with answers and explanations, plus the exam facts and study guides.

Start the free practice test

More questions

Other CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice questions