DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-0 practice question 6 of 25

The code block displayed below contains an error. The code block should configure Spark to split data in 20 parts when exchanging data between executors for joins or aggregations. Find the error. Code block: spark.conf.set(spark.sql.shuffle.partitions, 20)

Full set: 180 questions
Answer options
A: The code block uses the wrong command for setting an option.
B: The code block sets the wrong option.
C: The code block expresses the option incorrectly. Correct answer
D: The code block sets the incorrect number of parts.
E: The code block is missing a parameter.
Correct Answer:
C. The code block expresses the option incorrectly.
Explanation

Correct code block: spark.conf.set("spark.sql.shuffle.partitions", 20) The code block expresses the option incorrectly. Correct! The option should be expressed as a string. The code block sets the wrong option. No, spark.sql.shuffle.partitions is the correct option for the use case in the question. The code block sets the incorrect number of parts. Wrong, the code block correctly states 20 parts. The code block uses the wrong command for setting an option. No, in PySpark spark.conf.set() is the correct command for setting an option. The code block is missing a parameter. Incorrect, spark.conf.set() takes two parameters. More info: Configuration - Spark 3.1.2 Documentation

Scroll to Top