CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice question 16 of 25

Given: python CopyEdit spark.sparkContext.setLogLevel("") Which set contains the suitable configuration settings for Spark driver LOG_LEVELs?

Full set: 135 questions
Answer options
A: ALL, DEBUG, FAIL, INFO
B: ERROR, WARN, TRACE, OFF Correct answer
C: WARN, NONE, ERROR, FATAL
D: FATAL, NONE, INFO, DEBUG
Correct Answer:
B. ERROR, WARN, TRACE, OFF
Explanation

Apache Spark utilizes the log4j logging library. The spark.sparkContext.setLogLevel() method allows developers to control the verbosity of Spark's logging output during runtime. The accepted log levels are a standard set defined by log4j. These levels, in order of increasing severity, are ALL, TRACE, DEBUG, INFO, WARN, ERROR, FATAL, and OFF. The set provided in option B (ERROR, WARN, TRACE, OFF) consists entirely of these valid log levels. The other options contain invalid level names.

Why Incorrect

A. The level FAIL is not a valid log4j or Spark log level. The standard level for reporting failures is ERROR or FATAL.
C. The level NONE is not a valid log level. To disable all logging, the correct level is OFF.
D. The level NONE is not a valid log level. The correct level to turn off logging is OFF.

References

1. Apache Spark 3.5.1 Documentation

PySpark API Reference: In the documentation for the pyspark.SparkContext.setLogLevel method

the valid log levels are explicitly listed.

Source: pyspark.SparkContext.setLogLevel(logLevel)

Quote: "Control our logLevel. This overrides any user-defined log settings. Valid log levels include: ALL

DEBUG

ERROR

FATAL

INFO

OFF

TRACE

WARN"

Link: https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.SparkContext.setLogLevel.html

2. Apache Spark 3.5.1 Documentation

Configuration: The "Configuration" page under the "Configuring Logging" section describes how Spark's logging is configured using log4j

which underpins the available log levels.

Source: Configuration - Configuring Logging

Quote: "Spark uses log4j for logging... You can configure it by adding a log4j2.properties file in the conf directory." This confirms the use of the log4j standard

whose levels are listed in the API reference.

Link: https://spark.apache.org/docs/latest/configuration.html#configuring-logging

Scroll to Top