CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice question 7 of 25

A developer wants to test Spark Connect with an existing Spark application. What are the two alternative ways the developer can start a local Spark Connect server without changing their existing application code? (Choose 2 answers)

Full set: 135 questions
Answer options
A: Execute their pyspark shell with the option --remote "https://localhost"
B: Execute their pyspark shell with the option --remote "sc://localhost" Correct answer
C: Set the environment variable SPARK_REMOTE="sc://localhost" before starting the pyspark shell Correct answer
D: Add .remote("sc://localhost") to their SparkSession.builder calls in their Spark code
E: Ensure the Spark property spark.connect.grpc.binding.port is set to 15002 in the application code
Correct Answer:
B. Execute their pyspark shell with the option --remote "sc://localhost"; C. Set the environment variable SPARK_REMOTE="sc://localhost" before starting the pyspark shell
Explanation

To connect an existing Spark application to a Spark Connect server without code modification, the connection must be configured externally. The two primary methods are using a command-line flag when launching the application (like the PySpark shell) or setting an environment variable. The --remote "sc://localhost" flag instructs the PySpark shell to operate in Connect mode and connect to the specified server. Similarly, setting the SPARKREMOTE="sc://localhost" environment variable before launching any Spark application (including a shell or a submitted job) will configure the default SparkSession to connect to the specified remote server, overriding the default behavior of creating a local Spark session.

Why Incorrect

A. The https:// protocol is incorrect for Spark Connect; the correct protocol is sc://.
D. This option requires modifying the application's source code by adding .remote() to the SparkSession.builder call, which the question explicitly prohibits.
E. spark.connect.grpc.binding.port is a server-side configuration property that specifies the port the Spark Connect server listens on, not how a client connects to it.

References

1. Apache Spark 3.5.1 Documentation

Spark Connect Overview

"Start Spark Connect server and client": This section explicitly demonstrates the two valid methods.

For the command-line option: "To use Spark Connect with the PySpark shell

run the following command in the Spark root directory: ./bin/pyspark --remote sc://localhost". This validates option B.

For the environment variable: "Alternatively

you can set the SPARKREMOTE environment variable

for example

export SPARKREMOTE="sc://localhost"". This validates option C.

The documentation also shows the code-based method

.remote("sc://localhost")

confirming why option D is a valid method in general but incorrect for this specific question's constraint.

Question breakdown

Tests
Use Spark Connect without modifying application code
Difficulty
Medium

Key idea: A Spark Connect client can be pointed at a server from outside the code, either with the --remote launch option or with the SPARK_REMOTE environment variable, using the sc:// scheme.

How to reach the answer

  1. The constraint is 'without changing their existing application code', which rules out any edit to the builder.
  2. Spark Connect URIs use the sc:// scheme, not https://.
  3. The pyspark --remote "sc://localhost" flag sets Connect mode at launch.
  4. Exporting SPARK_REMOTE="sc://localhost" makes the default SparkSession connect remotely without any code change.

Each option

  • AIncorrect. Spark Connect endpoints use the sc:// scheme, not https://.
  • BCorrect. Launching pyspark with --remote "sc://localhost" switches the shell into Spark Connect mode with no code change.
  • CCorrect. Setting SPARK_REMOTE before launch makes SparkSession creation use Spark Connect automatically.
  • DIncorrect. Adding .remote() to SparkSession.builder works, but it changes the application code, which the question forbids.
  • EIncorrect. This property sets the port the server listens on; it does not tell a client to use Spark Connect, and it would require a code change.

Common trap: Option D is a valid way to use Spark Connect, so it is tempting, but it breaks the explicit no-code-change constraint.

In practice: Teams often move existing PySpark scripts to Spark Connect by setting SPARK_REMOTE in the environment or CI config, so the same code runs locally or against a remote server.

Remember: For Spark Connect without code changes, use --remote sc://... or SPARK_REMOTE=sc://...

Drafted with AI assistance and checked against the answer key.

Keep practicing

Take the free CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice test

Ten exam-style questions with answers and explanations, plus the exam facts and study guides.

Start the free practice test

More questions

Other CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK-3-5-QUESTIONS practice questions

Scroll to Top