44 of 55. A data engineer is working on a real-time analytics pipeline using Spark Structured Streaming. They want the system to process incoming…

The trigger(processingTime="5 seconds") configuration explicitly sets up a micro-batch processing model where the Spark engine initiates a new batch at a fixed interval. This means the query will check for new data and start processing it every 5 seconds, regardless of whether the previous batch has finished. This directly fulfills the requirement for a real-time analytics pipeline that processes data in micro-batches at a fixed 5-second interval.

B. trigger(continuous="5 seconds") enables continuous processing, a low-latency mode, not the requested fixed-interval micro-batch model. The interval here is for checkpointing, not batching.

C. trigger(once=True) processes only a single batch of all available data and then stops the query, which is not suitable for a continuous, real-time pipeline.

D. The default trigger (no trigger specified) starts a new micro-batch immediately after the previous one completes. This does not guarantee a fixed interval; the timing depends on processing duration.

1. Apache Spark 3.5.0 Documentation

Structured Streaming Programming Guide

Section: Triggers.

"If you specify a processing time trigger

Spark will check for new data when the previous processing has completed. If new data is available

Spark will start a new processing. If no new data is available

Spark will sleep until the processing time is reached

and then check for new data again." This describes the behavior of Trigger.ProcessingTime("5 seconds")

confirming option A. The documentation also explicitly contrasts this with the default behavior (option D)

Trigger.Once() (option C)

and Trigger.Continuous() (option B).

2. Databricks Documentation

Structured Streaming

"Configure Structured Streaming trigger intervals".

Under the heading "Fixed interval micro-batches

" the documentation states: "You can configure structured streaming queries to run with a fixed batch interval using processingTime. If you do not specify a processingTime trigger

queries run using the default micro-batch setting...". This directly validates that processingTime is the correct parameter for achieving a fixed interval.