CCDAK practice question 7 of 25

The exactly once guarantee in the Kafka Streams is for which flow of data?

Full set: 150 questions
Answer options
A: Kafka => Kafka Correct answer
B: Kafka => External
C: External => Kafka
Correct Answer:
A. Kafka => Kafka
Explanation

Kafka Streams provides an end-to-end exactly-once processing guarantee specifically for data flows that are contained entirely within the Kafka ecosystem. This is often referred to as a "Kafka-to-Kafka" flow. The guarantee is achieved by leveraging Kafka's transactional capabilities. When processing.guarantee is set to exactlyoncev2 (or exactlyonce), the entire read-process-write cycle—reading from a source topic, updating state, and writing to a sink topic—is executed as a single, atomic transaction. This ensures that consumer offsets are committed only when the corresponding output has been successfully written, preventing both data loss and duplicate processing in the event of a failure.

Why Incorrect

B. Kafka => External: Kafka Streams cannot extend its transactional guarantees to external systems (e.g., a database or a REST API), as it has no control over their transactional boundaries.
C. External => Kafka: When data originates from an external system, Kafka Streams cannot guarantee that the data was read exactly once from that source, as the source is outside its transactional control.

References

1. Confluent Documentation, "Processing Guarantees in Kafka Streams":

"Exactly-once semantics is the guarantee that a processed event affects the final state exactly one time. Kafka Streams provides this guarantee for any Kafka-to-Kafka data flows, where your application reads from Kafka topics, processes the data, and writes the results back to Kafka topics."

Source: Confluent Developer Documentation, streams/developer-guide/processing-guarantees.html, Section: "Exactly-once semantics".

2. Kreps, J. (2017). "Exactly-once Semantics are Possible: Here’s How Kafka Does it." Confluent Blog:

This foundational blog post by Kafka's co-creator explains the mechanism. It details how the transactional producer and consumer work together to achieve exactly-once semantics for streams of data flowing from one Kafka topic to another.

Source: Confluent Blog, published May 23, 2017. Section: "The solution: transactions".

3. Shapira, G., et al. (2021). Kafka: The Definitive Guide, 2nd Edition. O'Reilly Media.

Chapter 10, "Stream Processing," discusses the processing guarantees. It explicitly states that the exactly-once guarantee in Kafka Streams applies to operations where both the input and output are Kafka topics, by bundling reads, processing, and writes into atomic transactions.

Source: Chapter 10, Section: "Exactly-Once Semantics".

Scroll to Top