CCDAK practice question 4 of 25

A customer has many consumer applications that process messages from a Kafka topic. Each consumer application can only process 50 MB/s. Your customer wants to achieve a target throughput of 1 GB/s. What is the minimum number of partitions will you suggest to the customer for that particular topic?

Full set: 150 questions
Answer options
A: 10
B: 20 Correct answer
C: 1
D: 50
Correct Answer:
B. 20
Explanation

The maximum parallelism of a Kafka consumer group is determined by the number of partitions in the topic it consumes from. Each partition can be assigned to at most one consumer instance within the group at any given time. To calculate the minimum number of partitions, we first determine the number of parallel consumers required to meet the throughput target. Target Throughput: 1 GB/s (which is typically treated as 1000 MB/s in capacity planning). Throughput per Consumer: 50 MB/s. Minimum Consumers Required = Total Target Throughput / Throughput per Consumer Minimum Consumers Required = 1000 MB/s / 50 MB/s = 20 consumers. To allow 20 consumers to process data in parallel, the topic must have at least 20 partitions. Why Incorrect Options are Wrong
A. 10: With only 10 partitions, a maximum of 10 consumers can run in parallel, yielding a total throughput of 10 50 MB/s = 500 MB/s, which is below the target.
C. 1: A single partition allows for only one active consumer, limiting the throughput to 50 MB/s, which is insufficient.
D. 50: While 50 partitions would allow the target throughput to be achieved, the question asks for the minimum number required. 20 is the calculated minimum. References
1. Confluent Documentation, "Apache Kafka 101" Course: In the "Topics and Partitions" module, it is stated: "Partitions are the main way that Kafka provides for parallelism. For example, if a topic has four partitions, up to four consumers in a single consumer group can read from that topic in parallel." This establishes the direct link between the number of partitions and consumer parallelism. Source: Confluent Developer, Apache Kafka 101, Module: Topics and Partitions.
2. Narkhede, N., Shapira, G., & Palino, T. (2021). Kafka: The Definitive Guide (2nd ed.). O'Reilly Media. In Chapter 4, "Kafka Consumers," the section "Consumers and Consumer Groups" explains: "The number of partitions is the upper limit on the level of consumer parallelism... If you have more consumers than partitions, some of the consumers will be idle." This confirms that to run N consumers in parallel, you need at least N partitions. Source: Chapter 4, Page 70.
3. Apache Kafka Documentation, Section 1.4 "Distribution": The official documentation states, "By balancing consumers over the partitions, Kafka is able to provide both ordering and load balancing over a pool of consumer processes." This highlights that partitions are the mechanism for distributing load across consumers. Source: Apache Kafka Documentation, Introduction, Section 1.4.

Scroll to Top