Kafka Connect is running on a two node cluster in distributed mode. The connector is a source connector that pulls data from Postgres tables…

The connector configuration explicitly sets "tasks.max": "1", which restricts the entire data ingestion process for all three tables (users, payment, orders) to a single thread (task). This creates a significant performance bottleneck, as one task must sequentially poll all tables. By increasing the tasks.max value (e.g., to 3), you allow Kafka Connect to create more tasks. The JDBC source connector can then parallelize the work, potentially assigning each table to a separate task. These tasks can run concurrently on the available worker nodes, significantly improving throughput and reducing the data lag.

A. Increasing the number of Connect Nodes is ineffective because the connector is configured to use only one task, so there is no additional work to distribute to new nodes.

C. Increasing the number of partitions affects downstream consumer parallelism but does not speed up the source connector's ability to read data from the database.

D. Increasing the replication factor enhances data durability, not ingestion speed. It can slightly increase write latency, which would not help reduce the lag.

1. Confluent Documentation, Kafka Connect Concepts, Tasks: "For a source connector, tasks.max is the maximum number of tasks that can be created... For example, the JDBC Source Connector can create a task for each table it is configured to poll... The tasks are distributed among the Connect worker processes." This confirms that tasks.max is the primary mechanism for scaling a source connector.

2. Confluent Documentation, JDBC Source Connector for Confluent Platform, Configuration Properties: Under the tasks.max property description, it states: "The maximum number of tasks that should be created for this connector. The connector may create fewer tasks if it cannot find a way to partition the data." This directly links the configuration to the ability to partition and parallelize work.

3. Shapira, G., Narkhede, N., & Palino, T. (2021). Kafka: The Definitive Guide (2nd ed.). O'Reilly Media. In Chapter 11, "Kafka Connect," the section "Creating and Configuring Connectors" explains: "The tasks.max configuration controls the maximum number of tasks that a connector can be broken into... a JDBC connector might break the job by assigning a set of tables to each task. Increasing the number of tasks is the main way to scale a connector." (p. 289).