What must be the first PDI step in a child transformation ingesting records from Kafka?

In a standard Pentaho Data Integration (PDI) design pattern involving parent and child transformations, the parent is responsible for a specific task, such as data ingestion. In this scenario, the parent transformation would contain the Kafka Consumer step to poll and retrieve batches of records from the Kafka topic.

The parent then passes these records to the child transformation for processing. The standard mechanism for a child transformation to receive data from its parent is the Get rows from result step. Therefore, this must be the first step in the child transformation to accept the incoming data stream originating from Kafka via its parent.

B. Get records from stream: This step is primarily used within PDI's AEL (Adaptive Execution Layer) for processing data with Spark or Flink, not for standard parent-child data hand-offs.

C. Kafka Consumer: This step would be located in the parent transformation, which is responsible for the initial ingestion from Kafka, not in the child transformation that processes the results.

D. Get data from Kafka: This is not an official or valid name for a step in Pentaho Data Integration. The correct step name is Kafka Consumer.

1. Hitachi Vantara Documentation - Pentaho Data Integration Steps: The official documentation describes the Get rows from result step as the method for a transformation to retrieve the output rows from a previous transformation or job entry in a parent job.

Source: Hitachi Vantara Pentaho Documentation

"Get rows from result" step reference. (e.g.

in Pentaho 9.x documentation

this is a standard

well-defined behavior).

2. Hitachi Vantara Documentation - Transformation Executor Step: The Transformation Executor step

used in a parent to call a child

has configuration options to pass data. The documentation specifies using this in conjunction with Get rows from result in the child transformation to create modular data flows.

Source: Hitachi Vantara Pentaho Documentation

"Transformation Executor" step reference. Section on "Passing and accepting rows".

3. Pentaho Data Integration Cookbook

Third Edition (Academic/Instructional Context): While a commercial book

its principles are based directly on official PDI functionality and are used in academic settings. It outlines design patterns where a parent job/transformation fetches data and passes it to a child transformation for reusable processing logic

starting with the Get rows from result step.

Reference: Chapter on Job and Transformation Orchestration

detailing parent/child data passing mechanisms. This pattern is a fundamental concept taught in PDI training.