A data engineer must orchestrate a data pipeline that consists of one AWS Lambda function and one AWS Glue job. The solution must integrate with AWS…

AWS Step Functions is a serverless orchestration service that allows developers to coordinate multiple AWS services into flexible workflows. It provides direct, state-based integration for both AWS Lambda (Lambda:Invoke) and AWS Glue (Glue:StartJobRun). This serverless nature means there is no underlying infrastructure like servers or clusters to provision, patch, or manage, directly fulfilling the requirement for the "LEAST management overhead." The visual workflow also simplifies development, monitoring, and debugging of the multi-step pipeline.

1. AWS Step Functions Developer Guide: "AWS Step Functions is a serverless orchestration service that lets you combine AWS Lambda functions and other AWS services to build business-critical applications." It explicitly lists AWS Glue and AWS Lambda as supported service integrations.

Source: AWS Step Functions Developer Guide

"What is AWS Step Functions?" and "Supported AWS Service Integrations".

2. AWS Step Functions Developer Guide - Managing AWS Glue Jobs: "You can use Step Functions to start an AWS Glue job... Step Functions can wait for the job to complete

and then retrieve the results." This confirms the direct integration required by the scenario.

Source: AWS Step Functions Developer Guide

"Manage AWS Glue jobs with Step Functions".

3. AWS Glue Developer Guide - AWS Glue Workflows: "A workflow in AWS Glue is a container for a set of related jobs

crawlers

and triggers... You can use a workflow to create a complex multi-job extract

transform

and load (ETL) pipeline." The documentation focuses on orchestrating Glue-native components

not external services like Lambda as a primary step.

Source: AWS Glue Developer Guide

"Overview of workflows in AWS Glue".

4. AWS Whitepaper - Practicing Continuous Integration and Continuous Delivery on AWS: This paper discusses the operational burden of self-managed solutions. "When you manage your own... servers on Amazon EC2

you are responsible for scaling

managing

and maintaining the infrastructure... AWS-managed services... reduce your operational overhead." This principle supports choosing a managed service (Step Functions) over a self-hosted one (Airflow on EC2/EKS).

Source: AWS Whitepaper: "Practicing Continuous Integration and Continuous Delivery on AWS"

section on "Operational Responsibility".