You orchestrate ETL pipelines by using Cloud Composer One of the tasks in the Apache Airflow directed acyclic graph (DAG) relies on a third-party…

The onfailurecallback parameter is a standard feature of Apache Airflow operators. It allows you to specify a Python callable (a function) that will be executed only when a task instance fails. This is the most direct and appropriate mechanism to trigger a custom action, such as sending a notification, in response to a task failure. By assigning a function with notification logic to this parameter for the specific task, you ensure that you are alerted precisely when that task does not succeed after all configured retries have been exhausted.

A. The slamissed metric is triggered when a task does not complete within its defined Service Level Agreement (SLA) timeframe. A task can miss its SLA and still succeed, so this does not specifically track failures.

B. The slamisscallback function is executed when a task misses its SLA. This is related to the timeliness of task completion, not its success or failure status.

C. The onretrycallback function is executed each time a task is retried. A task might retry and then ultimately succeed, so this callback does not signify a final failure.

1. Apache Airflow Documentation, BaseOperator API Reference: The official documentation for Airflow's BaseOperator (the parent class for all operators) defines the callback parameters.

onfailurecallback: "a function to be called when a task instance of this task fails."

onretrycallback: "a function to be called when a task instance of this task is retried."

slamisscallback: "function to be called when a task instance of this task misses its SLA."

Source: Apache Airflow Documentation, apacheairflow.models.baseoperator.py.

2. Google Cloud Documentation, Writing DAGs (Airflow): Google's documentation for Cloud Composer explicitly directs users to the core Apache Airflow concepts for authoring DAGs, including the use of callbacks for notifications. It states, "Airflow provides several ways to set up notifications... A common way is to use Airflow's built-in callback mechanism."

Source: Google Cloud, Cloud Composer Documentation, "Writing DAGs (Airflow)", section on "Sending email notifications".

3. Google Cloud Documentation, Monitoring environments: This page lists the Airflow metrics exported to Cloud Monitoring. It includes airflow/dag/task/slamisses, confirming that SLA misses are tracked as a distinct metric from general task failures.

Source: Google Cloud, Cloud Composer Documentation, "Key metrics for monitoring environments", section "Airflow metrics".