A Big Data customer wants to run POI transformations on Spark on their production Hadoop cluster using Pentaho's Adaptive Execution Layer (AEL) What…

To install Pentaho's Adaptive Execution Layer (AEL) for use with Spark on a production Hadoop cluster, two key steps are required. First, the Spark application builder tool (spark-app-builder.sh) must be run. This utility packages the AEL daemon and all its dependencies into a single distributable zip file (ael-daemon-spark.zip). Second, the AEL daemon must be configured to run in YARN mode. This is accomplished by setting the ael.spark.master property to yarn in the config.properties file, which allows AEL to leverage the cluster's YARN resource manager for running Spark jobs, a standard practice for production Hadoop environments.

B. Configuring the AEL daemon in Local Mode is for single-machine development or testing and does not utilize the distributed resources of a production Hadoop cluster.

C. Oozie is a workflow scheduler used to manage Hadoop jobs. It is not the tool used to perform the initial installation of the AEL daemon service itself.

1. Hitachi Vantara Documentation

Pentaho 9.1

"Set Up the Adaptive Execution Layer (AEL) to Run on Spark": This official guide details the installation process.

Reference for A: In the section "Package the AEL Daemon into a Zip File

" the documentation states: "Run the Spark application builder tool to create the ael-daemon-spark.zip file..." This directly supports running the builder tool to obtain the daemon zip file.

Reference for D: In the section "Configure the AEL Daemon

" the documentation instructs users to edit the config.properties file and specifies: "Set the ael.spark.master property to yarn." This confirms the requirement to configure AEL for YARN mode on a cluster.

Reference for B & C: The same document outlines the installation process which does not involve an Oozie job. It also presents 'local' mode as an alternative to 'yarn' mode

clarifying its use for non-cluster environments

making it unsuitable for the production scenario described.