HCE-5920 Practice Questions

Pentaho Spoon, also known as Pentaho Data Integration (PDI), is the design tool used to create ETL (Extract, Transform, Load) jobs and transformations. When connecting to a data source like a secure Hadoop cluster, Spoon provides the graphical interface to configure the connection details. This includes setting up security parameters such as Kerberos authentication and enabling user impersonation, which allows a Pentaho job to run on the Hadoop cluster as a specific end-user.

A. Pentaho Report Designer: This tool is for creating pixel-perfect reports. It consumes data connections but is not the primary tool for configuring complex, low-level security settings like Hadoop impersonation.

C. Pentaho Security Manager: This is a component for managing the Pentaho platform's internal security, such as users, roles, and permissions, not for configuring security for external data source connections.

D. Pentaho Server: The Pentaho Server is the execution environment where jobs, transformations, and reports are published and run. The connection configurations themselves are designed and created in Spoon.

1. Hitachi Vantara Documentation

Pentaho 9.3

"Set up connections to Hadoop": This official guide details the process of creating and configuring a connection to a Hadoop cluster. The entire procedure

including security configuration for Kerberos and impersonation

is performed within the Pentaho Spoon (PDI) client. The "Hadoop cluster" connection dialog

which contains these settings

is a core feature of Spoon. (See section: Connect to a Hadoop cluster with PDI).

2. Hitachi Vantara Documentation

Pentaho 9.3

"Hadoop cluster": This document describes the specific fields within the Hadoop cluster connection window in Spoon. It explicitly mentions configuring security settings

which is where impersonation would be enabled as part of the Kerberos or other security protocol setup. (See section: Hadoop cluster window

specifically the Security tab description).

The correct syntax for using an encrypted password in the kettle.properties file is VARIABLENAME=Encrypted . The Pentaho Data Integration (PDI) engine recognizes the Encrypted prefix and automatically decrypts the password string at runtime. The encr script generates this entire value string (Encrypted XYZabc123).

Although option D contains typos (a hyphen - instead of an equals sign = and XY2 instead of XYZ), it is the only option that correctly reflects the required structure of the Encrypted keyword followed by a space and the encrypted value.

A. This option uses invalid function-like syntax with parentheses and quotes, ("..."), which is not the correct format for a value in a Java properties file.

B. This option uses an incorrect keyword (Stringr) and invalid syntax. PDI would not recognize this format for decryption.

C. This option lacks the mandatory Encrypted prefix. PDI would incorrectly interpret the value as a plaintext password, leading to connection failure.

1. Hitachi Vantara Pentaho Documentation

"Use Variables to Secure Database Connection Passwords": This official guide explicitly shows the correct syntax for using encrypted passwords in the kettle.properties file. In the "Set Variables in kettle.properties" section

it provides a clear example: DBPASSWORD=Encrypted 2be98afc86aa7f2e4cb79ce10df90acde. This confirms the KEY=Encrypted VALUE format.

Source: Hitachi Vantara Pentaho Documentation. Specific section on securing database passwords.

2. Hitachi Vantara Pentaho Documentation

"Encrypt Passwords in Configuration Files": This document details the usage of the encr.sh (or encr.bat) utility. It states

"The PDI client encrypts the password and returns the encrypted password to the command line with the prefix Encrypted." It then instructs the user to copy the entire output string into the relevant configuration file

reinforcing that the Encrypted prefix is part of the value itself.

Source: Hitachi Vantara Pentaho Documentation. Section on password encryption utilities.

In the Pentaho Data Integration (PDI) Spoon client, free-form text fields that support variable substitution are marked with a small icon, typically a green or grey diamond containing a dollar sign ($). This visual cue indicates that you can use the ${VARIABLENAME} or %%VARIABLENAME%% syntax. Additionally, pressing CTRL+SPACE within these fields usually displays a list of available variables for insertion. The symbols listed in the options (V, @, ') are not the standard indicators used in the PDI interface for this purpose.

A. A "V" symbol is not the standard PDI UI indicator for a variable-enabled field.

B. An "@" symbol is not the standard PDI UI indicator for a variable-enabled field.

C. A "'" (single quote) symbol is not the standard PDI UI indicator for a variable-enabled field.

D. Using all caps is a common naming convention for variables, not a UI marker on a field indicating its capability to accept variables.

1. Hitachi Vantara Official Documentation

Pentaho 9.3

"Use Variables in Steps and Entries": This official guide specifies the correct indicator. It states

"Many text boxes in step and entry dialog boxes have a key combination shortcut

CTRL-SPACE

that you can use to select from a list of available variables. Text boxes that have this feature are marked with a small

green diamond-shaped icon with a dollar sign in it."

2. Hitachi Vantara Official Documentation

Pentaho 9.3

"PDI steps and entries reference": In the documentation for numerous steps

such as the "Text file input" or "Table input" steps

screenshots of the dialog boxes consistently show the dollar sign ($) icon next to fields that accept variables (e.g.

the "File or directory" field or SQL query text areas). This provides visual confirmation of the correct UI element.

To install Pentaho's Adaptive Execution Layer (AEL) for use with Spark on a production Hadoop cluster, two key steps are required. First, the Spark application builder tool (spark-app-builder.sh) must be run. This utility packages the AEL daemon and all its dependencies into a single distributable zip file (ael-daemon-spark.zip). Second, the AEL daemon must be configured to run in YARN mode. This is accomplished by setting the ael.spark.master property to yarn in the config.properties file, which allows AEL to leverage the cluster's YARN resource manager for running Spark jobs, a standard practice for production Hadoop environments.

B. Configuring the AEL daemon in Local Mode is for single-machine development or testing and does not utilize the distributed resources of a production Hadoop cluster.

C. Oozie is a workflow scheduler used to manage Hadoop jobs. It is not the tool used to perform the initial installation of the AEL daemon service itself.

1. Hitachi Vantara Documentation

Pentaho 9.1

"Set Up the Adaptive Execution Layer (AEL) to Run on Spark": This official guide details the installation process.

Reference for A: In the section "Package the AEL Daemon into a Zip File

" the documentation states: "Run the Spark application builder tool to create the ael-daemon-spark.zip file..." This directly supports running the builder tool to obtain the daemon zip file.

Reference for D: In the section "Configure the AEL Daemon

" the documentation instructs users to edit the config.properties file and specifies: "Set the ael.spark.master property to yarn." This confirms the requirement to configure AEL for YARN mode on a cluster.

Reference for B & C: The same document outlines the installation process which does not involve an Oozie job. It also presents 'local' mode as an alternative to 'yarn' mode

clarifying its use for non-cluster environments

making it unsuitable for the production scenario described.

In Pentaho Data Integration (PDI), job workflow is controlled by hops that connect job entries. To execute a specific action, such as sending a notification, only when a job entry fails, you must use the "On Failure" hop. This hop type is represented by a red arrow and is also referred to as the "false evaluation" hop. It ensures that the subsequent job entry (e.g., a "Mail" or "Write to Log" step) is triggered exclusively upon the failure of the preceding entry, which directly addresses the scenario's requirement for failure-specific notification.

A. Define the true evaluation hop out of the job entry that you are going to verify.

This hop (On Success) executes the next step only when the job entry completes successfully, which is the opposite of the required failure-based trigger.

C. Define the unconditional evaluation hop out of the job entry that you are going to verify.

This hop executes the next step regardless of success or failure, so it cannot be used to specifically identify and act upon a failure event.

D. Do not define any evaluation hop out of the Job entry that are going to verify.

This would terminate the workflow path after the job entry, preventing any subsequent notification step from being executed at all.

1. Hitachi Vantara Documentation

Pentaho Data Integration Concepts

"Jobs". This official guide explains the fundamental components of PDI jobs. In the section describing job workflow

it details the different types of hops. It specifies that hops can be conditional based on the success (true) or failure (false) of the previous job entry. The "On Failure" hop is explicitly designated for error handling routines.

2. Hitachi Vantara Documentation

Pentaho Spoon User Guide

"Working with Jobs" section. This document provides practical instructions for building jobs. It illustrates how to create a hop between two job entries and presents the dialog to choose the evaluation condition: "Follow when result is true" (Success)

"Follow when result is false" (Failure)

or "Unconditional". This directly supports using the "false" condition for failure paths.

A file format is considered splittable in HDFS if a large file can be logically divided into chunks that can be processed independently and in parallel by different nodes.

- Text files (.txt): These are splittable because data is typically separated by newlines. A processing framework like MapReduce can read a block of data, and if the split starts in the middle of a line, it can easily discard the first partial line and synchronize at the beginning of the next full line.

- Parquet: This is a columnar storage format designed for parallel processing in the Hadoop ecosystem. It structures data into row groups, which are designed to be read independently, making the format inherently splittable.

- Avro: This is a row-based format that includes synchronization markers between blocks of data. These markers allow a reader to seek to any point in a file, find the next marker, and begin reading, which makes Avro files splittable.

B. xml: Standard XML files are not inherently splittable because there is no reliable way to find the start or end of a record within an arbitrary file block without parsing from the beginning.

D. xisx: Assuming this is a typo for XLSX (Excel), this format is a compressed ZIP archive. Compressed archives cannot be split for parallel processing as they must be read sequentially from the start to be decompressed.

1. Avro Splittability: The official Apache Avro specification details the Object Container File format

which includes periodic "sync markers." These markers enable splitting. A reader can seek to any position in a file and then scan forward to the next sync marker to begin processing records.

Source: Apache Avro™ 1.11.1 Specification

Section: "Object Container Files". The specification states

"A sync marker is used to permit efficient splitting of files for parallel processing."

2. Parquet Splittability: The Apache Parquet format is explicitly designed for parallel processing. Data is organized into "row groups

" and the file's metadata allows readers to seek directly to the start of any row group

making it highly splittable.

Source: Apache Parquet Official Documentation

"Apache Parquet Format" section. The documentation describes the file structure consisting of row groups

which are the fundamental unit of parallel execution.

3. Text File Splittability: Hadoop's TextInputFormat treats plain text files as splittable. Its LineRecordReader can handle splits that occur in the middle of a line by reading past the initial partial line and starting at the next full newline character.

Source: Courseware from the University of California

Berkeley

CS 162: Operating Systems and System Programming. Lecture 22: "Distributed File Systems

" slide 32

discusses how MapReduce splits input files and how LineRecordReader handles records that cross split boundaries.

When the 'Database Lookup' step in Pentaho Data Integration (PDI) is configured to cache and "Load all data from table," it pre-loads the entire lookup table into memory before processing the first input row. This has three key implications for the data flow:

1. The entire dataset is stored in the Java Virtual Machine's (JVM) heap, meaning sufficient heap space must be allocated to prevent OutOfMemoryError exceptions (B).

2. Once cached, the comparisons between the input stream's keys and the cached keys are performed in memory. By default, these Java-based string comparisons are case-sensitive (C).

3. The step is designed to return a single set of values for each input row. If multiple rows in the lookup table match the input key, the step will use the data from only the first matching row it encounters in the cache (E).

A. This is incorrect. The step performs an enrichment (similar to a left outer join). If no match is found, the new fields are populated with default values (nulls), and the original row is passed through.

D. This is a best practice for predictable results, not a requirement. The step will execute if multiple matches exist, but it will non-deterministically use only the first match found.

1. Hitachi Vantara - Pentaho Documentation

"Database lookup" step:

Reference for B: The documentation describes the "Load all data from table" option for the cache. This action inherently requires memory. The "Pentaho Performance Tuning" guide frequently discusses allocating sufficient heap space (-Xmx) for transformations that cache large datasets. Section: Cache.

Reference for E: The documentation explicitly states

"If you expect more than one row to be returned

the returned value will be the first one encountered." This confirms that only one matching row is used. Section: Options.

Reference for A (Incorrectness): The presence of fields to specify default values for lookup fields when a match is not found confirms that rows are not dropped

but rather are passed on with null/default values. Section: The fields to return.

2. Pentaho Data Integration Official Documentation (General Principles):

Reference for C: The underlying engine for PDI is Java. In-memory lookups using standard Java data structures like HashMap rely on the equals() and hashCode() methods of key objects. For java.lang.String

these methods are case-sensitive. This is a fundamental principle of the platform's operation when performing in-memory comparisons. This behavior is detailed in general Java and PDI developer guides.

The two primary methods for scheduling Pentaho Data Integration (PDI) jobs using external, command-line tools are with an OS-native scheduler like cron.

The most direct method is using the kitchen script, which is the designated command-line utility for executing PDI jobs. It can connect to the Pentaho repository to locate and run the specified job, and cron can be used to schedule its execution (C).

An indirect, but technically valid, method involves the pan script. While pan is designed to execute transformations, a transformation can contain a "Job Executor" step. This allows a wrapper transformation to call a PDI job. Therefore, one can use cron to schedule the pan script to run this wrapper transformation, which in turn executes the job (B).

A. Using a login script is not a reliable or standard enterprise scheduling method; it is dependent on user sessions and not suitable for unattended, time-based execution.

D. The Spoon client is the PDI design environment; it does not contain a native "Action > Schedule" feature for persistent scheduling. Scheduling is managed via the Pentaho Server or external tools.

1. Hitachi Vantara Documentation

Pentaho 9.1

"Use Carte

Pan

and Kitchen": This official document explicitly defines the roles of the command-line tools.

Section: "Kitchen": "Kitchen is a command line tool that executes PDI jobs. The job filename or repository location is specified on the command line." This supports answer C.

Section: "Pan": "Pan is a command line tool that executes PDI transformations. The transformation filename or repository location is specified on the command line." This clarifies that Pan's direct target is a transformation

not a job.

2. Hitachi Vantara Documentation

Pentaho 9.1

"Job Executor": This document describes the transformation step that makes answer B possible.

Section: "Job Executor": "The Job Executor step allows you to run a job from within your transformation... You can specify the job to execute by filename or by reference from a repository." This supports the indirect method described in answer B.

3. Hitachi Vantara Documentation

Pentaho 9.1

"Tools to Automate

Schedule

and Monitor Pentaho": This document outlines the standard scheduling approaches.

Section: "Use Kitchen or Pan with a Scheduler": "You can use Kitchen or Pan with your operating system's scheduler

such as Cron for Linux and macOS

or Task Scheduler for Windows

to run jobs and transformations at specific times." This confirms that using cron with these scripts is the intended method.

The HTTP job entry in Pentaho Data Integration (PDI) is specifically designed to interact with web servers using the HTTP or HTTPS protocol. It can be configured to perform a GET request to a specified URL, which is the standard method for retrieving or downloading files from a website. The entry allows you to define the target URL and specify a local file path where the downloaded content will be saved, directly fulfilling the requirement of the task.

B. Sqoop import: This job entry is used for transferring bulk data between Hadoop and structured datastores like relational databases, not for downloading files from a web server.

C. Get a files with FTP: This entry uses the File Transfer Protocol (FTP) to download files from an FTP server, which is a different protocol and server type than a standard website (HTTP/HTTPS).

D. Copy Files: This job entry is used for copying files and folders within accessible file systems (local or network shares), but it does not support web protocols like HTTP.

1. Hitachi Vantara - Pentaho Documentation: The official documentation describes the HTTP job entry's function. It states

"The HTTP job entry allows you to do a single HTTP call. You can specify the URL to call and a target file to save the result to."

Source: Pentaho Wiki

"HTTP (job entry)". (The Pentaho Wiki serves as a primary official documentation source for PDI components).

2. Hitachi Vantara - Pentaho Documentation: The documentation for the "Get a file with FTP" job entry confirms its specific purpose: "The Get a file with FTP job entry allows you to get files from a remote server using the File Transfer Protocol (FTP)." This distinguishes it from the HTTP protocol.

Source: Pentaho Wiki

"Get a file with FTP (job entry)".

3. Hitachi Vantara - Pentaho Documentation: The purpose of the "Copy Files" job entry is explicitly defined for file system operations: "The Copy Files job entry allows you to copy one or more files and/or folders to a destination folder."

Source: Pentaho Wiki

"Copy Files (job entry)".

The Repository Explorer in Pentaho Data Integration (PDI) is the primary interface for managing content stored within the Pentaho Repository. Its core purpose is to provide file management and version control functionalities. Users can perform standard organizational tasks such as moving, renaming, and deleting repository objects like jobs and transformations. Additionally, to support multi-user collaboration and prevent conflicting edits, the Repository Explorer allows users to manually lock and unlock these assets. Creating new jobs/transformations is done in the main design canvas, and managing JDBC drivers is a separate configuration task.

A. New jobs and transformations are created using the "File > New" menu in the main Spoon application canvas, not directly within the Repository Explorer pane.

C. JDBC drivers are not uploaded via the Repository Explorer. They must be manually placed in the appropriate library directory (e.g., .../data-integration/lib) of the PDI client.

1. Hitachi Vantara Documentation - Work with the Pentaho Repository in PDI: This official guide details the functionalities of the Repository Explorer.

For option B (Move

Rename

Delete): The sections "Move Files and Folders

" "Rename Files and Folders

" and "Delete Files and Folders" explicitly describe these capabilities.

For option D (Lock

Unlock): The section "Lock and Unlock Files" states

"When you open a file from the Pentaho Repository

it is automatically locked... You can also manually lock and unlock files."

For option A (Create): The document shows that creating new content is initiated from the main "File" menu

not the explorer.

2. Hitachi Vantara Documentation - Work with Database Connections: This document explains the prerequisite for database connectivity.

For option C (JDBC Drivers): The section "Prerequisites" clearly states

"The appropriate database driver for the connection must be installed in the .../data-integration/lib folder for the PDI client..." This confirms driver management is a file-system-level task

not a Repository Explorer function.