Question: 40 I Which orchestration feature helps optimize AI workload placement across CPU and GPU resources in HPE Private Cloud AI?

HPE Private Cloud AI is built on a cloud-native architecture that uses Kubernetes as its container orchestration engine. The standard Kubernetes scheduler is responsible for placing containerized workloads (pods) onto appropriate compute nodes. For AI workloads that require GPU acceleration, the NVIDIA GPU device plugin is a critical component. This plugin exposes the NVIDIA GPUs as a schedulable resource within the Kubernetes cluster, allowing the scheduler to intelligently place GPU-dependent workloads on nodes equipped with the necessary hardware, thereby optimizing resource utilization across both CPUs and GPUs.

B. BIOS power settings: These are firmware-level configurations that manage the power and performance states of hardware components but do not orchestrate or schedule software workloads.

C. iLO remote management: HPE iLO is an out-of-band management tool for server hardware (power, health, remote console) and does not participate in application workload placement.

D. RAID controllers: These are hardware components responsible for managing disk drives in a storage array and have no function related to scheduling compute tasks on CPUs or GPUs.

---

1. NVIDIA AI Enterprise Documentation

GPU Operator: The HPE Private Cloud AI solution is co-engineered with NVIDIA and includes NVIDIA AI Enterprise software. The documentation for this software explains the role of the GPU Operator

which includes the device plugin. "The NVIDIA device plugin for Kubernetes is a Daemonset that allows you to automatically: Expose the number of GPUs on each node in your cluster; ... Run GPU enabled containers in your Kubernetes cluster." This directly enables the Kubernetes scheduler to manage GPU resources.

Source: NVIDIA AI Enterprise Documentation

"GPU Operator" section. (Refer to the latest version's deployment guide for NVIDIA AI Enterprise).

2. HPE Private Cloud AI Solution Brief: This document outlines the solution's architecture

stating it is a "fully integrated AI private cloud solution" that "includes the NVIDIA AI Enterprise software platform". This establishes the foundation on which the Kubernetes and NVIDIA components operate.

Source: HPE Private Cloud AI Solution Brief

Document ID a00138323enw

Published February 2024

Page 2.

3. Kubernetes Official Documentation

"Schedule GPUs": The official Kubernetes documentation details how device plugins

such as the NVIDIA GPU plugin

work. "Kubernetes provides a device plugin framework that you can use to advertise system hardware resources to the Kubelet. ... Instead of customizing the code for Kubernetes itself

vendors can implement a device plugin that you deploy either manually or as a DaemonSet. The targeted devices include GPUs..."

Source: Kubernetes Documentation

Concepts > Configuration > Schedule GPUs.