NCP-AIO practice question 10 of 15

Your Kubernetes cluster is running a mixture of AI training and inference workloads. You want to ensure that inference services have higher priority over training jobs during peak resource usage times. How would you configure Kubernetes to prioritize inference workloads?

Full set: 66 questions
Answer options
A: Increase the number of replicas for inference services so they always have more resources than training jobs.
B: Set up a separate namespace for inference services and limit resource usage in other namespaces.
C: Use Horizontal Pod Autoscaling (HPA) based on memory usage to scale up inference services during peak times.
D: Implement ResourceQuotas and PriorityClasses to assign higher priority and resource guarantees to inference workloads over training jobs. Correct answer
Correct Answer:
D. Implement ResourceQuotas and PriorityClasses to assign higher priority and resource guarantees to inference workloads over training jobs.
Explanation

The most effective and native Kubernetes method for ensuring one workload type is favored over another during resource contention is using PriorityClass. A PriorityClass object defines a mapping from a priority name to an integer value; higher values indicate higher priority. When a high-priority pod cannot be scheduled due to insufficient resources, the scheduler can preempt (evict) lower-priority pods to make room. ResourceQuotas complement this by setting aggregate resource limits per namespace. This ensures that the namespace for high-priority inference workloads has guaranteed resource allocations, preventing lower-priority training jobs in other namespaces from consuming all available cluster resources. This combination directly addresses the requirement for prioritization and resource guarantees.

Why Incorrect

A. Increasing replicas is a scaling strategy to handle load, not a prioritization mechanism to resolve resource contention between different workloads.
B. Namespaces provide logical isolation, and limits can prevent resource hogging, but this approach lacks a preemption capability for true prioritization.
C. Horizontal Pod Autoscaling (HPA) adjusts the number of replicas based on metrics; it does not influence scheduling priority during resource scarcity.

References

1. Kubernetes Documentation on Pod Priority and Preemption: This official source details the use of PriorityClass objects to control scheduling order and enable preemption. It explicitly states, "Priority indicates the importance of a Pod relative to other Pods... If a Pod cannot be scheduled, the scheduler tries to preempt (evict) lower priority Pods to make scheduling of the pending Pod possible."

Source: Kubernetes Documentation, "Pod Priority and Preemption," Section: "How to use priority and preemption".

2. Kubernetes Documentation on Resource Quotas: This document explains how ResourceQuotas are used to constrain aggregate resource consumption per namespace, which is essential for guaranteeing resources for high-priority workloads.

Source: Kubernetes Documentation, "Resource Quotas," Section: "What is a Resource Quota?".

3. NVIDIA AI Enterprise Deployment Guide: This guide discusses best practices for managing GPU resources in a multi-tenant Kubernetes environment. It recommends using Kubernetes scheduling features like priorities and quotas to ensure fair resource allocation and service levels for different AI/ML workloads, aligning with the principles in the correct answer.

Source: NVIDIA AI Enterprise Deployment Guide, Version 4.1, Chapter 6: "Kubernetes," Section: "Best Practices for Multi-Tenancy".

Scroll to Top