H13-711 practice question 14 of 30

The order in which the YARN scheduler allocates resource requests, which of the following descriptions is correct.

Full set: 201 questions
Answer options
A: An arbitrary machine-> same rack-> Mochi resources
B: Any machine-> local resources-> same rack
C: Local resources-> Same rack-> Any machine Correct answer
D: Same rack-> Any machine-> Local Huangyuan
Correct Answer:
C. Local resources-> Same rack-> Any machine
Explanation

The YARN scheduler prioritizes data locality to minimize network I/O and improve job performance. When an ApplicationMaster requests a container for a data block, it provides location preferences. The scheduler attempts to fulfill the request in a specific hierarchical order. It first tries to allocate a container on the exact node where the data resides (node-local). If that fails due to resource unavailability, it relaxes the constraint and tries to find a container on any node within the same rack (rack-local). If a rack-local allocation is also not possible, the scheduler's final attempt is to allocate a container on any available node in the cluster (off-switch/any machine). This progressive relaxation ensures the best possible data locality is achieved.

Why Incorrect

A. This order is reversed, starting with the least desirable location (any machine) and moving towards the most desirable.
B. This order is illogical, as it does not follow the progressive relaxation of locality constraints from most specific to least specific.
D. This order incorrectly starts with the second preference (same rack) instead of the primary goal of node-local resource allocation.

References

1. Official Vendor Documentation (Apache Hadoop): The YARN architecture, on which Huawei's Big Data platform is based, follows this principle. The ResourceRequest API allows the ApplicationMaster to specify locality preferences. The scheduler then attempts to match these in the order of node-local, rack-local, and then off-switch.

Source: Apache Hadoop 3.3.6 Documentation, YARN Architecture, Section: "The Life of an Application". This section describes how the ApplicationMaster negotiates containers from the ResourceManager with resource requirements, including locality information.

2. University Courseware: University courses on distributed systems and big data explicitly teach this scheduling hierarchy as a fundamental concept of YARN.

Source: University of California, Berkeley, CS 162 Operating Systems and System Programming, Lecture 22: "Distributed Systems V: Hadoop/YARN", Slide 43. The slide details the YARN scheduling process: "Scheduler tries to match resource requests to nodes: 1. Tries to place on requested node. 2. Tries to place on requested rack. 3. Places on any node with available resources."

3. Academic/Technical Publications: Authoritative texts on Hadoop detail this process as a core feature of the YARN scheduler.

Source: White, T. (2015). Hadoop: The Definitive Guide (4th ed.). O'Reilly Media. Chapter 4, "YARN," in the section "Resource Requests," explains the data locality matching process: "YARN’s schedulers try to honor data locality requests... The best a scheduler can do is allocate a container on the node that the data is local to... If this is not possible, the scheduler tries to allocate a container on the same rack... As a last resort, the scheduler allocates a container on any node in the cluster."

Scroll to Top