PEGACPDS88V1 practice question 25 of 25

Lift is key metric for the performance of the adaptive models. To measure lift, you need a____________

Full set: 140 questions
Answer options
A: Standard model as a benchmark
B: Control group with random actions Correct answer
C: Control group with a new action
D: New adaptive model
Correct Answer:
B. Control group with random actions
Explanation

Lift is a crucial metric that quantifies the performance improvement gained by using an adaptive model compared to a random approach. To calculate lift, the model's predictions (the treatment group) are compared against a baseline. This baseline is established by a control group, which is a statistically significant sample of the population that receives random actions or offers. The success rate of the control group represents the performance without the model's intelligence. The lift is the ratio or difference between the model's success rate and the control group's success rate.

Why Incorrect

A. A standard model can be a benchmark in a champion-challenger setup, but the fundamental calculation of lift is against a random baseline, not necessarily another model.
C. A control group receiving a "new action" would constitute another test group (a challenger), not a baseline for measuring lift against random chance.
D. A new adaptive model is not required to measure the lift of an existing model; a baseline established by a control group is the necessary component.

References

1. Pega Academy

Pega Decisioning Consultant (8.8) Mission

Module: Measuring model performance

Lesson: Measuring model performance.

This module explicitly states: "To measure the actual lift that is generated by the Next-Best-Action strategy

you can configure a control group. A control group is a small group of customers who are given random offers. The behavior of the control group is then compared to the behavior of the customers who were targeted by the strategy."

2. Pega Documentation

"Adaptive model details

" Section: Performance.

This document defines the metrics on the adaptive model rule form. For the "Lift" metric

it states: "The increase in success rate that is due to using the model. The lift is calculated by comparing the model performance with the performance of a random model that is represented by the control group."

3. Pega Documentation

"Configuring a control group."

This guide details the purpose and setup of control groups: "A control group is a random sample of customers who are not subjected to the main treatment but are instead given a random offer or no offer at all. By comparing the behavior of the control group to the behavior of the customers who were targeted by your strategies

you can measure the actual lift that is generated by your business logic."

Scroll to Top