AB-100 practice question 35 of 35
You’re releasing an autonomous case-routing copilot that can open and close incidents across Dynamics 365 Customer Service and Sales. Production has…
Full set: 708 questionsThe requirements call for a repeatable, automated quality gate to prevent regressions in grounding and tool usage before weekly production releases, without using real customer data in non-prod environments. 'Agent evaluation with regression gates' directly addresses this. This process involves running the agent against a standardized, curated set of test prompts (a "golden set") that cover critical grounding and tool-use scenarios. The results are automatically compared against a baseline, and the release is blocked (gated) if key metrics degrade. This provides a reliable, repeatable, and automated quality control mechanism that fits within a CI/CD pipeline and respects data boundaries.
B. Manual testing is not repeatable or scalable for weekly releases and is prone to human error, failing the "repeatable test process" requirement.
C. A latency SLO gate only measures performance. It does not catch functional regressions in grounding or tool usage accuracy, which is a key requirement.
D. Canary releasing in production uses live data, which violates the constraint of not using real customer data outside the production region for testing.
1. Microsoft Power Platform
"Application lifecycle management (ALM) with Microsoft Power Platform
" Solution concepts section. This document outlines the principles of using solutions to package and deploy components
which is a prerequisite for automated testing and promotion through environments.
2. Microsoft Copilot Studio Documentation
"Test your copilot." This page describes the built-in testing capabilities
which are the foundation for building more advanced
automated evaluation suites.
3. Microsoft Azure AI Studio Documentation
"Evaluate copilots and AI models." This documentation details the framework for systematic evaluation of AI agents against predefined datasets and metrics
which is the core of the "Agent evaluation" process.
Keep practicing
Take the free AB-100 practice test
Ten exam-style questions with answers and explanations, plus the exam facts and study guides.
More questions
Other AB-100 practice questions
- Question 2A global support team uses Dynamics 365 Customer Service and wants Copilot to summarize c…
- Question 3A manufacturing group is deploying an agent that drafts maintenance work orders from oper…
- Question 5A global manufacturer is rolling out a Microsoft 365 Copilot agent for supplier onboardin…
- Question 6A regulated insurer is modernizing a model-driven claims app and wants an AI-driven works…
- Question 7A global retailer is standardizing prompt guidance for agents used in Copilot Studio and …
- Question 8A global manufacturer is piloting an agent that drafts incident summaries, computes KPI v…
