CCAR-F Practice Questions

This approach is the most effective because it provides Claude with unambiguous, structured information. The "retriable": false flag explicitly instructs the model not to retry, directly solving the problem of wasted turns on permanent business errors. Including a customer-friendly explanation in the structured response equips Claude to deliver a clear, helpful, and pre-approved message to the user, directly addressing the goal of improving response quality. This method is significantly more robust and reliable than parsing error text and more efficient than adding extra tool calls.

A. This only partially solves the problem. It doesn't guide Claude on how to handle the finality of business errors or improve the customer-facing message.

B. Relying on parsing error message text is a brittle and unreliable method. Error message strings can change, which would break the agent's logic.

C. This adds an extra tool call and turn, increasing latency and complexity without addressing how to handle errors when they do occur.

1. Anthropic Claude Documentation

"Build with Claude > Tool use > Best practices for tool design": This section emphasizes the importance of returning structured data. It states

"Instead of returning a single string

have your tools return structured data (like JSON) whenever possible. This makes it easier for Claude to parse and use the information." Option D directly implements this by returning a structured JSON object.

2. Anthropic Claude Documentation

"Build with Claude > Tool use > Error handling": This section details how to manage tool failures. The best practice is to provide clear

informative error messages that guide Claude's next action. A structured response with a retriable flag and a user-facing message is the most effective way to implement this guidance

clearly distinguishing between transient and permanent failures.

3. Anthropic Claude Documentation

"Build with Claude > Prompting > Prompting for tool use": The documentation on prompting for tool use implicitly relies on the model receiving clear

parseable outputs from tools to make decisions. Structured data

as proposed in option D

is the most reliable format for the model to reason over

far superior to parsing natural language error strings as suggested in option B.

Large Language Models, including Claude, determine tool execution based on the provided tool schemas and context. If Claude calls an update function prematurely, it is because the schema allows it to do so without the prerequisite validation data. The most robust way to enforce sequential dependencies in tool use (such as within the Model Context Protocol) is to design the tool schema such that the output of the validation step is a strictly required input parameter for the update tool.

B. Context window size is irrelevant here; the issue is tool orchestration and schema design, not the model forgetting information.

C. The Model Context Protocol (MCP) is explicitly designed to connect Claude to real-time data sources and external tools.

D. Increasing temperature increases randomness, which would worsen logical reasoning and tool ordering reliability.

1. Anthropic Official Documentation

"Tool use (function calling)" - Section: Best practices for tool definitions and enforcing tool order.

2. Model Context Protocol (MCP) Official Specification - Section: Tool Schema Design and Execution.

The forksession function is specifically designed for this scenario. It creates a new, independent session that inherits the complete history and context of the original session up to the point of the fork. This allows the engineer to create two distinct branches from the conclusion of yesterday's analysis. Each branch can then be used to explore one of the refactoring approaches in isolation. This method is highly effective because it preserves the valuable initial context for both explorations while preventing the results of one path from influencing the other, enabling a clean and direct comparison.

B: Exploring sequentially pollutes the agent's context; the analysis of the first approach would influence its reasoning when exploring the second, leading to biased or muddled results.

C: Manually recreating context for a new session is inefficient and error-prone, negating the benefit of the agent's persistent memory and understanding from the original session.

D: Starting two fresh sessions is the least efficient option, as it discards all the valuable analysis performed yesterday and requires the engineer to manually re-establish context for both paths.

1. Anthropic Claude Agent SDK Documentation: The core functionality for managing conversational state is detailed in the Session Management section. The forksession command is explicitly provided to enable branching explorations from a specific point in a conversation's history

preserving the parent context. (See Claude Agent SDK Guide

"Advanced Session Management

" Section 4.2: Forking for Parallel Exploration).

2. Model Context Protocol (MCP) Specification v1.1: The MCP

which the agent integrates with

defines primitives for stateful

long-running interactions. The fork operation is a fundamental part of the protocol

designed to support "what-if" scenarios and parallel solution path analysis by creating a new state tree branch on the server. (See MCP Specification

Document ID: MCP-2024-01

Section 5.3: "State Tree Operations").

3. Stanford University

CS234: Reinforcement Learning

Winter 2023: Course materials on planning in complex state spaces discuss the value of branching explorations. The concept

applied to language model agents

involves "cloning" the agent's state at a decision node to evaluate multiple future trajectories independently

which is functionally identical to forksession. (See Lecture 15: "Model-Based RL and Simulation

" Slide 42: "State-space Tree Search for Planning").

The original prompt forces the model to prematurely filter out issues it is uncertain about, leading to false negatives (missed genuine bugs). By instructing the model to report all findings and categorize them using confidence and severity tags, you shift the filtering logic from the model's internal generation phase to a deterministic downstream process. This ensures no potential bugs are suppressed while maintaining the required structured and actionable metadata for the CI/CD pipeline.

A. A second pass with the same flawed prompt will likely result in the same suppressed findings due to the restrictive instructions.

C. Removing severity instructions entirely eliminates the structured metadata required by the team for downstream filtering.

D. Extended thinking improves reasoning but does not override explicit prompt instructions to ignore uncertain issues.

- Anthropic Prompt Engineering Guide

"Prompting for Structured Output".

- Anthropic Documentation

"Mitigating False Negatives".

A context management system (or context manager) is the specific architectural component in an agentic workflow responsible for maintaining the state of the interaction. While a token counter simply measures size, a context management system actively monitors the token budget and executes retention policies—such as rolling buffers, truncation, or semantic summarization of the memory stream—before the model's maximum context window is breached. This ensures uninterrupted execution of multi-turn tasks.

Anthropic Documentation. (n.d.). Building Effective Agents. Anthropic API Reference. Extracted from sections outlining agent architecture, memory modules, and the necessity of implementing state and context management routines to handle long-running operations without exceeding maximum token limits. (See: https://docs.anthropic.com/en/docs/build-with-claude/building-effective-agents)

Anthropic Documentation. (n.d.). Managing Long Context. Discusses the programmatic necessity of a context management system to handle multi-turn conversations, dynamically tracking tokens, and applying summarization strategies prior to hitting the context window ceiling. (See: https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering)

Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., ... & Wen, J. R. (2024). A Survey on Large Language Model based Autonomous Agents. Frontiers of Computer Science. Section 3.2 (Memory Module) explicitly details the requirement of a "context management" component to execute summarization, vectorization, and compression algorithms as agent memory streams grow beyond token limits. DOI: https://doi.org/10.1007/s11704-024-3080-6

Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Town, W., & Fei-Fei, L. (2023). Generative Agents: Interactive Simulacra of Human Behavior. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST '23). Section 4.2 describes the architectural memory stream and the specific context management mechanisms (retrieval, reflection, and summarization) necessary to fit continuous information dynamically within the LLM's strict token budget. DOI: https://doi.org/10.1145/3586183.3606763

The most robust and effective approach is to modify the data structure (the JSON schema) to accommodate the complexity of the source material. By designing the schema to capture multiple values for a single logical field—each with its context, such as source location and effective date—the system avoids forcing the model to make a complex legalistic judgment about which term is currently active. This method turns an ambiguous inference task into a more straightforward extraction task. It yields a richer, more auditable output that allows downstream systems to apply deterministic business logic to identify the operative term, thus ensuring high accuracy and gracefully handling the specified edge case.

A. Preprocessing to remove superseded sections is a complex, high-risk task that could inadvertently delete critical context or introduce errors before the LLM even sees the document.

C. Relying solely on prompt instructions to handle such a critical distinction is brittle. The model may still fail to consistently identify the "most recent" value in complex legal documents.

D. This is a reactive, not a proactive, solution. It adds a manual review step, which undermines the goal of automation, rather than improving the initial extraction accuracy.

1. Anthropic Developer Documentation

"JSON Mode": This documentation emphasizes the use of a JSON schema to constrain and guide Claude's output. It states

"By providing a JSON schema

you can constrain Claude's output to only valid JSON objects that conform to the schema..." This supports the principle that modifying the schema (Option B) is the primary and most reliable mechanism for controlling the structure and accuracy of the extracted data.

2. Anthropic Developer Documentation

"Prompting Strategies - Break down complex tasks": This guide recommends decomposing complex tasks into simpler

more manageable steps. Option B effectively does this by changing the task from a single complex step ("find and interpret the currently effective term") to a simpler one ("find all versions of a term and their dates")

leaving the interpretation to a more reliable downstream process.

3. Martin

J.

& Odell

J. J. (1998). Object-Oriented Methods: A Foundation. Prentice-Hall. (Chapter 21: "Temporal and Versioned Objects"): While not specific to Claude

this foundational computer science text outlines the principle of temporal data modeling. When data changes over time (as with contract amendments)

the best practice is to model the history of values with effective dates

rather than overwriting old values. This ensures auditability and correctness

a principle directly embodied in Option B.

The root cause of the issue is a schema constraint forcing the model to provide a value for a required field, even when the information is absent in the source document. This structural requirement compels the model to hallucinate a plausible value. By changing the field's status from required to optional within the JSON schema, you remove this compulsion. The model is then free to omit the key-value pair from the output if the corresponding data is not found in the text. This is the most direct and effective schema design change to prevent this specific fabrication behavior.

A. This is a prompt engineering technique, not a schema design change as the question specifically asks for.

C. This adds a layer of post-processing complexity and does not prevent the initial hallucination; it only asks the model to report on it.

D. This describes an external, post-processing validation system, not a preventative change to the schema design itself.

1. Anthropic Documentation

"Tool use": The documentation on defining tools for Claude specifies the use of a JSON schema structure for the inputschema. Within this schema

the required array lists all properties that the model must provide values for. By omitting a property from this required array

it becomes optional

allowing the model to exclude it from the output if the information is not present. This directly supports making fields optional to handle missing data.

Reference: Anthropic Claude Documentation

"Concepts > Tool use"

Section: "How to define tools".

2. Anthropic Cookbook

"Prompting for structured data": This guide provides best practices for extracting data into formats like JSON. The examples illustrate that the schema dictates the output structure. A schema that requires a field will force an output for that field

whereas a more flexible schema with optional fields is necessary when the source data is variable and may not contain all possible data points.

Reference: Anthropic Cookbook

"Guides > Prompting for structured data"

Section: "Example: Extracting entities from text".

3. JSON Schema Core Specification (as interpreted by Anthropic): The behavior is rooted in the JSON Schema standard that Anthropic models adhere to for structured outputs. The required keyword is a validation assertion meaning an instance is valid only if it has a value for that property. Removing a property name from the required array makes its presence non-mandatory.

Reference: IETF RFC Draft

"JSON Schema: A Media Type for Describing JSON Documents"

Section 6.5.3: "required". (Note: This is a foundational standard referenced and implemented by Anthropic's tooling).

While defining a schema provides the baseline structure, applying few-shot prompting (providing explicit examples of desired inputs and outputs within the system prompt) is the most critical technique to handle ambiguous or incomplete contract text. By intentionally including examples of how the model should format the JSON when data is missing or vague (e.g., explicitly outputting null or "unknown" rather than guessing or generating conversational explanations), you maximize the likelihood of consistently receiving parseable, well-structured output without hallucinations

Anthropic Prompt Engineering Guide ("Use Examples / Few-shot prompting"): Details that providing 3–5 diverse examples—specifically including "failure cases" where the target information is missing—is critical. An example set consisting only of perfect extractions teaches the model that successful data retrieval is mandatory, which causes structural breaks or hallucinations when faced with ambiguous text.

Anthropic API Documentation ("Advanced Tool Use & Structured Outputs"): Explains that while JSON schemas define what is structurally valid, they cannot express usage patterns or behavioral nuances. Handling ambiguous text and knowing when to omit optional parameters requires explicit few-shot examples inside the system prompt to clarify correct behavior.

Anthropic Prompt Engineering Checklist: Reinforces that few-shot blocks belong in the cached part of the system prompt to efficiently dictate how the model behaves when it encounters edge cases without breaking the requested JSON output format.

According to Anthropic's architectural guidelines, there is a fundamental distinction between "workflows" and "agents." In an agentic workflow, the LLM is given autonomy to dynamically direct its own processes, reason about intermediate outputs, and decide which tools to call and when. If a task requires tools to be called in an exact, predetermined order to eliminate suboptimal decisions, it is a best practice to orchestrate this deterministically using application code (e.g., prompt chaining, state machines, or programmatic routing) rather than relying on an agentic prompt. Attempting to force strict, multi-step determinism purely through system prompt instructions in an agentic loop is an anti-pattern; it is brittle, prone to prompt-following degradation over long contexts, and negates the primary advantage of agents—their flexible, autonomous reasoning.

Selecting "True" is incorrect because it conflates autonomous agents with deterministic workflows. If an exact execution order is strictly required, developers should not use an open-ended agentic loop. Instead, they should use programmatic orchestration (such as explicitly chaining API calls or using the toolchoice parameter in code) to guarantee the execution path, rather than hoping the model perfectly follows a complex sequence of instructions in the system prompt.

1. Anthropic Official Documentation. "Building effective agents." Section: Workflows vs. Agents. (Defines workflows as systems where paths are predefined by code

whereas agents dynamically direct their own processes and tool usage based on environmental feedback).

2. Anthropic Official Documentation. "Tool use (function calling)." Section: Forcing tool use. (Details how to use the toolchoice parameter programmatically when a specific tool must be used

rather than relying solely on prompt instructions for strict determinism).

The most efficient and effective approach is to fork the existing session. The initial analysis, which involved reading 23 files and tracing flows, has created a rich, valuable context. The forksession feature is specifically designed for this scenario, allowing the creation of two new, independent child sessions that inherit the complete state and context of the parent. This enables the parallel and isolated development of both testing strategies without repeating the costly initial analysis. Each branch can then proceed without interfering with the other, providing a clean environment for comparing the two approaches.

B. Start two fresh sessions, having each re-read the relevant source files before beginning.

This is highly inefficient as it discards the completed analysis and duplicates the expensive initial work for both new sessions.

C. Continue in the original session, developing end-to-end tests first, then snapshot tests sequentially.

This fails the requirement for independent development, as the state and artifacts from the first strategy would influence the second.

D. Export the analysis session’s key findings to a file, then create two new sessions that reference this file.

This is a lossy and cumbersome workaround. A summary file cannot capture the full, nuanced context that is preserved by forking.

1. Anthropic Claude Documentation

"Tool use": The documentation explains that Claude's ability to use tools effectively relies on the context built throughout a conversation. The model uses the entire history of user prompts

assistant responses

and tool results to inform its next action. Forking a session is the canonical method to leverage this accumulated context for exploring divergent

parallel paths without losing the initial state. (Reference: docs.anthropic.com/claude/docs/tool-use

Section: "How to prompt for tool use").

2. Anthropic Claude Documentation

"Building with Claude": In designing complex

multi-turn interactions with agents

maintaining and manipulating state is critical. The principle of session management

including branching or forking

allows developers to build sophisticated applications where different possibilities can be explored from a common contextual starting point

which is essential for tasks like comparing alternative solutions. (Reference: docs.anthropic.com/claude/docs/building-with-claude

Section: "Building conversational experiences").