CCAR-P practice question 6 of 10

A financial services firm is building a document analysis system to extract key terms and clauses from thousands of contracts daily. The system must…

Full set: 142 questions
A financial services firm is building a document analysis system to extract key terms and clauses from thousands of contracts daily. The system must handle variable-length documents (some under 1,000 tokens, others exceeding 100,000 tokens), process them cost-effectively, and maintain extraction accuracy above 95%. Which Claude model and architectural approach would best address these requirements?
Answer options
A: Use Claude 3.5 Sonnet with a batching API architecture, and implement a sliding context window strategy for documents exceeding token limits to process sections independently and aggregate results Correct answer
B: Use Claude 3 Opus exclusively with a real-time processing architecture to maximize accuracy, accepting higher costs for guaranteed 100% precision on all documents
C: Use Claude 3.5 Haiku with a load-balanced queue to minimize costs, and accept that some longer documents will need to be truncated to stay within token budgets
D: Use Claude 3.5 Sonnet with direct synchronous API calls for all documents to ensure consistent latency across the processing pipeline
Correct Answer:
A. Use Claude 3.5 Sonnet with a batching API architecture, and implement a sliding context window strategy for documents exceeding token limits to process sections independently and aggregate results
Explanation

Claude 3.5 Sonnet provides the optimal balance of high intelligence (meeting the >95% accuracy requirement for complex contract analysis) and cost-efficiency. Because the firm processes thousands of documents daily without a stated need for real-time latency, utilizing the Message Batch API is the most cost-effective architectural choice, as it provides a 50% discount on standard API costs. Additionally, while Claude 3.5 Sonnet has a 200,000 token context window, implementing a sliding context window or chunking strategy is the standard architectural best practice for variable-length pipelines. This ensures that any outlier documents exceeding the maximum token limit are processed reliably without data loss, allowing the system to aggregate extracted terms accurately.

Why Incorrect

Option B: Claude 3 Opus is significantly more expensive, and guaranteeing 100% precision is unrealistic for LLMs. Real-time processing also misses the 50% cost savings of the Batch API.
Option C: Truncating documents causes data loss, meaning key clauses at the end of contracts will be missed, directly violating the strict >95% extraction accuracy requirement.
Option D: Direct synchronous API calls for thousands of daily documents miss the 50% cost reduction provided by the Message Batch API, failing the "cost-effectively" requirement.

References

1. Anthropic Official API Documentation. "Message Batches." Section: Introduction to Message Batches and Pricing (2024). (Confirms the 50% cost reduction for asynchronous batch processing of large document volumes).

2. Anthropic Official API Documentation. "Models Overview." Section: Claude 3.5 Family and Context Windows (2024). (Details Claude 3.5 Sonnet's 200K context window and its benchmarked superiority in cost-to-intelligence ratio for text extraction).

3. Stanford University, CS224N: Natural Language Processing with Deep Learning. Lecture 11: Prompting, Instruction Tuning, and RLHF. Section: Handling Long Contexts and Chunking Strategies (Spring 2023). (Validates the sliding context window and aggregation architecture for documents exceeding maximum LLM token limits).

Scroll to Top