A company wants to use a large language model (LLM) on Amazon Bedrock for sentiment analysis. The company wants to know how much information can fit…

The context window, also referred to as context length, is the specific parameter that defines the maximum number of tokens (a combination of the input prompt and the model's generated output) that a model can process at one time. It directly determines how much information can be fed into a single prompt. For a task like sentiment analysis on a large document, a model with a larger context window is necessary to ensure the entire text can be analyzed in one go, preserving the full context for an accurate assessment.

1. Amazon Web Services (AWS) Documentation: The Amazon Bedrock User Guide explicitly lists the "Max context length" or "Max tokens" for each foundation model. This value represents the context window. For example

for Anthropic's Claude models

it states

"Maximum context length (input + output): 200

000 tokens."

Source: Amazon Bedrock User Guide

"Base models in Amazon Bedrock

" section on "Model attributes."

2. University Courseware: Stanford University's course CS324: Large Language Models defines context length as a core property of a language model. Lecture slides and notes explain that the "context length" (or window) is the finite number of tokens that a model can take as input.

Source: Stanford University

CS324: Large Language Models

Winter 2023

Lecture 2: "Language Model Properties."

3. Academic Publication: The foundational paper on the Transformer architecture

which underpins most modern LLMs

establishes the concept of processing sequences of a fixed length

which is the basis for the context window.

Source: Vaswani

A.

et al. (2017). "Attention Is All You Need." Advances in Neural Information Processing Systems 30 (NIPS 2017). Section 3.1

"Encoder and Decoder Stacks

" describes processing input sequences of symbol representations. (DOI: https://doi.org/10.48550/arXiv.1706.03762)