Definition
A context window is the total amount of input and output context a model can consider at once, measured in tokens. Understanding it helps prevent truncation and manage long documents.
You hand an LLM a lengthy contract and ask it to "identify the risk clauses," only to get an error saying "input too long" -- sound familiar? The context window is the maximum total number of tokens (input plus output combined) that an LLM can handle in a single processing step, determining the range the model can "see" at once.
Consumed by Input and Output Combined
The context window is consumed by the sum of input tokens plus output tokens. If you pass a 100K-token document to a model with a 128K context window, only 28K tokens remain for the response. When working with long inputs, you need to factor in the expected output length as well. Since Japanese characters generally correspond to 1-2 tokens each, token efficiency is somewhat lower compared to English. Because API pricing is based on token count, how you use the context window directly affects cost.
Rapid Expansion from 2K to 1 Million Tokens
Context window expansion has progressed at a remarkable pace. GPT-3 in 2020 had only 2,048 tokens (roughly 1,000 Japanese characters). In 2023, GPT-4 expanded from 8K to 32K to 128K, and Anthropic's Claude made headlines with 100K tokens that same year. In 2024, Google's Gemini 1.5 Pro achieved 1 million tokens, equivalent to approximately 10 books. By 2025, Claude 3.5 reached 200K tokens and Gemini 2.0 expanded to 2 million tokens. This expansion was enabled by efficient attention computation methods like FlashAttention and RoPE (Rotary Position Embedding) scaling techniques.
The Lost in the Middle Problem
A larger context does not solve everything. A 2023 Stanford study identified the "Lost in the Middle" problem: information at the beginning and end of input is utilized well, but the middle portion tends to be overlooked. For example, when 20 documents are input for Q&A, accuracy drops most when the correct information is positioned around the 10th document. Practical countermeasures include placing important information at the beginning or end, strategically ordering chunked segments, or splitting queries across multiple calls.
Practical Usage Strategies
Context window size fundamentally changes how LLMs are used. In the era of short windows, RAG (Retrieval-Augmented Generation) was essential for searching and inserting only relevant documents. With windows of 100K or more now available, full contract reviews, entire codebase analysis, and cross-comparison of multiple papers are possible in a single shot. However, longer contexts increase inference costs and reduce response speed. In practice, it is important to choose between the "stuff everything into a long context" approach and the "extract only the necessary parts with RAG" approach based on the balance of task characteristics, cost, and accuracy.