How to build an AI agent with 1M token context.

Analysis

For years, the standard advice for feeding a large document to a language model was to break it into pieces. You'd chunk the text, build embeddings, store them in a vector database, and retrieve the parts that looked relevant at query time. It worked, but it was fiddly, and it leaked context. The model never saw the whole picture, only the slices you guessed it would need.

That constraint is loosening. Several models now accept roughly a million tokens in one go, enough for a mid-sized codebase, a long contract with all its appendices, or an entire support-ticket history. You can hand the model the full thing and ask your question, no retrieval pipeline required.

For an Australian business team, the practical upshot is fewer moving parts. The work that used to live in chunking logic and a vector store can sometimes collapse into a single API call. That doesn't make retrieval obsolete, and it isn't free, a million input tokens still costs real money, and the cheapest option depends on the job. But the engineering you can now skip is substantial.

This guide walks through building a 1M-context agent: counting tokens so you don't blow the limit, wiring up the agent, pre-filtering when even a million isn't enough, and routing each task to the model that does it cheapest.

Analysis

Prerequisites

• API keys for Anthropic, MiniMax, or Google
• Understanding of token counting
• A large document or codebase to process
• Python 3.10+

Step-by-Step Framework

Step 1: Count Tokens Accurately

Before you fire off a million tokens, you need to know how many you actually have:

# token_counter.py
import tiktoken

def count_tokens(text: str, model: str = "gpt-4") -> int:
 encoder = tiktoken.encoding_for_model(model)
 return len(encoder.encode(text))

def count_file_tokens(path: str) -> int:
 with open(path, 'r', encoding='utf-8', errors='ignore') as f:
 return count_tokens(f.read())

def count_directory_tokens(dir_path: str, extensions: list[str]) -> dict:
 import os
 from pathlib import Path

 results = {}
 total = 0

 for ext in extensions:
 for file in Path(dir_path).rglob(f"*.{ext}"):
 try:
 tokens = count_file_tokens(str(file))
 results[str(file)] = tokens
 total += tokens
 except Exception as e:
 results[str(file)] = 0

 return {"files": results, "total": total, "file_count": len(results)}

# Example: count a codebase
stats = count_directory_tokens("./src", ["ts", "tsx", "js"])
print(f"Total tokens: {stats['total']:,}") # e.g. 485,231 tokens
print(f"Files: {stats['file_count']}")

Step 2: Build the 1M Context Agent

# agent_1m.py
from anthropic import Anthropic
from typing import List, Optional
import json

class OneMillionContextAgent:
 def __init__(self, api_key: str, model: str = "claude-sonnet-4.6"):
 self.client = Anthropic(api_key=api_key)
 self.model = model
 self.max_context = 1_000_000
 self.max_output = 8192 # Can request up to 64K

 def build_prompt(self, context: str, instruction: str, examples: Optional[List[dict]] = None) -> str:
 """Build a prompt that fits within 1M tokens."""
 system_prompt = f"""You are an expert analyst with access to {self.max_context:,} tokens of context.
Analyse the provided materials thoroughly and follow the instruction precisely.
Be concise but comprehensive."""

 messages = [{"role": "user", "content": f"Context:
\n{context}\n\nInstruction: {instruction}"}]

 return system_prompt, messages

 def run(self, context: str, instruction: str, temperature: float = 0.3) -> str:
 """Execute with full context."""
 system, messages = self.build_prompt(context, instruction)

 # Verify token count
 context_tokens = len(self.client.count_tokens(context))
 print(f"Context tokens: {context_tokens:,} / {self.max_context:,}")

 if context_tokens > self.max_context:
 raise ValueError(
 f"Context too large: {context_tokens:,} > {self.max_context:,}. "
 "Use the pre-filtering method (Step 3)."
 )

 response = self.client.messages.create(
 model=self.model,
 max_tokens=self.max_output,
 temperature=temperature,
 system=system,
 messages=messages
 )

 return response.content[0].text

One correctness note on the snippet above: the current Anthropic Python SDK counts tokens via client.messages.count_tokens(...), which returns an object with an input_tokens field, not a client.count_tokens(text) call you can wrap in len(). Swap that line for the real API before you rely on the guard.

Step 3: Pre-Filtering for Very Large Inputs

When your context runs past 1M tokens, you filter before you send:

# prefilter.py
from sentence_transformers import SentenceTransformer
import numpy as np

class ContextPrefilter:
 def __init__(self):
 self.embedder = SentenceTransformer('all-MiniLM-L6-v2')

 def filter_relevant_sections(
 self,
 sections: List[str],
 query: str,
 max_tokens: int = 900_000
 ) -> str:
 """Keep only the most relevant sections to fit under the limit."""
 # Embed query
 query_embedding = self.embedder.encode(query)

 # Embed all sections
 section_embeddings = self.embedder.encode(sections)

 # Calculate similarity
 similarities = np.dot(section_embeddings, query_embedding)

 # Sort by relevance
 ranked = sorted(zip(sections, similarities), key=lambda x: x[1], reverse=True)

 # Take top sections until we hit the token limit
 selected = []
 total_tokens = 0

 for section, score in ranked:
 section_tokens = len(section.split()) * 1.3 # Rough estimate
 if total_tokens + section_tokens > max_tokens:
 break
 selected.append(section)
 total_tokens += section_tokens

 return "\n\n---\n\n".join(selected)

This is the part where chunking and embeddings still earn their keep. You're not retrieving slices to answer the question, you're trimming the input down to what fits, then letting the model read the rest in full.

Step 4: Use Case, Entire Codebase Analysis

# codebase_analysis.py
from agent_1m import OneMillionContextAgent
from pathlib import Path

def load_codebase(path: str) -> str:
 """Load all source files into a single context string."""
 files = []
 for ext in ['ts', 'tsx', 'js', 'json', 'md']:
 for file in Path(path).rglob(f"*.{ext}"):
 if 'node_modules' in str(file) or 'dist' in str(file):
 continue
 content = file.read_text(errors='ignore')
 files.append(f"=== {file} ===\n{content}")
 return "\n\n".join(files)

# Load and analyse
codebase = load_codebase("./my-project")
agent = OneMillionContextAgent(api_key="sk-ant-...")

analysis = agent.run(
 context=codebase,
 instruction="""Analyse this codebase and provide:
1. Architecture overview
2. Security vulnerabilities (if any)
3. Performance bottlenecks
4. Code quality issues
5. Suggested refactoring (top 5 by impact)

Format as structured markdown.""",
 temperature=0.2
)

print(analysis)

Step 5: Multi-Model Fallback Strategy

Pick the cheapest model that can still handle the job and the context size:

# multi_model.py
class ContextRouter:
 MODELS = {
 "gemini-3.5-flash": {"cost_per_1m": 0.35, "context": 1_000_000, "strengths": ["speed", "cost"]},
 "minimax-m3": {"cost_per_1m": 0.30, "context": 1_000_000, "strengths": ["cost", "output_length"]},
 "claude-sonnet-4.6": {"cost_per_1m": 3.00, "context": 1_000_000, "strengths": ["reasoning", "coding"]}
 }

 def select_model(self, task_type: str, context_tokens: int) -> str:
 if task_type in ["summarisation", "extraction", "classification"]:
 return "gemini-3.5-flash" # Fast and cheap

 if task_type == "code-generation" or context_tokens > 500_000:
 return "minimax-m3" # Best output length, cheapest

 return "claude-sonnet-4.6" # Best reasoning

# Usage
router = ContextRouter()
model = router.select_model("code-analysis", 800_000)
print(f"Selected: {model} (estimated cost: ${0.30})")

The cost_per_1m values baked into that dict are the article's original figures, and at least one is off, the Gemini Flash rate in particular runs well above $0.35 in practice (see the cost note below). Pull live pricing into this table rather than trusting the hard-coded numbers.

Step 6: Streaming for Long Outputs

If you're generating 10K-plus token outputs, stream them so the request doesn't time out:

# streaming.py
from anthropic import Anthropic

client = Anthropic()

with client.messages.stream(
 model="claude-sonnet-4.6",
 max_tokens=64000,
 messages=[{
 "role": "user",
 "content": f"Context: {large_context}\n\nGenerate a complete analysis..."
 }]
) as stream:
 for text in stream.text_stream:
 print(text, end="", flush=True)

Do/Don't

Cost Comparison (1M Input Tokens)

Two cautions on this table. The Gemini 3.5 Flash row reads $0.35/$0.70, but OpenRouter lists the model at $1.50 input / $9.00 output (opens in a new tab) per million tokens, roughly four to thirteen times higher, so the figures here are unconfirmed and almost certainly understated. The MiniMax M3 row holds up to 512K input; past that you're paying closer to $0.60/$2.40 (opens in a new tab). The GPT-5.5 $5/$30 pricing is accurate (opens in a new tab), but the "400K only" note isn't: the GPT-5.5 API offers a 1M-token window, and the 400K cap applies inside Codex.

Conclusion

A 1M token window lets you skip the retrieval pipeline for a lot of jobs. Rather than chunking, embedding, and fetching, you load the material and ask. Sonnet 4.6 gives you the strongest reasoning, MiniMax M3 the best value when you need long outputs, and Gemini 3.5 Flash quick turnaround for lighter work. Count your tokens first, pre-filter when the input won't fit, and price each model against today's published rates rather than the figures that were current when this was written, those move fast, and a couple of them already have.