Context engineering

Context engineering names the discipline of deciding what goes into an LLM’s context window, how it is structured, when it arrives, and what gets discarded. As AI agents move from single-turn assistants to long-running systems, the quality of that window increasingly determines whether the system works or not.

The phrase captures a shift in where the hard problems live. The Typical Set argues that code generation is no longer the bottleneck: the real constraints are specification clarity and shared organizational context, and agents amplify whatever alignment or misalignment already exists. Context engineering is what turns that observation into a practice.

At the structural level, several sources treat the context window itself as a state container. The 12-factor-agents project Factor 5 argues that execution state and business state should both be inferred from a single context-window-derived thread, making the system trivially serializable, debuggable, and resumable. Anthropic’s Managed Agents service externalizes the session log as a stable interface so the harness and sandbox can evolve independently. Their long-running agent harness uses a progress file and feature list to maintain coherent state across many context windows without drift.

Memory design is a recurring sub-problem. The zerostack agent uses plain Markdown files with regex retrieval rather than vector stores, a deliberate choice driven by RAM constraints and provider neutrality. Storybloq persists session context across Claude Code sessions via a .story/ directory of JSON files. A more principled critique from Jakedismo reframes memory as a belief-maintenance problem: systems that store assertions without provenance, confidence, or revision history propagate stale information into later context windows.

Retrieval is the other axis. The Karpathy LLM-wiki pattern, documented in two Reddit threads, builds curated Markdown knowledge bases that an LLM queries holistically. One builder reports that cross-document synthesis is genuinely superior to RAG for curated research, but that hallucinations baked in at ingest propagate structurally, making a lint step non-negotiable. VectifyAI’s PageIndex takes a different angle, replacing vector similarity with LLM reasoning over hierarchical tree indexes, achieving 98.7% accuracy on FinanceBench. Recursive Language Models as described by dbreunig address context rot by keeping data in a REPL and letting the model pull selectively into token space.

Compression matters when context is scarce or expensive. Headroom compresses tool outputs and RAG chunks before they reach the model, claiming 60-95% token reduction. KV caching treated as a persistent data asset can reduce prefill costs by up to 20x. A skeptical counterpoint from Przemek Mroczek warns that compression tools advertising 60-90% savings are often measuring vanity metrics rather than task accuracy, and that silent data loss in agent pipelines is a real risk.

At the tooling layer, WaveScope generates multi-resolution structural views of source code via wavelet transforms, providing token-efficient context without language-specific parsers. OpenAI’s internal data agent stacks schema metadata, human annotations, code enrichment, and self-improving memory to query 600+ petabytes accurately. Anthropic’s analytics stack achieves 95% accuracy through canonical datasets and curated skill docs; a critique from Genloop notes that replicating that accuracy requires months of senior data engineering most organizations cannot afford, which is itself an argument for treating context construction as a first-class engineering investment.