AI agents spend a surprising amount of their budget rereading. Tool outputs repeat metadata, logs contain thousands of irrelevant lines, code search returns overlapping snippets, and conversation history grows long after parts of it stop mattering.
Headroom inserts a local compression layer before that context reaches the LLM. It detects content type, applies specialized compressors to JSON, code, prose, images, and context windows, stabilizes prompt prefixes for provider caches, and can retain originals for retrieval when compressed context is insufficient.
It is Apache-2.0 software with Python and TypeScript libraries, an OpenAI-compatible proxy, wrappers for coding agents, and MCP tools. The objective is not merely shorter prompts; it is lower token usage while preserving the information needed to complete the task.
Structured JSON, source code, prose, images, and rolling context have different redundancy. One summarizer is not ideal for all of them.
Headroom can reduce tool and retrieval payloads before the provider sees them, while CacheAligner helps stable prompt prefixes continue hitting provider caches.
CCR stores originals locally within the configured lifetime and exposes retrieval so an agent can recover details omitted from the compact form.
What Headroom Does
Headroom sits between an agent or application and its model provider. It can be integrated in four main ways:
- call
compress(messages)from Python or TypeScript; - run a local OpenAI-compatible proxy with
headroom proxy; - launch supported agents through
headroom wrap; - expose compression, retrieval, and statistics through MCP.
Because the proxy speaks a familiar API shape, applications in any language can use it without embedding Headroom directly. Wrappers target Claude Code, Codex, Cursor, Aider, Copilot CLI, and OpenClaw. Framework integrations cover SDKs and agent stacks such as LangChain, Agno, LiteLLM, Strands, and Vercel AI SDK.
The Compression Pipeline
The architecture is more than a summarization prompt.
ContentRouter inspects payloads and selects an appropriate path. SmartCrusher targets JSON and nested tool output. CodeCompressor is AST-aware for languages including Python, JavaScript, Go, Rust, Java, and C++. Kompress-base is a Hugging Face model trained for text compression on agent traces. Other components can fit context by relevance or rolling windows and compress images.
CacheAligner addresses another source of cost: prompt-cache misses. Provider KV caches benefit from stable prefixes. If preprocessing constantly rewrites early prompt content, compression may reduce tokens while destroying cache reuse. Headroom attempts to align content so stable prefixes remain stable.
The pipeline runs locally, but the final compressed prompt still goes to the configured model provider unless the model itself is local.
CCR: Reversible Compression
Compression inevitably discards detail. Headroom’s Context Compression and Retrieval (CCR) design stores original content locally and gives the agent a retrieval path.
Instead of placing a complete 10,000-line log in the prompt, Headroom can send a compact representation plus references. If the model needs an omitted stack trace or exact record, it can call retrieval to fetch the original.
This is an important safety valve, not a guarantee. The model must recognize that information is missing and decide to retrieve it. Cache lifetime, storage limits, permissions, and deletion behavior therefore affect correctness as well as privacy.
Getting Started
The full Python installation requires Python 3.10+:
pip install "headroom-ai[all]"
headroom wrap codex
headroom perf
Or run a proxy:
headroom proxy --port 8787
TypeScript users can install headroom-ai from npm. Granular Python extras let teams avoid pulling every feature and model dependency. Docker images are also published.
Start with a reversible, observable workload such as verbose search output or logs. Compare task success before and after compression, then expand to conversation history and shared memory.
Published Benchmarks in Context
The repository reports large reductions on selected agent workloads: 92% for a 100-result code search, 92% for an SRE incident, 73% for GitHub issue triage, and 47% for codebase exploration. It also publishes small benchmark samples where accuracy was maintained or improved.
These results demonstrate possibility, not a universal 60–95% discount. Compression depends on redundancy, query type, model, algorithm, and required fidelity. A repetitive log is easier to compress than compact source code where one character changes semantics.
The benchmark tables should also be read with their sample sizes and methodology. Reproduce the evaluation and test representative internal workloads. Measure correctness, tool-call quality, latency, provider caching, and retrieval frequency alongside tokens.
Input and Output Token Reduction
Most Headroom features reduce input context. An optional output shaper targets the model’s response by steering verbosity and reducing reasoning effort on routine post-tool turns. It is disabled by default.
This can reduce ceremonial preambles, repeated code, and unnecessary deliberation, but it changes response behavior. Routine and high-stakes turns are not always easy to classify. Use a holdout group and measure outcomes rather than assuming shorter answers are equivalent.
The project explicitly labels counterfactual output savings as estimates with confidence ranges and supports unshaped holdouts for measured comparisons. That distinction is important because nobody observes the answer the model would have generated under the alternative policy in the same conversation.
Cross-Agent Memory and Learning
Headroom includes shared context storage so different agents can reuse compressed memory with provenance and deduplication. A team can move between Claude, Codex, and other tools without copying the entire history into every new session.
headroom learn analyzes failed sessions and proposes corrections for instruction files such as CLAUDE.md, AGENTS.md, and GEMINI.md. This can turn recurring mistakes into durable guidance, but automatically learned rules should be reviewed. A failure may come from a transient environment issue or bad task framing rather than a reusable principle.
Privacy and Security
“Local-first” reduces exposure to a hosted compression service, but Headroom handles sensitive prompts and stores recoverable originals. Production use requires answers to several questions:
- Where are CCR originals and shared memories stored?
- What TTL, encryption, access controls, and deletion policies apply?
- Can one agent retrieve another user’s content?
- Which telemetry, update checks, or model downloads require network access?
- Does the proxy preserve provider authentication and tenant isolation correctly?
- What happens when compression or retrieval fails?
Corporate environments may also need trusted certificates or offline copies for ONNX Runtime and Hugging Face assets.
When to Use—or Skip—Headroom
Headroom is a strong fit for agents that consume verbose tool results, large code searches, logs, repeated RAG chunks, or long shared histories. It is also useful when provider-neutral compression and reversible retrieval matter.
Skip or delay it when the provider’s native compaction is sufficient, local processes cannot run, prompts are already compact, or the workload cannot tolerate lossy preprocessing. Regulated and high-stakes workflows need domain evaluations before compression enters the decision path.
Final Take
Headroom treats context as a data pipeline rather than an ever-growing text buffer. It routes each content type to a specialized compressor, protects prompt-cache stability, and keeps an on-demand path back to original material.
That combination is more practical than blindly summarizing everything. But the engineering target is not the highest compression ratio. It is the lowest token footprint that preserves task success, recoverability, latency, privacy, and auditability.
Deploy it first in shadow or holdout mode, reproduce the published methodology on your own agent traces, inspect retrieval misses, and let measured quality determine how much headroom you can safely reclaim.
