ctx-gate: Reduce LLM Context Tokens Through a Proxy

Overview

ctx-gate sits between an IDE or coding assistant and an LLM provider. It reduces repeated conversation history and verbose tool output, and can retain prompt-relevant earlier details, with the goal of lowering token use without changing the client workflow.

It is most relevant to developers who use coding agents for long sessions and want to measure the effects of compression. The README describes the project as early-stage and reports benchmark results where different compression methods perform differently, so it is worth evaluating against your own workload rather than assuming a fixed saving.

Key Features

• Proxy endpoints for Anthropic Messages and OpenAI-compatible chat completions, including streaming.
• Context compression using summaries, relevance-based retention, file diffs, and tool-output trimming.
• Optional semantic relevance and code retrieval, with lexical or TF-IDF fallbacks.
• Task-shift detection and model-tier routing based on prompt signals.
• Session checkpoints, per-client state isolation, and persistent usage statistics.
• Faithfulness evaluation and benchmark commands for comparing token savings with fact retention.

Use Cases

• Developers using Claude Code who want to reduce repeated context and tool-output tokens without changing their normal client workflow.
• Teams using Cursor, Continue.dev, or OpenAI SDK clients that want to route requests through a local optimization proxy.
• Long-running coding sessions where logs and repeated file contents make up a significant share of the prompt.
• Maintainers testing context-compression changes who need a repeatable fact-retention check or CI gate.

Getting Started

Install and start the default Anthropic proxy:

pip install ctx-gate
ANTHROPIC_API_KEY=sk-ant-... python ctx_gate.py serve

For provider configuration, client setup, optional dependencies, and evaluation instructions, see the repository README.

Alternatives

• headroom: Headroom compresses tool outputs, files, logs, and retrieval context through library, proxy, or MCP integrations, while ctx-gate focuses on conversation context.

Considerations

• Token savings depend on the session. The README reports minimal compression for short Q&A and much larger savings for sessions with verbose logs; these are not guaranteed for other workloads.
• The bundled evaluation measures fact retention deterministically, while answer-accuracy evaluation requires an API key. Benchmark conclusions depend on the test corpus and method.
• Semantic relevance, RAG, and LLM-based summaries are optional and can add dependencies or request latency. Defaults use lexical relevance and extractive summaries.
• The README describes the project as early-stage. Check provider model IDs and configuration for your setup, and note that a streaming connection dropped after output begins is not retried.