tooltrim: Drastically Reduce LLM Agent Tool Output Tokens, Improve Accuracy

Introduction

tooltrim is an innovative Python library designed to address the significant challenge of bloated tool outputs in Large Language Model (LLM) agent contexts. It offers a drop-in solution to compress tool results, such as fetched web pages, JSON responses, or log dumps, before they consume valuable context window space. By reducing token counts by up to 99% while maintaining, and often improving, answer accuracy, tooltrim helps agents operate more efficiently and cost-effectively.

Why Use tooltrim and Its Advantages

LLM agents frequently encounter tool outputs that are excessively verbose, leading to increased token usage, slower responses, and potential distraction for the model. Traditional solutions like prompt compressors or semantic caches do not target this specific issue. tooltrim directly tackles this problem by compressing tool outputs at the point they enter the agent's context.

Key advantages include:

• Massive Token Savings: Benchmarks show reductions from tens of thousands of tokens to hundreds, resulting in a 35.6x smaller context.
• Accuracy Improvement: For smaller models, compression doesn't just preserve accuracy, it can significantly improve it by removing noise and distractions.
• Provider-Agnostic: Works with any LLM provider or framework, as it compresses strings, not APIs.
• Content-Aware Compression: Utilizes specific strategies for HTML, JSON, tabular data, logs, and free text, ensuring intelligent and relevant extraction.
• Lossless by Reference: Full original outputs are stashed and retrievable via a short reference, allowing agents to expand content on demand.
• Faithfulness-Tested: Includes a built-in benchmark to measure whether models maintain correct answers on compressed output, not just token savings.
• Proxy Deployment: An OpenAI-compatible proxy allows for zero-code integration, trimming role:"tool" messages in flight by simply changing a base_url.
• Framework Integrations: Seamlessly integrates with LangChain, LlamaIndex, and OpenAI Agents SDK with minimal code changes.

Installation

Installation is straightforward:

pip install tooltrim          # zero-dependency core (heuristic token counts)
pip install tooltrim[tokens]  # add tiktoken for exact token counts

Optional extras for specific integrations include tooltrim[langchain], tooltrim[redis], tooltrim[s3], tooltrim[mcp], and tooltrim[embeddings].

Examples

tooltrim offers flexible usage patterns:

1. Decorate a tool: Wrap your existing functions to automatically compress their outputs.

from tooltrim import compressed_tool

@compressed_tool(max_tokens=400)
def read_file(path: str) -> str:
    return open(path).read()

2. Make it query-aware: Provide a relevance query to guide the compression, ensuring critical information is retained.

from tooltrim import query_scope

with query_scope("find the customer's refund status"):
    result = run_agent_step()   # all @compressed_tool calls inside use this query

3. Imperative API + expand-on-demand: For fine-grained control, use the ToolCompressor directly and allow agents to retrieve full outputs when needed.

from tooltrim import ToolCompressor

tc = ToolCompressor(max_tokens=400)
res = tc.compress(huge_json_response, query="refund status for customer C-1007")

# ... later, if the agent needs the full context ...
full = tc.expand(res.ref)

4. Framework Integrations: Integrate with popular LLM frameworks like LangChain, LlamaIndex, or OpenAI Agents SDK with simple wrappers.

# LangChain example
from tooltrim.integrations import compress_langchain_tool

fetch = compress_langchain_tool(my_tool, max_tokens=400,
                                query_from=lambda query, **_: query)

5. Run as a Proxy: Deploy tooltrim as a proxy to compress tool results for any OpenAI or Anthropic client with zero code changes, just by updating the base_url.

python run_proxy.py --upstream https://api.openai.com/v1

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8800/v1", api_key="<upstream key>")

Links

For more details, benchmarks, and advanced usage, visit the official repository:

• GitHub Repository
• PyPI