3DAgentWorld

Toolkit-for-Prompt-Compression: Evaluate and Apply Prompt Compression

PCToolkit is a Python toolkit for applying and evaluating prompt-compression methods for large language models. It brings five compressors, datasets, and evaluation metrics behind modular interfaces, making it useful for comparing methods across language tasks.

View on GitHub Compare
Stars
293
Forks
10
Stars, last 30 days
+0

Our take

Health score

19/100

How it is scored
Activity
19
Community
17
Issues
21
Pull requests
20

+0 stars in the last 30 days

  • Fewer contributors than 86% of the projects we track

Project history

Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.

Selected snapshot

Stars

291

Forks

10

Change since Nov 15, 2026: 0 stars·0 forks

Overview

PCToolkit provides a shared interface for compressing prompts and evaluating the results. It addresses the fragmentation of prompt-compression implementations by bringing five methods, datasets, and metrics into a modular toolkit for large language model workflows.

It is aimed at researchers and developers who need to experiment with compression methods or compare them across tasks. The repository includes implementations of Selective Context, LLMLingua, LongLLMLingua, SCRL, and Keep it Simple, along with dataset and evaluation components. See the repository and its technical report for details.

Key Features

  • Unified interface for five prompt compressors: Selective Context, LLMLingua, LongLLMLingua, SCRL, and Keep it Simple.
  • Modular components for compressors, datasets, metrics, and evaluation runners.
  • Support for 11 datasets and more than five metrics, as described by the project.
  • Evaluation coverage across tasks including summarization, question answering, mathematical problems, and code completion.
  • Example Python workflows for compressing a prompt and running dataset evaluations.
  • Interfaces designed to make it possible to add or adapt compressors, datasets, and metrics.

Use Cases

  • Prompt-compression researchers can use a shared setup to compare supported methods on selected datasets and tasks.
  • LLM application developers can test whether reducing prompt length suits their workflow before integrating a compressor.
  • Evaluation engineers can run compression experiments across tasks using the toolkit's datasets, metrics, and runners.
  • Educators and students can inspect and modify implementations to learn how prompt compression approaches differ.

Getting Started

git clone https://github.com/3DAgentWorld/Toolkit-for-Prompt-Compression.git
cd Toolkit-for-Prompt-Compression
pip install -r requirements.txt

For model setup and usage examples, see the README.

Considerations

  • Some model files must be downloaded manually, especially those used by SCRL. Other models may download from Hugging Face Hub.
  • The evaluation workflow may require Hugging Face tokens and OpenAI API keys, which the README says to configure in pctoolkit/runners.py.
  • Running the example compressor uses a CUDA device, so check the selected compressor's model and hardware requirements before deployment.
  • Evaluation results depend on the chosen dataset, metrics, compressor settings, and external API configuration; the repository does not provide a single universal measure of compression quality.

Found this useful?

Share it with someone who would like Toolkit-for-Prompt-Compression.