{"name":"Toolkit-for-Prompt-Compression: A Unified Toolkit for LLM Prompt Compression","description":"PCToolkit is a unified, plug-and-play toolkit designed for efficient prompt compression in Large Language Models (LLMs). It provides state-of-the-art compression methods, diverse datasets, and comprehensive metrics for evaluating performance. This modular toolkit simplifies the process of condensing input prompts while preserving crucial information.","github":"https://github.com/3DAgentWorld/Toolkit-for-Prompt-Compression","url":"https://osrepos.com/repo/3dagentworld-toolkit-for-prompt-compression","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/3dagentworld-toolkit-for-prompt-compression","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/3dagentworld-toolkit-for-prompt-compression.md","json":"https://osrepos.com/repo/3dagentworld-toolkit-for-prompt-compression.json","topics":["Python","AI","Machine Learning","NLP","LLM","Prompt Compression","Toolkit"],"keywords":["Python","AI","Machine Learning","NLP","LLM","Prompt Compression","Toolkit"],"stars":null,"summary":"PCToolkit is a unified, plug-and-play toolkit designed for efficient prompt compression in Large Language Models (LLMs). It provides state-of-the-art compression methods, diverse datasets, and comprehensive metrics for evaluating performance. This modular toolkit simplifies the process of condensing input prompts while preserving crucial information.","content":"## Introduction\n\nPCToolkit is a unified, plug-and-play prompt compression toolkit for Large Language Models (LLMs). Prompt compression is an innovative method for efficiently condensing input prompts while preserving essential information, crucial for optimizing LLM inference efficiency. This toolkit offers a comprehensive solution, featuring cutting-edge prompt compressors, diverse datasets, and metrics for thorough performance evaluation.\n\nKey features of PCToolkit include:\n*   **State-of-the-art and reproducible methods:** It encompasses five distinct mainstream compression techniques: Selective Context, LLMLingua, LongLLMLingua, SCRL, and Keep it Simple.\n*   **User-friendly interfaces:** Designed for portability and easy adaptation, allowing for simple integration of new compressors, datasets, and metrics.\n*   **Modular design:** Organized into Compressor, Dataset, Metric, and Runner modules, simplifying transitions between different methods and evaluation components.\n\nPCToolkit includes 5 compression methods, 11 datasets, and over 5 metrics, evaluated across various natural language tasks like reconstruction, summarization, mathematical problem-solving, and question answering.\n\n## Installation\n\nTo get started with PCToolkit, follow these steps:\n\nFirst, clone the repository:\n\nshell\ngit clone https://github.com/3DAgentWorld/Toolkit-for-Prompt-Compression.git\n\n\nNavigate into the cloned directory:\n\nshell\ncd Toolkit-for-Prompt-Compression\n\n\nThen, install the required dependencies:\n\nshell\npip install -r requirements.txt\n\n\nPlease note that while most models can be automatically downloaded from Huggingface Hub, models for the `SCRL` method require manual download. Refer to the guidance within the `/models` folder for detailed instructions.\n\n## Examples\n\nPCToolkit provides straightforward interfaces for both prompt compression and evaluation.\n\n### Prompt Compression\n\nTo perform prompt compression, you can use the `PromptCompressor` class:\n\npython\nfrom pctoolkit.compressors import PromptCompressor\n\ncompressor = PromptCompressor(type='SCCompressor', device='cuda')\n\ntest_prompt = \"test prompt\"\nratio = 0.3\nresult = compressor.compressgo(test_prompt, ratio)\nprint(result)\n\n\n### Evaluation\n\nFor evaluating compressor performance, follow this example:\n\npython\nfrom pctoolkit.runners import run\nfrom pctoolkit.datasets import load_dataset\nfrom pctoolkit.metrics import load_metrics\nfrom pctoolkit.compressors import PromptCompressor\n\ncompressor = PromptCompressor(type='SCCompressor', device='cuda')\ndataset_name = 'arxiv'\ndataset = load_dataset(dataset_name)\n\nrun(compressor=compressor, dataset=dataset, metrics=load_metrics, ratio=0.1)\n\n\n\n**Hint**: Remember to fill in your Huggingface Tokens and API keys for OpenAI in `pctoolkit/runners.py`. You can also modify the URLs if you are using other OpenAI APIs. If you wish to change the metrics, specifically for the LongBench dataset, modify `pctoolkit/metrics.py`.\n\n## Why Use PCToolkit?\n\nPCToolkit offers a robust and versatile solution for anyone working with Large Language Models and seeking to optimize prompt efficiency. Its key advantages include:\n\n*   **Comprehensive Coverage:** Integrates five state-of-the-art prompt compression methods, providing a wide range of options for different use cases.\n*   **Ease of Use:** Features user-friendly interfaces that simplify the process of applying compression techniques and evaluating their impact.\n*   **Modularity and Extensibility:** Its modular design allows for easy integration of new datasets, metrics, and even custom compression methods, making it highly adaptable.\n*   **Extensive Evaluation:** Comes with support for 11 diverse datasets and over 5 metrics, enabling thorough and reproducible evaluation across various NLP tasks.\n*   **Research and Development Ready:** Ideal for researchers and developers looking to experiment with, compare, and build upon existing prompt compression techniques.\n\n## Links\n\n*   **GitHub Repository:** [https://github.com/3DAgentWorld/Toolkit-for-Prompt-Compression](https://github.com/3DAgentWorld/Toolkit-for-Prompt-Compression)\n*   **Technical Report (Paper):** [https://arxiv.org/abs/2403.17411](https://arxiv.org/abs/2403.17411)\n*   **Hugging Face Demo:** [https://huggingface.co/spaces/CjangCjengh/Prompt-Compression-Toolbox](https://huggingface.co/spaces/CjangCjengh/Prompt-Compression-Toolbox)","metrics":{"detailViews":4,"githubClicks":4},"dates":{"published":null,"modified":"2025-12-13T00:01:04.000Z"}}