files-to-prompt: Concatenate Files into a Single Prompt for LLMs

Summary
files-to-prompt is a command-line tool designed to concatenate the contents of multiple files from a directory into a single, structured prompt suitable for Large Language Models (LLMs). It offers flexible options for filtering files by extension, ignoring patterns, and supports various output formats including standard text, Claude XML, and Markdown. This utility streamlines the process of preparing complex codebases or documentation for AI analysis.
Repository Info
Tags
Click on any tag to explore related repositories
Introduction
files-to-prompt is a powerful command-line utility created by Simon Willison that simplifies the process of preparing textual data for Large Language Models (LLMs). It efficiently concatenates the contents of multiple files, or even entire directories, into a single, cohesive output. This tool is particularly useful for developers and researchers who need to feed large amounts of code, documentation, or other text files into LLMs for tasks like code analysis, summarization, or question-answering, ensuring the LLM receives a well-structured context.
Installation
Installing files-to-prompt is straightforward using pip:
pip install files-to-prompt
Examples
Here are some common ways to use files-to-prompt:
Basic Usage: Concatenate all files in a directory.
files-to-prompt path/to/directory
Filter by Extension: Include only files with specific extensions.
files-to-prompt path/to/directory -e py -e js
Output as Claude XML: Format the output specifically for Anthropic's Claude LLMs, optimizing for its context window.
files-to-prompt path/to/directory --cxml
Output as Markdown: Generate output with fenced code blocks, useful for pasting into Markdown documents.
files-to-prompt path/to/directory --markdown
Include Line Numbers: Add line numbers to each file's content in the output.
files-to-prompt path/to/directory -n
Ignore Files: Exclude files matching a specific pattern.
files-to-prompt path/to/directory --ignore "*.log"
Read from Stdin: Pipe file paths from another command, such as find.
find . -name "*.py" -print0 | files-to-prompt --null
Why Use files-to-prompt?
In the era of powerful LLMs, providing well-structured and relevant context is crucial for obtaining accurate and useful responses. files-to-prompt addresses this need by:
- Streamlining Context Preparation: Automatically gathers and formats content from multiple files, saving manual effort.
- Optimizing for LLMs: Offers specific output formats like Claude XML and Markdown, which can improve how LLMs process and understand the provided context.
- Flexibility and Control: Provides extensive options for filtering, ignoring, and customizing the output, allowing users to precisely control what content is included.
- Developer-Friendly: Integrates seamlessly into existing workflows, especially for projects involving codebases or extensive documentation.
Links
- GitHub Repository: https://github.com/simonw/files-to-prompt
- PyPI: https://pypi.org/project/files-to-prompt/
- Background Article: Building files-to-prompt entirely using Claude 3 Opus