# Promptfoo: LLM Evaluation and Red Teaming for AI Applications

This repository profile is provided by osrepos.com, an open source repository discovery platform.

Source: osrepos.com
Repository profile: https://osrepos.com/repo/promptfoo-promptfoo
Generated for open source discovery and AI-assisted research.

Promptfoo is an open-source CLI and library designed for evaluating and red-teaming Large Language Model (LLM) applications. It enables developers to test prompts, agents, and RAGs, compare model performance, and secure AI apps through vulnerability scanning. With simple declarative configs and CI/CD integration, Promptfoo helps ship reliable and secure AI solutions.

GitHub: https://github.com/promptfoo/promptfoo
OSRepos URL: https://osrepos.com/repo/promptfoo-promptfoo

## Summary

Promptfoo is an open-source CLI and library designed for evaluating and red-teaming Large Language Model (LLM) applications. It enables developers to test prompts, agents, and RAGs, compare model performance, and secure AI apps through vulnerability scanning. With simple declarative configs and CI/CD integration, Promptfoo helps ship reliable and secure AI solutions.

## Topics

- llm
- evaluation
- red-teaming
- prompt-engineering
- testing
- ci-cd
- TypeScript
- AI

## Repository Information

Last analyzed by OSRepos: Tue Mar 24 2026 12:18:12 GMT+0000 (Western European Standard Time)
Detail views: 4
GitHub clicks: 3

## Safety Notice

OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.

## Content

## Introduction

Promptfoo is a powerful open-source CLI and library for evaluating and red-teaming Large Language Model (LLM) applications. It helps developers move beyond trial-and-error, enabling them to ship secure and reliable AI apps. Used by organizations like OpenAI and Anthropic, Promptfoo provides a robust framework for testing prompts, agents, and RAGs, as well as comparing the performance of various LLMs such as GPT, Claude, Gemini, and Llama.

## Installation

Getting started with Promptfoo is straightforward. You can install it globally via npm, brew, or pip:

sh
npm install -g promptfoo


Alternatively, you can use `brew install promptfoo` or `pip install promptfoo`. For quick execution without installation, `npx promptfoo@latest` is also available.

Most LLM providers require an API key. Set yours as an environment variable:

sh
export OPENAI_API_KEY=sk-abc123


## Examples

Once installed, you can initialize an example project and run your first evaluation:

sh
promptfoo init --example getting-started
cd getting-started
promptfoo eval
promptfoo view


Promptfoo offers a comprehensive suite of features to streamline your LLM development workflow. You can test prompts and models with automated evaluations, secure your LLM apps with red teaming and vulnerability scanning, and compare models side-by-side across various providers. It also supports automating checks in CI/CD and reviewing pull requests for LLM-related security issues. The tool provides visual web viewers for evaluation matrices, command-line output, and detailed security vulnerability reports.

## Why use Promptfoo?

Promptfoo stands out for several key reasons:

*   **Developer-first**: It's fast, with features like live reload and caching, designed for developer efficiency.
*   **Private**: LLM evaluations run 100% locally, ensuring your prompts never leave your machine.
*   **Flexible**: It works seamlessly with any LLM API or programming language.
*   **Battle-tested**: Promptfoo powers LLM applications serving over 10 million users in production.
*   **Data-driven**: Make informed decisions based on concrete metrics, not just intuition.
*   **Open source**: It's MIT licensed, backed by an active and supportive community.

## Links

Explore Promptfoo further with these official resources:

*   [Website](https://www.promptfoo.dev)
*   [Getting Started](https://www.promptfoo.dev/docs/getting-started/)
*   [Full Documentation](https://www.promptfoo.dev/docs/intro/)
*   [Red Teaming Guide](https://www.promptfoo.dev/docs/red-team/)
*   [CLI Usage](https://www.promptfoo.dev/docs/usage/command-line/)
*   [Node.js Package](https://www.promptfoo.dev/docs/usage/node-package/)
*   [Supported Models](https://www.promptfoo.dev/docs/providers/)
*   [Code Scanning Guide](https://www.promptfoo.dev/docs/code-scanning/)
*   [Discord Community](https://discord.gg/promptfoo)
*   [GitHub Repository](https://github.com/promptfoo/promptfoo)