Our take
Activeregular commits and releases- Activity
- 87
- Community
- 87
- Issues
- 65
- Pull requests
- 83
+1,172 stars in the last 30 days
- Merges more pull requests than 93% of the projects we track
- Releases more often than 91% of the projects we track
- More contributors than 78% of the projects we track
Promptfoo is a capable, actively maintained choice for teams that want prompt evaluation and LLM red teaming in one TypeScript tool, though its current Node.js requirement may affect adoption.
Good fit if
- You need automated prompt or model evaluations, red-team testing, or both.
- Your team wants CLI or CI/CD workflows and can use Node.js 22.22.0 or later.
- You value an MIT-licensed project with regular commits and releases and contributions from a broad contributor base.
Look elsewhere if
- Your environment cannot meet the Node.js runtime requirement and you do not want to use the available Docker option.
- You need a tool centered on a single model or provider, rather than a framework for evaluating LLM-based applications.
- You need a different license from MIT.
All health signals
| Last commit | 2026-10-23 (today) |
|---|---|
| Commits, last 90 days | 100+ |
| Releases, last 12 months | 79 (latest 0.125.0, 2026-10-23) |
| Contributors | 100+ (top contributor: 33% of commits) |
| Issues closed, last 90 days | 3+ (typically closed in 1 day) |
| Pull requests merged, last 90 days | 90 (typically merged in 0 days) |
| Project age | 3 years |
Checked on 2026-10-24 with the GitHub API.
Project history
Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.
Stars
27,200Forks
2,739Change since Nov 14, 2026: +49 stars·+10 forks
Overview
Promptfoo is a TypeScript CLI and library for evaluating and red-teaming LLM-based apps. It helps developers move beyond trial and error by comparing prompt and model behavior and testing applications for security vulnerabilities. It supports multiple model providers, while keeping evaluations local according to the project README.
The project is aimed at developers building LLM applications, including teams that need repeatable evaluations, security checks, or CI/CD integration. It can also be used as a Node.js package, alongside its command-line interface.
Key Features
- Define automated evaluations for prompts and models.
- Compare model outputs side by side across supported providers.
- Run red-team tests and vulnerability scans against LLM applications.
- Add evaluation and security checks to CI/CD workflows.
- Review pull requests for LLM-related security and compliance issues through code scanning.
- Use the CLI or Node.js package, and share evaluation results with a team.
Use Cases
- Prompt engineers can compare prompt revisions against a consistent set of evaluation cases before shipping changes.
- Application teams can compare candidate models for a task using the same prompts and criteria.
- Security teams can red-team an LLM application and generate vulnerability reports before deployment.
- Development teams can run LLM evaluations and security checks in CI/CD as part of routine changes.
What you need
Detected in the repository
- Node.js >=22.22.0 (from package.json)
- A Dockerfile, so it can run in a container
- A Helm chart for Kubernetes
- A test suite and automated checks on GitHub Actions
License in plain words
MITpermissive
- Commercial use: yes
- Modify and redistribute: yes
- You must keep: the copyright and license notice
- Share your changes: no
A summary, not legal advice: the LICENSE file is what applies.
Getting Started
npm install -g promptfoo
promptfoo init --example getting-started
See the README and Getting Started documentation for setup and usage details.
Alternatives
- langwatch: LangWatch is a broader platform for production monitoring, simulations, and prompt management, while promptfoo centers on local CLI evaluations and red-teaming.
- opik: Opik focuses on tracing, experiment management, and production dashboards alongside evaluation, while promptfoo centers on local tests and security scans.
- agentevals: agentevals scores existing OpenTelemetry traces without rerunning agents, while promptfoo runs configurable prompt and model evaluations and red-team scans.
- agentevals: AgentEvals focuses on evaluating agent steps and tool calls, while promptfoo supports broader prompt, model, and vulnerability testing.
| Project | Language | License | Stars | Status |
|---|---|---|---|---|
| promptfoo | TypeScript | MIT | 26.4k | Active |
| langwatch | TypeScript | Apache-2.0 | 4.9k | Active |
| opik | Python | Apache-2.0 | 22.8k | Active |
| agentevals | Python | Apache-2.0 | 162 | Active |
| agentevals | Python | MIT | 737 | Active |
Considerations
- The measured project health indicates active development, with frequent commits and releases. That pace can mean behavior changes over time, so teams should check the current documentation and release notes when upgrading.
- The declared Node.js requirement is 22.22.0 or later. Docker and Helm support are also indicated in the project metadata, but the README's quick start uses npm.
- Evaluations typically require API keys for the providers being tested. Treat credentials and access to external model APIs as part of the deployment and testing setup.
- The project is MIT licensed. Review that license and any provider-specific terms relevant to your use.
Found this useful?
Share it with someone who would like promptfoo.