inspector vs promptfoo
MCP server inspection and LLM evaluation compared
inspector helps developers test and debug MCP servers and apps, including protocol behavior, OAuth, and model-driven evaluations. promptfoo evaluates prompts and models and tests LLM applications for security risks. Their main distinction is MCP integration debugging for inspector versus broader LLM quality and security testing for promptfoo.
inspector: Test and Debug MCP Servers and Apps
MCPJam Inspector helps developers test, inspect, and evaluate MCP servers and apps across clients and models. Use it to debug protocol and OAuth behavior, run repeatable evaluations, and check for regressions in CI.
promptfoo: Evaluate and Red-Team LLM Apps
Promptfoo is a TypeScript CLI and library for evaluating and red-teaming LLM-based apps. Compare prompts and models, scan for vulnerabilities, and run checks in CI/CD, with evaluations performed locally.
| inspector | promptfoo | |
|---|---|---|
| Language | TypeScript | TypeScript |
| License | NOASSERTION | MIT |
| Stars | 2.2k | 26.4k |
| Forks | 293 | 2.6k |
| Last analyzed | Oct 3, 2026 | Oct 24, 2026 |
Key differences
- inspector focuses on MCP servers and apps, with JSON-RPC traces, protocol checks, and OAuth debugging; promptfoo focuses on prompts, models, and LLM application evaluations and red-teaming.
- inspector offers hosted, desktop, and terminal apps plus an SDK; promptfoo is available as a CLI and library, with a web viewer for evaluation results.
- inspector supports local HTTP/S and STDIO server connections, with local STDIO workflows requiring the terminal or desktop app; promptfoo describes local evaluations with prompts staying on the machine.
- inspector's repository metadata lists its license as NOASSERTION, while its README says Apache License 2.0; promptfoo is MIT licensed.
- Both are TypeScript projects. promptfoo has 25.7k stars and 2.4k forks, compared with inspector's 2.2k stars and 293 forks.
Choose inspector if you…
- need to inspect MCP tools, resources, prompts, or JSON-RPC traces.
- want to debug OAuth or check MCP protocol conformance.
- need to compare MCP client behavior or run server evaluations in CI.
Choose promptfoo if you…
- want to compare prompts or model outputs across evaluation cases.
- need to red-team an LLM application or scan for vulnerabilities.
- want a CLI or library for repeatable LLM evaluations and CI/CD checks.
Compare the numbers side by side, or add more projects
This comparison is generated with AI from the OSRepos analyses of both projects. Always check each project's repository and documentation before choosing.