agentic_security vs promptfoo
LLM security testing and evaluation tools compared
agentic_security and promptfoo both help teams test LLM applications and identify potential security issues. agentic_security focuses on configurable vulnerability scans using adversarial prompts, while promptfoo combines red teaming with broader prompt and model evaluation.
agentic_security: Scan LLMs and Agent Workflows for Vulnerabilities
Agentic Security is a Python vulnerability scanner and red-teaming kit for LLMs and agent workflows. It runs prompt-based scans against HTTP APIs, using supplied or built-in datasets to probe jailbreaks, fuzzing, and multimodal inputs.
promptfoo: Evaluate and Red-Team LLM Apps
Promptfoo is a TypeScript CLI and library for evaluating and red-teaming LLM-based apps. Compare prompts and models, scan for vulnerabilities, and run checks in CI/CD, with evaluations performed locally.
| agentic_security | promptfoo | |
|---|---|---|
| Language | Python | TypeScript |
| License | Apache-2.0 | MIT |
| Stars | 2k | 26.4k |
| Forks | 305 | 2.6k |
| Last analyzed | Nov 11, 2026 | Oct 24, 2026 |
Key differences
- agentic_security is a Python toolkit that sends configured prompt probes to an HTTP endpoint; promptfoo is a TypeScript CLI and library for evaluating prompts, models, and applications.
- agentic_security supports prompt mutations, multi-step attack simulations, and documented text, image, and audio request formats; promptfoo emphasizes evaluation cases, side-by-side model comparisons, and a web viewer for results.
- agentic_security documents built-in or local prompt datasets and a CI-oriented scan command; promptfoo supports local evaluations and CI/CD checks, plus pull request code scanning for LLM-related security and compliance issues.
- agentic_security is licensed under Apache-2.0; promptfoo is licensed under MIT.
- The supplied project facts list agentic_security at 2k stars and 292 forks, and promptfoo at 25.7k stars and 2.4k forks.
- Both projects are not archived; promptfoo's README states that it is now part of OpenAI while remaining open source.
Choose agentic_security if you…
- need configurable adversarial scans against an API endpoint.
- want to use prompt mutations or simulate multi-step attacks.
- need to probe text, image, or audio interfaces using documented request formats.
Choose promptfoo if you…
- want to evaluate prompts and compare model outputs across supported providers.
- need a CLI and library for repeatable evaluations as well as red-team checks.
- want to inspect evaluation results in a web viewer or add LLM-related pull request checks.
Compare the numbers side by side, or add more projects
This comparison is generated with AI from the OSRepos analyses of both projects. Always check each project's repository and documentation before choosing.