agentic_security vs JailbreakEval
LLM security scanning and jailbreak evaluation compared
agentic_security probes LLM APIs and agent workflows with attack prompts to find unsafe behavior. JailbreakEval evaluates responses to jailbreak attempts using multiple scoring methods, so the main difference is scanning targets for vulnerabilities versus comparing evaluators of attack outcomes.
agentic_security: Scan LLMs and Agent Workflows for Vulnerabilities
Agentic Security is a Python vulnerability scanner and red-teaming kit for LLMs and agent workflows. It runs prompt-based scans against HTTP APIs, using supplied or built-in datasets to probe jailbreaks, fuzzing, and multimodal inputs.
JailbreakEval: Compare LLM Jailbreak Evaluators
JailbreakEval brings together automated methods for assessing whether language-model responses comply with jailbreak attempts. Researchers can compare evaluators across datasets, while developers can build and benchmark new evaluation methods.
| agentic_security | JailbreakEval | |
|---|---|---|
| Language | Python | Python |
| License | Apache-2.0 | MIT |
| Stars | 2k | 199 |
| Forks | 305 | 13 |
| Last analyzed | Nov 11, 2026 | Oct 3, 2026 |
Key differences
- agentic_security runs jailbreak, fuzzing, and multimodal probes against configurable HTTP endpoints; JailbreakEval scores responses to harmful prompts.
- agentic_security supports scans through a UI, CLI, or GitHub Actions; JailbreakEval offers a Python API and a CLI for CSV evaluation.
- agentic_security accepts text, image, and audio probes on compatible endpoints; JailbreakEval provides string-matching, chat-based, text-classification, and voting evaluators.
- agentic_security is licensed under Apache-2.0 and has 2k stars; JailbreakEval is MIT-licensed and has 199 stars.
- agentic_security is aimed at teams scanning their LLM APIs and workflows; JailbreakEval is aimed at researchers comparing evaluators and developers benchmarking evaluation methods.
Choose agentic_security if you…
- need to scan an LLM API or agent workflow for unsafe behavior.
- want to run prompt-based, fuzzing, or multimodal probes through a UI, CLI, or CI workflow.
- need configurable scan budgets and failure thresholds for CI checks.
Choose JailbreakEval if you…
- want to compare different automated evaluators on jailbreak attempts.
- need to evaluate CSV datasets and report agreement with supplied labels.
- plan to implement or benchmark a custom jailbreak evaluator.
Compare the numbers side by side, or add more projects
This comparison is generated with AI from the OSRepos analyses of both projects. Always check each project's repository and documentation before choosing.