giskard-oss vs promptfoo
LLM evaluation and red-teaming tools compared
giskard-oss and promptfoo help teams evaluate LLM applications and test for security risks. giskard-oss focuses on Python-based agent, multi-turn, and RAG evaluation, while promptfoo offers a CLI and library for prompt and model comparisons, with CI/CD workflows.
giskard-oss: Test and Red-Team LLM Agents
Giskard is a Python toolkit for evaluating agent behavior and probing AI systems for vulnerabilities. It suits teams building LLM agents or RAG applications that need repeatable checks, safety testing, and adversarial evaluation.
promptfoo: Evaluate and Red-Team LLM Apps
Promptfoo is a TypeScript CLI and library for evaluating and red-teaming LLM-based apps. Compare prompts and models, scan for vulnerabilities, and run checks in CI/CD, with evaluations performed locally.
| giskard-oss | promptfoo | |
|---|---|---|
| Language | Python | TypeScript |
| License | Apache-2.0 | MIT |
| Stars | 5.9k | 26.4k |
| Forks | 547 | 2.6k |
| Last analyzed | Oct 3, 2026 | Oct 24, 2026 |
Key differences
- giskard-oss is a Python toolkit centered on agent behavior, multi-turn interactions, RAG quality, and generated adversarial tests; promptfoo evaluates prompts and models and compares their outputs.
- giskard-oss provides scenario-based checks such as groundedness and semantic similarity; promptfoo emphasizes side-by-side model comparisons, a web results viewer, and evaluation workflows.
- giskard-oss is licensed under Apache-2.0, while promptfoo is licensed under MIT.
- giskard-oss v3 requires Python 3.12 or later and focuses on dynamic agent testing; promptfoo offers a CLI and library and describes local evaluation and CI/CD integration.
- giskard-oss keeps automatic tabular and ML scanning in its legacy v2, which is no longer actively maintained; promptfoo describes code scanning for LLM-related security and compliance issues in pull requests.
Choose giskard-oss if you…
- need to evaluate multi-turn agent behavior or traces.
- want checks for RAG answer quality and grounding.
- need adversarial tests focused on agent risks such as prompt injection or harmful content.
Choose promptfoo if you…
- want to compare prompt revisions or model outputs across providers.
- need a CLI or library for evaluations in local development and CI/CD.
- want to review evaluation results in a web viewer or scan pull requests for LLM-related risks.
Compare the numbers side by side, or add more projects
This comparison is generated with AI from the OSRepos analyses of both projects. Always check each project's repository and documentation before choosing.