agent-anvil: Test AI Agent Tool Use in CI

Overview

Agent Anvil is a Python evaluation harness for teams building AI agents that call tools. It runs YAML scenarios, records agent and tool traces, and checks whether behavior satisfies workflow and safety requirements, not just whether the final answer looks good.

Use it when you need repeatable checks for issues such as calling a destructive tool too early, passing invalid arguments, skipping clarification, or violating business rules. Runs can be integrated into CI, with reports and repair suggestions to help investigate failures.

Key Features

• Define agent evaluation scenarios in YAML and run repeatable trials.
• Check trace invariants such as tool-call order, call limits, arguments, tool results, and final output.
• Record traces for demo agents or external agents connected through command or HTTP adapters.
• Apply policy checks and optional OpenAI semantic grading alongside deterministic assertions.
• Group failures into clusters and generate reports and repair plans.
• Run offline scenarios without OpenAI credentials.
• Export run summaries for GitHub Actions and use the provided GitHub Action in CI.

Use Cases

• Agent developers can add regression tests for tool workflows before changing prompts, tools, or guardrails.
• Product and safety teams can check that consequential actions require the right prerequisites, such as verifying an order before issuing a refund.
• CI maintainers can fail builds when an agent's tool behavior regresses and attach run artifacts for review.
• Teams integrating an existing command-line or HTTP agent can check its compatibility and evaluate it without replacing its runtime.

Getting Started

The README documents installation with uv sync --group dev and an offline example run:

uv sync --group dev
uv run anvil run scenarios/external_jsonl_agent.yaml --offline

See the README for scenario authoring, adapters, CI setup, and additional commands.

Alternatives

• agentevals: AgentEvals focuses on reusable evaluators for agent steps and trajectories, while Agent Anvil runs scenario-based checks with trace rules and optional semantic grading.
• promptfoo: Promptfoo evaluates prompts, models, and application security broadly, while Agent Anvil centers on scenario-based testing of tool-using agent behavior.
• giskard-oss: Giskard combines agent and RAG evaluation with vulnerability probing, while Agent Anvil focuses on testing tool-using agents through scenarios and trace checks.

Considerations

• The README specifies Python 3.12 or newer.
• Semantic grading uses OpenAI models, while deterministic offline runs do not require OpenAI credentials.
• Local run artifacts retain raw traces. Review them before sharing, even though semantic-grading payloads are redacted by default.
• Generated repair plans and helper patches are suggestions and scaffolding, not an automatic fix or substitute for domain review.
• The repository has 9 stars and 2 forks in the supplied project facts, so teams should assess its maturity and fit before relying on it for critical evaluations.