agentevals vs weave
AI application tracing and evaluation compared
agentevals evaluates recorded OpenTelemetry traces to score agent behavior, while weave traces Python AI applications and supports evaluations across experiments and production. The main difference is that agentevals focuses on scoring existing traces, whereas weave combines tracing with application workflow and repeatable comparisons.
agentevals: Evaluate AI Agents from OpenTelemetry Traces
agentevals scores AI agent behavior from existing OpenTelemetry traces, without rerunning agents or making extra model calls. It suits teams building instrumented agents that need local evaluation, golden-set checks, or CI quality gates.
weave: Trace and Evaluate Generative AI Applications
Weave helps developers trace language model inputs, outputs, and function calls, then organize and evaluate application runs. It suits teams building AI features that need debugging and repeatable comparisons across experiments and production.
| agentevals | weave | |
|---|---|---|
| Language | Python | Python |
| License | Apache-2.0 | Apache-2.0 |
| Stars | 162 | 1.1k |
| Forks | 28 | 171 |
| Last analyzed | Oct 4, 2026 | Oct 3, 2026 |
Key differences
- agentevals reads Jaeger JSON and OTLP traces and evaluates recorded runs; weave captures inputs, outputs, and nested function calls in Python applications.
- agentevals supports golden-set checks for tool trajectories and responses, custom evaluators, and LLM-based judges; weave supports structured evaluations for comparing language model use cases.
- agentevals offers an offline CLI, a web UI with an OTLP receiver, and an MCP server; weave integrates tracing into Python code with the `weave.op` decorator.
- agentevals can run basic evaluations locally without a required cloud service or database; weave requires Python 3.10 or higher and a Weights & Biases account.
- Both projects use Python and the Apache-2.0 license; agentevals is under active development and warns of possible breaking changes, while weave describes cleanup work and a focus on tracing and evaluations.
Choose agentevals if you…
- need to score existing OpenTelemetry traces without rerunning agents or making extra model calls.
- want golden-set checks for tool trajectories or responses in local evaluation or CI.
- need an offline CLI or live trace inspection through a web UI and OTLP receiver.
Choose weave if you…
- want to trace inputs, outputs, and nested function calls in Python applications.
- need repeatable evaluations to compare model or prompt changes.
- want tracing and evaluation organized across experimentation and production.
Compare the numbers side by side, or add more projects
This comparison is generated with AI from the OSRepos analyses of both projects. Always check each project's repository and documentation before choosing.