future-agi

future-agi: Evaluate, Trace, and Improve AI Agents

Future AGI is a self-hostable platform for evaluating, observing, and improving LLM and AI agent applications, with OpenTelemetry-native tracing at its core. It connects simulations, evaluations, guardrails, and an LLM gateway in a shared feedback loop.

ActivePythonApache-2.0AI AgentsEvaluationObservability
Stars
2.3k
Forks
721
Last commit
today
Contributors
97
Releases, 12 months
73

Our take

Activeregular commits and releases

Health score

66/100

How it is scored
Activity
87
Community
82
Issues
46
Pull requests
47
  • Releases more often than 90% of the projects we track
  • Merges more pull requests than 78% of the projects we track
  • More contributors than 77% of the projects we track
  • Slower to merge pull requests than 83% of the projects we track

This is a broad, actively maintained platform for teams that want to bring agent testing, tracing, and runtime protections together, rather than adopt a single-purpose evaluation or observability tool.

Good fit if

  • You need simulations, evaluations, OpenTelemetry tracing, and guardrails in one agent-development workflow.
  • You want to self-host an Apache 2.0 platform and can operate its Docker-based stack.
  • You value an active project with regular releases and contributions from a broad group of developers.

Look elsewhere if

  • You need only a lightweight SDK or one standalone capability rather than a multi-component platform.
  • You cannot provide Docker Compose and the resources needed for its self-hosted setup.
  • Your deployment depends on a specific capability not listed in the README, such as a supported migration from a Standalone install to a Distributed one.
All health signals
Last commit2026-11-18 (today)
Commits, last 90 days100+
Releases, last 12 months73 (latest v1.58.0, 2026-11-03)
Contributors97 (top contributor: 32% of commits)
Issues closed, last 90 days1+
Pull requests merged, last 90 days62 (typically merged in 3 days)
Project age7 months

Checked on 2026-11-18 with the GitHub API.

Overview

Future AGI is an open-source platform for evaluating, observing, and improving LLM and AI agent applications. Built around OpenTelemetry-native tracing, it aims to solve the problem of teams stitching together separate tools for testing, production monitoring, and user protection without a shared feedback loop.

The platform combines simulations, evaluation, tracing, guardrails, an OpenAI-compatible gateway, and prompt optimization. Teams can use the managed cloud or self-host the platform; the README describes Docker Compose deployments and a Kubernetes Helm option. Its Apache 2.0 core is intended for teams that want to inspect and control their evaluation logic and data.

Key Features

  • Instrument AI applications with OpenTelemetry tracing, with integrations for more than 50 frameworks.
  • Evaluate outputs using metrics for areas such as groundedness, hallucinations, tool use, and PII.
  • Simulate conversations with personas, adversarial inputs, and edge cases, including voice-agent scenarios.
  • Apply built-in guardrail scanners or connect vendor adapters to check inputs and outputs.
  • Route model requests through an OpenAI-compatible gateway with provider integrations, routing strategies, and semantic caching.
  • Use production traces as feedback for prompt optimization and subsequent evaluations.

Use Cases

  • An AI application team can use simulations and evaluations to test an agent against edge cases before release.
  • An operations team can trace agent runs in production and inspect spans, latency, and token cost.
  • A team building a customer-facing agent can apply guardrails to screen for risks such as PII, jailbreaks, and prompt injection.
  • Developers working with voice agents can simulate and evaluate conversations across supported voice platforms.

What you need

Detected in the repository

  • Node.js (from package.json)
  • A Dockerfile, so it can run in a container; a Compose file sets up the related services
  • A test suite and automated checks on GitHub Actions

License in plain words

Apache-2.0permissive

  • Commercial use: yes
  • Modify and redistribute: yes
  • You must keep: the license, the NOTICE file and a note of your changes
  • Share your changes: no
  • Includes an explicit patent grant from the contributors:

A summary, not legal advice: the LICENSE file is what applies.

Getting Started

For self-hosting, the README requires Docker Desktop or Docker Engine with Compose v2.24 or newer. Clone the repository and run the installer:

git clone https://github.com/future-agi/future-agi.git
cd future-agi
./bin/install

See the README for configuration, cloud setup, deployment choices, and tracing examples.

Alternatives

  • phoenix: Phoenix focuses on self-hosted tracing, evaluation, and troubleshooting, while Future AGI also combines simulations, guardrails, and an LLM gateway.
  • langwatch: LangWatch combines observability and simulation testing with prompt management and governance, while Future AGI centers its feedback loop on OpenTelemetry-native tracing.
  • opik: Opik offers tracing, experiments, datasets, and prompt tools, while Future AGI additionally brings simulations, guardrails, and an LLM gateway into its platform.
  • weave: Weave focuses on tracing and evaluating application runs, while Future AGI adds agent simulations, guardrails, and an LLM gateway.
ProjectLanguageLicenseStarsStatus
future-agiPythonApache-2.02.3kActive
phoenixPythonOther11.7kActive
langwatchTypeScriptApache-2.04.9kActive
opikPythonApache-2.022.8kActive
weavePythonApache-2.01.1kActive

Considerations

  • The default self-hosted Standalone setup requires Docker to have 2 vCPUs and 4 GB of memory available; the installer downloads roughly 800 MB of images. The README lists higher resource needs for Distributed deployments.
  • The project is actively developed, with recent commits and releases and contributions from many people. Its measured issue handling is less clear: only a small number of issues were closed in the recent period, and there is no median close time available.
  • The README says data cannot be moved from a Standalone install to Distributed or Helm later through a supported path, so choose the deployment model before adding data.
  • Self-hosted telemetry is enabled by default. The README documents its payload and opt-out settings; teams with strict outbound-network policies should review these before deployment.

Found this useful?

Share it with someone who would like future-agi.