{"name":"future-agi: Evaluate, Trace, and Improve AI Agents","description":"Future AGI is a self-hostable platform for evaluating, observing, and improving LLM and AI agent applications, with OpenTelemetry-native tracing at its core. It connects simulations, evaluations, guardrails, and an LLM gateway in a shared feedback loop.","github":"https://github.com/future-agi/future-agi","url":"https://osrepos.com/repo/future-agi-future-agi","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/future-agi-future-agi","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/future-agi-future-agi.md","json":"https://osrepos.com/repo/future-agi-future-agi.json","topics":["python","ai-agents","evaluation","observability","self-hosted","llm-tracing","agent-simulation"],"keywords":["python","ai-agents","evaluation","observability","self-hosted","llm-tracing","agent-simulation"],"stars":null,"summary":"Future AGI is a self-hostable platform for evaluating, observing, and improving LLM and AI agent applications, with OpenTelemetry-native tracing at its core. It connects simulations, evaluations, guardrails, and an LLM gateway in a shared feedback loop.","content":"## Overview\n\nFuture AGI is an open-source platform for evaluating, observing, and improving LLM and AI agent applications. Built around OpenTelemetry-native tracing, it aims to solve the problem of teams stitching together separate tools for testing, production monitoring, and user protection without a shared feedback loop.\n\nThe platform combines simulations, evaluation, tracing, guardrails, an OpenAI-compatible gateway, and prompt optimization. Teams can use the managed cloud or self-host the platform; the README describes Docker Compose deployments and a Kubernetes Helm option. Its Apache 2.0 core is intended for teams that want to inspect and control their evaluation logic and data.\n\n## Key Features\n\n- Instrument AI applications with OpenTelemetry tracing, with integrations for more than 50 frameworks.\n- Evaluate outputs using metrics for areas such as groundedness, hallucinations, tool use, and PII.\n- Simulate conversations with personas, adversarial inputs, and edge cases, including voice-agent scenarios.\n- Apply built-in guardrail scanners or connect vendor adapters to check inputs and outputs.\n- Route model requests through an OpenAI-compatible gateway with provider integrations, routing strategies, and semantic caching.\n- Use production traces as feedback for prompt optimization and subsequent evaluations.\n\n## Use Cases\n\n- An AI application team can use simulations and evaluations to test an agent against edge cases before release.\n- An operations team can trace agent runs in production and inspect spans, latency, and token cost.\n- A team building a customer-facing agent can apply guardrails to screen for risks such as PII, jailbreaks, and prompt injection.\n- Developers working with voice agents can simulate and evaluate conversations across supported voice platforms.\n\n## Our Take\n\nThis is a broad, actively maintained platform for teams that want to bring agent testing, tracing, and runtime protections together, rather than adopt a single-purpose evaluation or observability tool.\n\n**Good fit if:**\n- You need simulations, evaluations, OpenTelemetry tracing, and guardrails in one agent-development workflow.\n- You want to self-host an Apache 2.0 platform and can operate its Docker-based stack.\n- You value an active project with regular releases and contributions from a broad group of developers.\n\n**Look elsewhere if:**\n- You need only a lightweight SDK or one standalone capability rather than a multi-component platform.\n- You cannot provide Docker Compose and the resources needed for its self-hosted setup.\n- Your deployment depends on a specific capability not listed in the README, such as a supported migration from a Standalone install to a Distributed one.\n\n## Project Health\n\n| Signal | Value |\n|---|---|\n| Status | **Active**: regular commits and releases |\n| Last commit | 2026-11-18 (today) |\n| Commits, last 90 days | 100+ |\n| Releases, last 12 months | 73 (latest v1.58.0, 2026-11-03) |\n| Contributors | 97 (top contributor: 32% of commits) |\n| Issues closed, last 90 days | 1+ |\n| Pull requests merged, last 90 days | 62 (typically merged in 3 days) |\n| Project age | 7 months |\n\nChecked on 2026-11-18 with the GitHub API.\n\n## Project Facts\n\n- Language: Python\n- License: Apache-2.0\n- Stars: 2.3k\n- Forks: 721\n- Topics: agent-evaluation, ai-agents, ai-evals, ai-gateway, ai-observability, evals, evaluation-framework, guardrails\n- Website: https://futureagi.com\n- Archived: no\n\n## What You Need\n\nDetected in the repository:\n\n- Node.js (from package.json)\n- A Dockerfile, so it can run in a container; a Compose file sets up the related services\n- A test suite and automated checks on GitHub Actions\n\n## Getting Started\n\nFor self-hosting, the README requires Docker Desktop or Docker Engine with Compose v2.24 or newer. Clone the repository and run the installer:\n\n```bash\ngit clone https://github.com/future-agi/future-agi.git\ncd future-agi\n./bin/install\n```\n\nSee the [README](https://github.com/future-agi/future-agi) for configuration, cloud setup, deployment choices, and tracing examples.\n\n## License in Plain Words\n\n**Apache-2.0** (permissive).\n\n- Commercial use: yes\n- Modify and redistribute: yes\n- You must keep: the license, the NOTICE file and a note of your changes\n- Share your changes: no\n- Includes an explicit patent grant from the contributors\n\nA summary, not legal advice: the LICENSE file is what applies.\n\n## Alternatives\n\n- [phoenix](https://osrepos.com/repo/arize-ai-phoenix): Phoenix focuses on self-hosted tracing, evaluation, and troubleshooting, while Future AGI also combines simulations, guardrails, and an LLM gateway.\n- [langwatch](https://osrepos.com/repo/langwatch-langwatch): LangWatch combines observability and simulation testing with prompt management and governance, while Future AGI centers its feedback loop on OpenTelemetry-native tracing.\n- [opik](https://osrepos.com/repo/comet-ml-opik): Opik offers tracing, experiments, datasets, and prompt tools, while Future AGI additionally brings simulations, guardrails, and an LLM gateway into its platform.\n- [weave](https://osrepos.com/repo/wandb-weave): Weave focuses on tracing and evaluating application runs, while Future AGI adds agent simulations, guardrails, and an LLM gateway.\n\n| Project | Language | License | Stars | Status |\n|---|---|---|---|---|\n| **future-agi** | Python | Apache-2.0 | 2.3k | Active |\n| [phoenix](https://osrepos.com/repo/arize-ai-phoenix) | Python | Other | 11.7k | Active |\n| [langwatch](https://osrepos.com/repo/langwatch-langwatch) | TypeScript | Apache-2.0 | 4.9k | Active |\n| [opik](https://osrepos.com/repo/comet-ml-opik) | Python | Apache-2.0 | 22.8k | Active |\n| [weave](https://osrepos.com/repo/wandb-weave) | Python | Apache-2.0 | 1.1k | Active |\n\n## Considerations\n\n- The default self-hosted Standalone setup requires Docker to have 2 vCPUs and 4 GB of memory available; the installer downloads roughly 800 MB of images. The README lists higher resource needs for Distributed deployments.\n- The project is actively developed, with recent commits and releases and contributions from many people. Its measured issue handling is less clear: only a small number of issues were closed in the recent period, and there is no median close time available.\n- The README says data cannot be moved from a Standalone install to Distributed or Helm later through a supported path, so choose the deployment model before adding data.\n- Self-hosted telemetry is enabled by default. The README documents its payload and opt-out settings; teams with strict outbound-network policies should review these before deployment.","metrics":{"detailViews":2,"githubClicks":1},"dates":{"published":null,"modified":"2026-11-18T05:20:37.000Z"}}