Our take
- Activity
- 68
- Community
- 89
- Issues
- 48
- Pull requests
- 38
+40 stars in the last 30 days
- Releases more often than 85% of the projects we track
- More contributors than 78% of the projects we track
- Slower to close issues than 89% of the projects we track
Project history
Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.
Stars
7,541Forks
1,194Change since Nov 17, 2026: 0 stars·+1 forks
Overview
OpenLLMetry is a Python project that extends OpenTelemetry with instrumentation for LLM applications. It helps developers inspect model and vector database activity alongside the rest of an application's telemetry, making it easier to investigate behavior and failures across an AI workflow.
Use it when you want LLM-specific traces in an existing OpenTelemetry-compatible observability system. The repository includes a convenience SDK as well as individual instrumentations that can be added to an application already using OpenTelemetry.
Key Features
- Instruments calls to LLM providers including OpenAI, Anthropic, Gemini, and Bedrock.
- Adds instrumentation for vector databases such as Chroma, Pinecone, Qdrant, and Weaviate.
- Supports frameworks and protocols including LangChain, LlamaIndex, OpenAI Agents, and MCP.
- Exports standard OpenTelemetry data to supported destinations, including Datadog, Grafana, Honeycomb, and the OpenTelemetry Collector.
- Offers a Python SDK for initializing instrumentation, or lets existing OpenTelemetry setups use instrumentations directly.
- Builds on OpenTelemetry instrumentation for other application components, such as databases and API calls.
Use Cases
- LLM application developers can trace model calls to understand errors and activity within an AI request.
- Platform and observability teams can send LLM telemetry to their existing OpenTelemetry-compatible stack.
- Teams using vector databases can inspect database activity alongside model-provider calls.
- Developers integrating frameworks such as LangChain or LlamaIndex can add framework instrumentation to their application telemetry.
Getting Started
Install the SDK and initialize it in your application:
pip install traceloop-sdk
from traceloop.sdk import Traceloop
Traceloop.init()
See the getting started guide and the README for configuration and integration details.
Alternatives
- Evidently: Evidently focuses on evaluating and monitoring model quality, drift, and LLM outputs, rather than exporting application traces and metrics through OpenTelemetry.
- agenttrail: agenttrail visualizes local AI coding-agent plans, tool calls, and file changes, rather than providing provider- and framework-level OpenTelemetry instrumentation.
| Project | Language | License | Stars | Status |
|---|---|---|---|---|
| openllmetry | Python | Apache-2.0 | 7.5k | Not checked yet |
| evidently | Jupyter Notebook | Apache-2.0 | 8k | Active |
| agenttrail | JavaScript | MIT | 913 | New |
Considerations
- You need an OpenTelemetry-compatible destination and configuration to export and inspect telemetry; the project supports multiple destinations, but setup varies by backend.
- Instrumentation depends on the providers, frameworks, and database integrations used by your application. Check the project's instrumentation documentation for relevant setup details.
- The repository reports 733 open issues, so review current issues and releases when evaluating compatibility or investigating integration problems.
Found this useful?
Share it with someone who would like openllmetry.