cfahlgren1

observers: Track and Store AI API Interactions

Observers wraps generative AI clients to capture interactions and sync them to storage backends. It suits Python teams that need lightweight observability across supported LLM providers, with storage options ranging from DuckDB to OpenTelemetry-compatible services.

View on GitHub Compare
Stars
255
Forks
28
Stars, last 30 days
+0

Our take

Health score

25/100

How it is scored
Activity
19
Community
40
Issues
21
Pull requests
20

+0 stars in the last 30 days

Project history

Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.

Selected snapshot

Stars

255

Forks

28

Change since Nov 15, 2026: 0 stars·0 forks

Overview

Observers is a Python library for recording interactions with generative AI APIs. It separates instrumentation from persistence: observers wrap supported clients, and stores sync captured observations to a selected backend.

This approach is useful when a team wants to inspect or query model requests and responses without replacing its existing client integration. The project supports local and hosted storage options, with features such as annotation and filtering varying by store.

Key Features

  • Wraps OpenAI-compatible clients, Hugging Face Transformers and Inference Client, AISuite, and LiteLLM.
  • Supports synchronous and asynchronous observer versions.
  • Captures interactions while keeping the wrapped client workflow familiar.
  • Stores observations in DuckDB, Hugging Face Datasets, Argilla, or OpenTelemetry-compatible backends.
  • Supports viewing or querying data through backend-specific tools, including SQL options for some stores.
  • Offers optional integrations through install extras for AISuite, LiteLLM, and OpenTelemetry.

Use Cases

  • Python application developers can record model requests and responses while testing or operating an LLM integration.
  • Teams comparing multiple OpenAI-compatible providers can apply a consistent observation layer across clients.
  • Developers who want local inspection can send observations to DuckDB and query them with SQL.
  • Teams using Argilla can sync observations there when they need its annotation and review workflow.
  • Organizations already using OpenTelemetry-compatible observability providers can route observations into their existing trace data workflow.

Getting Started

Install the package:

pip install observers

See the README for setup examples, provider support, and store configuration.

Alternatives

  • openllmetry: OpenLLMetry instruments LLM apps with OpenTelemetry traces and metrics, while observers captures client interactions and syncs them to storage backends.
  • LangWatch: LangWatch is a platform for LLM evaluation, agent testing, and production monitoring, while observers is a Python client-wrapping library for capturing and storing interactions.
ProjectLanguageLicenseStarsStatus
observersPython–255Not checked yet
openllmetryPythonApache-2.07.5kActive
langwatchTypeScriptApache-2.04.9kActive

Considerations

  • The README describes supported integrations, but does not specify a license in the provided repository metadata.
  • Store capabilities differ. Annotation and UI or SQL filtering are not available uniformly across backends.
  • Hosted stores may require provider accounts or configuration. For example, the README's Hugging Face Datasets workflow pushes data to a Hub workspace.
  • OpenTelemetry store capabilities depend on the provider being used.
  • The repository metadata shows its last push on 2025-02-20, so check the repository for current compatibility and maintenance details before adopting it.

Found this useful?

Share it with someone who would like observers.

Comparisons