# FreeLLMAPI: Stack 16 Free LLM Tiers for 1.7 Billion Tokens/Month

This repository profile is provided by osrepos.com, an open source repository discovery platform.

Source: osrepos.com
Repository profile: https://osrepos.com/repo/tashfeenahmed-freellmapi
Generated for open source discovery and AI-assisted research.

FreeLLMAPI is an OpenAI-compatible proxy that aggregates the free tiers of 16 LLM providers, offering access to approximately 1.7 billion tokens per month. It simplifies access to diverse models through a single endpoint, featuring smart routing, automatic failover, and encrypted key storage. This powerful tool is designed for personal experimentation, allowing developers to leverage multiple free LLM resources efficiently.

GitHub: https://github.com/tashfeenahmed/freellmapi
OSRepos URL: https://osrepos.com/repo/tashfeenahmed-freellmapi

## Summary

FreeLLMAPI is an OpenAI-compatible proxy that aggregates the free tiers of 16 LLM providers, offering access to approximately 1.7 billion tokens per month. It simplifies access to diverse models through a single endpoint, featuring smart routing, automatic failover, and encrypted key storage. This powerful tool is designed for personal experimentation, allowing developers to leverage multiple free LLM resources efficiently.

## Topics

- TypeScript
- LLM
- AI
- Proxy
- OpenAI
- API Gateway
- Developer Tools
- Open Source

## Repository Information

Last analyzed by OSRepos: Sat Jun 27 2026 13:18:15 GMT+0100 (Western European Summer Time)
Detail views: 2
GitHub clicks: 1

## Safety Notice

OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.

## Content

## Introduction

FreeLLMAPI is an innovative, self-hosted, OpenAI-compatible proxy designed to consolidate the free tiers of 16 different Large Language Model (LLM) providers. By stacking these free resources, it offers access to an impressive approximately 1.7 billion tokens per month through a single `/v1` endpoint. This project aims to simplify the use of various LLMs for personal experimentation, providing features like smart routing, automatic failover, and secure, encrypted key storage.

## Why Use FreeLLMAPI?

Individually, the free tiers offered by major AI labs often feel limited. However, managing multiple SDKs, navigating different rate limits, and handling potential request failures across many providers can be a significant challenge. FreeLLMAPI solves this by collapsing all these complexities into one unified, OpenAI-compatible endpoint. This allows you to point any OpenAI client library at your local FreeLLMAPI server, which then transparently routes requests across the providers for which you've added keys. The aggregated capacity transforms individual "toy" tiers into a substantial working inference capacity.

## Key Features

FreeLLMAPI comes packed with features to enhance your LLM experimentation:

*   **OpenAI-compatible API**: Works seamlessly with official OpenAI SDKs and any OpenAI-compatible client (LangChain, LlamaIndex, etc.) by simply changing the `base_url`.
*   **Anthropic Messages API Support**: Integrates with Anthropic's wire format, allowing Claude clients and SDKs to run against your free LLM pool.
*   **Image Generation & Text-to-Speech**: Routes requests for media models across supported providers.
*   **Streaming and Non-Streaming**: Supports both Server-Sent Events for streaming and standard JSON responses.
*   **Tool Calling**: Passes through OpenAI-style `tools` and `tool_choice` requests, supporting multi-step tool-use flows.
*   **Embeddings**: Provides a `/v1/embeddings` endpoint with family-based routing, ensuring failover only occurs between compatible models.
*   **Automatic Fallover**: Automatically retries requests on the next available model in your fallback chain if a provider returns a 429, 5xx, or times out.
*   **Per-Key Rate Tracking**: Monitors RPM, RPD, TPM, and TPD counters for each `(platform, model, key)` to stay within free-tier caps.
*   **Encrypted Key Storage**: API keys are encrypted with AES-256-GCM for enhanced security.
*   **Unified API Key**: Clients authenticate to your proxy with a single `freellmapi-...` bearer token, keeping upstream provider keys private.
*   **Admin Dashboard**: A React + Vite UI for managing keys, reordering the fallback chain, inspecting analytics, and using a prompt playground.
*   **Context Handoff**: Optionally injects a compact system message when switching models mid-conversation to improve continuity.

## Installation

Getting FreeLLMAPI up and running is straightforward.

### One-liner (Docker Required)

For a quick setup, use the provided install script:

bash
curl -fsSL https://freellmapi.co/install.sh | bash


This script sets up `~/freellmapi`, generates an encryption key, pulls the Docker image, and starts the container.

### Manual Docker Compose

If you prefer a manual Docker Compose setup:

1.  Clone the repository:
    bash
    git clone https://github.com/tashfeenahmed/freellmapi.git
    cd freellmapi
    
2.  Generate an encryption key and create a `.env` file:
    bash
    ENCRYPTION_KEY="$(openssl rand -hex 32)"
    printf "ENCRYPTION_KEY=%s\\nPORT=3001\\n" "$ENCRYPTION_KEY" > .env
    
3.  Start the services:
    bash
    docker compose up -d
    
    Access the dashboard at `http://localhost:3001`. Remember to add your provider keys and configure the fallback chain.

### Desktop App

A native menu-bar application is available for macOS and Windows, providing a local router and dashboard directly from your system tray. You can download the latest `.dmg` or `.exe` installer from the [GitHub Releases page](https://github.com/tashfeenahmed/freellmapi/releases/latest){:target="_blank"}.

## Examples

Once FreeLLMAPI is running, you can use any OpenAI-compatible client.

### Python

python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",  # let the router pick; or specify e.g. "gemini-2.5-flash"
    messages=[{"role": "user", "content": "Summarise the fall of Rome in one sentence."}],
)
print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))


### curl

bash
curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "hi"}]
  }'


### Streaming

python
stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Stream me a haiku about SQLite."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)


### Tool Calling

FreeLLMAPI supports OpenAI-style tool calling, allowing complex interactions with your LLMs.

python
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

# 1. Model asks for a tool call
first = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "What's the weather in Karachi?"}],
    tools=tools,
    tool_choice="required",
)
call = first.choices[0].message.tool_calls[0]

# 2. You execute the tool, feed the result back
final = client.chat.completions.create(
    model="auto",
    messages=[
        {"role": "user", "content": "What's the weather in Karachi?"},
        first.choices[0].message,
        {"role": "tool", "tool_call_id": call.id, "content": '{"temp_c": 32, "cond": "sunny"}'},
    ],
    tools=tools,
)
print(final.choices[0].message.content)


### Vision / Image Input

Send images using standard OpenAI `image_url` content blocks. The router automatically restricts requests to vision-capable models.

python
resp = client.chat.completions.create(
    model="auto",  # auto-routes to a vision model
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {"url": "data:image/png;base64,<...>"}},
        ],
    }],
)
print(resp.choices[0].message.content)


## Links

*   **GitHub Repository**: [tashfeenahmed/freellmapi](https://github.com/tashfeenahmed/freellmapi){:target="_blank"}
*   **Official Website**: [freellmapi.co](https://freellmapi.co){:target="_blank"}
*   **Desktop App Releases**: [GitHub Releases](https://github.com/tashfeenahmed/freellmapi/releases/latest){:target="_blank"}