{"name":"FreeLLMAPI: Stack 16 Free LLM Tiers for 1.7 Billion Tokens/Month","description":"FreeLLMAPI is an OpenAI-compatible proxy that aggregates the free tiers of 16 LLM providers, offering access to approximately 1.7 billion tokens per month. It simplifies access to diverse models through a single endpoint, featuring smart routing, automatic failover, and encrypted key storage. This powerful tool is designed for personal experimentation, allowing developers to leverage multiple free LLM resources efficiently.","github":"https://github.com/tashfeenahmed/freellmapi","url":"https://osrepos.com/repo/tashfeenahmed-freellmapi","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/tashfeenahmed-freellmapi","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/tashfeenahmed-freellmapi.md","json":"https://osrepos.com/repo/tashfeenahmed-freellmapi.json","topics":["TypeScript","LLM","AI","Proxy","OpenAI","API Gateway","Developer Tools","Open Source"],"keywords":["TypeScript","LLM","AI","Proxy","OpenAI","API Gateway","Developer Tools","Open Source"],"stars":null,"summary":"FreeLLMAPI is an OpenAI-compatible proxy that aggregates the free tiers of 16 LLM providers, offering access to approximately 1.7 billion tokens per month. It simplifies access to diverse models through a single endpoint, featuring smart routing, automatic failover, and encrypted key storage. This powerful tool is designed for personal experimentation, allowing developers to leverage multiple free LLM resources efficiently.","content":"## Introduction\n\nFreeLLMAPI is an innovative, self-hosted, OpenAI-compatible proxy designed to consolidate the free tiers of 16 different Large Language Model (LLM) providers. By stacking these free resources, it offers access to an impressive approximately 1.7 billion tokens per month through a single `/v1` endpoint. This project aims to simplify the use of various LLMs for personal experimentation, providing features like smart routing, automatic failover, and secure, encrypted key storage.\n\n## Why Use FreeLLMAPI?\n\nIndividually, the free tiers offered by major AI labs often feel limited. However, managing multiple SDKs, navigating different rate limits, and handling potential request failures across many providers can be a significant challenge. FreeLLMAPI solves this by collapsing all these complexities into one unified, OpenAI-compatible endpoint. This allows you to point any OpenAI client library at your local FreeLLMAPI server, which then transparently routes requests across the providers for which you've added keys. The aggregated capacity transforms individual \"toy\" tiers into a substantial working inference capacity.\n\n## Key Features\n\nFreeLLMAPI comes packed with features to enhance your LLM experimentation:\n\n*   **OpenAI-compatible API**: Works seamlessly with official OpenAI SDKs and any OpenAI-compatible client (LangChain, LlamaIndex, etc.) by simply changing the `base_url`.\n*   **Anthropic Messages API Support**: Integrates with Anthropic's wire format, allowing Claude clients and SDKs to run against your free LLM pool.\n*   **Image Generation & Text-to-Speech**: Routes requests for media models across supported providers.\n*   **Streaming and Non-Streaming**: Supports both Server-Sent Events for streaming and standard JSON responses.\n*   **Tool Calling**: Passes through OpenAI-style `tools` and `tool_choice` requests, supporting multi-step tool-use flows.\n*   **Embeddings**: Provides a `/v1/embeddings` endpoint with family-based routing, ensuring failover only occurs between compatible models.\n*   **Automatic Fallover**: Automatically retries requests on the next available model in your fallback chain if a provider returns a 429, 5xx, or times out.\n*   **Per-Key Rate Tracking**: Monitors RPM, RPD, TPM, and TPD counters for each `(platform, model, key)` to stay within free-tier caps.\n*   **Encrypted Key Storage**: API keys are encrypted with AES-256-GCM for enhanced security.\n*   **Unified API Key**: Clients authenticate to your proxy with a single `freellmapi-...` bearer token, keeping upstream provider keys private.\n*   **Admin Dashboard**: A React + Vite UI for managing keys, reordering the fallback chain, inspecting analytics, and using a prompt playground.\n*   **Context Handoff**: Optionally injects a compact system message when switching models mid-conversation to improve continuity.\n\n## Installation\n\nGetting FreeLLMAPI up and running is straightforward.\n\n### One-liner (Docker Required)\n\nFor a quick setup, use the provided install script:\n\nbash\ncurl -fsSL https://freellmapi.co/install.sh | bash\n\n\nThis script sets up `~/freellmapi`, generates an encryption key, pulls the Docker image, and starts the container.\n\n### Manual Docker Compose\n\nIf you prefer a manual Docker Compose setup:\n\n1.  Clone the repository:\n    bash\n    git clone https://github.com/tashfeenahmed/freellmapi.git\n    cd freellmapi\n    \n2.  Generate an encryption key and create a `.env` file:\n    bash\n    ENCRYPTION_KEY=\"$(openssl rand -hex 32)\"\n    printf \"ENCRYPTION_KEY=%s\\\\nPORT=3001\\\\n\" \"$ENCRYPTION_KEY\" > .env\n    \n3.  Start the services:\n    bash\n    docker compose up -d\n    \n    Access the dashboard at `http://localhost:3001`. Remember to add your provider keys and configure the fallback chain.\n\n### Desktop App\n\nA native menu-bar application is available for macOS and Windows, providing a local router and dashboard directly from your system tray. You can download the latest `.dmg` or `.exe` installer from the [GitHub Releases page](https://github.com/tashfeenahmed/freellmapi/releases/latest){:target=\"_blank\"}.\n\n## Examples\n\nOnce FreeLLMAPI is running, you can use any OpenAI-compatible client.\n\n### Python\n\npython\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"http://localhost:3001/v1\",\n    api_key=\"freellmapi-your-unified-key\",\n)\n\nresp = client.chat.completions.create(\n    model=\"auto\",  # let the router pick; or specify e.g. \"gemini-2.5-flash\"\n    messages=[{\"role\": \"user\", \"content\": \"Summarise the fall of Rome in one sentence.\"}],\n)\nprint(resp.choices[0].message.content)\nprint(\"Routed via:\", resp.headers.get(\"x-routed-via\"))\n\n\n### curl\n\nbash\ncurl http://localhost:3001/v1/chat/completions \\\n  -H \"Authorization: Bearer freellmapi-your-unified-key\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"auto\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"hi\"}]\n  }'\n\n\n### Streaming\n\npython\nstream = client.chat.completions.create(\n    model=\"auto\",\n    messages=[{\"role\": \"user\", \"content\": \"Stream me a haiku about SQLite.\"}],\n    stream=True,\n)\nfor chunk in stream:\n    print(chunk.choices[0].delta.content or \"\", end=\"\", flush=True)\n\n\n### Tool Calling\n\nFreeLLMAPI supports OpenAI-style tool calling, allowing complex interactions with your LLMs.\n\npython\ntools = [{\n    \"type\": \"function\",\n    \"function\": {\n        \"name\": \"get_weather\",\n        \"description\": \"Get current weather for a city.\",\n        \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\"city\": {\"type\": \"string\"}},\n            \"required\": [\"city\"],\n        },\n    },\n}]\n\n# 1. Model asks for a tool call\nfirst = client.chat.completions.create(\n    model=\"auto\",\n    messages=[{\"role\": \"user\", \"content\": \"What's the weather in Karachi?\"}],\n    tools=tools,\n    tool_choice=\"required\",\n)\ncall = first.choices[0].message.tool_calls[0]\n\n# 2. You execute the tool, feed the result back\nfinal = client.chat.completions.create(\n    model=\"auto\",\n    messages=[\n        {\"role\": \"user\", \"content\": \"What's the weather in Karachi?\"},\n        first.choices[0].message,\n        {\"role\": \"tool\", \"tool_call_id\": call.id, \"content\": '{\"temp_c\": 32, \"cond\": \"sunny\"}'},\n    ],\n    tools=tools,\n)\nprint(final.choices[0].message.content)\n\n\n### Vision / Image Input\n\nSend images using standard OpenAI `image_url` content blocks. The router automatically restricts requests to vision-capable models.\n\npython\nresp = client.chat.completions.create(\n    model=\"auto\",  # auto-routes to a vision model\n    messages=[{\n        \"role\": \"user\",\n        \"content\": [\n            {\"type\": \"text\", \"text\": \"What's in this image?\"},\n            {\"type\": \"image_url\", \"image_url\": {\"url\": \"data:image/png;base64,<...>\"}},\n        ],\n    }],\n)\nprint(resp.choices[0].message.content)\n\n\n## Links\n\n*   **GitHub Repository**: [tashfeenahmed/freellmapi](https://github.com/tashfeenahmed/freellmapi){:target=\"_blank\"}\n*   **Official Website**: [freellmapi.co](https://freellmapi.co){:target=\"_blank\"}\n*   **Desktop App Releases**: [GitHub Releases](https://github.com/tashfeenahmed/freellmapi/releases/latest){:target=\"_blank\"}","metrics":{"detailViews":2,"githubClicks":1},"dates":{"published":null,"modified":"2026-06-27T12:18:15.000Z"}}