{"name":"Harbor: Effortless Local LLM Stack Management with CLI and App","description":"Harbor is a comprehensive containerized toolkit designed to simplify the setup and management of local LLM environments. It allows users to effortlessly run LLM backends, APIs, frontends, and various related services with simple commands or via a companion application. This project streamlines experimentation and development with large language models on your local machine.","github":"https://github.com/av/harbor","url":"https://osrepos.com/repo/av-harbor","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/av-harbor","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/av-harbor.md","json":"https://osrepos.com/repo/av-harbor.json","topics":["ai","llm","local","docker","cli","python","self-hosted","llm-ops"],"keywords":["ai","llm","local","docker","cli","python","self-hosted","llm-ops"],"stars":null,"summary":"Harbor is a comprehensive containerized toolkit designed to simplify the setup and management of local LLM environments. It allows users to effortlessly run LLM backends, APIs, frontends, and various related services with simple commands or via a companion application. This project streamlines experimentation and development with large language models on your local machine.","content":"## Introduction\nHarbor is an innovative, containerized LLM toolkit that empowers users to effortlessly set up and manage their local Large Language Model (LLM) environments. It provides a powerful command-line interface (CLI) and a companion desktop application, enabling the seamless deployment of LLM backends, frontends, APIs, and a wide array of related services. Designed for both beginners and experienced developers, Harbor simplifies the complex process of running cutting-edge AI models and tools locally.\n\n## Installation\nGetting started with Harbor is straightforward. The project provides comprehensive guides for installing both the CLI and the companion application. Once installed, you can launch a default stack, including Open WebUI and Ollama, with a single command:\n\nbash\nharbor up\n\n\nFor more detailed installation instructions, refer to the [official documentation](https://github.com/av/harbor/wiki/1.0.-Installing-Harbor).\n\n## Examples\nHarbor's versatility shines through its wide range of supported services and functionalities. Here are a few examples of what you can achieve:\n\n### Running Local LLMs and Inference Engines\nEasily deploy popular LLM backends and advanced inference engines, all pre-connected to frontends like Open WebUI:\n\nbash\nharbor up ollama\nharbor up llamacpp\nharbor up vllm\nharbor up vllm llamacpp tgi litellm tabbyapi aphrodite sglang ktransformers mistralrs airllm\n\n\n### Web RAG and Deep Research\nIntegrate powerful search capabilities with your LLMs. Harbor includes SearXNG, which is pre-connected to various services for Web RAG:\n\nbash\nharbor up searxng\nharbor up searxng chatui\n\n\n### Image Generation\nUtilize image generation models like FLUX with ComfyUI, integrated directly into Open WebUI:\n\nbash\nharbor up comfyui\n\n\n### Access from Anywhere\nAccess your Harbor services from your phone via QR code or expose them to the internet with a built-in tunneling service (use with caution):\n\nbash\nharbor qr\nharbor tunnel\n\n\n### Eject to Docker Compose\nWhen you're ready to transition to a custom setup, Harbor can generate a `docker-compose` file replicating your current configuration:\n\nbash\nharbor eject searxng llamacpp > docker-compose.harbor.yml\n\n\n## Why Use Harbor\nHarbor offers significant advantages for anyone working with local LLM environments:\n\n*   **Simplicity and Convenience**: Run complex LLM stacks with minimal configuration, often with a single command or click.\n*   **Centralized Workflow**: All services, configurations, logs, and data files are managed from a single, consistent interface, reducing setup complexity and ensuring predictability.\n*   **Experimentation Hub**: It serves as an excellent starting point for experimenting with a vast array of LLMs and related services, providing a robust and ready-to-use environment.\n*   **Extensive Service Catalog**: Access a wide selection of UIs, backends, and satellite services, from chat interfaces to RAG tools and workflow automation.\n\n## Links\n*   **GitHub Repository**: [av/harbor](https://github.com/av/harbor)\n*   **Official Documentation**: [Harbor Wiki](https://github.com/av/harbor/wiki)\n*   **Discord Community**: [Join Discord](https://discord.gg/8nDRphrhSF)","metrics":{"detailViews":3,"githubClicks":5},"dates":{"published":null,"modified":"2025-10-12T04:15:16.000Z"}}