{"name":"vLLM CLI: A Powerful Command-Line Interface for Serving LLMs with vLLM","description":"vLLM CLI is an intuitive command-line interface tool designed to simplify serving Large Language Models using vLLM. It offers both interactive and direct CLI modes, enabling efficient model management, real-time server monitoring, and advanced configuration. This tool streamlines the deployment and management of LLMs, making it accessible for various use cases.","github":"https://github.com/Chen-zexi/vllm-cli","url":"https://osrepos.com/repo/chen-zexi-vllm-cli","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/chen-zexi-vllm-cli","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/chen-zexi-vllm-cli.md","json":"https://osrepos.com/repo/chen-zexi-vllm-cli.json","topics":["llm","llm-inference","llm-tools","vllm","Python","AI","Machine Learning","CLI"],"keywords":["llm","llm-inference","llm-tools","vllm","Python","AI","Machine Learning","CLI"],"stars":null,"summary":"vLLM CLI is an intuitive command-line interface tool designed to simplify serving Large Language Models using vLLM. It offers both interactive and direct CLI modes, enabling efficient model management, real-time server monitoring, and advanced configuration. This tool streamlines the deployment and management of LLMs, making it accessible for various use cases.","content":"## Introduction\n\nvLLM CLI is a powerful and intuitive command-line interface tool designed to simplify the process of serving Large Language Models (LLMs) using the vLLM library. It provides a comprehensive suite of features for managing, configuring, and monitoring your LLM inference servers, catering to both interactive use and automated scripting.\n\nKey features include a rich interactive terminal mode, direct CLI commands for automation, automatic discovery of local models with HuggingFace and Ollama support, and flexible configuration profiles. Recent updates have introduced an experimental Multi-Model Proxy server for unified API access to multiple LLMs, hardware-optimized profiles for GPT-OSS models on NVIDIA GPUs, and a convenient shortcuts system for quick launches.\n\n## Installation\n\n**Important: vLLM Installation Notes**\nvLLM contains pre-compiled CUDA kernels that must precisely match your PyTorch version. Installing mismatched versions will lead to errors. vLLM CLI does not install vLLM or PyTorch by default.\n\n**Option 1: Install vLLM separately and then install vLLM CLI (Recommended)**\n\nbash\n# Install vLLM -- Skip this step if you have vLLM installed in your environment\nuv venv --python 3.12 --seed\nsource .venv/bin/activate\nuv pip install vllm --torch-backend=auto\n# Or specify a backend: uv pip install vllm --torch-backend=cu128\n\n# Install vLLM CLI\nuv pip install --upgrade vllm-cli\nuv run vllm-cli\n\n# If you are using conda:\n# Activate the environment you have vLLM installed in\npip install vllm-cli\nvllm-cli\n\n\n**Option 2: Install vLLM CLI + vLLM**\n\nbash\npip install vllm-cli[vllm]\nvllm-cli\n\n\n**Prerequisites:**\n*   Python 3.9+\n*   CUDA-compatible GPU (recommended)\n*   vLLM package installed\n\n## Examples\n\n**Interactive Mode**\nLaunch the menu-driven interface for easy navigation and management.\n\nbash\nvllm-cli\n\n\n**Serve a Model**\nQuickly serve a specific model using a direct command.\n\nbash\nvllm-cli serve --model openai/gpt-oss-20b\n\n\n**Use a Shortcut**\nLaunch pre-configured model and profile combinations with a simple shortcut.\n\nbash\nvllm-cli serve --shortcut my-model\n\n\n## Why Use vLLM CLI?\n\nvLLM CLI streamlines the often complex process of deploying and managing LLMs with vLLM. It offers a user-friendly interactive terminal for easy configuration and monitoring, alongside powerful command-line options for automation. With features like automatic model discovery, real-time server monitoring, and optimized configuration profiles, it empowers users to efficiently serve various LLMs, including those from HuggingFace and Ollama, while ensuring optimal performance and resource utilization.\n\n## Links\n\n*   **GitHub Repository**: [Chen-zexi/vllm-cli](https://github.com/Chen-zexi/vllm-cli){:target=\"_blank\"}\n*   **Usage Guide**: [Learn how to use vLLM CLI](https://github.com/Chen-zexi/vllm-cli/blob/main/docs/usage-guide.md){:target=\"_blank\"}\n*   **Multi-Model Proxy Guide**: [Serve multiple models simultaneously](https://github.com/Chen-zexi/vllm-cli/blob/main/docs/multi-model-proxy.md){:target=\"_blank\"}\n*   **Profiles Guide**: [Detailed information on built-in profiles](https://github.com/Chen-zexi/vllm-cli/blob/main/docs/profiles.md){:target=\"_blank\"}\n*   **Troubleshooting**: [Common issues and solutions](https://github.com/Chen-zexi/vllm-cli/blob/main/docs/troubleshooting.md){:target=\"_blank\"}","metrics":{"detailViews":9,"githubClicks":13},"dates":{"published":null,"modified":"2026-01-20T20:01:16.000Z"}}