vllm-cli: Manage and Serve LLMs with vLLM

Overview

vllm-cli is a Python terminal tool for configuring and launching Large Language Model servers with vLLM. It combines a menu-driven interface with CLI commands, helping users manage model selection, server settings, and active instances without assembling those workflows manually.

It is useful for developers and self-hosters already working with vLLM who want reusable configurations and visibility into their local serving setup. Model discovery includes HuggingFace and Ollama support. A multi-model proxy is also described as experimental.

Key Features

• Interactive terminal interface and direct CLI commands for automation.
• Discovers local models, including HuggingFace and Ollama models.
• Provides built-in and user-defined configuration profiles for server parameters.
• Supports shortcuts for launching saved model and profile combinations.
• Monitors active vLLM servers and displays system information such as GPU and memory details.
• Offers GPU selection and configuration validation.
• Includes an experimental proxy for accessing multiple models through one API endpoint.

Use Cases

• Developers who already have vLLM installed and want a guided terminal workflow to start and configure model servers.
• Self-hosters who regularly switch between models or hardware-specific profiles and want to save launch configurations.
• Operators who need a CLI for scripting server startup, alongside an interactive interface for manual use.
• Users with Ollama models who want to discover them as part of their vLLM CLI model-management workflow.
• Experimenters who want to test serving multiple models behind one endpoint, with the understanding that the proxy is experimental.

Getting Started

Install the CLI in an environment configured for vLLM, then launch it:

pip install vllm-cli
vllm-cli

For vLLM installation guidance, compatibility notes, and other install options, see the README. vllm-cli does not install vLLM or PyTorch by default.

Considerations

• vLLM and PyTorch compatibility matters because vLLM includes compiled CUDA kernels. The project warns that mismatched versions can cause errors.
• A CUDA-compatible GPU is recommended, and vLLM must be installed separately unless using the optional vllm extra.
• The multi-model proxy is experimental and under active development.
• The repository describes both GPU and system-information features, but the available hardware and vLLM setup determine what can be used in a given environment.