VoxCPM: Generate and Clone Multilingual Speech

Overview

VoxCPM is an open-source text-to-speech system that generates speech from text using a tokenizer-free, diffusion autoregressive approach. The current VoxCPM2 release is designed for multilingual synthesis, natural-language voice design, and voice cloning with optional style control.

It is suited to developers and researchers building speech experiences who need control over voice and delivery, and who can run a substantial neural model. The repository includes Python and command-line entry points, with separate serving and edge-inference options documented for deployment.

Key Features

• Generates speech in 30 supported languages without requiring language tags.
• Creates a voice from a natural-language description, without reference audio.
• Clones a voice from a reference clip, with optional instructions for style, pace, or emotion.
• Supports audio continuation using reference audio and its transcript for closer voice reproduction.
• Produces 48 kHz audio output, according to the project documentation.
• Offers streaming generation and fine-tuning with SFT or LoRA.
• Provides Python and CLI workflows, plus integrations for serving and on-device inference.

Use Cases

• Speech product developers: add multilingual text-to-speech to an application when expressive, customizable voices matter.
• Content creators and studios: generate narration or localized speech and adjust voice characteristics through descriptions or style guidance.
• Accessibility teams: create spoken versions of written content for audiences across supported languages.
• Researchers and ML practitioners: experiment with tokenizer-free speech generation or adapt a model to a speaker or domain through fine-tuning.
• Infrastructure teams: evaluate self-hosted TTS serving when they can provide the GPU resources and operational support it requires.

Getting Started

Install the Python package:

pip install voxcpm

The README documents model loading, generation, cloning, CLI use, and deployment. See the VoxCPM repository and project documentation for setup details and model weights.

Alternatives

• MOSS-TTS: MOSS-TTS covers a broader range of speech and sound generation tasks, including dialogue, sound effects, and streaming TTS.
• Spark-TTS: Spark-TTS uses Qwen2.5 and focuses on efficient Chinese-English synthesis and zero-shot voice cloning.

Considerations

• The documented requirements include Python 3.10 to below 3.13, PyTorch 2.5.0 or newer, and CUDA 12.0 or newer. Hardware and memory needs depend on the inference setup; the README lists about 8 GB of VRAM for VoxCPM2.
• Voice design and controllable cloning can vary between runs, so results may require retries and evaluation for the intended use.
• Realistic voice cloning can be misused for impersonation or fraud. The project explicitly forbids those uses and recommends labeling AI-generated content.
• The Apache-2.0 license applies to the model weights and code. Production users should still test and assess safety for their own use case.