GPT-SoVITS: Few-Shot Voice Cloning and Text-to-Speech WebUI

Introduction

GPT-SoVITS-WebUI is a powerful, open-source project designed for few-shot voice conversion and text-to-speech (TTS). It stands out by enabling the training of high-quality TTS models with remarkably little data, often as little as one minute of vocal input. The project provides a user-friendly web interface, making advanced voice synthesis technologies accessible to a wider audience.

Key features include:

• Zero-shot TTS: Instant text-to-speech conversion using just a 5-second vocal sample.
• Few-shot TTS: Fine-tune the model with only 1 minute of training data for enhanced voice similarity and realism.
• Cross-lingual Support: Perform inference in languages different from the training dataset, currently supporting English, Japanese, Korean, Cantonese, and Chinese.
• Integrated WebUI Tools: Tools for voice accompaniment separation, automatic training set segmentation, ASR (Automatic Speech Recognition), and text labeling are included to assist in creating training datasets and GPT/SoVITS models.

Installation

GPT-SoVITS supports various environments including Windows, Linux, macOS, and Docker.

Prerequisites

Ensure you have Python 3.10-3.12 and a compatible PyTorch version installed. Conda is recommended for environment management.

Windows

For Windows users, an integrated package is available for direct download and execution. Alternatively, you can install via conda and pwsh script:

conda create -n GPTSoVits python=3.10
conda activate GPTSoVits
pwsh -F install.ps1 --Device <CU126|CU128|CPU> --Source <HF|HF-Mirror|ModelScope> [--DownloadUVR5]

Linux

Install on Linux using conda and the bash script:

conda create -n GPTSoVits python=3.10
conda activate GPTSoVits
bash install.sh --device <CU126|CU128|ROCM|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]

macOS

For macOS, installation also uses conda and a bash script. Note that GPU training on Macs may yield lower quality, so CPU is often used.

conda create -n GPTSoVits python=3.10
conda activate GPTSoVits
bash install.sh --device <MPS|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]

Docker

GPT-SoVITS can be run using Docker. Refer to the official repository for detailed Docker image selection, environment variables, and shared memory configuration.

docker compose run --service-ports <GPT-SoVITS-CU126-Lite|GPT-SoVITS-CU128-Lite|GPT-SoVITS-CU126|GPT-SoVITS-CU128>

Pretrained Models

After installation, download necessary pretrained models for GPT-SoVITS, G2PW, UVR5, and ASR from the links provided in the official documentation and place them in their respective directories.

Examples

Experience GPT-SoVITS in action through various demonstrations:

• Demo Video: Watch a comprehensive demonstration of its capabilities on Bilibili.
• Online Demo: Try the free online demo hosted on Hugging Face for a hands-on experience of high-speed inference.
• Few-shot Fine-tuning Demo: See unseen speakers few-shot fine-tuning in action directly on the GitHub repository.

To open the WebUI for finetuning or inference, use the provided scripts:

python webui.py <language(optional)>

Or for inference:

python GPT_SoVITS/inference_webui.py <language(optional)>

Why Use GPT-SoVITS?

GPT-SoVITS offers several compelling reasons for anyone interested in voice synthesis:

• Efficiency: Achieve high-quality voice cloning and TTS with minimal data, significantly reducing the effort and resources typically required.
• Versatility: Its cross-lingual capabilities and support for multiple languages make it suitable for a wide range of global applications.
• Comprehensive Toolset: The integrated WebUI tools simplify the entire process, from dataset preparation to model training and inference, making it beginner-friendly.
• High Performance: The project boasts impressive inference speeds, especially on modern GPUs, ensuring quick and responsive voice generation.
• Continuous Improvement: With ongoing development and multiple versions (V2, V3, V4, V2Pro) introducing new features and enhancements, GPT-SoVITS remains at the forefront of voice synthesis technology.

Links

• GitHub Repository: https://github.com/RVC-Boss/GPT-SoVITS
• English Documentation: https://rentry.co/GPT-SoVITS-guide#/
• Hugging Face Online Demo: https://lj1995-gpt-sovits-proplus.hf.space/
• Colab Training: https://colab.research.google.com/github/RVC-Boss/GPT-SoVITS/blob/main/Colab-WebUI.ipynb
• Demo Video (Bilibili): https://www.bilibili.com/video/BV12g4y1m7Uw