Open-LLM-VTuber: Your Offline, Cross-Platform AI Companion with Live2D

Summary

Open-LLM-VTuber is an innovative open-source project that brings a customizable, voice-interactive AI companion to your desktop. It supports real-time voice interaction, visual perception, and dynamic Live2D avatars, all running completely offline across Windows, macOS, and Linux. This project allows users to create a personalized AI companion with extensive model support and advanced interaction features, ensuring privacy and flexibility.

Use at your own risk

OSRepos shares public repositories for knowledge and discovery only. Any installation, execution, configuration, or use of code from these repositories is the user's own responsibility. Always review the repository, source code, dependencies, licenses, and security implications before running or installing anything. OSRepos is not responsible for issues, damages, or losses resulting from third-party repositories.

Introduction

Open-LLM-VTuber is a unique voice-interactive AI companion project, designed to run entirely offline on your personal computer. It combines real-time voice interaction, visual perception, and expressive Live2D avatars to create a highly personalized virtual companion. Whether you envision a virtual girlfriend, boyfriend, or a pet, this project offers extensive customization options for character appearance and personality. It supports Windows, macOS, and Linux, providing both web-based and desktop client experiences, including a transparent desktop pet mode that keeps your AI companion always by your side. The project's initial goal was to replicate the functionality of closed-source AI VTubers like Neuro-Sama using open-source, offline-capable solutions.

Installation

To get started with Open-LLM-VTuber, please refer to the official Quick Start Guide for detailed installation instructions. The project is actively developed, and it is recommended to follow the latest deployment guidelines for the best experience.

Examples

Open-LLM-VTuber offers a rich set of interactive features to enhance your AI companion experience. These include:

Visual Perception: The AI can "see" you and your screen through camera, screen recording, and screenshot support.
Voice Interruption: Engage in natural conversations with the ability to interrupt the AI without needing headphones.
Touch Feedback: Interact directly with your AI companion by clicking or dragging its Live2D avatar.
Live2D Expressions: Configure emotion mapping to allow the backend to control the model's expressions.
Pet Mode: Enjoy a transparent background, global top-level window, and mouse-through functionality, letting you place your AI companion anywhere on your screen.
AI Active Speech: The AI can initiate conversations.
AI Internal Monologue: Observe the AI's expressions, thoughts, and actions, which are visible but not spoken aloud.
Persistent Chat History: Seamlessly switch between previous conversations without losing any interaction moments.
TTS Translation Support: For example, chat in Chinese while the AI responds with a Japanese voice.

Why Use Open-LLM-VTuber?

This project stands out for its commitment to privacy, flexibility, and extensive customization. Running entirely offline, your conversations remain on your device. It boasts broad cross-platform support and a powerful client with advanced interaction features. Users can customize everything from Live2D models and character prompts to voice cloning. The modular design also allows for easy integration of various LLM, ASR, and TTS models, making it highly extensible. As one user enthusiastically put it, "Thanks to the developer for open-sourcing the girlfriend for everyone to use."