xiaozhi-esp32-server: Run a Backend for ESP32 Voice Devices

Overview

xiaozhi-esp32-server is the backend for the xiaozhi-esp32 open hardware project. It connects ESP32 devices to voice and AI services, and provides interfaces for device communication, management, and control.

It is intended for people who already have compatible ESP32 hardware and want to run their own backend rather than rely on another hosted service. Deployment options range from a minimal server setup to a fuller installation with database-backed management features.

Key Features

• Communicates with devices over MQTT+UDP and WebSocket, with HTTP endpoints also listed in the project documentation.
• Supports configurable speech recognition, language models, text-to-speech, and voice activity detection providers.
• Offers voiceprint recognition, visual-model interactions, and short-term or external memory options.
• Connects tools and device actions through MCP, client IoT protocols, custom functions, and MQTT command delivery.
• Includes a web console for user, device, and system management, with multiple language options.
• Provides minimal and full-module deployment paths, including Docker and local source deployment.

Use Cases

• ESP32 device owners can deploy a private backend when they want to configure the service and integrations themselves.
• Developers can prototype voice assistants that combine an ESP32 device with speech, LLM, and TTS services.
• Teams building device-control workflows can connect assistant tool calls to MQTT commands and supported MCP integrations.
• Operators who need multiple users, agents, or console-based administration can choose the full-module deployment rather than the minimal setup.

Project Facts

• Language: JavaScript
• License: MIT
• Stars: 10.7k
• Forks: 3.7k
• Topics: dify, esp32, mcp-server, xiaozhi, xiaozhi-ai, xiaozhi-esp32, xiaozhi-server
• Archived: No

Getting Started

Choose a deployment path in the deployment documentation. The README also links to the full-module deployment guide, configuration guidance, and FAQs. This service is designed to be used with compatible ESP32 hardware.

Alternatives

• ElatoAI: ElatoAI connects ESP32 devices to real-time voice AI using secure WebSockets and edge functions, rather than providing a self-hosted Xiaozhi-compatible backend.

Considerations

• The project warns that it is not complete and has not passed a cybersecurity assessment, and advises against production use. Public deployments need appropriate security protections.
• The backend depends on compatible ESP32 hardware and requires configuration of the communication protocol and selected AI or speech services.
• Some integrations use third-party APIs. The project disclaims responsibility for those providers' service quality, fees, and account security. Review provider terms and privacy policies before use.
• Resource requirements depend on the deployment and selected components. The README gives different recommendations for local FunASR and API-based configurations.