Our take
- Activity
- 19
- Community
- 28
- Issues
- 21
- Pull requests
- 20
+9 stars in the last 30 days
Project history
Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.
Stars
3,908Forks
479Change since Nov 14, 2026: 0 stars·0 forks
Overview
Riffusion-hobby applies stable diffusion to real-time music and audio generation. It connects text prompts and image conditioning with spectrogram processing, then converts generated spectrogram images into audio clips.
The repository is the core processing code, with a command-line interface, an interactive Streamlit playground, and a Flask inference API. It may fit developers exploring diffusion-based audio or building on the Riffusion workflow, rather than users seeking a currently maintained product.
Key Features
- Interpolates between prompts and uses image conditioning in a diffusion pipeline.
- Converts spectrogram images to audio clips and audio clips to spectrogram images.
- Provides a command-line interface for common processing tasks.
- Includes a Streamlit playground for interactive use.
- Serves model inference through a Flask API.
- Supports CPU, CUDA, and Apple Silicon MPS inference, with different performance and compatibility caveats.
- Allows a custom checkpoint directory or Hugging Face ID in diffusers format.
Use Cases
- Developers can experiment with generating audio from text prompts and interpolating between musical ideas.
- Researchers and hobbyists can explore how diffusion models connect spectrogram images with audio.
- Application builders can run the Flask inference server locally and connect it to the related Riffusion web app.
- Python users can convert existing audio and spectrogram files through the CLI or library code.
Getting Started
The README documents a Python 3.9 setup and dependency installation:
python -m pip install -r requirements.txt
For installation details, optional audio-format support, and setup of the model and backends, see the repository README.
Alternatives
- Magenta RT: Magenta RT generates music from text and audio prompts in real time, while riffusion-hobby uses stable diffusion with spectrogram-to-audio conversion and prompt interpolation.
| Project | Language | License | Stars | Status |
|---|---|---|---|---|
| riffusion-hobby | Python | MIT | 3.9k | Not checked yet |
| magenta-realtime | Python | Apache-2.0 | 1.8k | Active |
Considerations
The README states that the project is no longer actively maintained. CPU inference is supported but described as quite slow. Real-time generation requires a GPU capable of running stable diffusion in approximately five seconds, and MPS may fall back to CPU for some audio operations and is not deterministic. Audio formats beyond WAV require ffmpeg; the documented setup was tested with Python 3.9 and 3.10.
Found this useful?
Share it with someone who would like riffusion-hobby.