{"name":"Magenta RT: Live Music Generation on Your Local Device","description":"Magenta RealTime (Magenta RT) is an open-source Python library for live music audio generation on local devices. It allows users to create music using both text and audio prompts, serving as a powerful tool for real-time creative audio exploration. This library is the on-device companion to Google's MusicFX DJ Mode and the Lyria RealTime API.","github":"https://github.com/magenta/magenta-realtime","url":"https://osrepos.com/repo/magenta-magenta-realtime","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/magenta-magenta-realtime","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/magenta-magenta-realtime.md","json":"https://osrepos.com/repo/magenta-magenta-realtime.json","topics":["Python","Music Generation","AI","Real-time","Audio","Machine Learning","Open Source","Deep Learning"],"keywords":["Python","Music Generation","AI","Real-time","Audio","Machine Learning","Open Source","Deep Learning"],"stars":null,"summary":"Magenta RealTime (Magenta RT) is an open-source Python library for live music audio generation on local devices. It allows users to create music using both text and audio prompts, serving as a powerful tool for real-time creative audio exploration. This library is the on-device companion to Google's MusicFX DJ Mode and the Lyria RealTime API.","content":"## Introduction\nMagenta RealTime (Magenta RT) is an open-source Python library designed for live music audio generation directly on your local device. It serves as the on-device companion to Google's MusicFX DJ Mode and the Lyria RealTime API, enabling users to create music through both text and audio prompts. Magenta RT generates audio in short chunks (2 seconds) given a finite amount of past context (10 seconds), utilizing crossfading to mitigate boundary artifacts between chunks.\n\n## Installation\nGetting started with Magenta RT is straightforward, offering options for cloud-based demos or local setup.\n\nThe fastest way to experience Magenta RT is through its official [Colab Demo](https://colab.research.google.com/github/magenta/magenta-realtime/blob/main/notebooks/Magenta_RT_Demo.ipynb), which runs in real-time on freely available TPUs. Additionally, there are Colab demos supporting [live audio input](https://colab.research.google.com/github/magenta/magenta-realtime/blob/main/notebooks/Magenta_RT_Audio_Injection.ipynb) and [customization via finetuning](https://colab.research.google.com/github/magenta/magenta-realtime/blob/main/notebooks/Magenta_RT_Finetune.ipynb).\n\nFor local execution, you can choose between using Docker or performing a native installation.\n\n### Running Locally via Docker\nThis method requires a powerful GPU with 40GB memory, Linux, and [Docker](https://www.docker.com/get-started/).\n\nsh\nmkdir -p ~/.cache/magenta_rt\ndocker run -it \\\n  --gpus device=0 \\\n  -v ~/.cache/magenta_rt:/magenta-realtime/cache \\\n  -p 8000:8000 \\\n  us-docker.pkg.dev/brain-magenta/magenta-rt/magenta-rt:gpu\n\nAfter running the command, open the web demo at [https://localhost:8000](https://localhost:8000).\n\n### Local Installation\nIf you prefer to run Magenta RT natively rather than using Docker, follow these instructions.\n\n**Step 1: Install Python 3.12**\nsh\nsudo apt update\nsudo apt install software-properties-common -y\nsudo add-apt-repository ppa:deadsnakes/ppa\nsudo apt install python3.12 python3.12-venv python3.12-dev -y\n\n\n**Step 2: Install Magenta RT for GPU**\nsh\n# Clone Magenta RT\ngit clone https://github.com/magenta/magenta-realtime.git\ncd magenta-realtime\n# Create a virtual environment\npython3.12 -m venv .venv\nsource .venv/bin/activate\n# Patch and install t5x\ngit clone https://github.com/google-research/t5x.git && \\\n  pushd t5x && \\\n  git checkout 7781d16 && \\\n  patch setup.py < ../patch/t5x_setup.py.patch && \\\n  patch t5x/partitioning.py < ../patch/t5x_partitioning.py.patch && \\\n  pip install .[gpu] && \\\n  popd\n# Install Magenta RT\npip install -e .[gpu] && pip install tf2jax==0.3.8\n# Patch seqIO to remove tensorflow-text dependency\npatch .venv/lib/python3.12/site-packages/seqio/vocabularies.py < patch/seqio_vocabularies.py.patch\n\n\n**Step 2 (alternative): Install Magenta RT for TPU**\nsh\n# Create a virtual environment\npython3.12 -m venv .venv\nsource .venv/bin/activate\n# Install Magenta RT\ngit clone https://github.com/magenta/magenta-realtime.git\npip install -e magenta-realtime/[tpu] && pip install tf2jax==0.3.8 huggingface_hub\n\n\n**Step 3: Generate!**\nsh\npython -m magenta_rt.generate \\\n  --prompt=\"blissful ambient synth\" \\\n  --output=\"./output.mp3\"\n\n\n## Examples\nMagenta RT offers powerful capabilities for music generation and style blending.\n\n### Generating audio with Magenta RT\npython\nfrom magenta_rt import audio, system\nfrom IPython.display import display, Audio\n\nnum_seconds = 10\nmrt = system.MagentaRT()\nstyle = system.embed_style('funk')\n\nchunks = []\nstate = None\nfor i in range(round(num_seconds / mrt.config.chunk_length)):\n  state, chunk = mrt.generate_chunk(state=state, style=style)\n  chunks.append(chunk)\ngenerated = audio.concatenate(chunks)\ndisplay(Audio(generated.samples.swapaxes(0, 1), rate=mrt.sample_rate))\n\n\n### Blending text and audio styles with MusicCoCa\npython\nfrom magenta_rt import audio, musiccoca\nimport numpy as np\n\nstyle_model = musiccoca.MusicCoCa()\nmy_audio = audio.Waveform.from_file('myjam.mp3')\nweighted_styles = [\n  (2.0, my_audio),\n  (1.0, 'heavy metal'),\n]\nweights = np.array([w for w, _ in weighted_styles])\nstyles = style_model.embed([s for _, s in weighted_styles])\nweights_norm = weights / weights.sum()\nblended = (weights_norm[:, np.newaxis] * styles).mean(axis=0)\n\n\n### Tokenizing audio with SpectroStream\npython\nfrom magenta_rt import audio, spectrostream\n\ncodec = spectrostream.SpectroStream()\nmy_audio = audio.Waveform.from_file('jam.mp3')\nmy_tokens = codec.encode(my_audio)\nmy_audio_reconstruction = codec.decode(my_tokens)\n\n\n## Why Use Magenta RT?\nMagenta RT provides a unique platform for real-time, on-device music generation. Its ability to respond to both text and audio prompts makes it incredibly flexible for creators. The integration with powerful models like MusicCoCa for style blending and SpectroStream for audio tokenization offers advanced capabilities for sophisticated audio manipulation. Being open-source, it allows developers and musicians to experiment, customize, and integrate live music generation into their projects.\n\n## Links\n*   **GitHub Repository:** [https://github.com/magenta/magenta-realtime](https://github.com/magenta/magenta-realtime)\n*   **Official Blog Post:** [https://g.co/magenta/rt](https://g.co/magenta/rt)\n*   **Research Paper:** [https://arxiv.org/abs/2508.04651](https://arxiv.org/abs/2508.04651)\n*   **Model Card:** [https://github.com/magenta/magenta-realtime/blob/main/MODEL.md](https://github.com/magenta/magenta-realtime/blob/main/MODEL.md)\n*   **Colab Demo (Main):** [https://colab.research.google.com/github/magenta/magenta-realtime/blob/main/notebooks/Magenta_RT_Demo.ipynb](https://colab.research.google.com/github/magenta/magenta-realtime/blob/main/notebooks/Magenta_RT_Demo.ipynb)\n*   **Colab Demo (Audio Injection):** [https://colab.research.google.com/github/magenta/magenta-realtime/blob/main/notebooks/Magenta_RT_Audio_Injection.ipynb](https://colab.research.google.com/github/magenta/magenta-realtime/blob/main/notebooks/Magenta_RT_Audio_Injection.ipynb)\n*   **Colab Demo (Finetuning):** [https://colab.research.google.com/github/magenta/magenta-realtime/blob/main/notebooks/Magenta_RT_Finetune.ipynb](https://colab.research.google.com/github/magenta/magenta-realtime/blob/main/notebooks/Magenta_RT_Finetune.ipynb)\n*   **YouTube Video (Text Prompting):** [https://www.youtube.com/watch?v=Ae1Kz2zmh9M](https://www.youtube.com/watch?v=Ae1Kz2zmh9M)\n*   **YouTube Video (Audio Prompting):** [https://www.youtube.com/watch?v=vHIf2UKXmp4](https://www.youtube.com/watch?v=vHIf2UKXmp4)\n*   **YouTube Video (Colab Walkthrough):** [https://www.youtube.com/watch?v=SVTuEdeepVs](https://www.youtube.com/watch?v=SVTuEdeepVs)","metrics":{"detailViews":3,"githubClicks":4},"dates":{"published":null,"modified":"2026-03-06T16:55:26.000Z"}}