verl-project

verl-omni: Train Diffusion and Omni Models with verl

VeRL-Omni is a multimodal reinforcement-learning framework built on verl for post-training diffusion, unified vision-language-generation, and omni-modality models. It supports distributed training backends and vLLM-Omni rollouts.

Stars
1.5k
Forks
309
Stars, last 30 days
+238
Last commit
today
Contributors
58

Our take

Activeregular commits and releases

Health score

65/100

How it is scored
Activity
71
Community
82
Issues
57
Pull requests
50

+238 stars in the last 30 days

  • Merges more pull requests than 85% of the projects we track
  • Slower to merge pull requests than 83% of the projects we track

VeRL-Omni is an actively maintained, specialized option for teams building multimodal generative-model RL pipelines, rather than a general-purpose training framework.

Good fit if

  • You are post-training diffusion, unified multimodal, or omni-modality models and need the algorithms and recipes documented here.
  • Your workflow benefits from vLLM-Omni rollouts, asynchronous rewards, or the supported VeOmni and FSDP2 backends.
  • You can work with its Python-based stack and Apache-2.0 licensing, and value a project with regular releases and contributions from a broad contributor base.

Look elsewhere if

  • You need a general-purpose framework for text-only LLM RL rather than the multimodal model families this project targets.
  • Your models or algorithms are outside the project’s documented support and you need a ready-made recipe for them.
  • You need a mature project with a long adoption history; this repository is relatively young, despite its recent development activity.
All health signals
Last commit2026-11-10 (today)
Commits, last 90 days100+
Releases, last 12 months6 (latest v0.3.0, 2026-10-22)
Contributors58 (top contributor: 16% of commits)
Issues closed, last 90 days8+ (typically closed in 5 days)
Pull requests merged, last 90 days78 (typically merged in 3 days)
Project age7 months

Checked on 2026-11-11 with the GitHub API.

Project history

Daily GitHub snapshots since Oct 7, 2026. History covers stars and forks; health signals are not tracked historically.

Selected snapshot

Stars

1,521

Forks

321

Change since Nov 15, 2026: +6 stars·+1 forks

Overview

VeRL-Omni is a multimodal reinforcement-learning training framework built on verl from the verl-project. It addresses the distinct rollout, input/output, and compute demands of post-training diffusion and omni-modality generative models, including image, video, audio, and combined text-and-media workflows. For generation rollouts, it uses vLLM-Omni.

The project is aimed at teams training or adapting these models with reinforcement learning and related post-training algorithms. It offers selectable VeOmni and FSDP2 training backends, and supports Ascend A2, A3, and A5 NPUs as well as its documented training workflows.

Key Features

  • Training workflows for diffusion generators, unified multimodal understanding-and-generation models, and omni-modality models.
  • Algorithms include FlowGRPO, Flow-DPPO, DiffusionNFT, DPO, and GSPO, with model-specific support documented in the repository.
  • vLLM-Omni rollout support, with routing, batching, and embedding-cache optimizations.
  • Synchronous and asynchronous reward computation, including support for multiple reward services and HTTP scorers.
  • Selectable VeOmni and FSDP2 training backends, with combinations of USP, tensor, and data parallelism.
  • Rollout correction and deterministic training options for diffusion workflows.
  • Ascend NPU installation guidance and FlowGRPO quickstart support.

Use Cases

  • Research teams can optimize image-generation behavior, such as text rendering, with FlowGRPO and model-specific recipes.
  • Video-generation teams can apply reinforcement-learning training to supported models such as Wan2.2 and LTX2.3.
  • Multimodal post-training teams can run reward-based or preference-based workflows for models such as Qwen3-Omni.
  • Teams with Ascend NPUs can follow the documented setup and quickstart for supported training workflows.

What you need

Detected in the repository

  • Python >=3.11 (from pyproject.toml)
  • A test suite and automated checks on GitHub Actions

License in plain words

Apache-2.0permissive

  • Commercial use: yes
  • Modify and redistribute: yes
  • You must keep: the license, the NOTICE file and a note of your changes
  • Share your changes: no
  • Includes an explicit patent grant from the contributors:

A summary, not legal advice: the LICENSE file is what applies.

Getting Started

Start with the installation guide, then follow the FlowGRPO quickstart. The README links to additional model and algorithm recipes.

Alternatives

  • RL4LMs: RL4LMs focuses on on-policy preference fine-tuning for text-generation models, while verl-omni targets multimodal post-training, including diffusion models.
ProjectLanguageLicenseStarsStatus
verl-omniPythonApache-2.01.5kActive
RL4LMsPythonApache-2.02.4kInactive

Considerations

This is a young project whose feature set and recipes are still evolving. Its recent commits and releases indicate active development, and contributions are spread across many contributors rather than concentrated in one person. Issue and pull-request handling has been active, but teams should still validate that their particular model, algorithm, backend, and hardware combination is covered by a maintained recipe. The project requires Python 3.11 or newer; the repository does not specify a minimum GPU requirement in the provided information, while Ascend NPU support is documented for A2, A3, and A5.

Found this useful?

Share it with someone who would like verl-omni.

Comparisons