TRELLIS: Generate 3D Assets from Text or Images

Overview

TRELLIS is a 3D generation model and Python toolkit for turning text or images into detailed 3D assets. It addresses a practical limitation of 3D generation systems: outputs often need to fit different rendering and production workflows. TRELLIS uses a shared structured latent representation that can be decoded into meshes, 3D Gaussians, or radiance fields.

The project is aimed at researchers and developers exploring 3D generative models, as well as creators who need generated assets in more than one representation. It includes pretrained models, inference examples, a web demo, training code, and the TRELLIS-500K dataset. The repository recommends image-conditioned generation for better results, and specifically suggests creating an image from a text prompt first when detail and creativity matter.

Key Features

• Generates 3D assets from image or text prompts.
• Decodes a shared structured latent into meshes, 3D Gaussians, and radiance fields.
• Offers pretrained image and text models, from 342M to 2.0B parameters.
• Supports exporting generated outputs as GLB meshes or PLY Gaussian files.
• Provides asset variant generation and local editing capabilities.
• Includes training code, configuration files, and distributed training support.
• Provides TRELLIS-500K, a curated dataset of 500K 3D assets.

Use Cases

• 3D researchers can study or fine-tune generation models using the included training framework and configs.
• Developers building 3D content tools can generate assets in the representation their renderer or downstream workflow expects.
• Artists and prototypers can use image prompts to create initial 3D assets, then export meshes or Gaussian representations.
• Teams with collections of 3D objects can explore training or evaluation workflows using TRELLIS-500K as a reference for dataset preparation.

Getting Started

Clone the repository with its submodules and install the documented dependencies:

git clone --recurse-submodules https://github.com/microsoft/TRELLIS.git
cd TRELLIS
. ./setup.sh --new-env --basic --xformers --flash-attn --diffoctreerast --spconv --mipgaussian --kaolin --nvdiffrast

See the README for setup options, pretrained model loading, inference examples, and training instructions. The project page and Hugging Face demo provide additional ways to explore the project.

Considerations

• Local inference requires an NVIDIA GPU with at least 16 GB of memory. The documented setup is tested on Linux and CUDA 11.8 or 12.2; Windows support is not fully tested.
• Installation involves CUDA compilation and multiple specialized dependencies, so this is not a lightweight, platform-independent package.
• Text-conditioned models may be less creative and detailed than image-conditioned models because of data limitations. The README recommends an image-generation step followed by image-to-3D for text-driven workflows.
• The repository reports training and editing capabilities, but users should assess generated assets for their own quality and production requirements.
• Most code and models use the MIT License, but the README notes that some submodules have separate licenses.