TRELLIS: Structured 3D Latents for Scalable and Versatile 3D Generation

Introduction

TRELLIS is the official repository for the paper "Structured 3D Latents for Scalable and Versatile 3D Generation," recognized as a CVPR'25 Spotlight. Developed by Microsoft, TRELLIS introduces a powerful 3D asset generation model that can create high-quality 3D assets from both text and image prompts. Its core innovation lies in a unified Structured LATent (SLAT) representation, enabling versatile decoding into various 3D formats, including Radiance Fields, 3D Gaussians, and meshes. This project offers large-scale pre-trained models and advanced capabilities for flexible 3D editing.

Installation

To get started with TRELLIS, follow these steps:

Prerequisites

System: Linux is currently tested.
Hardware: An NVIDIA GPU with at least 16GB of memory (A100, A6000 recommended).
Software: CUDA Toolkit (11.8 or 12.2), Conda for dependency management, Python 3.8 or higher.

Installation Steps

Clone the repository:

git clone --recurse-submodules https://github.com/microsoft/TRELLIS.git
cd TRELLIS

Install dependencies: This command creates a new conda environment named trellis and installs all necessary dependencies. Refer to . ./setup.sh --help for detailed options and flags.
```
. ./setup.sh --new-env --basic --xformers --flash-attn --diffoctreerast --spconv --mipgaussian --kaolin --nvdiffrast
```

Examples

TRELLIS provides straightforward ways to generate 3D assets. Here's a minimal Python example for image-to-3D generation:

import os
import imageio
from PIL import Image
from trellis.pipelines import TrellisImageTo3DPipeline
from trellis.utils import render_utils, postprocessing_utils

# Load a pipeline from a model folder or a Hugging Face model hub.
pipeline = TrellisImageTo3DPipeline.from_pretrained("microsoft/TRELLIS-image-large")
pipeline.cuda()

# Load an image
image = Image.open("assets/example_image/T.png")

# Run the pipeline
outputs = pipeline.run(image, seed=1)

# Render the outputs and save as videos
video = render_utils.render_video(outputs['gaussian'][0])['color']
imageio.mimsave("sample_gs.mp4", video, fps=30)

# Extract GLB files
glb = postprocessing_utils.to_glb(
    outputs['gaussian'][0],
    outputs['mesh'][0],
    simplify=0.95,
    texture_size=1024,
)
glb.export("sample.glb")

# Save Gaussians as PLY files
outputs['gaussian'][0].save_ply("sample.ply")

This example demonstrates loading a pre-trained model, taking an input image, generating 3D assets in various formats, and saving them as videos, GLB, and PLY files.

Additionally, TRELLIS offers a web demo based on Gradio. After installing demo-specific dependencies (. ./setup.sh --demo), you can run it with python app.py.

Why Use TRELLIS

TRELLIS stands out as a leading solution for 3D generation due to several compelling features:

High Quality: It consistently produces diverse 3D assets with exceptional shape and texture details.
Versatility: TRELLIS accepts both text and image prompts, generating a wide array of final 3D representations, including Radiance Fields, 3D Gaussians, and meshes, to suit diverse downstream applications.
Flexible Editing: The platform allows for easy modifications of generated 3D assets, such as creating variants of an object or performing local edits.
Scalable Models: It leverages large-scale pre-trained models, some with up to 2 billion parameters, trained on an extensive dataset of 500K diverse 3D objects.
Cutting-edge Research: Backed by a CVPR'25 Spotlight paper, TRELLIS represents the forefront of 3D generative AI research.

TRELLIS: Structured 3D Latents for Scalable and Versatile 3D Generation

Summary

Repository Info

Tags