ComfyUI-TRELLIS2_Motion

A comprehensive ComfyUI node package that uses Microsoft's TRELLIS2 to transform video footage into 3D mesh reconstructions with advanced rendering effects.

Features

• Image to 3D Mesh: Convert single images to high-quality 3D meshes with PBR materials
• Video Processing: Batch process video frames through TRELLIS2
• Multiple Rendering Modes: PBR, wireframe, cel/toon shading, normal map, depth map
• Onion Skinning: Motion ghosting effect with configurable interval (every 5, 10, 30 frames)
• Camera Control: Orbit, dolly, pan trajectories with animation support
• Lighting Presets: Studio, outdoor, dramatic, flat lighting configurations
• Post-Processing: Bloom, depth of field, motion blur, outlines
• Progress Bars: Visual feedback on all long-running operations
• Auto Model Download: Automatically downloads TRELLIS2 model (~8GB) on first use

Nodes

Node	Description
Trellis2Predict	Convert single image to 3D mesh with PBR materials
VideoToMeshes	Batch process video frames through TRELLIS2
CameraTrajectory	Define camera movement (orbit, dolly, pan, static)
MultiAngleCamera	Generate multiple viewing angles for grid preview
MeshRenderer	Render mesh from camera viewpoint with effects
MeshSequenceRenderer	Render video sequence with consistent camera
OnionSkinning	Add motion ghosting effect to rendered frames
RenderingEffects	Apply bloom, DoF, cel shading, outlines
Trellis2VideoAngleShift	All-in-one convenience node

Installation

1. Clone the Repository

cd ComfyUI/custom_nodes
git clone https://github.com/yourusername/ComfyUI-TRELLIS2_Motion.git
cd ComfyUI-TRELLIS2_Motion

2. Install Dependencies

python install.py

3. Install TRELLIS2 (Required)

pip install git+https://github.com/microsoft/TRELLIS.git

4. Install nvdiffrast (Required for Rendering)

pip install git+https://github.com/NVlabs/nvdiffrast.git

5. Restart ComfyUI

The nodes will appear under the TRELLIS2 category.

Requirements

• Python: 3.10+
• CUDA: Required (tested on CUDA 11.8+)
• GPU: NVIDIA GPU with 24GB+ VRAM recommended (A100, RTX 4090)
• ComfyUI: Latest version

Workflows

Pre-built workflows are included in the workflows/ folder:

Workflow	Description
single_image_novel_view.json	Basic image to 3D with camera control
video_angle_shift.json	Full video processing pipeline
onion_skinning_demo.json	Motion ghosting showcase
effects_showcase.json	Cel shading and post-processing demo
multiview_preview.json	8-angle grid preview

Usage Examples

Single Image Novel View

LoadImage → Trellis2Predict → MeshRenderer → SaveImage
                                    ↑
                           CameraTrajectory

Video Angle Shift

VHS_LoadVideo → VideoToMeshes → MeshSequenceRenderer → VHS_VideoCombine
                                        ↑
                               CameraTrajectory

With Onion Skinning

VHS_LoadVideo → VideoToMeshes → MeshSequenceRenderer → OnionSkinning → VHS_VideoCombine

Onion Skinning Parameters

Parameter	Description	Default
ghost_count	Number of ghost frames	3
frame_interval	Frames between ghosts	5
ghost_opacity	Base opacity of ghosts	0.3
opacity_falloff	linear, exponential, constant	linear
ghost_color_mode	original, tinted, silhouette	tinted
past_tint	Color for past frames	Blue (#0066FF)
future_tint	Color for future frames	Orange (#FF6600)

Rendering Modes

• PBR: Full physically-based rendering with materials
• Wireframe: Edge visualization
• Cel/Toon: Quantized shading with outlines
• Normal: RGB-encoded surface normals
• Depth: Grayscale depth visualization

Lighting Presets

• Studio: Professional 3-point lighting
• Outdoor: HDR sky with sun
• Dramatic: High contrast single light
• Flat: Even lighting for cel shading

Performance

Resolution	TRELLIS2 Time	Render Time	Total/Frame
512x512	~3s	~0.1s	~3.1s
1024x1024	~10s	~0.2s	~10.2s

Tested on NVIDIA A100 80GB

Comparison with SHARP

Aspect	TRELLIS2	SHARP
Output	PBR Mesh + Materials	Gaussian Splats
Processing Time	3-60s per frame	<1s per frame
Materials	Full PBR	SH coefficients
Rendering	nvdiffrast	gsplat
Quality	Higher fidelity meshes	Real-time splatting

Troubleshooting

"CUDA out of memory"

• Reduce resolution from 1024 to 512
• Increase frame_skip to process fewer frames
• Use a GPU with more VRAM

"nvdiffrast not found"

pip install git+https://github.com/NVlabs/nvdiffrast.git

"TRELLIS not found"

pip install git+https://github.com/microsoft/TRELLIS.git

Credits

• Microsoft TRELLIS - Core 3D reconstruction model
• nvdiffrast - CUDA mesh rendering
• ComfyUI - Node-based UI framework

License

MIT License - See LICENSE file for details.