nodes/paulh4x-ComfyUI_PHRenderFormerWrapper
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

added gitignore and new advanced workflow example containing animated light

Browse Source
This commit is contained in:
paulh4x
2025-07-14 14:56:30 +02:00
parent 63e98f1abf
commit 3b5a4920a4
8 changed files with 116 additions and 118 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+4
View File
@@ -0,0 +1,4 @@
# Ignore the assets folder
/assets/
/renderformer/
PLAN.md
README.md
+45 -36
View File
@@ -70,9 +70,9 @@ For video, the workflow is similar. You would use a `RenderFormer Camera Target`
---
### 🚀 Advanced Usage: Building Complex Scenes & Animations
### 🚀 Advanced Usage: Building Complex Scenes & Multi-Light Animations
For more complex projects, you can combine multiple meshes and create camera animations. The `ph_renderformer_advanced_01.json` workflow demonstrates this by building a scene with several objects and animating the camera.
For more complex projects, you can combine multiple meshes and create sophisticated camera and lighting animations. The `ph_renderformer_advanced_01.json` workflow demonstrates this by building a scene with several object groups and animating multiple lights simultaneously.
```mermaid
graph TD
@@ -82,28 +82,34 @@ graph TD
subgraph "Step 2: Define Scene Elements"
subgraph "Compose Meshes"
B1["Mesh 1 (Main Object)"] --> C["Mesh Combine"]
B2["Mesh 2 (Fox)"] --> C
B3["Mesh 3 (Spheres)"] --> C
B4["Mesh 4 (Background Walls)"] --> C
B1["Mesh Group 1 (e.g., Characters)"] --> C1["Mesh Combine 1"]
B2["Mesh Group 2 (e.g., Environment)"] --> C2["Mesh Combine 2"]
C1 --> C3["Final Mesh Combine"]
C2 --> C3
end
subgraph "Animate Camera"
D1["RenderFormer Camera (Start)"] --> D2["RenderFormer Camera Target (End)"]
end
E["RenderFormer Lighting"]
subgraph "Animate Multiple Lights"
L1_Start["Light 1 (Start)"] --> L1_Target["Light 1 Target"]
L2_Start["Light 2 (Start)"] --> L2_Target["Light 2 Target"]
L1_Target --> LC["Lighting Combine"]
L2_Target --> LC
end
end
subgraph "Step 3: Build Scene Sequence"
C --> F["RenderFormer Scene Builder"]
D2 -- CAMERA_SEQUENCE --> F
E --> F
A --> F["RenderFormer Scene Builder"]
C3 -- MESH --> F
D1 -- required CAMERA --> F
D2 -- optional CAMERA_SEQUENCE --> F
LC -- LIGHTING & LIGHTING_SEQUENCE --> F
end
subgraph "Step 4: Render (Sample)"
A --> G["RenderFormer Sampler"]
F -- SCENE_SEQUENCE --> G
F -- SCENE_SEQUENCE --> G["RenderFormer Sampler"]
end
subgraph "Step 5: Create & Save Video"
@@ -114,28 +120,33 @@ graph TD
style A fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style B1 fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style B2 fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style B3 fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style B4 fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style C fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style C1 fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style C2 fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style C3 fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style D1 fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style D2 fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style E fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style L1_Start fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style L2_Start fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style L1_Target fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style L2_Target fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style LC fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style F fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style G fill:#FDC501,stroke:#111417,stroke-width:2px,color:#111417
style H fill:#a0a0a0,stroke:#111417,stroke-width:2px,color:#111417
style I fill:#a0a0a0,stroke:#111417,stroke-width:2px,color:#111417
```
1. **Compose Your Scene**: Instead of using one large model, you can load multiple `RenderFormer Mesh Loader` nodes for different parts of your scene (e.g., characters, props, background elements).
2. **Combine Meshes**: Use one or more `RenderFormer Mesh Combine` nodes to merge all your individual meshes into a single `MESH` output. This creates a clean, organized workflow and makes it easy to manage complex scenes.
3. **Animate the Camera**:
* Define the starting view with a `RenderFormer Camera` node.
* Connect its `CAMERA` output to the `start_camera` input of a `RenderFormer Camera Target` node.
* In the `Camera Target` node, define the end position, look-at point, and FOV for your animation, along with the total number of frames. This generates a `CAMERA_SEQUENCE`.
4. **Build the Scene Sequence**: Connect your combined `MESH`, the `CAMERA_SEQUENCE`, and your `LIGHTING` to the `RenderFormer Scene Builder`. It will automatically create a sequence of scenes, one for each frame of the animation.
5. **Render and Save**:
* The `RenderFormer Sampler` will receive the `SCENE_SEQUENCE` and render all frames, outputting them as an `IMAGES` batch.
* Connect this batch to a `Create Video` node (or another video-handling node) and then to a `Save Video` node to get your final animation file.
1. **Compose Your Scene**: Build complex scenes by loading multiple `RenderFormer Mesh Loader` nodes. Group them logically using `RenderFormer Mesh Combine` nodes. You can even chain `Combine` nodes together for highly organized, large-scale scenes.
2. **Animate the Camera**: Define the starting view with a `RenderFormer Camera` node and the ending view with a `RenderFormer Camera Target` node.
3. **Animate Multiple Lights**:
* For each light you want to animate, create a pair of `RenderFormer Lighting` (for the start state) and `RenderFormer Lighting Target` (for the end state) nodes.
* Feed all the `Lighting Target` outputs into a single `RenderFormer Lighting Combine` node. This node intelligently gathers all start and end states.
4. **Build the Scene Sequence**:
* Connect your final combined `MESH` to the `RenderFormer Scene Builder`.
* Connect your starting `RenderFormer Camera` to the required `camera` input.
* Connect your `RenderFormer Camera Target` to the optional `camera_sequence` input.
* The `RenderFormer Lighting Combine` node provides two outputs. Connect its `LIGHTING` output to the required `lighting` input of the Scene Builder, and its `LIGHTING_SEQUENCE` output to the optional `lighting_sequence` input.
5. **Render and Save**: The `RenderFormer Sampler` will receive the `SCENE_SEQUENCE` and render all frames, which can then be saved as a video.
---
@@ -246,12 +257,12 @@ This wrapper provides a comprehensive set of nodes to build 3D scenes.
- **RenderFormer Lighting**: Creates a configurable emissive light source.
#### Animation
- **RenderFormer Camera Target**: Creates a camera animation sequence by interpolating between a start and end camera state (position, look-at, FOV).
- **RenderFormer Lighting Target**: Creates a light animation sequence by interpolating between a start and end light state (position, rotation, scale, and emission).
- **RenderFormer Camera Target**: Creates a camera animation sequence by defining a start and end camera state (position, look-at, FOV).
- **RenderFormer Lighting Target**: Creates a light animation sequence by defining a start and end light state (position, rotation, scale, and emission).
#### Utilities
- **RenderFormer Mesh Combine**: Combines multiple `MESH` outputs into a single object list.
- **RenderFormer Lighting Combine**: A powerful node that enables multi-light animation. It accepts any combination of static lights and animated light sequences, and outputs two complete lists: one for the start frame and one for the end frame. This allows for complex, synchronized lighting animations.
- **RenderFormer Lighting Combine**: A powerful node that enables multi-light animation. It accepts any combination of static lights (`LIGHTING`) and animated lights (`LIGHTING_SEQUENCE`), and outputs a single, unified `LIGHTING_SEQUENCE` ready for the Scene Builder. It also outputs the `start_frame_lighting` separately for a direct, convenient connection.
- **RenderFormer Remesh**: Simplifies the geometry of a mesh to a target face count using `pymeshlab`.
- **RenderFormer Random Colors**: Applies random vertex colors to a mesh for creative effects or debugging.
@@ -263,13 +274,11 @@ This wrapper provides a comprehensive set of nodes to build 3D scenes.
### 📜 Version History
#### Version 0.4.0 - Multi-Light Animation
- **Feature: Multi-Light Animation:** The lighting system has been completely overhauled to support the animation of multiple lights simultaneously.
- **Workflow:** The `RenderFormerLightingCombine` node is now the central hub for all lighting. It takes any combination of static and animated lights and produces a `start_frame_lighting` and `end_frame_lighting` list.
- **Workflow:** The `RenderFormerSceneBuilder` now uses the `end_lighting` input to receive the end-frame data, simplifying the animation triggering logic. This removes the need for a separate `lighting_sequence` input.
#### Version 0.3.1 - Flexible Lighting Workflow
- **Workflow:** The `RenderFormerLightingCombine` node has been overhauled. It now accepts both static `LIGHTING` and animated `LIGHTING_SEQUENCE` inputs simultaneously. It intelligently separates the animation data, allowing you to combine multiple static lights with one animated light in a single, streamlined workflow.
#### Version 0.5.0 - Workflow Optimization
- **Workflow:** The animation workflow has been significantly simplified and made more robust. The `RenderFormerSceneBuilder` now has a **required** `lighting` input that explicitly defines the start-frame lighting.
- **Workflow:** The `end_lighting` input on the `Scene Builder` has been **removed**. All lighting animation is now handled by the optional `lighting_sequence` input, which provides the end-frame state.
- **Data Flow:** The `RenderFormerLightingTarget` and `RenderFormerLightingCombine` nodes have been updated to output a standardized `LIGHTING_SEQUENCE` dictionary (`{ "start_lights": [...], "end_lights": [...] }`), ensuring consistent data flow.
- **Convenience:** The `RenderFormerLightingCombine` node now provides a separate `start_frame_lighting` output for a direct connection to the `Scene Builder`'s required `lighting` input, preserving a clean and logical graph.
#### Version 0.3.0 - Animation Overhaul and Performance Boost
- **Feature: Full Light Animation:** The `RenderFormerLightingTarget` node now supports animating a light's position, rotation, scale, and emissive strength, enabling complex dynamic lighting effects.
__pycache__/__init__.cpython-312.pyc
BIN
View File
Binary file not shown.
__pycache__/nodes.cpython-312.pyc
BIN
View File
Binary file not shown.
assets/img/phrenderformer_wip_02.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 786 KiB

After

Width:  |  Height:  |  Size: 837 KiB

example_workflows/ph_renderformer_advanced_01.json
+1 -1
View File
File diff suppressed because one or more lines are too long
example_workflows/ph_renderformer_basic_01.json
+1 -1
View File
File diff suppressed because one or more lines are too long
nodes.py
+65 -80
View File
@@ -435,9 +435,9 @@ class RenderFormerLightingTarget:
return {
"required": {
"start_lighting": ("LIGHTING",),
"end_pos_x": ("FLOAT", {"default": 0.0, "min": -10.0, "max": 10.0, "step": 0.01}),
"end_pos_y": ("FLOAT", {"default": 0.0, "min": -10.0, "max": 10.0, "step": 0.01}),
"end_pos_z": ("FLOAT", {"default": 2.1, "min": -10.0, "max": 10.0, "step": 0.01}),
"end_pos_x": ("FLOAT", {"default": 0.0, "min": -10.0, "max": 10.0, "step": 0.001}),
"end_pos_y": ("FLOAT", {"default": 0.0, "min": -10.0, "max": 10.0, "step": 0.001}),
"end_pos_z": ("FLOAT", {"default": 2.1, "min": -10.0, "max": 10.0, "step": 0.001}),
"end_rot_x": ("FLOAT", {"default": 0.0, "min": -360.0, "max": 360.0, "step": 0.1}),
"end_rot_y": ("FLOAT", {"default": 0.0, "min": -360.0, "max": 360.0, "step": 0.1}),
"end_rot_z": ("FLOAT", {"default": 0.0, "min": -360.0, "max": 360.0, "step": 0.1}),
@@ -446,21 +446,18 @@ class RenderFormerLightingTarget:
}
}
RETURN_TYPES = ("LIGHTING_SEQUENCE",)
RETURN_TYPES = ("LIGHTING",)
FUNCTION = "get_lighting_sequence"
CATEGORY = "PHRenderFormer"
def get_lighting_sequence(self, start_lighting, end_pos_x, end_pos_y, end_pos_z, end_rot_x, end_rot_y, end_rot_z, end_scale, end_emissive_strength):
# This node receives the single light to be animated.
if not start_lighting or not isinstance(start_lighting, list) or not start_lighting[0]:
raise Exception("RenderFormerLightingTarget: Invalid start_lighting input.")
start_light_def = start_lighting[0]
# Create a deep copy for the end state to ensure complete data isolation.
end_light_def = copy.deepcopy(start_light_def)
# Modify the deep-copied data with the new end-state values.
end_light_def["transform"]["translation"] = [end_pos_x, end_pos_y, end_pos_z]
end_light_def["transform"]["rotation"] = [end_rot_x, end_rot_y, end_rot_z]
end_light_def["transform"]["scale"] = [end_scale, end_scale, end_scale]
@@ -469,9 +466,12 @@ class RenderFormerLightingTarget:
base_color = [c / max(start_emissive) if max(start_emissive) > 0 else 0 for c in start_emissive]
end_light_def["material"]["emissive"] = [c * end_emissive_strength for c in base_color]
# The output sequence contains ONLY the start and end states for the animated light.
# The output is a dictionary containing both start and end states.
# It is returned under the generic 'LIGHTING' type to allow it to be connected
# to the Combine node.
lighting_output = {
"sequence": [start_light_def, end_light_def]
"start_lights": [start_light_def],
"end_lights": [end_light_def]
}
return (lighting_output,)
@@ -480,56 +480,65 @@ class RenderFormerLightingTarget:
class RenderFormerLightingCombine:
@classmethod
def INPUT_TYPES(cls):
# All inputs are now of the generic 'LIGHTING' type.
# The node's logic will distinguish between static lists and animated dicts.
return {
"optional": {
"lighting_1": ("*",), "lighting_2": ("*",), "lighting_3": ("*",), "lighting_4": ("*",),
"lighting_5": ("*",), "lighting_6": ("*",), "lighting_7": ("*",), "lighting_8": ("*",),
"lighting_1": ("LIGHTING",), "lighting_2": ("LIGHTING",),
"lighting_3": ("LIGHTING",), "lighting_4": ("LIGHTING",),
"lighting_5": ("LIGHTING",), "lighting_6": ("LIGHTING",),
"lighting_7": ("LIGHTING",), "lighting_8": ("LIGHTING",),
}
}
RETURN_TYPES = ("LIGHTING", "LIGHTING",)
RETURN_NAMES = ("start_frame_lighting", "end_frame_lighting",)
RETURN_NAMES = ("LIGHTING", "LIGHTING_SEQUENCE",)
FUNCTION = "combine_lighting"
CATEGORY = "PHRenderFormer"
def combine_lighting(self, **kwargs):
start_frame_lights = []
end_frame_lights = []
all_start_lights = []
animated_end_map = {}
has_animation = False
# First pass to gather all lights and check for animations
all_inputs = []
# First pass: Collect all start lights and map animated end states by UUID
for i in range(1, 9):
light_input = kwargs.get(f"lighting_{i}")
if light_input:
all_inputs.append(light_input)
if isinstance(light_input, dict) and "sequence" in light_input:
has_animation = True
if not light_input:
continue
# Process all inputs
for light_input in all_inputs:
# It's an animation sequence
if isinstance(light_input, dict) and "sequence" in light_input:
sequence = light_input.get("sequence", [])
if len(sequence) >= 2:
start_frame_lights.append(sequence[0])
end_frame_lights.append(sequence[1])
# It's a static light (or list of static lights)
# It's an animation sequence (a dictionary)
if isinstance(light_input, dict):
start_defs = light_input.get("start_lights", [])
end_defs = light_input.get("end_lights", [])
all_start_lights.extend(start_defs)
# Map the end state of each light in the sequence by its UUID
for j, start_def in enumerate(start_defs):
if "ph_uuid" in start_def and j < len(end_defs):
animated_end_map[start_def["ph_uuid"]] = end_defs[j]
# It's a static light (a list of light definitions)
elif isinstance(light_input, list):
for light_def in light_input:
# Add the static light to both start and end frames
start_frame_lights.append(light_def)
if has_animation:
# If we are animating ANY light, static lights must be in the end list too
end_frame_lights.append(copy.deepcopy(light_def))
all_start_lights.extend(light_input)
# If there was no animation, the end_frame_lights list will be empty.
# In this case, we should return None for the end frame to signal no animation.
if not has_animation:
end_frame_lights = None
# Second pass: Build the end_frame_lights list to ensure 1-to-1 correspondence
all_end_lights = []
for start_light in all_start_lights:
uuid = start_light.get("ph_uuid")
# If this light has a defined end state in our map, use it.
if uuid and uuid in animated_end_map:
all_end_lights.append(animated_end_map[uuid])
# Otherwise, it's a static light, so duplicate its start state for the end frame.
else:
all_end_lights.append(copy.deepcopy(start_light))
return (start_frame_lights, end_frame_lights)
# The combined output is a dictionary, but it's returned as type 'LIGHTING'.
lighting_sequence = {
"start_lights": all_start_lights,
"end_lights": all_end_lights
}
# Return the start frame lights first (for the top output) and then the full sequence.
return (all_start_lights, lighting_sequence)
class RenderFormerSceneBuilder:
"""
@@ -545,8 +554,8 @@ class RenderFormerSceneBuilder:
},
"optional": {
"camera_sequence": ("CAMERA_SEQUENCE",),
"end_lighting": ("LIGHTING",),
"lighting_sequence": ("LIGHTING_SEQUENCE",), # For backward compatibility
"camera_sequence": ("CAMERA_SEQUENCE",),
"lighting_sequence": ("LIGHTING",),
"num_frames": ("INT", {"default": 1, "min": 1, "max": 1000}),
"add_default_background": ("BOOLEAN", {"default": False}),
}
@@ -556,11 +565,11 @@ class RenderFormerSceneBuilder:
FUNCTION = "build_scene"
CATEGORY = "PHRenderFormer"
def build_scene(self, mesh, lighting, camera, camera_sequence=None, end_lighting=None, lighting_sequence=None, num_frames=1, add_default_background=False):
def build_scene(self, mesh, lighting, camera, camera_sequence=None, lighting_sequence=None, num_frames=1, add_default_background=False):
output_scene = None
output_sequence = None
is_animation = num_frames > 1 and (camera_sequence is not None or end_lighting is not None or lighting_sequence is not None)
is_animation = num_frames > 1 and (camera_sequence is not None or lighting_sequence is not None)
if not is_animation:
# Build a single static scene
@@ -568,43 +577,19 @@ class RenderFormerSceneBuilder:
output_scene = self._build_single_scene(tmpdir, mesh, camera, lighting, add_default_background)
return (output_scene, None)
# --- Animation Sequence Building (High-Performance Tensor Interpolation) ---
print(f"PHRenderFormer: Building video scene with {num_frames} frames using tensor interpolation.")
# --- Animation Sequence Building ---
print(f"PHRenderFormer: Building video scene with {num_frames} frames.")
pbar = comfy.utils.ProgressBar(num_frames)
# --- Determine Start and End Configurations ---
# The 'camera' and 'lighting' inputs are ALWAYS the start frame.
start_cam_config = camera
end_cam_config = camera
if camera_sequence:
start_cam_config = camera_sequence["sequence"][0]
end_cam_config = camera_sequence["sequence"][-1]
start_lighting_config = lighting
end_lighting_config = None # Initialize to None
if end_lighting is not None:
# New, preferred workflow for multi-light animation
end_lighting_config = end_lighting
elif lighting_sequence is not None:
# Old, deprecated workflow for single-light animation
print("PHRenderFormer WARNING: The 'lighting_sequence' input is deprecated. Please use the new 'RenderFormerLightingCombine' node and connect its 'end_frame_lighting' output to the 'end_lighting' input of the Scene Builder for a more robust workflow.")
end_lighting_config = copy.deepcopy(lighting) # Start with a copy of the start lights
start_light_def_anim = lighting_sequence["sequence"][0]
end_light_def_anim = lighting_sequence["sequence"][-1]
anim_light_id = start_light_def_anim.get("ph_uuid")
if anim_light_id:
found = False
for i, light_def in enumerate(end_lighting_config):
if light_def.get("ph_uuid") == anim_light_id:
end_lighting_config[i] = end_light_def_anim
found = True
break
if not found:
print(f"Warning: Animated light with UUID {anim_light_id} not found in the main lighting list.")
if end_lighting_config is None:
# If no animation data was provided for lighting, the end state is the same as the start
end_lighting_config = start_lighting_config
# The '..._sequence' inputs provide the end frame. If not provided, the end state is the same as the start.
end_cam_config = camera_sequence["sequence"][-1] if camera_sequence and "sequence" in camera_sequence and camera_sequence["sequence"] else start_cam_config
# Check if lighting_sequence is a dictionary (i.e., an animation) before accessing its keys.
end_lighting_config = lighting_sequence["end_lights"] if lighting_sequence and isinstance(lighting_sequence, dict) and "end_lights" in lighting_sequence else start_lighting_config
# --- Build Keyframe Scenes in completely isolated directories ---
start_scene = None
@@ -770,7 +755,7 @@ class RenderFormerSceneBuilder:
with h5py.File(h5_buffer, "w") as f:
f.create_dataset("triangles", data=all_triangles.astype(np.float32), compression="gzip")
f.create_dataset("vn", data=all_vn.astype(np.float32), compression="gzip")
f.create_dataset("texture", data=all_texture.astype(np.float16), compression="gzip")
f.create_dataset("texture", data=all_texture.astype(np.float32), compression="gzip")
f.create_dataset("c2w", data=np.stack(all_c2w).astype(np.float32), compression="gzip")
f.create_dataset("fov", data=np.array(all_fov).astype(np.float32), compression="gzip")
@@ -1070,7 +1055,7 @@ class RenderFormerExampleScene:
with h5py.File(output_h5_buffer, "w") as f:
f.create_dataset("triangles", data=all_triangles.astype(np.float32), compression="gzip", compression_opts=9)
f.create_dataset("vn", data=all_vn.astype(np.float32), compression="gzip", compression_opts=9)
f.create_dataset("texture", data=all_texture.astype(np.float16), compression="gzip", compression_opts=9)
f.create_dataset("texture", data=all_texture.astype(np.float32), compression="gzip", compression_opts=9)
f.create_dataset("c2w", data=all_c2w.astype(np.float32), compression="gzip", compression_opts=9)
f.create_dataset("fov", data=all_fov.astype(np.float32), compression="gzip", compression_opts=9)
@@ -1539,7 +1524,7 @@ class RenderFormerFromJSON:
with h5py.File(output_h5_buffer, "w") as f:
f.create_dataset("triangles", data=all_triangles.astype(np.float32), compression="gzip", compression_opts=9)
f.create_dataset("vn", data=all_vn.astype(np.float32), compression="gzip", compression_opts=9)
f.create_dataset("texture", data=all_texture.astype(np.float16), compression="gzip", compression_opts=9)
f.create_dataset("texture", data=all_texture.astype(np.float32), compression="gzip", compression_opts=9)
f.create_dataset("c2w", data=all_c2w.astype(np.float32), compression="gzip", compression_opts=9)
f.create_dataset("fov", data=all_fov.astype(np.float32), compression="gzip", compression_opts=9)