ComfyUI-CXN

Custom ComfyUI nodes for loading and rendering FBX 3D models.

Nodes

Load FBX Model

Loads an FBX file from ComfyUI/input/3d/, reorients it into the pack's canonical shoe frame (+X = lateral, +Y = up, +Z = toe), and optionally applies up to five image textures to the materials in scene order. Outputs a MODEL_3D object.

The canonical orientation is frontal: at the default Render FBX camera (0, 0, 1) with no rotation, the shoe is seen head-on with the toe pointing toward the camera (+Z), the top of the shoe up (+Y), and the lateral axis right (+X).

Inputs:

Param	Type	Default	Notes
model_file	combo	—	FBX file under ComfyUI/input/3d/.
toe_axis	combo	+Y	Which FBX-local axis points toward the toe in the source file. Defaults match PS5I-style models; pick another value if your FBX is authored differently.
up_axis	combo	+X	Which FBX-local axis points to the top of the shoe (opposite of the sole). Must be on a different axis from toe_axis.
texture_1…5	IMAGE	—	Optional. Applied to materials in scene order.

How to find the right axes: load the FBX with the defaults, run Render FBX at x=0, y=0, z=1, all angles 0 — you should see the shoe front-on with the toe pointing at the camera, the laces facing up, and the lateral side to the right. If the toe end isn't facing the camera, try a different toe_axis. If you see the sole instead of the laces, try a different up_axis. Once the default view is the canonical frontal view, Match Foot Pose and Render FBX downstream will compose correctly.

Render FBX

Renders a MODEL_3D to an IMAGE using an offscreen OpenGL (EGL) renderer with a built-in 3-point + ambient lighting setup so the object is always visible.

Required inputs:

Param	Type	Default	Notes
x,y,z	FLOAT	0,0,3	Camera world position
pitch	FLOAT	0	Camera rotation around local X axis (degrees)
yaw	FLOAT	0	Camera rotation around local Y axis (degrees)
roll	FLOAT	0	Camera rotation around local Z axis (degrees)
width	INT	512	Output image width
height	INT	512	Output image height
fov	FLOAT	45	Vertical field of view in degrees

Optional inputs (model pose; identity by default):

Param	Type	Default	Notes
model_x/y/z	FLOAT	0	Translate the model in world space
model_pitch	FLOAT	0	Model rotation around X axis (degrees)
model_yaw	FLOAT	0	Model rotation around Y axis (degrees)
model_roll	FLOAT	0	Model rotation around Z axis (degrees)

Convention: right-handed, Y-up. Camera looks down its local −Z axis. Rotation order is YXZ (yaw → pitch → roll). The model pose is applied first, then the camera transform.

Match Foot Pose (SAM3D Body)

Takes a shoe MODEL_3D plus the skeleton output of ComfyUI-SAM3DBody's SAM 3D Body: Process Image node (custom type SKELETON), detects a foot in the joint cloud, and emits the shoe's model_pose (x,y,z, pitch,yaw,roll) plus the photo's vertical FOV — wired into Render FBX they place the shoe exactly where the foot would project in the original photo.

Inputs:

Param	Type	Default	Notes
model_3d	MODEL_3D	—	Right-foot shoe in canonical orientation (+X = lateral, +Y = up, +Z = toe, frontal view at default camera). Use Load FBX's toe_axis / up_axis to bring any FBX to this convention.
skeleton	SKELETON	—	The skeleton output of SAM3D Body's Process Image node. Reads joint_positions, camera (=pred_cam_t), focal_length.
left_or_right	BOOLEAN	right	Detects the requested foot on its own side of the body.
foot_y_fraction	FLOAT	0.20	Fraction of the side's own Y-range (bottom-up) used as the foot region. Decrease (~0.10) to focus on the foot proper and exclude shin joints — important for tilted poses.
image_height	INT	0	Height in pixels of the original photo (e.g. 1200). Used with focal_length to compute vertical FOV. Leave 0 to disable FOV calculation.
image_width	INT	0	Width in pixels of the original photo. Use it to set Render FBX's width so the rendered aspect matches the photo.

Outputs:

Output	Type	Wire to (on Render FBX)
model_3d	MODEL_3D	model_3d
cam_x	FLOAT	x
cam_y	FLOAT	y
cam_z	FLOAT	z
pitch	FLOAT	model_pitch
yaw	FLOAT	model_yaw
roll	FLOAT	model_roll
fov	FLOAT	fov

The new contract: camera moves to the foot, the shoe stays at world origin and rotates in place. cam_x, cam_y, cam_z go straight to Render FBX's camera inputs; pitch, yaw, roll go to model_pitch / model_yaw / model_roll. Leave Render FBX's model_x, model_y, model_z and camera pitch, yaw, roll at their zero defaults. The rendered image is mathematically equivalent to the previous "model-at-foot, camera-at-origin" wiring (same pixel positions, same pose), but the inputs are easier to override manually — for instance, you can nudge a single rotation channel without disturbing the position, or move the camera to inspect the shoe from a different angle.

How the foot is found

1. Read joint_positions, add pred_cam_t (body moves into camera space), negate Y and Z (CV → graphics: Y up, Z back, camera at origin looking -Z).
2. Drop any joint that sits more than 25 cm below the next-lowest joint in Y. The MHR 127-joint output has a global root anchor at joint 0 that sits ~60 cm below the rest of the skeleton and would otherwise dominate the foot region.
3. Split the remaining joints by the body's lateral centerline (median X) into LEFT (X > median) and RIGHT (X ≤ median) — cam_t shifts the body off X=0, so X=0 is not a reliable split.
4. Within the requested side, take the bottom-foot_y_fraction of joints by Y. Their centroid is the foot position; their XZ-PCA gives the toe-forward axis.
5. If left_or_right is left, mirror the shoe model along its medial-lateral axis so the right-foot canonical mesh becomes a left-foot one. The pose itself comes from the actual LEFT-side joints — no positional mirroring.

Caveat: SAM3D's right-side joints can sit higher than the left's

For some bodies/poses SAM3D's MHR regressor places lower-leg keypoints on the two sides at different heights — i.e. the lowest joint on the right side might be a calf point while the lowest on the left side is a true foot point. When that happens the two shoes will render at different Y levels even though the real person is standing on level ground. If you see this, options:

• Increase foot_y_fraction until both sides include actual ankle-area joints.
• Render each foot in a separate Render FBX, then manually equalize Y in the lower one with model_y before compositing.
• Fall back to driving both feet from the mesh-data variant of this idea (one detected, the other mirrored) if precise photo-overlay isn't required.

Workflow: overlay the shoe on the original photo

[Load Image] ──┬─> [SAM 3D Body: Process Image] ──(mesh_data)──┐
               │                                                │
               └─> width × height                               │
                                                                ▼
[Load FBX Model] ──(model_3d)─────────────────────> [Match Foot Pose] ──(x,y,z,pitch,yaw,roll,fov)──> [Render FBX]
                                                                                                          │
                                                                                       (camera at 0,0,0,  ┘
                                                                                        all angles 0,
                                                                                        width/height = photo size)

Critical wiring rules:

1. Send x, y, z, pitch, yaw, roll from Match Foot Pose into model_x, model_y, model_z, model_pitch, model_yaw, model_roll on Render FBX — NOT into the camera inputs. The camera inputs control the viewpoint; the model inputs place the object in the scene.
2. Set the Render FBX camera to x=0, y=0, z=0, pitch=0, yaw=0, roll=0 — SAM3D's camera lives at the world origin looking -Z.
3. Send fov into Render FBX's fov. (Default 45° will be the wrong scale.)
4. Set Render FBX's width and height to the original photo's pixel dimensions (or any multiple of its aspect ratio).
5. Pass image_height into Match Foot Pose so it can compute FOV.

Common wiring mistakes:

• Wiring the 6 pose floats into the camera inputs instead of the model inputs → shoe appears close-up filling the frame.
• Leaving the camera at the default z=3 → shoe appears small and at the wrong projected position.
• Leaving FOV at the default 45° → shoe scale won't match the photo's perspective.

Install

# 1. System deps (Linux)
sudo apt install libassimp-dev libegl1 libegl-mesa0

# 2. Python deps
pip install -r requirements.txt

Drop FBX files into ComfyUI/input/3d/.

Running ComfyUI in Docker

If you run ComfyUI inside a Docker container, add the system packages to your Dockerfile:

RUN apt-get update && apt-get install -y \
    libassimp-dev libegl1 libegl-mesa0

The custom node's requirements.txt (auto-installed by typical ComfyUI entrypoint scripts) takes care of the Python side. Note: pyrender pins PyOpenGL 3.1.0 (from 2014), which is incompatible with NumPy 2.x — our requirements.txt forces PyOpenGL>=3.1.7 to fix this.

Headless rendering

pyrender is forced to the EGL platform (PYOPENGL_PLATFORM=egl). NVIDIA GPUs provide a hardware EGL backend once libegl1 is installed. If you have no EGL, install libosmesa6 instead and set PYOPENGL_PLATFORM=osmesa before launching ComfyUI.

License

MIT