iGavroche-rocm-ninodes/docs/LOADER_NODES_GUIDE.md

ROCm Loader Nodes Guide

This guide explains the different loader nodes available in ROCm Ninodes and when to use each one.

Overview

ROCm Ninodes provides three specialized loader nodes, each optimized for different model formats and use cases:

1. ROCm Checkpoint Loader - For full checkpoints (MODEL + CLIP + VAE)
2. ROCm Diffusion Loader - For standalone diffusion models (MODEL only)
3. ROCm GGUF Loader - For GGUF format models (quantized binary format)

ROCm Checkpoint Loader

Category: ROCm Ninodes/Loaders
Display Name: ROCm Checkpoint Loader
Class: ROCMOptimizedCheckpointLoader

What It Loads

Full Stable Diffusion checkpoints that contain all three components:

• MODEL (diffusion model/UNet)
• CLIP (text encoder)
• VAE (variational autoencoder)

Supported Formats

• .safetensors (recommended - safe format)
• .ckpt (PyTorch checkpoint)
• .pt (PyTorch model)
• .pth (PyTorch model)

When to Use

• Loading traditional Stable Diffusion checkpoints
• You need all three components (MODEL, CLIP, VAE) from a single file
• Working with standard SD 1.x, SD 2.x, or SDXL models

Example Use Case

Checkpoint: stable-diffusion-v1-5.safetensors
Output: MODEL, CLIP, VAE (all three components)

Features

• ROCm-optimized memory management
• Automatic component detection
• Quantized model support
• Compatibility mode for unusual models
• Optional in-memory cache for repeated API runs: when use_cache is True (default), the same checkpoint is not reloaded on subsequent runs; use force_reload to load from disk and refresh the cache. Single-slot eviction when switching checkpoints keeps RAM bounded (see CHECKPOINT_CACHE_API.md).

---

ROCm Diffusion Loader

Category: ROCm Ninodes/Loaders
Display Name: ROCm Diffusion Loader
Class: ROCmDiffusionLoader

What It Loads

Standalone diffusion models (UNet only) without CLIP or VAE:

• MODEL (diffusion model/UNet only)

Supported Formats

• .safetensors (recommended)
• .ckpt (PyTorch checkpoint)
• .pt (PyTorch model)
• .pth (PyTorch model)
• .bin (binary format)
• .onnx (ONNX format)

Note: Does NOT support GGUF files. Use ROCm GGUF Loader for GGUF format.

When to Use

• Loading standalone diffusion models (WAN, Flux, etc.)
• You already have CLIP and VAE loaded separately
• Working with models that don't include CLIP/VAE
• Loading just the UNet component

Example Use Case

Model: flux-dev.safetensors (diffusion model only)
Output: MODEL (no CLIP or VAE)

Features

• ROCm-optimized for diffusion models
• FP8 precision support (fp8_e4m3fn, fp8_e5m2)
• Automatic fp8_scaled detection
• Memory-efficient loading

---

ROCm GGUF Loader

Category: ROCm Ninodes/Loaders
Display Name: ROCm GGUF Loader
Class: ROCmGGUFLoader

What It Loads

GGUF (GPT-Generated Unified Format) models:

• MODEL (diffusion model in GGUF format)

Supported Formats

• .gguf (GGUF binary format only)

When to Use

• Loading quantized models in GGUF format
• Working with WAN, Flux, or other models distributed as GGUF files
• You need memory-efficient quantized models (Q4, Q5, Q8, etc.)
• The model is only available in GGUF format

Example Use Case

Model: wan2.2_i2v_low_noise_14B_Q4_K_M.gguf
Output: MODEL (GGUF format, quantized)

Features

• Native GGUF format support
• Quantized model detection (Q4, Q5, Q8, etc.)
• ROCm-optimized memory handling
• Efficient tensor conversion from GGUF to PyTorch

Important Notes

• GGUF files are not PyTorch pickle files - they use a different binary format
• GGUF format is commonly used for quantized models to reduce memory usage
• If loading fails, you may need to convert the GGUF file to .safetensors format

---

Comparison Table

Feature	Checkpoint Loader	Diffusion Loader	GGUF Loader
Outputs	MODEL, CLIP, VAE	MODEL only	MODEL only
Use Case	Full checkpoints	Standalone UNet	GGUF quantized
Formats	.safetensors, .ckpt, .pt, .pth	.safetensors, .ckpt, .pt, .pth, .bin, .onnx	.gguf only
GGUF Support	❌ No	❌ No	✅ Yes
Quantization	✅ Yes	✅ Yes (FP8)	✅ Yes (Q4/Q5/Q8)
Memory Optimized	✅ Yes	✅ Yes	✅ Yes

Choosing the Right Loader

Scenario 1: Standard Stable Diffusion Checkpoint

File: stable-diffusion-v1-5.safetensors
→ Use: ROCm Checkpoint Loader
→ Output: MODEL, CLIP, VAE

Scenario 2: Standalone Diffusion Model (WAN, Flux)

File: flux-dev.safetensors (no CLIP/VAE included)
→ Use: ROCm Diffusion Loader
→ Output: MODEL only
→ Note: You'll need separate CLIP and VAE loaders

Scenario 3: Quantized GGUF Model

File: wan2.2_i2v_low_noise_14B_Q4_K_M.gguf
→ Use: ROCm GGUF Loader
→ Output: MODEL only (quantized)

Scenario 4: Mixed Workflow

1. Load checkpoint with ROCm Checkpoint Loader → Get MODEL, CLIP, VAE
2. Or load diffusion model with ROCm Diffusion Loader → Get MODEL
3. Load CLIP separately if needed
4. Load VAE separately if needed

Technical Details

Memory Management

All three loaders use ROCm-optimized memory management:

• No aggressive memory cleanup (prevents fragmentation)
• Trusts PyTorch 2.7+ modern allocators
• Designed for gfx1151 architecture with unified memory

Quantization Support

• Checkpoint Loader: Detects quantized models from filename
• Diffusion Loader: Supports FP8 precision (fp8_e4m3fn, fp8_e5m2)
• GGUF Loader: Native support for Q4, Q5, Q8 quantization levels

Error Handling

All loaders provide:

• ROCm-specific diagnostics
• Memory status logging
• Helpful error messages with troubleshooting steps
• OOM (Out of Memory) detection and suggestions

Troubleshooting

"File not found" Error

• Check that the file is in the correct folder:
  • Checkpoints: ComfyUI/models/checkpoints/
  • Diffusion models: ComfyUI/models/diffusion_models/
  • GGUF files: Can be in diffusion_models/, unet/, or unet_gguf/

"Invalid load key" Error (GGUF files)

• GGUF files use a different binary format than PyTorch
• Use the ROCm GGUF Loader instead of other loaders
• If loading still fails, the GGUF file may need conversion to .safetensors

Out of Memory (OOM) Error

• Try using quantized models (Q8, Q5, Q4 for GGUF)
• Run ComfyUI with --cache-none flag
• Restart ComfyUI to clear memory fragmentation
• Load only one large model at a time

Best Practices

1. Use .safetensors format when possible - Safer and more efficient
2. Match the loader to your file type - Don't use Diffusion Loader for full checkpoints
3. Use GGUF Loader for GGUF files - Other loaders won't work
4. Check file format before loading - Verify the file extension matches the loader
5. Monitor memory usage - All loaders provide memory diagnostics

Version Information

• PyTorch: 2.7+ required
• ROCm: 6.4+ required
• Architecture: Optimized for gfx1151 (AMD Radeon RX 7900 series)