iGavroche
a098c72d77
- Added optional in-memory cache to ROCmOptimizedCheckpointLoader to avoid redundant disk loads during repeated API runs. - Implemented cache eviction strategy to manage memory efficiently on gfx1151 architecture. - Updated INPUT_TYPES to include parameters for cache usage and forced reload. - Added integration and unit tests to verify cache behavior and functionality. - Documented caching mechanism in CHECKPOINT_CACHE_API.md and updated LOADER_NODES_GUIDE.md for clarity.
2.0 KiB
2.0 KiB
Checkpoint cache for repeated API runs
When you run the same ComfyUI workflow multiple times via the API (e.g. same prompt, same checkpoint), each run normally re-executes the whole graph and reloads the checkpoint from disk. That can cause high NVMe usage and RAM growth until the system becomes unstable.
Solution: use the ROCm Checkpoint Loader with cache
- Use ROCm Checkpoint Loader instead of ComfyUI’s built-in CheckpointLoaderSimple in your workflow (or API template).
- Leave use_cache enabled (default: True) so that if the same checkpoint is already loaded, the node returns the cached (model, clip, vae) and does not reload from disk.
- Use force_reload only when you need a fresh load (e.g. after replacing the checkpoint file).
Behavior
- First run (or first run after loading a different checkpoint): checkpoint is loaded from disk and cached.
- Later runs with the same checkpoint: cache hit; no disk load, same (model, clip, vae) returned.
- Switching checkpoint: loading a different
ckpt_nameevicts the previous cache (single-slot), runs a gentle memory cleanup, then loads the new checkpoint. This avoids unbounded RAM on gfx1151.
Inputs
| Input | Default | Description |
|---|---|---|
ckpt_name |
(required) | Checkpoint filename. |
use_cache |
True | If True and this checkpoint is already cached, return cached tuple (no reload). |
force_reload |
False | If True, load from disk and update cache (ignores cache for this call). |
compatibility_mode |
False | Extra validation for quantized/unusual models. |
When to use
- Workflows executed repeatedly via API with the same checkpoint.
- Avoiding NVMe and RAM growth from repeated full checkpoint loads.
- gfx1151 (and other architectures) when you want at most one checkpoint in memory at a time (single-slot eviction).
See also
- LOADER_NODES_GUIDE.md – ROCm Checkpoint Loader overview and features.