BlockSwapLTX (QT-capable, .blocks-aware) + --disable-pinned-memory +
--reserve-vram resolves both the second-expert invalid-argument crash
and the full-load OOM; 12.5 min end to end on a 24 GB RTX 4090.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The runtime-LoRA / bf16-scale failures documented in the last commit are
NOT a bake-path bug. cudaHostRegister on mmap-backed checkpoint pages
(comfy-aimdo ModelMMAP loader) in legacy --disable-dynamic-vram mode queues
async CUDA errors that poison the context: silent H2D transfer corruption
plus deferred 'CUDA error: invalid argument'. Verified fix: launch with
--disable-pinned-memory — krea2_turbo_bf16 24.5 GB + style-reference LoRA
+ multi-reference edit now runs correctly on a 24 GB GPU (12.3 s/it,
~+5% vs pinned), stable across free/reload cycles.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TCjmaeny4KRS2TzXyYEgZS
Krea2OstrisEditModelPatch object-patches diffusion_model.forward and drives
block submodules directly (block.mod / block.attn / block.mlp), so the
forward-wrapper swap never fires on the reference-latent paths. Every Krea2
block path begins with block.mod and ends with block.mlp, so this node
triggers the swap with a forward_pre_hook on mod and a forward_hook on mlp.
Verified on RTX 4090 / ComfyUI 0.26.2 with krea2_turbo_fp8 (no LoRA),
blocks_to_swap=14: native, multi-reference and kv_cache paths all bit-exact
vs no-swap, 6.4 GB resident instead of 12.9 GB. Runtime LoRA and bf16-scale
checkpoints are documented as known-broken (CUDA invalid argument; shared
bake-path suspect, see README).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TCjmaeny4KRS2TzXyYEgZS
BS_DEBUG_SYNC=1 makes BlockSwapLTX's ON_LOAD synchronize and log after each
swap/resident/non-block phase so a queued CUDA error can be bisected to the
exact re-layout step. No-op (and zero overhead) unless the env var is set.
Findings recorded with it: all ON_LOAD phases synchronize cleanly even in
the failing 43GB-bf16 case - the subsequent 'invalid argument' is the OS
refusing new memory commit (commit charge exhaustion; RAM+pagefile ceiling),
not a poisoned CUDA context. See README known-issue / todo-013.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- blockswap.py restored byte-identical to the Wan-verified implementation
(a403b51): class BlockSwap, .blocks only - zero regression risk for Wan
- blockswap_ltx.py: class BlockSwapLTX at the fp8-verified engine (b5dc522:
transformer_blocks detection, QuantizedTensor attr-level swap, base.to()
unpin guard); distinct ON_LOAD callback key
- the 155ff95 hardening (mmap materialization / swap-set partial unload /
comfy-pin release) is reverted: it did not fix the open bf16-43GB issue
and regressed the previously passing fp8 smoke (its partial-unload
override also fired in model_unload context and crashed there)
- bundled workflows migrated to BlockSwapLTX; README: two-node table
Verified: fp8 no-LoRA 360x360x49f via BlockSwapLTX 41.6s (parity with
pre-split 40.2s); both nodes registered in /object_info.
Known issue (todo-013, BlockSwapLTX): 43GB bf16 multi-model WF still
poisons the CUDA context after pass-1 - unchanged, see README.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- release comfy's load-time host registrations on swap blocks (unpin_all)
before installing our own pins - they are dead weight against the Windows
MAX_PINNED_MEMORY budget (RAM*0.40)
- materialize file-mapped (safetensors mmap) tensors into plain RAM before
using them as swap masters
- replace ModelPatcher.partially_unload on blockswapped patchers: free VRAM
by growing the swap set instead of comfy's backup-restore path
- extract _swap_out_block() shared by ON_LOAD and the partial-unload override
Known issue (open, todo-013): 43GB bf16 + 30-block swap still poisons the
CUDA context ('invalid argument' at the next module .to()) after pass-1
sampling in 2-pass workflows; see README. fp8 (29GB/14 blocks) unaffected.
Wan-family regression re-test pending after these changes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- comfy_kitchen QuantizedTensor params cannot be moved via .data repointing
(wrapper device fixed at construction, payload in _qdata): swap them at the
module-attribute level, pin the inner _qdata instead of the wrapper
- guard base model .to(): a sibling patcher clone's unpatch_model rebuilds
QT wrappers via Module._apply and would free still-pinned _qdata (dangling
cudaHostRegister -> later 'CUDA error: invalid argument'); deactivate and
unpin all swap state before delegating
- bundle 4 LTX fp8 test workflows; README: LTX-2.3 section + fp8+LoRA caveat
Tested: ltx-2.3-22b-dev-fp8 360x360x49f, blocks_to_swap 12->14 auto,
40.2s vs 47.4s no-swap, 17.9GB resident. Wan path unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LTX-2.3's diffusion_model (LTXAVModel) names its repeated-block
ModuleList "transformer_blocks" (diffusers-style), not "blocks" like
Wan 2.2 / Bernini-R. Detect either name generically so BlockSwap works
across both model families without a regression for the existing Wan
path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verified on-device (2026-07-09): with the flag the node swaps 20/40 blocks
(13.4 GB pinned) and completes; without it the workflow still runs to
completion with only a console warning, which is easy to miss from the UI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Avoid collision with existing *Blockswap custom-node repos (wanBlockswap,
HidreamBlockswap, hvBlockswap, NativeBlockSwap). Node class name (BlockSwap)
and display name are unchanged so existing workflows keep working.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Run DiT models larger than VRAM (Wan 2.2 / Bernini-R 14B fp16, 28.6 GB on a
24 GB GPU) by keeping the first N transformer blocks in pinned system RAM and
streaming each block to the GPU just-in-time during forward.
- ModelPatcher ON_LOAD hook: swap blocks get LoRA patches baked in once,
resident part is made fully GPU-resident (no per-layer LowVramPatch cast)
- Synchronized H2D transfers (async pinned copies busy-loop hang on WDDM)
- ON_DETACH restore; safe across model switching and LoRA changes
- Requires --disable-dynamic-vram; Wan-family (.blocks) tested
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
