From d0ef3b5601ae72c49e735a324daf300567fac68d Mon Sep 17 00:00:00 2001 From: kijai <40791699+kijai@users.noreply.github.com> Date: Thu, 6 Nov 2025 16:37:50 +0200 Subject: [PATCH] Update readme.md --- readme.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/readme.md b/readme.md index 024219d..723769c 100644 --- a/readme.md +++ b/readme.md @@ -1,3 +1,25 @@ +## Update notification that can affect memory use in old workflows + +In a recent update I changed how unmerged LoRA weights are handled: + +Previously mostly due to my laziness they were always loaded from RAM when used, this was of course inefficient and also made using torch.compile for LoRA applying difficult, thus forcing a graph break when using unmerged LoRAs. + +Now the LoRA weights are assigned as buffers to the corresponding modules, so they are part of the blocks and obey the block swapping unifying the offloading and allowing LoRA weights to benefit from the prefetch feature for async offoading. Downside is that this means if you did not use block swap, you will see increased memory use as the LoRAs are part of the model and all on VRAM. + +If you use block swap, the LoRAs are swapped along the rest of the block, but the block size is now larger, this means you may have to compensate with couple of more blocks swapped. + +Example situation: you use 1GB LoRA unmerged and swap 20 blocks on 14B model, we can divide the LoRA size by block count, single block grows by 25MB, 20 blocks grow by 500MB, so your VRAM usage would be 500MB more than before, to compensate you swap 2 more blocks. + +### Unrelated other VRAM issue with torch.compile + +After any update that modifies the model code and when using torch.compile it's common to run into issues with VRAM, this can be caused by using older pytorch/triton version without latest compile fixes, and/or from old triton caches, mostly in Windows. This manifests in the issue that first run of new input size may have drastically increased memory use, which can clear from simply running it again, and once cached, not manifest again. Again I've only seen this happen in Windows. + +To clear your Triton cache you can delete the contents of following (default) folders: + +`C:\Users\\.triton` +`C:\Users\\AppData\Local\Temp\torchinductor_` + + ## Note: Due to the stupid amount of bots or people thinking this is some of video generation service, I've blocked new accounts from posting issues for now. # ComfyUI wrapper nodes for [WanVideo](https://github.com/Wan-Video/Wan2.1) and related models.