Compare commits

..
73 Commits
Author SHA1 Message Date
scraed 1393a46d67 Bump version from 1.4.9 to 1.4.10 2026-01-13 22:17:42 +08:00
scraed 27421363bf Merge pull request #71 from godnight10061/fix/issue69-nan-multivariatenormal
Avoid MultivariateNormal crash on non-finite dynamics
2026-01-13 22:16:34 +08:00
scraed 01400a541d Update test_sho_regression.py 2026-01-13 22:13:30 +08:00
scraed c981354387 Update test_sho_regression.py 2026-01-13 21:46:56 +08:00
scraed c92c8bfb1f Update test_sho_regression.py 2026-01-13 21:38:49 +08:00
scraed 9334025801 Update test_sho_regression.py 2026-01-13 21:31:15 +08:00
scraed 144893ecd2 fall back to overdamped update if nan appears 2026-01-13 18:41:27 +08:00
scraed bce8505ce2 Revert "Avoid MultivariateNormal crash on non-finite dynamics"
This reverts commit 9432a34c38.
2026-01-13 18:29:06 +08:00
godnight10061 0992657e54 Fix CI workflows for torch and default branch 2026-01-13 09:25:41 +08:00
godnight10061 2fa036992b Add unit regression for NaN oscillator mean 2026-01-13 08:56:46 +08:00
godnight10061 9432a34c38 Avoid MultivariateNormal crash on non-finite dynamics 2026-01-13 01:39:48 +08:00
scraed d4e8ee28fb Merge pull request #70 from godnight10061/fix/reshape-mask-3d
Thanks for the contribution! Tested and merged.
2026-01-11 18:22:14 +08:00
godnight10061 34b8e83831 Make CI tooling importable 2026-01-11 11:07:41 +08:00
godnight10061 e29d480a1a Fix reshape_mask for 3D masks 2026-01-10 14:46:14 +08:00
charrywhite 9479106c1d Fix Flux2 Dev Inpaint workflow size bug 2026-01-10 12:43:40 +08:00
charrywhite db4bbbe9f2 Update Flux.2.Dev workflow Preview pic 2026-01-10 11:47:29 +08:00
charrywhite 1a3e15964c Merge branch 'master' of https://github.com/scraed/LanPaint 2026-01-10 11:41:58 +08:00
charrywhite 4cc30f8a1d Fix Flux2.Dev Inpainting Workflow MaskBlend Bug 2026-01-10 11:41:53 +08:00
scraed d1e609192f Bump version from 1.4.8 to 1.4.9 2026-01-06 23:13:59 +08:00
charrywhite bf46831196 Update README.md 2026-01-06 23:01:04 +08:00
charrywhite c9d9a79b18 Update README.md 2026-01-06 22:59:21 +08:00
charrywhite 175156af86 Add Flux.2 Dev support 2026-01-06 22:57:42 +08:00
charrywhite 3281946b32 Merge branch 'master' of https://github.com/scraed/LanPaint 2026-01-06 22:42:16 +08:00
charrywhite 83f557ff4c Support Flux.2 Dev Inpainting 2026-01-06 22:42:07 +08:00
charrywhite 5e7fe4d5e4 Enhance README with new Discord info and features
Updated README to highlight new Discord channel and features.
2026-01-06 21:06:25 +08:00
scraed 69f2aded4d Update README with Discord link and new features
Added Discord link for community engagement and announced new inpainting and outpainting features.
2026-01-06 20:02:15 +08:00
scraed 5f1b6d8989 add discord link 2026-01-06 20:01:41 +08:00
scraed 7b0d144db9 Update Qwen Image Edit 2511 support
Updated references for Qwen Image Edit to include version 2511 alongside 2509.
2025-12-28 18:40:41 +08:00
scraed 27ebb6e7af fix qwen edit shape error 2025-12-25 23:18:51 +08:00
charrywhite f148e4b631 Update paper title in README.md 2025-12-17 22:33:46 +08:00
charrywhite cc9ec8873a Revise citation and add TMLR link
Updated citation format and added TMLR link.
2025-12-17 22:30:06 +08:00
scraed 565087e8f8 fix z image resize error for inpaint 2025-12-15 18:15:15 +08:00
scraed ebdc5dcc02 Merge branch 'master' of https://github.com/scraed/LanPaint 2025-12-06 10:26:06 +08:00
scraed 9336523576 more clear bug report 2025-12-06 10:26:01 +08:00
scraed 21520b38fb Update README.md 2025-12-05 09:39:13 +08:00
scraed 868bbaeefe fix detail display 2025-12-04 15:01:50 +08:00
scraed 3581d6c217 outpaint example 2025-12-04 14:58:16 +08:00
scraed d4da280a92 update version 2025-12-04 14:37:10 +08:00
scraed d0bcd459cc update version 2025-12-04 14:20:58 +08:00
scraed 469012d183 Z image update 2025-12-04 14:17:29 +08:00
scraed 3f4a5cdb5a update pyproject.yaml 2025-11-23 01:11:08 +08:00
scraed e0ce52b1be fix blur issue in Wan 2.2 14B 2025-11-23 00:46:25 +08:00
scraed 2c8f4baec0 add wan 2.2 5B 2025-11-17 16:09:45 +08:00
scraed daf1bdf4be update version 2025-11-17 16:05:42 +08:00
scraed f4d4dfdd61 Merge branch 'master' of https://github.com/scraed/LanPaint 2025-11-17 16:04:51 +08:00
scraed 6d624e7e1e support Wan 2.2 5B 2025-11-17 16:04:48 +08:00
scraed e0940059e6 Update README.md 2025-11-10 13:37:49 +08:00
scraed 76ce20b51f version change 2025-10-26 23:29:23 +08:00
scraed 29ed0b3da5 Add **kwargs to CFGGuider_LanPaint sampling methods
Updated the outer_sample and inner_sample method calls in CFGGuider_LanPaint to accept and forward arbitrary keyword arguments (**kwargs). This allows for greater flexibility and compatibility with future changes or additional parameters.
2025-10-26 23:28:35 +08:00
scraed 3d5aa3a3fa Add Hunyuan supports
Added new example workflows for Hunyuan, including images and JSON workflow files. Updated README with Hunyuan T2I inpainting instructions and links. Increased default LanPaint step size to 0.2 in nodes and advanced sampler, and refactored scale_latent_inpaint logic in lanpaint.py for improved compatibility.
2025-10-16 15:59:13 +08:00
charrywhite 6109df6591 Upload a 41frames outpaint workflows 2025-10-03 22:32:19 +08:00
scraed 536716a7b9 typo fix 2025-10-03 09:59:01 +08:00
scraed ece228ab10 Bump version to 1.4.0 in pyproject.toml
Updated the project version from 1.3.3 to 1.4.0 to prepare for a new release.
2025-10-02 23:24:21 +08:00
charrywhite c048230250 Update Original Video (Fix to 40 frames, fps=12) 2025-10-02 21:48:44 +08:00
charrywhite aa885d9fa6 Update README.md 2025-10-02 21:33:37 +08:00
charrywhite 80d55a47b0 Add 81 frames example 2025-10-02 21:31:19 +08:00
charrywhite 79db1765e3 Add video inpainting (Beta) 2025-10-02 21:13:43 +08:00
charrywhite 929328d08d Update README.md 2025-09-27 00:42:43 +08:00
scraed c12897040b Update README with inpainting result image
Replaced the workflow & masks link with an inpainting result image to better showcase the output of the project.
2025-09-24 18:49:34 +08:00
scraed 48b9a59ca3 Update README with new model support and examples
Added references to Wan2.2 and Qwen Image Edit 2509 support, and included a link to workflow and mask examples for improved documentation.
2025-09-24 18:48:46 +08:00
scraed d55f8c61d0 Add Qwen Image Edit 2509 example to README
Introduced a new section for Qwen Image Edit 2509 inpainting example, including workflow and model download links, and an example image. Updated the table of contents to reference the new example.
2025-09-24 18:43:41 +08:00
scraed a097b9c9b2 Update LanPaintQwen_04.jpg example image
Replaces the existing LanPaintQwen_04.jpg in the examples directory with a new version. This may reflect updated content or improved image quality.
2025-09-24 18:40:25 +08:00
scraed 70cc94cc6a update readme 2025-09-24 18:30:04 +08:00
scraed e653f45103 Support 2509 2025-09-24 18:27:11 +08:00
scraed c9c237c2de Add WAN 2.2 partial inpaint workflow template 2025-09-18 16:55:48 +08:00
scraed c5aa0116d5 Add Example 16 for partial inpainting with reference
Added a new example (Example 16) demonstrating partial inpainting using reference images. Updated README with instructions and links for the new example, and added related images to the examples directory.
2025-09-18 15:40:22 +08:00
scraed 1820d88594 fix link 2025-09-17 15:59:35 +08:00
scraed c265bcc989 Add Wan2.2 inpainting example and bump version
Added a new example for Wan2.2 inpainting with image and workflow links in the README. Updated project version to 1.3.2 in pyproject.toml.
2025-09-17 15:55:28 +08:00
scraed b8973c570b Support Wan 2.2 Txt 2 Img 2025-09-17 15:46:11 +08:00
scraed 1b50a90424 qwen edit resize fix 2025-09-07 13:07:17 +08:00
charrywhite dab41249da Update README.md 2025-08-30 11:15:11 +08:00
charrywhite b2b2b21bf1 Update README.md 2025-08-30 11:14:53 +08:00
scraed 099d4137a9 Clarify speed improvement suggestions
Reworded instructions for boosting speed in README.
2025-08-26 12:10:11 +08:00
78 changed files with 11224 additions and 928 deletions
+1
View File
@@ -26,6 +26,7 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install .[dev]
pip install torch --extra-index-url https://download.pytorch.org/whl/cpu
- name: Run Linting
run: |
ruff check .
+2
View File
@@ -11,3 +11,5 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: comfy-org/node-diff@main
with:
base_ref: ${{ github.event.repository.default_branch }}
+2
View File
@@ -98,3 +98,5 @@ cookiecutter-pypackage-env/
# vscode settings
.history/
*.code-workspace
.vscode/
/.vscode
+4
View File
@@ -5,4 +5,8 @@
"/PATH/TO/ComfyUI/",
"/PATH/TO/ComfyUI/custom_nodes/"
],
"cursorpyright.analysis.extraPaths": [
"/PATH/TO/ComfyUI/",
"/PATH/TO/ComfyUI/custom_nodes/"
],
}
+261 -22
View File
@@ -1,46 +1,96 @@
<div align="center">
# LanPaint: Universal Inpainting Sampler with "Think Mode"
[![arXiv](https://img.shields.io/badge/Arxiv-2502.03491-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2502.03491)
[![TMLR PDF](https://img.shields.io/badge/TMLR-PDF-8A2BE2?logo=openreview&logoColor=white)](https://openreview.net/pdf?id=JPC8JyOUSW)
[![Python Benchmark](https://img.shields.io/badge/🐍-Python_Benchmark-3776AB?logo=python)](https://github.com/scraed/LanPaintBench)
[![ComfyUI Extension](https://img.shields.io/badge/ComfyUI-Extension-7B5DFF)](https://github.com/comfyanonymous/ComfyUI)
[![Hugging Face](https://img.shields.io/badge/Hugging%20Face-yellow?logo=huggingface&logoColor=white)](https://huggingface.co/charrywhite/LanPaint)
[![Blog](https://img.shields.io/badge/📝-Blog-9cf)](https://scraed.github.io/scraedBlog/)
[![GitHub stars](https://img.shields.io/github/stars/scraed/LanPaint)](https://github.com/scraed/LanPaint/stargazers)
[![Discord](https://img.shields.io/badge/Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/aCGZutBV)
</div>
Universally applicable inpainting ability for every model. LanPaint sampler lets the model "think" through multiple iterations before denoising, enabling you to invest more computation time for superior inpainting quality.
This is the official implementation of ["Lanpaint: Training-Free Diffusion Inpainting with Exact and Fast Conditional Inference"](https://arxiv.org/abs/2502.03491). The repository is for ComfyUI extension. Local Python benchmark code is published here: [LanPaintBench](https://github.com/scraed/LanPaintBench).
This is the official implementation of ["LanPaint: Training-Free Diffusion Inpainting with Asymptotically Exact and Fast Conditional Sampling"](https://arxiv.org/abs/2502.03491), accepted by TMLR. The repository is for ComfyUI extension. Local Python benchmark code is published here: [LanPaintBench](https://github.com/scraed/LanPaintBench).
![Qwen Result 2](https://github.com/scraed/LanPaint/blob/master/examples/LanPaintQwen_03.jpg)
Check [Mased Qwen Edit Workflow](https://github.com/scraed/LanPaint/tree/master/examples/Example_14). You need to follow the ComfyUI version of [Qwen Image Edit workflow](https://docs.comfy.org/tutorials/image/qwen/qwen-image-edit) to download and install the model.
## Citation
![Qwen Result 1](https://github.com/scraed/LanPaint/blob/master/examples/LanPaintQwen_01.jpg)
```
@article{
zheng2025lanpaint,
title={LanPaint: Training-Free Diffusion Inpainting with Asymptotically Exact and Fast Conditional Sampling},
author={Candi Zheng and Yuan Lan and Yang Wang},
journal={Transactions on Machine Learning Research},
issn={2835-8856},
year={2025},
url={https://openreview.net/forum?id=JPC8JyOUSW},
note={}
}
```
**🎉 NEW 2026: Join our discord!**
Also check [Qwen Inpaint Workflow](https://github.com/scraed/LanPaint/tree/master/examples/Example_13) and [Qwen Outpaint Workflow](https://github.com/scraed/LanPaint/tree/master/examples/Example_12). You need to follow the ComfyUI version of [Qwen Image workflow](https://docs.comfy.org/tutorials/image/qwen/qwen-image) to download and install the model.
[Join our Discord](https://discord.gg/aCGZutBV) to share experiences, discuss features, and explore future development.
**🎬 NEW: LanPaint now supports inpainting and outpainting based on Z-Image!**
| Original | Masked | Inpainted |
|:--------:|:------:|:---------:|
| ![Original Z-image](https://github.com/scraed/LanPaint/blob/master/examples/Example_21/Original_No_Mask.png) | ![Masked Z-image](https://github.com/scraed/LanPaint/blob/master/examples/Example_21/Masked_Load_Me_in_Loader.png) | ![Inpainted Z-image](https://github.com/scraed/LanPaint/blob/master/examples/Example_21/InPainted_Drag_Me_to_ComfyUI.png) |
**🎬 NEW: LanPaint now supports video inpainting and outpainting based on Wan 2.2!**
<div align="center">
| Original Video | Mask (edit T-shirt text) | Inpainted Result |
|:--------------:|:----:|:----------------:|
| ![Original](https://github.com/scraed/LanPaint/blob/master/examples/Original_No_Mask-example18.gif) | ![Mask](https://github.com/scraed/LanPaint/blob/master/examples/Example_18/Masked_Load_Me_in_Loader.png) | ![Result](https://github.com/scraed/LanPaint/blob/master/examples/Inpainted_81frames_Drag_Me_to_ComfyUI_example18.gif) |
*Video Inpainting Example: 81 frames with temporal consistency*
</div>
Check our latest [Wan 2.2 Video Examples](#video-examples-beta), [Wan 2.2 Image Examples](#example-wan22-inpaintlanpaint-k-sampler-5-steps-of-thinking), and
[Qwen Image Edit 2509](#example-qwen-edit-2509-inpaint) support.
## Table of Contents
- [Features](#features)
- [Quickstart](#quickstart)
- [How to Use Examples](#how-to-use-examples)
- [Examples](#examples)
- [Video Examples (Beta)](#video-examples-beta)
- [Wan 2.2 Video Inpainting](#wan-22-video-inpainting)
- [Wan 2.2 5B Video Inpainting](#wan-22-5b-video-inpainting)
- [Wan 2.2 Video Outpainting](#wan-22-video-outpainting)
- [Resource Consumption](#resource-consumption)
- [Image Examples](#image-examples)
- [Flux.2.Dev](#example-flux2dev-inpaintlanpaint-k-sampler-5-steps-of-thinking)
- [Z-image](#example-z-image-inpaintlanpaint-k-sampler-5-steps-of-thinking)
- [Hunyuan T2I](#example-hunyuan-t2i-inpaintlanpaint-k-sampler-5-steps-of-thinking)
- [Wan 2.2 T2I](#example-wan22-inpaintlanpaint-k-sampler-5-steps-of-thinking)
- [Wan 2.2 T2I with reference](#example-wan22-partial-inpaintlanpaint-k-sampler-5-steps-of-thinking)
- [Qwen Image Edit 2511 2509](#example-qwen-edit-2509-inpaint)
- [Qwen Image Edit 2508](#example-qwen-edit-2508-inpaint)
- [Qwen Image](#example-qwen-image-inpaintlanpaint-k-sampler-5-steps-of-thinking)
- [HiDream](#example-hidream-inpaint-lanpaint-k-sampler-5-steps-of-thinking)
- [SD 3.5](#example-sd-35-inpaintlanpaint-k-sampler-5-steps-of-thinking)
- [Flux](#example-flux-inpaintlanpaint-k-sampler-5-steps-of-thinking)
- [SDXL Examples](#example-sdxl-0-character-consistency-side-view-generation-lanpaint-k-sampler-5-steps-of-thinking)
- [SDXL](#example-sdxl-0-character-consistency-side-view-generation-lanpaint-k-sampler-5-steps-of-thinking)
- [Usage](#usage)
- [Basic Sampler](#basic-sampler)
- [Advanced Sampler](#lanpaint-ksampler-advanced)
- [Tuning Guide](#lanpaint-ksampler-advanced-tuning-guide)
- [Community Showcase](#community-showcase-)
- [Community Showcase](#community-showcase-)
- [FAQ](#faq)
- [Updates](#updates)
- [ToDo](#todo)
- [Citation](#citation)
## Features
- **Universal Compatibility** – Works instantly with almost any model (**SD 1.5, XL, 3.5, Flux, HiDream, Qwen-Image or custom LoRAs**) and ControlNet.
- **Universal Compatibility** – Works instantly with almost any model (**Z-image, Hunyuan, Wan 2.2, Qwen Image/Edit, HiDream, SD 3.5, Flux-series, SDXL, SD 1.5 or custom LoRAs**) and ControlNet.
![Inpainting Result 13](https://github.com/scraed/LanPaint/blob/master/examples/InpaintChara_13.jpg)
- **No Training Needed** – Works out of the box with your existing model.
- **Easy to Use** – Same workflow as standard ComfyUI KSampler.
@@ -76,10 +126,175 @@ Once installed, you'll find the LanPaint nodes under the "sampling" category in
- **[VAE Encode for Inpainting](https://comfyanonymous.github.io/ComfyUI_examples/inpaint/)**
- **[Set Latent Noise Mask](https://comfyui-wiki.com/en/tutorial/basic/how-to-inpaint-an-image-in-comfyui)**
## Examples
## Video Examples (Beta)
LanPaint now supports video inpainting with Wan 2.2, enabling you to seamlessly inpaint masked regions across video frames while maintaining temporal consistency.
**Note:** LanPaint supports video inpainting for longer sequences (e.g., 81 frames), but processing time increases significantly (please check the [Resource Consumption](#resource-consumption) section for details) and performance may become unstable. For optimal results and stability, we recommend limiting video inpainting to **40 frames or fewer**.
### Wan 2.2 Video Inpainting
*Example: Wan2.2 t2v 14B, 480p video (11:6), 40 frames, LanPaint K Sampler, 2 steps of thinking*
| Original Video | Mask (Add a white hat) | Inpainted Result |
|:--------------:|:----:|:----------------:|
| ![Original Video](https://github.com/scraed/LanPaint/blob/master/examples/Original_No_Mask_example17.gif) | ![Mask](https://github.com/scraed/LanPaint/blob/master/examples/Example_17/Masked_Load_Me_in_Loader.png) | ![Inpainted Result](https://github.com/scraed/LanPaint/blob/master/examples/Inpainted_40frames_Drag_Me_to_ComfyUI_example17.gif) |
[View Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_17)
You need to follow the ComfyUI version of [Wan2.2 T2V workflow](https://docs.comfy.org/tutorials/video/wan/wan2_2) to download and install the T2V model.
### Wan 2.2 5B Video Inpainting
Similar to Wan 2.2 14B with slightly different workflow. [View Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_17)
### Wan 2.2 Video Outpainting
Extend your videos beyond their original boundaries with LanPaint's video outpainting capability based on Wan 2.2. This feature allows you to expand the canvas of your videos while maintaining coherent motion and context.
*Example: Wan2.2 t2v 14B, 480p video (1:1 outpaint to 11:6), 40 frames, LanPaint K Sampler, 2 steps of thinking*
| Original Video | Mask (Expand to 880x480) | Outpainted Result |
|:--------------:|:----:|:-----------------:|
| ![Original Video](https://github.com/scraed/LanPaint/blob/master/examples/Original_Load_Me_in_Loader_example19.gif) | ![Mask](https://github.com/scraed/LanPaint/blob/master/examples/Mask_Example19_.png) | ![Outpainted Result](https://github.com/scraed/LanPaint/blob/master/examples/Outpainted_40frames_Drag_Me_to_ComfyUI_example19.gif) |
[View Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_19)
You need to follow the ComfyUI version of [Wan2.2 T2V workflow](https://docs.comfy.org/tutorials/video/wan/wan2_2) to download and install the T2V model.
### Resource Consumption
<table>
<thead>
<tr>
<th align="left">Processing Mode</th>
<th align="left">Resolution</th>
<th align="left">Frames Processed</th>
<th align="left">VRAM Required</th>
<th align="left">Total Runtime (20 steps)</th>
</tr>
</thead>
<tbody>
<tr style="background-color: #e8f4f8;">
<td><strong>Inpainting</strong></td>
<td>880×480 (11:6)</td>
<td>40 frames</td>
<td>39.8 GB</td>
<td><strong>05:37 min</strong></td>
</tr>
<tr style="background-color: #e8f4f8;">
<td><strong>Inpainting</strong></td>
<td>480×480 (1:1)</td>
<td>40 frames</td>
<td>38.0 GB</td>
<td><strong>05:35 min</strong></td>
</tr>
<tr style="background-color: #e8f4f8;">
<td><strong>Outpainting</strong></td>
<td>880×480 (11:6)</td>
<td>40 frames</td>
<td>40.2 GB</td>
<td><strong>05:36 min</strong></td>
</tr>
<tr style="background-color: #fff4e6;">
<td><strong>Inpainting</strong></td>
<td>880×480 (11:6)</td>
<td>81 frames</td>
<td>43.3 GB</td>
<td><strong>16:23 min</strong></td>
</tr>
<tr style="background-color: #fff4e6;">
<td><strong>Inpainting</strong></td>
<td>480×480 (1:1)</td>
<td>81 frames</td>
<td>39.8 GB</td>
<td><strong>14:25 min</strong></td>
</tr>
<tr style="background-color: #fff4e6;">
<td><strong>Outpainting</strong></td>
<td>880×480 (11:6)</td>
<td>81 frames</td>
<td>42.6 GB</td>
<td><strong>13:46 min</strong></td>
</tr>
</tbody>
</table>
<sub>**Test Platform**: All tests were conducted on an NVIDIA RTX Pro 6000.<br>
**Model Used**: `wan2.2_t2v_low_noise_14B_fp8_scaled.safetensors` and `wan2.2_t2v_high_noise_14B_fp8_scaled.safetensors`.<br>
**Processing Steps**: 20 sampling steps x 2 (LanPaint steps of thinking).</sub>
**Note:** Vram is required by the model, not LanPaint. To further reduce VRAM requirements, we recommend generating less frames and loading CLIP on CPU.
## Image Examples
### Example Hunyuan T2I: InPaint(LanPaint K Sampler, 5 steps of thinking)
We are excited to announce that LanPaint now supports inpainting with Hunyuan text to image generation.
[View Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_20)
You need to follow the ComfyUI version of [Hunyuan workflow](https://docs.comfy.org/tutorials/video/hunyuan-video#hunyuan-text-to-video-workflow) to download and install the model.
### Example Wan2.2: InPaint(LanPaint K Sampler, 5 steps of thinking)
We are excited to announce that LanPaint now supports Wan2.2 text to image generation with Wan2.2 T2V model.
![Inpainting Result 45](https://github.com/scraed/LanPaint/blob/master/examples/InpaintChara_45.jpg)
[View Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_15)
You need to follow the ComfyUI version of [Wan2.2 T2V workflow](https://docs.comfy.org/tutorials/video/wan/wan2_2) to download and install the T2V model.
### Example Z-image: InPaint(LanPaint K Sampler, 5 steps of thinking)
LanPaint also supports inpainting with the Z-image text-to-image model.
<details open>
<summary>View Original / Masked / Inpainted Comparison</summary>
| Original | Masked | Inpainted |
|:--------:|:------:|:---------:|
| ![Original Z-image](https://github.com/scraed/LanPaint/blob/master/examples/Example_21/Original_No_Mask.png) | ![Masked Z-image](https://github.com/scraed/LanPaint/blob/master/examples/Example_21/Masked_Load_Me_in_Loader.png) | ![Inpainted Z-image](https://github.com/scraed/LanPaint/blob/master/examples/Example_21/InPainted_Drag_Me_to_ComfyUI.png) |
</details>
[View Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_21)
<details open>
<summary>View Z-image Outpainting (Original / Masked / Outpainted)</summary>
| Original | Masked | Outpainted |
|:--------:|:------:|:----------:|
| ![Original Z-image Outpaint](https://github.com/scraed/LanPaint/blob/master/examples/Example_22/Original_No_Mask.png) | ![Masked Z-image Outpaint](https://github.com/scraed/LanPaint/blob/master/examples/Example_22/Masked_Load_Me_in_Loader.png) | ![Outpainted Z-image](https://github.com/scraed/LanPaint/blob/master/examples/Example_22/InPainted_Drag_Me_to_ComfyUI.png) |
</details>
[View Outpaint Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_22)
You can download the Z-image model for ComfyUI from [Z-image](https://docs.comfy.org/zh-CN/tutorials/image/z-image/z-image-turbo).
### Example Wan2.2: Partial InPaint(LanPaint K Sampler, 5 steps of thinking)
Sometimes we don't want to inpaint completely new content, but rather let the inpainted image reference the original image. One option to achieve this is to inpaint with an edit model like Qwen Image Edit. Another option is to perform a partial inpaint: allowing the diffusion process to start at some middle steps rather than from 0.
![Inpainting Result 46](https://github.com/scraed/LanPaint/blob/master/examples/InpaintChara_46.jpg)
[View Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_16)
You need to follow the ComfyUI version of [Wan2.2 T2V workflow](https://docs.comfy.org/tutorials/video/wan/wan2_2) to download and install the T2V model.
### Example Qwen Edit 2509: InPaint
Check our latest updated [Mased Qwen Edit Workflow](https://github.com/scraed/LanPaint/tree/master/examples/Example_14) for Qwen Image Edit 2509. Download the model at [Qwen Image Edit 2509 Comfy](https://huggingface.co/Comfy-Org/Qwen-Image-Edit_ComfyUI/tree/main/split_files/diffusion_models). This workflow also supports Qwen Image Edit 2511.
![Qwen Result 3](https://github.com/scraed/LanPaint/blob/master/examples/LanPaintQwen_04.jpg)
### Example Qwen Edit 2508: InPaint
![Qwen Result 2](https://github.com/scraed/LanPaint/blob/master/examples/LanPaintQwen_03.jpg)
Check [Mased Qwen Edit Workflow](https://github.com/scraed/LanPaint/tree/master/examples/Example_14). You need to follow the ComfyUI version of [Qwen Image Edit workflow](https://docs.comfy.org/tutorials/image/qwen/qwen-image-edit) to download and install the model.
### Example Qwen Image: InPaint(LanPaint K Sampler, 5 steps of thinking)
We are excited to announce that LanPaint now supports Qwen Image, providing powerful inpainting capabilities for image editing.
![Inpainting Result 14](https://github.com/scraed/LanPaint/blob/master/examples/InpaintChara_14.jpg)
[View Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_11)
@@ -89,6 +304,9 @@ You need to follow the ComfyUI version of [Qwen Image workflow](https://docs.com
The following examples utilize a random seed of 0 to generate a batch of 4 images for variance demonstration and fair comparison. (Note: Generating 4 images may exceed your GPU memory; please adjust the batch size as necessary.)
![Qwen Result 1](https://github.com/scraed/LanPaint/blob/master/examples/LanPaintQwen_01.jpg)
Also check [Qwen Inpaint Workflow](https://github.com/scraed/LanPaint/tree/master/examples/Example_13) and [Qwen Outpaint Workflow](https://github.com/scraed/LanPaint/tree/master/examples/Example_12). You need to follow the ComfyUI version of [Qwen Image workflow](https://docs.comfy.org/tutorials/image/qwen/qwen-image) to download and install the model.
### Example HiDream: InPaint (LanPaint K Sampler, 5 steps of thinking)
![Inpainting Result 8](https://github.com/scraed/LanPaint/blob/master/examples/InpaintChara_11.jpg)
[View Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_8)
@@ -107,6 +325,23 @@ You need to follow the ComfyUI version of [HiDream workflow](https://docs.comfy.
You need to follow the ComfyUI version of [SD 3.5 workflow](https://comfyui-wiki.com/en/tutorial/advanced/stable-diffusion-3-5-comfyui-workflow) to download and install the model.
### Example Flux.2.Dev: InPaint(LanPaint K Sampler, 5 steps of thinking)
<details open>
<summary>View Original / Masked / Inpainted Comparison</summary>
| Original | Masked | Inpainted |
|:--------:|:------:|:---------:|
| ![Original Flux.2.Dev](https://github.com/scraed/LanPaint/blob/master/examples/Example_23/Original_No_Mask.png) | ![Masked Flux.2.Dev](https://github.com/scraed/LanPaint/blob/master/examples/Example_23/Masked_Load_Me_in_Loader.png) | ![Inpainted Flux.2.Dev](https://github.com/scraed/LanPaint/blob/master/examples/Example_23/InPainted_Drag_Me_to_ComfyUI.png) |
</details>
[View Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_23)
[Model Used in This Example](https://huggingface.co/Comfy-Org/flux2-dev)
(Note: Prompt First mode is disabled on Flux.2.Dev. As it does not use CFG guidance.)
### Example Flux: InPaint(LanPaint K Sampler, 5 steps of thinking)
![Inpainting Result 7](https://github.com/scraed/LanPaint/blob/master/examples/InpaintChara_10.jpg)
[View Workflow & Masks](https://github.com/scraed/LanPaint/tree/master/examples/Example_7)
@@ -198,7 +433,7 @@ For challenging inpainting tasks:
Increase **total number of sampling steps** (very important!), **LanPaint_NumSteps** (thinking iterations) or **LanPaint_Lambda** if the inpainted result does not meet your expectations.
2️⃣ **Boost Speed**
If you want better results but still need fewer steps, consider:
Decrease **LanPaint_NumSteps** to accelerate generation! If you want better results but still need fewer steps, consider:
- **Increasing LanPaint_StepSize** to speed up the thinking process.
- **Decreasing LanPaint_Friction** to make the Langevin dynamics converges more faster.
@@ -223,6 +458,9 @@ Discover how the community is using LanPaint! Here are some user-created tutoria
Submit a PR to add your tutorial/video here, or open an [Issue](https://github.com/scraed/LanPaint/issues) with details!
## FAQ
[Working togather with crop&stitch](https://github.com/scraed/LanPaint/issues/46)
## Updates
- 2025/08/08
- Add Qwen image support
@@ -251,17 +489,19 @@ Submit a PR to add your tutorial/video here, or open an [Issue](https://github.c
- Try Implement Detailer
- ~~Provide inference code on without GUI.~~ Check our local Python benchmark code [LanPaintBench](https://github.com/scraed/LanPaintBench).
## Citation
```
@misc{zheng2025lanpainttrainingfreediffusioninpainting,
title={Lanpaint: Training-Free Diffusion Inpainting with Exact and Fast Conditional Inference},
author={Candi Zheng and Yuan Lan and Yang Wang},
year={2025},
eprint={2502.03491},
archivePrefix={arXiv},
primaryClass={eess.IV},
url={https://arxiv.org/abs/2502.03491},
@article{
zheng2025lanpaint,
title={LanPaint: Training-Free Diffusion Inpainting with Asymptotically Exact and Fast Conditional Sampling},
author={Candi Zheng and Yuan Lan and Yang Wang},
journal={Transactions on Machine Learning Research},
issn={2835-8856},
year={2025},
url={https://openreview.net/forum?id=JPC8JyOUSW},
note={}
}
```
@@ -269,4 +509,3 @@ Submit a PR to add your tutorial/video here, or open an [Issue](https://github.c
+80 -2
View File
@@ -10,7 +10,85 @@ __author__ = """LanPaint"""
__email__ = "czhengac@connect.ust.hk"
__version__ = "0.0.1"
from .src.LanPaint.nodes import NODE_CLASS_MAPPINGS
from .src.LanPaint.nodes import NODE_DISPLAY_NAME_MAPPINGS
def _install_lightweight_runtime_stubs() -> None:
"""Install lightweight stubs so tooling can import this package without ComfyUI.
This is used by CI tooling (e.g., comfy-org/node-diff) that imports NODE_CLASS_MAPPINGS
in an environment where ComfyUI isn't installed.
"""
import sys
import types
# `src/LanPaint/nodes.py` uses `torch.Tensor` in type annotations.
try:
import torch # noqa: F401
except ModuleNotFoundError:
torch_mod = types.ModuleType("torch")
class Tensor: # noqa: N801 (match torch naming)
pass
torch_mod.Tensor = Tensor
torch_mod.nn = types.SimpleNamespace(functional=types.SimpleNamespace())
sys.modules["torch"] = torch_mod
if "comfyui_version" not in sys.modules:
comfyui_version_mod = types.ModuleType("comfyui_version")
comfyui_version_mod.__version__ = "0.0.0"
sys.modules["comfyui_version"] = comfyui_version_mod
sys.modules.setdefault("nodes", types.ModuleType("nodes"))
sys.modules.setdefault("latent_preview", types.ModuleType("latent_preview"))
if "comfy" not in sys.modules:
comfy_mod = types.ModuleType("comfy")
comfy_mod.__path__ = []
comfy_utils_mod = types.ModuleType("comfy.utils")
def repeat_to_batch_size(tensor, batch_size): # type: ignore[no-untyped-def]
if getattr(tensor, "shape", ())[0] == batch_size:
return tensor
return tensor
comfy_utils_mod.repeat_to_batch_size = repeat_to_batch_size
comfy_samplers_mod = types.ModuleType("comfy.samplers")
class DummyKSAMPLER: # noqa: N801 (match ComfyUI naming)
pass
comfy_samplers_mod.KSAMPLER = DummyKSAMPLER
comfy_model_base_mod = types.ModuleType("comfy.model_base")
class ModelType: # noqa: N801 (match ComfyUI naming)
FLUX = "FLUX"
FLOW = "FLOW"
class WAN22: # noqa: N801 (match ComfyUI naming)
pass
comfy_model_base_mod.ModelType = ModelType
comfy_model_base_mod.WAN22 = WAN22
comfy_mod.utils = comfy_utils_mod
comfy_mod.samplers = comfy_samplers_mod
comfy_mod.model_base = comfy_model_base_mod
sys.modules["comfy"] = comfy_mod
sys.modules["comfy.utils"] = comfy_utils_mod
sys.modules["comfy.samplers"] = comfy_samplers_mod
sys.modules["comfy.model_base"] = comfy_model_base_mod
try:
from .src.LanPaint.nodes import NODE_CLASS_MAPPINGS
from .src.LanPaint.nodes import NODE_DISPLAY_NAME_MAPPINGS
except ModuleNotFoundError:
_install_lightweight_runtime_stubs()
from .src.LanPaint.nodes import NODE_CLASS_MAPPINGS
from .src.LanPaint.nodes import NODE_DISPLAY_NAME_MAPPINGS
WEB_DIRECTORY = "./web"
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

File diff suppressed because it is too large Load Diff
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 MiB

+786
View File
@@ -0,0 +1,786 @@
{
"id": "978d3a45-3d13-43c6-8ef9-89dc3e74d6ba",
"revision": 0,
"last_node_id": 82,
"last_link_id": 220,
"nodes": [
{
"id": 66,
"type": "SetLatentNoiseMask",
"pos": [
514.1915893554688,
781.9396362304688
],
"size": [
264.5999755859375,
46
],
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "samples",
"type": "LATENT",
"link": 216
},
{
"name": "mask",
"type": "MASK",
"link": 189
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"slot_index": 0,
"links": [
186
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.23",
"Node name for S&R": "SetLatentNoiseMask"
},
"widgets_values": []
},
{
"id": 78,
"type": "CLIPTextEncode",
"pos": [
314.8565368652344,
255.63235473632812
],
"size": [
422.84503173828125,
164.31304931640625
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"label": "clip",
"name": "clip",
"type": "CLIP",
"link": 194
}
],
"outputs": [
{
"label": "CONDITIONING",
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
195
]
}
],
"title": "CLIP Text Encode (Positive Prompt)",
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.26",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"cute anime girl with massive fluffy fennec ears and a big fluffy tail blonde messy long hair blue eyes wearing a maid outfit with a long black gold leaf pattern dress and a white apron mouth open placing a fancy black forest cake with candles on top of a dinner table of an old dark Victorian mansion lit by candlelight with a bright window to the foggy forest and very expensive stuff everywhere there are paintings on the walls"
],
"color": "#232",
"bgcolor": "#353"
},
{
"id": 80,
"type": "CLIPTextEncode",
"pos": [
362.7684020996094,
481.0662536621094
],
"size": [
422.84503173828125,
164.31304931640625
],
"flags": {
"collapsed": true
},
"order": 3,
"mode": 0,
"inputs": [
{
"label": "clip",
"name": "clip",
"type": "CLIP",
"link": 196
}
],
"outputs": [
{
"label": "CONDITIONING",
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
210
]
}
],
"title": "CLIP Text Encode (Negative Prompt)",
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.26",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
""
],
"color": "#322",
"bgcolor": "#533"
},
{
"id": 79,
"type": "FluxGuidance",
"pos": [
529.1380615234375,
156.20236206054688
],
"size": [
211.60000610351562,
58
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"label": "conditioning",
"name": "conditioning",
"type": "CONDITIONING",
"link": 195
}
],
"outputs": [
{
"label": "CONDITIONING",
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
207
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.26",
"Node name for S&R": "FluxGuidance"
},
"widgets_values": [
3.5
]
},
{
"id": 77,
"type": "CheckpointLoaderSimple",
"pos": [
-66.25785064697266,
484.081787109375
],
"size": [
315,
98
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"label": "MODEL",
"name": "MODEL",
"type": "MODEL",
"slot_index": 0,
"links": [
197
]
},
{
"label": "CLIP",
"name": "CLIP",
"type": "CLIP",
"slot_index": 1,
"links": [
194,
196
]
},
{
"label": "VAE",
"name": "VAE",
"type": "VAE",
"slot_index": 2,
"links": [
200,
203
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.26",
"Node name for S&R": "CheckpointLoaderSimple"
},
"widgets_values": [
"flux1-dev-fp8.safetensors"
]
},
{
"id": 73,
"type": "LanPaint_KSampler",
"pos": [
1005.0399780273438,
296.9553527832031
],
"size": [
413.6495666503906,
596
],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 197
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 207
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 210
},
{
"name": "latent_image",
"type": "LATENT",
"link": 186
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"slot_index": 0,
"links": [
187
]
}
],
"properties": {
"cnr_id": "LanPaint",
"ver": "56bd6c04e89124cd06682b304245d6ddf8b20522",
"Node name for S&R": "LanPaint_KSampler"
},
"widgets_values": [
0,
"fixed",
30,
1,
"euler",
"simple",
1,
5,
"Image First",
"LanPaint KSampler. Recommend steps 50, LanPaint NumSteps 1-20 depending on the difficulty of task. LanPaint_EndSigma = 3.0 for anime style, 0.6 for realistic style. For more information, visit https://github.com/scraed/LanPaint",
"🖼️ Image Inpainting"
]
},
{
"id": 65,
"type": "VAEEncode",
"pos": [
200.1034698486328,
776.3561401367188
],
"size": [
210,
46
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "pixels",
"type": "IMAGE",
"link": 188
},
{
"name": "vae",
"type": "VAE",
"link": 203
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"slot_index": 0,
"links": [
216
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.23",
"Node name for S&R": "VAEEncode"
},
"widgets_values": []
},
{
"id": 48,
"type": "SaveImage",
"pos": [
1075.49169921875,
1167.2703857421875
],
"size": [
311.2532653808594,
484.7096252441406
],
"flags": {},
"order": 9,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 103
}
],
"outputs": [],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.23"
},
"widgets_values": [
"ComfyUI"
]
},
{
"id": 81,
"type": "LanPaint_MaskBlend",
"pos": [
1773.0189208984375,
1222.9478759765625
],
"size": [
210,
98
],
"flags": {},
"order": 10,
"mode": 0,
"inputs": [
{
"name": "image1",
"type": "IMAGE",
"link": 219
},
{
"name": "image2",
"type": "IMAGE",
"link": 218
},
{
"name": "mask",
"type": "MASK",
"link": 220
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
217
]
}
],
"properties": {
"cnr_id": "LanPaint",
"ver": "4d3d5d17f0105b673df92da5b084cce567c9c712",
"Node name for S&R": "LanPaint_MaskBlend"
},
"widgets_values": [
9
]
},
{
"id": 82,
"type": "SaveImage",
"pos": [
2106.45166015625,
1090.682861328125
],
"size": [
311.2532653808594,
484.7096252441406
],
"flags": {},
"order": 11,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 217
}
],
"outputs": [],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.23"
},
"widgets_values": [
"ComfyUI"
]
},
{
"id": 8,
"type": "VAEDecode",
"pos": [
1211.46484375,
1065.318359375
],
"size": [
210,
46
],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"name": "samples",
"type": "LATENT",
"link": 187
},
{
"name": "vae",
"type": "VAE",
"link": 200
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"slot_index": 0,
"links": [
103,
218
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.23",
"Node name for S&R": "VAEDecode"
},
"widgets_values": []
},
{
"id": 75,
"type": "LoadImage",
"pos": [
45.954593658447266,
1150.45556640625
],
"size": [
266.13720703125,
487.1314697265625
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
188,
219
]
},
{
"name": "MASK",
"type": "MASK",
"links": [
189,
220
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.27",
"Node name for S&R": "LoadImage"
},
"widgets_values": [
"clipspace/clipspace-mask-8865503.5.png [input]",
"image"
]
}
],
"links": [
[
103,
8,
0,
48,
0,
"IMAGE"
],
[
186,
66,
0,
73,
3,
"LATENT"
],
[
187,
73,
0,
8,
0,
"LATENT"
],
[
188,
75,
0,
65,
0,
"IMAGE"
],
[
189,
75,
1,
66,
1,
"MASK"
],
[
194,
77,
1,
78,
0,
"CLIP"
],
[
195,
78,
0,
79,
0,
"CONDITIONING"
],
[
196,
77,
1,
80,
0,
"CLIP"
],
[
197,
77,
0,
73,
0,
"MODEL"
],
[
200,
77,
2,
8,
1,
"VAE"
],
[
203,
77,
2,
65,
1,
"VAE"
],
[
207,
79,
0,
73,
1,
"CONDITIONING"
],
[
210,
80,
0,
73,
2,
"CONDITIONING"
],
[
216,
65,
0,
66,
0,
"LATENT"
],
[
217,
81,
0,
82,
0,
"IMAGE"
],
[
218,
8,
0,
81,
1,
"IMAGE"
],
[
219,
75,
0,
81,
0,
"IMAGE"
],
[
220,
75,
1,
81,
2,
"MASK"
]
],
"groups": [
{
"id": 1,
"title": "Mask image for inpainting.",
"bounding": [
36.04227828979492,
989.7313232421875,
278.89093017578125,
669.3414916992188
],
"color": "#3f789e",
"font_size": 24,
"flags": {}
},
{
"id": 2,
"title": "Convert Latents for LanPaint",
"bounding": [
286.0640563964844,
714.343505859375,
489.16796875,
197.81044006347656
],
"color": "#3f789e",
"font_size": 24,
"flags": {}
},
{
"id": 3,
"title": "Load Model and Set Prompts",
"bounding": [
-78.9311294555664,
176.08712768554688,
1006.1721801757812,
514.258544921875
],
"color": "#3f789e",
"font_size": 24,
"flags": {}
},
{
"id": 4,
"title": "Inpaint with the LanPaint KSampler",
"bounding": [
960.8922729492188,
179.17588806152344,
474.8909606933594,
630.4742431640625
],
"color": "#3f789e",
"font_size": 24,
"flags": {}
},
{
"id": 5,
"title": "LanPaint OutPut",
"bounding": [
1085.6029052734375,
994.0775756835938,
345.4561767578125,
669.4969482421875
],
"color": "#3f789e",
"font_size": 24,
"flags": {}
},
{
"id": 11,
"title": "LanPaint",
"bounding": [
-262.59381103515625,
140.46656799316406,
1737.328857421875,
797.4443359375
],
"color": "#3f789e",
"font_size": 24,
"flags": {}
}
],
"config": {},
"extra": {
"ds": {
"scale": 0.35049389948139237,
"offset": [
348.866804381099,
308.65057628971834
]
},
"frontendVersion": "1.27.10",
"node_versions": {
"comfy-core": "0.3.18",
"LanPaint": "0f509469ed2cd60c6032f739e282aad5dfc06166"
}
},
"version": 0.4
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 670 KiB

File diff suppressed because it is too large Load Diff
Binary file not shown.

After

Width:  |  Height:  |  Size: 674 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 674 KiB

File diff suppressed because it is too large Load Diff
Binary file not shown.

After

Width:  |  Height:  |  Size: 921 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 810 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

File diff suppressed because it is too large Load Diff
Binary file not shown.

After

Width:  |  Height:  |  Size: 683 KiB

File diff suppressed because it is too large Load Diff
Binary file not shown.

After

Width:  |  Height:  |  Size: 687 KiB

File diff suppressed because it is too large Load Diff
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 MiB

After

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 221 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 301 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 461 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.8 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 801 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 551 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.8 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.5 MiB

+4 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "LanPaint"
version = "1.3.1"
version = "1.4.10"
description = "Achieve seamless inpainting results without needing a specialized inpainting model."
authors = [
{name = "LanPaint", email = "czhengac@connect.ust.hk"}
@@ -75,5 +75,8 @@ select = [
# See all rules here: https://docs.astral.sh/ruff/rules/#pyflakes-f
]
[tool.ruff.lint.per-file-ignores]
"src/LanPaint/nodes.py" = ["F403", "F405"]
[tool.ruff.lint.flake8-quotes]
inline-quotes = "double"
+72 -21
View File
@@ -1,5 +1,5 @@
import torch
from .utils import *
from .utils import StochasticHarmonicOscillator
from functools import partial
class LanPaint():
@@ -13,6 +13,7 @@ class LanPaint():
self.friction = Friction
self.chara_beta = Beta
self.img_dim_size = None
def add_none_dims(self, array):
# Create a tuple with ':' for the first dimension and 'None' repeated num_nones times
index = (slice(None),) + (None,) * (self.img_dim_size-1)
@@ -25,19 +26,23 @@ class LanPaint():
self.img_dim_size = len(x.shape)
self.latent_image = latent_image
self.noise = noise
if torch.mean(torch.abs(self.noise)) < 1e-8:
self.noise = torch.randn_like(self.noise)
if n_steps is None:
n_steps = self.n_steps
return self.LanPaint(x, sigma, latent_mask, current_times, n_steps, model_options, seed, self.IS_FLUX, self.IS_FLOW)
def LanPaint(self, x, sigma, latent_mask, current_times, n_steps, model_options, seed, IS_FLUX, IS_FLOW):
VE_Sigma, abt, Flow_t = current_times
step_size = self.step_size * (1 - abt)
step_size = self.add_none_dims(step_size)
# self.inner_model.inner_model.scale_latent_inpaint returns variance exploding x_t values
# This is the replace step
x = x * (1 - latent_mask) + self.inner_model.inner_model.scale_latent_inpaint(x=x, sigma=sigma, noise=self.noise, latent_image=self.latent_image)* latent_mask
def scale_latent_inpaint(x, sigma, noise, latent_image):
return self.inner_model.inner_model.model_sampling.noise_scaling(sigma.reshape([sigma.shape[0]] + [1] * (len(noise.shape) - 1)), noise, latent_image)
x = x * (1 - latent_mask) + scale_latent_inpaint(x=x, sigma=sigma, noise=self.noise, latent_image=self.latent_image)* latent_mask
if IS_FLUX or IS_FLOW:
x_t = x * ( self.add_none_dims(abt)**0.5 + (1-self.add_none_dims(abt))**0.5 )
else:
@@ -60,7 +65,6 @@ class LanPaint():
return out
def score_model(self, x_t, y, mask, abt, sigma, tflow, model_options, seed):
lamb = self.chara_lamb
if self.IS_FLUX or self.IS_FLOW:
# compute t for flow model, with a small epsilon compensating for numerical error.
@@ -111,24 +115,71 @@ class LanPaint():
x_t = x_t.to(dtype)
v = v.to(dtype)
return x_t, v
if args is None:
#v = torch.zeros_like(x_t)
v = None
C = Coef_C(x_t)
#print(torch.squeeze(dtx), torch.squeeze(dty))
x_t, v = advance_time(x_t, v, dt, Gamma, A, C, D)
else:
v, C = args
x_t, v = advance_time(x_t, v, dt/2, Gamma, A, C, D)
def advance_time_overdamped(x_t, dt, A, C, D):
"""
Overdamped (Gamma -> infinity) limit:
dx = -A x dt + C dt + D dW_t
with C treated as constant over this substep.
"""
dtype = x_t.dtype
with torch.autocast(device_type=x_t.device.type, dtype=torch.float32):
A_dt = A * dt
exp_neg = torch.exp(-A_dt)
C_new = Coef_C(x_t)
v = v + Gamma**0.5 * ( C_new - C) *dt
eps = 1e-8
abs_A = torch.abs(A)
# k = (1 - exp(-A dt)) / A -> dt when A -> 0
k = torch.where(abs_A < eps, dt, (-torch.expm1(-A_dt)) / A)
# k2 = (1 - exp(-2 A dt)) / (2 A) -> dt when A -> 0
k2 = torch.where(abs_A < eps, dt, (-torch.expm1(-2 * A_dt)) / (2 * A))
x_t, v = advance_time(x_t, v, dt/2, Gamma, A, C, D)
mean = exp_neg * x_t + k * C
var = (D ** 2) * k2
noise = torch.randn_like(x_t) * torch.sqrt(torch.clamp(var, min=0.0))
x_t = mean + noise
return x_t.to(dtype)
def run_damped(x_t, args):
if args is None:
v = None
C = Coef_C(x_t)
x_t, v = advance_time(x_t, v, dt, Gamma, A, C, D)
else:
v, C = args
x_t, v = advance_time(x_t, v, dt/2, Gamma, A, C, D)
C_new = Coef_C(x_t)
v = v + Gamma**0.5 * ( C_new - C) *dt
x_t, v = advance_time(x_t, v, dt/2, Gamma, A, C, D)
C = C_new
return x_t, (v, C)
def run_overdamped(x_t, args):
if args is None:
C = Coef_C(x_t)
x_t = advance_time_overdamped(x_t, dt, A, C, D)
else:
_, C = args
x_t = advance_time_overdamped(x_t, dt / 2, A, C, D)
C_new = Coef_C(x_t)
x_t = x_t + (C_new - C) * dt
x_t = advance_time_overdamped(x_t, dt / 2, A, C, D)
C = C_new
return x_t, (None, C)
try:
x_t_next, (v_next, C_next) = run_damped(x_t, args)
if torch.isnan(x_t_next).any() or torch.isnan(v_next).any():
raise ValueError("NaN detected")
x_t = x_t_next
v = v_next
C = C_next
except Exception:
x_t, (v, C) = run_overdamped(x_t, args)
C = C_new
return x_t, (v, C)
def prepare_step_size(self, current_times, step_size, sigma_x, sigma_y):
@@ -140,7 +191,7 @@ class LanPaint():
# Compute time step (dtx, dty) for x and y branches.
dtx = 2 * step_size * sigma_x
dty = 2 * step_size * sigma_y
# -------------------------------------------------------------------------
# Define friction parameter Gamma_hat for each branch.
# Using dtx**0 provides a tensor of the proper device/dtype.
@@ -170,4 +221,4 @@ class LanPaint():
def x0_evalutation(self, x_t, score, sigma, args):
x0 = x_t + score(x_t)
return x0
return x0
+133 -70
View File
@@ -1,29 +1,82 @@
from contextlib import contextmanager
from inspect import cleandoc
import inspect
import math
# import nodes.py
import comfy
import nodes
import latent_preview
from functools import partial
import torch
from comfy.utils import repeat_to_batch_size
from comfy.samplers import *
from comfy.model_base import ModelType
from .utils import *
from .lanpaint import LanPaint
from comfy.model_base import WAN22
import comfyui_version
def reshape_mask(input_mask, output_shape):
def reshape_mask(input_mask, output_shape,video_inpainting=False):
dims = len(output_shape) - 2
print('output shape',output_shape)
scale_mode = "nearest-exact"
mask = torch.nn.functional.interpolate(input_mask, size=output_shape[-2:], mode=scale_mode)
if mask.shape[1] < output_shape[1]:
mask = mask.repeat((1, output_shape[1]) + (1,) * dims)[:,:output_shape[1]]
mask = repeat_to_batch_size(mask, output_shape[0])
print('input mask',input_mask.shape,type(input_mask),torch.max(input_mask),torch.min(input_mask))
print('target output_shape',output_shape)
print('input_mask.ndim:', input_mask.ndim, 'output_shape len:', len(output_shape))
# Handle input mask dimensions
if input_mask.ndim == 2:
input_mask = input_mask.unsqueeze(0).unsqueeze(0)
elif input_mask.ndim == 3:
input_mask = input_mask.unsqueeze(1)
# Handle 5D output shape (B, C, F, H, W) by ensuring input is 5D
if len(output_shape) == 5 and input_mask.ndim == 4:
if comfyui_version.__version__ >= "0.6.0":
input_mask = input_mask.unsqueeze(2) # (B, C, 1, H, W)
# Handle video case with temporal dimension
if video_inpainting: # Video case: (batch, channels, frames, height, width)
target_frames = output_shape[2]
target_height, target_width = output_shape[-2:]
print('Video case - input_mask initial shape:', input_mask.shape)
# First reshape input_mask to have proper dimensions for video processing
# Assume input is (frames, channels, height, width) -> (1, channels, frames, height, width)
## if comfy version < 0.6.0
if comfyui_version.__version__ < "0.6.0":
input_mask = input_mask.permute(1, 0, 2, 3).unsqueeze(0)
print('Video case - input_mask after reshaping:', input_mask.shape)
# Ensure we have the correct 5D shape: (batch, channels, frames, height, width)
batch_size, channels, frames, height, width = input_mask.shape
print('Video case - dimensions: batch_size={}, channels={}, frames={}, height={}, width={}'.format(batch_size, channels, frames, height, width))
print('Video case - target size:', (target_frames, target_height, target_width))
# 3D nearest-exact interpolation: (batch, channels, frames, height, width) -> (batch, channels, target_frames, target_height, target_width)
temp_mask = torch.nn.functional.interpolate(
input_mask,
size=(target_frames, target_height, target_width),
mode=scale_mode,
)
# temp_mask is already 5D: (batch, channels, target_frames, target_height, target_width)
mask = temp_mask
print('after mask',mask.shape)
# Handle channel dimension expansion if needed
if mask.shape[1] < output_shape[1]:
mask = mask.repeat(1, output_shape[1], 1, 1, 1)[:, :output_shape[1]]
# Handle batch dimension
mask = repeat_to_batch_size(mask, output_shape[0])
else: # Original 2D image case
if comfyui_version.__version__ < "0.6.0":
mask = torch.nn.functional.interpolate(input_mask, size=output_shape[-2:], mode=scale_mode)
else:
mask = torch.nn.functional.interpolate(input_mask, size=output_shape[2:], mode=scale_mode)
if mask.shape[1] < output_shape[1]:
mask = mask.repeat((1, output_shape[1]) + (1,) * dims)[:,:output_shape[1]]
mask = repeat_to_batch_size(mask, output_shape[0])
return mask
def prepare_mask(noise_mask, shape, device):
return reshape_mask(noise_mask, shape).to(device)
def prepare_mask(noise_mask, shape, device,video_inpainting=False):
return reshape_mask(noise_mask, shape,video_inpainting).to(device)
def sampling_function_LanPaint(model, x, timestep, uncond, cond, cond_scale, cond_scale_BIG, model_options={}, seed=None):
if math.isclose(cond_scale, 1.0) and model_options.get("disable_cfg1_optimization", False) == False:
uncond_ = None
@@ -42,13 +95,18 @@ def sampling_function_LanPaint(model, x, timestep, uncond, cond, cond_scale, con
class CFGGuider_LanPaint:
def outer_sample(self, noise, latent_image, sampler, sigmas, denoise_mask=None, callback=None, disable_pbar=False, seed=None):
def outer_sample(self, noise, latent_image, sampler, sigmas, denoise_mask=None, callback=None, disable_pbar=False, seed=None, **kwargs):
print("CFGGuider outer_sample")
self.inner_model, self.conds, self.loaded_models = comfy.sampler_helpers.prepare_sampling(self.model_patcher, noise.shape, self.conds, self.model_options)
device = self.model_patcher.load_device
if isinstance(self.inner_model, WAN22):
print("WAN22 detected")
self.inner_model.extra_conds = super(WAN22, self.inner_model).extra_conds
if denoise_mask is not None:
denoise_mask = prepare_mask(denoise_mask, noise.shape, device)
video_inpainting = self.model_options.get("video_inpainting", False)
denoise_mask = prepare_mask(denoise_mask, noise.shape, device, video_inpainting)
noise = noise.to(device)
latent_image = latent_image.to(device)
@@ -57,7 +115,7 @@ class CFGGuider_LanPaint:
try:
self.model_patcher.pre_run()
output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed, **kwargs)
finally:
self.model_patcher.cleanup()
@@ -86,14 +144,16 @@ class KSamplerX0Inpaint:
IS_FLUX = self.inner_model.inner_model.model_type == ModelType.FLUX
IS_FLOW = self.inner_model.inner_model.model_type == ModelType.FLOW
#print("model class", type(self.inner_model.inner_model))
#print("model type", self.inner_model.inner_model.model_type, "IS_FLUX", IS_FLUX, "IS_FLOW", IS_FLOW)
#print("sigma", torch.mean(sigma).item(), torch.min(sigma).item(), torch.max(sigma).item())
# unify the notations into variance exploding diffusion model
if IS_FLUX or IS_FLOW:
Flow_t = sigma
abt = (1 - Flow_t)**2 / ((1 - Flow_t)**2 + Flow_t**2 )
VE_Sigma = Flow_t / (1 - Flow_t)
#print("t", torch.mean( sigma ).item(), "VE_Sigma", torch.mean( VE_Sigma ).item())
else:
VE_Sigma = sigma
@@ -118,7 +178,7 @@ class KSamplerX0Inpaint:
out = self.PaintMethod(x, self.latent_image, self.noise, sigma, latent_mask, current_times, model_options, seed)
else:
out, _ = self.inner_model(x, sigma, model_options=model_options, seed=seed)
# Add TAESD preview support - directly use the latent_preview module
current_step = model_options.get("i", kwargs.get("i", 0))
total_steps = model_options.get("total_steps", 0)
@@ -129,7 +189,7 @@ class KSamplerX0Inpaint:
callback = model_options.get("callback", None)
if callback is not None:
callback({"i": current_step, "denoised": out, "x": x})
return out
# Custom sampler class extending ComfyUI's KSAMPLER for LanPaint
@@ -245,6 +305,7 @@ class LanPaint_KSampler():
"LanPaint_NumSteps": ("INT", {"default": 5, "min": 0, "max": 100, "tooltip": "The number of steps for the Langevin dynamics, representing the turns of thinking per step."}),
"LanPaint_PromptMode": (["Image First", "Prompt First"], {"tooltip": "Image First: emphasis image quality, Prompt First: emphasis prompt following"}),
"LanPaint_Info": ("STRING", {"default": "LanPaint KSampler. For more info, visit https://github.com/scraed/LanPaint. If you find it useful, please give a star ⭐️!", "multiline": True}),
"Inpainting_mode": (["🖼️ Image Inpainting", "🎬 Video Inpainting"], {"default": "🖼️ Image Inpainting", "tooltip": "Choose Image mode for photos or Video mode for video frames with temporal consistency"}),
}
}
@@ -255,9 +316,9 @@ class LanPaint_KSampler():
CATEGORY = "sampling"
DESCRIPTION = "Uses the provided model, positive and negative conditioning to denoise the latent image."
def sample(self, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=1.0, LanPaint_NumSteps=5, LanPaint_PromptMode = "Image First", LanPaint_Info=""):
def sample(self, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=1.0, LanPaint_NumSteps=5, LanPaint_PromptMode="Image First", LanPaint_Info="",Inpainting_mode="🖼️ Image Inpainting"):
model.LanPaint_StepSize = 0.15
model.LanPaint_StepSize = 0.2
model.LanPaint_Lambda = 16.0
model.LanPaint_Beta = 1.
model.LanPaint_NumSteps = LanPaint_NumSteps
@@ -267,6 +328,13 @@ class LanPaint_KSampler():
model.LanPaint_cfg_BIG = cfg
else:
model.LanPaint_cfg_BIG = 0*cfg - 0.5
# Convert inpainting_mode to boolean for video_inpainting
video_inpainting = (Inpainting_mode == "🎬 Video Inpainting")
if not hasattr(model, 'model_options') or model.model_options is None:
model.model_options = {}
model.model_options["video_inpainting"] = video_inpainting
with override_sample_function():
return nodes.common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
class LanPaint_KSamplerAdvanced:
@@ -288,12 +356,13 @@ class LanPaint_KSamplerAdvanced:
"return_with_leftover_noise": (["disable", "enable"], ),
"LanPaint_NumSteps": ("INT", {"default": 5, "min": 0, "max": 100, "tooltip": "The number of steps for the Langevin dynamics, representing the turns of thinking per step."}),
"LanPaint_Lambda": ("FLOAT", {"default": 16., "min": 0.1, "max": 50.0, "step": 0.1, "round": 0.1, "tooltip": "The bidirectional guidance scale. Higher values align with known regions more closely, but may result in instability."}),
"LanPaint_StepSize": ("FLOAT", {"default": 0.15, "min": 0.0001, "max": 1., "step": 0.01, "round": 0.001, "tooltip": "The step size for the Langevin dynamics. Higher values result in faster convergence but may be unstable."}),
"LanPaint_StepSize": ("FLOAT", {"default": 0.2, "min": 0.0001, "max": 1., "step": 0.01, "round": 0.001, "tooltip": "The step size for the Langevin dynamics. Higher values result in faster convergence but may be unstable."}),
"LanPaint_Beta": ("FLOAT", {"default": 1., "min": 0.0001, "max": 5, "step": 0.1, "round": 0.1, "tooltip": "The step size ratio between masked / unmasked regions. Lower value can compensate high values of LanPaint_Lambda."}),
"LanPaint_Friction": ("FLOAT", {"default": 15, "min": 0., "max": 50.0, "step": 0.1, "round": 0.1, "tooltip": "The friction parameter for fast langevin, lower values result in faster convergence but may be unstable."}),
"LanPaint_PromptMode": (["Image First", "Prompt First"], {"tooltip": "Image First: emphasis image quality, Prompt First: emphasis prompt following"}),
"LanPaint_EarlyStop": ("INT", {"default": 1, "min": 0, "max": 10000, "tooltip": "The number of steps to stop the LanPaint early, useful for preventing the image from irregular patterns."}),
"LanPaint_Info": ("STRING", {"default": "LanPaint KSampler Adv. For more info, visit https://github.com/scraed/LanPaint. If you find it useful, please give a star ⭐️!", "multiline": True}),
"Inpainting_mode": (["🖼️ Image Inpainting", "🎬 Video Inpainting"], {"default": "🖼️ Image Inpainting", "tooltip": "Choose Image mode for photos or Video mode for video frames with temporal consistency"}),
},
}
@@ -302,7 +371,7 @@ class LanPaint_KSamplerAdvanced:
CATEGORY = "sampling"
def sample(self, model, add_noise, noise_seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, start_at_step, end_at_step, return_with_leftover_noise, denoise=1.0, LanPaint_StepSize=0.05, LanPaint_Lambda=5, LanPaint_Beta=1, LanPaint_NumSteps=5, LanPaint_Friction=5, LanPaint_PromptMode = "Image First", LanPaint_EarlyStop = 1, LanPaint_Info=""):
def sample(self, model, add_noise, noise_seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, start_at_step, end_at_step, return_with_leftover_noise, LanPaint_NumSteps=5, LanPaint_Lambda=16.0, LanPaint_StepSize=0.2, LanPaint_Beta=1.0, LanPaint_Friction=15.0, LanPaint_PromptMode="Image First", LanPaint_EarlyStop=1, LanPaint_Info="", Inpainting_mode="🖼️ Image Inpainting"):
force_full_denoise = True
if return_with_leftover_noise == "enable":
force_full_denoise = False
@@ -320,8 +389,14 @@ class LanPaint_KSamplerAdvanced:
else:
model.LanPaint_cfg_BIG = 0*cfg - 0.5
# Convert inpainting_mode to boolean for video_inpainting
video_inpainting = (Inpainting_mode == "🎬 Video Inpainting")
if not hasattr(model, 'model_options') or model.model_options is None:
model.model_options = {}
model.model_options["video_inpainting"] = video_inpainting
with override_sample_function():
return nodes.common_ksampler(model, noise_seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise, disable_noise=disable_noise, start_step=start_at_step, last_step=end_at_step, force_full_denoise=force_full_denoise)
return nodes.common_ksampler(model, noise_seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=1.0, disable_noise=disable_noise, start_step=start_at_step, last_step=end_at_step, force_full_denoise=force_full_denoise)
class MaskBlend:
@@ -349,15 +424,23 @@ class MaskBlend:
# for each mask pixel, find out the nearest 1 pixel, and set the mask value to the distance between the two pixels
# check the size of mask and image1, image2, if not the same, assert error
if image1.shape[1] != image2.shape[1] or image1.shape[2] != image2.shape[2]:
raise ValueError("Make sure your image size is a multiple of 8. Otherwise the mask will not be aligned with the output image.")
raise ValueError(
"Image size mismatch: Image1 and Image2 must have the same dimensions.\n"
"Additionally, ensure both images have width and height that are multiples of 8.\n"
"This is required because VAE decode always generates images with dimensions that are multiples of 8.\n"
"If your input images are not multiples of 8, a size mismatch will occur during the decoding process.\n"
"Please resize your images using an image resize node to ensure compatibility.\n"
"Current sizes - Image1: {}x{}, Image2: {}x{}".format(
image1.shape[2], image1.shape[1], image2.shape[2], image2.shape[1]
)
)
mask = mask.float()
mask = torch.nn.functional.max_pool2d(mask, kernel_size=blend_overlap, stride=1, padding=blend_overlap//2)
# apply Gaussian blur with kernel size blend_overlap
kernel = self.gaussian_kernel(blend_overlap)
kernel = kernel.to(image1.device)
kernel = kernel[None, None, ...]
mask = torch.nn.functional.conv2d(mask[:,None,:,:], kernel, padding=blend_overlap//2)[:,0,:,:]
@@ -416,7 +499,7 @@ class LanPaint_SamplerCustom:
CATEGORY = "sampling/custom_sampling"
def sample(self, model, sampler, sigmas, add_noise, noise_seed, cfg, positive, negative, latent_image, LanPaint_NumSteps, LanPaint_PromptMode, LanPaint_Info=""):
model.LanPaint_StepSize = 0.15
model.LanPaint_StepSize = 0.2
model.LanPaint_Lambda = 16.0
model.LanPaint_Beta = 1.
model.LanPaint_NumSteps = LanPaint_NumSteps
@@ -455,22 +538,19 @@ class LanPaint_SamplerCustom:
else:
out_denoised = out
return (out, out_denoised)
class LanPaint_SamplerCustomAdvanced:
@classmethod
def INPUT_TYPES(s):
return {"required":
{"noise": ("NOISE",),
"guider": ("GUIDER",),
"sampler": ("SAMPLER",),
"sigmas": ("SIGMAS",),
"latent_image": ("LATENT",),
"start_at_step": ("INT", {"default": 0, "min": 0, "max": 10000}),
"end_at_step": ("INT", {"default": 10000, "min": 0, "max": 10000}),
"return_with_leftover_noise": (["disable", "enable"], ),
"guider": ("GUIDER", ),
"sampler": ("SAMPLER", ),
"sigmas": ("SIGMAS", ),
"latent_image": ("LATENT", ),
"LanPaint_NumSteps": ("INT", {"default": 5, "min": 0, "max": 100, "tooltip": "Number of steps for Langevin dynamics, representing turns of thinking per step."}),
"LanPaint_Lambda": ("FLOAT", {"default": 16.0, "min": 0.1, "max": 50.0, "step": 0.1, "tooltip": "Bidirectional guidance scale. Higher values align with known regions but may cause instability."}),
"LanPaint_StepSize": ("FLOAT", {"default": 0.15, "min": 0.0001, "max": 1.0, "step": 0.01, "tooltip": "Step size for Langevin dynamics. Higher values speed convergence but may be unstable."}),
"LanPaint_StepSize": ("FLOAT", {"default": 0.2, "min": 0.0001, "max": 1.0, "step": 0.01, "tooltip": "Step size for Langevin dynamics. Higher values speed convergence but may be unstable."}),
"LanPaint_Beta": ("FLOAT", {"default": 1.0, "min": 0.0001, "max": 5.0, "step": 0.1, "tooltip": "Step size ratio between masked/unmasked regions. Lower values balance high Lambda."}),
"LanPaint_Friction": ("FLOAT", {"default": 15.0, "min": 0.0, "max": 50.0, "step": 0.1, "tooltip": "Friction parameter for fast Langevin. Lower values speed convergence but may be unstable."}),
"LanPaint_PromptMode": (["Image First", "Prompt First"], {"tooltip": "Image First: prioritizes image quality; Prompt First: prioritizes prompt adherence."}),
@@ -479,17 +559,14 @@ class LanPaint_SamplerCustomAdvanced:
}
}
RETURN_TYPES = ("LATENT", "LATENT")
RETURN_TYPES = ("LATENT","LATENT")
RETURN_NAMES = ("output", "denoised_output")
FUNCTION = "sample"
CATEGORY = "sampling/custom_sampling"
def sample(self, noise, guider, sampler, sigmas, latent_image, start_at_step, end_at_step, return_with_leftover_noise, LanPaint_NumSteps, LanPaint_Lambda, LanPaint_StepSize, LanPaint_Beta, LanPaint_Friction, LanPaint_PromptMode, LanPaint_EarlyStop, LanPaint_Info=""):
force_full_denoise = True
if end_at_step <= start_at_step:
raise ValueError('end_at_step must be larger than start_at_step')
if return_with_leftover_noise == "enable":
force_full_denoise = False
def sample(self, noise, guider, sampler, sigmas, latent_image, LanPaint_NumSteps, LanPaint_Lambda, LanPaint_StepSize, LanPaint_Beta, LanPaint_Friction, LanPaint_PromptMode, LanPaint_EarlyStop, LanPaint_Info=""):
model = guider.model_patcher
model.LanPaint_StepSize = LanPaint_StepSize
model.LanPaint_Lambda = LanPaint_Lambda
@@ -502,47 +579,33 @@ class LanPaint_SamplerCustomAdvanced:
else:
model.LanPaint_cfg_BIG = 0 * guider.cfg - 0.5
with override_sample_function():
latent = latent_image.copy()
latent_image_samples = latent["samples"]
latent_image_samples = comfy.sample.fix_empty_latent_channels(model, latent_image_samples)
latent["samples"] = latent_image_samples
latent = latent_image
latent_image = latent["samples"]
latent = latent.copy()
latent_image = comfy.sample.fix_empty_latent_channels(guider.model_patcher, latent_image)
latent["samples"] = latent_image
noise_mask = None
if "noise_mask" in latent:
noise_mask = latent["noise_mask"]
# From base comfy samplers.py
if end_at_step is not None and end_at_step < (len(sigmas) - 1):
sigmas = sigmas[:end_at_step + 1]
if force_full_denoise:
sigmas[-1] = 0
if start_at_step is not None:
if start_at_step < (len(sigmas) - 1):
sigmas = sigmas[start_at_step:]
else:
if latent_image is not None:
return latent_image
else:
return torch.zeros_like(noise)
x0_output = {}
callback = latent_preview.prepare_callback(model, sigmas.shape[-1] - 1, x0_output)
callback = latent_preview.prepare_callback(guider.model_patcher, sigmas.shape[-1] - 1, x0_output)
disable_pbar = not comfy.utils.PROGRESS_BAR_ENABLED
samples = guider.sample( noise.generate_noise(latent), latent_image_samples, sampler, sigmas, denoise_mask=noise_mask, callback=callback,disable_pbar=disable_pbar, seed=noise.seed
)
samples = guider.sample(noise.generate_noise(latent), latent_image, sampler, sigmas, denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=noise.seed)
samples = samples.to(comfy.model_management.intermediate_device())
out = latent.copy()
out["samples"] = samples
if "x0" in x0_output:
out_denoised = latent.copy()
out_denoised["samples"] = model.model.process_latent_out(x0_output["x0"].cpu())
out_denoised["samples"] = guider.model_patcher.model.process_latent_out(x0_output["x0"].cpu())
else:
out_denoised = out
return (out, out_denoised)
# A dictionary that contains all nodes you want to export with their names
# NOTE: names should be globally unique
NODE_CLASS_MAPPINGS = {
+20 -21
View File
@@ -28,11 +28,11 @@ def expm1mxmhx2_x3(x):
def exp_1mcosh_GD(gamma_t, delta):
"""
Compute e^(-Γt) * (1 - cosh(Γt√Δ))/ ( (Γt)**2 Δ )
Parameters:
gamma_t: Γ*t term (could be a scalar or tensor)
delta: Δ term (could be a scalar or tensor)
Returns:
Result of the computation with numerical stability handling
"""
@@ -68,7 +68,6 @@ def exp_sinh_GsqrtD(gamma_t, delta):
sqrt_abs_delta = torch.sqrt(torch.abs(delta))
gamma_t_sqrt_delta = gamma_t * sqrt_abs_delta
numerator_pos = (torch.exp(gamma_t * (sqrt_abs_delta - 1)) - torch.exp(gamma_t * (-sqrt_abs_delta - 1))) / 2
denominator_pos = gamma_t_sqrt_delta
result_pos = numerator_pos / gamma_t_sqrt_delta
result_pos = torch.where(torch.isfinite(result_pos), result_pos, torch.zeros_like(result_pos))
@@ -117,15 +116,15 @@ def zeta1(gamma_t, delta):
exp_cosh_term = exp_cosh(half_gamma_t, delta)
exp_sinh_term = exp_sinh_sqrtD(half_gamma_t, delta)
# Main computation
numerator = 1 - (exp_cosh_term + exp_sinh_term)
denominator = gamma_t * (1 - delta) / 4
result = 1 - numerator / denominator
# Handle numerical instability
result = torch.where(torch.isfinite(result), result, torch.zeros_like(result))
# Taylor expansion for small x (similar to your epxm1Dx approach)
mask = torch.abs(denominator) < 5e-3
term1 = epxm1_x(-gamma_t)
@@ -133,17 +132,17 @@ def zeta1(gamma_t, delta):
term3 = expm1mxmhx2_x3(-gamma_t)
taylor = term1 + (1/2.+ term1-3*term2)*denominator + (-1/6. + term1/2 - 4 * term2 + 10 * term3) * denominator**2
result = torch.where(mask, taylor, result)
return result
def exp_cosh_minus_terms(gamma_t, delta):
"""
Compute E^(-tΓ) * (Cosh[tΓ] - 1 - (Cosh[tΓ√Δ] - 1)/Δ) / (tΓ(1 - Δ))
Parameters:
gamma_t: Γ*t term (could be a scalar or tensor)
delta: Δ term (could be a scalar or tensor)
Returns:
Result of the computation with numerical stability handling
"""
@@ -151,17 +150,17 @@ def exp_cosh_minus_terms(gamma_t, delta):
# Compute individual terms
exp_cosh_term = exp_cosh(gamma_t, gamma_t**0) - exp_term # E^(-tΓ) (Cosh[tΓ] - 1) term
exp_cosh_delta_term = - gamma_t**2 * exp_1mcosh_GD(gamma_t, delta) # E^(-tΓ) (Cosh[tΓ√Δ] - 1)/Δ term
#exp_1mcosh_GD e^(-Γt) * (1 - cosh(Γt√Δ))/ ( (Γt)**2 Δ )
# Main computation
numerator = exp_cosh_term - exp_cosh_delta_term
denominator = gamma_t * (1 - delta)
result = numerator / denominator
# Handle numerical instability
result = torch.where(torch.isfinite(result), result, torch.zeros_like(result))
# Taylor expansion for small gamma_t and delta near 1
mask = (torch.abs(denominator) < 1e-1)
exp_1mcosh_GD_term = exp_1mcosh_GD(gamma_t, delta**0)
@@ -170,7 +169,7 @@ def exp_cosh_minus_terms(gamma_t, delta):
- denominator / 4 * ( 0.5 * exp_cosh(gamma_t, delta**0) - 4 * exp_1mcosh_GD_term - 5 /2 * exp_sinh_GsqrtD(gamma_t, delta**0) )
)
result = torch.where(mask, taylor, result)
return result
@@ -185,7 +184,7 @@ def sig11(gamma_t, delta):
def Zcoefs(gamma_t, delta):
Zeta1 = zeta1(gamma_t, delta)
Zeta2 = zeta2(gamma_t, delta)
sq_total = 1 - Zeta1 + gamma_t * (delta - 1) * (Zeta1 - 1)**2 / 8
amplitude = torch.sqrt(sq_total)
Zcoef1 = ( gamma_t**0.5 * Zeta2 / 2 **0.5 ) / amplitude
@@ -208,7 +207,7 @@ class StochasticHarmonicOscillator:
dq(t) = -Γ A y(t) dt + Γ C dt + Γ D dw(t) - Γ q(t) dt
Also define v(t) = q(t) / √Γ, which is numerically more stable.
Where:
y(t) - Position variable
q(t) - Velocity variable
@@ -239,7 +238,7 @@ class StochasticHarmonicOscillator:
Returns:
tuple: (y(t), v(t))
"""
dummyzero = y0.new_zeros(1) # convert scalar to tensor with same device and dtype as y0
Delta = self.Delta + dummyzero
Gamma_hat = self.Gamma * t + dummyzero
@@ -254,12 +253,12 @@ class StochasticHarmonicOscillator:
if v0 is None:
v0 = torch.randn_like(y0) * D / 2 ** 0.5
#v0 = (C - A * y0)/Gamma**0.5
# Calculate mean position and velocity
term1 = (1 - zeta_1) * (C * t - A * t * y0) + zeta_2 * (Gamma ** 0.5) * v0 * t
y_mean = term1 + y0
v_mean = (1 - EE)*(C - A * y0) / (Gamma ** 0.5) + (EE - A * t * (1 - zeta_1)) * v0
cov_yy = D**2 * t * self.sig22(Gamma_hat, Delta)
cov_vv = D**2 * self.sig11(Gamma_hat, Delta) / 2
cov_yv = (zeta2(Gamma_hat, Delta) * Gamma_hat * D ) **2 / 2 / (Gamma ** 0.5)
@@ -274,7 +273,7 @@ class StochasticHarmonicOscillator:
cov_matrix[..., 1, 1] = cov_vv
# Compute the Cholesky decomposition to get scale_tril
#scale_tril = torch.linalg.cholesky(cov_matrix)
scale_tril = torch.zeros(*batch_shape, 2, 2, device=y0.device, dtype=y0.dtype)
@@ -298,4 +297,4 @@ class StochasticHarmonicOscillator:
scale_tril=scale_tril
).sample()
return new_yv[...,0], new_yv[...,1]
return new_yv[...,0], new_yv[...,1]
+4 -3
View File
@@ -1,4 +1,5 @@
[pytest]
testpaths = . # Run tests in the current directory
python_files = test_*.py # Run tests in files that start with "test_"
norecursedirs = .. # Don't run tests in the parent directory
# Keep settings value-only; pytest does not treat inline `# ...` as comments.
testpaths = .
python_files = test_*.py
norecursedirs = ..
+9 -17
View File
@@ -1,21 +1,13 @@
#!/usr/bin/env python
"""Basic import tests for LanPaint.
"""Tests for `LanPaint` package."""
The ComfyUI runtime dependencies (e.g. `comfy`) are intentionally optional for unit tests.
"""
import pytest
from src.LanPaint.nodes import Example
@pytest.fixture
def example_node():
"""Fixture to create an Example node instance."""
return Example()
def test_package_imports_without_comfy() -> None:
import LanPaint
def test_example_node_initialization(example_node):
"""Test that the node can be instantiated."""
assert isinstance(example_node, Example)
def test_return_types():
"""Test the node's metadata."""
assert Example.RETURN_TYPES == ("IMAGE",)
assert Example.FUNCTION == "test"
assert Example.CATEGORY == "Example"
assert isinstance(LanPaint.NODE_CLASS_MAPPINGS, dict)
assert isinstance(LanPaint.NODE_DISPLAY_NAME_MAPPINGS, dict)
assert "LanPaint_KSampler" in LanPaint.NODE_CLASS_MAPPINGS
assert LanPaint.WEB_DIRECTORY == "./web"
+74
View File
@@ -0,0 +1,74 @@
import importlib
import sys
import types
import pytest
import torch
def _repeat_to_batch_size(tensor: torch.Tensor, batch_size: int) -> torch.Tensor:
if tensor.shape[0] == batch_size:
return tensor
if tensor.shape[0] == 1:
return tensor.repeat((batch_size,) + (1,) * (tensor.ndim - 1))
repeats = (batch_size + tensor.shape[0] - 1) // tensor.shape[0]
return tensor.repeat((repeats,) + (1,) * (tensor.ndim - 1))[:batch_size]
def _import_nodes(monkeypatch, comfyui_version: str):
comfy_mod = types.ModuleType("comfy")
comfy_mod.__path__ = []
comfy_utils_mod = types.ModuleType("comfy.utils")
comfy_utils_mod.repeat_to_batch_size = _repeat_to_batch_size
comfy_samplers_mod = types.ModuleType("comfy.samplers")
class DummyKSAMPLER: ...
comfy_samplers_mod.KSAMPLER = DummyKSAMPLER
comfy_model_base_mod = types.ModuleType("comfy.model_base")
class ModelType:
FLUX = "FLUX"
FLOW = "FLOW"
class WAN22: ...
comfy_model_base_mod.ModelType = ModelType
comfy_model_base_mod.WAN22 = WAN22
comfyui_version_mod = types.ModuleType("comfyui_version")
comfyui_version_mod.__version__ = comfyui_version
comfy_mod.utils = comfy_utils_mod
comfy_mod.samplers = comfy_samplers_mod
comfy_mod.model_base = comfy_model_base_mod
monkeypatch.setitem(sys.modules, "comfy", comfy_mod)
monkeypatch.setitem(sys.modules, "comfy.utils", comfy_utils_mod)
monkeypatch.setitem(sys.modules, "comfy.samplers", comfy_samplers_mod)
monkeypatch.setitem(sys.modules, "comfy.model_base", comfy_model_base_mod)
monkeypatch.setitem(sys.modules, "nodes", types.ModuleType("nodes"))
monkeypatch.setitem(sys.modules, "latent_preview", types.ModuleType("latent_preview"))
monkeypatch.setitem(sys.modules, "comfyui_version", comfyui_version_mod)
sys.modules.pop("src.LanPaint.nodes", None)
return importlib.import_module("src.LanPaint.nodes")
@pytest.mark.parametrize("comfyui_version", ["0.5.0", "0.6.0"])
def test_reshape_mask_accepts_bhw_and_5d_output_shape(monkeypatch, comfyui_version: str) -> None:
lanpaint_nodes = _import_nodes(monkeypatch, comfyui_version)
input_mask = torch.zeros((1, 4, 4))
output_shape = (1, 16, 1, 8, 8)
out = lanpaint_nodes.reshape_mask(input_mask, output_shape, video_inpainting=False)
assert tuple(out.shape) == output_shape
def test_prepare_mask_accepts_hw_and_moves_device(monkeypatch) -> None:
lanpaint_nodes = _import_nodes(monkeypatch, "0.5.0")
input_mask = torch.zeros((4, 4))
output_shape = (2, 3, 8, 8)
out = lanpaint_nodes.prepare_mask(input_mask, output_shape, device=torch.device("cpu"), video_inpainting=False)
assert tuple(out.shape) == output_shape
assert out.device.type == "cpu"
+51
View File
@@ -0,0 +1,51 @@
import torch
from unittest.mock import MagicMock, patch
from src.LanPaint.lanpaint import LanPaint
def test_langevin_dynamics_fallback_on_nan() -> None:
"""Test that langevin_dynamics falls back to overdamped dynamics if damped dynamics produces NaNs."""
torch.manual_seed(0)
# Setup minimal LanPaint instance
lp = LanPaint(Model=MagicMock(), NSteps=10, Friction=1.0, Lambda=1.0, Beta=1.0, StepSize=0.1)
# Dummy inputs
# Shape: (Batch, Channel, Height, Width)
x_t = torch.randn(1, 4, 8, 8)
lp.img_dim_size = 4
mask = torch.zeros_like(x_t)
# Simple score function
def score(x):
return torch.zeros_like(x)
step_size = torch.tensor([0.1])
# (sigma, abt, flow_t)
current_times = (torch.tensor([0.5]), torch.tensor([0.5]), torch.tensor([0.5]))
# Mock StochasticHarmonicOscillator to return NaNs
# We patch it where it is used (imported) in lanpaint.py
with patch("src.LanPaint.lanpaint.StochasticHarmonicOscillator") as MockSHO:
mock_instance = MockSHO.return_value
# Configure dynamics to return NaNs
nan_tensor = torch.full_like(x_t, float('nan'))
mock_instance.dynamics.return_value = (nan_tensor, nan_tensor)
# Execute langevin_dynamics
# This should try run_damped -> get NaNs -> raise ValueError -> catch -> run_overdamped
x_out, (v_out, C_out) = lp.langevin_dynamics(x_t, score, mask, step_size, current_times, sigma_y=1.0)
# Verify that SHO was initialized and dynamics called
MockSHO.assert_called()
mock_instance.dynamics.assert_called()
# Verify result is finite (indicating fallback to overdamped logic was successful)
assert torch.isfinite(x_out).all(), "Output contains NaNs, fallback failed"
assert v_out is None or torch.isfinite(v_out).all()
def test_langevin_dynamics_fallback_on_exception() -> None:
"""Test that langevin_dynamics falls back to overdamped dynamics if damped dynamics raises Exception."""
torch.manual_seed(0)
lp = LanPaint(Model=MagicMock(), NSteps=10, Friction=1.0, Lambda=1.0, Beta=1.0, StepSize=0.1)
x_t = torch.randn(1, 4, 8, 8)
lp.img_dim_size = 4
mask = torch.zeros_like(x_t)
score = lambda x: torch.zeros_like(x)
step_size = torch.tensor([0.1])
current_times = (torch.tensor([0.5]), torch.tensor([0.5]), torch.tensor([0.5]))
with patch("src.LanPaint.lanpaint.StochasticHarmonicOscillator") as MockSHO:
mock_instance = MockSHO.return_value
# Configure dynamics to raise Exception
mock_instance.dynamics.side_effect = RuntimeError("Simulation exploded")
x_out, (v_out, C_out) = lp.langevin_dynamics(x_t, score, mask, step_size, current_times, sigma_y=1.0)
assert torch.isfinite(x_out).all()