Krea 2 (RAW + Turbo) integration + VAE-Utils vendored pack + modal polish
- Krea 2 RAW + Turbo: model configs, prompts, and templates (t2i/i2i, highdetail, turbo-t2i-3d-regional) — 3D-poser depth control bridges into Krea 2 via SDXL i2i - Bundled ComfyUI-VAE-Utils pack: tiled detail-VAE decode + 2x latent-upscale weights - Upscale/inpaint engines: native qwen_image_vae for krea2 sectional passes, euler the krea2 default sampler, real per-engine Steps/CFG defaults, CN-less tile denoise caps, krea2 resolution-shift sync - Model-settings + install-section wiring for the new models and bundled pack
This commit is contained in:
@@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -0,0 +1,11 @@
|
||||
from .nodes import ConditioningKrea2Rebalance
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"ConditioningKrea2Rebalance": ConditioningKrea2Rebalance,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"ConditioningKrea2Rebalance": "Conditioning Krea2 Rebalance",
|
||||
}
|
||||
|
||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
|
||||
@@ -0,0 +1,85 @@
|
||||
import torch
|
||||
|
||||
|
||||
def _scale_cond_tensor(t: torch.Tensor, multiplier, per_layer_weights=None):
|
||||
"""Scale a conditioning tensor, optionally with per-layer weighting.
|
||||
|
||||
Krea2 conditioning arrives as (B, seq, 12*2560) — the 12 Qwen3-VL taps flattened
|
||||
into the feature dim. When per_layer_weights is given we reshape to
|
||||
(B, seq, 12, D), apply a different gain to each tap and flatten back.
|
||||
"""
|
||||
if per_layer_weights is None:
|
||||
return t * multiplier
|
||||
|
||||
flat = t.shape[-1]
|
||||
n_layers = len(per_layer_weights)
|
||||
if n_layers > 1 and flat % n_layers == 0:
|
||||
layer_dim = flat // n_layers
|
||||
orig_dtype = t.dtype
|
||||
t = t.float()
|
||||
t = t.view(*t.shape[:-1], n_layers, layer_dim)
|
||||
gains = torch.tensor(per_layer_weights, dtype=t.dtype, device=t.device)
|
||||
t = t * gains.view(*([1] * (t.dim() - 2)), n_layers, 1)
|
||||
t = t.view(*t.shape[:-2], flat)
|
||||
return t.to(orig_dtype) * multiplier
|
||||
return t * multiplier
|
||||
|
||||
|
||||
def _parse_per_layer(s: str):
|
||||
"""Parse a comma-separated list of floats. Returns None if empty/invalid."""
|
||||
if not s:
|
||||
return None
|
||||
s = s.strip()
|
||||
if not s:
|
||||
return None
|
||||
try:
|
||||
vals = [float(x) for x in s.replace(";", ",").split(",") if x.strip() != ""]
|
||||
except ValueError:
|
||||
return None
|
||||
if len(vals) < 2:
|
||||
return None
|
||||
return vals
|
||||
|
||||
|
||||
def scale_conditioning(structure, multiplier, per_layer_weights=None):
|
||||
"""leaving masks / pooled output intact."""
|
||||
if isinstance(structure, list):
|
||||
out = []
|
||||
for item in structure:
|
||||
if isinstance(item, (list, tuple)) and len(item) == 2 \
|
||||
and isinstance(item[0], torch.Tensor) and isinstance(item[1], dict):
|
||||
cond_t, extras = item
|
||||
new_cond = _scale_cond_tensor(cond_t, multiplier, per_layer_weights)
|
||||
out.append([new_cond, dict(extras)])
|
||||
else:
|
||||
out.append(scale_conditioning(item, multiplier, per_layer_weights))
|
||||
return out
|
||||
if isinstance(structure, torch.Tensor):
|
||||
return _scale_cond_tensor(structure, multiplier, per_layer_weights)
|
||||
if isinstance(structure, dict):
|
||||
return {k: scale_conditioning(v, multiplier, per_layer_weights)
|
||||
for k, v in structure.items()}
|
||||
return structure
|
||||
|
||||
|
||||
class ConditioningKrea2Rebalance:
|
||||
|
||||
DEFAULT_WEIGHTS = "1.0,1.0,1.0,1.0,1.0,1.0,1.0,2.5,5.0,1.1,4.0,1.0"
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {"required": {
|
||||
"conditioning": ("CONDITIONING",),
|
||||
"multiplier": ("FLOAT", {"default": 4.0, "min": -1000000000.0, "max": 1000000000.0, "step": 0.01}),
|
||||
"per_layer_weights": ("STRING", {"default": cls.DEFAULT_WEIGHTS, "multiline": False}),
|
||||
}}
|
||||
|
||||
RETURN_TYPES = ("CONDITIONING",)
|
||||
RETURN_NAMES = ("conditioning",)
|
||||
FUNCTION = "main"
|
||||
CATEGORY = "conditioning"
|
||||
|
||||
def main(self, conditioning, multiplier, per_layer_weights=None):
|
||||
plw = _parse_per_layer(per_layer_weights) if per_layer_weights else None
|
||||
c = scale_conditioning(conditioning, multiplier, per_layer_weights=plw)
|
||||
return (c,)
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 spacepxl
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,9 @@
|
||||
from .nodes import COMBINED_MAPPINGS
|
||||
|
||||
NODE_CLASS_MAPPINGS = {}
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {}
|
||||
for k, v in COMBINED_MAPPINGS.items():
|
||||
NODE_CLASS_MAPPINGS[k] = v[0]
|
||||
NODE_DISPLAY_NAME_MAPPINGS[k] = v[1]
|
||||
|
||||
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']
|
||||
@@ -0,0 +1,56 @@
|
||||
import os
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
from safetensors.torch import load_file
|
||||
|
||||
|
||||
class CausalLatentProjector(nn.Module):
|
||||
def __init__(
|
||||
self,
|
||||
in_channels = 16,
|
||||
out_channels = 3,
|
||||
scale_t = 4,
|
||||
scale_s = 8,
|
||||
num_projections = 3,
|
||||
):
|
||||
super().__init__()
|
||||
self.scale_t = scale_t
|
||||
self.scale_s = scale_s
|
||||
|
||||
self.projections = nn.ModuleList([
|
||||
nn.Linear(in_channels, out_channels * scale_s ** 2)
|
||||
])
|
||||
|
||||
for _ in range(num_projections - 1):
|
||||
self.projections.append(
|
||||
nn.Linear(in_channels, out_channels * scale_t * scale_s ** 2)
|
||||
)
|
||||
|
||||
def forward(self, latents):
|
||||
batch, channels, frames, height, width = latents.shape
|
||||
|
||||
pixels = []
|
||||
for frame in range(frames):
|
||||
idx = min(frame, len(self.projections) - 1)
|
||||
|
||||
feat = self.projections[idx](
|
||||
latents[:, :, frame].movedim(1, -1) # BHWC
|
||||
).movedim(-1, 1) # BCHW
|
||||
|
||||
feat = F.pixel_shuffle(feat, self.scale_s).unsqueeze(2) # BCFHW
|
||||
|
||||
if frame == 0:
|
||||
pixels.append(feat)
|
||||
else:
|
||||
feat = feat.chunk(self.scale_t, dim=1)
|
||||
pixels.extend(list(feat))
|
||||
|
||||
return torch.cat(pixels, dim=2)
|
||||
|
||||
|
||||
def Wan21_latent_projector():
|
||||
model = CausalLatentProjector()
|
||||
model_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "wan21_latent_projector.safetensors")
|
||||
model.load_state_dict(load_file(model_path))
|
||||
return model
|
||||
@@ -0,0 +1,155 @@
|
||||
import os
|
||||
import math
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
from safetensors.torch import load_file
|
||||
|
||||
|
||||
class LayerNorm3d(nn.LayerNorm):
|
||||
def __init__(self, num_channels, eps=1e-6, affine=True):
|
||||
super().__init__(num_channels, eps=eps, elementwise_affine=affine)
|
||||
|
||||
def forward(self, x: torch.Tensor) -> torch.Tensor:
|
||||
x = x.permute(0, 2, 3, 4, 1)
|
||||
x = F.layer_norm(x, self.normalized_shape, self.weight, self.bias, self.eps)
|
||||
x = x.permute(0, 4, 1, 2, 3)
|
||||
return x
|
||||
|
||||
|
||||
class DiCoBlock3d(nn.Module):
|
||||
def __init__(self, hidden_size, mlp_ratio=4.0, kernel_size=3):
|
||||
super().__init__()
|
||||
|
||||
# self.conv1 = nn.Conv3d(hidden_size, hidden_size, kernel_size=1)
|
||||
|
||||
self.conv2 = nn.Conv3d(
|
||||
hidden_size,
|
||||
hidden_size,
|
||||
kernel_size=(1, kernel_size, kernel_size),
|
||||
padding=(0, kernel_size//2, kernel_size//2),
|
||||
# groups=hidden_size,
|
||||
padding_mode="replicate",
|
||||
)
|
||||
|
||||
self.conv3 = nn.Conv3d(hidden_size, hidden_size, kernel_size=1)
|
||||
|
||||
self.cca = nn.Sequential(
|
||||
nn.AdaptiveAvgPool3d(1),
|
||||
nn.Conv3d(hidden_size, hidden_size , kernel_size=1),
|
||||
nn.Sigmoid(),
|
||||
)
|
||||
|
||||
ffn_channel = int(mlp_ratio * hidden_size)
|
||||
self.conv4 = nn.Conv3d(hidden_size, ffn_channel, kernel_size=1)
|
||||
self.conv5 = nn.Conv3d(ffn_channel, hidden_size, kernel_size=1)
|
||||
|
||||
self.norm1 = LayerNorm3d(hidden_size, affine=False)
|
||||
self.norm2 = LayerNorm3d(hidden_size, affine=False)
|
||||
|
||||
def forward(self, inp):
|
||||
x = self.norm1(inp)
|
||||
# x = F.gelu(self.conv2(self.conv1(x)))
|
||||
x = F.gelu(self.conv2(x))
|
||||
x = self.conv3(x * self.cca(x))
|
||||
x = inp + x
|
||||
|
||||
y = self.norm2(x)
|
||||
y = self.conv5(F.gelu(self.conv4(y)))
|
||||
return x + y
|
||||
|
||||
|
||||
class LatentModel3d(nn.Module):
|
||||
def __init__(
|
||||
self,
|
||||
in_channels = 16,
|
||||
out_channels = 16,
|
||||
hidden_size = 128,
|
||||
kernel_size = 3,
|
||||
mlp_ratio = 4.0,
|
||||
depth = 8,
|
||||
upscale = 2,
|
||||
):
|
||||
super().__init__()
|
||||
self.upscale = int(upscale)
|
||||
|
||||
self.conv_in = nn.Conv3d(
|
||||
in_channels,
|
||||
hidden_size,
|
||||
kernel_size=(1, 3, 3),
|
||||
padding=(0, 1, 1),
|
||||
padding_mode="replicate",
|
||||
)
|
||||
|
||||
self.blocks = nn.ModuleList([
|
||||
DiCoBlock3d(hidden_size, mlp_ratio, kernel_size)
|
||||
for _ in range(depth)
|
||||
])
|
||||
|
||||
out_ch = out_channels * self.upscale ** 2
|
||||
self.conv_out = nn.Conv3d(
|
||||
hidden_size,
|
||||
out_ch,
|
||||
kernel_size=(1, 3, 3),
|
||||
padding=(0, 1, 1),
|
||||
padding_mode="replicate",
|
||||
)
|
||||
|
||||
# self.initialize_weights()
|
||||
|
||||
def initialize_weights(self):
|
||||
def _basic_init(module):
|
||||
if isinstance(module, nn.Linear) or isinstance(module, nn.Conv3d):
|
||||
torch.nn.init.xavier_uniform_(module.weight)
|
||||
if module.bias is not None:
|
||||
nn.init.constant_(module.bias, 0)
|
||||
self.apply(_basic_init)
|
||||
|
||||
def forward(self, x):
|
||||
"""
|
||||
x: (B, C, F, H, W) tensor of spatial inputs (images or latent representations of images)
|
||||
"""
|
||||
hidden = self.conv_in(x)
|
||||
|
||||
for block in self.blocks:
|
||||
hidden = block(hidden)
|
||||
|
||||
output = self.conv_out(hidden)
|
||||
|
||||
if self.upscale > 1:
|
||||
output = F.pixel_shuffle(output.movedim(1, 2), self.upscale).movedim(2, 1)
|
||||
x = F.interpolate(x, scale_factor=(1, self.upscale, self.upscale), mode="nearest-exact")
|
||||
return output + x
|
||||
|
||||
|
||||
def Wan21_latent_upscale_2x():
|
||||
model = LatentModel3d(
|
||||
in_channels = 16,
|
||||
out_channels = 16,
|
||||
hidden_size = 128,
|
||||
kernel_size = 3,
|
||||
mlp_ratio = 4.0,
|
||||
depth = 8,
|
||||
upscale = 2,
|
||||
)
|
||||
model_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "wan21_latent_upscale_2x.safetensors")
|
||||
model.load_state_dict(load_file(model_path))
|
||||
return model
|
||||
|
||||
|
||||
latent_upscale_models = {
|
||||
"Wan 2.1 latent upscale 2x": Wan21_latent_upscale_2x,
|
||||
}
|
||||
|
||||
|
||||
if __name__=="__main__":
|
||||
model = LatentModel3d(upscale=2).to("cuda")
|
||||
|
||||
x = torch.randn(1, 16, 4, 64, 64).to("cuda")
|
||||
out = model(x)
|
||||
print(out.shape)
|
||||
|
||||
total = sum([p.numel() for p in model.parameters() if p.requires_grad])
|
||||
print("trainable parameters: %.2f M" % (total / 1e6))
|
||||
|
||||
print(model)
|
||||
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,450 @@
|
||||
import torch
|
||||
import torch.nn.functional as F
|
||||
import copy
|
||||
import math
|
||||
from tqdm.auto import tqdm
|
||||
|
||||
import comfy.utils
|
||||
import comfy.model_management
|
||||
import comfy.latent_formats
|
||||
import folder_paths
|
||||
from nodes import VAELoader
|
||||
from .src.sd import CustomVAE
|
||||
from .latent_upscale.model import latent_upscale_models
|
||||
from .latent_upscale.latent_projector import Wan21_latent_projector
|
||||
|
||||
|
||||
class VAEUtils_CustomVAELoader(VAELoader):
|
||||
@staticmethod
|
||||
def vae_list():
|
||||
return folder_paths.get_filename_list("vae")
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"vae_name": (s.vae_list(), ),
|
||||
"disable_offload": ("BOOLEAN", {"default": True}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("VAE",)
|
||||
FUNCTION = "load_vae"
|
||||
CATEGORY = "VAE-Utils"
|
||||
|
||||
def load_vae(self, vae_name, disable_offload):
|
||||
if vae_name == "pixel_space":
|
||||
sd = {}
|
||||
sd["pixel_space_vae"] = torch.tensor(1.0)
|
||||
elif vae_name in ["taesd", "taesdxl", "taesd3", "taef1"]:
|
||||
sd = self.load_taesd(vae_name)
|
||||
else:
|
||||
vae_path = folder_paths.get_full_path_or_raise("vae", vae_name)
|
||||
sd = comfy.utils.load_torch_file(vae_path)
|
||||
vae = CustomVAE(sd=sd)
|
||||
vae.throw_exception_if_invalid()
|
||||
vae.disable_offload = disable_offload
|
||||
return (vae, )
|
||||
|
||||
|
||||
class VAEUtils_DisableVAEOffload:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"vae": ("VAE", ),
|
||||
"disable_offload": ("BOOLEAN", {"default": True}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("VAE",)
|
||||
FUNCTION = "set_offload"
|
||||
CATEGORY = "VAE-Utils"
|
||||
|
||||
def set_offload(self, vae, disable_offload):
|
||||
vae = copy.copy(vae)
|
||||
vae.disable_offload = disable_offload
|
||||
return (vae, )
|
||||
|
||||
|
||||
class VAEUtils_VAEDecodeTiled:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"samples": ("LATENT", ),
|
||||
"vae": ("VAE", ),
|
||||
"upscale": ("INT", {"default": -1, "min": -1, "tooltip": "Post upscale factor, -1=auto"}),
|
||||
"tile": ("BOOLEAN", {"default": False}),
|
||||
"tile_size": ("INT", {"default": 512, "min": 64, "max": 4096, "step": 32}),
|
||||
"overlap": ("INT", {"default": 64, "min": 0, "max": 4096, "step": 32}),
|
||||
"temporal_size": ("INT", {"default": 4096, "min": 8, "max": 4096, "step": 4, "tooltip": "Only used for video VAEs: Amount of frames to decode at a time."}),
|
||||
"temporal_overlap": ("INT", {"default": 64, "min": 4, "max": 4096, "step": 4, "tooltip": "Only used for video VAEs: Amount of frames to overlap."}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("IMAGE",)
|
||||
FUNCTION = "decode"
|
||||
CATEGORY = "VAE-Utils"
|
||||
|
||||
def decode(self, samples, vae, upscale, tile, tile_size, overlap, temporal_size, temporal_overlap):
|
||||
if tile_size < overlap * 4:
|
||||
overlap = tile_size // 4
|
||||
if temporal_size < temporal_overlap * 2:
|
||||
temporal_overlap = temporal_overlap // 2
|
||||
temporal_compression = vae.temporal_compression_decode()
|
||||
if temporal_compression is not None:
|
||||
temporal_size = max(2, temporal_size // temporal_compression)
|
||||
temporal_overlap = max(1, min(temporal_size // 2, temporal_overlap // temporal_compression))
|
||||
else:
|
||||
temporal_size = None
|
||||
temporal_overlap = None
|
||||
|
||||
compression = vae.spacial_compression_decode()
|
||||
|
||||
if tile:
|
||||
images = vae.decode_tiled(samples["samples"], tile_x=tile_size // compression, tile_y=tile_size // compression, overlap=overlap // compression, tile_t=temporal_size, overlap_t=temporal_overlap)
|
||||
else:
|
||||
images = vae.decode(samples["samples"])
|
||||
|
||||
if len(images.shape) == 5: #Combine batches
|
||||
images = images.reshape(-1, images.shape[-3], images.shape[-2], images.shape[-1])
|
||||
|
||||
# Safety guard for ComfyUI PRs #11405 / #11406 regression:
|
||||
# If process_output was not applied through the inherited VAE.decode() path the raw
|
||||
# decoder output lands in [-1, 1]. Values below 0 are clamped to black by save/preview
|
||||
# nodes, producing the dark/burned look. Detect and correct here.
|
||||
# This guard is safe on correctly-decoded images because they are always in [0, 1]
|
||||
# so min >= 0 and the branch never fires.
|
||||
if images.min() < -0.1:
|
||||
images = torch.clamp((images.float() + 1.0) / 2.0, min=0.0, max=1.0)
|
||||
|
||||
if upscale < 1:
|
||||
ch = images.shape[-1]
|
||||
if ch == 3:
|
||||
upscale = 1
|
||||
else:
|
||||
if ch % 3 == 0:
|
||||
upscale = round((ch // 3) ** 0.5)
|
||||
else:
|
||||
raise Exception("Couldn't determine upscale factor, try setting the value manually instead")
|
||||
|
||||
if upscale > 1:
|
||||
images = F.pixel_shuffle(images.movedim(-1, 1), upscale_factor=int(upscale)).movedim(1, -1)
|
||||
|
||||
return (images, )
|
||||
|
||||
|
||||
class VAEUtils_LatentUpscale:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"samples": ("LATENT", ),
|
||||
"model": (list(latent_upscale_models.keys()), ),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("LATENT",)
|
||||
FUNCTION = "upscale"
|
||||
CATEGORY = "VAE-Utils"
|
||||
|
||||
def upscale(self, samples, model):
|
||||
device = comfy.model_management.get_torch_device()
|
||||
model = latent_upscale_models[model]().to(device)
|
||||
|
||||
latents = samples["samples"].to(dtype=torch.float32, device=device)
|
||||
upscaled_latents = model(latents).to(comfy.model_management.intermediate_device())
|
||||
|
||||
samples = copy.deepcopy(samples)
|
||||
samples["samples"] = upscaled_latents
|
||||
|
||||
return (samples, )
|
||||
|
||||
|
||||
class VAEUtils_WanLatentPreview:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"samples": ("LATENT", ),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("IMAGE",)
|
||||
FUNCTION = "upscale"
|
||||
CATEGORY = "VAE-Utils"
|
||||
|
||||
def upscale(self, samples):
|
||||
device = comfy.model_management.intermediate_device()
|
||||
projector = Wan21_latent_projector().to(device)
|
||||
|
||||
latents = samples["samples"].to(dtype=torch.float32, device=device)
|
||||
pixels = projector(latents).to(comfy.model_management.intermediate_device())
|
||||
pixels = pixels * 0.5 + 0.5
|
||||
|
||||
f, h, w = pixels.shape[-3:]
|
||||
pixels = F.interpolate(pixels, size=(f, h//8, w//8), mode="area")
|
||||
|
||||
pixels = [b.movedim(0, -1) for b in pixels] # CFHW -> FHWC
|
||||
pixels = torch.cat(pixels, dim=0) # (BF)HWC
|
||||
return (pixels, )
|
||||
|
||||
|
||||
def get_tiles(length, tile_size, min_overlap):
|
||||
if length <= tile_size:
|
||||
return [(0, length)]
|
||||
|
||||
max_step = tile_size - min_overlap
|
||||
total_shiftable = length - tile_size
|
||||
|
||||
gaps_needed = math.ceil(total_shiftable / max_step) if total_shiftable > 0 else 0
|
||||
num_tiles = gaps_needed + 1
|
||||
|
||||
if num_tiles == 1:
|
||||
raise Exception("this shouldn't happen")
|
||||
|
||||
gap_base = total_shiftable // (num_tiles - 1)
|
||||
remainder = total_shiftable % (num_tiles - 1)
|
||||
|
||||
starts = []
|
||||
acc = 0
|
||||
for i in range(num_tiles):
|
||||
starts.append(acc)
|
||||
if i < num_tiles - 1:
|
||||
acc += gap_base + (1 if i < remainder else 0)
|
||||
|
||||
slices = [(s, s + tile_size) for s in starts]
|
||||
return slices
|
||||
|
||||
|
||||
def get_1d_mask(idx, tiles, drop_first=0):
|
||||
tile_start, tile_end = tiles[idx]
|
||||
mask = torch.ones(tile_end - tile_start)
|
||||
|
||||
if idx > 0:
|
||||
prev_end = tiles[idx - 1][1]
|
||||
size = prev_end - tile_start - drop_first
|
||||
ramp = (torch.arange(size) + 1) / size
|
||||
mask[drop_first : size + drop_first] *= ramp
|
||||
mask[:drop_first] *= 0
|
||||
|
||||
if idx < (len(tiles) - 1):
|
||||
next_start = tiles[idx + 1][0]
|
||||
size = tile_end - next_start
|
||||
ramp = (torch.flip(torch.arange(size), [0]) + 1) / size
|
||||
mask[-size:] *= ramp
|
||||
|
||||
return mask
|
||||
|
||||
|
||||
class VAEUtils_TileModelPatch:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"model": ("MODEL", ),
|
||||
"tile_t": ("INT", {"default": 13, "min": 3, "step": 1}),
|
||||
"tile_h": ("INT", {"default": 78, "min": 32, "step": 2}),
|
||||
"tile_w": ("INT", {"default": 78, "min": 32, "step": 2}),
|
||||
"min_overlap_t": ("INT", {"default": 5, "min": 1, "step": 1}),
|
||||
"min_overlap_h": ("INT", {"default": 16, "min": 0, "step": 2}),
|
||||
"min_overlap_w": ("INT", {"default": 16, "min": 0, "step": 2}),
|
||||
"drop_first_t": ("INT", {"default": 2, "min": 0, "step": 1}),
|
||||
"patch_memory_estimate": ("BOOLEAN", {"default": True}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("MODEL",)
|
||||
FUNCTION = "patch"
|
||||
CATEGORY = "VAE-Utils"
|
||||
|
||||
def patch(self, model, tile_t, tile_h, tile_w, min_overlap_t, min_overlap_h, min_overlap_w, drop_first_t, patch_memory_estimate):
|
||||
model = model.clone()
|
||||
|
||||
def model_function_tile_wrapper(apply_model, args):
|
||||
# print(args["c"]["transformer_options"]["patches"]["noise_refiner"][0].encoded_image.shape)
|
||||
|
||||
# collect inputs
|
||||
x = args["input"]
|
||||
t = args["timestep"]
|
||||
c = args["c"]
|
||||
c_concat = c.get("c_concat", None)
|
||||
vace_context = c.get("vace_context", None)
|
||||
# transformer_options = c.get("transformer_options", {})
|
||||
|
||||
if c_concat is not None:
|
||||
assert c_concat.shape[2:] == x.shape[2:], f"c_concat shape {c_concat.shape} doesn't match x shape {x.shape}"
|
||||
|
||||
t_tiles = get_tiles(x.shape[-3], tile_t, min_overlap_t) if x.ndim == 5 else None
|
||||
h_tiles = get_tiles(x.shape[-2], tile_h, min_overlap_h)
|
||||
w_tiles = get_tiles(x.shape[-1], tile_w, min_overlap_w)
|
||||
|
||||
total_tiles = len(h_tiles) * len(w_tiles)
|
||||
message = f"{len(h_tiles)}h {len(w_tiles)}w"
|
||||
|
||||
if t_tiles is not None:
|
||||
total_tiles *= len(t_tiles)
|
||||
message = f"{len(t_tiles)}t " + message
|
||||
|
||||
x_out = torch.zeros_like(x)
|
||||
x_alpha = torch.zeros_like(x)
|
||||
progress_bar = tqdm(range(0, total_tiles), desc=message)
|
||||
|
||||
for w_idx, (w_start, w_end) in enumerate(w_tiles):
|
||||
for h_idx, (h_start, h_end) in enumerate(h_tiles):
|
||||
if t_tiles is not None:
|
||||
for t_idx, (t_start, t_end) in enumerate(t_tiles):
|
||||
# slice inputs (todo: any other conditions? controlnets etc)
|
||||
x_tile = x[..., t_start:t_end, h_start:h_end, w_start:w_end]
|
||||
c_tile = c.copy()
|
||||
|
||||
if c_concat is not None:
|
||||
c_tile["c_concat"] = c_concat[..., t_start:t_end, h_start:h_end, w_start:w_end]
|
||||
|
||||
if vace_context is not None:
|
||||
c_tile["vace_context"] = vace_context[..., t_start:t_end, h_start:h_end, w_start:w_end]
|
||||
|
||||
tile_out = apply_model(x_tile, t, **c_tile)
|
||||
|
||||
# prepare mask for edge blending
|
||||
mask_t = get_1d_mask(t_idx, t_tiles, drop_first_t).view(1, 1, -1, 1, 1)
|
||||
mask_h = get_1d_mask(h_idx, h_tiles).view(1, 1, 1, -1, 1)
|
||||
mask_w = get_1d_mask(w_idx, w_tiles).view(1, 1, 1, 1, -1)
|
||||
|
||||
mask = torch.ones_like(tile_out)
|
||||
mask *= mask_t.to(mask)
|
||||
mask *= mask_h.to(mask)
|
||||
mask *= mask_w.to(mask)
|
||||
|
||||
# accumulate tile into full image
|
||||
x_out[:, :, t_start:t_end, h_start:h_end, w_start:w_end] += tile_out * mask
|
||||
x_alpha[:, :, t_start:t_end, h_start:h_end, w_start:w_end] += mask
|
||||
progress_bar.update(1)
|
||||
|
||||
else:
|
||||
x_tile = x[..., h_start:h_end, w_start:w_end]
|
||||
c_tile = c.copy()
|
||||
|
||||
if c_concat is not None:
|
||||
c_tile["c_concat"] = c_concat[..., h_start:h_end, w_start:w_end]
|
||||
|
||||
tile_out = apply_model(x_tile, t, **c_tile)
|
||||
|
||||
mask_h = get_1d_mask(h_idx, h_tiles).view(1, 1, -1, 1)
|
||||
mask_w = get_1d_mask(w_idx, w_tiles).view(1, 1, 1, -1)
|
||||
|
||||
mask = torch.ones_like(tile_out)
|
||||
mask *= mask_h.to(mask)
|
||||
mask *= mask_w.to(mask)
|
||||
|
||||
x_out[:, :, h_start:h_end, w_start:w_end] += tile_out * mask
|
||||
x_alpha[:, :, h_start:h_end, w_start:w_end] += mask
|
||||
progress_bar.update(1)
|
||||
|
||||
assert x_alpha.min() > 0, "missing tile coverage"
|
||||
x_out = x_out / x_alpha
|
||||
return x_out
|
||||
|
||||
model.set_model_unet_function_wrapper(model_function_tile_wrapper)
|
||||
|
||||
def update_input_shape(input_shape):
|
||||
new_input_shape = input_shape.copy()
|
||||
if len(new_input_shape) == 5:
|
||||
new_input_shape = new_input_shape[:2] + [tile_t, tile_h, tile_w]
|
||||
elif len(new_input_shape) == 4:
|
||||
new_input_shape = new_input_shape[:2] + [tile_h, tile_w]
|
||||
return new_input_shape
|
||||
|
||||
original_memory_required = model.model.memory_required
|
||||
|
||||
def memory_required_wrapper(input_shape, cond_shapes={}):
|
||||
new_input_shape = update_input_shape(input_shape)
|
||||
|
||||
for c in model.model.memory_usage_factor_conds:
|
||||
shape = cond_shapes.get(c, None)
|
||||
if shape is not None:
|
||||
new_shape = update_input_shape(shape)
|
||||
cond_shapes[c] = new_shape
|
||||
|
||||
return original_memory_required(new_input_shape, cond_shapes)
|
||||
|
||||
if patch_memory_estimate:
|
||||
model.model.memory_required = memory_required_wrapper
|
||||
|
||||
return (model, )
|
||||
|
||||
|
||||
class VAEUtils_VisualizeTiles:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"length": ("INT", {"default": 21, "min": 1, "step": 1}),
|
||||
"tile_size": ("INT", {"default": 13, "min": 1, "step": 1}),
|
||||
"min_overlap": ("INT", {"default": 5, "min": 0, "step": 1}),
|
||||
"drop_first": ("INT", {"default": 2, "min": 0, "step": 1}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("IMAGE",)
|
||||
FUNCTION = "visualize"
|
||||
CATEGORY = "VAE-Utils"
|
||||
|
||||
def visualize(self, length, tile_size, min_overlap, drop_first):
|
||||
tile_slices = get_tiles(length, tile_size, min_overlap)
|
||||
|
||||
image = torch.zeros(1, 3, len(tile_slices), length)
|
||||
|
||||
for idx, (start, end) in enumerate(tile_slices):
|
||||
print(idx, start, end)
|
||||
mask = get_1d_mask(idx, tile_slices, drop_first)
|
||||
image[:, :, idx, start:end] = mask
|
||||
|
||||
image = F.interpolate(image, scale_factor=8, mode="nearest-exact")
|
||||
image = image.movedim(1, -1)
|
||||
|
||||
return (image, )
|
||||
|
||||
|
||||
latent_formats = {name: obj for name, obj in vars(comfy.latent_formats).items() if isinstance(obj, type)}
|
||||
del latent_formats["LatentFormat"]
|
||||
|
||||
|
||||
class VAEUtils_ScaleLatents:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"latents": ("LATENT", ),
|
||||
"direction": (["scale", "unscale"], ),
|
||||
"latent_type": (list(latent_formats.keys()), ),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("LATENT",)
|
||||
FUNCTION = "scale"
|
||||
CATEGORY = "VAE-Utils"
|
||||
|
||||
def scale(self, latents, direction, latent_type):
|
||||
latents = copy.deepcopy(latents)
|
||||
latent_format = latent_formats[latent_type]()
|
||||
|
||||
if direction == "scale":
|
||||
latents["samples"] = latent_format.process_in(latents["samples"])
|
||||
else:
|
||||
latents["samples"] = latent_format.process_out(latents["samples"])
|
||||
|
||||
return (latents, )
|
||||
|
||||
|
||||
COMBINED_MAPPINGS = {
|
||||
"VAEUtils_CustomVAELoader": (VAEUtils_CustomVAELoader, "Load VAE (VAE Utils)"),
|
||||
"VAEUtils_DisableVAEOffload": (VAEUtils_DisableVAEOffload, "Disable VAE Offload (VAE Utils)"),
|
||||
"VAEUtils_VAEDecodeTiled": (VAEUtils_VAEDecodeTiled, "VAE Decode (VAE Utils)"),
|
||||
"VAEUtils_LatentUpscale": (VAEUtils_LatentUpscale, "Latent Upscale (VAE Utils)"),
|
||||
"VAEUtils_WanLatentPreview": (VAEUtils_WanLatentPreview, "Wan Latent Preview (VAE Utils)"),
|
||||
"VAEUtils_TileModelPatch": (VAEUtils_TileModelPatch, "Tile Model Patch (VAE Utils)"),
|
||||
"VAEUtils_VisualizeTiles": (VAEUtils_VisualizeTiles, "Visualize Tiles (VAE Utils)"),
|
||||
"VAEUtils_ScaleLatents": (VAEUtils_ScaleLatents, "Scale/Unscale Latents (VAE Utils)"),
|
||||
}
|
||||
@@ -0,0 +1,420 @@
|
||||
import math
|
||||
import json
|
||||
import torch
|
||||
import logging
|
||||
|
||||
from comfy import model_management
|
||||
from comfy.ldm.models.autoencoder import AutoencoderKL, AutoencodingEngine
|
||||
# from comfy.ldm.cascade.stage_a import StageA
|
||||
# from comfy.ldm.cascade.stage_c_coder import StageC_coder
|
||||
# from comfy.ldm.audio.autoencoder import AudioOobleckVAE
|
||||
# import comfy.ldm.genmo.vae.model
|
||||
# import comfy.ldm.lightricks.vae.causal_video_autoencoder
|
||||
# import comfy.ldm.cosmos.vae
|
||||
from .wan.vae import WanVAE
|
||||
import comfy.ldm.wan.vae2_2
|
||||
# import comfy.ldm.hunyuan3d.vae
|
||||
# import comfy.ldm.ace.vae.music_dcae_pipeline
|
||||
# import comfy.ldm.hunyuan_video.vae
|
||||
# import comfy.ldm.mmaudio.vae.autoencoder
|
||||
# import comfy.pixel_space_convert
|
||||
|
||||
import comfy.utils
|
||||
|
||||
import comfy.model_patcher
|
||||
import comfy.taesd.taesd
|
||||
|
||||
from comfy.sd import VAE
|
||||
|
||||
class CustomVAE(VAE):
|
||||
def __init__(self, sd=None, device=None, config=None, dtype=None, metadata=None):
|
||||
if model_management.is_amd():
|
||||
VAE_KL_MEM_RATIO = 2.73
|
||||
else:
|
||||
VAE_KL_MEM_RATIO = 1.0
|
||||
|
||||
self.memory_used_encode = lambda shape, dtype: (1767 * shape[2] * shape[3]) * model_management.dtype_size(dtype) * VAE_KL_MEM_RATIO #These are for AutoencoderKL and need tweaking (should be lower)
|
||||
self.memory_used_decode = lambda shape, dtype: (2178 * shape[2] * shape[3] * 64) * model_management.dtype_size(dtype) * VAE_KL_MEM_RATIO
|
||||
self.downscale_ratio = 8
|
||||
self.upscale_ratio = 8
|
||||
self.latent_channels = 4
|
||||
self.latent_dim = 2
|
||||
self.input_channels = 3
|
||||
self.output_channels = 3
|
||||
self.real_output_channels = 3
|
||||
self.pad_channel_value = None
|
||||
self.process_input = lambda image: image * 2.0 - 1.0
|
||||
self.process_output = lambda image: torch.clamp((image + 1.0) / 2.0, min=0.0, max=1.0)
|
||||
self.working_dtypes = [torch.bfloat16, torch.float32]
|
||||
self.disable_offload = False
|
||||
self.not_video = False
|
||||
self.size = None
|
||||
|
||||
self.downscale_index_formula = None
|
||||
self.upscale_index_formula = None
|
||||
self.extra_1d_channel = None
|
||||
self.crop_input = True
|
||||
|
||||
if config is None:
|
||||
if "decoder.mid.block_1.mix_factor" in sd:
|
||||
encoder_config = {'double_z': True, 'z_channels': 4, 'resolution': 256, 'in_channels': 3, 'out_ch': 3, 'ch': 128, 'ch_mult': [1, 2, 4, 4], 'num_res_blocks': 2, 'attn_resolutions': [], 'dropout': 0.0}
|
||||
decoder_config = encoder_config.copy()
|
||||
decoder_config["video_kernel_size"] = [3, 1, 1]
|
||||
decoder_config["alpha"] = 0.0
|
||||
self.first_stage_model = AutoencodingEngine(regularizer_config={'target': "comfy.ldm.models.autoencoder.DiagonalGaussianRegularizer"},
|
||||
encoder_config={'target': "comfy.ldm.modules.diffusionmodules.model.Encoder", 'params': encoder_config},
|
||||
decoder_config={'target': "comfy.ldm.modules.temporal_ae.VideoDecoder", 'params': decoder_config})
|
||||
elif "taesd_decoder.1.weight" in sd:
|
||||
self.latent_channels = sd["taesd_decoder.1.weight"].shape[1]
|
||||
self.first_stage_model = comfy.taesd.taesd.TAESD(latent_channels=self.latent_channels)
|
||||
# elif "vquantizer.codebook.weight" in sd: #VQGan: stage a of stable cascade
|
||||
# self.first_stage_model = StageA()
|
||||
# self.downscale_ratio = 4
|
||||
# self.upscale_ratio = 4
|
||||
# self.process_input = lambda image: image
|
||||
# self.process_output = lambda image: image
|
||||
# elif "backbone.1.0.block.0.1.num_batches_tracked" in sd: #effnet: encoder for stage c latent of stable cascade
|
||||
# self.first_stage_model = StageC_coder()
|
||||
# self.downscale_ratio = 32
|
||||
# self.latent_channels = 16
|
||||
# new_sd = {}
|
||||
# for k in sd:
|
||||
# new_sd["encoder.{}".format(k)] = sd[k]
|
||||
# sd = new_sd
|
||||
# elif "blocks.11.num_batches_tracked" in sd: #previewer: decoder for stage c latent of stable cascade
|
||||
# self.first_stage_model = StageC_coder()
|
||||
# self.latent_channels = 16
|
||||
# new_sd = {}
|
||||
# for k in sd:
|
||||
# new_sd["previewer.{}".format(k)] = sd[k]
|
||||
# sd = new_sd
|
||||
# elif "encoder.backbone.1.0.block.0.1.num_batches_tracked" in sd: #combined effnet and previewer for stable cascade
|
||||
# self.first_stage_model = StageC_coder()
|
||||
# self.downscale_ratio = 32
|
||||
# self.latent_channels = 16
|
||||
elif "decoder.conv_in.weight" in sd:
|
||||
# if sd['decoder.conv_in.weight'].shape[1] == 64:
|
||||
# ddconfig = {"block_out_channels": [128, 256, 512, 512, 1024, 1024], "in_channels": 3, "out_channels": 3, "num_res_blocks": 2, "ffactor_spatial": 32, "downsample_match_channel": True, "upsample_match_channel": True}
|
||||
# self.latent_channels = ddconfig['z_channels'] = sd["decoder.conv_in.weight"].shape[1]
|
||||
# self.downscale_ratio = 32
|
||||
# self.upscale_ratio = 32
|
||||
# self.working_dtypes = [torch.float16, torch.bfloat16, torch.float32]
|
||||
# self.first_stage_model = AutoencodingEngine(regularizer_config={'target': "comfy.ldm.models.autoencoder.DiagonalGaussianRegularizer"},
|
||||
# encoder_config={'target': "comfy.ldm.hunyuan_video.vae.Encoder", 'params': ddconfig},
|
||||
# decoder_config={'target': "comfy.ldm.hunyuan_video.vae.Decoder", 'params': ddconfig})
|
||||
|
||||
# self.memory_used_encode = lambda shape, dtype: (700 * shape[2] * shape[3]) * model_management.dtype_size(dtype)
|
||||
# self.memory_used_decode = lambda shape, dtype: (700 * shape[2] * shape[3] * 32 * 32) * model_management.dtype_size(dtype)
|
||||
# elif sd['decoder.conv_in.weight'].shape[1] == 32:
|
||||
# ddconfig = {"block_out_channels": [128, 256, 512, 1024, 1024], "in_channels": 3, "out_channels": 3, "num_res_blocks": 2, "ffactor_spatial": 16, "ffactor_temporal": 4, "downsample_match_channel": True, "upsample_match_channel": True, "refiner_vae": False}
|
||||
# self.latent_channels = ddconfig['z_channels'] = sd["decoder.conv_in.weight"].shape[1]
|
||||
# self.working_dtypes = [torch.float16, torch.bfloat16, torch.float32]
|
||||
# self.upscale_ratio = (lambda a: max(0, a * 4 - 3), 16, 16)
|
||||
# self.upscale_index_formula = (4, 16, 16)
|
||||
# self.downscale_ratio = (lambda a: max(0, math.floor((a + 3) / 4)), 16, 16)
|
||||
# self.downscale_index_formula = (4, 16, 16)
|
||||
# self.latent_dim = 3
|
||||
# self.not_video = True
|
||||
# self.first_stage_model = AutoencodingEngine(regularizer_config={'target': "comfy.ldm.models.autoencoder.DiagonalGaussianRegularizer"},
|
||||
# encoder_config={'target': "comfy.ldm.hunyuan_video.vae_refiner.Encoder", 'params': ddconfig},
|
||||
# decoder_config={'target': "comfy.ldm.hunyuan_video.vae_refiner.Decoder", 'params': ddconfig})
|
||||
|
||||
# self.memory_used_encode = lambda shape, dtype: (2800 * shape[-2] * shape[-1]) * model_management.dtype_size(dtype)
|
||||
# self.memory_used_decode = lambda shape, dtype: (2800 * shape[-3] * shape[-2] * shape[-1] * 16 * 16) * model_management.dtype_size(dtype)
|
||||
# else:
|
||||
#default SD1.x/SD2.x VAE parameters
|
||||
ddconfig = {'double_z': True, 'z_channels': 4, 'resolution': 256, 'in_channels': 3, 'out_ch': 3, 'ch': 128, 'ch_mult': [1, 2, 4, 4], 'num_res_blocks': 2, 'attn_resolutions': [], 'dropout': 0.0}
|
||||
|
||||
if 'encoder.down.2.downsample.conv.weight' not in sd and 'decoder.up.3.upsample.conv.weight' not in sd: #Stable diffusion x4 upscaler VAE
|
||||
ddconfig['ch_mult'] = [1, 2, 4]
|
||||
self.downscale_ratio = 4
|
||||
self.upscale_ratio = 4
|
||||
|
||||
self.latent_channels = ddconfig['z_channels'] = sd["decoder.conv_in.weight"].shape[1]
|
||||
if 'post_quant_conv.weight' in sd:
|
||||
self.first_stage_model = AutoencoderKL(ddconfig=ddconfig, embed_dim=sd['post_quant_conv.weight'].shape[1])
|
||||
else:
|
||||
self.first_stage_model = AutoencodingEngine(regularizer_config={'target': "comfy.ldm.models.autoencoder.DiagonalGaussianRegularizer"},
|
||||
encoder_config={'target': "comfy.ldm.modules.diffusionmodules.model.Encoder", 'params': ddconfig},
|
||||
decoder_config={'target': "comfy.ldm.modules.diffusionmodules.model.Decoder", 'params': ddconfig})
|
||||
# elif "decoder.layers.1.layers.0.beta" in sd:
|
||||
# self.first_stage_model = AudioOobleckVAE()
|
||||
# self.memory_used_encode = lambda shape, dtype: (1000 * shape[2]) * model_management.dtype_size(dtype)
|
||||
# self.memory_used_decode = lambda shape, dtype: (1000 * shape[2] * 2048) * model_management.dtype_size(dtype)
|
||||
# self.latent_channels = 64
|
||||
# self.output_channels = 2
|
||||
# self.upscale_ratio = 2048
|
||||
# self.downscale_ratio = 2048
|
||||
# self.latent_dim = 1
|
||||
# self.process_output = lambda audio: audio
|
||||
# self.process_input = lambda audio: audio
|
||||
# self.working_dtypes = [torch.float16, torch.bfloat16, torch.float32]
|
||||
# self.disable_offload = True
|
||||
# elif "blocks.2.blocks.3.stack.5.weight" in sd or "decoder.blocks.2.blocks.3.stack.5.weight" in sd or "layers.4.layers.1.attn_block.attn.qkv.weight" in sd or "encoder.layers.4.layers.1.attn_block.attn.qkv.weight" in sd: #genmo mochi vae
|
||||
# if "blocks.2.blocks.3.stack.5.weight" in sd:
|
||||
# sd = comfy.utils.state_dict_prefix_replace(sd, {"": "decoder."})
|
||||
# if "layers.4.layers.1.attn_block.attn.qkv.weight" in sd:
|
||||
# sd = comfy.utils.state_dict_prefix_replace(sd, {"": "encoder."})
|
||||
# self.first_stage_model = comfy.ldm.genmo.vae.model.VideoVAE()
|
||||
# self.latent_channels = 12
|
||||
# self.latent_dim = 3
|
||||
# self.memory_used_decode = lambda shape, dtype: (1000 * shape[2] * shape[3] * shape[4] * (6 * 8 * 8)) * model_management.dtype_size(dtype)
|
||||
# self.memory_used_encode = lambda shape, dtype: (1.5 * max(shape[2], 7) * shape[3] * shape[4] * (6 * 8 * 8)) * model_management.dtype_size(dtype)
|
||||
# self.upscale_ratio = (lambda a: max(0, a * 6 - 5), 8, 8)
|
||||
# self.upscale_index_formula = (6, 8, 8)
|
||||
# self.downscale_ratio = (lambda a: max(0, math.floor((a + 5) / 6)), 8, 8)
|
||||
# self.downscale_index_formula = (6, 8, 8)
|
||||
# self.working_dtypes = [torch.float16, torch.float32]
|
||||
# elif "decoder.up_blocks.0.res_blocks.0.conv1.conv.weight" in sd: #lightricks ltxv
|
||||
# tensor_conv1 = sd["decoder.up_blocks.0.res_blocks.0.conv1.conv.weight"]
|
||||
# version = 0
|
||||
# if tensor_conv1.shape[0] == 512:
|
||||
# version = 0
|
||||
# elif tensor_conv1.shape[0] == 1024:
|
||||
# version = 1
|
||||
# if "encoder.down_blocks.1.conv.conv.bias" in sd:
|
||||
# version = 2
|
||||
# vae_config = None
|
||||
# if metadata is not None and "config" in metadata:
|
||||
# vae_config = json.loads(metadata["config"]).get("vae", None)
|
||||
# self.first_stage_model = comfy.ldm.lightricks.vae.causal_video_autoencoder.VideoVAE(version=version, config=vae_config)
|
||||
# self.latent_channels = 128
|
||||
# self.latent_dim = 3
|
||||
# self.memory_used_decode = lambda shape, dtype: (900 * shape[2] * shape[3] * shape[4] * (8 * 8 * 8)) * model_management.dtype_size(dtype)
|
||||
# self.memory_used_encode = lambda shape, dtype: (70 * max(shape[2], 7) * shape[3] * shape[4]) * model_management.dtype_size(dtype)
|
||||
# self.upscale_ratio = (lambda a: max(0, a * 8 - 7), 32, 32)
|
||||
# self.upscale_index_formula = (8, 32, 32)
|
||||
# self.downscale_ratio = (lambda a: max(0, math.floor((a + 7) / 8)), 32, 32)
|
||||
# self.downscale_index_formula = (8, 32, 32)
|
||||
# self.working_dtypes = [torch.bfloat16, torch.float32]
|
||||
# elif "decoder.conv_in.conv.weight" in sd and sd['decoder.conv_in.conv.weight'].shape[1] == 32:
|
||||
# ddconfig = {"block_out_channels": [128, 256, 512, 1024, 1024], "in_channels": 3, "out_channels": 3, "num_res_blocks": 2, "ffactor_spatial": 16, "ffactor_temporal": 4, "downsample_match_channel": True, "upsample_match_channel": True}
|
||||
# ddconfig['z_channels'] = sd["decoder.conv_in.conv.weight"].shape[1]
|
||||
# self.latent_channels = 64
|
||||
# self.upscale_ratio = (lambda a: max(0, a * 4 - 3), 16, 16)
|
||||
# self.upscale_index_formula = (4, 16, 16)
|
||||
# self.downscale_ratio = (lambda a: max(0, math.floor((a + 3) / 4)), 16, 16)
|
||||
# self.downscale_index_formula = (4, 16, 16)
|
||||
# self.latent_dim = 3
|
||||
# self.not_video = True
|
||||
# self.working_dtypes = [torch.float16, torch.bfloat16, torch.float32]
|
||||
# self.first_stage_model = AutoencodingEngine(regularizer_config={'target': "comfy.ldm.models.autoencoder.EmptyRegularizer"},
|
||||
# encoder_config={'target': "comfy.ldm.hunyuan_video.vae_refiner.Encoder", 'params': ddconfig},
|
||||
# decoder_config={'target': "comfy.ldm.hunyuan_video.vae_refiner.Decoder", 'params': ddconfig})
|
||||
|
||||
# self.memory_used_encode = lambda shape, dtype: (1400 * shape[-2] * shape[-1]) * model_management.dtype_size(dtype)
|
||||
# self.memory_used_decode = lambda shape, dtype: (1400 * shape[-3] * shape[-2] * shape[-1] * 16 * 16) * model_management.dtype_size(dtype)
|
||||
elif "decoder.conv_in.conv.weight" in sd:
|
||||
ddconfig = {'double_z': True, 'z_channels': 4, 'resolution': 256, 'in_channels': 3, 'out_ch': 3, 'ch': 128, 'ch_mult': [1, 2, 4, 4], 'num_res_blocks': 2, 'attn_resolutions': [], 'dropout': 0.0}
|
||||
ddconfig["conv3d"] = True
|
||||
ddconfig["time_compress"] = 4
|
||||
self.upscale_ratio = (lambda a: max(0, a * 4 - 3), 8, 8)
|
||||
self.upscale_index_formula = (4, 8, 8)
|
||||
self.downscale_ratio = (lambda a: max(0, math.floor((a + 3) / 4)), 8, 8)
|
||||
self.downscale_index_formula = (4, 8, 8)
|
||||
self.latent_dim = 3
|
||||
self.latent_channels = ddconfig['z_channels'] = sd["decoder.conv_in.conv.weight"].shape[1]
|
||||
self.first_stage_model = AutoencoderKL(ddconfig=ddconfig, embed_dim=sd['post_quant_conv.weight'].shape[1])
|
||||
self.memory_used_decode = lambda shape, dtype: (1500 * shape[2] * shape[3] * shape[4] * (4 * 8 * 8)) * model_management.dtype_size(dtype)
|
||||
self.memory_used_encode = lambda shape, dtype: (900 * max(shape[2], 2) * shape[3] * shape[4]) * model_management.dtype_size(dtype)
|
||||
self.working_dtypes = [torch.bfloat16, torch.float16, torch.float32]
|
||||
# elif "decoder.unpatcher3d.wavelets" in sd:
|
||||
# self.upscale_ratio = (lambda a: max(0, a * 8 - 7), 8, 8)
|
||||
# self.upscale_index_formula = (8, 8, 8)
|
||||
# self.downscale_ratio = (lambda a: max(0, math.floor((a + 7) / 8)), 8, 8)
|
||||
# self.downscale_index_formula = (8, 8, 8)
|
||||
# self.latent_dim = 3
|
||||
# self.latent_channels = 16
|
||||
# ddconfig = {'z_channels': 16, 'latent_channels': self.latent_channels, 'z_factor': 1, 'resolution': 1024, 'in_channels': 3, 'out_channels': 3, 'channels': 128, 'channels_mult': [2, 4, 4], 'num_res_blocks': 2, 'attn_resolutions': [32], 'dropout': 0.0, 'patch_size': 4, 'num_groups': 1, 'temporal_compression': 8, 'spacial_compression': 8}
|
||||
# self.first_stage_model = comfy.ldm.cosmos.vae.CausalContinuousVideoTokenizer(**ddconfig)
|
||||
# TODO: these values are a bit off because this is not a standard VAE
|
||||
# self.memory_used_decode = lambda shape, dtype: (50 * shape[2] * shape[3] * shape[4] * (8 * 8 * 8)) * model_management.dtype_size(dtype)
|
||||
# self.memory_used_encode = lambda shape, dtype: (50 * (round((shape[2] + 7) / 8) * 8) * shape[3] * shape[4]) * model_management.dtype_size(dtype)
|
||||
# self.working_dtypes = [torch.bfloat16, torch.float32]
|
||||
elif "decoder.middle.0.residual.0.gamma" in sd:
|
||||
if "decoder.upsamples.0.upsamples.0.residual.2.weight" in sd: # Wan 2.2 VAE
|
||||
self.upscale_ratio = (lambda a: max(0, a * 4 - 3), 16, 16)
|
||||
self.upscale_index_formula = (4, 16, 16)
|
||||
self.downscale_ratio = (lambda a: max(0, math.floor((a + 3) / 4)), 16, 16)
|
||||
self.downscale_index_formula = (4, 16, 16)
|
||||
self.latent_dim = 3
|
||||
self.latent_channels = 48
|
||||
ddconfig = {"dim": 160, "z_dim": self.latent_channels, "dim_mult": [1, 2, 4, 4], "num_res_blocks": 2, "attn_scales": [], "temperal_downsample": [False, True, True], "dropout": 0.0}
|
||||
self.first_stage_model = comfy.ldm.wan.vae2_2.WanVAE(**ddconfig)
|
||||
self.working_dtypes = [torch.bfloat16, torch.float16, torch.float32]
|
||||
self.memory_used_encode = lambda shape, dtype: 3300 * shape[3] * shape[4] * model_management.dtype_size(dtype)
|
||||
self.memory_used_decode = lambda shape, dtype: 8000 * shape[3] * shape[4] * (16 * 16) * model_management.dtype_size(dtype)
|
||||
else: # Wan 2.1 VAE
|
||||
self.upscale_ratio = (lambda a: max(0, a * 4 - 3), 8, 8)
|
||||
self.upscale_index_formula = (4, 8, 8)
|
||||
self.downscale_ratio = (lambda a: max(0, math.floor((a + 3) / 4)), 8, 8)
|
||||
self.downscale_index_formula = (4, 8, 8)
|
||||
self.input_channels = sd["encoder.conv1.weight"].shape[1]
|
||||
# self.output_channels = sd["decoder.head.2.weight"].shape[0]
|
||||
self.output_channels = self.input_channels
|
||||
self.real_output_channels = sd["decoder.head.2.weight"].shape[0]
|
||||
self.latent_dim = 3
|
||||
self.latent_channels = 16
|
||||
self.pad_channel_value = 1.0
|
||||
|
||||
ddconfig = {
|
||||
"in_channels": self.input_channels,
|
||||
# "out_channels": self.output_channels,
|
||||
"out_channels": self.real_output_channels,
|
||||
"dim": 96,
|
||||
"z_dim": self.latent_channels,
|
||||
"dim_mult": [1, 2, 4, 4],
|
||||
"num_res_blocks": 2,
|
||||
"attn_scales": [],
|
||||
"temperal_downsample": [False, True, True],
|
||||
"dropout": 0.0,
|
||||
}
|
||||
self.first_stage_model = WanVAE(**ddconfig)
|
||||
self.working_dtypes = [torch.bfloat16, torch.float16, torch.float32]
|
||||
self.memory_used_encode = lambda shape, dtype: 6000 * shape[3] * shape[4] * model_management.dtype_size(dtype)
|
||||
self.memory_used_decode = lambda shape, dtype: 7000 * shape[3] * shape[4] * (8 * 8) * model_management.dtype_size(dtype)
|
||||
# Hunyuan 3d v2 2.0 & 2.1
|
||||
# elif "geo_decoder.cross_attn_decoder.ln_1.bias" in sd:
|
||||
# self.latent_dim = 1
|
||||
|
||||
# def estimate_memory(shape, dtype, num_layers = 16, kv_cache_multiplier = 2):
|
||||
# batch, num_tokens, hidden_dim = shape
|
||||
# dtype_size = model_management.dtype_size(dtype)
|
||||
|
||||
# total_mem = batch * num_tokens * hidden_dim * dtype_size * (1 + kv_cache_multiplier * num_layers)
|
||||
# return total_mem
|
||||
|
||||
# better memory estimations
|
||||
# self.memory_used_encode = lambda shape, dtype, num_layers = 8, kv_cache_multiplier = 0:\
|
||||
# estimate_memory(shape, dtype, num_layers, kv_cache_multiplier)
|
||||
|
||||
# self.memory_used_decode = lambda shape, dtype, num_layers = 16, kv_cache_multiplier = 2: \
|
||||
# estimate_memory(shape, dtype, num_layers, kv_cache_multiplier)
|
||||
|
||||
# self.first_stage_model = comfy.ldm.hunyuan3d.vae.ShapeVAE()
|
||||
# self.working_dtypes = [torch.float16, torch.bfloat16, torch.float32]
|
||||
# elif "vocoder.backbone.channel_layers.0.0.bias" in sd: #Ace Step Audio
|
||||
# self.first_stage_model = comfy.ldm.ace.vae.music_dcae_pipeline.MusicDCAE(source_sample_rate=44100)
|
||||
# self.memory_used_encode = lambda shape, dtype: (shape[2] * 330) * model_management.dtype_size(dtype)
|
||||
# self.memory_used_decode = lambda shape, dtype: (shape[2] * shape[3] * 87000) * model_management.dtype_size(dtype)
|
||||
# self.latent_channels = 8
|
||||
# self.output_channels = 2
|
||||
# self.upscale_ratio = 4096
|
||||
# self.downscale_ratio = 4096
|
||||
# self.latent_dim = 2
|
||||
# self.process_output = lambda audio: audio
|
||||
# self.process_input = lambda audio: audio
|
||||
# self.working_dtypes = [torch.bfloat16, torch.float16, torch.float32]
|
||||
# self.disable_offload = True
|
||||
# self.extra_1d_channel = 16
|
||||
# elif "pixel_space_vae" in sd:
|
||||
# self.first_stage_model = comfy.pixel_space_convert.PixelspaceConversionVAE()
|
||||
# self.memory_used_encode = lambda shape, dtype: (1 * shape[2] * shape[3]) * model_management.dtype_size(dtype)
|
||||
# self.memory_used_decode = lambda shape, dtype: (1 * shape[2] * shape[3]) * model_management.dtype_size(dtype)
|
||||
# self.downscale_ratio = 1
|
||||
# self.upscale_ratio = 1
|
||||
# self.latent_channels = 3
|
||||
# self.latent_dim = 2
|
||||
# self.output_channels = 3
|
||||
# elif "vocoder.activation_post.downsample.lowpass.filter" in sd: #MMAudio VAE
|
||||
# sample_rate = 16000
|
||||
# if sample_rate == 16000:
|
||||
# mode = '16k'
|
||||
# else:
|
||||
# mode = '44k'
|
||||
|
||||
# self.first_stage_model = comfy.ldm.mmaudio.vae.autoencoder.AudioAutoencoder(mode=mode)
|
||||
# self.memory_used_encode = lambda shape, dtype: (30 * shape[2]) * model_management.dtype_size(dtype)
|
||||
# self.memory_used_decode = lambda shape, dtype: (90 * shape[2] * 1411.2) * model_management.dtype_size(dtype)
|
||||
# self.latent_channels = 20
|
||||
# self.output_channels = 2
|
||||
# self.upscale_ratio = 512 * (44100 / sample_rate)
|
||||
# self.downscale_ratio = 512 * (44100 / sample_rate)
|
||||
# self.latent_dim = 1
|
||||
# self.process_output = lambda audio: audio
|
||||
# self.process_input = lambda audio: audio
|
||||
# self.working_dtypes = [torch.float32]
|
||||
# self.crop_input = False
|
||||
else:
|
||||
logging.warning("WARNING: No VAE weights detected, VAE not initalized.")
|
||||
self.first_stage_model = None
|
||||
return
|
||||
else:
|
||||
self.first_stage_model = AutoencoderKL(**(config['params']))
|
||||
self.first_stage_model = self.first_stage_model.eval()
|
||||
|
||||
m, u = self.first_stage_model.load_state_dict(sd, strict=False)
|
||||
if len(m) > 0:
|
||||
logging.warning("Missing VAE keys {}".format(m))
|
||||
|
||||
if len(u) > 0:
|
||||
logging.debug("Leftover VAE keys {}".format(u))
|
||||
|
||||
if device is None:
|
||||
device = model_management.vae_device()
|
||||
self.device = device
|
||||
offload_device = model_management.vae_offload_device()
|
||||
if dtype is None:
|
||||
dtype = model_management.vae_dtype(self.device, self.working_dtypes)
|
||||
self.vae_dtype = dtype
|
||||
self.first_stage_model.to(self.vae_dtype)
|
||||
self.output_device = model_management.intermediate_device()
|
||||
|
||||
self.patcher = comfy.model_patcher.ModelPatcher(self.first_stage_model, load_device=self.device, offload_device=offload_device)
|
||||
logging.info("VAE load device: {}, offload device: {}, dtype: {}".format(self.device, offload_device, self.vae_dtype))
|
||||
|
||||
def decode(self, samples_in):
|
||||
"""
|
||||
Override VAE.decode() to fix a regression introduced in ComfyUI PRs #11405 / #11406
|
||||
(merged Dec 18 2025). Those PRs changed the 3D untiled decode path inside
|
||||
VAE.decode() so that it no longer routes through decode_tiled_3d for subclasses;
|
||||
it now does inline channel allocation and trimming that breaks CustomVAE because it
|
||||
only sees self.output_channels (= input_channels = 3) and ignores
|
||||
self.real_output_channels (the actual decoder head output count, e.g. 12 for the
|
||||
Wan 2x upscale VAE). process_output was also no longer guaranteed to be applied
|
||||
through that path.
|
||||
|
||||
For 2D VAEs (latent_dim != 3) we delegate to the parent as before.
|
||||
For 3D VAEs (WanVAE family: Wan 2.1, Wan 2.2, Qwen-Image, Wan 2x upscale) we do
|
||||
the model-loading bookkeeping ourselves, then call our own decode_tiled_3d which
|
||||
already handles real_output_channels and process_output correctly.
|
||||
"""
|
||||
if self.latent_dim != 3:
|
||||
return super().decode(samples_in)
|
||||
|
||||
# Mirror the memory-management preamble from the parent's decode().
|
||||
memory_used = self.memory_used_decode(samples_in.shape, self.vae_dtype)
|
||||
model_management.load_models_gpu([self.patcher], memory_required=memory_used)
|
||||
|
||||
# decode_tiled_3d returns process_output(tiled_scale_multidim(...)) whose shape is
|
||||
# [B, real_output_channels, T_out, H_out, W_out] (5-D for multi-frame video)
|
||||
# or [B, real_output_channels, H_out, W_out] (4-D if tiled_scale_multidim
|
||||
# squeezed the temporal dim)
|
||||
images = self.decode_tiled_3d(samples_in)
|
||||
|
||||
# Normalise to [B*T, H, W, C] — same layout that the parent returns and that
|
||||
# VAEUtils_VAEDecodeTiled.decode() expects (it has its own 5-D reshape guard too).
|
||||
if images.ndim == 5:
|
||||
B, C, T, H, W = images.shape
|
||||
images = images.permute(0, 2, 3, 4, 1).reshape(B * T, H, W, C)
|
||||
elif images.ndim == 4:
|
||||
# Already [B, C, H, W] — just move channels to last dim.
|
||||
images = images.movedim(1, -1)
|
||||
# ndim == 3 or anything unexpected: leave as-is and let downstream handle it.
|
||||
|
||||
return images
|
||||
|
||||
def decode_tiled_3d(self, samples, tile_t=999, tile_x=32, tile_y=32, overlap=(1, 8, 8)):
|
||||
decode_fn = lambda a: self.first_stage_model.decode(a.to(self.vae_dtype).to(self.device)).float()
|
||||
return self.process_output(
|
||||
comfy.utils.tiled_scale_multidim(
|
||||
samples,
|
||||
decode_fn,
|
||||
tile=(tile_t, tile_x, tile_y),
|
||||
overlap=overlap,
|
||||
upscale_amount=self.upscale_ratio,
|
||||
out_channels=self.real_output_channels,
|
||||
index_formulas=self.upscale_index_formula,
|
||||
output_device=self.output_device,
|
||||
)
|
||||
)
|
||||
@@ -0,0 +1,521 @@
|
||||
# original version: https://github.com/Wan-Video/Wan2.1/blob/main/wan/modules/vae.py
|
||||
# Copyright 2024-2025 The Alibaba Wan Team Authors. All rights reserved.
|
||||
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
from einops import rearrange
|
||||
from comfy.ldm.modules.diffusionmodules.model import vae_attention
|
||||
|
||||
import comfy.ops
|
||||
ops = comfy.ops.disable_weight_init
|
||||
|
||||
CACHE_T = 2
|
||||
|
||||
|
||||
class CausalConv3d(ops.Conv3d):
|
||||
"""
|
||||
Causal 3d convolusion.
|
||||
"""
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
self._padding = (self.padding[2], self.padding[2], self.padding[1],
|
||||
self.padding[1], 2 * self.padding[0], 0)
|
||||
self.padding = (0, 0, 0)
|
||||
|
||||
def forward(self, x, cache_x=None, cache_list=None, cache_idx=None):
|
||||
if cache_list is not None:
|
||||
cache_x = cache_list[cache_idx]
|
||||
cache_list[cache_idx] = None
|
||||
|
||||
padding = list(self._padding)
|
||||
if cache_x is not None and self._padding[4] > 0:
|
||||
cache_x = cache_x.to(x.device)
|
||||
x = torch.cat([cache_x, x], dim=2)
|
||||
padding[4] -= cache_x.shape[2]
|
||||
del cache_x
|
||||
x = F.pad(x, padding)
|
||||
|
||||
return super().forward(x)
|
||||
|
||||
|
||||
class RMS_norm(nn.Module):
|
||||
|
||||
def __init__(self, dim, channel_first=True, images=True, bias=False):
|
||||
super().__init__()
|
||||
broadcastable_dims = (1, 1, 1) if not images else (1, 1)
|
||||
shape = (dim, *broadcastable_dims) if channel_first else (dim,)
|
||||
|
||||
self.channel_first = channel_first
|
||||
self.scale = dim**0.5
|
||||
self.gamma = nn.Parameter(torch.ones(shape))
|
||||
self.bias = nn.Parameter(torch.zeros(shape)) if bias else None
|
||||
|
||||
def forward(self, x):
|
||||
return F.normalize(
|
||||
x, dim=(1 if self.channel_first else -1)) * self.scale * self.gamma.to(x) + (self.bias.to(x) if self.bias is not None else 0)
|
||||
|
||||
|
||||
class Resample(nn.Module):
|
||||
|
||||
def __init__(self, dim, mode):
|
||||
assert mode in ('none', 'upsample2d', 'upsample3d', 'downsample2d',
|
||||
'downsample3d')
|
||||
super().__init__()
|
||||
self.dim = dim
|
||||
self.mode = mode
|
||||
|
||||
# layers
|
||||
if mode == 'upsample2d':
|
||||
self.resample = nn.Sequential(
|
||||
nn.Upsample(scale_factor=(2., 2.), mode='nearest-exact'),
|
||||
ops.Conv2d(dim, dim // 2, 3, padding=1))
|
||||
elif mode == 'upsample3d':
|
||||
self.resample = nn.Sequential(
|
||||
nn.Upsample(scale_factor=(2., 2.), mode='nearest-exact'),
|
||||
ops.Conv2d(dim, dim // 2, 3, padding=1))
|
||||
self.time_conv = CausalConv3d(
|
||||
dim, dim * 2, (3, 1, 1), padding=(1, 0, 0))
|
||||
|
||||
elif mode == 'downsample2d':
|
||||
self.resample = nn.Sequential(
|
||||
nn.ZeroPad2d((0, 1, 0, 1)),
|
||||
ops.Conv2d(dim, dim, 3, stride=(2, 2)))
|
||||
elif mode == 'downsample3d':
|
||||
self.resample = nn.Sequential(
|
||||
nn.ZeroPad2d((0, 1, 0, 1)),
|
||||
ops.Conv2d(dim, dim, 3, stride=(2, 2)))
|
||||
self.time_conv = CausalConv3d(
|
||||
dim, dim, (3, 1, 1), stride=(2, 1, 1), padding=(0, 0, 0))
|
||||
|
||||
else:
|
||||
self.resample = nn.Identity()
|
||||
|
||||
def forward(self, x, feat_cache=None, feat_idx=[0]):
|
||||
b, c, t, h, w = x.size()
|
||||
if self.mode == 'upsample3d':
|
||||
if feat_cache is not None:
|
||||
idx = feat_idx[0]
|
||||
if feat_cache[idx] is None:
|
||||
feat_cache[idx] = 'Rep'
|
||||
feat_idx[0] += 1
|
||||
else:
|
||||
|
||||
cache_x = x[:, :, -CACHE_T:, :, :].clone()
|
||||
if cache_x.shape[2] < 2 and feat_cache[
|
||||
idx] is not None and feat_cache[idx] != 'Rep':
|
||||
# cache last frame of last two chunk
|
||||
cache_x = torch.cat([
|
||||
feat_cache[idx][:, :, -1, :, :].unsqueeze(2).to(
|
||||
cache_x.device), cache_x
|
||||
],
|
||||
dim=2)
|
||||
if cache_x.shape[2] < 2 and feat_cache[
|
||||
idx] is not None and feat_cache[idx] == 'Rep':
|
||||
cache_x = torch.cat([
|
||||
torch.zeros_like(cache_x).to(cache_x.device),
|
||||
cache_x
|
||||
],
|
||||
dim=2)
|
||||
if feat_cache[idx] == 'Rep':
|
||||
x = self.time_conv(x)
|
||||
else:
|
||||
x = self.time_conv(x, feat_cache[idx])
|
||||
feat_cache[idx] = cache_x
|
||||
feat_idx[0] += 1
|
||||
|
||||
x = x.reshape(b, 2, c, t, h, w)
|
||||
x = torch.stack((x[:, 0, :, :, :, :], x[:, 1, :, :, :, :]),
|
||||
3)
|
||||
x = x.reshape(b, c, t * 2, h, w)
|
||||
t = x.shape[2]
|
||||
x = rearrange(x, 'b c t h w -> (b t) c h w')
|
||||
x = self.resample(x)
|
||||
x = rearrange(x, '(b t) c h w -> b c t h w', t=t)
|
||||
|
||||
if self.mode == 'downsample3d':
|
||||
if feat_cache is not None:
|
||||
idx = feat_idx[0]
|
||||
if feat_cache[idx] is None:
|
||||
feat_cache[idx] = x.clone()
|
||||
feat_idx[0] += 1
|
||||
else:
|
||||
|
||||
cache_x = x[:, :, -1:, :, :].clone()
|
||||
# if cache_x.shape[2] < 2 and feat_cache[idx] is not None and feat_cache[idx]!='Rep':
|
||||
# # cache last frame of last two chunk
|
||||
# cache_x = torch.cat([feat_cache[idx][:, :, -1, :, :].unsqueeze(2).to(cache_x.device), cache_x], dim=2)
|
||||
|
||||
x = self.time_conv(
|
||||
torch.cat([feat_cache[idx][:, :, -1:, :, :], x], 2))
|
||||
feat_cache[idx] = cache_x
|
||||
feat_idx[0] += 1
|
||||
return x
|
||||
|
||||
|
||||
class ResidualBlock(nn.Module):
|
||||
|
||||
def __init__(self, in_dim, out_dim, dropout=0.0):
|
||||
super().__init__()
|
||||
self.in_dim = in_dim
|
||||
self.out_dim = out_dim
|
||||
|
||||
# layers
|
||||
self.residual = nn.Sequential(
|
||||
RMS_norm(in_dim, images=False), nn.SiLU(),
|
||||
CausalConv3d(in_dim, out_dim, 3, padding=1),
|
||||
RMS_norm(out_dim, images=False), nn.SiLU(), nn.Dropout(dropout),
|
||||
CausalConv3d(out_dim, out_dim, 3, padding=1))
|
||||
self.shortcut = CausalConv3d(in_dim, out_dim, 1) \
|
||||
if in_dim != out_dim else nn.Identity()
|
||||
|
||||
def forward(self, x, feat_cache=None, feat_idx=[0]):
|
||||
old_x = x
|
||||
for layer in self.residual:
|
||||
if isinstance(layer, CausalConv3d) and feat_cache is not None:
|
||||
idx = feat_idx[0]
|
||||
cache_x = x[:, :, -CACHE_T:, :, :].clone()
|
||||
if cache_x.shape[2] < 2 and feat_cache[idx] is not None:
|
||||
# cache last frame of last two chunk
|
||||
cache_x = torch.cat([
|
||||
feat_cache[idx][:, :, -1, :, :].unsqueeze(2).to(
|
||||
cache_x.device), cache_x
|
||||
],
|
||||
dim=2)
|
||||
x = layer(x, cache_list=feat_cache, cache_idx=idx)
|
||||
feat_cache[idx] = cache_x
|
||||
feat_idx[0] += 1
|
||||
else:
|
||||
x = layer(x)
|
||||
return x + self.shortcut(old_x)
|
||||
|
||||
|
||||
class AttentionBlock(nn.Module):
|
||||
"""
|
||||
Causal self-attention with a single head.
|
||||
"""
|
||||
|
||||
def __init__(self, dim):
|
||||
super().__init__()
|
||||
self.dim = dim
|
||||
|
||||
# layers
|
||||
self.norm = RMS_norm(dim)
|
||||
self.to_qkv = ops.Conv2d(dim, dim * 3, 1)
|
||||
self.proj = ops.Conv2d(dim, dim, 1)
|
||||
self.optimized_attention = vae_attention()
|
||||
|
||||
def forward(self, x):
|
||||
identity = x
|
||||
b, c, t, h, w = x.size()
|
||||
x = rearrange(x, 'b c t h w -> (b t) c h w')
|
||||
x = self.norm(x)
|
||||
# compute query, key, value
|
||||
|
||||
q, k, v = self.to_qkv(x).chunk(3, dim=1)
|
||||
x = self.optimized_attention(q, k, v)
|
||||
|
||||
# output
|
||||
x = self.proj(x)
|
||||
x = rearrange(x, '(b t) c h w-> b c t h w', t=t)
|
||||
return x + identity
|
||||
|
||||
|
||||
class Encoder3d(nn.Module):
|
||||
|
||||
def __init__(self,
|
||||
in_channels=3,
|
||||
dim=128,
|
||||
z_dim=4,
|
||||
dim_mult=[1, 2, 4, 4],
|
||||
num_res_blocks=2,
|
||||
attn_scales=[],
|
||||
temperal_downsample=[True, True, False],
|
||||
dropout=0.0):
|
||||
super().__init__()
|
||||
self.in_channels = in_channels
|
||||
self.dim = dim
|
||||
self.z_dim = z_dim
|
||||
self.dim_mult = dim_mult
|
||||
self.num_res_blocks = num_res_blocks
|
||||
self.attn_scales = attn_scales
|
||||
self.temperal_downsample = temperal_downsample
|
||||
|
||||
# dimensions
|
||||
dims = [dim * u for u in [1] + dim_mult]
|
||||
scale = 1.0
|
||||
|
||||
# init block
|
||||
self.conv1 = CausalConv3d(in_channels, dims[0], 3, padding=1)
|
||||
|
||||
# downsample blocks
|
||||
downsamples = []
|
||||
for i, (in_dim, out_dim) in enumerate(zip(dims[:-1], dims[1:])):
|
||||
# residual (+attention) blocks
|
||||
for _ in range(num_res_blocks):
|
||||
downsamples.append(ResidualBlock(in_dim, out_dim, dropout))
|
||||
if scale in attn_scales:
|
||||
downsamples.append(AttentionBlock(out_dim))
|
||||
in_dim = out_dim
|
||||
|
||||
# downsample block
|
||||
if i != len(dim_mult) - 1:
|
||||
mode = 'downsample3d' if temperal_downsample[
|
||||
i] else 'downsample2d'
|
||||
downsamples.append(Resample(out_dim, mode=mode))
|
||||
scale /= 2.0
|
||||
self.downsamples = nn.Sequential(*downsamples)
|
||||
|
||||
# middle blocks
|
||||
self.middle = nn.Sequential(
|
||||
ResidualBlock(out_dim, out_dim, dropout), AttentionBlock(out_dim),
|
||||
ResidualBlock(out_dim, out_dim, dropout))
|
||||
|
||||
# output blocks
|
||||
self.head = nn.Sequential(
|
||||
RMS_norm(out_dim, images=False), nn.SiLU(),
|
||||
CausalConv3d(out_dim, z_dim, 3, padding=1))
|
||||
|
||||
def forward(self, x, feat_cache=None, feat_idx=[0]):
|
||||
if feat_cache is not None:
|
||||
idx = feat_idx[0]
|
||||
cache_x = x[:, :, -CACHE_T:, :, :].clone()
|
||||
if cache_x.shape[2] < 2 and feat_cache[idx] is not None:
|
||||
# cache last frame of last two chunk
|
||||
cache_x = torch.cat([
|
||||
feat_cache[idx][:, :, -1, :, :].unsqueeze(2).to(
|
||||
cache_x.device), cache_x
|
||||
],
|
||||
dim=2)
|
||||
x = self.conv1(x, feat_cache[idx])
|
||||
feat_cache[idx] = cache_x
|
||||
feat_idx[0] += 1
|
||||
else:
|
||||
x = self.conv1(x)
|
||||
|
||||
## downsamples
|
||||
for layer in self.downsamples:
|
||||
if feat_cache is not None:
|
||||
x = layer(x, feat_cache, feat_idx)
|
||||
else:
|
||||
x = layer(x)
|
||||
|
||||
## middle
|
||||
for layer in self.middle:
|
||||
if isinstance(layer, ResidualBlock) and feat_cache is not None:
|
||||
x = layer(x, feat_cache, feat_idx)
|
||||
else:
|
||||
x = layer(x)
|
||||
|
||||
## head
|
||||
for layer in self.head:
|
||||
if isinstance(layer, CausalConv3d) and feat_cache is not None:
|
||||
idx = feat_idx[0]
|
||||
cache_x = x[:, :, -CACHE_T:, :, :].clone()
|
||||
if cache_x.shape[2] < 2 and feat_cache[idx] is not None:
|
||||
# cache last frame of last two chunk
|
||||
cache_x = torch.cat([
|
||||
feat_cache[idx][:, :, -1, :, :].unsqueeze(2).to(
|
||||
cache_x.device), cache_x
|
||||
],
|
||||
dim=2)
|
||||
x = layer(x, feat_cache[idx])
|
||||
feat_cache[idx] = cache_x
|
||||
feat_idx[0] += 1
|
||||
else:
|
||||
x = layer(x)
|
||||
return x
|
||||
|
||||
|
||||
class Decoder3d(nn.Module):
|
||||
|
||||
def __init__(self,
|
||||
out_channels=3,
|
||||
dim=128,
|
||||
z_dim=4,
|
||||
dim_mult=[1, 2, 4, 4],
|
||||
num_res_blocks=2,
|
||||
attn_scales=[],
|
||||
temperal_upsample=[False, True, True],
|
||||
dropout=0.0):
|
||||
super().__init__()
|
||||
self.out_channels = out_channels
|
||||
self.dim = dim
|
||||
self.z_dim = z_dim
|
||||
self.dim_mult = dim_mult
|
||||
self.num_res_blocks = num_res_blocks
|
||||
self.attn_scales = attn_scales
|
||||
self.temperal_upsample = temperal_upsample
|
||||
|
||||
# dimensions
|
||||
dims = [dim * u for u in [dim_mult[-1]] + dim_mult[::-1]]
|
||||
scale = 1.0 / 2**(len(dim_mult) - 2)
|
||||
|
||||
# init block
|
||||
self.conv1 = CausalConv3d(z_dim, dims[0], 3, padding=1)
|
||||
|
||||
# middle blocks
|
||||
self.middle = nn.Sequential(
|
||||
ResidualBlock(dims[0], dims[0], dropout), AttentionBlock(dims[0]),
|
||||
ResidualBlock(dims[0], dims[0], dropout))
|
||||
|
||||
# upsample blocks
|
||||
upsamples = []
|
||||
for i, (in_dim, out_dim) in enumerate(zip(dims[:-1], dims[1:])):
|
||||
# residual (+attention) blocks
|
||||
if i == 1 or i == 2 or i == 3:
|
||||
in_dim = in_dim // 2
|
||||
for _ in range(num_res_blocks + 1):
|
||||
upsamples.append(ResidualBlock(in_dim, out_dim, dropout))
|
||||
if scale in attn_scales:
|
||||
upsamples.append(AttentionBlock(out_dim))
|
||||
in_dim = out_dim
|
||||
|
||||
# upsample block
|
||||
if i != len(dim_mult) - 1:
|
||||
mode = 'upsample3d' if temperal_upsample[i] else 'upsample2d'
|
||||
upsamples.append(Resample(out_dim, mode=mode))
|
||||
scale *= 2.0
|
||||
self.upsamples = nn.Sequential(*upsamples)
|
||||
|
||||
# output blocks
|
||||
self.head = nn.Sequential(
|
||||
RMS_norm(out_dim, images=False), nn.SiLU(),
|
||||
CausalConv3d(out_dim, out_channels, 3, padding=1))
|
||||
|
||||
def forward(self, x, feat_cache=None, feat_idx=[0]):
|
||||
## conv1
|
||||
if feat_cache is not None:
|
||||
idx = feat_idx[0]
|
||||
cache_x = x[:, :, -CACHE_T:, :, :].clone()
|
||||
if cache_x.shape[2] < 2 and feat_cache[idx] is not None:
|
||||
# cache last frame of last two chunk
|
||||
cache_x = torch.cat([
|
||||
feat_cache[idx][:, :, -1, :, :].unsqueeze(2).to(
|
||||
cache_x.device), cache_x
|
||||
],
|
||||
dim=2)
|
||||
x = self.conv1(x, feat_cache[idx])
|
||||
feat_cache[idx] = cache_x
|
||||
feat_idx[0] += 1
|
||||
else:
|
||||
x = self.conv1(x)
|
||||
|
||||
## middle
|
||||
for layer in self.middle:
|
||||
if isinstance(layer, ResidualBlock) and feat_cache is not None:
|
||||
x = layer(x, feat_cache, feat_idx)
|
||||
else:
|
||||
x = layer(x)
|
||||
|
||||
## upsamples
|
||||
for layer in self.upsamples:
|
||||
if feat_cache is not None:
|
||||
x = layer(x, feat_cache, feat_idx)
|
||||
else:
|
||||
x = layer(x)
|
||||
|
||||
## head
|
||||
for layer in self.head:
|
||||
if isinstance(layer, CausalConv3d) and feat_cache is not None:
|
||||
idx = feat_idx[0]
|
||||
cache_x = x[:, :, -CACHE_T:, :, :].clone()
|
||||
if cache_x.shape[2] < 2 and feat_cache[idx] is not None:
|
||||
# cache last frame of last two chunk
|
||||
cache_x = torch.cat([
|
||||
feat_cache[idx][:, :, -1, :, :].unsqueeze(2).to(
|
||||
cache_x.device), cache_x
|
||||
],
|
||||
dim=2)
|
||||
x = layer(x, feat_cache[idx])
|
||||
feat_cache[idx] = cache_x
|
||||
feat_idx[0] += 1
|
||||
else:
|
||||
x = layer(x)
|
||||
return x
|
||||
|
||||
|
||||
def count_conv3d(model):
|
||||
count = 0
|
||||
for m in model.modules():
|
||||
if isinstance(m, CausalConv3d):
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
class WanVAE(nn.Module):
|
||||
|
||||
def __init__(self,
|
||||
in_channels=3,
|
||||
out_channels=3,
|
||||
dim=128,
|
||||
z_dim=4,
|
||||
dim_mult=[1, 2, 4, 4],
|
||||
num_res_blocks=2,
|
||||
attn_scales=[],
|
||||
temperal_downsample=[True, True, False],
|
||||
dropout=0.0):
|
||||
super().__init__()
|
||||
self.in_channels = in_channels
|
||||
self.out_channels = out_channels
|
||||
self.dim = dim
|
||||
self.z_dim = z_dim
|
||||
self.dim_mult = dim_mult
|
||||
self.num_res_blocks = num_res_blocks
|
||||
self.attn_scales = attn_scales
|
||||
self.temperal_downsample = temperal_downsample
|
||||
self.temperal_upsample = temperal_downsample[::-1]
|
||||
|
||||
# modules
|
||||
self.encoder = Encoder3d(in_channels, dim, z_dim * 2, dim_mult, num_res_blocks,
|
||||
attn_scales, self.temperal_downsample, dropout)
|
||||
self.conv1 = CausalConv3d(z_dim * 2, z_dim * 2, 1)
|
||||
self.conv2 = CausalConv3d(z_dim, z_dim, 1)
|
||||
self.decoder = Decoder3d(out_channels, dim, z_dim, dim_mult, num_res_blocks,
|
||||
attn_scales, self.temperal_upsample, dropout)
|
||||
|
||||
def encode(self, x):
|
||||
conv_idx = [0]
|
||||
feat_map = [None] * count_conv3d(self.decoder)
|
||||
## cache
|
||||
t = x.shape[2]
|
||||
iter_ = 1 + (t - 1) // 4
|
||||
## 对encode输入的x,按时间拆分为1、4、4、4....
|
||||
for i in range(iter_):
|
||||
conv_idx = [0]
|
||||
if i == 0:
|
||||
out = self.encoder(
|
||||
x[:, :, :1, :, :],
|
||||
feat_cache=feat_map,
|
||||
feat_idx=conv_idx)
|
||||
else:
|
||||
out_ = self.encoder(
|
||||
x[:, :, 1 + 4 * (i - 1):1 + 4 * i, :, :],
|
||||
feat_cache=feat_map,
|
||||
feat_idx=conv_idx)
|
||||
out = torch.cat([out, out_], 2)
|
||||
mu, log_var = self.conv1(out).chunk(2, dim=1)
|
||||
return mu
|
||||
|
||||
def decode(self, z):
|
||||
conv_idx = [0]
|
||||
feat_map = [None] * count_conv3d(self.decoder)
|
||||
# z: [b,c,t,h,w]
|
||||
|
||||
iter_ = z.shape[2]
|
||||
x = self.conv2(z)
|
||||
for i in range(iter_):
|
||||
conv_idx = [0]
|
||||
if i == 0:
|
||||
out = self.decoder(
|
||||
x[:, :, i:i + 1, :, :],
|
||||
feat_cache=feat_map,
|
||||
feat_idx=conv_idx)
|
||||
else:
|
||||
out_ = self.decoder(
|
||||
x[:, :, i:i + 1, :, :],
|
||||
feat_cache=feat_map,
|
||||
feat_idx=conv_idx)
|
||||
out = torch.cat([out, out_], 2)
|
||||
return out
|
||||
@@ -22,6 +22,8 @@ Source provenance (owner/repo @ commit — license):
|
||||
ComfyUI-SeedVR2_VideoUpscaler numz/ComfyUI-SeedVR2_VideoUpscaler @ 4490bd1f482e026674543386bb2a4d176da245b9 Apache-2.0
|
||||
ComfyUI_IPAdapter_plus cubiq/ComfyUI_IPAdapter_plus @ a0f451a5 GPL-3.0
|
||||
ComfyUI_UltimateSDUpscale ssitu/ComfyUI_UltimateSDUpscale @ d6b575a (patched) GPL-3.0
|
||||
ComfyUI-ConditioningKrea2Rebalance nova452/ComfyUI-ConditioningKrea2Rebalance @ 9ab5315e6aa8 Apache-2.0
|
||||
ComfyUI-VAE-Utils spacepxl/ComfyUI-VAE-Utils @ d69e7afaa9ed (assets/workflow imgs removed) MIT
|
||||
|
||||
UltimateSDUpscale local patches (in its __init__.py, both preserved on copy-out):
|
||||
1. The bundled Auto1111 repositories/ submodule is shipped pre-filled, so the
|
||||
|
||||
@@ -57,6 +57,9 @@ SECTIONS: list[dict] = [
|
||||
{"id": "face_detailer", "label": "Face Detailer", "size": "~6 MB",
|
||||
"desc": "Region-correct face detailing",
|
||||
"members": [_np("RegionalDetailer")]},
|
||||
{"id": "krea2_highdetail", "label": "Krea 2 High Detail", "size": "~510 MB",
|
||||
"desc": "Clean 2× decoder + conditioning rebalance for the Krea 2 High-Detail templates",
|
||||
"members": [_np("Krea2HighDetail")]},
|
||||
{"id": "controlnet", "label": "ControlNet", "size": "~5 GB SDXL · up to ~22 GB all",
|
||||
"desc": "Depth / pose / canny / edge guidance (SDXL · Flux · Z-Image)",
|
||||
"members": [_np("ControlNet"), _np("FluxControlNet"),
|
||||
|
||||
@@ -98,6 +98,16 @@ async def _api_list_models(request):
|
||||
and model_settings.is_primary_model_file(m["filename"]))
|
||||
or not model_settings.is_catalog_filename(m["filename"])
|
||||
]
|
||||
# The fingerprint scanner assigns architecture from header patterns; newer
|
||||
# DiT families (e.g. krea2) have no pattern yet and read as "unknown", which
|
||||
# drops them from the upscale picker's engine list. Their curated config
|
||||
# (matched by filename) carries the authoritative architecture — adopt it so
|
||||
# the right re-detail engine is offered.
|
||||
for m in filtered:
|
||||
if m.get("architecture") in ("", "unknown", None):
|
||||
config = model_settings.find_config_by_filename(m["filename"])
|
||||
if config and config.get("architecture"):
|
||||
m["architecture"] = config["architecture"]
|
||||
return web.json_response({"models": filtered})
|
||||
|
||||
|
||||
|
||||
@@ -147,6 +147,20 @@ NODE_PACKS: dict[str, dict] = {
|
||||
"dest": "ultralytics/bbox/face_yolov8n.pt"},
|
||||
],
|
||||
},
|
||||
"Krea2HighDetail": {
|
||||
"label": "Krea 2 High Detail",
|
||||
# The High-Detail krea2 templates (t2i + i2i) need two packs: nova452's
|
||||
# ConditioningKrea2Rebalance (a cond amplifier) and spacepxl's VAE-Utils
|
||||
# (its VAEUtils_VAEDecodeTiled + CustomVAELoader run the clean 2x decode).
|
||||
# Neither has pip deps. VAE-Utils ships no model downloader, so we fetch
|
||||
# the spacepxl decoder weight into models/vae for CustomVAELoader to pick.
|
||||
"bundle": ["ComfyUI-ConditioningKrea2Rebalance", "ComfyUI-VAE-Utils"],
|
||||
"proof_nodes": ["ConditioningKrea2Rebalance", "VAEUtils_VAEDecodeTiled"],
|
||||
"models": [
|
||||
{"url": "https://huggingface.co/spacepxl/Wan2.1-VAE-upscale2x/resolve/main/Wan2.1_VAE_upscale2x_imageonly_real_v1.safetensors",
|
||||
"dest": "vae/Wan2.1_VAE_upscale2x_imageonly_real_v1.safetensors"},
|
||||
],
|
||||
},
|
||||
"StyleReference": {
|
||||
"label": "Style Reference",
|
||||
# cubiq's IPAdapter pack is BUNDLED (bundled_packs/ComfyUI_IPAdapter_plus),
|
||||
|
||||
@@ -0,0 +1,164 @@
|
||||
{
|
||||
"display_name": "Krea 2 - RAW",
|
||||
"model_name": "Krea 2 RAW",
|
||||
"version": "RAW",
|
||||
"architecture": "krea2",
|
||||
"family": "krea2_raw",
|
||||
"author": "Krea AI",
|
||||
"description": "Krea 2 RAW — the undistilled 12B base checkpoint (full-step, real cfg). Krea ships it for fine-tuning/LoRA training, but because it is NOT step-distilled it re-details cleanly at a LOW denoise — which makes it the right model for a faithful, blotch-free TILED upscale (the distilled Turbo can't: low denoise blotches, high denoise seams). Same Qwen3-VL encoder + Qwen Image VAE as Turbo.",
|
||||
"license": "Krea 2 Community License Agreement v.1 (open weights; commercial use requires an Enterprise License once total annual revenue reaches $1M)",
|
||||
"tags": [
|
||||
"krea",
|
||||
"krea2",
|
||||
"dit",
|
||||
"transformer",
|
||||
"photorealistic",
|
||||
"raw",
|
||||
"base",
|
||||
"undistilled"
|
||||
],
|
||||
"url": "https://huggingface.co/Comfy-Org/Krea-2",
|
||||
"release_date": "2026-06-18",
|
||||
"tag_sources": [],
|
||||
"tag_format": "spaces",
|
||||
"prompt_style": "natural",
|
||||
"trigger": "",
|
||||
"default_prompt_id": "krea2-default",
|
||||
"negative": "",
|
||||
"notes": "Undistilled base: full-step sampling with a REAL cfg. Render recipe (52 steps / cfg 3.5 / er_sde+simple) is DOCUMENTED — Krea's official CLI is `--checkpoint oss_raw --steps 52 --cfg 3.5`. Negatives use the real CFG branch but efficacy on the Qwen3-VL encoder is A/B-pending (may be inert like Qwen-Image). Shift is baked into the model (no ModelSamplingAuraFlow needed; mu~1.15 at 1024). Shares qwen3vl_4b + qwen_image_vae with Turbo. ~13 GB fp8 / ~26 GB bf16. Slow (no distillation) — for UPSCALE re-detail the JS engine uses fewer steps at low denoise, this 52-step recipe is for from-scratch t2i. RESOLUTION: RAW is DOCUMENTED 'up to 1024' — keep BOTH dims <=1024 (1024x1024 is its sweet spot; long edge >1024 like 1152/1216 melts faces — confirmed). Turbo is the 1k-2k model. For bigger output, render <=1024 then upscale (RAW tiles at 1024 = exactly its trained size, why the tiled upscale is so clean). SHIFT FIX (critical): ComfyUI hardcodes flow-shift mu=1.15 (Turbo's value) for the WHOLE krea2 class, but RAW needs RESOLUTION-DYNAMIC shifting (mu~0.906 @ 1024^2, smaller below) — the over-shift compresses the detail tail and melts faces. The raw t2i/i2i templates insert a ModelSamplingFlux node (base_shift 0.5, max_shift 0.90625, width/height = render size) which REPLACES the bad baked shift (verified against ComfyUI master: supported_models.py Krea2 shift=1.15, ModelType.FLUX). SHIP RECIPE (quality-research-tuned): 52 steps / cfg 5.0 / er_sde / beta — DOCUMENTED crispest-photoreal combo; euler/simple is doubly-soft (euler smooths micro-texture, simple under-weights the detail band). cfg 5.0 ~ Krea --cfg 3.5 in ComfyUI's (1+g) convention + diffusers' native 4.5. ModelSamplingFlux width/height MUST match render size or mu over-shifts on non-square buckets (1024^2->0.906, 896x1024->0.852, 832x1024->0.825, 768x1024->0.798). Turbo needs no shift node (its real mu IS 1.15). Without the ModelSamplingFlux node a RAW render melts.",
|
||||
"files": [
|
||||
{
|
||||
"label": "Diffusion Model",
|
||||
"folder": "diffusion_models",
|
||||
"variants": [
|
||||
{
|
||||
"precision": "fp8",
|
||||
"filename": "krea2_raw_fp8_scaled.safetensors",
|
||||
"size_bytes": 13141730784,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "diffusion_models/krea2_raw_fp8_scaled.safetensors"
|
||||
},
|
||||
"quick_hash": null,
|
||||
"sha256": null
|
||||
},
|
||||
{
|
||||
"precision": "bf16",
|
||||
"filename": "krea2_raw_bf16.safetensors",
|
||||
"size_bytes": 26283332608,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "diffusion_models/krea2_raw_bf16.safetensors"
|
||||
},
|
||||
"quick_hash": null,
|
||||
"sha256": null
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"label": "Text Encoder",
|
||||
"folder": "text_encoders",
|
||||
"variants": [
|
||||
{
|
||||
"precision": "fp8",
|
||||
"filename": "qwen3vl_4b_fp8_scaled.safetensors",
|
||||
"size_bytes": 5242467968,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "text_encoders/qwen3vl_4b_fp8_scaled.safetensors"
|
||||
},
|
||||
"sha256": null
|
||||
},
|
||||
{
|
||||
"precision": "bf16",
|
||||
"filename": "qwen3vl_4b_bf16.safetensors",
|
||||
"size_bytes": 8875719384,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "text_encoders/qwen3vl_4b_bf16.safetensors"
|
||||
},
|
||||
"sha256": null
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"label": "VAE",
|
||||
"folder": "vae",
|
||||
"filename": "qwen_image_vae.safetensors",
|
||||
"size_bytes": 253806246,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "vae/qwen_image_vae.safetensors"
|
||||
},
|
||||
"sha256": null
|
||||
}
|
||||
],
|
||||
"nodes": {
|
||||
"KSampler": {
|
||||
"steps": 52,
|
||||
"cfg": 5.0,
|
||||
"sampler_name": "er_sde",
|
||||
"scheduler": "beta",
|
||||
"steps_range": [
|
||||
40,
|
||||
60
|
||||
],
|
||||
"cfg_range": [
|
||||
3.5,
|
||||
6.0
|
||||
],
|
||||
"sampler_name_options": [
|
||||
"er_sde",
|
||||
"dpmpp_2m",
|
||||
"euler",
|
||||
"dpmpp_sde",
|
||||
"euler_ancestral"
|
||||
],
|
||||
"scheduler_options": [
|
||||
"beta",
|
||||
"simple",
|
||||
"sgm_uniform",
|
||||
"normal"
|
||||
]
|
||||
},
|
||||
"EmptySD3LatentImage": {
|
||||
"width": 1024,
|
||||
"height": 1024,
|
||||
"_resolutions": [
|
||||
{
|
||||
"width": 1024,
|
||||
"height": 1024
|
||||
},
|
||||
{
|
||||
"width": 832,
|
||||
"height": 1024
|
||||
},
|
||||
{
|
||||
"width": 1024,
|
||||
"height": 832
|
||||
},
|
||||
{
|
||||
"width": 768,
|
||||
"height": 1024
|
||||
},
|
||||
{
|
||||
"width": 1024,
|
||||
"height": 768
|
||||
},
|
||||
{
|
||||
"width": 896,
|
||||
"height": 1024
|
||||
},
|
||||
{
|
||||
"width": 1024,
|
||||
"height": 896
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,192 @@
|
||||
{
|
||||
"display_name": "Krea 2 - Turbo",
|
||||
"model_name": "Krea 2 Turbo",
|
||||
"version": "Turbo",
|
||||
"architecture": "krea2",
|
||||
"family": "krea2_turbo",
|
||||
"author": "Krea AI",
|
||||
"description": "Krea 2 Turbo — a distilled 12B dense Diffusion Transformer for fast, high-quality, aesthetic-first text-to-image at native 2K. Uses a Qwen3-VL text encoder and the Qwen Image VAE. 8-step, cfg 1.0. nvfp4 variant is the fastest on Blackwell (RTX 50-series) native FP4.",
|
||||
"license": "Krea 2 Community License Agreement v.1 (open weights; commercial use requires an Enterprise License once total annual revenue reaches $1M)",
|
||||
"tags": [
|
||||
"krea",
|
||||
"krea2",
|
||||
"dit",
|
||||
"transformer",
|
||||
"photorealistic",
|
||||
"aesthetic",
|
||||
"turbo",
|
||||
"fast"
|
||||
],
|
||||
"url": "https://huggingface.co/Comfy-Org/Krea-2",
|
||||
"release_date": "2026-06-18",
|
||||
"tag_sources": [],
|
||||
"tag_format": "spaces",
|
||||
"prompt_style": "natural",
|
||||
"trigger": "",
|
||||
"default_prompt_id": "krea2-default",
|
||||
"negative": "",
|
||||
"notes": "8 steps, cfg 1.0, euler+simple (the official Krea recipe — default; er_sde+simple is available for extra grain on photoreal skin), denoise 1.0. Distilled turbo — keep cfg at 1.0 (no real negative). Native 2K (supports 1K-2K). Reuses the Qwen Image VAE (qwen_image_vae.safetensors). Requires a ComfyUI build with the 'krea2' CLIP type (Krea 2 release, June 2026 or newer). RAW-trained Krea 2 LoRAs express on Turbo.",
|
||||
"files": [
|
||||
{
|
||||
"label": "Diffusion Model",
|
||||
"folder": "diffusion_models",
|
||||
"variants": [
|
||||
{
|
||||
"precision": "fp8",
|
||||
"filename": "krea2_turbo_fp8_scaled.safetensors",
|
||||
"size_bytes": 13141730784,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "diffusion_models/krea2_turbo_fp8_scaled.safetensors"
|
||||
},
|
||||
"quick_hash": null,
|
||||
"sha256": null
|
||||
},
|
||||
{
|
||||
"precision": "bf16",
|
||||
"filename": "krea2_turbo_bf16.safetensors",
|
||||
"size_bytes": 26283332608,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "diffusion_models/krea2_turbo_bf16.safetensors"
|
||||
},
|
||||
"quick_hash": null,
|
||||
"sha256": null
|
||||
},
|
||||
{
|
||||
"precision": "mxfp8",
|
||||
"filename": "krea2_turbo_mxfp8.safetensors",
|
||||
"size_bytes": 13532318080,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "diffusion_models/krea2_turbo_mxfp8.safetensors"
|
||||
},
|
||||
"quick_hash": null,
|
||||
"sha256": null
|
||||
},
|
||||
{
|
||||
"precision": "nvfp4",
|
||||
"filename": "krea2_turbo_nvfp4.safetensors",
|
||||
"size_bytes": 7673668288,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "diffusion_models/krea2_turbo_nvfp4.safetensors"
|
||||
},
|
||||
"quick_hash": null,
|
||||
"sha256": null
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"label": "Text Encoder",
|
||||
"folder": "text_encoders",
|
||||
"variants": [
|
||||
{
|
||||
"precision": "fp8",
|
||||
"filename": "qwen3vl_4b_fp8_scaled.safetensors",
|
||||
"size_bytes": 5242467968,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "text_encoders/qwen3vl_4b_fp8_scaled.safetensors"
|
||||
},
|
||||
"sha256": null
|
||||
},
|
||||
{
|
||||
"precision": "bf16",
|
||||
"filename": "qwen3vl_4b_bf16.safetensors",
|
||||
"size_bytes": 8875719384,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "text_encoders/qwen3vl_4b_bf16.safetensors"
|
||||
},
|
||||
"sha256": null
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"label": "VAE",
|
||||
"folder": "vae",
|
||||
"filename": "qwen_image_vae.safetensors",
|
||||
"size_bytes": 253806246,
|
||||
"source": {
|
||||
"provider": "huggingface",
|
||||
"repo": "Comfy-Org/Krea-2",
|
||||
"path": "vae/qwen_image_vae.safetensors"
|
||||
},
|
||||
"sha256": null
|
||||
}
|
||||
],
|
||||
"nodes": {
|
||||
"KSampler": {
|
||||
"steps": 8,
|
||||
"cfg": 1.0,
|
||||
"sampler_name": "euler",
|
||||
"scheduler": "simple",
|
||||
"steps_range": [
|
||||
6,
|
||||
12
|
||||
],
|
||||
"cfg_range": [
|
||||
1.0,
|
||||
1.5
|
||||
],
|
||||
"sampler_name_options": [
|
||||
"er_sde",
|
||||
"euler",
|
||||
"res_multistep",
|
||||
"dpmpp_2m",
|
||||
"euler_ancestral"
|
||||
],
|
||||
"scheduler_options": [
|
||||
"simple",
|
||||
"beta",
|
||||
"sgm_uniform",
|
||||
"normal"
|
||||
]
|
||||
},
|
||||
"EmptySD3LatentImage": {
|
||||
"width": 1024,
|
||||
"height": 1536,
|
||||
"_resolutions": [
|
||||
{
|
||||
"width": 1024,
|
||||
"height": 1536
|
||||
},
|
||||
{
|
||||
"width": 1536,
|
||||
"height": 1024
|
||||
},
|
||||
{
|
||||
"width": 1024,
|
||||
"height": 1024
|
||||
},
|
||||
{
|
||||
"width": 1152,
|
||||
"height": 1536
|
||||
},
|
||||
{
|
||||
"width": 1536,
|
||||
"height": 1152
|
||||
},
|
||||
{
|
||||
"width": 1408,
|
||||
"height": 1408
|
||||
},
|
||||
{
|
||||
"width": 1152,
|
||||
"height": 2048
|
||||
},
|
||||
{
|
||||
"width": 2048,
|
||||
"height": 1152
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Binary file not shown.
@@ -0,0 +1,343 @@
|
||||
{
|
||||
"name": "Image-to-Image (High Detail)",
|
||||
"category": "Generation",
|
||||
"order": 7,
|
||||
"scope": {
|
||||
"type": "family",
|
||||
"architecture": "krea2",
|
||||
"family": "krea2_raw"
|
||||
},
|
||||
"created_at": 1782371000,
|
||||
"id": "krea2-raw-i2i-highdetail",
|
||||
"nodes": [
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
900.0,
|
||||
0.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
900.0,
|
||||
180.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "KSampler",
|
||||
"offset": [
|
||||
1740.0,
|
||||
160.0
|
||||
],
|
||||
"size": [
|
||||
330,
|
||||
474
|
||||
],
|
||||
"widgets_values": [
|
||||
826502184750291,
|
||||
"randomize",
|
||||
40,
|
||||
5.0,
|
||||
"er_sde",
|
||||
"beta",
|
||||
0.6
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "KSampler"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEUtils_VAEDecodeTiled",
|
||||
"offset": [
|
||||
1790.0,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
220
|
||||
],
|
||||
"widgets_values": [
|
||||
-1,
|
||||
false,
|
||||
512,
|
||||
64,
|
||||
4096,
|
||||
64
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEUtils_VAEDecodeTiled"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "SaveImage",
|
||||
"offset": [
|
||||
2150.0,
|
||||
-10.0
|
||||
],
|
||||
"size": [
|
||||
276.984375,
|
||||
322.984375
|
||||
],
|
||||
"widgets_values": [
|
||||
"Krea2-RAW/img2img-highdetail/%date:yyyy-MM-dd_hhmmss%"
|
||||
],
|
||||
"properties": {}
|
||||
},
|
||||
{
|
||||
"type": "CLIPLoader",
|
||||
"offset": [
|
||||
860.0,
|
||||
360.0
|
||||
],
|
||||
"size": [
|
||||
346.78125,
|
||||
117.328125
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen3vl_4b_fp8_scaled.safetensors",
|
||||
"krea2",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "UNETLoader",
|
||||
"offset": [
|
||||
1240.0,
|
||||
0.0
|
||||
],
|
||||
"size": [
|
||||
371.65625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"krea2_raw_bf16.safetensors",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "UNETLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "LoadImage",
|
||||
"offset": [
|
||||
1240.0,
|
||||
220.0
|
||||
],
|
||||
"size": [
|
||||
270,
|
||||
340
|
||||
],
|
||||
"widgets_values": [
|
||||
"",
|
||||
"image"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "LoadImage"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEUtils_CustomVAELoader",
|
||||
"offset": [
|
||||
1740.0,
|
||||
680.0
|
||||
],
|
||||
"size": [
|
||||
366.40625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"Wan2.1_VAE_upscale2x_imageonly_real_v1.safetensors",
|
||||
true
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEUtils_CustomVAELoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "ConditioningKrea2Rebalance",
|
||||
"offset": [
|
||||
1180.0,
|
||||
0.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
106
|
||||
],
|
||||
"widgets_values": [
|
||||
2.0,
|
||||
"1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.6,2.5,1.05,2.0,1.0"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "ConditioningKrea2Rebalance"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "ModelSamplingFlux",
|
||||
"offset": [
|
||||
1460.0,
|
||||
90.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
130
|
||||
],
|
||||
"widgets_values": [
|
||||
0.90625,
|
||||
0.5,
|
||||
1024,
|
||||
1024
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "ModelSamplingFlux"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEEncode",
|
||||
"offset": [
|
||||
1560.0,
|
||||
600.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
72
|
||||
],
|
||||
"widgets_values": [],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAELoader",
|
||||
"offset": [
|
||||
1180.0,
|
||||
600.0
|
||||
],
|
||||
"size": [
|
||||
366.40625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen_image_vae.safetensors"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAELoader"
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": [
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 9,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 9,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 1,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 2
|
||||
},
|
||||
{
|
||||
"from_node_idx": 6,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 10,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 10,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 11,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 3
|
||||
},
|
||||
{
|
||||
"from_node_idx": 7,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 11,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 12,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 11,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 2,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 8,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 3,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 4,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 0
|
||||
}
|
||||
],
|
||||
"anchorConnections": [
|
||||
{
|
||||
"from_slot": 1,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_slot": 2,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,294 @@
|
||||
{
|
||||
"name": "Image-to-Image (RAW base)",
|
||||
"category": "Generation",
|
||||
"order": 5,
|
||||
"scope": {
|
||||
"type": "family",
|
||||
"architecture": "krea2",
|
||||
"family": "krea2_raw"
|
||||
},
|
||||
"created_at": 1782328500,
|
||||
"id": "krea2-raw-i2i",
|
||||
"nodes": [
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
899.9999977111815,
|
||||
0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
899.9999977111815,
|
||||
180.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "KSampler",
|
||||
"offset": [
|
||||
1699.9999685974124,
|
||||
160.0
|
||||
],
|
||||
"size": [
|
||||
330,
|
||||
474
|
||||
],
|
||||
"widgets_values": [
|
||||
826502184750291,
|
||||
"randomize",
|
||||
40,
|
||||
5.0,
|
||||
"er_sde",
|
||||
"beta",
|
||||
0.6
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "KSampler"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEDecode",
|
||||
"offset": [
|
||||
1750.0000429382326,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
72
|
||||
],
|
||||
"widgets_values": [],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEDecode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "SaveImage",
|
||||
"offset": [
|
||||
2090.0000285339356,
|
||||
-10.0
|
||||
],
|
||||
"size": [
|
||||
276.984375,
|
||||
322.984375
|
||||
],
|
||||
"widgets_values": [
|
||||
"Krea2-RAW/img2img/%date:yyyy-MM-dd_hhmmss%"
|
||||
],
|
||||
"properties": {}
|
||||
},
|
||||
{
|
||||
"type": "CLIPLoader",
|
||||
"offset": [
|
||||
860.0000519714356,
|
||||
340.0
|
||||
],
|
||||
"size": [
|
||||
346.78125,
|
||||
117.328125
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen3vl_4b_fp8_scaled.safetensors",
|
||||
"krea2",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "UNETLoader",
|
||||
"offset": [
|
||||
1239.9999069433584,
|
||||
0
|
||||
],
|
||||
"size": [
|
||||
371.65625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"krea2_raw_fp8_scaled.safetensors",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "UNETLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "LoadImage",
|
||||
"offset": [
|
||||
1240.0,
|
||||
210.0
|
||||
],
|
||||
"size": [
|
||||
270,
|
||||
340
|
||||
],
|
||||
"widgets_values": [
|
||||
"",
|
||||
"image"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "LoadImage"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAELoader",
|
||||
"offset": [
|
||||
1239.993419293212,
|
||||
590.0
|
||||
],
|
||||
"size": [
|
||||
366.40625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen_image_vae.safetensors"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAELoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEEncode",
|
||||
"offset": [
|
||||
1610.0,
|
||||
210.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
72
|
||||
],
|
||||
"widgets_values": [],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "ModelSamplingFlux",
|
||||
"offset": [
|
||||
1450.0,
|
||||
90.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
130
|
||||
],
|
||||
"widgets_values": [
|
||||
0.90625,
|
||||
0.5,
|
||||
1024,
|
||||
1024
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "ModelSamplingFlux"
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": [
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 1,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 2
|
||||
},
|
||||
{
|
||||
"from_node_idx": 2,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 3,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 4,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 6,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 10,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 10,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 8,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 9,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 3
|
||||
},
|
||||
{
|
||||
"from_node_idx": 7,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 9,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 8,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 9,
|
||||
"to_slot": 1
|
||||
}
|
||||
],
|
||||
"anchorConnections": [
|
||||
{
|
||||
"from_slot": 1,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_slot": 2,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,300 @@
|
||||
{
|
||||
"name": "Text-to-Image (High Detail)",
|
||||
"category": "Generation",
|
||||
"order": 6,
|
||||
"scope": {
|
||||
"type": "family",
|
||||
"architecture": "krea2",
|
||||
"family": "krea2_raw"
|
||||
},
|
||||
"created_at": 1782370000,
|
||||
"id": "krea2-raw-t2i-highdetail",
|
||||
"nodes": [
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
900.0,
|
||||
0.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
900.0,
|
||||
180.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "KSampler",
|
||||
"offset": [
|
||||
1740.0,
|
||||
160.0
|
||||
],
|
||||
"size": [
|
||||
330,
|
||||
474
|
||||
],
|
||||
"widgets_values": [
|
||||
826502184750291,
|
||||
"randomize",
|
||||
52,
|
||||
5.0,
|
||||
"er_sde",
|
||||
"beta",
|
||||
1
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "KSampler"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEUtils_VAEDecodeTiled",
|
||||
"offset": [
|
||||
1790.0,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
220
|
||||
],
|
||||
"widgets_values": [
|
||||
-1,
|
||||
false,
|
||||
512,
|
||||
64,
|
||||
4096,
|
||||
64
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEUtils_VAEDecodeTiled"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "SaveImage",
|
||||
"offset": [
|
||||
2150.0,
|
||||
-10.0
|
||||
],
|
||||
"size": [
|
||||
276.984375,
|
||||
322.984375
|
||||
],
|
||||
"widgets_values": [
|
||||
"Krea2-RAW/highdetail/%date:yyyy-MM-dd_hhmmss%"
|
||||
],
|
||||
"properties": {}
|
||||
},
|
||||
{
|
||||
"type": "CLIPLoader",
|
||||
"offset": [
|
||||
860.0,
|
||||
360.0
|
||||
],
|
||||
"size": [
|
||||
346.78125,
|
||||
117.328125
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen3vl_4b_fp8_scaled.safetensors",
|
||||
"krea2",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "UNETLoader",
|
||||
"offset": [
|
||||
1240.0,
|
||||
0.0
|
||||
],
|
||||
"size": [
|
||||
371.65625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"krea2_raw_bf16.safetensors",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "UNETLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "EmptySD3LatentImage",
|
||||
"offset": [
|
||||
1250.0,
|
||||
220.0
|
||||
],
|
||||
"size": [
|
||||
324,
|
||||
144
|
||||
],
|
||||
"widgets_values": [
|
||||
1024,
|
||||
1024,
|
||||
1
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "EmptySD3LatentImage"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEUtils_CustomVAELoader",
|
||||
"offset": [
|
||||
1240.0,
|
||||
470.0
|
||||
],
|
||||
"size": [
|
||||
366.40625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"Wan2.1_VAE_upscale2x_imageonly_real_v1.safetensors",
|
||||
true
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEUtils_CustomVAELoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "ConditioningKrea2Rebalance",
|
||||
"offset": [
|
||||
1180.0,
|
||||
0.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
106
|
||||
],
|
||||
"widgets_values": [
|
||||
2.0,
|
||||
"1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.6,2.5,1.05,2.0,1.0"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "ConditioningKrea2Rebalance"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "ModelSamplingFlux",
|
||||
"offset": [
|
||||
1460.0,
|
||||
90.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
130
|
||||
],
|
||||
"widgets_values": [
|
||||
0.90625,
|
||||
0.5,
|
||||
1024,
|
||||
1024
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "ModelSamplingFlux"
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": [
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 9,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 9,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 1,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 2
|
||||
},
|
||||
{
|
||||
"from_node_idx": 6,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 10,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 10,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 7,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 3
|
||||
},
|
||||
{
|
||||
"from_node_idx": 2,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 8,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 3,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 4,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 0
|
||||
}
|
||||
],
|
||||
"anchorConnections": [
|
||||
{
|
||||
"from_slot": 1,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_slot": 2,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,268 @@
|
||||
{
|
||||
"name": "Text-to-Image (RAW base)",
|
||||
"category": "Generation",
|
||||
"order": 4,
|
||||
"scope": {
|
||||
"type": "family",
|
||||
"architecture": "krea2",
|
||||
"family": "krea2_raw"
|
||||
},
|
||||
"created_at": 1782328400,
|
||||
"id": "c5a9e2f0-1b3d-4e6a-8c7f-9d2b4a6e1f08",
|
||||
"nodes": [
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
899.9999977111815,
|
||||
0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
899.9999977111815,
|
||||
180.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "KSampler",
|
||||
"offset": [
|
||||
1699.9999685974124,
|
||||
160.0
|
||||
],
|
||||
"size": [
|
||||
330,
|
||||
474
|
||||
],
|
||||
"widgets_values": [
|
||||
826502184750291,
|
||||
"randomize",
|
||||
52,
|
||||
5.0,
|
||||
"er_sde",
|
||||
"beta",
|
||||
1
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "KSampler"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEDecode",
|
||||
"offset": [
|
||||
1750.0000429382326,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
72
|
||||
],
|
||||
"widgets_values": [],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEDecode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "SaveImage",
|
||||
"offset": [
|
||||
2090.0000285339356,
|
||||
-10.0
|
||||
],
|
||||
"size": [
|
||||
276.984375,
|
||||
322.984375
|
||||
],
|
||||
"widgets_values": [
|
||||
"Krea2-RAW/%date:yyyy-MM-dd_hhmmss%"
|
||||
],
|
||||
"properties": {}
|
||||
},
|
||||
{
|
||||
"type": "CLIPLoader",
|
||||
"offset": [
|
||||
860.0000519714356,
|
||||
340.0
|
||||
],
|
||||
"size": [
|
||||
346.78125,
|
||||
117.328125
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen3vl_4b_fp8_scaled.safetensors",
|
||||
"krea2",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "UNETLoader",
|
||||
"offset": [
|
||||
1239.9999069433584,
|
||||
0
|
||||
],
|
||||
"size": [
|
||||
371.65625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"krea2_raw_fp8_scaled.safetensors",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "UNETLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "EmptySD3LatentImage",
|
||||
"offset": [
|
||||
1249.9999315600576,
|
||||
200.0
|
||||
],
|
||||
"size": [
|
||||
324,
|
||||
144
|
||||
],
|
||||
"widgets_values": [
|
||||
1024,
|
||||
1024,
|
||||
1
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "EmptySD3LatentImage"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAELoader",
|
||||
"offset": [
|
||||
1239.993419293212,
|
||||
450.0
|
||||
],
|
||||
"size": [
|
||||
366.40625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen_image_vae.safetensors"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAELoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "ModelSamplingFlux",
|
||||
"offset": [
|
||||
1450.0,
|
||||
90.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
130
|
||||
],
|
||||
"widgets_values": [
|
||||
0.90625,
|
||||
0.5,
|
||||
1024,
|
||||
1024
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "ModelSamplingFlux"
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": [
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 1,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 2
|
||||
},
|
||||
{
|
||||
"from_node_idx": 2,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 3,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 4,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 6,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 9,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 9,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 7,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 3
|
||||
},
|
||||
{
|
||||
"from_node_idx": 8,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 1
|
||||
}
|
||||
],
|
||||
"anchorConnections": [
|
||||
{
|
||||
"from_slot": 1,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_slot": 2,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,289 @@
|
||||
{
|
||||
"name": "Image-to-Image (High Detail)",
|
||||
"category": "Generation",
|
||||
"order": 4,
|
||||
"scope": {
|
||||
"type": "family",
|
||||
"architecture": "krea2",
|
||||
"family": "krea2_turbo"
|
||||
},
|
||||
"created_at": 1782364000,
|
||||
"id": "b7c3e8a1-4f2d-4a6b-9c0e-2a1f6d3b9f0a",
|
||||
"nodes": [
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
820.0,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "KSampler",
|
||||
"offset": [
|
||||
1440.0,
|
||||
120.0
|
||||
],
|
||||
"size": [
|
||||
330,
|
||||
474
|
||||
],
|
||||
"widgets_values": [
|
||||
826502184750291,
|
||||
"randomize",
|
||||
8,
|
||||
1,
|
||||
"euler",
|
||||
"simple",
|
||||
0.6
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "KSampler"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEUtils_VAEDecodeTiled",
|
||||
"offset": [
|
||||
1800.0,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
220
|
||||
],
|
||||
"widgets_values": [
|
||||
-1,
|
||||
false,
|
||||
512,
|
||||
64,
|
||||
4096,
|
||||
64
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEUtils_VAEDecodeTiled"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "SaveImage",
|
||||
"offset": [
|
||||
2140.0,
|
||||
-10.0
|
||||
],
|
||||
"size": [
|
||||
276.984375,
|
||||
322.984375
|
||||
],
|
||||
"widgets_values": [
|
||||
"Krea2/img2img-highdetail/%date:yyyy-MM-dd_hhmmss%"
|
||||
],
|
||||
"properties": {}
|
||||
},
|
||||
{
|
||||
"type": "CLIPLoader",
|
||||
"offset": [
|
||||
820.0,
|
||||
240.0
|
||||
],
|
||||
"size": [
|
||||
346.78125,
|
||||
117.328125
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen3vl_4b_fp8_scaled.safetensors",
|
||||
"krea2",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "UNETLoader",
|
||||
"offset": [
|
||||
1100.0,
|
||||
280.0
|
||||
],
|
||||
"size": [
|
||||
371.65625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"krea2_turbo_bf16.safetensors",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "UNETLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "LoadImage",
|
||||
"offset": [
|
||||
780.0,
|
||||
420.0
|
||||
],
|
||||
"size": [
|
||||
270,
|
||||
340
|
||||
],
|
||||
"widgets_values": [
|
||||
"",
|
||||
"image"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "LoadImage"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEUtils_CustomVAELoader",
|
||||
"offset": [
|
||||
1440.0,
|
||||
620.0
|
||||
],
|
||||
"size": [
|
||||
366.40625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"Wan2.1_VAE_upscale2x_imageonly_real_v1.safetensors",
|
||||
true
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEUtils_CustomVAELoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "ConditioningKrea2Rebalance",
|
||||
"offset": [
|
||||
1100.0,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
106
|
||||
],
|
||||
"widgets_values": [
|
||||
2.0,
|
||||
"1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.6,2.5,1.05,2.0,1.0"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "ConditioningKrea2Rebalance"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEEncode",
|
||||
"offset": [
|
||||
1130.0,
|
||||
500.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
72
|
||||
],
|
||||
"widgets_values": [],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAELoader",
|
||||
"offset": [
|
||||
780.0,
|
||||
790.0
|
||||
],
|
||||
"size": [
|
||||
366.40625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen_image_vae.safetensors"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAELoader"
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": [
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 8,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 8,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 2
|
||||
},
|
||||
{
|
||||
"from_node_idx": 1,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 7,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 2,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 4,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 9,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 3
|
||||
},
|
||||
{
|
||||
"from_node_idx": 6,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 9,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 10,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 9,
|
||||
"to_slot": 1
|
||||
}
|
||||
],
|
||||
"anchorConnections": [
|
||||
{
|
||||
"from_slot": 1,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,240 @@
|
||||
{
|
||||
"name": "Image-to-Image",
|
||||
"category": "Generation",
|
||||
"order": 1,
|
||||
"scope": {
|
||||
"type": "family",
|
||||
"architecture": "krea2",
|
||||
"family": "krea2_turbo"
|
||||
},
|
||||
"created_at": 1782327800,
|
||||
"id": "krea2-turbo-i2i",
|
||||
"nodes": [
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
899.9999977111815,
|
||||
0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "KSampler",
|
||||
"offset": [
|
||||
1699.9999685974124,
|
||||
160.0
|
||||
],
|
||||
"size": [
|
||||
330,
|
||||
474
|
||||
],
|
||||
"widgets_values": [
|
||||
826502184750291,
|
||||
"randomize",
|
||||
8,
|
||||
1,
|
||||
"euler",
|
||||
"simple",
|
||||
0.6
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "KSampler"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEDecode",
|
||||
"offset": [
|
||||
1750.0000429382326,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
72
|
||||
],
|
||||
"widgets_values": [],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEDecode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "SaveImage",
|
||||
"offset": [
|
||||
2090.0000285339356,
|
||||
-10.0
|
||||
],
|
||||
"size": [
|
||||
276.984375,
|
||||
322.984375
|
||||
],
|
||||
"widgets_values": [
|
||||
"Krea2/img2img/%date:yyyy-MM-dd_hhmmss%"
|
||||
],
|
||||
"properties": {}
|
||||
},
|
||||
{
|
||||
"type": "CLIPLoader",
|
||||
"offset": [
|
||||
860.0000519714356,
|
||||
200.0
|
||||
],
|
||||
"size": [
|
||||
346.78125,
|
||||
117.328125
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen3vl_4b_fp8_scaled.safetensors",
|
||||
"krea2",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "UNETLoader",
|
||||
"offset": [
|
||||
1239.9999069433584,
|
||||
0
|
||||
],
|
||||
"size": [
|
||||
371.65625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"krea2_turbo_fp8_scaled.safetensors",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "UNETLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "LoadImage",
|
||||
"offset": [
|
||||
1240.0,
|
||||
210.0
|
||||
],
|
||||
"size": [
|
||||
270,
|
||||
340
|
||||
],
|
||||
"widgets_values": [
|
||||
"",
|
||||
"image"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "LoadImage"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAELoader",
|
||||
"offset": [
|
||||
1239.993419293212,
|
||||
390.0
|
||||
],
|
||||
"size": [
|
||||
366.40625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen_image_vae.safetensors"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAELoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEEncode",
|
||||
"offset": [
|
||||
1610.0,
|
||||
210.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
72
|
||||
],
|
||||
"widgets_values": [],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEEncode"
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": [
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 2
|
||||
},
|
||||
{
|
||||
"from_node_idx": 1,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 2,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 4,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 7,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 8,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 3
|
||||
},
|
||||
{
|
||||
"from_node_idx": 6,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 8,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 7,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 8,
|
||||
"to_slot": 1
|
||||
}
|
||||
],
|
||||
"anchorConnections": [
|
||||
{
|
||||
"from_slot": 1,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,246 @@
|
||||
{
|
||||
"name": "Text-to-Image (High Detail)",
|
||||
"category": "Generation",
|
||||
"order": 3,
|
||||
"scope": {
|
||||
"type": "family",
|
||||
"architecture": "krea2",
|
||||
"family": "krea2_turbo"
|
||||
},
|
||||
"created_at": 1782362000,
|
||||
"id": "b7c3e8a1-4f2d-4a6b-9c0e-2a1f6d3b9f09",
|
||||
"nodes": [
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
820.0,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "KSampler",
|
||||
"offset": [
|
||||
1440.0,
|
||||
120.0
|
||||
],
|
||||
"size": [
|
||||
330,
|
||||
474
|
||||
],
|
||||
"widgets_values": [
|
||||
826502184750291,
|
||||
"randomize",
|
||||
8,
|
||||
1,
|
||||
"euler",
|
||||
"simple",
|
||||
1
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "KSampler"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEUtils_VAEDecodeTiled",
|
||||
"offset": [
|
||||
1800.0,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
220
|
||||
],
|
||||
"widgets_values": [
|
||||
-1,
|
||||
false,
|
||||
512,
|
||||
64,
|
||||
4096,
|
||||
64
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEUtils_VAEDecodeTiled"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "SaveImage",
|
||||
"offset": [
|
||||
2140.0,
|
||||
-10.0
|
||||
],
|
||||
"size": [
|
||||
276.984375,
|
||||
322.984375
|
||||
],
|
||||
"widgets_values": [
|
||||
"Krea2/highdetail/%date:yyyy-MM-dd_hhmmss%"
|
||||
],
|
||||
"properties": {}
|
||||
},
|
||||
{
|
||||
"type": "CLIPLoader",
|
||||
"offset": [
|
||||
820.0,
|
||||
240.0
|
||||
],
|
||||
"size": [
|
||||
346.78125,
|
||||
117.328125
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen3vl_4b_fp8_scaled.safetensors",
|
||||
"krea2",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "UNETLoader",
|
||||
"offset": [
|
||||
1100.0,
|
||||
280.0
|
||||
],
|
||||
"size": [
|
||||
371.65625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"krea2_turbo_bf16.safetensors",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "UNETLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "EmptySD3LatentImage",
|
||||
"offset": [
|
||||
1100.0,
|
||||
420.0
|
||||
],
|
||||
"size": [
|
||||
324,
|
||||
144
|
||||
],
|
||||
"widgets_values": [
|
||||
1024,
|
||||
1536,
|
||||
1
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "EmptySD3LatentImage"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEUtils_CustomVAELoader",
|
||||
"offset": [
|
||||
1440.0,
|
||||
620.0
|
||||
],
|
||||
"size": [
|
||||
366.40625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"Wan2.1_VAE_upscale2x_imageonly_real_v1.safetensors",
|
||||
true
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEUtils_CustomVAELoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "ConditioningKrea2Rebalance",
|
||||
"offset": [
|
||||
1100.0,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
300,
|
||||
106
|
||||
],
|
||||
"widgets_values": [
|
||||
2.0,
|
||||
"1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.6,2.5,1.05,2.0,1.0"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "ConditioningKrea2Rebalance"
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": [
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 8,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 8,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 2
|
||||
},
|
||||
{
|
||||
"from_node_idx": 1,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 7,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 2,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 4,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 6,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 3
|
||||
}
|
||||
],
|
||||
"anchorConnections": [
|
||||
{
|
||||
"from_slot": 1,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,214 @@
|
||||
{
|
||||
"name": "Text-to-Image",
|
||||
"category": "Generation",
|
||||
"order": 0,
|
||||
"scope": {
|
||||
"type": "family",
|
||||
"architecture": "krea2",
|
||||
"family": "krea2_turbo"
|
||||
},
|
||||
"created_at": 1782327800,
|
||||
"id": "b7c3e8a1-4f2d-4a6b-9c0e-2a1f6d3b9e07",
|
||||
"nodes": [
|
||||
{
|
||||
"type": "CLIPTextEncode",
|
||||
"offset": [
|
||||
899.9999977111815,
|
||||
0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
116
|
||||
],
|
||||
"widgets_values": [
|
||||
""
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "KSampler",
|
||||
"offset": [
|
||||
1699.9999685974124,
|
||||
160.0
|
||||
],
|
||||
"size": [
|
||||
330,
|
||||
474
|
||||
],
|
||||
"widgets_values": [
|
||||
826502184750291,
|
||||
"randomize",
|
||||
8,
|
||||
1,
|
||||
"euler",
|
||||
"simple",
|
||||
1
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "KSampler"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAEDecode",
|
||||
"offset": [
|
||||
1750.0000429382326,
|
||||
20.0
|
||||
],
|
||||
"size": [
|
||||
225,
|
||||
72
|
||||
],
|
||||
"widgets_values": [],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEDecode"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "SaveImage",
|
||||
"offset": [
|
||||
2090.0000285339356,
|
||||
-10.0
|
||||
],
|
||||
"size": [
|
||||
276.984375,
|
||||
322.984375
|
||||
],
|
||||
"widgets_values": [
|
||||
"Krea2/%date:yyyy-MM-dd_hhmmss%"
|
||||
],
|
||||
"properties": {}
|
||||
},
|
||||
{
|
||||
"type": "CLIPLoader",
|
||||
"offset": [
|
||||
860.0000519714356,
|
||||
200.0
|
||||
],
|
||||
"size": [
|
||||
346.78125,
|
||||
117.328125
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen3vl_4b_fp8_scaled.safetensors",
|
||||
"krea2",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "UNETLoader",
|
||||
"offset": [
|
||||
1239.9999069433584,
|
||||
0
|
||||
],
|
||||
"size": [
|
||||
371.65625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"krea2_turbo_fp8_scaled.safetensors",
|
||||
"default"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "UNETLoader"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "EmptySD3LatentImage",
|
||||
"offset": [
|
||||
1249.9999315600576,
|
||||
200.0
|
||||
],
|
||||
"size": [
|
||||
324,
|
||||
144
|
||||
],
|
||||
"widgets_values": [
|
||||
1024,
|
||||
1536,
|
||||
1
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "EmptySD3LatentImage"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "VAELoader",
|
||||
"offset": [
|
||||
1239.993419293212,
|
||||
390.0
|
||||
],
|
||||
"size": [
|
||||
366.40625,
|
||||
82.65625
|
||||
],
|
||||
"widgets_values": [
|
||||
"qwen_image_vae.safetensors"
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAELoader"
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": [
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 1
|
||||
},
|
||||
{
|
||||
"from_node_idx": 0,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 2
|
||||
},
|
||||
{
|
||||
"from_node_idx": 1,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 2,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 3,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 4,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 5,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 0
|
||||
},
|
||||
{
|
||||
"from_node_idx": 6,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 1,
|
||||
"to_slot": 3
|
||||
},
|
||||
{
|
||||
"from_node_idx": 7,
|
||||
"from_slot": 0,
|
||||
"to_node_idx": 2,
|
||||
"to_slot": 1
|
||||
}
|
||||
],
|
||||
"anchorConnections": [
|
||||
{
|
||||
"from_slot": 1,
|
||||
"to_node_idx": 0,
|
||||
"to_slot": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
+117
-18
@@ -18,6 +18,7 @@ import { api } from "../../../scripts/api.js";
|
||||
import { externallyTrackedPrompts, refreshEntryWorkflows, armExternalQueue, disarmExternalQueue } from "./history.js";
|
||||
import { sourceSavePrefixInfo, fetchSavedUsduSettings, fetchEngineModels, engineAssetSupport, QWEN_ENHANCE_INSTRUCTION, FLUX1_TILE_GUIDANCE } from "./upscale-from-image.js";
|
||||
import { resolvePre, hasPre } from "./model-settings-bridge.js";
|
||||
import { offscreenIdBase } from "./offscreen-graph.js";
|
||||
|
||||
function toast(severity, detail) {
|
||||
app.extensionManager?.toast?.add({ severity, summary: "Inpaint", detail, life: 7000 });
|
||||
@@ -182,6 +183,7 @@ function inpaintArch(entries) {
|
||||
if (has("CheckpointLoaderSimple", "CheckpointLoader")) return "sdxl";
|
||||
const clipType = String(entries.find((n) => n?.class_type === "CLIPLoader")?.inputs?.type || "").toLowerCase();
|
||||
if (clipType === "flux2") return "flux2";
|
||||
if (clipType === "krea2") return "krea2"; // Krea 2 DiT: UNETLoader+CLIPLoader(type krea2) — must not fall through to flux1 (would mis-wire PuLID)
|
||||
if (has("DualCLIPLoader") || clipType === "flux" || has("UNETLoader", "UnetLoaderGGUF")) return "flux1";
|
||||
return "other";
|
||||
}
|
||||
@@ -313,7 +315,11 @@ export async function prepareInpaint(entry) {
|
||||
// (dual clip + flux VAE) are part of that check.
|
||||
engineModels = (await fetchEngineModels()).filter((m) => m.architecture === "sdxl"
|
||||
|| (m.architecture === "qwen_edit" && support.qwen)
|
||||
|| (m.architecture === "flux" && support.flux1));
|
||||
|| (m.architecture === "flux" && support.flux1)
|
||||
// Krea 2 DiT: UNETLoader + CLIPLoader(type krea2) + qwen_image VAE, all
|
||||
// self-contained in buildKrea2InpaintPrompt — admit unless the support
|
||||
// probe explicitly reports the krea2 assets absent.
|
||||
|| (m.architecture === "krea2" && support.krea2 !== false));
|
||||
} catch { /* engine list is optional — source surgery still works */ }
|
||||
if (!sourceUsable && !engineModels.length) {
|
||||
// Apply would die in buildInpaintPrompt — refuse with the reason up front.
|
||||
@@ -496,19 +502,34 @@ export function buildInpaintPrompt(apiPrompt, data, options, maskName) {
|
||||
// VAE: whatever decodes today, else the anchor's own vae input (MaskedDetail
|
||||
// and USDU carry one; a pruned inpaint graph has no VAEDecode), else the
|
||||
// checkpoint's.
|
||||
const decId = idByClass("VAEDecode");
|
||||
const vaeRef = (decId && isLink(graph[decId].inputs.vae) && graph[decId].inputs.vae)
|
||||
const decId = idByClass("VAEDecode", "VAEUtils_VAEDecodeTiled", "VAEDecodeTiled");
|
||||
let vaeRef = (decId && isLink(graph[decId].inputs.vae) && graph[decId].inputs.vae)
|
||||
|| (isLink(src.inputs.vae) && src.inputs.vae)
|
||||
|| (ckptId && [ckptId, 2]);
|
||||
if (!clipRef || !vaeRef) throw new Error("couldn't trace the clip/vae lines");
|
||||
if (!clipRef) throw new Error("couldn't trace the clip/vae lines");
|
||||
|
||||
let nextId = 1 + Object.keys(graph).reduce((m, id) => Math.max(m, Number.parseInt(id, 10) || 0), 0);
|
||||
// Grafted nodes go far above both the source graph's ids and the live canvas's,
|
||||
// so the background run's execution images never paint onto a live node.
|
||||
let nextId = offscreenIdBase();
|
||||
const add = (class_type, inputs) => {
|
||||
const id = String(nextId++);
|
||||
graph[id] = { class_type, inputs };
|
||||
return id;
|
||||
};
|
||||
|
||||
// The traced VAE may be a decode-only/2x custom loader (the spacepxl VAEUtils
|
||||
// decoder on High-Detail krea2 renders) that CAN'T encode — inpaint needs both
|
||||
// encode and decode. If the trace failed or landed on that custom loader,
|
||||
// graft a fresh stock VAELoader on the shared krea2/qwen latent (prefer
|
||||
// qwen_image_vae, else the clean wan_2.1_vae).
|
||||
const tracedVae = isLink(vaeRef) ? graph[String(vaeRef[0])] : null;
|
||||
if (!vaeRef || (tracedVae && tracedVae.class_type === "VAEUtils_CustomVAELoader")) {
|
||||
const vaeName = probeComboValue("VAELoader", "vae_name", (o) => /qwen.*image.*vae/i.test(o))
|
||||
|| probeComboValue("VAELoader", "vae_name", (o) => /wan.*2[._]?1.*vae/i.test(o) && !/upscale|2x/i.test(o));
|
||||
if (!vaeName) throw new Error("couldn't trace the clip/vae lines");
|
||||
vaeRef = [add("VAELoader", { vae_name: vaeName }), 0];
|
||||
}
|
||||
|
||||
const loadId = add("LoadImage", { image: data.input_ref });
|
||||
const maskId = add("LoadImageMask", { image: maskName, channel: "red" });
|
||||
const pcId = add("PromptChain_PromptChain", {
|
||||
@@ -583,8 +604,8 @@ export function buildInpaintPrompt(apiPrompt, data, options, maskName) {
|
||||
image: [loadId, 0], mask: [maskId, 0],
|
||||
positive: posRef, negative: negRef,
|
||||
seed: Math.floor(Math.random() * 0xffffffffff),
|
||||
steps: lit(src.inputs.steps) ?? 20,
|
||||
cfg: lit(src.inputs.cfg) ?? 7,
|
||||
steps: numOr(options.steps, lit(src.inputs.steps) ?? 20),
|
||||
cfg: numOr(options.cfg, lit(src.inputs.cfg) ?? 7),
|
||||
sampler_name: options.sampler || lit(src.inputs.sampler_name) || "euler",
|
||||
scheduler: options.scheduler || lit(src.inputs.scheduler) || "normal",
|
||||
denoise: typeof options.denoise === "number" ? options.denoise : 0.5,
|
||||
@@ -626,7 +647,7 @@ export function buildInpaintEnginePrompt(data, options, maskName) {
|
||||
const filename = options.engineModel?.filename;
|
||||
if (!filename) throw new Error("no engine model picked");
|
||||
const graph = {};
|
||||
let nextId = 1;
|
||||
let nextId = offscreenIdBase(); // render-node ids must stay off the live canvas — else execution images paint onto a live node sharing the id (see offscreen-graph.js)
|
||||
const add = (class_type, inputs) => {
|
||||
const id = String(nextId++);
|
||||
graph[id] = { class_type, inputs };
|
||||
@@ -656,8 +677,8 @@ export function buildInpaintEnginePrompt(data, options, maskName) {
|
||||
image: [loadId, 0], mask: [maskId, 0],
|
||||
positive: [posId, 0], negative: [negId, 0],
|
||||
seed: Math.floor(Math.random() * 0xffffffffff),
|
||||
steps: numOr(gen.steps, 25),
|
||||
cfg: numOr(gen.cfg, 5),
|
||||
steps: numOr(options.steps, numOr(gen.steps, 25)),
|
||||
cfg: numOr(options.cfg, numOr(gen.cfg, 5)),
|
||||
sampler_name: options.sampler || gen.sampler || "dpmpp_2m",
|
||||
scheduler: options.scheduler || gen.scheduler || "karras",
|
||||
denoise: typeof options.denoise === "number" ? options.denoise : 0.5,
|
||||
@@ -690,7 +711,7 @@ export function buildFlux1InpaintPrompt(data, options, maskName) {
|
||||
if (!clipL || !t5 || !vaeFile) throw new Error("the FLUX text encoders / VAE aren't installed");
|
||||
|
||||
const graph = {};
|
||||
let nextId = 1;
|
||||
let nextId = offscreenIdBase(); // render-node ids must stay off the live canvas — else execution images paint onto a live node sharing the id (see offscreen-graph.js)
|
||||
const add = (class_type, inputs) => {
|
||||
const id = String(nextId++);
|
||||
graph[id] = { class_type, inputs };
|
||||
@@ -716,7 +737,7 @@ export function buildFlux1InpaintPrompt(data, options, maskName) {
|
||||
image: [loadId, 0], mask: [maskId, 0],
|
||||
positive: [guidId, 0], negative: [negId, 0],
|
||||
seed: Math.floor(Math.random() * 0xffffffffff),
|
||||
steps: numOr(gen.steps, 25),
|
||||
steps: numOr(options.steps, numOr(gen.steps, 25)),
|
||||
// cfg pinned, never inherited — a gen cfg would double sampling and burn.
|
||||
cfg: 1.0,
|
||||
sampler_name: options.sampler || gen.sampler || "euler",
|
||||
@@ -730,6 +751,82 @@ export function buildFlux1InpaintPrompt(data, options, maskName) {
|
||||
return { graph, detailId };
|
||||
}
|
||||
|
||||
// KREA 2 engine inpaint: a DiT that wires like FLUX.1 through the same
|
||||
// MaskedDetail crop path, minus FLUX's guidance regime. Shift is baked into the
|
||||
// model class (no ModelSampling) and it's cfg-native (no FluxGuidance), so the
|
||||
// UNET feeds the sampler straight and the empty negative is just the required
|
||||
// input — cfg is pinned 1.0, the same inert-negative posture as flux1/zimage.
|
||||
// Self-contained loaders (UNET + single CLIPLoader(type krea2) + qwen_image VAE)
|
||||
// mean an imported/arbitrary image needs no embedded render data. guide_size
|
||||
// 1024 / max_size 1536: krea2's native working res, matching the flux1 crop.
|
||||
export function buildKrea2InpaintPrompt(data, options, maskName) {
|
||||
const filename = options.engineModel?.filename;
|
||||
if (!filename) throw new Error("no engine model picked");
|
||||
// Resolve the actual installed encoder/VAE (mirrors the flux1 builder + the krea2
|
||||
// tile stage) so a differently-named file still binds and a missing one fails early.
|
||||
const probe = (type, widget) => {
|
||||
try {
|
||||
const node = window.LiteGraph?.createNode?.(type);
|
||||
return node?.widgets?.find((w) => w.name === widget)?.options?.values || [];
|
||||
} catch { return []; }
|
||||
};
|
||||
const clipOpts = probe("CLIPLoader", "clip_name");
|
||||
const clipFile = clipOpts.find((o) => /qwen3vl.*4b/i.test(o)) || clipOpts.find((o) => /qwen3.?vl.*4b/i.test(o));
|
||||
const vaeFile = probe("VAELoader", "vae_name").find((o) => /qwen.*image.*vae/i.test(o));
|
||||
if (!clipFile || !vaeFile) throw new Error("the Krea 2 text encoder / VAE aren't installed");
|
||||
|
||||
const graph = {};
|
||||
let nextId = offscreenIdBase(); // render-node ids must stay off the live canvas — else execution images paint onto a live node sharing the id (see offscreen-graph.js)
|
||||
const add = (class_type, inputs) => {
|
||||
const id = String(nextId++);
|
||||
graph[id] = { class_type, inputs };
|
||||
return id;
|
||||
};
|
||||
const unetId = add("UNETLoader", { unet_name: filename, weight_dtype: "default" });
|
||||
const clipId = add("CLIPLoader", { clip_name: clipFile, type: "krea2", device: "default" });
|
||||
const vaeId = add("VAELoader", { vae_name: vaeFile });
|
||||
const loadId = add("LoadImage", { image: data.input_ref });
|
||||
const maskId = add("LoadImageMask", { image: maskName, channel: "red" });
|
||||
const pcId = add("PromptChain_PromptChain", {
|
||||
prompt: options.prompt || "", mode: "combine", switch_index: 1,
|
||||
locked: false, disabled: false, cached_output: "", cached_neg_output: "",
|
||||
iterate_index: 0, iterate_cycle: 1, collapsed: false,
|
||||
wildcard_modes: "", cached_regions: "",
|
||||
});
|
||||
const posId = add("CLIPTextEncode", { text: [pcId, 1], clip: [clipId, 0] });
|
||||
// Dummy negative: cfg is pinned 1.0 so it never influences sampling, but
|
||||
// MaskedDetail requires the input — the UNET feeds the sampler straight
|
||||
// (no ModelSampling, no FluxGuidance).
|
||||
const negId = add("CLIPTextEncode", { text: [pcId, 2], clip: [clipId, 0] });
|
||||
// RAW is the undistilled base: it needs full steps + a real cfg + the resolution
|
||||
// shift, NOT Turbo's 8-step / cfg-1 / no-shift recipe (which under-resolves it and
|
||||
// over-shifts → melted faces). The crop samples at guide_size 1024, so a static
|
||||
// 1024 ModelSamplingFlux is correct here. Turbo keeps its pinned fast recipe.
|
||||
const raw = /krea2_raw/i.test(filename);
|
||||
let modelRef = [unetId, 0];
|
||||
if (raw) {
|
||||
const msfId = add("ModelSamplingFlux", { model: [unetId, 0], max_shift: 0.90625, base_shift: 0.5, width: 1024, height: 1024 });
|
||||
modelRef = [msfId, 0];
|
||||
}
|
||||
const detailId = add("PromptChain_MaskedDetail", {
|
||||
model: modelRef, vae: [vaeId, 0],
|
||||
image: [loadId, 0], mask: [maskId, 0],
|
||||
positive: [posId, 0], negative: [negId, 0],
|
||||
seed: Math.floor(Math.random() * 0xffffffffff),
|
||||
// Defaults are RAW/Turbo-aware but overridable from the modal (steps/cfg/sampler).
|
||||
steps: numOr(options.steps, raw ? 30 : 8),
|
||||
cfg: numOr(options.cfg, raw ? 3.5 : 1.0),
|
||||
sampler_name: options.sampler || (raw ? "er_sde" : "euler"),
|
||||
scheduler: options.scheduler || (raw ? "beta" : "simple"),
|
||||
denoise: typeof options.denoise === "number" ? options.denoise : 0.5,
|
||||
guide_size: 1024, max_size: 1536, crop_factor: 2.0,
|
||||
feather: numOr(options.feather, 24), grow: numOr(options.grow, 0),
|
||||
});
|
||||
add("PreviewImage", { images: [detailId, 0] });
|
||||
console.log("[PromptChain][inpaint] built krea2 engine prompt:", JSON.stringify(graph, null, 2));
|
||||
return { graph, detailId };
|
||||
}
|
||||
|
||||
const round8 = (v) => Math.max(8, Math.round(v / 8) * 8);
|
||||
|
||||
// Crop window around the painted mask: bbox grown 2× (MaskedDetail's
|
||||
@@ -785,7 +882,7 @@ export function buildQwenEditInpaintPrompt(data, options, maskName) {
|
||||
const scaled = qwenWorkingSize(crop.w, crop.h);
|
||||
|
||||
const graph = {};
|
||||
let nextId = 1;
|
||||
let nextId = offscreenIdBase(); // render-node ids must stay off the live canvas — else execution images paint onto a live node sharing the id (see offscreen-graph.js)
|
||||
const add = (class_type, inputs) => {
|
||||
const id = String(nextId++);
|
||||
graph[id] = { class_type, inputs };
|
||||
@@ -834,8 +931,8 @@ export function buildQwenEditInpaintPrompt(data, options, maskName) {
|
||||
const samplerId = add("KSampler", {
|
||||
model: [normId, 0], positive: [refPosId, 0], negative: [refNegId, 0], latent_image: [noisedId, 0],
|
||||
seed: Math.floor(Math.random() * 0xffffffffff),
|
||||
steps: numOr(gen.steps, 40),
|
||||
cfg: numOr(gen.cfg, 4),
|
||||
steps: numOr(options.steps, numOr(gen.steps, 40)),
|
||||
cfg: numOr(options.cfg, numOr(gen.cfg, 4)),
|
||||
sampler_name: options.sampler || gen.sampler || "euler",
|
||||
scheduler: options.scheduler || gen.scheduler || "simple",
|
||||
// 1.0 is structural: partial denoise blends the shifted re-render; full
|
||||
@@ -912,9 +1009,11 @@ export function runInpaint(prepared, options = {}) {
|
||||
? buildQwenEditInpaintPrompt(data, options, maskName)
|
||||
: options.engine === "flux1-unet" && options.engineModel
|
||||
? buildFlux1InpaintPrompt(data, options, maskName)
|
||||
: options.engine === "sdxl-ckpt" && options.engineModel
|
||||
? buildInpaintEnginePrompt(data, options, maskName)
|
||||
: buildInpaintPrompt(data.prompt, data, options, maskName);
|
||||
: options.engine === "krea2-unet" && options.engineModel
|
||||
? buildKrea2InpaintPrompt(data, options, maskName)
|
||||
: options.engine === "sdxl-ckpt" && options.engineModel
|
||||
? buildInpaintEnginePrompt(data, options, maskName)
|
||||
: buildInpaintPrompt(data.prompt, data, options, maskName);
|
||||
} catch (e) {
|
||||
console.error("[PromptChain][inpaint] build failed (rev abc3dbbd+):", e);
|
||||
tracker.emit({ phase: "error", message: `build failed: ${e.message}` });
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
// Krea 2 RAW uses a resolution-dynamic flow-shift: the ModelSamplingFlux node's
|
||||
// width/height decide mu, and they MUST match the actual render resolution or the
|
||||
// shift is mis-applied — a 1024² shift on an 832×1024 render over-shifts mu ~10%
|
||||
// and compresses the detail tail (the same failure mode the node exists to fix,
|
||||
// milder). The model Settings resolution row mirrors this live for t2i, but that
|
||||
// misses i2i (size comes from the loaded image — there is no resolution row) and
|
||||
// any graph whose resolution was set on-canvas or loaded from disk. So re-derive
|
||||
// every krea2_raw ModelSamplingFlux's width/height from its true render size just
|
||||
// before each queue — the one chokepoint that catches all cases. Event-driven on
|
||||
// the queue (not a timer). No-op for any graph without a krea2_raw shift node.
|
||||
|
||||
import { app } from "../../../scripts/app.js";
|
||||
import { getLink } from "./slot-utils.js";
|
||||
|
||||
const SHIFT_TYPE = "ModelSamplingFlux";
|
||||
const LATENT_DIM_TYPES = new Set(["EmptySD3LatentImage", "EmptyLatentImage", "EmptyFlux2LatentImage"]);
|
||||
const IMAGE_TYPES = new Set(["LoadImage"]);
|
||||
|
||||
// Nearest ancestor of `startNode` (walking input links) whose class is in `types`.
|
||||
function findUpstreamByType(startNode, types) {
|
||||
const graph = startNode?.graph || app.graph;
|
||||
if (!graph || !startNode) return null;
|
||||
const visited = new Set();
|
||||
const queue = [startNode];
|
||||
while (queue.length) {
|
||||
const n = queue.shift();
|
||||
if (!n || visited.has(n.id)) continue;
|
||||
visited.add(n.id);
|
||||
for (const input of n.inputs || []) {
|
||||
if (input.link == null) continue;
|
||||
const link = getLink(graph, input.link);
|
||||
if (!link) continue;
|
||||
const src = graph.getNodeById(link.origin_id);
|
||||
if (!src) continue;
|
||||
if (types.has(src.comfyClass)) return src;
|
||||
queue.push(src);
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function widgetValue(node, name) {
|
||||
return node?.widgets?.find((w) => w.name === name)?.value;
|
||||
}
|
||||
|
||||
function setWidget(node, name, value) {
|
||||
const w = node?.widgets?.find((x) => x.name === name);
|
||||
if (!w || w.value === value) return;
|
||||
w.value = value;
|
||||
w.callback?.(value);
|
||||
}
|
||||
|
||||
// The render resolution feeding a sampler: the latent node's width/height widgets
|
||||
// (t2i), or — when the latent is encoded from an input image and has no such
|
||||
// widgets (i2i) — the loaded image's natural pixel dimensions.
|
||||
function samplerResolution(samplerNode) {
|
||||
const latent = findUpstreamByType(samplerNode, LATENT_DIM_TYPES);
|
||||
if (latent) {
|
||||
const w = widgetValue(latent, "width");
|
||||
const h = widgetValue(latent, "height");
|
||||
if (typeof w === "number" && typeof h === "number") return { width: w, height: h };
|
||||
}
|
||||
const img = findUpstreamByType(samplerNode, IMAGE_TYPES)?.imgs?.[0];
|
||||
if (img?.naturalWidth) return { width: img.naturalWidth, height: img.naturalHeight };
|
||||
return null;
|
||||
}
|
||||
|
||||
// Scope the sync to Krea 2 RAW: only its UNET wants a resolution-tracking shift.
|
||||
// Turbo bakes a fixed mu (and carries no shift node); Flux's shift is its own
|
||||
// concern. Gating on the upstream UNET filename keeps every other arch untouched.
|
||||
function isKrea2RawShift(shiftNode) {
|
||||
const name = widgetValue(findUpstreamByType(shiftNode, new Set(["UNETLoader"])), "unet_name");
|
||||
return typeof name === "string" && /krea2_raw/i.test(name);
|
||||
}
|
||||
|
||||
// Re-derive every krea2_raw ModelSamplingFlux's width/height from its render size.
|
||||
export function syncShiftNodes() {
|
||||
const graph = app?.graph;
|
||||
if (!graph) return;
|
||||
for (const sampler of graph._nodes || []) {
|
||||
const cls = sampler.comfyClass || sampler.type || "";
|
||||
if (!cls.includes("KSampler") || cls === "KSamplerSelect") continue;
|
||||
const shift = findUpstreamByType(sampler, new Set([SHIFT_TYPE]));
|
||||
if (!shift || !isKrea2RawShift(shift)) continue;
|
||||
const res = samplerResolution(sampler);
|
||||
if (!res) continue;
|
||||
const before = `${widgetValue(shift, "width")}×${widgetValue(shift, "height")}`;
|
||||
setWidget(shift, "width", res.width);
|
||||
setWidget(shift, "height", res.height);
|
||||
const after = `${res.width}×${res.height}`;
|
||||
if (before !== after) console.log(`[PromptChain] krea2 RAW shift synced ${before} → ${after}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Wrap queuePrompt so the shift is corrected for the live graph before it serializes.
|
||||
export function setupShiftSync() {
|
||||
const orig = app.queuePrompt?.bind(app);
|
||||
if (!orig) return;
|
||||
app.queuePrompt = async function (...args) {
|
||||
try { syncShiftNodes(); } catch (e) { console.warn("[PromptChain] krea2 shift-sync skipped:", e); }
|
||||
return orig(...args);
|
||||
};
|
||||
}
|
||||
@@ -508,6 +508,16 @@ function findUpstreamByType(startNode, targetTypes) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// A ModelSamplingFlux belongs to a Krea 2 RAW graph when its model line traces up
|
||||
// to a krea2_raw UNET. Keeps the resolution→shift mirror scoped to RAW (Turbo bakes
|
||||
// a fixed mu; Flux owns its own shift), so only RAW's shift node is slaved.
|
||||
function isKrea2RawShift(shiftNode) {
|
||||
const unet = findUpstreamByType(shiftNode, new Set(["UNETLoader"]));
|
||||
if (!unet) return false;
|
||||
const name = readWidgetValue(unet, "unet_name");
|
||||
return typeof name === "string" && /krea2_raw/i.test(name);
|
||||
}
|
||||
|
||||
// The face-detail leg of a model-patch injectable (PuLID / Style Reference) is a
|
||||
// second Apply node feeding the RegionalDetailer's couple-free model line. It's
|
||||
// an implementation detail, not its own tunable — the settings panel shows one
|
||||
@@ -668,6 +678,17 @@ export function detectSupportedNodes(pcNode) {
|
||||
detected.push({ node: latentNode, config: LATENT_NODES[cls], type: cls });
|
||||
}
|
||||
|
||||
// Krea 2 RAW's ModelSamplingFlux sets the resolution-dynamic flow-shift (mu),
|
||||
// whose width/height MUST track the render size. Detected only so the resolution
|
||||
// row can slave its width/height to the latent — mirrorOnly: no visible section
|
||||
// and never written into editors, so it is never baked back into the saved
|
||||
// config (the shift must stay dynamic). Gated to RAW (see isKrea2RawShift).
|
||||
const shiftNode = findUpstreamByType(pivotNode, new Set(["ModelSamplingFlux"]));
|
||||
if (shiftNode && !seenIds.has(shiftNode.id) && isKrea2RawShift(shiftNode)) {
|
||||
seenIds.add(shiftNode.id);
|
||||
detected.push({ node: shiftNode, config: { label: "Shift", widgets: {}, mirrorOnly: true }, type: "ModelSamplingFlux" });
|
||||
}
|
||||
|
||||
const pulidTypes = new Set(["ApplyPulidAdvanced", "ApplyPulid", "ApplyPulidFlux", "ApplyPuLIDFlux2"]);
|
||||
const pulidNode = findUpstreamByType(pivotNode, pulidTypes);
|
||||
if (pulidNode && !seenIds.has(pulidNode.id)) {
|
||||
@@ -746,6 +767,15 @@ function applyArchSamplerDefaults(defaults, architecture, family) {
|
||||
// high-σ regime where tiles come out clean (heavier repaint is the trade).
|
||||
if ("denoise" in defaults) defaults.denoise = 0.7;
|
||||
}
|
||||
} else if (arch === "krea2") {
|
||||
// Krea 2 Turbo: distilled 8-step, guidance baked (cfg 1), euler/simple. Qwen-family
|
||||
// is quant/prompt-sensitive, so a tile re-render wants a higher denoise floor than
|
||||
// Z-Image-Base to avoid underbaking the few-step tail (GPU-tunable).
|
||||
defaults.cfg = 1.0;
|
||||
defaults.sampler_name = "euler";
|
||||
defaults.scheduler = "simple";
|
||||
if ("steps" in defaults) defaults.steps = 8;
|
||||
if ("denoise" in defaults) defaults.denoise = 0.45;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1260,9 +1290,13 @@ export function injectUpscaler(pcNode, modelConfig, opts = {}) {
|
||||
// exists (regional or classic), else straight off the decode.
|
||||
const detailNode = downstream.find(n => n.comfyClass === "PromptChain_RegionalDetailer")
|
||||
|| downstream.find(n => n.comfyClass === "FaceDetailer");
|
||||
const vaeDecodeNode = downstream.find(n => n.comfyClass === "VAEDecode");
|
||||
const vaeDecodeNode = downstream.find(n => n.comfyClass === "VAEDecode"
|
||||
|| n.comfyClass === "VAEUtils_VAEDecodeTiled" || n.comfyClass === "VAEDecodeTiled");
|
||||
const imageSource = detailNode || vaeDecodeNode;
|
||||
if (!imageSource) return false;
|
||||
if (!imageSource) {
|
||||
console.warn(`[PromptChain][graft] FAIL: no decode/detail image source. downstream=[${downstream.map((n) => n.comfyClass).join(",")}]`);
|
||||
return false;
|
||||
}
|
||||
|
||||
// Sampler anchor: classic KSampler graphs, or the Flux 2 split where
|
||||
// SamplerCustomAdvanced samples and a guider (CFGGuider on Klein base,
|
||||
@@ -1357,6 +1391,24 @@ export function injectUpscaler(pcNode, modelConfig, opts = {}) {
|
||||
}
|
||||
splitModelNode = ms;
|
||||
splitVaeNode = findUpstreamByType(vaeDecodeNode, new Set(["VAELoader"]));
|
||||
if (!splitVaeNode) {
|
||||
// The source decodes through a custom/tiled VAE (e.g. the spacepxl
|
||||
// VAEUtils decoder on High-Detail krea2 renders) whose 2x/12-ch output
|
||||
// can't drive USDU's encode->sample->decode tiling. Graft a fresh stock
|
||||
// VAELoader on the shared krea2/qwen latent — prefer the clean wan_2.1_vae
|
||||
// (no grid artifact), else qwen_image_vae, else whatever is installed.
|
||||
const freshVae = LG.createNode("VAELoader");
|
||||
const vaeW = freshVae?.widgets?.find(w => w.name === "vae_name");
|
||||
const vaeOpts = vaeW?.options?.values || [];
|
||||
const pick = vaeOpts.find(o => /wan.*2[._]?1.*vae/i.test(o) && !/upscale|2x/i.test(o))
|
||||
|| vaeOpts.find(o => /qwen.*image.*vae/i.test(o)) || vaeOpts[0];
|
||||
if (freshVae && vaeW && pick) {
|
||||
vaeW.value = pick; vaeW.callback?.(pick);
|
||||
freshVae.pos = [vaeDecodeNode.pos[0], vaeDecodeNode.pos[1] + 220];
|
||||
graph.add(freshVae);
|
||||
splitVaeNode = freshVae;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const loaderNode = LG.createNode("UpscaleModelLoader");
|
||||
@@ -1847,7 +1899,7 @@ function injectRegional(pcNode, modelConfig) {
|
||||
// $blocks cohere into ONE scene). Other DiTs (Flux/SD3/…) have no ported couple yet, so
|
||||
// they fall back to ComfyUI-native masked conditioning here.
|
||||
const arch = modelConfig?.architecture || "";
|
||||
const NATIVE_REGIONAL_ARCH = new Set(["flux", "flux2", "sd3", "hidream", "qwen_image", "qwen_edit"]);
|
||||
const NATIVE_REGIONAL_ARCH = new Set(["flux", "flux2", "sd3", "hidream", "qwen_image", "qwen_edit", "krea2"]);
|
||||
if (NATIVE_REGIONAL_ARCH.has(arch)) {
|
||||
const rc = LG.createNode("PromptChain_RegionalConditioning");
|
||||
if (!rc) return false;
|
||||
@@ -2873,6 +2925,17 @@ export async function showModelModal(pcNode, modelInfo, anchorEl, { tab = null }
|
||||
],
|
||||
});
|
||||
}
|
||||
// Krea 2 has no published ControlNet (and ComfyUI's krea2 model class exposes
|
||||
// no control hook), but native cond-mask regional works — PromptChain_RegionalConditioning
|
||||
// is model-agnostic. Offer Regional only.
|
||||
if (arch === "krea2") {
|
||||
items.push({
|
||||
label: "ControlNet",
|
||||
children: [
|
||||
{ label: "Regional", value: "ControlNet:regional" },
|
||||
],
|
||||
});
|
||||
}
|
||||
// Z-Image ControlNet (DiffSynth model-patch path; core nodes + aux
|
||||
// preprocessors). The control model is family-specific: Turbo's is
|
||||
// step-distilled, base's isn't, and they're NOT cross-compatible — so a
|
||||
|
||||
@@ -384,14 +384,19 @@ function SettingsTab($$anchor, $$props) {
|
||||
if (!get(editors)[nodeType]) get(editors)[nodeType] = {};
|
||||
get(editors)[nodeType][widgetName] = val;
|
||||
}
|
||||
function mirrorIdeogramScheduler(latentNode) {
|
||||
const sched = detected().find((d) => d.type === "Ideogram4Scheduler");
|
||||
if (!sched) return;
|
||||
for (const dim of ["width", "height"]) {
|
||||
const v = $$props.readWidgetValue(latentNode, dim);
|
||||
if (v !== void 0) {
|
||||
$$props.writeWidgetValue(sched.node, dim, v);
|
||||
setEditorValue("Ideogram4Scheduler", dim, v);
|
||||
const RES_DEPENDENTS = [
|
||||
{ type: "Ideogram4Scheduler", persist: true },
|
||||
{ type: "ModelSamplingFlux", persist: false }
|
||||
];
|
||||
function mirrorResolutionDependents(latentNode) {
|
||||
for (const { type: depType, persist } of RES_DEPENDENTS) {
|
||||
const dep = detected().find((d) => d.type === depType);
|
||||
if (!dep) continue;
|
||||
for (const dim of ["width", "height"]) {
|
||||
const v = $$props.readWidgetValue(latentNode, dim);
|
||||
if (v === void 0) continue;
|
||||
$$props.writeWidgetValue(dep.node, dim, v);
|
||||
if (persist) setEditorValue(depType, dim, v);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -484,7 +489,10 @@ function SettingsTab($$anchor, $$props) {
|
||||
});
|
||||
}
|
||||
var node_2 = sibling(node_1, 2);
|
||||
each(node_2, 17, detected, index, ($$anchor2, $$item) => {
|
||||
each(node_2, 17, () => detected().filter((d) => {
|
||||
var _a2;
|
||||
return !((_a2 = d.config) == null ? void 0 : _a2.mirrorOnly);
|
||||
}), index, ($$anchor2, $$item) => {
|
||||
let node = () => get($$item).node;
|
||||
let config = () => get($$item).config;
|
||||
let type = () => get($$item).type;
|
||||
@@ -634,7 +642,7 @@ function SettingsTab($$anchor, $$props) {
|
||||
$$props.writeWidgetValue(node(), "height", get(p).height);
|
||||
setEditorValue(type(), "width", get(p).width);
|
||||
setEditorValue(type(), "height", get(p).height);
|
||||
mirrorIdeogramScheduler(node());
|
||||
mirrorResolutionDependents(node());
|
||||
presetMenuOpen[type()] = false;
|
||||
});
|
||||
delegated("click", span_2, (e) => {
|
||||
@@ -702,13 +710,13 @@ function SettingsTab($$anchor, $$props) {
|
||||
const v = parseInt(e.target.value) || 512;
|
||||
setEditorValue(type(), "width", v);
|
||||
$$props.writeWidgetValue(node(), "width", v);
|
||||
mirrorIdeogramScheduler(node());
|
||||
mirrorResolutionDependents(node());
|
||||
});
|
||||
delegated("change", input_2, (e) => {
|
||||
const v = parseInt(e.target.value) || 512;
|
||||
setEditorValue(type(), "height", v);
|
||||
$$props.writeWidgetValue(node(), "height", v);
|
||||
mirrorIdeogramScheduler(node());
|
||||
mirrorResolutionDependents(node());
|
||||
});
|
||||
append($$anchor5, div_6);
|
||||
};
|
||||
|
||||
+480
-375
File diff suppressed because it is too large
Load Diff
+118
-15
@@ -8,6 +8,7 @@ import { api } from "../../../scripts/api.js";
|
||||
import { getLink } from "./slot-utils.js";
|
||||
import {
|
||||
buildUpscaleGraph,
|
||||
logUpscaleRun,
|
||||
pickSeedvr2DitModel,
|
||||
SEEDVR2_STAGE_MAX_SHORT_EDGE,
|
||||
SEEDVR2_INPUT_NOISE,
|
||||
@@ -42,7 +43,7 @@ function createTracker() {
|
||||
// own job counting (ultimate-upscale.py): main pass rows*cols, plus the seam
|
||||
// pass per seam_fix_mode. Lets the modal show OVERALL progress — the ws
|
||||
// progress events are per-tile sampler bars that reset every tile.
|
||||
function computeTilePlan(graph, data) {
|
||||
function computeTilePlan(graph, data, options = {}) {
|
||||
const usNode = graph._nodes.find((n) => n.comfyClass === "UltimateSDUpscale" || n.comfyClass === "UltimateSDUpscaleNoUpscale");
|
||||
if (!usNode) return { steps: 0, totalTiles: 0 };
|
||||
const w = (name) => usNode.widgets?.find((x) => x.name === name)?.value;
|
||||
@@ -52,13 +53,19 @@ function computeTilePlan(graph, data) {
|
||||
let outW = (data.width || 0) * scale;
|
||||
let outH = (data.height || 0) * scale;
|
||||
if (usNode.comfyClass === "UltimateSDUpscaleNoUpscale") {
|
||||
// NoUpscale tiles whatever arrives on upscaled_image — the final dims
|
||||
// live on the exact-fit ImageScale feeding it.
|
||||
const input = usNode.inputs?.find((i) => i.name === "upscaled_image");
|
||||
const link = input?.link != null ? getLink(graph, input.link) : null;
|
||||
const origin = link && graph.getNodeById(link.origin_id);
|
||||
outW = Number(origin?.widgets?.find((x) => x.name === "width")?.value) || (data.width || 0);
|
||||
outH = Number(origin?.widgets?.find((x) => x.name === "height")?.value) || (data.height || 0);
|
||||
if (options.preEnlargedW && options.preEnlargedH) {
|
||||
// Cache hit: upscaled_image is a LoadImage of the cached base (no width/
|
||||
// height widgets to read), so take the resolved target dims from the cache.
|
||||
outW = options.preEnlargedW; outH = options.preEnlargedH;
|
||||
} else {
|
||||
// NoUpscale tiles whatever arrives on upscaled_image — the final dims
|
||||
// live on the exact-fit ImageScale feeding it.
|
||||
const input = usNode.inputs?.find((i) => i.name === "upscaled_image");
|
||||
const link = input?.link != null ? getLink(graph, input.link) : null;
|
||||
const origin = link && graph.getNodeById(link.origin_id);
|
||||
outW = Number(origin?.widgets?.find((x) => x.name === "width")?.value) || (data.width || 0);
|
||||
outH = Number(origin?.widgets?.find((x) => x.name === "height")?.value) || (data.height || 0);
|
||||
}
|
||||
}
|
||||
const rows = Math.ceil(outH / tileH) || 1;
|
||||
const cols = Math.ceil(outW / tileW) || 1;
|
||||
@@ -122,7 +129,7 @@ function collectRecordMeta(graph, data) {
|
||||
};
|
||||
}
|
||||
|
||||
function watchExecution({ promptId, workflowId, meta, plan, startedAt, tracker, run }) {
|
||||
function watchExecution({ promptId, workflowId, meta, plan, startedAt, tracker, run, enlargedUrl }) {
|
||||
let lastImage = null;
|
||||
// Tile bookkeeping: each sampled tile runs its own progress bar (1..steps),
|
||||
// so a value reset = next tile. Bars with a different max are the model
|
||||
@@ -213,6 +220,9 @@ function watchExecution({ promptId, workflowId, meta, plan, startedAt, tracker,
|
||||
resultUrl: lastImage
|
||||
? api.apiURL(`/view?filename=${encodeURIComponent(lastImage.filename)}&subfolder=${encodeURIComponent(lastImage.subfolder || "")}&type=${lastImage.type || "output"}&rand=${Math.random()}`)
|
||||
: null,
|
||||
// The enlarged base the tiles re-detailed — the compare's "Before" (so you see
|
||||
// the tile pass's contribution over the sharpened base, not vs the raw source).
|
||||
enlargedUrl: enlargedUrl || null,
|
||||
elapsedSec,
|
||||
});
|
||||
toast("success", `Upscale finished in ${elapsedSec}s${lastImage ? `: ${lastImage.filename}` : ""}.`);
|
||||
@@ -245,9 +255,13 @@ function comboValues(nodeType, widgetName) {
|
||||
// noise (clean renders carry no degradation for it to invert), and the ESRGAN
|
||||
// climb only happens when the target is still further out. Exported for the
|
||||
// headless build-only harness.
|
||||
export function buildEnlargePrompt(inputRef, targetW, targetH, engine = "ultrasharp") {
|
||||
export function buildEnlargePrompt(inputRef, targetW, targetH, engine = "ultrasharp", climbModel = null) {
|
||||
const models = comboValues("UpscaleModelLoader", "model_name");
|
||||
const modelName = models.find((o) => /ultrasharp/i.test(o)) || models.find((o) => /^4x/i.test(o)) || models[0];
|
||||
// Mirror upscale-from-image.js defaultUpscaleModelPick: exclude UltraSharpV2 (its
|
||||
// micro-speckle becomes tile-CN cross-hatch) so the cached base matches the fused build.
|
||||
const modelName = (climbModel && models.includes(climbModel) ? climbModel : null)
|
||||
|| models.find((o) => /ultrasharp(?!v2)/i.test(o)) || models.find((o) => /ultrasharp/i.test(o))
|
||||
|| models.find((o) => /^4x/i.test(o)) || models[0];
|
||||
if (!modelName) throw new Error("no upscale model installed");
|
||||
|
||||
if (engine === "seedvr2") {
|
||||
@@ -297,10 +311,10 @@ export function buildEnlargePrompt(inputRef, targetW, targetH, engine = "ultrash
|
||||
// to temp. Used by the Edit grow-canvas path to produce the new Background
|
||||
// while the region render diffuses, and for the under-layer pass. Plain JSON
|
||||
// prompt — no workflow tab involved.
|
||||
export function enlargeImageInBackground(inputRef, targetW, targetH, engine = "ultrasharp") {
|
||||
export function enlargeImageInBackground(inputRef, targetW, targetH, engine = "ultrasharp", climbModel = null) {
|
||||
let output;
|
||||
try {
|
||||
output = buildEnlargePrompt(inputRef, targetW, targetH, engine);
|
||||
output = buildEnlargePrompt(inputRef, targetW, targetH, engine, climbModel);
|
||||
} catch (e) {
|
||||
return Promise.reject(e);
|
||||
}
|
||||
@@ -350,6 +364,61 @@ export function enlargeImageInBackground(inputRef, targetW, targetH, engine = "u
|
||||
});
|
||||
}
|
||||
|
||||
// ── enlarged-base cache (NoUpscale tile engines) ───────────────────────────
|
||||
// The UltraSharp/SeedVR2 enlarge ("pre-process") is deterministic, so a retry
|
||||
// that only changes tile dials reuses the saved base instead of re-running it.
|
||||
// Keyed on the RESOLVED enlarge identity (input + engine + climb model + target
|
||||
// dims) — tile dials are excluded by construction. Only the four NoUpscale tile
|
||||
// engines are eligible; fused/diffusion engines (source/qwen/flux2/plain) build
|
||||
// the enlarge inseparably and stay on the normal fused path. The whole pre-pass
|
||||
// is best-effort: any failure falls through to the fused build (never breaks).
|
||||
const TILE_ENGINES = new Set(["sdxl-ckpt", "flux1-unet", "zimage-unet", "krea2-unet"]);
|
||||
const _enlargeCache = new Map(); // key -> input-dir ref ("subfolder/name.png")
|
||||
|
||||
function resolveEnlargeTarget(options, data) {
|
||||
const w = data.width || 0, h = data.height || 0;
|
||||
const longest = Math.max(w, h);
|
||||
const scale = typeof options.upscaleBy === "number" && options.upscaleBy >= 1
|
||||
? options.upscaleBy
|
||||
: longest > 0 ? Math.min(8, Math.max(1, Math.round((4096 / longest) * 100) / 100)) : 2;
|
||||
return { targetW: Math.round(w * scale) || 2048, targetH: Math.round(h * scale) || 2048, scale };
|
||||
}
|
||||
|
||||
function enlargeCacheKey(options, data, targetW, targetH) {
|
||||
const engine = options.climbStage === "seedvr2" ? "seedvr2" : "ultrasharp";
|
||||
return [data.input_ref || "", engine, options.climbModel || "default", targetW, targetH].join("|");
|
||||
}
|
||||
|
||||
// Short deterministic hash of the cache key → a stable cache filename, so re-MISSes
|
||||
// for the same enlarge identity OVERWRITE one file instead of piling new ones up.
|
||||
function hashKey(s) {
|
||||
let h = 5381;
|
||||
for (let i = 0; i < s.length; i++) h = ((h << 5) + h + s.charCodeAt(i)) >>> 0;
|
||||
return h.toString(36);
|
||||
}
|
||||
|
||||
// A LoadImage reads only the input dir, so copy the freshly-enlarged temp image into
|
||||
// input/ (under a key-stable name) so the tile graph can load it on a cache hit.
|
||||
async function copyEnlargedToInput(img, nameBase) {
|
||||
const url = api.apiURL(`/view?filename=${encodeURIComponent(img.filename)}&subfolder=${encodeURIComponent(img.subfolder || "")}&type=${img.type || "temp"}`);
|
||||
const blob = await (await fetch(url)).blob();
|
||||
const form = new FormData();
|
||||
form.append("image", new File([blob], `upcache_${nameBase}.png`, { type: "image/png" }));
|
||||
form.append("subfolder", "promptchain_upscale_cache");
|
||||
form.append("overwrite", "true");
|
||||
const res = await api.fetchApi("/upload/image", { method: "POST", body: form });
|
||||
const j = await res.json();
|
||||
return j.subfolder ? `${j.subfolder}/${j.name}` : j.name;
|
||||
}
|
||||
|
||||
// /view URL for a cached enlarged base (input-dir ref) — the compare's "Before".
|
||||
function enlargedRefUrl(ref) {
|
||||
const slash = ref.lastIndexOf("/");
|
||||
const sub = slash >= 0 ? ref.slice(0, slash) : "";
|
||||
const name = slash >= 0 ? ref.slice(slash + 1) : ref;
|
||||
return api.apiURL(`/view?filename=${encodeURIComponent(name)}&subfolder=${encodeURIComponent(sub)}&type=input&rand=${Math.random()}`);
|
||||
}
|
||||
|
||||
export function upscaleImageInBackground(prepared, options = {}) {
|
||||
const tracker = createTracker();
|
||||
// Cancellation context, shared between the pipeline and tracker.cancel().
|
||||
@@ -400,6 +469,38 @@ export function upscaleImageInBackground(prepared, options = {}) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Enlarge cache (NoUpscale tile engines): pre-enlarge once, reuse on retry so
|
||||
// only the tiles re-run. Wrapped so ANY failure falls through to the normal
|
||||
// fused build — this can never break the upscale, only skip the optimization.
|
||||
try {
|
||||
// Skip <=1.02: there the fused build does a plain lanczos (no ESRGAN climb), so
|
||||
// there's nothing expensive to cache AND a cached ESRGAN base would diverge from
|
||||
// the fused output. Above 1.02 the cached base equals the fused climb (same
|
||||
// climbModel, same lanczos fit), so the result is unchanged — just cached.
|
||||
const { targetW, targetH, scale } = TILE_ENGINES.has(options.engine || "source") && !options.preEnlargedRef
|
||||
? resolveEnlargeTarget(options, prepared.data)
|
||||
: { scale: 0 };
|
||||
if (scale > 1.02) {
|
||||
const key = enlargeCacheKey(options, prepared.data, targetW, targetH);
|
||||
let baseRef = _enlargeCache.get(key);
|
||||
if (baseRef) {
|
||||
console.log(`[upscale-cache] HIT — reusing enlarged base, skipping pre-process (${targetW}x${targetH})`);
|
||||
} else {
|
||||
tracker.emit({ phase: "running", prep: true });
|
||||
const enlEngine = options.climbStage === "seedvr2" ? "seedvr2" : "ultrasharp";
|
||||
const temp = await enlargeImageInBackground(prepared.data.input_ref, targetW, targetH, enlEngine, options.climbModel);
|
||||
if (run.cancelled) return;
|
||||
baseRef = await copyEnlargedToInput(temp, hashKey(key));
|
||||
_enlargeCache.set(key, baseRef);
|
||||
console.log(`[upscale-cache] MISS — enlarged + cached base (${targetW}x${targetH})`);
|
||||
}
|
||||
options = { ...options, preEnlargedRef: baseRef, preEnlargedW: targetW, preEnlargedH: targetH };
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn("[PromptChain] enlarge cache pre-pass skipped — using fused build:", e);
|
||||
options = { ...options, preEnlargedRef: null };
|
||||
}
|
||||
|
||||
tracker.emit({ phase: "building" });
|
||||
const built = await buildUpscaleGraph(prepared, options);
|
||||
if (!built) {
|
||||
@@ -411,6 +512,7 @@ export function upscaleImageInBackground(prepared, options = {}) {
|
||||
|
||||
tracker.emit({ phase: "queueing" });
|
||||
const serialized = await app.graphToPrompt(graph);
|
||||
logUpscaleRun(serialized.output, options.engine);
|
||||
let promptId = null;
|
||||
// Arm the external-prompt window across the queue round-trip so the main
|
||||
// graph's execution_start gate recognizes this run even if its event beats
|
||||
@@ -438,7 +540,7 @@ export function upscaleImageInBackground(prepared, options = {}) {
|
||||
run.promptId = promptId;
|
||||
const startedAt = Date.now();
|
||||
const meta = collectRecordMeta(graph, prepared.data);
|
||||
const plan = computeTilePlan(graph, prepared.data);
|
||||
const plan = computeTilePlan(graph, prepared.data, options);
|
||||
|
||||
if (!promptId) {
|
||||
tracker.emit({ phase: "error", message: "queue returned no prompt id" });
|
||||
@@ -457,7 +559,8 @@ export function upscaleImageInBackground(prepared, options = {}) {
|
||||
return;
|
||||
}
|
||||
tracker.emit({ phase: "running" });
|
||||
watchExecution({ promptId, workflowId, meta, plan, startedAt, tracker, run });
|
||||
const enlargedUrl = options.preEnlargedRef ? enlargedRefUrl(options.preEnlargedRef) : null;
|
||||
watchExecution({ promptId, workflowId, meta, plan, startedAt, tracker, run, enlargedUrl });
|
||||
})().catch((e) => {
|
||||
console.error("[PromptChain] background upscale failed", e);
|
||||
tracker.emit({ phase: "error", message: e?.message || "unexpected failure" });
|
||||
|
||||
+341
-80
@@ -21,6 +21,7 @@
|
||||
import { app } from "../../../scripts/app.js";
|
||||
import { api } from "../../../scripts/api.js";
|
||||
import { getLink } from "./slot-utils.js";
|
||||
import { offscreenIdBase } from "./offscreen-graph.js";
|
||||
import {
|
||||
injectUpscaler,
|
||||
buildRegionalUpscaleCond,
|
||||
@@ -54,6 +55,36 @@ export function reportClientError(where, e, context = null) {
|
||||
} catch { /* reporting must never break the flow it reports on */ }
|
||||
}
|
||||
|
||||
// Diagnostic: log exactly what an upscale Apply submitted, into comfyui.log
|
||||
// (as "[upscale-run]") so a run's model/denoise/recipe is recoverable without
|
||||
// reverse-engineering the output PNG. apiPrompt is app.graphToPrompt().output.
|
||||
export function logUpscaleRun(apiPrompt, engine) {
|
||||
try {
|
||||
if (!apiPrompt || typeof apiPrompt !== "object") return;
|
||||
const vals = Object.values(apiPrompt);
|
||||
const find = (pred) => vals.find(pred) || null;
|
||||
const unet = find((n) => n.class_type === "UNETLoader")?.inputs?.unet_name
|
||||
|| find((n) => n.class_type === "CheckpointLoaderSimple" || n.class_type === "CheckpointLoader")?.inputs?.ckpt_name
|
||||
|| null;
|
||||
const us = find((n) => (n.class_type || "").includes("UltimateSDUpscale"))?.inputs
|
||||
|| find((n) => n.class_type === "KSampler" || n.class_type === "PromptChain_MaskedDetail")?.inputs
|
||||
|| {};
|
||||
const climb = find((n) => n.class_type === "UpscaleModelLoader")?.inputs?.model_name || null;
|
||||
const vae = find((n) => n.class_type === "VAELoader")?.inputs?.vae_name
|
||||
|| find((n) => n.class_type === "VAEUtils_CustomVAELoader")?.inputs?.vae_name || null;
|
||||
const tile = us.tile_width != null ? `${us.tile_width}x${us.tile_height}` : null;
|
||||
const summary = {
|
||||
engine: engine || "?", model: unet, vae, climb,
|
||||
denoise: us.denoise, cfg: us.cfg, steps: us.steps,
|
||||
sampler: us.sampler_name, scheduler: us.scheduler,
|
||||
tile, upscale_by: us.upscale_by, seam: us.seam_fix_mode, mode: us.mode_type,
|
||||
};
|
||||
reportClientError("upscale-run",
|
||||
{ message: `engine=${engine} model=${unet} vae=${vae} denoise=${us.denoise} cfg=${us.cfg} steps=${us.steps} sampler=${us.sampler_name}/${us.scheduler} tile=${tile} upscale_by=${us.upscale_by} seam=${us.seam_fix_mode} climb=${climb}` },
|
||||
summary);
|
||||
} catch { /* never break the run */ }
|
||||
}
|
||||
|
||||
function freshWorkflowId() {
|
||||
if (globalThis.crypto?.randomUUID) return crypto.randomUUID();
|
||||
return `${Date.now().toString(16)}-${Math.random().toString(16).slice(2, 10)}`;
|
||||
@@ -144,10 +175,13 @@ export function analyzeUpscaleCaps(data) {
|
||||
// renders when the SeedVR2 pack is actually installed.
|
||||
const seedvr2Available = !!window.LiteGraph?.registered_node_types?.["SeedVR2VideoUpscaler"];
|
||||
|
||||
// High-Detail krea2 renders decode through VAEUtils_VAEDecodeTiled (spacepxl),
|
||||
// not the stock VAEDecode — still a graftable sampler->decode pipeline, so
|
||||
// accept the tiled decoders too or those renders fall to plain ESRGAN only.
|
||||
const graftable = has("UltimateSDUpscale") || (
|
||||
has("PromptChain_PromptChain")
|
||||
&& (has("KSampler") || has("KSamplerAdvanced") || has("SamplerCustomAdvanced"))
|
||||
&& has("VAEDecode")
|
||||
&& (has("VAEDecode") || has("VAEUtils_VAEDecodeTiled") || has("VAEDecodeTiled"))
|
||||
);
|
||||
if (!graftable) {
|
||||
return {
|
||||
@@ -290,7 +324,13 @@ export function engineAssetSupport() {
|
||||
&& comboProbe("UNETLoader", "unet_name").some((o) => /z.?image/i.test(o))
|
||||
&& comboProbe("CLIPLoader", "clip_name").some((o) => /qwen.*3[._]4b|lumina2/i.test(o))
|
||||
&& comboProbe("VAELoader", "vae_name").some((o) => /(^|[\\/])ae\.(safetensors|sft)$/i.test(o));
|
||||
return { sdxl, qwen, flux1, zimage };
|
||||
// Krea 2 (DiT, mirrors zimage but no ModelSamplingAuraFlow — shift is baked
|
||||
// into the model class): krea2 UNET + qwen3vl_4b text encoder + qwen_image_vae.
|
||||
const krea2 = !!reg["UltimateSDUpscaleNoUpscale"] && esrgan
|
||||
&& comboProbe("UNETLoader", "unet_name").some((o) => /krea2/i.test(o))
|
||||
&& comboProbe("CLIPLoader", "clip_name").some((o) => /qwen3vl.*4b|qwen3.?vl.*4b/i.test(o))
|
||||
&& comboProbe("VAELoader", "vae_name").some((o) => /qwen.*image.*vae/i.test(o));
|
||||
return { sdxl, qwen, flux1, zimage, krea2 };
|
||||
}
|
||||
|
||||
// The modal's defaults for a picked engine model. SDXL: the CN-locked tile
|
||||
@@ -356,6 +396,50 @@ function engineModelDefaults(architecture, config) {
|
||||
},
|
||||
};
|
||||
}
|
||||
if (architecture === "krea2") {
|
||||
// Krea 2 Turbo is step-distilled (cfg 1.0, 8 steps), cfg native (no
|
||||
// FluxGuidance/ModelSampling). TILE denoise stays LOW: a tile routine needs
|
||||
// each tile anchored to the source or adjacent tiles regenerate different
|
||||
// content and won't stitch (visible seams). The catch — at low denoise the
|
||||
// distilled few-step field under-resolves the tile (some residual voronoi/
|
||||
// blotch on smooth skin; NOT a VAE artifact, persists through a clean
|
||||
// wan_2.1_vae decode). Turbo can't fully win either way in a tile pass; the
|
||||
// non-distilled RAW krea2 is the proper clean+coherent tile upscaler.
|
||||
// Turbo defaults to euler+simple (the official Krea recipe; suits the smooth
|
||||
// matte-PVC figurine finish). RAW keeps er_sde (grain-preserving for photoreal).
|
||||
const usdu = nodes.UltimateSDUpscale || {};
|
||||
// RAW = the undistilled base checkpoint. Non-distilled, so it resolves a LOW
|
||||
// denoise tile cleanly (no blotch) while staying anchored to the source (no
|
||||
// seams) — the proper clean+coherent tiled upscaler. It runs a REAL cfg +
|
||||
// full steps; Turbo stays the soft-but-coherent 0.45 / cfg 1.0 / 8-step
|
||||
// compromise. (Same wan_2.1_vae decode + qwen3vl CLIP either way.)
|
||||
const fn = config?.filename || "";
|
||||
const raw = /raw/i.test(fn) && !/turbo/i.test(fn);
|
||||
return {
|
||||
denoise: raw ? 0.35 : 0.45,
|
||||
// No tile ControlNet exists for Krea 2 (standalone DiT, not Flux), so each tile
|
||||
// is anchored only by the img2img floor — push denoise too far and a tile that
|
||||
// contains the whole subject regenerates it (doubling). Cap to the CN-less safe
|
||||
// band; Turbo can't go much below 0.45 without blotching, so 0.45/0.5 is its
|
||||
// narrow usable window. (Stronger re-detail without doubling needs smaller tiles.)
|
||||
denoiseMax: raw ? 0.45 : 0.5,
|
||||
sampler: raw ? "er_sde" : "euler",
|
||||
scheduler: "simple",
|
||||
advanced: {
|
||||
steps: usdu.steps ?? (raw ? 30 : 8),
|
||||
cfg: raw ? 3.5 : 1.0,
|
||||
tile_width: usdu.tile_width ?? 1024,
|
||||
tile_height: usdu.tile_height ?? 1024,
|
||||
mask_blur: 32,
|
||||
tile_padding: 64,
|
||||
mode_type: "Chess", // checkerboard order — neighbours redraw before a tile's seam (cleaner than Linear)
|
||||
seam_fix_mode: raw ? "Half Tile" : "None",
|
||||
// RAW's Half-Tile seam pass must NOT full-repaint (the node default is 1.0, which
|
||||
// makes seam bands hallucinate content that won't match the lightly-detailed tiles).
|
||||
seam_fix_denoise: raw ? 0.3 : undefined,
|
||||
},
|
||||
};
|
||||
}
|
||||
const usdu = nodes.UltimateSDUpscale || {};
|
||||
return {
|
||||
denoise: 0.15,
|
||||
@@ -395,7 +479,7 @@ export async function fetchEngineModels() {
|
||||
if (!listRes.ok) return [];
|
||||
const models = ((await listRes.json())?.models || [])
|
||||
.filter((m) => m.architecture === "sdxl" || m.architecture === "qwen_edit"
|
||||
|| m.architecture === "zimage"
|
||||
|| m.architecture === "zimage" || m.architecture === "krea2"
|
||||
// flux = the whole FLUX.1 family; edit/control/util variants don't
|
||||
// belong in a faithful tile pass (kontext re-renders, schnell is
|
||||
// step-distilled, fill/canny/depth/redux are tool models).
|
||||
@@ -481,6 +565,7 @@ function recommendedSamplerFor(architecture) {
|
||||
const arch = architecture || "";
|
||||
if (arch === "flux" || arch === "flux2") return { sampler: "euler", scheduler: "simple" };
|
||||
if (arch === "zimage") return { sampler: "res_multistep", scheduler: "beta" };
|
||||
if (arch === "krea2") return { sampler: "euler", scheduler: "simple" };
|
||||
return { sampler: "dpmpp_2m_sde", scheduler: "karras" };
|
||||
}
|
||||
|
||||
@@ -533,6 +618,10 @@ function usduBuildDefaults(architecture, saved) {
|
||||
const arch = architecture || "";
|
||||
if (arch === "flux" || arch === "flux2") d.cfg = 1.0;
|
||||
else if (arch === "zimage") d.cfg = 1.3;
|
||||
// Krea 2 carries cfg natively at 1.0 (no FluxGuidance); euler/simple/0.45
|
||||
// denoise are applied in buildKrea2TileStage's defaults, not here (this dict
|
||||
// only holds USDU shape knobs).
|
||||
else if (arch === "krea2") d.cfg = 1.0;
|
||||
for (const key of Object.keys(d)) {
|
||||
if (saved && saved[key] !== undefined) d[key] = saved[key];
|
||||
}
|
||||
@@ -570,13 +659,16 @@ export async function prepareUpscale(entry) {
|
||||
const recommended = recommendedSamplerFor(architecture);
|
||||
// The modal's prompt editor scopes templates/autocomplete to this when
|
||||
// the source engine is selected (a picked engine model overrides it).
|
||||
caps.sourceModelInfo = hash ? { hash, architecture } : null;
|
||||
// Z-Image Turbo is step-distilled: at low denoise ComfyUI truncates the sigma
|
||||
// schedule to its low-σ tail, which the few-step field can't resolve -> blotchy
|
||||
// skin. 0.7 lands back in its trained high-σ regime (clean, heavier repaint).
|
||||
// Base is non-distilled and resolves a low-denoise tail fine.
|
||||
caps.sourceModelInfo = hash ? { hash, architecture, family } : null;
|
||||
// Krea 2 Turbo source-graft is a TILE pass: denoise stays LOW so each tile
|
||||
// anchors to the source and they stitch (high denoise = each tile a full
|
||||
// re-render = seams). The distillation means low-denoise tiles still carry
|
||||
// some residual blotch on smooth skin — that's a Turbo limitation a tile
|
||||
// routine can't escape; the non-distilled RAW model is the real clean+
|
||||
// coherent upscaler.
|
||||
caps.defaultDenoise = typeof settings.denoise === "number" ? settings.denoise
|
||||
: (architecture === "zimage" ? (family === "zimage_base" ? 0.15 : 0.7) : 0.3);
|
||||
: (architecture === "zimage" ? (family === "zimage_base" ? 0.15 : 0.7)
|
||||
: architecture === "krea2" ? 0.45 : 0.3);
|
||||
// Recommendation chain: a USDU-specific saved value > the MODEL's own
|
||||
// recommended gen sampler (its config's KSampler section) > generic arch
|
||||
// default. The model knows itself better than our tiling default does.
|
||||
@@ -586,9 +678,14 @@ export async function prepareUpscale(entry) {
|
||||
caps.samplerAlternates = ksampler.sampler_name_options || [];
|
||||
caps.schedulerAlternates = ksampler.scheduler_options || [];
|
||||
// …but the SELECTED default is what the image was generated with, so a
|
||||
// deliberate deviation carries through to the upscale.
|
||||
// deliberate deviation carries through to the upscale — EXCEPT Krea 2, whose
|
||||
// re-detail recipe is its own (euler is the house default). Inheriting a
|
||||
// stale gen sampler from an older er_sde render would silently override that
|
||||
// default; same deliberate skip as the flux2 refine below.
|
||||
const inherited = workflowSamplerChoice(data.workflow, combos.samplers, combos.schedulers);
|
||||
caps.defaultSampler = inherited?.sampler || caps.recommendedSampler;
|
||||
caps.defaultSampler = architecture === "krea2"
|
||||
? caps.recommendedSampler
|
||||
: (inherited?.sampler || caps.recommendedSampler);
|
||||
caps.defaultScheduler = inherited?.scheduler || caps.recommendedScheduler;
|
||||
caps.advancedDefaults = usduBuildDefaults(architecture, settings);
|
||||
caps.architecture = architecture;
|
||||
@@ -620,7 +717,8 @@ export async function prepareUpscale(entry) {
|
||||
(m.architecture === "sdxl" && support.sdxl)
|
||||
|| (m.architecture === "qwen_edit" && support.qwen)
|
||||
|| (m.architecture === "flux" && support.flux1)
|
||||
|| (m.architecture === "zimage" && support.zimage));
|
||||
|| (m.architecture === "zimage" && support.zimage)
|
||||
|| (m.architecture === "krea2" && support.krea2));
|
||||
caps.enginePromptDefaults = { qwen: QWEN_ENHANCE_INSTRUCTION, sdxlFallback: SDXL_TILE_NEUTRAL_POSITIVE };
|
||||
// Climb-model picker: the whole installed ESRGAN-family list, with the
|
||||
// empirically-settled v1 pick as the recommendation. Degradation-trained
|
||||
@@ -629,8 +727,15 @@ export async function prepareUpscale(entry) {
|
||||
caps.recommendedUpscaleModel = defaultUpscaleModelPick(caps.upscaleModelOptions);
|
||||
// Restore recommendation, by SOURCE: a jpeg or small input likely carries
|
||||
// degradation a plain GAN climb would smear (the 736px webcam eye-smear);
|
||||
// clean renders gain nothing from the restore round-trip.
|
||||
caps.recommendedRestore = caps.seedvr2Available
|
||||
// clean renders gain nothing from the restore round-trip. A GENERATED render
|
||||
// (embedded sampler) is clean by construction — a small/square one (1024²) is
|
||||
// NOT degraded, so don't let the size rule false-trigger SeedVR2 on it; only
|
||||
// raw imported photos (no generation workflow) get the restore recommendation.
|
||||
// Matches the prior A/B: UltraSharp beats SeedVR2 on clean renders.
|
||||
const srcNodes = data.workflow?.nodes || [];
|
||||
const isRender = srcNodes.some((n) => n.type === "KSampler" || n.type === "KSamplerAdvanced"
|
||||
|| n.type === "SamplerCustom" || n.type === "SamplerCustomAdvanced");
|
||||
caps.recommendedRestore = caps.seedvr2Available && !isRender
|
||||
&& (/\.jpe?g$/i.test(data.filename || "") || Math.max(data.width || 0, data.height || 0) < 1100)
|
||||
? "seedvr2" : "none";
|
||||
// DIAGNOSTIC: why the engine picker may show only "Model climb only". If
|
||||
@@ -1108,10 +1213,11 @@ function setNamedWidget(node, name, value) {
|
||||
|
||||
// SeedVR2 floor: restore at trained scale, then the classic ESRGAN climb to
|
||||
// the same 4× target the UltraSharp floor produces, lanczos-fit to exact dims.
|
||||
function buildSeedvr2FloorGraph(data, savePrefix = "", previewOnly = false, climbModel = null) {
|
||||
function buildSeedvr2FloorGraph(data, savePrefix = "", previewOnly = false, climbModel = null, scale = 0) {
|
||||
const LG = window.LiteGraph;
|
||||
const workflowId = freshWorkflowId();
|
||||
const graph = new LG.LGraph(structuredClone({ ...EMPTY_WORKFLOW, id: workflowId }));
|
||||
graph.last_node_id = offscreenIdBase(); // render-node ids must stay off the live canvas — see offscreen-graph.js
|
||||
const load = LG.createNode("LoadImage");
|
||||
const dit = LG.createNode("SeedVR2LoadDiTModel");
|
||||
const vae = LG.createNode("SeedVR2LoadVAEModel");
|
||||
@@ -1119,8 +1225,9 @@ function buildSeedvr2FloorGraph(data, savePrefix = "", previewOnly = false, clim
|
||||
const save = LG.createNode(previewOnly ? "PreviewImage" : "SaveImage");
|
||||
if (!load || !dit || !vae || !up || !save) return null;
|
||||
|
||||
const targetW = (data.width || 0) * 4;
|
||||
const targetH = (data.height || 0) * 4;
|
||||
const eff = scale > 0 ? scale : 4; // honor the Scale slider; default to 4x when none given
|
||||
const targetW = (data.width || 0) * eff;
|
||||
const targetH = (data.height || 0) * eff;
|
||||
const targetShort = Math.min(targetW, targetH);
|
||||
const stageShort = targetShort > 0
|
||||
? Math.min(SEEDVR2_STAGE_MAX_SHORT_EDGE, targetShort)
|
||||
@@ -1447,7 +1554,11 @@ function buildFlux2RefineGraph(graph, workflowId, data, options, config) {
|
||||
const prefix = (options.savePrefix || "").trim() || "upscale/upscale";
|
||||
if (prefixW) { prefixW.value = prefix; prefixW.callback?.(prefix); }
|
||||
}
|
||||
if (typeof options.prompt === "string" && options.prompt.trim()) {
|
||||
// A cleared prompt (empty string) is an explicit choice — override with it so
|
||||
// the upscale drops the source render's character conditioning instead of
|
||||
// silently re-detailing every tile with it (the "Ryu's face on a cropped arm"
|
||||
// bug). Only skip when the modal passed no prompt at all (undefined).
|
||||
if (typeof options.prompt === "string") {
|
||||
overrideGraphPrompt(graph, options.prompt);
|
||||
}
|
||||
return { graph, workflowId };
|
||||
@@ -1457,14 +1568,15 @@ function buildFlux2RefineGraph(graph, workflowId, data, options, config) {
|
||||
// images with no usable metadata (external files, non-PromptChain workflows).
|
||||
// engine "seedvr2" puts a restoration stage in front; it falls back to the
|
||||
// plain pair if the cluster can't build (pack uninstalled mid-session).
|
||||
function buildEsrganFloorGraph(data, savePrefix = "", previewOnly = false, engine = "ultrasharp", climbModel = null) {
|
||||
function buildEsrganFloorGraph(data, savePrefix = "", previewOnly = false, engine = "ultrasharp", climbModel = null, scale = 0) {
|
||||
if (engine === "seedvr2") {
|
||||
const built = buildSeedvr2FloorGraph(data, savePrefix, previewOnly, climbModel);
|
||||
const built = buildSeedvr2FloorGraph(data, savePrefix, previewOnly, climbModel, scale);
|
||||
if (built) return built;
|
||||
}
|
||||
const LG = window.LiteGraph;
|
||||
const workflowId = freshWorkflowId();
|
||||
const graph = new LG.LGraph(structuredClone({ ...EMPTY_WORKFLOW, id: workflowId }));
|
||||
graph.last_node_id = offscreenIdBase(); // render-node ids must stay off the live canvas — see offscreen-graph.js
|
||||
const load = LG.createNode("LoadImage");
|
||||
const loader = LG.createNode("UpscaleModelLoader");
|
||||
const apply = LG.createNode("ImageUpscaleWithModel");
|
||||
@@ -1480,15 +1592,34 @@ function buildEsrganFloorGraph(data, savePrefix = "", previewOnly = false, engin
|
||||
load.pos = [80, 200];
|
||||
loader.pos = [80, 540];
|
||||
apply.pos = [500, 240];
|
||||
save.pos = [780, 240];
|
||||
graph.add(load); graph.add(loader); graph.add(apply); graph.add(save);
|
||||
graph.add(load); graph.add(loader); graph.add(apply);
|
||||
setImageWidget(load, data.input_ref);
|
||||
pickUpscaleModel(loader, climbModel);
|
||||
const prefix = save.widgets?.find((w) => w.name === "filename_prefix");
|
||||
if (prefix) { prefix.value = savePrefix || "upscale/upscale"; prefix.callback?.(prefix.value); }
|
||||
load.connect(0, apply, inputIdx(apply, "image", 1));
|
||||
loader.connect(0, apply, inputIdx(apply, "upscale_model", 0));
|
||||
apply.connect(0, save, 0);
|
||||
|
||||
// The model upscales by its own fixed factor (e.g. 4x); a lanczos fit lets the
|
||||
// Scale slider actually target a size (ESRGAN climbs to its factor, then fits
|
||||
// up/down to source*scale). No scale given → keep the raw model-factor output.
|
||||
let out = apply;
|
||||
const w = data.width || 0, h = data.height || 0;
|
||||
if (scale > 0 && w && h) {
|
||||
const fit = LG.createNode("ImageScale");
|
||||
if (fit) {
|
||||
fit.pos = [780, 240];
|
||||
graph.add(fit);
|
||||
setNamedWidget(fit, "upscale_method", "lanczos");
|
||||
setNamedWidget(fit, "width", Math.round(w * scale));
|
||||
setNamedWidget(fit, "height", Math.round(h * scale));
|
||||
apply.connect(0, fit, inputIdx(fit, "image", 0));
|
||||
out = fit;
|
||||
}
|
||||
}
|
||||
save.pos = [out.pos[0] + 280, 240];
|
||||
graph.add(save);
|
||||
const prefix = save.widgets?.find((w) => w.name === "filename_prefix");
|
||||
if (prefix) { prefix.value = savePrefix || "upscale/upscale"; prefix.callback?.(prefix.value); }
|
||||
out.connect(0, save, 0);
|
||||
return { graph, workflowId };
|
||||
}
|
||||
|
||||
@@ -1595,7 +1726,9 @@ function buildFlux1TileStage(graph, LG, place, opts) {
|
||||
let unetFile = opts.unetName || null;
|
||||
if (unetFile && Array.isArray(unetOpts) && !unetOpts.includes(unetFile)) unetOpts.push(unetFile);
|
||||
if (!unetFile) {
|
||||
unetFile = unetOpts.find((o) => /krea/i.test(o))
|
||||
// /krea(?!2)/ matches the FLUX.1 "flux1-krea-dev" UNET but must NOT grab
|
||||
// the separate "krea2_turbo_*" DiT (its own engine, different VAE/CLIP).
|
||||
unetFile = unetOpts.find((o) => /krea(?!2)/i.test(o))
|
||||
|| unetOpts.find((o) => /flux1/i.test(o) && !/kontext|schnell|fill|canny|depth|redux/i.test(o));
|
||||
}
|
||||
const clip = LG.createNode("DualCLIPLoader");
|
||||
@@ -1721,6 +1854,76 @@ function buildZImageTileStage(graph, LG, place, opts) {
|
||||
return { usdu, sideInputIds: [unet.id, clip.id, vae.id, aura.id, posEnc.id, negEnc.id] };
|
||||
}
|
||||
|
||||
// Krea 2 tile cluster (DiT, closest to Z-Image): UNET feeds the USDU model
|
||||
// DIRECTLY — Krea 2's schedule shift is baked into the model class, so there's
|
||||
// NO ModelSamplingAuraFlow. CLIP (qwen3vl_4b, type "krea2") drives a neutral
|
||||
// positive + empty negative; krea2 carries cfg natively (cfg 1.0), so NO
|
||||
// FluxGuidance and NO tile ControlNet. Turbo-distilled: euler/simple/8 steps.
|
||||
function buildKrea2TileStage(graph, LG, place, opts) {
|
||||
const reg = LG.registered_node_types || {};
|
||||
if (!reg["UltimateSDUpscaleNoUpscale"]) return null;
|
||||
const unet = LG.createNode("UNETLoader");
|
||||
const unetW = unet?.widgets?.find((x) => x.name === "unet_name");
|
||||
const unetOpts = unetW?.options?.values || [];
|
||||
let unetFile = opts.unetName || null;
|
||||
if (unetFile && Array.isArray(unetOpts) && !unetOpts.includes(unetFile)) unetOpts.push(unetFile);
|
||||
if (!unetFile) unetFile = unetOpts.find((o) => /krea2/i.test(o));
|
||||
const clip = LG.createNode("CLIPLoader");
|
||||
const clipFile = (clip?.widgets?.find((x) => x.name === "clip_name")?.options?.values || [])
|
||||
.find((o) => /qwen3vl.*4b|qwen3.?vl.*4b/i.test(o));
|
||||
const vae = LG.createNode("VAELoader");
|
||||
// Use Krea 2's native VAE (qwen_image_vae) for the tile encode AND decode. A
|
||||
// re-detailed section is composited back into pixels that were generated and
|
||||
// decoded with qwen_image_vae; decoding the section through any other VAE
|
||||
// (e.g. wan_2.1_vae) shifts its colour/texture rendition and leaves an
|
||||
// unblendable seam — and encoding through it lands the tile in a latent the
|
||||
// Krea 2 UNET reads slightly wrong. A ground-truth plain i2i on the same crop
|
||||
// is clean with this VAE. Fall back to a Wan 2.1 VAE only if it isn't present.
|
||||
const vaeOpts = vae?.widgets?.find((x) => x.name === "vae_name")?.options?.values || [];
|
||||
const vaeFile = vaeOpts.find((o) => /qwen.*image.*vae/i.test(o))
|
||||
|| vaeOpts.find((o) => /wan.*2[._]?1.*vae/i.test(o) && !/upscale|2x/i.test(o));
|
||||
const posEnc = LG.createNode("CLIPTextEncode");
|
||||
const negEnc = LG.createNode("CLIPTextEncode");
|
||||
const usdu = LG.createNode("UltimateSDUpscaleNoUpscale");
|
||||
if (!(unet && unetFile && clip && clipFile && vae && vaeFile && posEnc && negEnc && usdu)) return null;
|
||||
place(unet, 0, 0);
|
||||
place(clip, 0, 180);
|
||||
place(vae, 0, 360);
|
||||
place(posEnc, 300, -80);
|
||||
place(negEnc, 300, 100);
|
||||
place(usdu, 860, 280);
|
||||
setNamedWidget(unet, "unet_name", unetFile);
|
||||
setNamedWidget(unet, "weight_dtype", "default");
|
||||
setNamedWidget(clip, "clip_name", clipFile);
|
||||
setNamedWidget(clip, "type", "krea2");
|
||||
setNamedWidget(vae, "vae_name", vaeFile);
|
||||
setNamedWidget(posEnc, "text", opts.positiveText || SDXL_TILE_NEUTRAL_POSITIVE);
|
||||
setNamedWidget(negEnc, "text", "");
|
||||
clip.connect(0, posEnc, inputIdx(posEnc, "clip", 0));
|
||||
clip.connect(0, negEnc, inputIdx(negEnc, "clip", 0));
|
||||
opts.imageNode.connect(opts.imageSlot || 0, usdu, inputIdx(usdu, "upscaled_image", 0));
|
||||
unet.connect(0, usdu, inputIdx(usdu, "model", 1));
|
||||
posEnc.connect(0, usdu, inputIdx(usdu, "positive", 2));
|
||||
negEnc.connect(0, usdu, inputIdx(usdu, "negative", 3));
|
||||
vae.connect(0, usdu, inputIdx(usdu, "vae", 4));
|
||||
setNamedWidget(usdu, "seed", Math.floor(Math.random() * 2 ** 32));
|
||||
setNamedWidget(usdu, "steps", opts.steps ?? 8);
|
||||
setNamedWidget(usdu, "cfg", opts.cfg ?? 1.0);
|
||||
setNamedWidget(usdu, "sampler_name", opts.sampler || "er_sde");
|
||||
setNamedWidget(usdu, "scheduler", opts.scheduler || "simple");
|
||||
setNamedWidget(usdu, "denoise", typeof opts.denoise === "number" ? opts.denoise : 0.45);
|
||||
setNamedWidget(usdu, "tile_width", opts.tileWidth ?? 1024);
|
||||
setNamedWidget(usdu, "tile_height", opts.tileHeight ?? 1024);
|
||||
setNamedWidget(usdu, "mask_blur", opts.maskBlur ?? 32);
|
||||
setNamedWidget(usdu, "tile_padding", opts.tilePadding ?? 64);
|
||||
setNamedWidget(usdu, "seam_fix_mode", opts.seamFixMode || "None");
|
||||
if (opts.modeType) setNamedWidget(usdu, "mode_type", opts.modeType);
|
||||
if (typeof opts.seamFixDenoise === "number") setNamedWidget(usdu, "seam_fix_denoise", opts.seamFixDenoise);
|
||||
setNamedWidget(usdu, "force_uniform_tiles", true);
|
||||
setNamedWidget(usdu, "tiled_decode", false);
|
||||
return { usdu, sideInputIds: [unet.id, clip.id, vae.id, posEnc.id, negEnc.id] };
|
||||
}
|
||||
|
||||
// The modal's tuning knobs in tile-stage shape — shared by every tile engine.
|
||||
function tileStageOpts(options) {
|
||||
return {
|
||||
@@ -1749,13 +1952,10 @@ function buildTiledEngineGraph(data, options = {}, buildStage) {
|
||||
const LG = window.LiteGraph;
|
||||
const workflowId = freshWorkflowId();
|
||||
const graph = new LG.LGraph(structuredClone({ ...EMPTY_WORKFLOW, id: workflowId }));
|
||||
graph.last_node_id = offscreenIdBase(); // render-node ids must stay off the live canvas — see offscreen-graph.js
|
||||
const placeAt = (node, x, y) => { node.pos = [x, y]; graph.add(node); };
|
||||
const load = LG.createNode("LoadImage");
|
||||
const save = LG.createNode(options.previewOnly ? "PreviewImage" : "SaveImage");
|
||||
if (!load || !save) return null;
|
||||
if (data.filename) load.title = data.filename;
|
||||
placeAt(load, 80, 200);
|
||||
setImageWidget(load, data.input_ref);
|
||||
if (!save) return null;
|
||||
|
||||
const w = data.width || 0;
|
||||
const h = data.height || 0;
|
||||
@@ -1766,54 +1966,74 @@ function buildTiledEngineGraph(data, options = {}, buildStage) {
|
||||
const targetW = Math.round(w * chosenScale) || 2048;
|
||||
const targetH = Math.round(h * chosenScale) || 2048;
|
||||
|
||||
let imageSource = load;
|
||||
let needsModelClimb = chosenScale > 1.02;
|
||||
// Optional SeedVR2 restoration in front of the climb: degraded sources
|
||||
// (webcam blur, jpeg) smear through a plain GAN climb — A/B'd on a 736px
|
||||
// jpeg, the eye region smeared via UltraSharp but came back clean when
|
||||
// SeedVR2 rebuilt structure at its trained scale first. The tile CN then
|
||||
// pins restored structure instead of pinning the smear.
|
||||
if (options.climbStage === "seedvr2" && LG.registered_node_types?.["SeedVR2VideoUpscaler"]) {
|
||||
const dit = LG.createNode("SeedVR2LoadDiTModel");
|
||||
const seedvrVae = LG.createNode("SeedVR2LoadVAEModel");
|
||||
const restore = LG.createNode("SeedVR2VideoUpscaler");
|
||||
if (dit && seedvrVae && restore) {
|
||||
placeAt(dit, 80, 460);
|
||||
placeAt(seedvrVae, 80, 660);
|
||||
placeAt(restore, 400, 200);
|
||||
const ditModels = dit.widgets?.find((x) => x.name === "model")?.options?.values || [];
|
||||
if (ditModels.length) setNamedWidget(dit, "model", pickSeedvr2DitModel(ditModels));
|
||||
const targetShort = Math.min(targetW, targetH);
|
||||
const stageShort = Math.min(SEEDVR2_STAGE_MAX_SHORT_EDGE, targetShort);
|
||||
setNamedWidget(restore, "resolution", stageShort);
|
||||
setNamedWidget(restore, "batch_size", 1);
|
||||
setNamedWidget(restore, "input_noise_scale", SEEDVR2_INPUT_NOISE);
|
||||
load.connect(0, restore, inputIdx(restore, "image", 0));
|
||||
dit.connect(0, restore, inputIdx(restore, "dit", 1));
|
||||
seedvrVae.connect(0, restore, inputIdx(restore, "vae", 2));
|
||||
imageSource = restore;
|
||||
needsModelClimb = targetShort > stageShort;
|
||||
// `fit` is the image (already at final/target dims) the engine stage tiles.
|
||||
// Normally we climb the source here (optional SeedVR2 restore → UltraSharp →
|
||||
// exact-fit lanczos). On a cache hit the enlarge already ran in a prior
|
||||
// pre-pass, so load that saved base directly and skip the whole climb.
|
||||
let fit;
|
||||
if (options.preEnlargedRef) {
|
||||
fit = LG.createNode("LoadImage");
|
||||
if (!fit) return null;
|
||||
fit.title = "enlarged base (cached)";
|
||||
placeAt(fit, 740, 200);
|
||||
setImageWidget(fit, options.preEnlargedRef);
|
||||
} else {
|
||||
const load = LG.createNode("LoadImage");
|
||||
if (!load) return null;
|
||||
if (data.filename) load.title = data.filename;
|
||||
placeAt(load, 80, 200);
|
||||
setImageWidget(load, data.input_ref);
|
||||
|
||||
let imageSource = load;
|
||||
let needsModelClimb = chosenScale > 1.02;
|
||||
// Optional SeedVR2 restoration in front of the climb: degraded sources
|
||||
// (webcam blur, jpeg) smear through a plain GAN climb — A/B'd on a 736px
|
||||
// jpeg, the eye region smeared via UltraSharp but came back clean when
|
||||
// SeedVR2 rebuilt structure at its trained scale first. The tile CN then
|
||||
// pins restored structure instead of pinning the smear.
|
||||
if (options.climbStage === "seedvr2" && LG.registered_node_types?.["SeedVR2VideoUpscaler"]) {
|
||||
const dit = LG.createNode("SeedVR2LoadDiTModel");
|
||||
const seedvrVae = LG.createNode("SeedVR2LoadVAEModel");
|
||||
const restore = LG.createNode("SeedVR2VideoUpscaler");
|
||||
if (dit && seedvrVae && restore) {
|
||||
placeAt(dit, 80, 460);
|
||||
placeAt(seedvrVae, 80, 660);
|
||||
placeAt(restore, 400, 200);
|
||||
const ditModels = dit.widgets?.find((x) => x.name === "model")?.options?.values || [];
|
||||
if (ditModels.length) setNamedWidget(dit, "model", pickSeedvr2DitModel(ditModels));
|
||||
const targetShort = Math.min(targetW, targetH);
|
||||
const stageShort = Math.min(SEEDVR2_STAGE_MAX_SHORT_EDGE, targetShort);
|
||||
setNamedWidget(restore, "resolution", stageShort);
|
||||
setNamedWidget(restore, "batch_size", 1);
|
||||
setNamedWidget(restore, "input_noise_scale", SEEDVR2_INPUT_NOISE);
|
||||
setNamedWidget(restore, "seed", 42); // match buildEnlargePrompt so the cached base == this fused base
|
||||
load.connect(0, restore, inputIdx(restore, "image", 0));
|
||||
dit.connect(0, restore, inputIdx(restore, "dit", 1));
|
||||
seedvrVae.connect(0, restore, inputIdx(restore, "vae", 2));
|
||||
imageSource = restore;
|
||||
needsModelClimb = targetShort > stageShort;
|
||||
}
|
||||
}
|
||||
if (needsModelClimb) {
|
||||
const upLoader = LG.createNode("UpscaleModelLoader");
|
||||
const upApply = LG.createNode("ImageUpscaleWithModel");
|
||||
if (!upLoader || !upApply) return null;
|
||||
placeAt(upLoader, 80, 880);
|
||||
placeAt(upApply, 460, 460);
|
||||
pickUpscaleModel(upLoader, options.climbModel);
|
||||
imageSource.connect(0, upApply, inputIdx(upApply, "image", 1));
|
||||
upLoader.connect(0, upApply, inputIdx(upApply, "upscale_model", 0));
|
||||
imageSource = upApply;
|
||||
}
|
||||
// The tile pass runs AT final resolution — exact-fit first, then re-detail.
|
||||
fit = LG.createNode("ImageScale");
|
||||
if (!fit) return null;
|
||||
placeAt(fit, 740, 200);
|
||||
setNamedWidget(fit, "upscale_method", "lanczos");
|
||||
setNamedWidget(fit, "width", targetW);
|
||||
setNamedWidget(fit, "height", targetH);
|
||||
imageSource.connect(0, fit, inputIdx(fit, "image", 0));
|
||||
}
|
||||
if (needsModelClimb) {
|
||||
const upLoader = LG.createNode("UpscaleModelLoader");
|
||||
const upApply = LG.createNode("ImageUpscaleWithModel");
|
||||
if (!upLoader || !upApply) return null;
|
||||
placeAt(upLoader, 80, 880);
|
||||
placeAt(upApply, 460, 460);
|
||||
pickUpscaleModel(upLoader, options.climbModel);
|
||||
imageSource.connect(0, upApply, inputIdx(upApply, "image", 1));
|
||||
upLoader.connect(0, upApply, inputIdx(upApply, "upscale_model", 0));
|
||||
imageSource = upApply;
|
||||
}
|
||||
// The tile pass runs AT final resolution — exact-fit first, then re-detail.
|
||||
const fit = LG.createNode("ImageScale");
|
||||
if (!fit) return null;
|
||||
placeAt(fit, 740, 200);
|
||||
setNamedWidget(fit, "upscale_method", "lanczos");
|
||||
setNamedWidget(fit, "width", targetW);
|
||||
setNamedWidget(fit, "height", targetH);
|
||||
imageSource.connect(0, fit, inputIdx(fit, "image", 0));
|
||||
|
||||
const stage = buildStage(graph, LG,
|
||||
(node, dx, dy) => placeAt(node, 1040 + dx, 480 + dy), fit);
|
||||
@@ -1867,6 +2087,19 @@ function buildZImageEngineGraph(data, options = {}) {
|
||||
}));
|
||||
}
|
||||
|
||||
// "Re-detail with Krea 2": picked Krea 2 Turbo UNET in the cfg-1/euler/simple
|
||||
// 8-step regime, model wired straight to the USDU (no ModelSampling, no
|
||||
// FluxGuidance, no tile CN). 0.45 denoise re-details while holding structure.
|
||||
function buildKrea2EngineGraph(data, options = {}) {
|
||||
return buildTiledEngineGraph(data, options, (graph, LG, place, fit) =>
|
||||
buildKrea2TileStage(graph, LG, place, {
|
||||
imageNode: fit,
|
||||
unetName: options.engineModel?.filename || null,
|
||||
positiveText: (options.prompt || "").trim() || SDXL_TILE_NEUTRAL_POSITIVE,
|
||||
...tileStageOpts(options),
|
||||
}));
|
||||
}
|
||||
|
||||
// Standalone Qwen-Image-Edit engine: instruction-driven whole-frame re-render
|
||||
// at ~2MP (the model's working size — bigger grows plastic skin, and the
|
||||
// stock encoder downsamples its conditioning copy regardless), then the
|
||||
@@ -1879,6 +2112,7 @@ function buildQwenEditEngineGraph(data, options = {}) {
|
||||
|| !reg["ModelSamplingAuraFlow"] || !reg["CFGNorm"]) return null;
|
||||
const workflowId = freshWorkflowId();
|
||||
const graph = new LG.LGraph(structuredClone({ ...EMPTY_WORKFLOW, id: workflowId }));
|
||||
graph.last_node_id = offscreenIdBase(); // render-node ids must stay off the live canvas — see offscreen-graph.js
|
||||
const placeAt = (node, x, y) => { node.pos = [x, y]; graph.add(node); };
|
||||
|
||||
const load = LG.createNode("LoadImage");
|
||||
@@ -2022,14 +2256,17 @@ export async function buildUpscaleGraph(prepared, options = {}) {
|
||||
const engine = options.engine || "source";
|
||||
// Picker engines build standalone graphs — any source image works,
|
||||
// graftable or not.
|
||||
if (engine === "sdxl-ckpt" || engine === "qwen-edit" || engine === "flux1-unet" || engine === "zimage-unet") {
|
||||
if (engine === "sdxl-ckpt" || engine === "qwen-edit" || engine === "flux1-unet"
|
||||
|| engine === "zimage-unet" || engine === "krea2-unet") {
|
||||
const built = engine === "sdxl-ckpt"
|
||||
? buildSdxlCkptEngineGraph(data, { ...options, savePrefix })
|
||||
: engine === "flux1-unet"
|
||||
? buildFlux1EngineGraph(data, { ...options, savePrefix })
|
||||
: engine === "zimage-unet"
|
||||
? buildZImageEngineGraph(data, { ...options, savePrefix })
|
||||
: buildQwenEditEngineGraph(data, { ...options, savePrefix });
|
||||
: engine === "krea2-unet"
|
||||
? buildKrea2EngineGraph(data, { ...options, savePrefix })
|
||||
: buildQwenEditEngineGraph(data, { ...options, savePrefix });
|
||||
if (built) return built;
|
||||
toast("warn", "Couldn't build that upscale engine — using a plain model upscale instead.");
|
||||
return buildEsrganFloorGraph(data, savePrefix, options.previewOnly, "ultrasharp", options.climbModel);
|
||||
@@ -2037,7 +2274,11 @@ export async function buildUpscaleGraph(prepared, options = {}) {
|
||||
// Plain engines are explicit main-engine choices now; non-graftable
|
||||
// images keep them as their only option.
|
||||
if (!caps.graftable || engine === "ultrasharp" || engine === "seedvr2") {
|
||||
return buildEsrganFloorGraph(data, savePrefix, options.previewOnly, engine, options.climbModel);
|
||||
const longest = Math.max(data.width || 0, data.height || 0);
|
||||
const scale = typeof options.upscaleBy === "number" && options.upscaleBy >= 1
|
||||
? options.upscaleBy
|
||||
: longest > 0 ? Math.min(8, Math.max(1, Math.round((4096 / longest) * 100) / 100)) : 2;
|
||||
return buildEsrganFloorGraph(data, savePrefix, options.previewOnly, engine, options.climbModel, scale);
|
||||
}
|
||||
|
||||
const mode = options.mode || caps.defaultMode || "prompt";
|
||||
@@ -2069,6 +2310,7 @@ export async function buildUpscaleGraph(prepared, options = {}) {
|
||||
return buildEsrganFloorGraph(data, savePrefix, options.previewOnly, options.engine, options.climbModel);
|
||||
}
|
||||
if (!pcNode || !injectUpscaler(pcNode, config, { regional: mode === "regional", pinFlux2TileModel: true })) {
|
||||
console.warn(`[PromptChain][upscale] source graft -> PLAIN fallback: pcNode=${pcNode?.id ?? "NONE"} arch=${config?.architecture || caps.architecture} graphTypes=[${graph._nodes.map((n) => n.comfyClass).join(",")}]`);
|
||||
toast("warn", "No attachment point in this image's workflow — using a plain model upscale instead.");
|
||||
return buildEsrganFloorGraph(data, savePrefix, options.previewOnly, options.engine, options.climbModel);
|
||||
}
|
||||
@@ -2076,6 +2318,25 @@ export async function buildUpscaleGraph(prepared, options = {}) {
|
||||
} else if (mode === "regional") {
|
||||
ensureRegionalConditioning(graph, LG, usNode);
|
||||
}
|
||||
// Krea 2 renders carry the Qwen-Image VAE, whose decoder bakes a periodic
|
||||
// grid/scale artifact into every tile USDU re-decodes. The Wan 2.1 VAE
|
||||
// shares the SAME frozen encoder (tiles encode to an identical Krea2-
|
||||
// compatible latent) but its decoder has no grid artifact — repoint the
|
||||
// grafted decode to it so the source-model upscale comes out clean, same as
|
||||
// the standalone Krea2 tile engine. Excludes the 2x variant (12-ch/2x output
|
||||
// would break tiling); only fires for a Krea 2 source on the Qwen VAE.
|
||||
const isKrea2Source = caps.architecture === "krea2"
|
||||
|| graph._nodes.some((n) => n.comfyClass === "CLIPLoader"
|
||||
&& n.widgets?.find((w) => w.name === "type")?.value === "krea2");
|
||||
if (isKrea2Source) {
|
||||
for (const vl of graph._nodes.filter((n) => n.comfyClass === "VAELoader")) {
|
||||
const vaeW = vl.widgets?.find((w) => w.name === "vae_name");
|
||||
if (!vaeW || !/qwen.*image.*vae/i.test(vaeW.value || "")) continue;
|
||||
const wan = (vaeW.options?.values || [])
|
||||
.find((o) => /wan.*2[._]?1.*vae/i.test(o) && !/upscale|2x/i.test(o));
|
||||
if (wan) { vaeW.value = wan; vaeW.callback?.(wan); }
|
||||
}
|
||||
}
|
||||
if (mode === "regional" && usNode) {
|
||||
// The installed USDU port's crop_mask only remaps the FIRST crop region
|
||||
// of a batch — batched tiles would smear every region mask across the
|
||||
|
||||
+16
-3
@@ -23,6 +23,7 @@ import { createHighlightExtension } from "./lib/active-label-highlight.js";
|
||||
import { createWildcardBadgeExtension } from "./lib/wildcard-badge.js";
|
||||
import { updateDynamicOutputs, initializeOutputSlots, healRegionsLinks, invalidateFluxCache, autoConnectToSampler } from "./lib/dynamic-outputs.js";
|
||||
import { setupIterateListeners, advanceAndCascade } from "./lib/iterate-hierarchy.js";
|
||||
import { setupShiftSync } from "./lib/krea2-shift-sync.js";
|
||||
import { getSwitchOptions, countLabeledOptions } from "./lib/label-utils.js";
|
||||
import { getWorkflowId, installSaveInterceptor, healWorkflowIdDrift } from "./lib/workflow-id.js";
|
||||
import { recordGeneration, fetchWorkflowImages, fetchWorkflowCount, subscribe as subscribeHistory, invalidateCache, getCachedImages, externallyTrackedPrompts, isExternalPrompt } from "./lib/history.js";
|
||||
@@ -366,8 +367,19 @@ function setupImagePreviewListeners() {
|
||||
// here describe the user's active graph, not that prompt's.
|
||||
if (detail.prompt_id && externallyTrackedPrompts.has(detail.prompt_id)) return;
|
||||
const nodeId = Number(detail.node);
|
||||
const pcNodes = _outputToPC.get(nodeId);
|
||||
if (!pcNodes?.length) return;
|
||||
let pcNodes = _outputToPC.get(nodeId);
|
||||
let wfId = _executionWorkflowId;
|
||||
if (!pcNodes?.length) {
|
||||
// The SaveImage didn't map to a PromptChain node via the execution_start
|
||||
// graph scan (trace miss / subgraph / workflow-id drift). Don't silently
|
||||
// drop the generation — diagnose, then attribute it to the active graph's
|
||||
// PromptChain node so the generation panel still catches it.
|
||||
if (!wfId) wfId = getWorkflowId();
|
||||
const fallbackPC = (app.graph?._nodes || []).find((n) => n.comfyClass === NODE_TYPE);
|
||||
console.warn(`[PromptChain][record] unmapped output node=${detail.node}->${nodeId} mapKeys=[${[..._outputToPC.keys()]}] wfId=${wfId} graphNodes=${app.graph?._nodes?.length} rootGraph=${!!app.rootGraph} fallbackPC=${fallbackPC?.id ?? "none"}`);
|
||||
if (!fallbackPC || !wfId) return;
|
||||
pcNodes = [fallbackPC];
|
||||
}
|
||||
|
||||
const img = detail.output.images[detail.output.images.length - 1];
|
||||
const params = new URLSearchParams({
|
||||
@@ -404,7 +416,7 @@ function setupImagePreviewListeners() {
|
||||
// the panel awaits it on filename-click to resolve the exact hash even
|
||||
// if the click lands before the POST response writes the entry to cache.
|
||||
const genPromise = recordGeneration(
|
||||
_executionWorkflowId,
|
||||
wfId,
|
||||
img.filename,
|
||||
img.subfolder || "",
|
||||
img.type || "output",
|
||||
@@ -882,6 +894,7 @@ app.registerExtension({
|
||||
|
||||
setup() {
|
||||
setupIterateListeners();
|
||||
setupShiftSync(); // krea2 RAW: keep the flow-shift node's w/h matched to render size
|
||||
setupImagePreviewListeners();
|
||||
checkOnboarding();
|
||||
loadSvelteNode(); // preload Svelte module in background
|
||||
|
||||
+1
-1
@@ -5,7 +5,7 @@
|
||||
|
||||
[project]
|
||||
name = "comfyui-promptchain"
|
||||
version = "0.1.3"
|
||||
version = "0.1.4"
|
||||
description = "Prompt-authoring and image-iteration suite for ComfyUI: structured prompts, 3D posing, model awareness, regional control, and layered editing."
|
||||
license = { file = "LICENSE" }
|
||||
requires-python = ">=3.10"
|
||||
|
||||
@@ -316,8 +316,9 @@
|
||||
// clicks skip the re-flatten AND let the server reuse its encoder cache.
|
||||
let flatBlobCache = null; // { blob, key }
|
||||
function flatKey() {
|
||||
// histIndex covers committed pixel edits; the rest covers uncommitted
|
||||
// visibility/opacity/blend toggles (they never enter history)
|
||||
// histIndex covers committed edits; the per-layer suffix also keys on
|
||||
// visibility/opacity/blend/offset so the cache stays correct mid-drag,
|
||||
// before an opacity release commits.
|
||||
return `${histIndex}|${activeIndex}|${layers.map((L) => `${L.id}:${L.visible}:${L.opacity}:${L.blend || ""}:${L.ox || 0}:${L.oy || 0}`).join(",")}`;
|
||||
}
|
||||
async function flattenedBlobCached() {
|
||||
@@ -3456,8 +3457,12 @@
|
||||
loadActiveBitmap();
|
||||
commitAction("Duplicate Layer");
|
||||
}
|
||||
function setLayerVisible(i, v) { layers[i] = { ...layers[i], visible: v }; }
|
||||
function setLayerOpacity(i, o) { layers[i] = { ...layers[i], opacity: o }; }
|
||||
function setLayerVisible(i, v) { layers[i] = { ...layers[i], visible: v }; commitAction("Visibility"); }
|
||||
// Opacity drags fire on every tick — mutate in place for the live preview and
|
||||
// commit ONCE on release (see the slider's onpointerdown/onchange) so the
|
||||
// change is undoable and dirties the doc without flooding the history ring.
|
||||
let opacityDragStart = null;
|
||||
function setLayerOpacity(i, o) { if (layers[i].opacity === o) return; layers[i] = { ...layers[i], opacity: o }; }
|
||||
|
||||
// Blend modes — CSS mix-blend-mode keywords double as canvas
|
||||
// globalCompositeOperation names, so the live stacked-canvas view and every
|
||||
@@ -3523,7 +3528,7 @@
|
||||
function commitRename() {
|
||||
const i = layers.findIndex((L) => L.id === renamingId);
|
||||
const name = renameText.trim();
|
||||
if (i >= 0 && name && name !== layers[i].name) layers[i] = { ...layers[i], name };
|
||||
if (i >= 0 && name && name !== layers[i].name) { layers[i] = { ...layers[i], name }; commitAction("Rename Layer"); }
|
||||
renamingId = null;
|
||||
}
|
||||
|
||||
@@ -3546,6 +3551,7 @@
|
||||
layers = rest;
|
||||
activeIndex = layers.findIndex((L) => L.id === activeId);
|
||||
dragId = null;
|
||||
commitAction("Reorder Layers");
|
||||
}
|
||||
|
||||
// Composite a set of layer indices (z-order) into one ImageData.
|
||||
@@ -4684,7 +4690,10 @@
|
||||
ondblclick={(e) => { e.stopPropagation(); startRename(L); }}>{L.name}{#if L.mask}<span class="pcr-ed-layer-blendtag" title="Has a layer mask">▦</span>{/if}{#if layerBlendCss(L)}<span class="pcr-ed-layer-blendtag">{L.blend}</span>{/if}</span>
|
||||
{/if}
|
||||
<input class="pcr-ed-layer-opacity" type="range" min="0" max="1" step="0.05"
|
||||
value={L.opacity} oninput={(e) => setLayerOpacity(i, +e.currentTarget.value)}
|
||||
value={L.opacity}
|
||||
onpointerdown={() => { opacityDragStart = L.opacity; }}
|
||||
oninput={(e) => setLayerOpacity(i, +e.currentTarget.value)}
|
||||
onchange={(e) => { if (opacityDragStart === null || +e.currentTarget.value !== opacityDragStart) commitAction("Opacity"); opacityDragStart = null; }}
|
||||
onclick={(e) => e.stopPropagation()} title="Opacity" />
|
||||
</div>
|
||||
{/each}
|
||||
|
||||
@@ -196,7 +196,16 @@
|
||||
.then((d) => { editDocLayers = d?.docs || {}; })
|
||||
.catch(() => {});
|
||||
}
|
||||
$effect(() => { if (fetchApi) refreshEditDocs(); });
|
||||
// A "Save edits" inside the editor writes the sidecar and dispatches this
|
||||
// event; without the listener the viewer's own ▦ layers badge stayed stale
|
||||
// until remount (only the gallery listened), so an in-place save looked like
|
||||
// it did nothing.
|
||||
$effect(() => {
|
||||
if (fetchApi) refreshEditDocs();
|
||||
const onEditDocsChanged = () => refreshEditDocs();
|
||||
window.addEventListener("promptchain:edit-docs-changed", onEditDocsChanged);
|
||||
return () => window.removeEventListener("promptchain:edit-docs-changed", onEditDocsChanged);
|
||||
});
|
||||
|
||||
// lineage state
|
||||
let lineageData = $state(null);
|
||||
@@ -1173,7 +1182,10 @@
|
||||
showNewGenBanner(state.elapsedSec);
|
||||
viewUpscaleResult(state.resultHash);
|
||||
} else if (state.phase === "cancelled") {
|
||||
cancelUpscale(); // back to the viewer exactly as it was
|
||||
// Back to the setup form with settings intact — don't kill the whole
|
||||
// modal on a cancelled run (matches inpaint/repose/extend/smooth). The
|
||||
// explicit Close button still calls cancelUpscale for a full exit.
|
||||
upscaleProgress = null;
|
||||
}
|
||||
});
|
||||
return;
|
||||
@@ -2946,7 +2958,7 @@
|
||||
height={editPrepared?.data?.height || 0}
|
||||
filename={editPrepared?.data?.filename || ""}
|
||||
caps={editPrepared?.caps}
|
||||
editDocHash={editPrepared ? (displayedHash || "") : ""}
|
||||
editDocHash={editPrepared && /^[0-9a-f]{64}$/.test(displayedHash || "") ? displayedHash : ""}
|
||||
{fetchApi}
|
||||
{apiURL}
|
||||
onSave={(blob, prefix) => onEditSave(editPrepared, blob, prefix)}
|
||||
|
||||
@@ -46,6 +46,8 @@
|
||||
let denoise = $state(0.5);
|
||||
let sampler = $state("");
|
||||
let scheduler = $state("");
|
||||
let steps = $state(0); // pre-filled to the engine's recipe on open (engineRecipe); 0 = inherit (source)
|
||||
let cfg = $state(0); // pre-filled to the engine's recipe on open; 0 = inherit (source)
|
||||
// Mask shaping (Advanced): grow dilates the painted mask, feather softens its
|
||||
// edge — both forwarded to PromptChain_MaskedDetail.
|
||||
let grow = $state(0);
|
||||
@@ -76,11 +78,12 @@
|
||||
let engineSel = $state("source");
|
||||
let engineEntry = $derived(caps?.engineModels?.find((m) => m.hash === engineSel) || null);
|
||||
let engineKind = $derived(engineEntry
|
||||
? (engineEntry.architecture === "qwen_edit" ? "qwen" : engineEntry.architecture === "flux" ? "flux1" : "sdxl")
|
||||
? (engineEntry.architecture === "qwen_edit" ? "qwen" : engineEntry.architecture === "flux" ? "flux1" : engineEntry.architecture === "krea2" ? "krea2" : "sdxl")
|
||||
: "source");
|
||||
let engineGroups = $derived({
|
||||
sdxl: (caps?.engineModels || []).filter((m) => m.architecture === "sdxl"),
|
||||
flux1: (caps?.engineModels || []).filter((m) => m.architecture === "flux"),
|
||||
krea2: (caps?.engineModels || []).filter((m) => m.architecture === "krea2"),
|
||||
qwen: (caps?.engineModels || []).filter((m) => m.architecture === "qwen_edit"),
|
||||
});
|
||||
// SearchableSelect feed — same rows/groups the native <select> carried.
|
||||
@@ -94,20 +97,40 @@
|
||||
...(engineGroups.flux1.length
|
||||
? [{ label: "FLUX.1 — re-render the masked region", options: engineGroups.flux1.map((m) => ({ value: m.hash, label: m.displayName })) }]
|
||||
: []),
|
||||
...(engineGroups.krea2.length
|
||||
? [{ label: "Krea 2 — re-render the masked region", options: engineGroups.krea2.map((m) => ({ value: m.hash, label: m.displayName })) }]
|
||||
: []),
|
||||
...(engineGroups.qwen.length
|
||||
? [{ label: "Qwen Edit — instruction edit (masked)", options: engineGroups.qwen.map((m) => ({ value: m.hash, label: m.displayName })) }]
|
||||
: []),
|
||||
]);
|
||||
let recSampler = $derived(engineKind === "qwen" || engineKind === "flux1"
|
||||
let recSampler = $derived(engineKind === "qwen" || engineKind === "flux1" || engineKind === "krea2"
|
||||
? (engineEntry?.gen?.sampler || "euler")
|
||||
: (engineEntry?.gen?.sampler || caps?.recommendedSampler));
|
||||
let recScheduler = $derived(engineKind === "qwen" || engineKind === "flux1"
|
||||
let recScheduler = $derived(engineKind === "qwen" || engineKind === "flux1" || engineKind === "krea2"
|
||||
? (engineEntry?.gen?.scheduler || "simple")
|
||||
: (engineEntry?.gen?.scheduler || caps?.recommendedScheduler));
|
||||
// Steps/CFG the modal pre-fills per engine, mirroring the inpaint builders'
|
||||
// own defaults (RAW 30/3.5 · Turbo 8/1.0 · SDXL 25/5 · Qwen 40/4 · Flux 25/1)
|
||||
// so the sliders show real values instead of a bare 0. The source model has
|
||||
// no fixed recipe — 0 tells the builder to inherit the image's own steps/cfg.
|
||||
function engineRecipe(entry) {
|
||||
const kind = entry
|
||||
? (entry.architecture === "qwen_edit" ? "qwen" : entry.architecture === "flux" ? "flux1" : entry.architecture === "krea2" ? "krea2" : "sdxl")
|
||||
: "source";
|
||||
const isRaw = kind === "krea2" && /raw/i.test(entry?.filename || "") && !/turbo/i.test(entry?.filename || "");
|
||||
if (kind === "krea2") return { steps: isRaw ? 30 : 8, cfg: isRaw ? 3.5 : 1.0 };
|
||||
if (kind === "sdxl") return { steps: entry?.gen?.steps || 25, cfg: entry?.gen?.cfg ?? 5 };
|
||||
if (kind === "qwen") return { steps: entry?.gen?.steps || 40, cfg: entry?.gen?.cfg ?? 4 };
|
||||
if (kind === "flux1") return { steps: entry?.gen?.steps || 25, cfg: 1.0 };
|
||||
return { steps: 0, cfg: 0 };
|
||||
}
|
||||
let recSteps = $derived(engineRecipe(engineEntry).steps);
|
||||
let recCfg = $derived(engineRecipe(engineEntry).cfg);
|
||||
function conditionOk(key) {
|
||||
// PuLID/IPAdapter patch an SDXL checkpoint UNET — the qwen/flux1
|
||||
// PuLID/IPAdapter patch an SDXL checkpoint UNET — the qwen/flux1/krea2
|
||||
// builders have no splice point.
|
||||
if (engineKind === "qwen" || engineKind === "flux1") return false;
|
||||
if (engineKind === "qwen" || engineKind === "flux1" || engineKind === "krea2") return false;
|
||||
return engineKind === "sdxl" || !!caps?.conditions?.[key]?.ok;
|
||||
}
|
||||
|
||||
@@ -176,11 +199,11 @@
|
||||
engineSel = value;
|
||||
const entry = caps?.engineModels?.find((m) => m.hash === value) || null;
|
||||
const kind = entry
|
||||
? (entry.architecture === "qwen_edit" ? "qwen" : entry.architecture === "flux" ? "flux1" : "sdxl")
|
||||
? (entry.architecture === "qwen_edit" ? "qwen" : entry.architecture === "flux" ? "flux1" : entry.architecture === "krea2" ? "krea2" : "sdxl")
|
||||
: "source";
|
||||
if (entry) {
|
||||
// The picked model's own gen-time sampler pick beats the source's.
|
||||
const fluxLike = kind === "qwen" || kind === "flux1";
|
||||
const fluxLike = kind === "qwen" || kind === "flux1" || kind === "krea2";
|
||||
sampler = entry.gen?.sampler || (fluxLike ? "euler" : sampler);
|
||||
scheduler = entry.gen?.scheduler || (fluxLike ? "simple" : scheduler);
|
||||
if (fluxLike && condition !== "none") condition = "none";
|
||||
@@ -190,6 +213,8 @@
|
||||
// Back on source: a condition the source architecture can't carry resets.
|
||||
if (condition !== "none" && !caps?.conditions?.[condition]?.ok) condition = "none";
|
||||
}
|
||||
const r = engineRecipe(entry);
|
||||
steps = r.steps; cfg = r.cfg;
|
||||
// The mode rows only apply to the source engine; switching engine changes
|
||||
// what the prompt MEANS, so re-seed deterministically (edits discarded).
|
||||
memoryPrompt = null;
|
||||
@@ -290,6 +315,8 @@
|
||||
const preEntry = caps.engineModels?.find((m) => m.hash === engineSel);
|
||||
if (preEntry?.gen?.sampler) sampler = preEntry.gen.sampler;
|
||||
if (preEntry?.gen?.scheduler) scheduler = preEntry.gen.scheduler;
|
||||
const r0 = engineRecipe(preEntry);
|
||||
steps = r0.steps; cfg = r0.cfg;
|
||||
if (memEngineValid && typeof mem.denoise === "number") denoise = mem.denoise;
|
||||
grow = 0;
|
||||
feather = 24;
|
||||
@@ -361,7 +388,7 @@
|
||||
if (typeof s.maskOpacity === "number") maskOpacity = s.maskOpacity;
|
||||
// Condition cluster — only when the resolved engine can carry it and the
|
||||
// condition is actually installed (flux/qwen engines can't).
|
||||
const fluxLike = engineKind === "flux1" || engineKind === "qwen";
|
||||
const fluxLike = engineKind === "flux1" || engineKind === "qwen" || engineKind === "krea2";
|
||||
// conditionOk (not the raw caps flag) so an SDXL-engine pick re-enables the
|
||||
// condition the same way the live dropdown does — else it's silently dropped.
|
||||
if (typeof s.condition === "string" && !(fluxLike && s.condition !== "none")
|
||||
@@ -753,6 +780,7 @@
|
||||
activeTab = "output";
|
||||
tracker = onRun({
|
||||
maskBlob, prompt, denoise, grow, feather, sampler: sampler || null, scheduler: scheduler || null,
|
||||
steps: steps > 0 ? steps : undefined, cfg: cfg > 0 ? cfg : undefined,
|
||||
// Source engine carries the mode so a Basic pick forces a flat encode
|
||||
// even if the prompt still has $blocks; engine models ignore it. (Moved
|
||||
// content rides regional + the ImageViewer-injected condOffset, which
|
||||
@@ -760,6 +788,7 @@
|
||||
mode: engineKind === "source" ? mode : undefined,
|
||||
engine: engineKind === "sdxl" ? "sdxl-ckpt"
|
||||
: engineKind === "flux1" ? "flux1-unet"
|
||||
: engineKind === "krea2" ? "krea2-unet"
|
||||
: engineKind === "qwen" ? "qwen-edit" : "source",
|
||||
engineModel: engineEntry ? { hash: engineEntry.hash, filename: engineEntry.filename } : undefined,
|
||||
engineGen: engineEntry?.gen || undefined,
|
||||
@@ -1018,13 +1047,15 @@
|
||||
<div class="pcr-ip-hint">renders the painted region with this checkpoint — no render metadata needed, and PuLID / Style Reference work on any source</div>
|
||||
{:else if engineKind === "flux1"}
|
||||
<div class="pcr-ip-hint">re-renders the painted region with this FLUX.1 model at the denoise below — describe the content, strong on faces and eyes</div>
|
||||
{:else if engineKind === "krea2"}
|
||||
<div class="pcr-ip-hint">re-renders the painted region with this Krea 2 model at the denoise below — describe the content, sharp detail and clean structure</div>
|
||||
{:else if engineKind === "qwen"}
|
||||
<div class="pcr-ip-hint">follows your instruction inside the mask only — unmasked pixels stay untouched; the region renders at full model resolution</div>
|
||||
{/if}
|
||||
</div>
|
||||
{/if}
|
||||
{#if caps?.conditions && engineKind !== "qwen" && engineKind !== "flux1"}
|
||||
<!-- conditions patch an SDXL checkpoint UNET — the qwen/flux1
|
||||
{#if caps?.conditions && engineKind !== "qwen" && engineKind !== "flux1" && engineKind !== "krea2"}
|
||||
<!-- conditions patch an SDXL checkpoint UNET — the qwen/flux1/krea2
|
||||
engines have no splice point, so the whole block disappears
|
||||
rather than offering dead options -->
|
||||
<div class="pcr-mcard">
|
||||
@@ -1196,6 +1227,17 @@
|
||||
<SettingsSlider min={0} max={128} step={1} value={feather}
|
||||
savedValue={24} onChange={(v) => { feather = v; }} />
|
||||
</div>
|
||||
<div class="pcr-ip-denoise">
|
||||
<span class="pcr-ip-label">Steps</span>
|
||||
<SettingsSlider min={0} max={60} step={1} value={steps}
|
||||
savedValue={recSteps} onChange={(v) => { steps = v; }} />
|
||||
</div>
|
||||
<div class="pcr-ip-denoise">
|
||||
<span class="pcr-ip-label">CFG</span>
|
||||
<SettingsSlider min={0} max={10} step={0.1} value={cfg}
|
||||
savedValue={recCfg} onChange={(v) => { cfg = v; }} />
|
||||
</div>
|
||||
<div style="opacity:0.55;font-size:11px;padding:2px 0 0 2px">Pre-filled with this engine's recipe (Turbo 8/1.0 · RAW 30/3.5 · SDXL 25/5). On the source model, 0 = inherit the image's own.</div>
|
||||
{/if}
|
||||
{#if !onUseInEdit}
|
||||
<!-- Edit handoffs return the render as a layer — Edit's own Save persists. -->
|
||||
|
||||
@@ -67,16 +67,18 @@
|
||||
let engineKind = $derived(engineEntry
|
||||
? (engineEntry.architecture === "qwen_edit" ? "qwen"
|
||||
: engineEntry.architecture === "zimage" ? "zimage"
|
||||
: engineEntry.architecture === "krea2" ? "krea2"
|
||||
: engineEntry.architecture === "flux" ? "flux1" : "sdxl")
|
||||
: (engineSel === "source" ? "source" : "plain"));
|
||||
let sourceGraftable = $derived(engineKind === "source" && !!caps?.graftable);
|
||||
// Tile engines share the USDU recipe surface (denoise/climb/tile knobs);
|
||||
// qwen is the whole-frame re-render with its own reduced surface.
|
||||
let tileEngine = $derived(engineKind === "sdxl" || engineKind === "flux1" || engineKind === "zimage");
|
||||
let tileEngine = $derived(engineKind === "sdxl" || engineKind === "flux1" || engineKind === "zimage" || engineKind === "krea2");
|
||||
let engineGroups = $derived({
|
||||
sdxl: (caps?.engineModels || []).filter((m) => m.architecture === "sdxl"),
|
||||
flux1: (caps?.engineModels || []).filter((m) => m.architecture === "flux"),
|
||||
zimage: (caps?.engineModels || []).filter((m) => m.architecture === "zimage"),
|
||||
krea2: (caps?.engineModels || []).filter((m) => m.architecture === "krea2"),
|
||||
qwen: (caps?.engineModels || []).filter((m) => m.architecture === "qwen_edit"),
|
||||
});
|
||||
// Degenerate picker (no models, nothing to choose) stays hidden.
|
||||
@@ -95,19 +97,22 @@
|
||||
...(engineGroups.zimage.length
|
||||
? [{ label: "Z-Image — tiled re-detail", options: engineGroups.zimage.map((m) => ({ value: m.hash, label: m.displayName })) }]
|
||||
: []),
|
||||
...(engineGroups.krea2.length
|
||||
? [{ label: "Krea 2 — tiled re-detail", options: engineGroups.krea2.map((m) => ({ value: m.hash, label: m.displayName })) }]
|
||||
: []),
|
||||
...(engineGroups.qwen.length
|
||||
? [{ label: "Qwen Edit — instruction enhance", options: engineGroups.qwen.map((m) => ({ value: m.hash, label: m.displayName })) }]
|
||||
: []),
|
||||
{ label: "Plain upscale", options: [{ value: "plain", label: "Model climb only — no re-detail pass" }] },
|
||||
]);
|
||||
let recSampler = $derived(engineKind === "sdxl" ? "dpmpp_2m" : engineKind === "zimage" ? "res_multistep" : engineKind === "flux1" || engineKind === "qwen" ? "euler" : caps?.recommendedSampler);
|
||||
let recScheduler = $derived(engineKind === "sdxl" ? "karras" : engineKind === "zimage" ? "beta" : engineKind === "flux1" || engineKind === "qwen" ? "simple" : caps?.recommendedScheduler);
|
||||
let recSampler = $derived(engineKind === "sdxl" ? "dpmpp_2m" : engineKind === "zimage" ? "res_multistep" : engineKind === "krea2" ? "euler" : engineKind === "flux1" || engineKind === "qwen" ? "euler" : caps?.recommendedSampler);
|
||||
let recScheduler = $derived(engineKind === "sdxl" ? "karras" : engineKind === "zimage" ? "beta" : engineKind === "flux1" || engineKind === "qwen" || engineKind === "krea2" ? "simple" : caps?.recommendedScheduler);
|
||||
let advDefaults = $derived(engineEntry ? (engineEntry.defaults?.advanced || {}) : (caps?.advancedDefaults || {}));
|
||||
let denoiseMax = $derived(tileEngine
|
||||
? (engineEntry?.defaults?.denoiseMax || 0.4)
|
||||
: (caps?.denoiseMax || 0.7));
|
||||
let denoiseDefault = $derived(tileEngine
|
||||
? (engineEntry?.defaults?.denoise ?? (engineKind === "flux1" ? 0.2 : 0.15))
|
||||
? (engineEntry?.defaults?.denoise ?? (engineKind === "flux1" ? 0.2 : engineKind === "krea2" ? 0.45 : 0.15))
|
||||
: caps?.defaultDenoise);
|
||||
|
||||
// Re-default per open: each image's workflow supports different modes.
|
||||
@@ -142,6 +147,17 @@
|
||||
// resets BEFORE the seed so prefillFor can't echo a stale prompt; the
|
||||
// seed itself is mode-aware (a regional defaultMode opens on $blocks).
|
||||
memoryPrompt = null;
|
||||
// A Krea 2 RAW *source* (image rendered with the undistilled base) upscales
|
||||
// cleanest through the RAW tile engine, so default to it. Scoped to a RAW
|
||||
// source ONLY: a Turbo (or any other) source keeps its own model as the
|
||||
// default engine — preserving "default engine = the image's own model" —
|
||||
// with RAW still one pick away in the dropdown. pickEngine applies RAW's
|
||||
// tile-tuned denoise/sampler below.
|
||||
const rawEngine = caps.sourceModelInfo?.family === "krea2_raw"
|
||||
? (caps.engineModels || []).find((m) => m.architecture === "krea2"
|
||||
&& /raw/i.test(m.filename || "") && !/turbo/i.test(m.filename || ""))
|
||||
: null;
|
||||
const rawHash = rawEngine?.hash || null;
|
||||
engineSel = caps.graftable ? "source" : "plain";
|
||||
prompt = prefillFor(caps.graftable ? "source" : "plain");
|
||||
// Plain pass defaults to SeedVR2 whenever it's installed (installed =
|
||||
@@ -184,6 +200,13 @@
|
||||
if (mem.climbModel && caps.upscaleModelOptions?.includes(mem.climbModel)) climbModel = mem.climbModel;
|
||||
if (mem.plainEngine === "ultrasharp" || (mem.plainEngine === "seedvr2" && caps.seedvr2Available)) plainEngine = mem.plainEngine;
|
||||
}
|
||||
// RAW source only (rawHash is null for every other source): prefer the RAW
|
||||
// tile engine over the source graft, which would carry the from-scratch
|
||||
// render recipe (52 steps) — wrong for a low-denoise tile pass. FINAL word,
|
||||
// so it survives the memory restore above; fires when nothing was explicitly
|
||||
// chosen for this image or the remembered choice was "source". An explicit
|
||||
// pick (incl. RAW on a Turbo image) is respected. pickEngine sets RAW's dials.
|
||||
if (rawHash && (!memEngineValid || memEngine === "source")) pickEngine(rawHash);
|
||||
requestAnimationFrame(() => confirmBtn?.focus());
|
||||
});
|
||||
});
|
||||
@@ -201,7 +224,7 @@
|
||||
// silently drop every region the moment the user touched the text.
|
||||
function basePrefillFor(kind) {
|
||||
if (kind === "qwen") return caps?.enginePromptDefaults?.qwen || "";
|
||||
if (kind === "sdxl" || kind === "flux1" || kind === "zimage") {
|
||||
if (kind === "sdxl" || kind === "flux1" || kind === "zimage" || kind === "krea2") {
|
||||
// An edit-model source's prompt is an INSTRUCTION ("make the person in
|
||||
// image 1 do a pose…"), not a scene description — pushing it into a
|
||||
// tile pass conditions the tiles on the instruction's words. Neutral
|
||||
@@ -226,7 +249,7 @@
|
||||
function pickEngine(value) {
|
||||
engineSel = value;
|
||||
const entry = caps?.engineModels?.find((m) => m.hash === value) || null;
|
||||
const kind = entry ? (entry.architecture === "qwen_edit" ? "qwen" : entry.architecture === "zimage" ? "zimage" : entry.architecture === "flux" ? "flux1" : "sdxl")
|
||||
const kind = entry ? (entry.architecture === "qwen_edit" ? "qwen" : entry.architecture === "zimage" ? "zimage" : entry.architecture === "krea2" ? "krea2" : entry.architecture === "flux" ? "flux1" : "sdxl")
|
||||
: (value === "source" ? "source" : "plain");
|
||||
advanced = {};
|
||||
// Restore/climbModel deliberately survive: they describe the SOURCE and
|
||||
@@ -457,6 +480,7 @@
|
||||
: engineKind === "plain" ? (climbStage === "seedvr2" && caps?.seedvr2Available ? "seedvr2" : "ultrasharp")
|
||||
: engineKind === "qwen" ? "qwen-edit"
|
||||
: engineKind === "zimage" ? "zimage-unet"
|
||||
: engineKind === "krea2" ? "krea2-unet"
|
||||
: engineKind === "flux1" ? "flux1-unet" : "sdxl-ckpt";
|
||||
if (engineEntry) base.engineModel = { hash: engineEntry.hash, filename: engineEntry.filename };
|
||||
if (tileEngine && climbStage !== "ultrasharp") base.climbStage = climbStage;
|
||||
@@ -473,7 +497,13 @@
|
||||
// doesn't — the graft's own source already says exactly that.
|
||||
base.prompt = prompt;
|
||||
}
|
||||
if (Object.keys(advanced).length) base.advanced = { ...advanced };
|
||||
// Tile/engine builds read cfg/steps/tile from options.advanced and fall back
|
||||
// to TURBO-shaped constants if absent — so a picked engine's OWN defaults
|
||||
// (e.g. RAW's cfg 3.5 / 24 steps) must be sent, not just user overrides, or
|
||||
// RAW silently runs on Turbo's 8/1.0. The source graft sets its own node
|
||||
// defaults, so it only needs the user's overrides.
|
||||
if (engineEntry) base.advanced = { ...advDefaults, ...advanced };
|
||||
else if (Object.keys(advanced).length) base.advanced = { ...advanced };
|
||||
storeModalMemory("upscale", imageKey, {
|
||||
engine: engineKind === "source" ? "source" : engineSel,
|
||||
prompt,
|
||||
@@ -715,10 +745,10 @@
|
||||
natural box, so only the added detail differs. -->
|
||||
<div class="pcr-up-split-before" style="clip-path: inset(0 calc(100% - {splitX}px) 0 0);">
|
||||
<div class="pcr-up-zoomwrap" style="transform: translate({panX}px, {panY}px) scale({zoom});">
|
||||
<img class="pcr-up-preview" src={previewUrl} alt="" draggable="false" width={imgNatW} height={imgNatH} />
|
||||
<img class="pcr-up-preview" src={progress?.enlargedUrl || previewUrl} alt="" draggable="false" width={imgNatW} height={imgNatH} />
|
||||
</div>
|
||||
</div>
|
||||
<div class="pcr-up-split-label before">Before</div>
|
||||
<div class="pcr-up-split-label before">{progress?.enlargedUrl ? "Enlarged" : "Before"}</div>
|
||||
<div class="pcr-up-split-label after">After</div>
|
||||
<!-- svelte-ignore a11y_no_static_element_interactions -->
|
||||
<div class="pcr-up-split-divider" style="left: {splitX}px;"
|
||||
@@ -770,9 +800,9 @@
|
||||
<button class="pcr-up-restore-chip" onclick={applySavedSetup}
|
||||
title="Re-apply the dials from your last upscale of this image">↩ Restore last setup</button>
|
||||
{/if}
|
||||
{#if !caps?.graftable && engineKind === "plain"}
|
||||
{#if !caps?.graftable && engineKind === "plain" && engineChoices <= 1}
|
||||
<p class="pcr-up-floor-msg">
|
||||
This image has no usable prompt metadata — a plain model upscale (ESRGAN) will be used.
|
||||
No re-detail engine is available for this image — a plain model upscale (ESRGAN) will be used.
|
||||
</p>
|
||||
{/if}
|
||||
{#if engineChoices > 1}
|
||||
@@ -792,6 +822,8 @@
|
||||
? "the climb model below pushes to the target, then this FLUX.1 model re-details every tile with the prompt below — low denoise holds structure without a ControlNet"
|
||||
: engineKind === "zimage"
|
||||
? "the climb model below pushes to the target, then this Z-Image model re-details every tile with the prompt below — Base holds structure at low denoise; Turbo (distilled) needs ~0.7 or it blotches"
|
||||
: engineKind === "krea2"
|
||||
? "the climb model below pushes to the target, then this Krea 2 model re-details every tile with the prompt below — distilled turbo re-detail at moderate denoise, no ControlNet"
|
||||
: "the climb model below pushes to the target, then this checkpoint re-details every tile (structure locked by a tile ControlNet) with the prompt below"}
|
||||
</span>
|
||||
{:else if engineKind === "qwen"}
|
||||
@@ -861,7 +893,7 @@
|
||||
{/if}
|
||||
<div class="pcr-mcard">
|
||||
<div class="pcr-mcard-title">Settings</div>
|
||||
{#if sourceGraftable || tileEngine || engineKind === "qwen"}
|
||||
{#if sourceGraftable || tileEngine || engineKind === "qwen" || engineKind === "plain"}
|
||||
<div class="pcr-up-sliders">
|
||||
<div class="pcr-up-slider-row" class:pcr-up-slider-row-ticks={!!scaleSlider.ticks}>
|
||||
<span class="pcr-up-slider-label">Scale</span>
|
||||
@@ -907,7 +939,7 @@
|
||||
{/if}
|
||||
{#if sourceGraftable || tileEngine}
|
||||
<!-- qwen has no denoise knob: 1.0 is structural (the
|
||||
conditioning image is the anchor, not the latent) -->
|
||||
conditioning image is the anchor, not the latent). -->
|
||||
<div class="pcr-up-slider-row">
|
||||
<span class="pcr-up-slider-label">Denoise</span>
|
||||
<SettingsSlider
|
||||
|
||||
@@ -128,19 +128,27 @@
|
||||
editors[nodeType][widgetName] = val;
|
||||
}
|
||||
|
||||
// Ideogram's noise schedule is computed by Ideogram4Scheduler, which carries its
|
||||
// OWN width/height that must match the latent or the schedule and the image size
|
||||
// disagree. The resolution row writes only the latent node, so slave the
|
||||
// scheduler's width/height to it here (it is intentionally not shown as a second
|
||||
// resolution row). No-op on any graph without an Ideogram4Scheduler section.
|
||||
function mirrorIdeogramScheduler(latentNode) {
|
||||
const sched = detected.find((d) => d.type === "Ideogram4Scheduler");
|
||||
if (!sched) return;
|
||||
for (const dim of ["width", "height"]) {
|
||||
const v = readWidgetValue(latentNode, dim);
|
||||
if (v !== undefined) {
|
||||
writeWidgetValue(sched.node, dim, v);
|
||||
setEditorValue("Ideogram4Scheduler", dim, v);
|
||||
// Some nodes carry their OWN width/height that must match the latent or their
|
||||
// schedule/shift disagrees with the image size. The resolution row writes only the
|
||||
// latent node, so slave those dependents to it here (they are intentionally not
|
||||
// shown as a second resolution row). Ideogram4Scheduler (noise schedule) persists
|
||||
// its width/height with the saved model config; Krea 2 RAW's ModelSamplingFlux
|
||||
// (resolution-dynamic flow-shift) is mirror-only — written to the live node but
|
||||
// never into editors, so handleSave can never re-bake a static shift (it must stay
|
||||
// dynamic). No-op on any graph without these sections.
|
||||
const RES_DEPENDENTS = [
|
||||
{ type: "Ideogram4Scheduler", persist: true },
|
||||
{ type: "ModelSamplingFlux", persist: false },
|
||||
];
|
||||
function mirrorResolutionDependents(latentNode) {
|
||||
for (const { type: depType, persist } of RES_DEPENDENTS) {
|
||||
const dep = detected.find((d) => d.type === depType);
|
||||
if (!dep) continue;
|
||||
for (const dim of ["width", "height"]) {
|
||||
const v = readWidgetValue(latentNode, dim);
|
||||
if (v === undefined) continue;
|
||||
writeWidgetValue(dep.node, dim, v);
|
||||
if (persist) setEditorValue(depType, dim, v);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -252,7 +260,7 @@
|
||||
<div class="pcr-model-panel-empty">No supported nodes detected.</div>
|
||||
{/if}
|
||||
|
||||
{#each detected as { node, config, type }}
|
||||
{#each detected.filter((d) => !d.config?.mirrorOnly) as { node, config, type }}
|
||||
{@const savedWidgets = savedWidgetsFor(type)}
|
||||
{@const isExpanded = rangeMode || !!expandedState[type]}
|
||||
<div class="pcr-model-panel-section">
|
||||
@@ -338,7 +346,7 @@
|
||||
writeWidgetValue(node, "height", p.height);
|
||||
setEditorValue(type, "width", p.width);
|
||||
setEditorValue(type, "height", p.height);
|
||||
mirrorIdeogramScheduler(node);
|
||||
mirrorResolutionDependents(node);
|
||||
presetMenuOpen[type] = false;
|
||||
}}>
|
||||
<span class="pcr-resolution-preset-item-label">{p.width}×{p.height}</span>
|
||||
@@ -374,7 +382,7 @@
|
||||
const v = parseInt(e.target.value) || 512;
|
||||
setEditorValue(type, "width", v);
|
||||
writeWidgetValue(node, "width", v);
|
||||
mirrorIdeogramScheduler(node);
|
||||
mirrorResolutionDependents(node);
|
||||
}} />
|
||||
<span class="pcr-resolution-sep">×</span>
|
||||
<input type="number" class="pcr-resolution-input"
|
||||
@@ -385,7 +393,7 @@
|
||||
const v = parseInt(e.target.value) || 512;
|
||||
setEditorValue(type, "height", v);
|
||||
writeWidgetValue(node, "height", v);
|
||||
mirrorIdeogramScheduler(node);
|
||||
mirrorResolutionDependents(node);
|
||||
}} />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user