aszc
d90546b6bb
* docs(extraction): E0 seam inventory + correct stale spec assumptions Resolve all pre-flight greps for the converter-extraction seam: - conversion/* and lcm/unet.py confirmed comfy-free - converter.py: only folder_paths reach-in is get_out_path - lcm/converter.py: folder_paths + comfy.model_management to cut; dup helpers and SimianLuo HF hardcode confirmed - no attention module-global; already per-call Correct two stale assumptions verified against current source: - ml-stable-diffusion is already fully removed (#58); CoreMLModel is a local coremltools wrapper, not Apple's. Drop the dep-pinning blocker and the package/suite dep lines that assumed it. - model_version discovery must emit .name (node reverses via ModelVersion[...]); the .value form in the draft would KeyError on every saved workflow. * feat(extraction): E1 coreml_diffusion package + discovery API Stand up the framework-free coreml_diffusion namespace and freeze its versioned discovery contract. The package re-exports from already comfy-free coreml_suite sources (model_version, attention, core.naming); the conversion implementation moves in E2. - list_model_versions/list_attention_impls/list_quant_modes return today's exact dropdown strings, so wiring the node onto them (E3) changes no value and breaks no saved workflow. - Status/_MODEL_STATUS registry gates VERIFIED vs EXPERIMENTAL in the package, so promoting a model expands the node dropdown with no Suite change (additive-only contract; CONTRACT_VERSION=1.0). - Tier-0 test pins the contract and proves comfy/diffusers/coremltools are not pulled on import. Node untouched; zero behavior change. * refactor(extraction): E2 move conversion mechanics into coreml_diffusion Physically relocate the framework-free conversion code into the package and collapse the duplicated LCM/main helpers, behavior-preserving. - coreml_suite/conversion/ -> coreml_diffusion/conversion/ (attention, shapes, trace, unet) - coreml_suite/core/naming.py -> coreml_diffusion/naming.py (the cache-key contract now lives with the package; tests re-pointed) - coreml_suite/converter.py logic -> coreml_diffusion/convert.py, with convert() made keyword-only past (ckpt_path, model_version, out_path) per the interface contract; out_path is injected (no folder_paths) - dedup: load_coreml_model / convert_to_coreml / get_coreml_inputs / add_cnet_support / get_encoder_hidden_states_shape / inputs-spec now defined once in the package; get_sample_input gains an optional scheduler arg so the LCM path shares it (same keys/order/dtypes) - coreml_suite.{converter,lcm.converter} reduced to comfy-side shims: folder_paths path resolution and the LCM scheduler's comfy.model_management stay here; the package imports neither - __init__ keeps discovery + compose_out_name eager; convert is lazy via __getattr__ so 'import coreml_diffusion' stays Tier-0 pure Nodes untouched (E3 thins them onto the package). Tier-0 (109) and smoke (3, real coremltools conversion) green; [M2-ANE] golden pending a server. * refactor(extraction): E3 thin nodes onto coreml_diffusion + discovery dropdowns The CoreMLConverter node now calls coreml_diffusion directly instead of the coreml_suite.converter shim, and its dropdowns are populated at runtime from the package's discovery API. - INPUT_TYPES dropdowns (model_version / attention_implementation / quantize_nbits) now come from a fail-soft _discover() that calls coreml_diffusion.list_*; a missing/old package falls back to a literal list and logs a warning instead of de-registering the node. Installing a newer coreml_diffusion surfaces new conversion types with no Suite change. - folder_paths path resolution moved inline into the node; the package's convert() takes the output path as an injected positional. - compose_out_name / lora_names_from_params now imported from coreml_diffusion (lazily, inside convert) — no node-side copy. - deleted the dead coreml_suite/converter.py and coreml_suite/core/naming.py shims (no remaining importers). Field names, RETURN_TYPES/NAMES and NODE_*_MAPPINGS unchanged; dropdown values are a superset of the prior literals (additive-only). Tier-0 (109) and smoke (3) green; [M2-ANE] golden re-runs on push. * refactor(extraction): E5 depend on external coreml-diffusion package Conversion code now lives in the standalone coreml-diffusion repo. The Suite deletes its in-tree copy and depends on the package instead. - removed coreml_diffusion/ (whole package), coreml_suite/model_version.py and coreml_suite/attention.py (moved to the package as its source of truth), and the tests that moved with them (discovery, conversion_helpers, out_name; smoke synthetic_unet + split_einsum) - re-pointed ModelVersion imports (config.py, nodes.py, lcm/converter.py) to coreml_diffusion - pyproject: drop the coreml_diffusion package include and the conversion-only deps (peft/omegaconf/transformers, now transitive via coreml-diffusion); add coreml-diffusion as a dependency with a local path source until it is published (switch to git tag/PyPI once the repo exists, so CI can resolve it) Suite Tier-0 green (75); conversion code fully absent from the Suite. The comfy node still imports coreml_diffusion (installed package) for ModelVersion + the discovery dropdowns + convert. * build(extraction): pin coreml-diffusion to git tag v0.1.0 Switch the coreml-diffusion source from a local path to the published git tag so CI can resolve it. Suite Tier-0 green resolving from the tag. * ci(extraction): drop Suite smoke tier (moved to coreml-diffusion) The conversion smoke tests moved to the coreml-diffusion repo, which runs its own Tier 1. The Suite's smoke lane had no tests left (pytest exit 5). The Suite keeps Tier 0 (inference units) and the m2 golden e2e. * chore(release): v2.1.0; wire coreml-diffusion into requirements.txt Minor bump: the conversion path moved to the external coreml-diffusion package (node graph + artifact cache keys unchanged, golden-verified). requirements.txt (used by ComfyUI Manager) now installs coreml-diffusion from the v0.1.0 tag and drops the conversion-only deps now provided transitively.
116 lines
3.5 KiB
Python
116 lines
3.5 KiB
Python
import torch
|
|
|
|
from comfy import supported_models_base
|
|
from comfy import latent_formats
|
|
from comfy.model_detection import convert_config
|
|
|
|
from coreml_diffusion import ModelVersion
|
|
|
|
|
|
config_map = {
|
|
ModelVersion.SD15: {
|
|
"use_checkpoint": False,
|
|
"image_size": 32,
|
|
"out_channels": 4,
|
|
"use_spatial_transformer": True,
|
|
"legacy": False,
|
|
"adm_in_channels": None,
|
|
"dtype": torch.float16,
|
|
"in_channels": 4,
|
|
"model_channels": 320,
|
|
"num_res_blocks": 2,
|
|
"attention_resolutions": [1, 2, 4],
|
|
"transformer_depth": [1, 1, 1, 0],
|
|
"channel_mult": [1, 2, 4, 4],
|
|
"transformer_depth_middle": 1,
|
|
"use_linear_in_transformer": False,
|
|
"context_dim": 768,
|
|
"num_heads": 8,
|
|
"disable_unet_model_creation": True,
|
|
},
|
|
ModelVersion.SDXL: {
|
|
"use_checkpoint": False,
|
|
"image_size": 32,
|
|
"out_channels": 4,
|
|
"use_spatial_transformer": True,
|
|
"legacy": False,
|
|
"num_classes": "sequential",
|
|
"adm_in_channels": 2816,
|
|
"dtype": torch.float16,
|
|
"in_channels": 4,
|
|
"model_channels": 320,
|
|
"num_res_blocks": 2,
|
|
"attention_resolutions": [2, 4],
|
|
"transformer_depth": [0, 2, 10],
|
|
"channel_mult": [1, 2, 4],
|
|
"transformer_depth_middle": 10,
|
|
"use_linear_in_transformer": True,
|
|
"context_dim": 2048,
|
|
"num_head_channels": 64,
|
|
"disable_unet_model_creation": True,
|
|
},
|
|
ModelVersion.SDXL_REFINER: {
|
|
"use_checkpoint": False,
|
|
"image_size": 32,
|
|
"out_channels": 4,
|
|
"use_spatial_transformer": True,
|
|
"legacy": False,
|
|
"num_classes": "sequential",
|
|
"adm_in_channels": 2560,
|
|
"dtype": torch.float16,
|
|
"in_channels": 4,
|
|
"model_channels": 384,
|
|
"num_res_blocks": 2,
|
|
"attention_resolutions": [2, 4],
|
|
"transformer_depth": [0, 4, 4, 0],
|
|
"channel_mult": [1, 2, 4, 4],
|
|
"transformer_depth_middle": 4,
|
|
"use_linear_in_transformer": True,
|
|
"context_dim": 1280,
|
|
"num_head_channels": 64,
|
|
"disable_unet_model_creation": True,
|
|
},
|
|
}
|
|
|
|
latent_format_map = {
|
|
ModelVersion.SD15: latent_formats.SD15,
|
|
ModelVersion.SDXL: latent_formats.SDXL,
|
|
ModelVersion.SDXL_REFINER: latent_formats.SDXL,
|
|
}
|
|
|
|
|
|
def get_model_config(model_version: ModelVersion):
|
|
unet_config = convert_config(config_map[model_version])
|
|
config = supported_models_base.BASE(unet_config)
|
|
config.latent_format = latent_format_map[model_version]()
|
|
return config
|
|
|
|
|
|
def unet_config_from_diffusers_unet(state_dict):
|
|
match = {}
|
|
attention_resolutions = []
|
|
|
|
attn_res = 1
|
|
for i in range(5):
|
|
k = "down_blocks.{}.attentions.1.transformer_blocks.0.attn2.to_k.weight".format(
|
|
i
|
|
)
|
|
if k in state_dict:
|
|
match["context_dim"] = state_dict[k].shape[1]
|
|
attention_resolutions.append(attn_res)
|
|
attn_res *= 2
|
|
|
|
match["attention_resolutions"] = attention_resolutions
|
|
|
|
match["model_channels"] = state_dict["conv_in.weight"].shape[0]
|
|
match["in_channels"] = state_dict["conv_in.weight"].shape[1]
|
|
match["adm_in_channels"] = None
|
|
if "class_embedding.linear_1.weight" in state_dict:
|
|
match["adm_in_channels"] = state_dict["class_embedding.linear_1.weight"].shape[
|
|
1
|
|
]
|
|
elif "add_embedding.linear_1.weight" in state_dict:
|
|
match["adm_in_channels"] = state_dict["add_embedding.linear_1.weight"].shape[1]
|
|
|
|
print(match)
|