diff --git a/README.md b/README.md index 8290b9c..94fc883 100644 --- a/README.md +++ b/README.md @@ -104,7 +104,7 @@ and no longer depends on that package: UNet conversion runs natively on `diffusers`' `UNet2DConditionModel`, the ANE attention path (`SPLIT_EINSUM`, `SPLIT_EINSUM_V2`) is reimplemented as standalone `diffusers` attention processors, and the toolchain tracks current ComfyUI (NumPy 2, Torch 2.7+, -coremltools 9, Python 3.11/3.12). Conversion now lives in the separate +coremltools 9, Python 3.12+). Conversion now lives in the separate [coreml-diffusion](https://github.com/aszc-dev/coreml-diffusion) package. ## Support diff --git a/docs/conversion.md b/docs/conversion.md index d7ce600..1c7deb5 100644 --- a/docs/conversion.md +++ b/docs/conversion.md @@ -3,8 +3,10 @@ ## Conversion is the only supported path You always start from a Stable Diffusion checkpoint (`.safetensors` / `.ckpt`) -and convert it with the **Convert Checkpoint to Core ML** (or **Convert LCM**) -node. Pre-converted Core ML models from elsewhere are not supported, because: +and convert it with the **Convert Checkpoint to Core ML** node. The model +version (SD1.5, SDXL, SDXL refiner, full-distill LCM) is auto-detected from the +checkpoint. Pre-converted Core ML models from elsewhere are not supported, +because: - The suite uses its own input **dimensions**, **naming convention**, and **metadata**, all produced by the @@ -23,7 +25,7 @@ or `coremlcompiler` dependency**. Conversion runs **once**, not on every queue. The converter encodes all conversion parameters into the output filename (via `coreml_diffusion.compose_out_name`, -called in `coreml_suite/nodes.py:313`): +called in `coreml_suite/nodes.py`): - checkpoint name, `batch_size`, `width`, `height` - `controlnet_support`, `attention_implementation` @@ -46,7 +48,7 @@ it short-circuits to the cached file.) ## Quantization -Both converter nodes accept an optional `quantize_nbits` dropdown that runs +The converter node accepts an optional `quantize_nbits` dropdown that runs k-means weight palettization (`coremltools.optimize.coreml.palettize_weights`) on the UNet before saving. @@ -89,7 +91,3 @@ The conversion engine was extracted into the standalone The nodes in this suite resolve ComfyUI paths and call into it; node names, inputs, and outputs are unchanged, so the split has effectively no user-facing impact beyond `pip install` pulling one more dependency. - -One detail: the LCM converter still imports `diffusers` directly (in -`coreml_suite/lcm/converter.py`) to download the hardcoded LCM model from Hugging -Face. This is an internal note, not something you need to act on. diff --git a/docs/faq.md b/docs/faq.md index 74c1505..d5bab34 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -67,8 +67,8 @@ avoids. Use the native sampler unless you specifically need a `MODEL` (e.g. ## Which Python versions work? -Both 3.11 and 3.12. Older 3.12 install failures came from the now-removed -`ml-stable-diffusion` build, not from this suite. +Python 3.12 or newer (`requires-python >=3.12`). Older 3.12 install failures +came from the now-removed `ml-stable-diffusion` build, not from this suite. ## Long prompts crash my workflow diff --git a/docs/limitations.md b/docs/limitations.md index 11fc80e..f455462 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -10,7 +10,7 @@ | Inpainting checkpoints (9-channel) | ❌ Unsupported | | | ControlNet | ✅ Supported | Convert the checkpoint with `controlnet_support = True` | | LoRA | ⚠️ Experimental | Inconsistent per-LoRA; baked at conversion, immutable afterward | -| LCM | ⚠️ Experimental | Hardcoded to LCM Dreamshaper v7 | +| LCM | ⚠️ Experimental | Full-distill LCM checkpoints auto-detected by the converter | | SVD | ❌ Not supported | | | AnimateDiff | ❌ Not supported | Motion modules need pre-conversion injection; not feasible today | | IPAdapter | ❌ Not supported | Needs a real `MODEL` the Core ML wrapper can't provide | @@ -42,7 +42,7 @@ degraded versus the safetensors original (an upstream conversion artifact). Use The Adapter wraps a Core ML model to look like a standard ComfyUI `MODEL`, which covers many standard and custom nodes. But it can't fully emulate a real model: operations that need genuine `MODEL` internals — model merges, IPAdapter, some -LoRA flows, detailers without the size hook — generally won't work, and the model's +LoRA flows, detailers — generally won't work, and the model's fixed input shapes aren't validated, so mismatches error at runtime. Prefer the native Core ML Sampler when you don't need the `MODEL` type. diff --git a/docs/nodes.md b/docs/nodes.md index f046cbf..fd02f91 100644 --- a/docs/nodes.md +++ b/docs/nodes.md @@ -11,7 +11,6 @@ All nodes live in the **Core ML Suite** category. Right-click the canvas → | Core ML Adapter (Experimental) | `CoreMLModelAdapter` | Wrap as a standard `MODEL` | | Load LoRA to use with Core ML | `Core ML LoRA Loader` | Bake LoRA(s) at conversion | | Convert Checkpoint to Core ML | `Core ML Converter` | Convert a checkpoint | -| Convert LCM to Core ML | `Core ML LCM Converter` | Convert LCM Dreamshaper v7 | --- @@ -122,14 +121,15 @@ only affects CLIP and can be changed freely. After conversion, when loading with ![Checkpoint Converter](../assets/checkpoint_converter.png?raw=true) -Converts a SD1.5- or SDXL-based checkpoint from `models/checkpoints` to a Core ML -`.mlpackage` in `models/unet`. The conversion parameters are encoded in the -output name, so an already-converted model is reused instead of re-converted. -See [conversion](conversion.md) for details. +Converts a checkpoint from `models/checkpoints` to a Core ML `.mlpackage` in +`models/unet`. The model version (SD1.5, SDXL, SDXL refiner, or full-distill +LCM) is auto-detected from the checkpoint's architecture — there is no version +dropdown. The conversion parameters are encoded in the output name, so an +already-converted model is reused instead of re-converted. See +[conversion](conversion.md) for details. - **Inputs** - `ckpt_name` — checkpoint in `models/checkpoints`. - - `model_version` — `SD15` or `SDXL` (list is discovered from `coreml-diffusion`). - `height`, `width` — target image size; any positive multiple of 8 (default 512). The model's input size is fixed at these values. - `batch_size` — default 1; raise to convert a batch-capable model. @@ -150,24 +150,8 @@ See [conversion](conversion.md) for details. > named like the checkpoint (e.g. `juggernaut.safetensors` → > `juggernaut.yaml`); it is loaded automatically during conversion. ---- - -## Convert LCM to Core ML (`Core ML LCM Converter`) - -![LCM Converter](../assets/lcm_converter.png?raw=true) - -Converts [SimianLuo/LCM_Dreamshaper_v7](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7) -to Core ML in `models/unet`. As with the checkpoint converter, the parameters are -encoded in the name and an existing model is reused. - -- **Inputs** - - `height`, `width` — 512–768, multiple of 8 (default 512). - - `batch_size` — default 1. - - `compute_unit` — used only when loading. - - `controlnet_support` — default `False`. -- **Output** - - `coreml_model`. - > [!NOTE] -> When sampling an LCM model, set `sampler_name` to `lcm` and `scheduler` to -> `sgm_uniform`. Conversion can take a while. +> Full-distill LCM checkpoints (e.g. +> [LCM_Dreamshaper_v7](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7)) are +> detected and converted like any other checkpoint. When sampling an LCM model, +> set `sampler_name` to `lcm` and `scheduler` to `sgm_uniform`. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index bae081d..b12bf20 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -60,11 +60,11 @@ and can't be changed afterward. ## FaceDetailer / detailers error on size Detailers rescale latents internally (e.g. 512 → 1024), which breaks the model's -fixed input shape. +fixed input shape. There is no workaround node — a Core ML model only accepts +the resolution it was converted for. -**Fix:** use the `CoreMLDetailerHookProvider` node to pin the detailer's internal -size to the model's converted resolution. Note it only offers preset sizes, so -non-standard resolutions may not be selectable. +**Fix:** convert a second model at the detailer's internal resolution and use it +for the detailing pass, or run the detailer with a standard (non–Core ML) model. ## Inpainting checkpoint errors (`tensor size 9 vs 4`) diff --git a/docs/workflows.md b/docs/workflows.md index 5c81032..51f8a64 100644 --- a/docs/workflows.md +++ b/docs/workflows.md @@ -77,8 +77,11 @@ passed through the same LoRA nodes used at conversion. Since `lora_name` and ## LCM conversion with ControlNet -Convert [LCM_Dreamshaper_v7](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7) -with the LCM converter, then use it with or without ControlNet. +Convert a full-distill LCM checkpoint (e.g. +[LCM_Dreamshaper_v7](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7)) with +the standard **Convert Checkpoint to Core ML** node — the LCM architecture is +auto-detected. Use it with or without ControlNet. When sampling, set +`sampler_name` to `lcm` and `scheduler` to `sgm_uniform`. ![LCM + ControlNet](../assets/lcm+controlnet.png?raw=true)