docs: sync with converter consolidation
Docs described the pre-#67 suite: a removed standalone LCM converter node, a removed model_version input, a nonexistent CoreMLDetailerHookProvider node, and a deleted lcm/converter.py file. - drop LCM converter node docs; LCM checkpoints are auto-detected by the consolidated CoreMLConverter - remove model_version input and invented 512-768 resolution range - replace phantom detailer-hook fix with real workarounds - fix Python support claim (3.12+ per requires-python) - update LCM support-matrix row, drop stale line reference Refs #67, #68
This commit is contained in:
@@ -104,7 +104,7 @@ and no longer depends on that package: UNet conversion runs natively on
|
||||
`diffusers`' `UNet2DConditionModel`, the ANE attention path (`SPLIT_EINSUM`,
|
||||
`SPLIT_EINSUM_V2`) is reimplemented as standalone `diffusers` attention
|
||||
processors, and the toolchain tracks current ComfyUI (NumPy 2, Torch 2.7+,
|
||||
coremltools 9, Python 3.11/3.12). Conversion now lives in the separate
|
||||
coremltools 9, Python 3.12+). Conversion now lives in the separate
|
||||
[coreml-diffusion](https://github.com/aszc-dev/coreml-diffusion) package.
|
||||
|
||||
## Support
|
||||
|
||||
+6
-8
@@ -3,8 +3,10 @@
|
||||
## Conversion is the only supported path
|
||||
|
||||
You always start from a Stable Diffusion checkpoint (`.safetensors` / `.ckpt`)
|
||||
and convert it with the **Convert Checkpoint to Core ML** (or **Convert LCM**)
|
||||
node. Pre-converted Core ML models from elsewhere are not supported, because:
|
||||
and convert it with the **Convert Checkpoint to Core ML** node. The model
|
||||
version (SD1.5, SDXL, SDXL refiner, full-distill LCM) is auto-detected from the
|
||||
checkpoint. Pre-converted Core ML models from elsewhere are not supported,
|
||||
because:
|
||||
|
||||
- The suite uses its own input **dimensions**, **naming convention**, and
|
||||
**metadata**, all produced by the
|
||||
@@ -23,7 +25,7 @@ or `coremlcompiler` dependency**.
|
||||
|
||||
Conversion runs **once**, not on every queue. The converter encodes all
|
||||
conversion parameters into the output filename (via `coreml_diffusion.compose_out_name`,
|
||||
called in `coreml_suite/nodes.py:313`):
|
||||
called in `coreml_suite/nodes.py`):
|
||||
|
||||
- checkpoint name, `batch_size`, `width`, `height`
|
||||
- `controlnet_support`, `attention_implementation`
|
||||
@@ -46,7 +48,7 @@ it short-circuits to the cached file.)
|
||||
|
||||
## Quantization
|
||||
|
||||
Both converter nodes accept an optional `quantize_nbits` dropdown that runs
|
||||
The converter node accepts an optional `quantize_nbits` dropdown that runs
|
||||
k-means weight palettization (`coremltools.optimize.coreml.palettize_weights`) on
|
||||
the UNet before saving.
|
||||
|
||||
@@ -89,7 +91,3 @@ The conversion engine was extracted into the standalone
|
||||
The nodes in this suite resolve ComfyUI paths and call into it; node names,
|
||||
inputs, and outputs are unchanged, so the split has effectively no user-facing
|
||||
impact beyond `pip install` pulling one more dependency.
|
||||
|
||||
One detail: the LCM converter still imports `diffusers` directly (in
|
||||
`coreml_suite/lcm/converter.py`) to download the hardcoded LCM model from Hugging
|
||||
Face. This is an internal note, not something you need to act on.
|
||||
|
||||
+2
-2
@@ -67,8 +67,8 @@ avoids. Use the native sampler unless you specifically need a `MODEL` (e.g.
|
||||
|
||||
## Which Python versions work?
|
||||
|
||||
Both 3.11 and 3.12. Older 3.12 install failures came from the now-removed
|
||||
`ml-stable-diffusion` build, not from this suite.
|
||||
Python 3.12 or newer (`requires-python >=3.12`). Older 3.12 install failures
|
||||
came from the now-removed `ml-stable-diffusion` build, not from this suite.
|
||||
|
||||
## Long prompts crash my workflow
|
||||
|
||||
|
||||
+2
-2
@@ -10,7 +10,7 @@
|
||||
| Inpainting checkpoints (9-channel) | ❌ Unsupported | |
|
||||
| ControlNet | ✅ Supported | Convert the checkpoint with `controlnet_support = True` |
|
||||
| LoRA | ⚠️ Experimental | Inconsistent per-LoRA; baked at conversion, immutable afterward |
|
||||
| LCM | ⚠️ Experimental | Hardcoded to LCM Dreamshaper v7 |
|
||||
| LCM | ⚠️ Experimental | Full-distill LCM checkpoints auto-detected by the converter |
|
||||
| SVD | ❌ Not supported | |
|
||||
| AnimateDiff | ❌ Not supported | Motion modules need pre-conversion injection; not feasible today |
|
||||
| IPAdapter | ❌ Not supported | Needs a real `MODEL` the Core ML wrapper can't provide |
|
||||
@@ -42,7 +42,7 @@ degraded versus the safetensors original (an upstream conversion artifact). Use
|
||||
The Adapter wraps a Core ML model to look like a standard ComfyUI `MODEL`, which
|
||||
covers many standard and custom nodes. But it can't fully emulate a real model:
|
||||
operations that need genuine `MODEL` internals — model merges, IPAdapter, some
|
||||
LoRA flows, detailers without the size hook — generally won't work, and the model's
|
||||
LoRA flows, detailers — generally won't work, and the model's
|
||||
fixed input shapes aren't validated, so mismatches error at runtime. Prefer the
|
||||
native Core ML Sampler when you don't need the `MODEL` type.
|
||||
|
||||
|
||||
+10
-26
@@ -11,7 +11,6 @@ All nodes live in the **Core ML Suite** category. Right-click the canvas →
|
||||
| Core ML Adapter (Experimental) | `CoreMLModelAdapter` | Wrap as a standard `MODEL` |
|
||||
| Load LoRA to use with Core ML | `Core ML LoRA Loader` | Bake LoRA(s) at conversion |
|
||||
| Convert Checkpoint to Core ML | `Core ML Converter` | Convert a checkpoint |
|
||||
| Convert LCM to Core ML | `Core ML LCM Converter` | Convert LCM Dreamshaper v7 |
|
||||
|
||||
---
|
||||
|
||||
@@ -122,14 +121,15 @@ only affects CLIP and can be changed freely. After conversion, when loading with
|
||||
|
||||

|
||||
|
||||
Converts a SD1.5- or SDXL-based checkpoint from `models/checkpoints` to a Core ML
|
||||
`.mlpackage` in `models/unet`. The conversion parameters are encoded in the
|
||||
output name, so an already-converted model is reused instead of re-converted.
|
||||
See [conversion](conversion.md) for details.
|
||||
Converts a checkpoint from `models/checkpoints` to a Core ML `.mlpackage` in
|
||||
`models/unet`. The model version (SD1.5, SDXL, SDXL refiner, or full-distill
|
||||
LCM) is auto-detected from the checkpoint's architecture — there is no version
|
||||
dropdown. The conversion parameters are encoded in the output name, so an
|
||||
already-converted model is reused instead of re-converted. See
|
||||
[conversion](conversion.md) for details.
|
||||
|
||||
- **Inputs**
|
||||
- `ckpt_name` — checkpoint in `models/checkpoints`.
|
||||
- `model_version` — `SD15` or `SDXL` (list is discovered from `coreml-diffusion`).
|
||||
- `height`, `width` — target image size; any positive multiple of 8 (default
|
||||
512). The model's input size is fixed at these values.
|
||||
- `batch_size` — default 1; raise to convert a batch-capable model.
|
||||
@@ -150,24 +150,8 @@ See [conversion](conversion.md) for details.
|
||||
> named like the checkpoint (e.g. `juggernaut.safetensors` →
|
||||
> `juggernaut.yaml`); it is loaded automatically during conversion.
|
||||
|
||||
---
|
||||
|
||||
## Convert LCM to Core ML (`Core ML LCM Converter`)
|
||||
|
||||

|
||||
|
||||
Converts [SimianLuo/LCM_Dreamshaper_v7](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7)
|
||||
to Core ML in `models/unet`. As with the checkpoint converter, the parameters are
|
||||
encoded in the name and an existing model is reused.
|
||||
|
||||
- **Inputs**
|
||||
- `height`, `width` — 512–768, multiple of 8 (default 512).
|
||||
- `batch_size` — default 1.
|
||||
- `compute_unit` — used only when loading.
|
||||
- `controlnet_support` — default `False`.
|
||||
- **Output**
|
||||
- `coreml_model`.
|
||||
|
||||
> [!NOTE]
|
||||
> When sampling an LCM model, set `sampler_name` to `lcm` and `scheduler` to
|
||||
> `sgm_uniform`. Conversion can take a while.
|
||||
> Full-distill LCM checkpoints (e.g.
|
||||
> [LCM_Dreamshaper_v7](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7)) are
|
||||
> detected and converted like any other checkpoint. When sampling an LCM model,
|
||||
> set `sampler_name` to `lcm` and `scheduler` to `sgm_uniform`.
|
||||
|
||||
@@ -60,11 +60,11 @@ and can't be changed afterward.
|
||||
## FaceDetailer / detailers error on size
|
||||
|
||||
Detailers rescale latents internally (e.g. 512 → 1024), which breaks the model's
|
||||
fixed input shape.
|
||||
fixed input shape. There is no workaround node — a Core ML model only accepts
|
||||
the resolution it was converted for.
|
||||
|
||||
**Fix:** use the `CoreMLDetailerHookProvider` node to pin the detailer's internal
|
||||
size to the model's converted resolution. Note it only offers preset sizes, so
|
||||
non-standard resolutions may not be selectable.
|
||||
**Fix:** convert a second model at the detailer's internal resolution and use it
|
||||
for the detailing pass, or run the detailer with a standard (non–Core ML) model.
|
||||
|
||||
## Inpainting checkpoint errors (`tensor size 9 vs 4`)
|
||||
|
||||
|
||||
+5
-2
@@ -77,8 +77,11 @@ passed through the same LoRA nodes used at conversion. Since `lora_name` and
|
||||
|
||||
## LCM conversion with ControlNet
|
||||
|
||||
Convert [LCM_Dreamshaper_v7](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7)
|
||||
with the LCM converter, then use it with or without ControlNet.
|
||||
Convert a full-distill LCM checkpoint (e.g.
|
||||
[LCM_Dreamshaper_v7](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7)) with
|
||||
the standard **Convert Checkpoint to Core ML** node — the LCM architecture is
|
||||
auto-detected. Use it with or without ControlNet. When sampling, set
|
||||
`sampler_name` to `lcm` and `scheduler` to `sgm_uniform`.
|
||||
|
||||

|
||||
|
||||
|
||||
Reference in New Issue
Block a user