aszc-dev-ComfyUI-CoreMLSuite/docs/workflows.md

Example Workflows

Note

The models referenced are examples — substitute your own. Every workflow starts from a checkpoint you convert yourself (see conversion); there is no Core ML model to download.

Basic txt2img

Convert a SD1.5 checkpoint, then sample from it. CLIP and VAE come from standard ComfyUI nodes — either loaded separately or pulled from the checkpoint.

1. Place a SD1.5 checkpoint in models/checkpoints (e.g. v1-5-pruned-emaonly).
2. Convert Checkpoint to Core ML → queue once → a .mlpackage lands in models/unet.
3. Load Core ML UNet (or wire the converter output straight in) → Core ML Sampler → VAE Decode.

CLIP and VAE from the checkpoint:

CLIP and VAE loaded separately — use any SD1.5-compatible CLIP and VAE, placed in models/clip and models/vae:

ControlNet

Convert the checkpoint with controlnet_support = True, then wire a standard ComfyUI ControlNet. The ControlNet model itself needs no conversion. Place it in models/controlnet (e.g. control_v11p_sd15_scribble).

Checkpoint conversion

The minimal conversion graph. See Convert Checkpoint to Core ML.

Conversion with LoRA

Bake LoRA(s) into the model at conversion. Read the LoRA caveats first — baked weights are immutable, and support is inconsistent per-LoRA.

LCM LoRA conversion

Chain multiple LoRA loaders to use several LoRAs with one model.

Important

Here the model goes through the Core ML Adapter and ModelSamplingDiscrete into the standard ComfyUI KSampler (not the Core ML Sampler). ModelSamplingDiscrete is required to sample LCM LoRAs correctly.

Loading a model with baked LoRAs

Load a model that already has LoRAs baked in. CLIP must be loaded separately and passed through the same LoRA nodes used at conversion. Since lora_name and strength_model are baked in, they need not be passed to the loader.

Important

As above, the model goes through the Core ML Adapter + ModelSamplingDiscrete into the standard KSampler.

LCM conversion with ControlNet

Convert a full-distill LCM checkpoint (e.g. LCM_Dreamshaper_v7) with the standard Convert Checkpoint to Core ML node — the LCM architecture is auto-detected. Use it with or without ControlNet. When sampling, set sampler_name to lcm and scheduler to sgm_uniform.

SDXL Base + Refiner

A basic SDXL graph. Add LoRAs and ControlNets as in the SD1.5 examples; the refiner step is optional.

Models: base + text encoders, refiner, VAE.

Important

SDXL does not run on the ANE. Convert with ORIGINAL and load with CPU_AND_GPU (or CPU_ONLY). If loading hangs on CPU_AND_NE, that is the cause. See limitations.