Commit Graph
117 Commits
Author SHA1 Message Date
aszc 8d28964831 feat(convert): consolidate into one auto-detecting converter node (#67)
* feat(lcm): convert any full-distill LCM checkpoint

- COREML_CONVERT_LCM gains a ckpt_name input: any checkpoint from the
  checkpoints folder, with the canonical SimianLuo single file as the
  default auto-download entry, so workflows saved before this input
  existed keep the old behavior
- conversion routes through the unified
  coreml_diffusion.convert(model_version=LCM) path; the bespoke
  trace/convert pipeline in lcm/converter.py and the dead
  UNet2DConditionModelLCM wrapper are removed
- output naming via compose_out_name; existing cached LCM .mlpackages
  reconvert once due to the new _se attention suffix in the name
- LCM-LoRA merged checkpoints (plain SD1.5 architecture, no guidance
  embedding) are rejected by the package with a pointer to the standard
  converter node + LCM scheduler
- requires coreml-diffusion>=0.1.4 (generic LCM conversion fix)

Verified against a local ComfyUI checkout: the default entry resolves
the Hugging Face single file and cache-hits a previously converted
.mlpackage exposing timestep_cond; an LCM-LoRA merge raises the
explanatory ValueError.

* feat(convert): consolidate conversion into one auto-detecting node

The standard CoreMLConverter now auto-detects the model version from the
checkpoint (coreml-diffusion>=0.1.5, convert(model_version=None)), so:

- the model_version dropdown is gone — one node converts SD15 / SDXL / SDXL
  refiner / full-distill LCM, the version inferred from the UNet architecture
- the dedicated "Core ML LCM Converter" node, its single-model autodownload,
  and coreml_suite/lcm/nodes.py are removed; the converter UX was previously
  inconsistent (LCM only reachable through a separate autodownload-only node,
  while the standard converter did not list LCM at all)

lcm/utils.py (sampler-side timestep_cond patching) is unchanged — runtime LCM
support still keys off the converted UNet exposing timestep_cond.

diffusers is dropped from the dependencies (no longer imported directly after
the LCM converter removal). The e2e workflow fixture drops its now-invalid
model_version input.

* fix(deps): require coreml-diffusion>=0.1.5, keep requires-python <3.13

The auto-detect consolidation needs convert(model_version=None) from
coreml-diffusion 0.1.5. requires-python stays pinned to <3.13 to match the
library (coremltools-driven); relaxing it past the library's own cap makes the
dependency unresolvable for the 3.13+ range.
2026-06-13 14:11:20 +02:00
aszc a199e749bc fix(deps): unpin torch; require coreml-diffusion>=0.1.1 (#65)
The transitive torch<2.8 cap (lifted in coreml-diffusion 0.1.1) downgraded
the host's torch on install and broke ComfyUI startup
(operator torchvision::nms does not exist). Drop the redundant direct torch
pin; torch is provided by the host and pulled transitively as >=2.7.
2026-05-27 05:56:45 +02:00
aszc a3d555cdc5 fix(registry): depend on PyPI coreml-diffusion, set icon, fix license (#64)
* fix(registry): depend on PyPI coreml-diffusion, set icon, fix license

The git+ dependency on coreml-diffusion tripped the registry security
scan (versions flagged unsafe -> Latest pinned to 1.0.1) and broke the
node-extraction sandbox (import failed -> 'No nodes found').

- depend on coreml-diffusion from PyPI (>=0.1.0,<0.2); drop git source
- set the registry Icon to the bundled 512x512 snake.png
- relicense LICENSE file to MIT to match pyproject (was stray GPLv3)
- bump version to 2.1.1 to trigger a fresh registry publish

Note: uv.lock still references the old git source; refresh with
'uv lock' once coreml-diffusion 0.1.0 is live on PyPI.

* chore: refresh lockfile against PyPI coreml-diffusion
2026-05-27 02:54:12 +02:00
aszc d90546b6bb feat(extraction): split conversion into the coreml-diffusion package (E0–E5) (#63)
* docs(extraction): E0 seam inventory + correct stale spec assumptions

Resolve all pre-flight greps for the converter-extraction seam:
- conversion/* and lcm/unet.py confirmed comfy-free
- converter.py: only folder_paths reach-in is get_out_path
- lcm/converter.py: folder_paths + comfy.model_management to cut; dup
  helpers and SimianLuo HF hardcode confirmed
- no attention module-global; already per-call

Correct two stale assumptions verified against current source:
- ml-stable-diffusion is already fully removed (#58); CoreMLModel is a
  local coremltools wrapper, not Apple's. Drop the dep-pinning blocker
  and the package/suite dep lines that assumed it.
- model_version discovery must emit .name (node reverses via
  ModelVersion[...]); the .value form in the draft would KeyError on
  every saved workflow.

* feat(extraction): E1 coreml_diffusion package + discovery API

Stand up the framework-free coreml_diffusion namespace and freeze its
versioned discovery contract. The package re-exports from already
comfy-free coreml_suite sources (model_version, attention, core.naming);
the conversion implementation moves in E2.

- list_model_versions/list_attention_impls/list_quant_modes return
  today's exact dropdown strings, so wiring the node onto them (E3)
  changes no value and breaks no saved workflow.
- Status/_MODEL_STATUS registry gates VERIFIED vs EXPERIMENTAL in the
  package, so promoting a model expands the node dropdown with no Suite
  change (additive-only contract; CONTRACT_VERSION=1.0).
- Tier-0 test pins the contract and proves comfy/diffusers/coremltools
  are not pulled on import.

Node untouched; zero behavior change.

* refactor(extraction): E2 move conversion mechanics into coreml_diffusion

Physically relocate the framework-free conversion code into the package
and collapse the duplicated LCM/main helpers, behavior-preserving.

- coreml_suite/conversion/ -> coreml_diffusion/conversion/ (attention,
  shapes, trace, unet)
- coreml_suite/core/naming.py -> coreml_diffusion/naming.py (the cache-key
  contract now lives with the package; tests re-pointed)
- coreml_suite/converter.py logic -> coreml_diffusion/convert.py, with
  convert() made keyword-only past (ckpt_path, model_version, out_path)
  per the interface contract; out_path is injected (no folder_paths)
- dedup: load_coreml_model / convert_to_coreml / get_coreml_inputs /
  add_cnet_support / get_encoder_hidden_states_shape / inputs-spec now
  defined once in the package; get_sample_input gains an optional
  scheduler arg so the LCM path shares it (same keys/order/dtypes)
- coreml_suite.{converter,lcm.converter} reduced to comfy-side shims:
  folder_paths path resolution and the LCM scheduler's
  comfy.model_management stay here; the package imports neither
- __init__ keeps discovery + compose_out_name eager; convert is lazy via
  __getattr__ so 'import coreml_diffusion' stays Tier-0 pure

Nodes untouched (E3 thins them onto the package). Tier-0 (109) and smoke
(3, real coremltools conversion) green; [M2-ANE] golden pending a server.

* refactor(extraction): E3 thin nodes onto coreml_diffusion + discovery dropdowns

The CoreMLConverter node now calls coreml_diffusion directly instead of the
coreml_suite.converter shim, and its dropdowns are populated at runtime from
the package's discovery API.

- INPUT_TYPES dropdowns (model_version / attention_implementation /
  quantize_nbits) now come from a fail-soft _discover() that calls
  coreml_diffusion.list_*; a missing/old package falls back to a literal list
  and logs a warning instead of de-registering the node. Installing a newer
  coreml_diffusion surfaces new conversion types with no Suite change.
- folder_paths path resolution moved inline into the node; the package's
  convert() takes the output path as an injected positional.
- compose_out_name / lora_names_from_params now imported from coreml_diffusion
  (lazily, inside convert) — no node-side copy.
- deleted the dead coreml_suite/converter.py and coreml_suite/core/naming.py
  shims (no remaining importers).

Field names, RETURN_TYPES/NAMES and NODE_*_MAPPINGS unchanged; dropdown values
are a superset of the prior literals (additive-only). Tier-0 (109) and smoke
(3) green; [M2-ANE] golden re-runs on push.

* refactor(extraction): E5 depend on external coreml-diffusion package

Conversion code now lives in the standalone coreml-diffusion repo. The Suite
deletes its in-tree copy and depends on the package instead.

- removed coreml_diffusion/ (whole package), coreml_suite/model_version.py and
  coreml_suite/attention.py (moved to the package as its source of truth), and
  the tests that moved with them (discovery, conversion_helpers, out_name; smoke
  synthetic_unet + split_einsum)
- re-pointed ModelVersion imports (config.py, nodes.py, lcm/converter.py) to
  coreml_diffusion
- pyproject: drop the coreml_diffusion package include and the conversion-only
  deps (peft/omegaconf/transformers, now transitive via coreml-diffusion); add
  coreml-diffusion as a dependency with a local path source until it is published
  (switch to git tag/PyPI once the repo exists, so CI can resolve it)

Suite Tier-0 green (75); conversion code fully absent from the Suite. The comfy
node still imports coreml_diffusion (installed package) for ModelVersion + the
discovery dropdowns + convert.

* build(extraction): pin coreml-diffusion to git tag v0.1.0

Switch the coreml-diffusion source from a local path to the published git tag
so CI can resolve it. Suite Tier-0 green resolving from the tag.

* ci(extraction): drop Suite smoke tier (moved to coreml-diffusion)

The conversion smoke tests moved to the coreml-diffusion repo, which runs its
own Tier 1. The Suite's smoke lane had no tests left (pytest exit 5). The Suite
keeps Tier 0 (inference units) and the m2 golden e2e.

* chore(release): v2.1.0; wire coreml-diffusion into requirements.txt

Minor bump: the conversion path moved to the external coreml-diffusion package
(node graph + artifact cache keys unchanged, golden-verified). requirements.txt
(used by ComfyUI Manager) now installs coreml-diffusion from the v0.1.0 tag and
drops the conversion-only deps now provided transitively.
2026-05-26 22:12:56 +02:00
aszc-dev f054bbe991 docs: update Stable Diffusion 1.5 links 2026-05-26 22:01:31 +02:00
aszc 20ec450f9b fix: allow custom converter dimensions (#60) 2026-05-26 16:17:32 +02:00
aszc d0cca3c3f4 fix(conversion): load .mlpackage instead of unloadable .mlmodelc (#59)
* fix(conversion): load .mlpackage instead of unloadable .mlmodelc

The native ct.models.MLModel runtime added in the diffusers conversion
path cannot load compiled .mlmodelc directories (no Manifest.json), so
the converter's compiled output failed at load with "A valid manifest
does not exist". Both converters now return the .mlpackage directly and
the loader lists only .mlpackage. Removes the now-dead coremlcompiler
wrappers.

* chore(release): bump version to 2.0.1
2026-05-26 16:04:38 +02:00
aszc 65a2de2fab feat!: modernize toolchain and replace apple/ml-stable-diffusion with native diffusers conversion (#58)
* feat(deps): support ComfyUI's numpy 2 toolchain; make conversion optional

The runtime package now installs and runs under numpy 2 / coremltools 9 /
torch 2.7 — matching current ComfyUI — without apple/ml-stable-diffusion.

- Drop the heavy converter stack (ml-stable-diffusion, diffusers, peft,
  omegaconf, overrides, transformers) from runtime dependencies; require
  numpy>=2.
- Vendor the runtime pieces: a slim CoreMLModel wrapper around coremltools
  and the attention-implementation constants.
- Lazy-import the converters; the Convert nodes raise a clear error when the
  legacy conversion dependencies are absent. Loading and sampling existing
  Core ML models no longer needs them.
- CI: Tier 0 tracks the numpy 2 / torch 2.7 toolchain; drop the Tier 2
  golden-image lane (it converts at runtime, which now requires the legacy
  stack) and its fixtures.

* feat(conversion): replace apple/ml-stable-diffusion with native diffusers path

Reimplement Core ML UNet conversion on top of diffusers instead of the
apple/ml-stable-diffusion git dependency, so the full suite (including
conversion) installs through ComfyUI Manager without extras on the NumPy 2
toolchain.

- Add coreml_suite/conversion package: split-einsum attention processors,
  a conv2d output-shape helper, Transformer2D trace patches, and a UNet
  input-adapter wrapper preserving the historical Core ML I/O contract.
- Drop python_coreml_stable_diffusion and overrides; route SD15, SDXL,
  SDXL refiner, and LCM conversion through diffusers UNet2DConditionModel.
- Declare diffusers, peft, omegaconf, and transformers as runtime deps.
- Add characterization tests asserting split-einsum matches reference
  attention math; extend the synthetic-UNet smoke test for the wrapper.
- Bump to 1.1.0 and set requires-comfyui to a semver constraint (>=0.3.27)
  so the Comfy Registry publish succeeds.

* refactor(conversion)!: native diffusers context layout; drop legacy fallbacks

Address PR review feedback:

- Drop the legacy converter ImportError fallbacks and LEGACY_CONVERTER_MODULES
  guards in nodes.py and lcm/nodes.py. Conversion dependencies are mandatory in
  pyproject, so the indirection is dead code.
- Tier 0 CI resolves its toolchain from pyproject via uv (uv sync + uv run)
  instead of hand-pinned pip installs, removing duplicated version maintenance.
- Document the conversion lineage: credit apple/ml-stable-diffusion as the
  origin, note the implementation has diverged to a native diffusers path, and
  state the intent to iterate independently. Fix stale README links that pointed
  users to apple/ml-stable-diffusion for conversion.
- Drop the unused `sources` argument from CoreMLModel.

BREAKING CHANGE: the converted Core ML UNet now takes encoder_hidden_states in
the native diffusers layout (batch, tokens, hidden) instead of
(batch, hidden, 1, tokens). This removes the boundary transposes in
CoreMLUNetWrapper and CoreMLInputs. Core ML models converted with earlier
versions are incompatible and must be re-converted. Bump to 2.0.0.

* test: widen split-einsum allclose tolerance for cross-platform float drift

The split-einsum attention reorders float32 reductions relative to the
reference, so equality holds only up to rounding. The default allclose atol
(1e-8) is too tight on Linux x86 BLAS and failed Tier 0 CI; use atol=1e-6 to
match the existing chunked-path characterization test.

* ci: run macOS smoke tier on the self-hosted Apple Silicon runner

GitHub-hosted macOS carries a 10x minute multiplier and exhausts the included
Actions minutes too quickly. Move the Tier 1 smoke job onto the self-hosted
Apple Silicon runner ([self-hosted, macOS, ARM64, coreml]) so macOS coverage no
longer consumes hosted minutes. Tier 0 stays on hosted ubuntu (1x).

* ci: fix uv setup for both tiers

astral-sh/setup-uv@v3 was retagged and its old commit garbage-collected, so
codeload 404s when Actions resolves the stale SHA. Bump Tier 0 (ubuntu) to
setup-uv@v7, and drop the action entirely from Tier 1 since the self-hosted
runner already provides uv.

* test(ci): restore golden-image correctness gate on the self-hosted runner

The Tier 2 end-to-end correctness check (real SD1.5 -> Core ML -> image,
gated on SHA/PSNR vs a golden) was dropped during the modernization. With the
breaking 3D-context change, the synthetic smoke and shape/attention
characterization tests no longer cover real-model conversion correctness.

Restore tier2.yml (on the [self-hosted, macOS, ARM64, coreml] runner shared
with Tier 1), the golden-image test, and the e2e workflow. Adapt the pinned
ComfyUI resolution to the requires-comfyui semver tag (vX.Y.Z) instead of a
commit SHA, and re-register the m2 marker. The golden is intentionally not
committed: the first self-hosted run regenerates it under the new 3D contract
and fails for review, per the test's documented bootstrap.

* test(ci): add golden image for SD1.5 seed 42 under the 3D-context contract

Generated by the first Tier 2 self-hosted run after the native diffusers
conversion change. The decoded image is a coherent SD1.5 generation, confirming
the (batch, tokens, hidden) Core ML contract produces correct output
end-to-end. Subsequent runs gate on this golden (SHA-strict, PSNR fallback).

* test(ci): force fresh conversion in Tier 2; drop stale golden

The converter skips conversion when a same-named model already exists, keyed on
conversion parameters but not the conversion code/toolchain. The self-hosted
runner held a pre-existing v1-5 .mlmodelc (4D-context, old toolchain), so the
Tier 2 runs reused it (~8-30s) instead of converting — the gate validated a
stale model, not the new native diffusers path.

Purge the cached Core ML UNets before running so every Tier 2 run does a real
convert -> compile -> sample. Drop the golden generated from the stale cache;
the next run regenerates it from a genuine 3D-contract conversion and fails for
review.

* test(ci): add golden image from a genuine 3D-contract conversion

Regenerated by a Tier 2 run with the model cache purged, so the converter
actually ran (62s, not a cache hit). The fresh model exposes the new 3D
encoder_hidden_states input [1, 77, 768], and its decoded SD1.5 seed-42 image
is byte-identical to the prior baseline — confirming the native diffusers
conversion is behavior-preserving end to end.
2026-05-26 15:46:09 +02:00
aszc 02b6e8ece3 feat: modernize toolchain, refactor core, add tiered CI and opt-in quantization
Modernizes ComfyUI-CoreMLSuite onto Python 3.12 / torch 2.7 / coremltools 9
with a characterization-test safety net. The default conversion path is
unchanged; existing saved workflows produce identical output.

- Toolchain bump (Python 3.12, torch 2.7, coremltools 9, numpy <2) with the
  blocking upstream pins overridden.
- Framework-free logic moved into coreml_suite/core/ (no comfy/coremltools
  imports); old module paths re-export from there.
- Opt-in quantize_nbits dropdown (none|8|6|4) for k-means weight
  palettization; default none is byte-for-byte identical to before.
- Tiered CI: Tier 0 (Linux unit), Tier 1 (macOS-ARM smoke), Tier 2
  (self-hosted Apple Silicon golden-image check on the ANE).
2026-05-25 19:11:49 +02:00
snomiao 7678a07ed5 chore(publish): update GitHub Actions workflow for node publishing
- Added permissions for issue writing
- Updated action version to v1 for publish-node-action
- Added condition to run job only for 'aszc-dev' repository owner
2025-04-01 23:45:31 +02:00
snomiao 43b77e8471 chore(licence-update): Update PyProject Toml - License 2024-08-15 20:37:19 +02:00
aszc-dev c96059ff0b Add basic conversion integration test 2024-07-04 08:44:37 +02:00
aszc-dev 3224d62342 Restructure tests directory 2024-07-04 08:44:37 +02:00
aszc-dev 2fb135df03 Fix set_timestamps for new LCMScheduler implementation 2024-07-04 08:44:37 +02:00
aszc-dev 66e83c2f2f Change syntax to support older Python versions 2024-07-04 08:44:37 +02:00
aszc fb7188e5a2 Update pyproject.toml to test registry workflow 2024-07-03 16:15:37 +02:00
haohaocreates 4096466f8c chore(publish): Add Github Action for Publishing to Comfy Registry 2024-07-03 16:13:36 +02:00
aszc b8c263b763 Update pyproject.toml 2024-07-03 16:08:02 +02:00
haohaocreates 56cff2bd91 chore(pyproject): Add pyproject.toml for Custom Node Registry 2024-07-03 16:08:02 +02:00
Chris Chance 7b3f8fc29e Update ModelSamplingDiscreteLCM to Distilled for latest comfyui 2023-12-01 01:09:14 +01:00
Chris Chance adaecd3f66 Lowered minimum CoreML Size to 256x256 2023-11-28 17:49:43 +01:00
aszc-dev aa60cda09b Add installation using ComfyUI-Manager instructions 2023-11-24 15:10:33 +01:00
aszc-dev 5f7fcd6df3 Add note on SD2.1 to readme 2023-11-24 12:42:33 +01:00
aszc-dev e89cff6d01 Update readme with SDXL info 2023-11-24 12:14:15 +01:00
aszc-dev 9f90083126 Update converter docs and workflows 2023-11-24 12:14:15 +01:00
aszc-dev 5c774ddc5e Remove LCM option from converter for now 2023-11-24 12:14:15 +01:00
aszc-dev b8197c21ef Converting refiner works 2023-11-24 12:14:15 +01:00
aszc-dev 0c78803b25 Base SDXL conversion works 2023-11-24 12:14:15 +01:00
aszc-dev 763ca3961b Handle SDXL config 2023-11-24 12:14:15 +01:00
aszc-dev ae9a9874c5 Add Advanced Sampler node 2023-11-24 12:14:15 +01:00
aszc-dev 67c902f761 Generating SDXL with Core ML Sampler works 2023-11-24 12:14:15 +01:00
aszc-dev bb44b4a35f Link to ComfyUI repo 2023-11-24 12:14:15 +01:00
aszc-dev 46d1124573 Update REAMDE.md (Conversion and LoRA) 2023-11-17 22:55:13 +01:00
aszc-dev ead01c08dd Remove lora.py 2023-11-17 22:55:13 +01:00
aszc-dev b10effc7c2 Add conversion/lora workflows 2023-11-17 22:55:13 +01:00
aszc-dev b1d2e82677 Add peft and omegaconf to requirements 2023-11-17 22:55:13 +01:00
aszc-dev 9f650acb79 Load .yaml config if present 2023-11-17 22:55:13 +01:00
aszc-dev c6d6917827 Setting LoRA model weights works 2023-11-17 22:55:13 +01:00
aszc-dev 63377ebd73 Store lora_params in dict 2023-11-17 22:55:13 +01:00
aszc-dev 42ff10cd43 Add node to load LoRAs 2023-11-17 22:55:13 +01:00
aszc-dev da3a8e13d3 Add logging during conversion 2023-11-17 22:55:13 +01:00
aszc-dev 8092a19173 Enable choosing attention implementation during conversion 2023-11-17 22:55:13 +01:00
aszc-dev 5477e3d71a Remove CLIP loader from nodes 2023-11-17 22:55:13 +01:00
aszc-dev a8d2d6ec46 Move lora related code around, remove clip stuff 2023-11-17 22:55:13 +01:00
aszc-dev 44cffbb8b8 Move load_lora to lora.py 2023-11-17 22:55:13 +01:00
aszc-dev 6907d4910f Remove ckpt loading when loading lora clip 2023-11-17 22:55:13 +01:00
aszc-dev 1930be5c98 Remove CLIP related code 2023-11-17 22:55:13 +01:00
aszc-dev 45be6761d1 Basic conversion + LoRA support works 2023-11-17 22:55:13 +01:00
aszc-dev fc1132a5d5 Fix category for all Core ML nodes 2023-11-17 22:55:13 +01:00
aszc-dev e440f725a4 Specify diffusers and coremltools versions in requirements.txt 2023-11-14 18:43:15 +01:00