The frozen 'comfy' uv group cannot track a moving host by hand: a latest
ComfyUI checkout already needs comfyui-frontend-package==1.44.19, comfy_aimdo,
alembic and blake3 that the old pin never listed, so 'import comfy' fails
outright against latest. Make Tier 2 source ComfyUI's deps from upstream
instead of a hand-frozen list, keyed by trigger:
- schedule (nightly) -> latest origin/master + ComfyUI's own requirements.txt,
capped by constraints/comfy-ceiling.txt (torch<2.8, numpy<2, coremltools 9).
Early-warning canary; a hard upstream conflict fails on purpose, signalling
a needed toolchain bump rather than silently floating past the ANE ceiling.
- PR label / dispatch -> pinned requires-comfyui SHA + frozen 'comfy' group.
Reproducible gate, immune to overnight drift.
Make the runner self-contained so there's no manual local fiddling:
- clone ComfyUI into COMFY_DIR on first run; checkout the resolved ref.
- symlink custom_nodes/ComfyUI-CoreMLSuite -> GITHUB_WORKSPACE in-workflow,
refusing to clobber a real directory (guards a misconfigured COMFY_DIR).
- convert-if-missing for all UNet variants (none/8/6/4), cached across runs;
only the checkpoint stays a runner-local artifact.
- record resolved ComfyUI SHA + mode in the job step summary.
Add the Phase 6 quant tradeoff matrix (bench/scripts/quant_matrix.py) to the
bench lane and upload .md alongside .json. Pass --no-sync to every 'uv run' so
the post-install steps keep the deps just installed instead of re-syncing to
the lock and dropping them.
Tier 2 is the canary for a moving host, so make it test against latest
ComfyUI explicitly instead of whatever happens to sit on the runner:
- add an 'Update ComfyUI to latest master' step that resets $COMFY_DIR to
origin/master each run and records the resolved SHA (GITHUB_ENV +
step summary) so failures name the commit they hit.
- replace the startup-banner grep with an HTTP readiness probe against
/system_stats, robust to colored-log / banner changes in floating latest.
- drop the self-referential 'env: COMFY_DIR: ${{ env.COMFY_DIR }}' that
could shadow the runner .env value with an empty string.
- document the one-time custom_nodes symlink so the server loads the
checked-out PR, not a stale node copy, plus a ComfyUI-version section
clarifying canary-latest vs the requires-comfyui published pin.
Phase 4 of the modernization plan: institutionalize the 3-tier strategy
so future changes are guarded automatically, and pin down the
self-hosted M2 path the maintainer's hardware needs.
Tier dispatch
- Makefile targets test-unit / test-smoke / test-m2 / bench (plus
ci-tier0 / ci-tier1 wrappers that echo env first). check-macos-arm
fails fast on non-Apple-Silicon hosts.
Tier 1 smoke
- tests/smoke/test_synthetic_unet.py: builds a TinyUNet (conv-in,
time/text projections, conv-out), traces it, ct.convert to
mlprogram + fp16 CPU_ONLY, loads back via CoreMLModel and asserts
expected_inputs + named output. Runs in ~2s; auto-skips on
non-Apple-Silicon. Catches coremltools / ml-stable-diffusion API
drift without needing a real SD checkpoint or the ANE.
GitHub Actions
- .github/workflows/tier0.yml: ubuntu-latest on every push/PR, ~10
min budget, minimal-deps install (torch==2.0.1, numpy<1.25, pytest)
-> pytest -m unit.
- .github/workflows/tier1.yml: macos-14 (M1) on push/PR; opt-in via
run-tier1 label on labeled PRs to spare external-doc PRs.
- .github/workflows/tier2.yml: self-hosted [macOS, ARM64, coreml] on
PR label run-m2 / nightly cron / workflow_dispatch. Starts ComfyUI
with --cpu-vae, runs pytest -m m2 + bench/run.py, uploads bench
results.
Integration coverage moved
- Removed tests/integration/test_basic_conversion_1_5.py: it required
an MPS reference image (broken on macOS 26 + torch 2.0.1, see
Phase 1 Gate) and a checkpoint the maintainer doesn't have on disk
(dreamshaper_8). The same coverage now lives in
tests/m2/test_golden_image.py: deterministic numerical pass/fail
(SHA256 + PSNR fallback) against a stored golden, Core ML pipeline
only. No more human eyeballing.
Docs
- docs/ci-m2.md: one-time runner registration steps, COMFY_DIR
persistence, baseline model pre-conversion, trigger semantics, what
to do when the runner is offline, and the migration note from
integration -> m2 golden.
Sanity check
- Temporarily set convert_to="BREAKAGE_CANARY_NOT_A_REAL_FORMAT" in
the smoke test; Tier 1 surfaced
NotImplementedError: Backend converter BREAKAGE_CANARY_NOT_A_REAL_FORMAT not implemented
immediately. Reverted.
Local verification
- make test-unit -> 88/88 passed in 2.09s
- make test-smoke -> 1/1 passed in 1.99s
- Added permissions for issue writing
- Updated action version to v1 for publish-node-action
- Added condition to run job only for 'aszc-dev' repository owner
aszc-dev