feat: modernize toolchain, refactor core, add tiered CI and opt-in quantization
Modernizes ComfyUI-CoreMLSuite onto Python 3.12 / torch 2.7 / coremltools 9 with a characterization-test safety net. The default conversion path is unchanged; existing saved workflows produce identical output. - Toolchain bump (Python 3.12, torch 2.7, coremltools 9, numpy <2) with the blocking upstream pins overridden. - Framework-free logic moved into coreml_suite/core/ (no comfy/coremltools imports); old module paths re-export from there. - Opt-in quantize_nbits dropdown (none|8|6|4) for k-means weight palettization; default none is byte-for-byte identical to before. - Tiered CI: Tier 0 (Linux unit), Tier 1 (macOS-ARM smoke), Tier 2 (self-hosted Apple Silicon golden-image check on the ANE).
This commit is contained in:
aszc
@@ -0,0 +1,124 @@
|
||||
name: Tier 2 — M2 / ANE (self-hosted)
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
# `labeled` fires when run-m2 is first added; `synchronize`/`reopened`
|
||||
# re-run on every subsequent push while the label is present, so the
|
||||
# result tracks the PR head instead of going stale. The `if` below keeps
|
||||
# the run gated on the run-m2 label for all pull_request events.
|
||||
types: [labeled, synchronize, reopened]
|
||||
schedule:
|
||||
# Nightly at 04:00 UTC (~05/06 in PL). Keeps the M2 path honest
|
||||
# without burning the runner on every PR.
|
||||
- cron: "0 4 * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
m2:
|
||||
if: |
|
||||
github.event_name == 'schedule' ||
|
||||
github.event_name == 'workflow_dispatch' ||
|
||||
(github.event_name == 'pull_request' &&
|
||||
contains(github.event.pull_request.labels.*.name, 'run-m2'))
|
||||
# Self-hosted Apple Silicon runner. Prerequisites: COMFY_DIR pointing at
|
||||
# a runner-owned ComfyUI clone, plus a cached SD1.5 checkpoint.
|
||||
runs-on: [self-hosted, macOS, ARM64, coreml]
|
||||
timeout-minutes: 90
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Hybrid ComfyUI strategy:
|
||||
# - schedule (nightly) -> latest origin/master + ComfyUI's own
|
||||
# requirements.txt (constrained). Canary for upstream API breakage.
|
||||
# - PR label / dispatch -> the pinned requires-comfyui SHA + the frozen
|
||||
# `comfy` uv group. Reproducible merge gate, immune to overnight drift.
|
||||
- name: Resolve ComfyUI ref + mode
|
||||
run: |
|
||||
if [ "$GITHUB_EVENT_NAME" = "schedule" ]; then
|
||||
echo "COMFY_MODE=latest" >> "$GITHUB_ENV"
|
||||
echo "COMFY_REF=master" >> "$GITHUB_ENV"
|
||||
else
|
||||
PIN="$(sed -nE 's/^requires-comfyui *= *"==?([0-9a-f]+)".*/\1/p' pyproject.toml)"
|
||||
if [ -z "$PIN" ]; then echo "could not parse requires-comfyui from pyproject.toml"; exit 1; fi
|
||||
echo "COMFY_MODE=pinned" >> "$GITHUB_ENV"
|
||||
echo "COMFY_REF=$PIN" >> "$GITHUB_ENV"
|
||||
fi
|
||||
|
||||
- name: Set up ComfyUI checkout
|
||||
# COMFY_DIR is exported by the self-hosted runner's .env and MUST be a
|
||||
# runner-owned ComfyUI clone (never your dev checkout — this step does
|
||||
# git reset --hard and rewrites custom_nodes). Cloned on first run.
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${COMFY_DIR:-}" ]; then echo "COMFY_DIR unset"; exit 1; fi
|
||||
# Init-in-place rather than `git clone`: COMFY_DIR may already hold the
|
||||
# cached checkpoint (models/checkpoints) or converted .mlmodelc, and
|
||||
# `git clone` refuses a non-empty target. init + fetch + `checkout -f`
|
||||
# populates the ComfyUI tree while leaving untracked files (the
|
||||
# checkpoint, the cached models) untouched — so setup order is free.
|
||||
if [ ! -d "$COMFY_DIR/.git" ]; then
|
||||
echo "initialising ComfyUI repo in $COMFY_DIR"
|
||||
mkdir -p "$COMFY_DIR"
|
||||
git -C "$COMFY_DIR" init -q
|
||||
fi
|
||||
git -C "$COMFY_DIR" remote get-url origin >/dev/null 2>&1 \
|
||||
|| git -C "$COMFY_DIR" remote add origin https://github.com/comfyanonymous/ComfyUI.git
|
||||
git -C "$COMFY_DIR" fetch --quiet origin
|
||||
if [ "$COMFY_MODE" = "latest" ]; then
|
||||
git -C "$COMFY_DIR" checkout -f -B master origin/master
|
||||
else
|
||||
git -C "$COMFY_DIR" checkout -f "$COMFY_REF"
|
||||
fi
|
||||
COMFY_SHA="$(git -C "$COMFY_DIR" rev-parse HEAD)"
|
||||
echo "COMFY_SHA=$COMFY_SHA" >> "$GITHUB_ENV"
|
||||
echo "Tier 2 mode=$COMFY_MODE, ComfyUI \`$COMFY_SHA\`" >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
# Point ComfyUI's custom-node loader at this checkout. Refresh the
|
||||
# symlink only; refuse to clobber a real directory (guards against a
|
||||
# COMFY_DIR that is accidentally a dev checkout).
|
||||
NODE_LINK="$COMFY_DIR/custom_nodes/ComfyUI-CoreMLSuite"
|
||||
if [ -e "$NODE_LINK" ] && [ ! -L "$NODE_LINK" ]; then
|
||||
echo "ERROR: $NODE_LINK is a real directory, not a symlink."
|
||||
echo "COMFY_DIR must be a runner-owned ComfyUI, not your dev checkout."
|
||||
exit 1
|
||||
fi
|
||||
mkdir -p "$COMFY_DIR/custom_nodes"
|
||||
ln -sfn "$GITHUB_WORKSPACE" "$NODE_LINK"
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ "$COMFY_MODE" = "latest" ]; then
|
||||
# Node deps (our coremltools-9 toolchain), then ComfyUI's own
|
||||
# requirements for the pulled SHA, capped by the toolchain ceiling.
|
||||
uv sync
|
||||
uv pip install -r "$COMFY_DIR/requirements.txt" \
|
||||
-c constraints/comfy-ceiling.txt
|
||||
else
|
||||
# Pinned gate: the frozen group mirrors the known-good pinned SHA.
|
||||
uv sync --group comfy
|
||||
fi
|
||||
|
||||
- name: Start ComfyUI server (background)
|
||||
run: |
|
||||
cd "$COMFY_DIR"
|
||||
nohup "$GITHUB_WORKSPACE/.venv/bin/python" main.py --port 8188 --cpu-vae > /tmp/comfyui-ci.log 2>&1 &
|
||||
# Poll the HTTP endpoint for readiness — robust to startup-banner
|
||||
# wording / colored-log changes in a floating-latest ComfyUI.
|
||||
for _ in $(seq 1 90); do
|
||||
if curl -sf -o /dev/null http://127.0.0.1:8188/system_stats; then
|
||||
echo "comfy ready (ComfyUI ${COMFY_SHA:-unknown})"; exit 0
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
echo "comfy failed to start"; tail -100 /tmp/comfyui-ci.log; exit 1
|
||||
|
||||
- name: Run Tier 2 (m2 marker)
|
||||
# The golden-image workflow drives the Core ML Converter node, so the
|
||||
# UNet is converted on demand on the first run and reused from the
|
||||
# runner-local cache afterwards.
|
||||
run: uv run --no-sync pytest -m m2 tests/ -v
|
||||
|
||||
- name: Stop ComfyUI server
|
||||
if: always()
|
||||
run: pkill -f "main.py.*8188" || true
|
||||
Reference in New Issue
Block a user