ci(phase4): track latest ComfyUI in Tier 2 and fix runner wiring

Tier 2 is the canary for a moving host, so make it test against latest
ComfyUI explicitly instead of whatever happens to sit on the runner:

- add an 'Update ComfyUI to latest master' step that resets $COMFY_DIR to
  origin/master each run and records the resolved SHA (GITHUB_ENV +
  step summary) so failures name the commit they hit.
- replace the startup-banner grep with an HTTP readiness probe against
  /system_stats, robust to colored-log / banner changes in floating latest.
- drop the self-referential 'env: COMFY_DIR: ${{ env.COMFY_DIR }}' that
  could shadow the runner .env value with an empty string.
- document the one-time custom_nodes symlink so the server loads the
  checked-out PR, not a stale node copy, plus a ComfyUI-version section
  clarifying canary-latest vs the requires-comfyui published pin.
This commit is contained in:
aszc-dev
2026-05-25 10:51:45 +02:00
parent 1240524201
commit 085509d83e
2 changed files with 61 additions and 6 deletions
+24 -5
View File
@@ -26,16 +26,35 @@ jobs:
- name: uv sync
run: uv sync
- name: Update ComfyUI to latest master (canary)
# Tier 2 deliberately tracks the moving host: every run resets the
# runner's ComfyUI checkout to origin/master and records the resolved
# SHA, so upstream API breakage surfaces here instead of in a user's
# install. (pyproject `requires-comfyui` stays the published-compat
# declaration; this lane is the early-warning canary, not the contract.)
# COMFY_DIR is exported by the self-hosted runner's .env (see docs/ci-m2.md).
# The node under test lives at $COMFY_DIR/custom_nodes/ComfyUI-CoreMLSuite,
# symlinked to $GITHUB_WORKSPACE at runner-setup time, so this reset does
# not touch the checkout being validated.
run: |
git -C "$COMFY_DIR" fetch --quiet origin
git -C "$COMFY_DIR" checkout --quiet master \
|| git -C "$COMFY_DIR" checkout --quiet -b master origin/master
git -C "$COMFY_DIR" reset --hard --quiet origin/master
COMFY_SHA="$(git -C "$COMFY_DIR" rev-parse HEAD)"
echo "COMFY_SHA=$COMFY_SHA" >> "$GITHUB_ENV"
echo "Tier 2 tested against ComfyUI \`$COMFY_SHA\` (latest master)" \
>> "$GITHUB_STEP_SUMMARY"
- name: Start ComfyUI server (background)
env:
COMFY_DIR: ${{ env.COMFY_DIR }}
run: |
cd "$COMFY_DIR"
nohup "$GITHUB_WORKSPACE/.venv/bin/python" main.py --port 8188 --cpu-vae > /tmp/comfyui-ci.log 2>&1 &
# Wait for readiness, fail fast if it never comes up.
# Poll the HTTP endpoint for readiness — robust to startup-banner
# wording / colored-log changes in a floating-latest ComfyUI.
for _ in $(seq 1 60); do
if grep -q "To see the GUI" /tmp/comfyui-ci.log 2>/dev/null; then
echo "comfy ready"; exit 0
if curl -sf -o /dev/null http://127.0.0.1:8188/system_stats; then
echo "comfy ready (ComfyUI ${COMFY_SHA:-unknown})"; exit 0
fi
sleep 2
done
+37 -1
View File
@@ -39,7 +39,24 @@ half of the matrix has to live on real hardware.
echo 'COMFY_DIR=/Users/<you>/dev/ComfyUI' >> ~/actions-runner/.env
```
4. **Pre-convert the baseline SD1.5 model.** The bench step expects
4. **Symlink the node under test into ComfyUI.** `actions/checkout` clones
the PR into `$GITHUB_WORKSPACE` (`~/actions-runner/_work/<repo>/<repo>`),
but ComfyUI only loads custom nodes from `$COMFY_DIR/custom_nodes/`.
Without a link, Tier 2 would spin up the server against a *stale* copy of
the node instead of the checked-out PR. Point the load path at the
runner's workspace once (the workspace path is stable for a self-hosted
runner):
```bash
rm -rf "$COMFY_DIR/custom_nodes/ComfyUI-CoreMLSuite"
ln -s ~/actions-runner/_work/ComfyUI-CoreMLSuite/ComfyUI-CoreMLSuite \
"$COMFY_DIR/custom_nodes/ComfyUI-CoreMLSuite"
```
After this, `uv sync`, `pytest`, and the ComfyUI server all run against
the same tree.
5. **Pre-convert the baseline SD1.5 model.** The bench step expects
`$COMFY_DIR/models/unet/v1-5-pruned-emaonly_1x512x512_se_unet.mlmodelc`.
Run the conversion once manually:
@@ -60,6 +77,25 @@ The Tier 2 workflow (`.github/workflows/tier2.yml`) runs:
- **Nightly at 04:00 UTC** via `schedule:`.
- **Manually** via the workflow_dispatch button.
## ComfyUI version under test
Every Tier 2 run resets `$COMFY_DIR` to **latest `origin/master`** before
starting the server (`Update ComfyUI to latest master` step). This is a
deliberate early-warning canary: the suite tracks a moving host, so upstream
API breakage should surface here — in CI — rather than in a user's install.
The resolved ComfyUI SHA is written to the job's step summary (and `COMFY_SHA`
in the env) so any failure says exactly which commit it was tested against.
This is separate from `pyproject.toml`'s `requires-comfyui` pin, which is the
**published-compatibility declaration** for the Comfy registry, not the CI
target. Bump that pin deliberately once a newer ComfyUI is validated; do not
expect it to match the floating SHA Tier 2 reports.
Because the step does `git reset --hard`, the runner's ComfyUI checkout must
not hold local commits you care about — treat it as disposable. The symlinked
`custom_nodes/ComfyUI-CoreMLSuite` lives outside that repo's tracked tree, so
the reset never touches the node under test.
## Artifacts
- Bench JSON/MD are uploaded as `bench-results`.