Compare commits

..
Author SHA1 Message Date
Vito Sansevero ab38656f05 ci(comfyui): run the real-host job in the pinned Playwright image
The real ComfyUI compatibility job installed Chromium with
`playwright install --with-deps`, which pulls the browser's system
packages from Ubuntu's apt mirror on every run. Three runs (#349 run
40367579020, #378, #382 job 122287230985: "Fetched 19.7 MB in 12min 23s
(26.5 kB/s)") spent 11-12 minutes there and were cancelled at the
24-minute limit after all 89 tests had passed.

The job now runs inside mcr.microsoft.com/playwright:v1.64.1-noble, the
same image as the frontend coverage and page workflow jobs, which ships
Chromium and its libraries, so the job has no apt step at all. The
whole job moves rather than only the browser part: the Playwright
suites launch the managed ComfyUI host themselves (PM_TEST_PYTHON), so
splitting the browser off would mean running the host on the runner and
the browser in a container across a network boundary, a larger change
than moving one job whose host is plain Python and pip wheels.

Python follows the other image jobs: the image's own Python 3.12 with a
user-space venv bootstrapped from a checksum-pinned pip wheel (toolcache
setup-python builds do not run in the container). A step fails the job
when the image Python differs from pythonVersion in
tools/vendor/comfyui-runtime.json, so the pinned host stays honest;
torch, torchvision and ComfyUI's requirements install from the same
pins as before. pip and npm download caches replace setup-python's and
setup-node's.

tests/vendor/browser-image.test.mjs now checks this job like the other
browser jobs: image tag equals @playwright/test in the lockfile, no
browser install or apt, no setup-python/setup-node, checksum-pinned pip,
Node 24, and the runtime Python check. The 24-minute limit is unchanged.
2026-10-27 09:10:12 -07:00
Vito Sansevero b558a7e3d6 test(history): drain the writer before every bounded close or dispose
The merge queue's Windows py3.12 job failed
test_saturated_start_cannot_capture_outputs_or_retain_previous_roots:
dispose(2) ran while the writer still worked through 1,024 queued
commands on a busy runner. Like #370, the test now drains first; the 2 s
bound is not what it tests. Four more closes that follow a released
block get the same drain, so the next slow runner can't fail them one at
a time.
2026-10-27 08:27:26 -07:00
Vito Sansevero ef5080642c test(history): give waits that span a database restore their own ceiling
The merge queue's Windows py3.12 job failed
test_restore_or_generation_rotation_between_transactions_discards_evidence
(restore=True): the restore it waits for, a full verify/migrate/backup/
replace cycle, took over the 10 s BLOCK_SECONDS on a busy runner. The two
waits that span a restore now use RESTORE_SECONDS (90 s), which stays under
the suite's 120 s hang watchdog, so a real hang still fails with its thread
dump. Behavioral waits keep BLOCK_SECONDS.
2026-10-27 07:18:32 -07:00
Vito Sansevero ef2d65716c ci: run the required workflows for the merge queue
main no longer requires a branch to be up to date before it merges, so a
PR no longer re-runs CI after every other merge. The Tests, Code Quality
and Viewer.js workflows also run on merge_group, so the merge queue can
test each PR against the latest main once, in order. The tracked
protection record now says strict: false.
2026-10-27 06:16:52 -07:00
Vito d868745194 fix(lora): cache LoraManager's layout and close the #355 follow-ups (#362) (#364)
* feat(lora): cache LoraManager's layout behind file signatures (#362, phase 1)

A new py/lora_layout_cache.py keeps one (install, layout) entry and reuses it
while the configured path, the plugin settings file's and the settings file's
signatures (st_mtime_ns, st_size), the install and every media root's real
path are unchanged and the entry is younger than 30 s. A load whose watched
file changed within 3 s (git's racy rule), could not be checked, or whose
settings were unreadable serves its request only. clear() bumps a generation
so a load that started earlier stores nothing. Hits take no lock.

Not wired yet; phase 2 puts it behind the image hot paths.

* fix(lora): treat an offline Windows share as unvalidated and make cache tests portable (#362)

Review findings on phase 1: Windows raises FileNotFoundError with winerror
53/67 for an offline share, which is a temporary failure, not a missing file.
Test times are whole milliseconds (NTFS keeps 100 ns steps) and a file system
that rounds them skips; cleanup no longer ignores errors; tests added for
ValueError, NotADirectoryError, a failed reload, resolve failures and the
watched-file set.

* perf(lora): serve the image hot paths from the layout cache (#362, phase 2)

_lora_image_dirs takes (install, layout) from lora_utils.cached_lora_manager_layout
instead of resolving the install and reading LoraManager's settings files on
every Library, inspect, reveal and serve request. serve_image builds the
allow-list in the executor instead of on the event loop. A finished import
(System job or legacy scan, both through replace_lora_imports) clears the
cache, so files linked under a folder LoraManager just created are served at
once. The integration-off check and the configured-path rule still run before
the cache.

* fix(lora): fingerprint every scanned sidecar, kept or skipped (#362, phase 3)

Discovery keeps every metadata file it saw (scanned_files, resolved and
sorted; scanned is now its length). The fingerprint hashes each one's path,
size, modification time and whether it was kept, so a skipped sidecar added,
removed or edited between the review and Start refuses Start with
plan-changed, as the review's counts promised.

* fix(lora): warn about refused default LoRA folders and log what failed (#362, phase 4)

ComfyUI's default LoRA folders already pass the folder guard; a refused one
that exists is now listed in the layout's refused_defaults, and the layout
cache warns once per newly refused folder. The cache writes one INFO line
when a stored layout differs from the last one logged. BAD_SIDECAR_ROOT and
the module docstring name a folder that contains the home folder. A relative
preview_url writes a DEBUG line before the suffix search; a non-OSError
failure while loading trigger words logs its traceback. Review follow-ups
harden two serve tests.

* fix(lora): skip a sidecar that is not UTF-8 instead of failing the load (#362)

Review follow-up on phase 4: read_lora_metadata caught JSONDecodeError but not
UnicodeDecodeError, so one non-UTF-8 sidecar failed the whole trigger-word
load (now logged with a traceback every retry) and the import review. It is
skipped like any unreadable sidecar. The relative preview_url DEBUG line uses
lazy formatting.

* docs(testing): the LoRA linked-only gate, layout cache and fingerprint scope (#362, phase 5)

TESTING.md's LoRA reference import contract now states what #358 and #362
shipped: the linked-only gate and _media_roots, linked_only media, the cover
link order, the guard on ComfyUI's default LoRA folders and refused_defaults,
the folder-unreadable code, the trigger cache's catch and 60 s retry, the
fingerprint over every scanned sidecar, and the layout cache's key,
signatures, racy window, maximum age, root re-check and clears. The targeted
command includes tests.test_lora_layout_cache and tests.test_api_images.

* fix(lora): a cached layout needs the same home folder; give the rebuild poll its time (#362)

Final review follow-ups: the folder guard compares against the home folder,
so a cache hit now also needs Path.home() unchanged. TESTING.md says the
configured-path rule runs on the cache's result and that a non-UTF-8
sidecar is skipped.

CI: 'Rebuild all is explicit...' (webkit) polled the job for the default
5 s while it regenerated 24 previews; the failure screenshot shows 12 of 24
done at the deadline on the busy runner (0.17 s locally). The poll now
allows 30 s for that real work.
2026-10-27 05:00:55 -07:00
8 changed files with 83 additions and 19 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
{
"required_status_checks": {
"strict": true,
"strict": false,
"contexts": [
"py3.12 on ubuntu-latest",
"py3.13 on ubuntu-latest",
+1
View File
@@ -4,6 +4,7 @@ permissions:
contents: read
on:
merge_group:
push:
branches: [main]
pull_request:
+38 -14
View File
@@ -4,6 +4,7 @@ permissions:
contents: read
on:
merge_group:
pull_request:
branches: [main]
push:
@@ -63,16 +64,18 @@ jobs:
comfyui:
name: real ComfyUI compatibility
runs-on: ubuntu-latest
# Chromium and its system libraries come from the pinned Playwright image (tag = @playwright/test in
# tools/vendor/package-lock.json; tests/vendor/browser-image.test.mjs checks it). The runner install
# pulled them from Ubuntu's apt mirror on every run, and three runs (#349, #378, #382) crawled at
# ~26 kB/s for 11-12 minutes until the job hit its limit after every test had passed.
container:
image: mcr.microsoft.com/playwright:v1.64.1-noble
options: --user 1001
timeout-minutes: 24
steps:
- uses: actions/checkout@v7
with:
persist-credentials: false
- uses: actions/setup-node@v7
with:
node-version: "24"
cache: npm
cache-dependency-path: tools/vendor/package-lock.json
- name: Read the reviewed runtime pins
id: runtime
run: |
@@ -97,21 +100,42 @@ jobs:
ref: ${{ steps.runtime.outputs.kikoRevision }}
path: .ci-kikotools
persist-credentials: false
- uses: actions/setup-python@v7
# The image's own Python and Node 24: setup-python/setup-node install the runner's toolcache builds,
# whose entry points name host paths that do not exist in the container. The image has no ensurepip
# and the job has no root, so a checksum-pinned pip wheel bootstraps a user-space venv. The image
# Python must be the reviewed host Python, so a pin or image bump that disagrees fails here.
- name: Python environment on the reviewed host Python
env:
PIP_WHEEL: https://files.pythonhosted.org/packages/f3/6e/1736e5b4ae2b778ef2f81c47d797de9f891d4d8acb047a24ca37a60294dd/pip-26.2.1-py3-none-any.whl
PIP_SHA256: 71138adf1f4ca900cdb7d289c21b7494329f2332b6d85f0e1c42108c0384ed3e
PINNED_PYTHON: ${{ steps.runtime.outputs.pythonVersion }}
run: |
image="$(python3 -c 'import sys; print("%d.%d" % sys.version_info[:2])')"
[ "$image" = "$PINNED_PYTHON" ] || { echo "::error::image Python $image is not the pinned $PINNED_PYTHON"; exit 1; }
venv="$RUNNER_TEMP/venv" wheel="$RUNNER_TEMP/pip-26.2.1-py3-none-any.whl"
python3 -m venv --without-pip "$venv"
curl -fsSL --retry 3 -o "$wheel" "$PIP_WHEEL"
echo "$PIP_SHA256 $wheel" | sha256sum -c -
"$venv/bin/python" "$wheel/pip" install --no-index --no-cache-dir "$wheel"
echo "$venv/bin" >> "$GITHUB_PATH"
- run: python --version && node --version && node --version | grep -q '^v24\.'
# pip's download cache (torch alone is ~200 MB); inside the container ~ is the job's HOME (/github/home).
- name: Cache pip downloads
uses: actions/cache@v6
with:
python-version: ${{ steps.runtime.outputs.pythonVersion }}
cache: pip
cache-dependency-path: |
requirements-dev.txt
.ci-comfyui/requirements.txt
tools/vendor/comfyui-runtime.json
path: ~/.cache/pip
key: pip-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('requirements-dev.txt', '.ci-comfyui/requirements.txt', 'tools/vendor/comfyui-runtime.json') }}
- name: Install CPU runtime and test dependencies
run: |
python -m pip install --upgrade pip
pip install "torch==${{ steps.runtime.outputs.torchVersion }}" "torchvision==${{ steps.runtime.outputs.torchvisionVersion }}" --index-url https://download.pytorch.org/whl/cpu
pip install -r .ci-comfyui/requirements.txt -r requirements-dev.txt "torch==${{ steps.runtime.outputs.torchVersion }}" "torchvision==${{ steps.runtime.outputs.torchvisionVersion }}"
# npm's download cache, keyed on the lockfile. node_modules itself is still built fresh by npm ci.
- name: Cache npm downloads
uses: actions/cache@v6
with:
path: ~/.npm
key: npm-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('tools/vendor/package-lock.json') }}
- run: npm ci --prefix tools/vendor --ignore-scripts
- run: npx --prefix tools/vendor playwright install --with-deps chromium
- name: Classic and Nodes 2.0 startup and compatibility
env:
PM_COMFY_ROOT: ${{ github.workspace }}/.ci-comfyui
+1
View File
@@ -1,6 +1,7 @@
name: Viewer.js vendor and compatibility
on:
merge_group:
pull_request:
branches: [main]
push:
+2
View File
@@ -356,6 +356,8 @@ class HistoryHooksTests(unittest.TestCase):
self.assertGreaterEqual(time.monotonic() - began, 1.8)
self.assertLess(time.monotonic() - began, CEILING_SECONDS)
release.set()
# Drain first: the 2 s bound is not under test here (#370).
self.drain(registration)
self.assertTrue(registration.dispose(2))
def test_restore_disables_capture_and_clears_handles_before_fresh_epoch(self):
+4
View File
@@ -399,6 +399,10 @@ class OutputHooksTests(fixtures.unittest.TestCase):
self.queue.task_done(1, history("rejected.png"), fixtures.status())
registration.writer.invalidate()
release.set()
# Let the writer work through the 1,024 queued commands first
# (#370's pattern): on a busy Windows runner that took longer
# than dispose's 2 s bound (merge queue, #382).
self.drain(registration)
self.assertTrue(registration.dispose(2))
self.assertEqual(self.outputs(0)["state"], "not_captured")
self.assertEqual(len(self.rows()), 1)
+15 -4
View File
@@ -28,6 +28,11 @@ except ImportError:
# Upper bound a blocked fake write waits for its release. Tests prove ordering
# with events, so this only caps a failing run and never decides a pass.
BLOCK_SECONDS = 10
# A restore is a full verify/migrate/backup/replace cycle and is bound by the
# disk: a busy Windows runner took over 10 s (merge queue, #382). Waits that
# span a restore use this ceiling, which stays under the runner's 120 s hang
# watchdog, so a real hang still fails with the thread dump.
RESTORE_SECONDS = 90
class OutputWriterTests(fixtures.unittest.TestCase):
@@ -317,8 +322,8 @@ class OutputWriterTests(fixtures.unittest.TestCase):
Returns the patch and an event set once that rotation has returned on
the worker. A restore is a full verify/migrate/backup/replace cycle, so
the handoff waits on this event with the generous ``BLOCK_SECONDS``
ceiling instead of the short worker-behavior deadline (TESTING.md,
the handoff waits on this event with the ``RESTORE_SECONDS`` ceiling
instead of the short worker-behavior deadline (TESTING.md,
Windows portability preflight).
"""
backup = os.path.join(self.directory.name, f"restore-{start.run_id}.db")
@@ -361,7 +366,7 @@ class OutputWriterTests(fixtures.unittest.TestCase):
)
with rotation:
self.finalization(writer, start)
self.assertTrue(rotated.wait(BLOCK_SECONDS))
self.assertTrue(rotated.wait(RESTORE_SECONDS))
self.wait_for(lambda: writer.status().pending == 0)
self.assertEqual(
self.db.get_execution_outputs(start.run_id)["state"],
@@ -403,7 +408,7 @@ class OutputWriterTests(fixtures.unittest.TestCase):
self.assertEqual(writer.status().pending, 1)
self.assertEqual(writer.context_budget.charged_bytes, charge)
release.set()
self.assertTrue(rotated.wait(BLOCK_SECONDS))
self.assertTrue(rotated.wait(RESTORE_SECONDS))
self.wait_for(lambda: writer.status().pending == 0)
self.assertEqual(writer.context_budget.charged_bytes, 0)
self.assertFalse(writer.status().accepting)
@@ -567,6 +572,8 @@ class OutputWriterTests(fixtures.unittest.TestCase):
self.assertIs(replacement.context_budget, writer.context_budget)
self.assertIsNone(replacement.context_budget.reserve(8388608))
release.set()
# Drain first: the 2 s bound is not under test here (#370).
drain_writer(writer)
self.assertTrue(writer.close(2))
finally:
release.set()
@@ -608,6 +615,8 @@ class OutputWriterTests(fixtures.unittest.TestCase):
writer.invalidate()
self.assertEqual(writer.context_budget.charged_bytes, 0)
release.set()
# Drain first: the 2 s bound is not under test here (#370).
drain_writer(writer)
self.assertTrue(writer.close(2))
def test_malformed_context_omission_preserves_independent_valid_output(self):
@@ -698,6 +707,8 @@ class OutputWriterTests(fixtures.unittest.TestCase):
release.set()
invalidator.join(BLOCK_SECONDS)
self.assertFalse(invalidator.is_alive())
# Drain first: the 2 s bound is not under test here (#370).
drain_writer(writer)
self.assertTrue(writer.close(2))
self.assertEqual(
self.db.get_execution_outputs(start.run_id)["state"], "complete"
+21
View File
@@ -28,3 +28,24 @@ for (const name of ['frontend-coverage', 'browser-workflows']) {
assert.match(body, /node --version \| grep -q '\^v24\\\.'/, 'the image Node is the Node 24 the project requires');
});
}
// The real-host job drives Chromium against a managed ComfyUI. `playwright install --with-deps` pulled the browser's
// system packages from Ubuntu's apt mirror on every run, and three runs (#349, #378, #382) crawled at ~26 kB/s until
// the job hit its limit after every test had passed. The pinned image ships Chromium and its libraries, so the job runs
// there; the image Python must be the reviewed host Python from tools/vendor/comfyui-runtime.json.
test('real ComfyUI compatibility runs in the pinned Playwright image on the reviewed host Python', () => {
const testWorkflow = readFileSync(new URL('../../.github/workflows/test.yml', import.meta.url), 'utf8');
const runtime = JSON.parse(readFileSync(new URL('../../tools/vendor/comfyui-runtime.json', import.meta.url), 'utf8'));
const match = testWorkflow.match(/^ {2}comfyui:\n([\s\S]*?)(?=^ {2}[a-z][a-z-]*:\n|(?![\s\S]))/m);
assert.ok(match, 'comfyui job exists');
const body = match[1];
assert.match(body, new RegExp(`^ container:\\n image: mcr\\.microsoft\\.com/playwright:v${version.replaceAll('.', '\\.')}-noble$`, 'm'));
assert.doesNotMatch(body, /playwright install/, 'Chromium and its system libraries come from the image, not apt');
assert.doesNotMatch(body, /apt-get|actions\/setup-(python|node)@/);
assert.match(body, /python3 -m venv --without-pip /, 'a user-space venv on the image Python');
assert.match(body, /sha256sum -c/, 'the pip wheel that bootstraps the venv is checksum-pinned');
assert.match(body, /node --version \| grep -q '\^v24\\\.'/, 'the image Node is Node 24');
// The image Python is checked against the pin at run time, so a pin or image bump that disagrees fails loudly.
assert.match(body, /steps\.runtime\.outputs\.pythonVersion/, 'the job compares the image Python with the reviewed pin');
assert.match(runtime.pythonVersion, /^3\.12$/, 'the reviewed host Python is the image Python (3.12 on noble)');
});