Compare commits

..
Author SHA1 Message Date
Vito Sansevero 74df0584cc Merge remote-tracking branch 'origin/main' into fix/362-lora-followups 2026-10-27 04:34:15 -07:00
Vito Sansevero afa796b121 Merge remote-tracking branch 'origin/main' into fix/362-lora-followups 2026-10-27 04:05:38 -07:00
Vito Sansevero 150d1d63df fix(lora): a cached layout needs the same home folder; give the rebuild poll its time (#362)
Final review follow-ups: the folder guard compares against the home folder,
so a cache hit now also needs Path.home() unchanged. TESTING.md says the
configured-path rule runs on the cache's result and that a non-UTF-8
sidecar is skipped.

CI: 'Rebuild all is explicit...' (webkit) polled the job for the default
5 s while it regenerated 24 previews; the failure screenshot shows 12 of 24
done at the deadline on the busy runner (0.17 s locally). The poll now
allows 30 s for that real work.
2026-10-26 23:09:51 -07:00
Vito Sansevero f6898699ba docs(testing): the LoRA linked-only gate, layout cache and fingerprint scope (#362, phase 5)
TESTING.md's LoRA reference import contract now states what #358 and #362
shipped: the linked-only gate and _media_roots, linked_only media, the cover
link order, the guard on ComfyUI's default LoRA folders and refused_defaults,
the folder-unreadable code, the trigger cache's catch and 60 s retry, the
fingerprint over every scanned sidecar, and the layout cache's key,
signatures, racy window, maximum age, root re-check and clears. The targeted
command includes tests.test_lora_layout_cache and tests.test_api_images.
2026-10-26 23:01:01 -07:00
Vito Sansevero ee8672a26c fix(lora): skip a sidecar that is not UTF-8 instead of failing the load (#362)
Review follow-up on phase 4: read_lora_metadata caught JSONDecodeError but not
UnicodeDecodeError, so one non-UTF-8 sidecar failed the whole trigger-word
load (now logged with a traceback every retry) and the import review. It is
skipped like any unreadable sidecar. The relative preview_url DEBUG line uses
lazy formatting.
2026-10-26 23:00:59 -07:00
Vito Sansevero f020f1d751 fix(lora): warn about refused default LoRA folders and log what failed (#362, phase 4)
ComfyUI's default LoRA folders already pass the folder guard; a refused one
that exists is now listed in the layout's refused_defaults, and the layout
cache warns once per newly refused folder. The cache writes one INFO line
when a stored layout differs from the last one logged. BAD_SIDECAR_ROOT and
the module docstring name a folder that contains the home folder. A relative
preview_url writes a DEBUG line before the suffix search; a non-OSError
failure while loading trigger words logs its traceback. Review follow-ups
harden two serve tests.
2026-10-26 22:59:07 -07:00
Vito Sansevero 5b0468eaee fix(lora): fingerprint every scanned sidecar, kept or skipped (#362, phase 3)
Discovery keeps every metadata file it saw (scanned_files, resolved and
sorted; scanned is now its length). The fingerprint hashes each one's path,
size, modification time and whether it was kept, so a skipped sidecar added,
removed or edited between the review and Start refuses Start with
plan-changed, as the review's counts promised.
2026-10-26 22:47:13 -07:00
Vito Sansevero 0e28fd8c37 perf(lora): serve the image hot paths from the layout cache (#362, phase 2)
_lora_image_dirs takes (install, layout) from lora_utils.cached_lora_manager_layout
instead of resolving the install and reading LoraManager's settings files on
every Library, inspect, reveal and serve request. serve_image builds the
allow-list in the executor instead of on the event loop. A finished import
(System job or legacy scan, both through replace_lora_imports) clears the
cache, so files linked under a folder LoraManager just created are served at
once. The integration-off check and the configured-path rule still run before
the cache.
2026-10-26 22:47:12 -07:00
Vito Sansevero 88e44bb068 fix(lora): treat an offline Windows share as unvalidated and make cache tests portable (#362)
Review findings on phase 1: Windows raises FileNotFoundError with winerror
53/67 for an offline share, which is a temporary failure, not a missing file.
Test times are whole milliseconds (NTFS keeps 100 ns steps) and a file system
that rounds them skips; cleanup no longer ignores errors; tests added for
ValueError, NotADirectoryError, a failed reload, resolve failures and the
watched-file set.
2026-10-26 22:46:53 -07:00
Vito Sansevero 9a03766090 feat(lora): cache LoraManager's layout behind file signatures (#362, phase 1)
A new py/lora_layout_cache.py keeps one (install, layout) entry and reuses it
while the configured path, the plugin settings file's and the settings file's
signatures (st_mtime_ns, st_size), the install and every media root's real
path are unchanged and the entry is younger than 30 s. A load whose watched
file changed within 3 s (git's racy rule), could not be checked, or whose
settings were unreadable serves its request only. clear() bumps a generation
so a load that started earlier stores nothing. Hits take no lock.

Not wired yet; phase 2 puts it behind the image hot paths.
2026-10-26 22:36:15 -07:00
8 changed files with 19 additions and 83 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
{
"required_status_checks": {
"strict": false,
"strict": true,
"contexts": [
"py3.12 on ubuntu-latest",
"py3.13 on ubuntu-latest",
-1
View File
@@ -4,7 +4,6 @@ permissions:
contents: read
on:
merge_group:
push:
branches: [main]
pull_request:
+14 -38
View File
@@ -4,7 +4,6 @@ permissions:
contents: read
on:
merge_group:
pull_request:
branches: [main]
push:
@@ -64,18 +63,16 @@ jobs:
comfyui:
name: real ComfyUI compatibility
runs-on: ubuntu-latest
# Chromium and its system libraries come from the pinned Playwright image (tag = @playwright/test in
# tools/vendor/package-lock.json; tests/vendor/browser-image.test.mjs checks it). The runner install
# pulled them from Ubuntu's apt mirror on every run, and three runs (#349, #378, #382) crawled at
# ~26 kB/s for 11-12 minutes until the job hit its limit after every test had passed.
container:
image: mcr.microsoft.com/playwright:v1.64.1-noble
options: --user 1001
timeout-minutes: 24
steps:
- uses: actions/checkout@v7
with:
persist-credentials: false
- uses: actions/setup-node@v7
with:
node-version: "24"
cache: npm
cache-dependency-path: tools/vendor/package-lock.json
- name: Read the reviewed runtime pins
id: runtime
run: |
@@ -100,42 +97,21 @@ jobs:
ref: ${{ steps.runtime.outputs.kikoRevision }}
path: .ci-kikotools
persist-credentials: false
# The image's own Python and Node 24: setup-python/setup-node install the runner's toolcache builds,
# whose entry points name host paths that do not exist in the container. The image has no ensurepip
# and the job has no root, so a checksum-pinned pip wheel bootstraps a user-space venv. The image
# Python must be the reviewed host Python, so a pin or image bump that disagrees fails here.
- name: Python environment on the reviewed host Python
env:
PIP_WHEEL: https://files.pythonhosted.org/packages/f3/6e/1736e5b4ae2b778ef2f81c47d797de9f891d4d8acb047a24ca37a60294dd/pip-26.2.1-py3-none-any.whl
PIP_SHA256: 71138adf1f4ca900cdb7d289c21b7494329f2332b6d85f0e1c42108c0384ed3e
PINNED_PYTHON: ${{ steps.runtime.outputs.pythonVersion }}
run: |
image="$(python3 -c 'import sys; print("%d.%d" % sys.version_info[:2])')"
[ "$image" = "$PINNED_PYTHON" ] || { echo "::error::image Python $image is not the pinned $PINNED_PYTHON"; exit 1; }
venv="$RUNNER_TEMP/venv" wheel="$RUNNER_TEMP/pip-26.2.1-py3-none-any.whl"
python3 -m venv --without-pip "$venv"
curl -fsSL --retry 3 -o "$wheel" "$PIP_WHEEL"
echo "$PIP_SHA256 $wheel" | sha256sum -c -
"$venv/bin/python" "$wheel/pip" install --no-index --no-cache-dir "$wheel"
echo "$venv/bin" >> "$GITHUB_PATH"
- run: python --version && node --version && node --version | grep -q '^v24\.'
# pip's download cache (torch alone is ~200 MB); inside the container ~ is the job's HOME (/github/home).
- name: Cache pip downloads
uses: actions/cache@v6
- uses: actions/setup-python@v7
with:
path: ~/.cache/pip
key: pip-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('requirements-dev.txt', '.ci-comfyui/requirements.txt', 'tools/vendor/comfyui-runtime.json') }}
python-version: ${{ steps.runtime.outputs.pythonVersion }}
cache: pip
cache-dependency-path: |
requirements-dev.txt
.ci-comfyui/requirements.txt
tools/vendor/comfyui-runtime.json
- name: Install CPU runtime and test dependencies
run: |
python -m pip install --upgrade pip
pip install "torch==${{ steps.runtime.outputs.torchVersion }}" "torchvision==${{ steps.runtime.outputs.torchvisionVersion }}" --index-url https://download.pytorch.org/whl/cpu
pip install -r .ci-comfyui/requirements.txt -r requirements-dev.txt "torch==${{ steps.runtime.outputs.torchVersion }}" "torchvision==${{ steps.runtime.outputs.torchvisionVersion }}"
# npm's download cache, keyed on the lockfile. node_modules itself is still built fresh by npm ci.
- name: Cache npm downloads
uses: actions/cache@v6
with:
path: ~/.npm
key: npm-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('tools/vendor/package-lock.json') }}
- run: npm ci --prefix tools/vendor --ignore-scripts
- run: npx --prefix tools/vendor playwright install --with-deps chromium
- name: Classic and Nodes 2.0 startup and compatibility
env:
PM_COMFY_ROOT: ${{ github.workspace }}/.ci-comfyui
-1
View File
@@ -1,7 +1,6 @@
name: Viewer.js vendor and compatibility
on:
merge_group:
pull_request:
branches: [main]
push:
-2
View File
@@ -356,8 +356,6 @@ class HistoryHooksTests(unittest.TestCase):
self.assertGreaterEqual(time.monotonic() - began, 1.8)
self.assertLess(time.monotonic() - began, CEILING_SECONDS)
release.set()
# Drain first: the 2 s bound is not under test here (#370).
self.drain(registration)
self.assertTrue(registration.dispose(2))
def test_restore_disables_capture_and_clears_handles_before_fresh_epoch(self):
-4
View File
@@ -399,10 +399,6 @@ class OutputHooksTests(fixtures.unittest.TestCase):
self.queue.task_done(1, history("rejected.png"), fixtures.status())
registration.writer.invalidate()
release.set()
# Let the writer work through the 1,024 queued commands first
# (#370's pattern): on a busy Windows runner that took longer
# than dispose's 2 s bound (merge queue, #382).
self.drain(registration)
self.assertTrue(registration.dispose(2))
self.assertEqual(self.outputs(0)["state"], "not_captured")
self.assertEqual(len(self.rows()), 1)
+4 -15
View File
@@ -28,11 +28,6 @@ except ImportError:
# Upper bound a blocked fake write waits for its release. Tests prove ordering
# with events, so this only caps a failing run and never decides a pass.
BLOCK_SECONDS = 10
# A restore is a full verify/migrate/backup/replace cycle and is bound by the
# disk: a busy Windows runner took over 10 s (merge queue, #382). Waits that
# span a restore use this ceiling, which stays under the runner's 120 s hang
# watchdog, so a real hang still fails with the thread dump.
RESTORE_SECONDS = 90
class OutputWriterTests(fixtures.unittest.TestCase):
@@ -322,8 +317,8 @@ class OutputWriterTests(fixtures.unittest.TestCase):
Returns the patch and an event set once that rotation has returned on
the worker. A restore is a full verify/migrate/backup/replace cycle, so
the handoff waits on this event with the ``RESTORE_SECONDS`` ceiling
instead of the short worker-behavior deadline (TESTING.md,
the handoff waits on this event with the generous ``BLOCK_SECONDS``
ceiling instead of the short worker-behavior deadline (TESTING.md,
Windows portability preflight).
"""
backup = os.path.join(self.directory.name, f"restore-{start.run_id}.db")
@@ -366,7 +361,7 @@ class OutputWriterTests(fixtures.unittest.TestCase):
)
with rotation:
self.finalization(writer, start)
self.assertTrue(rotated.wait(RESTORE_SECONDS))
self.assertTrue(rotated.wait(BLOCK_SECONDS))
self.wait_for(lambda: writer.status().pending == 0)
self.assertEqual(
self.db.get_execution_outputs(start.run_id)["state"],
@@ -408,7 +403,7 @@ class OutputWriterTests(fixtures.unittest.TestCase):
self.assertEqual(writer.status().pending, 1)
self.assertEqual(writer.context_budget.charged_bytes, charge)
release.set()
self.assertTrue(rotated.wait(RESTORE_SECONDS))
self.assertTrue(rotated.wait(BLOCK_SECONDS))
self.wait_for(lambda: writer.status().pending == 0)
self.assertEqual(writer.context_budget.charged_bytes, 0)
self.assertFalse(writer.status().accepting)
@@ -572,8 +567,6 @@ class OutputWriterTests(fixtures.unittest.TestCase):
self.assertIs(replacement.context_budget, writer.context_budget)
self.assertIsNone(replacement.context_budget.reserve(8388608))
release.set()
# Drain first: the 2 s bound is not under test here (#370).
drain_writer(writer)
self.assertTrue(writer.close(2))
finally:
release.set()
@@ -615,8 +608,6 @@ class OutputWriterTests(fixtures.unittest.TestCase):
writer.invalidate()
self.assertEqual(writer.context_budget.charged_bytes, 0)
release.set()
# Drain first: the 2 s bound is not under test here (#370).
drain_writer(writer)
self.assertTrue(writer.close(2))
def test_malformed_context_omission_preserves_independent_valid_output(self):
@@ -707,8 +698,6 @@ class OutputWriterTests(fixtures.unittest.TestCase):
release.set()
invalidator.join(BLOCK_SECONDS)
self.assertFalse(invalidator.is_alive())
# Drain first: the 2 s bound is not under test here (#370).
drain_writer(writer)
self.assertTrue(writer.close(2))
self.assertEqual(
self.db.get_execution_outputs(start.run_id)["state"], "complete"
-21
View File
@@ -28,24 +28,3 @@ for (const name of ['frontend-coverage', 'browser-workflows']) {
assert.match(body, /node --version \| grep -q '\^v24\\\.'/, 'the image Node is the Node 24 the project requires');
});
}
// The real-host job drives Chromium against a managed ComfyUI. `playwright install --with-deps` pulled the browser's
// system packages from Ubuntu's apt mirror on every run, and three runs (#349, #378, #382) crawled at ~26 kB/s until
// the job hit its limit after every test had passed. The pinned image ships Chromium and its libraries, so the job runs
// there; the image Python must be the reviewed host Python from tools/vendor/comfyui-runtime.json.
test('real ComfyUI compatibility runs in the pinned Playwright image on the reviewed host Python', () => {
const testWorkflow = readFileSync(new URL('../../.github/workflows/test.yml', import.meta.url), 'utf8');
const runtime = JSON.parse(readFileSync(new URL('../../tools/vendor/comfyui-runtime.json', import.meta.url), 'utf8'));
const match = testWorkflow.match(/^ {2}comfyui:\n([\s\S]*?)(?=^ {2}[a-z][a-z-]*:\n|(?![\s\S]))/m);
assert.ok(match, 'comfyui job exists');
const body = match[1];
assert.match(body, new RegExp(`^ container:\\n image: mcr\\.microsoft\\.com/playwright:v${version.replaceAll('.', '\\.')}-noble$`, 'm'));
assert.doesNotMatch(body, /playwright install/, 'Chromium and its system libraries come from the image, not apt');
assert.doesNotMatch(body, /apt-get|actions\/setup-(python|node)@/);
assert.match(body, /python3 -m venv --without-pip /, 'a user-space venv on the image Python');
assert.match(body, /sha256sum -c/, 'the pip wheel that bootstraps the venv is checksum-pinned');
assert.match(body, /node --version \| grep -q '\^v24\\\.'/, 'the image Node is Node 24');
// The image Python is checked against the pin at run time, so a pin or image bump that disagrees fails loudly.
assert.match(body, /steps\.runtime\.outputs\.pythonVersion/, 'the job compares the image Python with the reviewed pin');
assert.match(runtime.pythonVersion, /^3\.12$/, 'the reviewed host Python is the image Python (3.12 on noble)');
});