a20e59f6744f9b5ec96fe8cd0b2a84dfcee20fcd
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vito
|
d0339128e4 |
test: isolate API fixture databases from other suites; CI lint pins, failing bandit hook, npm cache (#346, #352, #409) (#411)
* ci: pin black and flake8, let the bandit hook fail, cache npm in the JS job (#352) The lint job installs black==26.10.1 and flake8==7.4.1 from requirements-dev.txt (the versions CI resolved on main today), so a new release cannot turn CI red with no code change. The local bandit hook ran 'bash -c ... "$@"; true': it could never fail, and bash bound the first staged file to $0 so it was never scanned. It now runs 'bandit -lll -q' directly, failing on high severity exactly like code-quality.yml. The frontend modules job's setup-node caches npm keyed on tools/vendor/package-lock.json. * test(history): wait on the writer's state, not short wall-clock pauses (#352) Four 0.08 s pauses before asserting that the worker had not probed yet now wait until the idle worker has run a whole loop pass at the current manual clock value (three clock reads), so a slow runner cannot end the pause before the worker decides. A mutation that probes 20 ms early now fails two tests deterministically. The real-clock bounded close no longer asserts close(0.02) returned in under 0.2 s wall time; it asserts the in-flight write is still blocked and the worker alive, like the output writer test. The manual-clock bound stays exact. The last 0.1 s non-blocking submit check uses the 1 s NON_BLOCKING_CEILING from #351. * test(api): give each API fixture database its own temp folder (#346) Every PromptManagerAPI sweeps its database's folder at startup for the stages, uploads and safety-backup copies an interrupted restore left. The API test fixtures created their database directly in the system temp folder, so any other test process starting an API there (another worktree's suite, a pre-push hook, an agent's targeted run) deleted this process's in-flight restore files. The restore, epoch and retirement tests then failed with 'staged restore changed', 'unable to open database file' or 503 instead of 409. Busy machines fail because they are running other suites, not because they are slow: the same CPU load with a private TMPDIR passes. private_temp_db() puts the fixture database in a new folder removed at cleanup (with any safety backups a restore left beside it). A regression test sweeps a stand-in shared folder in the middle of a multipart restore, as another suite's startup would. * test(history): count worker parks, hold saturation blocks until released (#352) wait_for_worker_pass now counts the idle worker's parks on its wake event (two parks from now = one whole pass begun after the call) instead of assuming a fixed number of clock reads per pass, which a future clock read would have made vacuous. The real-clock bounded close keeps a loose upper bound (0.02 s + NON_BLOCKING_CEILING). The two saturation tests (history and output writer) blocked the worker with release.wait(3) while they queued 1023 more items. On a loaded runner the fill took longer than 3 s, the worker unblocked on its own and drained part of the queue (pending 868..1017 != 1024), and cleanup then reported a leaked worker. The block now holds until the test releases it; 60 s is only a hang guard, and cleanup always sets the release. Under one core with 8 busy loops: 8/8 runs failed before, 6/6 pass after. * test(maintenance): wait on duplicate jobs with a disk-bound hang guard (#409) On Windows py3.13 CI (PR #405) a one-file duplicate cleanup missed its 10 s WAIT. The test already waits on the job's own task; the 10 s bound acted as a deadline on disk-bound work. In the same 50 s window the following tests' setups took up to 9 s against 0.1-0.6 s around it while the test bodies stayed fast, which is consistent with a process-wide disk I/O stall on the runner rather than a lease, handle, executor or job stall (the log has no thread dump to show more). tests/disk_waits.py holds the one disk-bound hang guard (90 s per wait, under the 120 s per-test watchdog), shared with RESTORE_SECONDS (#382). Waits on find and cleanup job tasks in the duplicate-files tests use it; gates and server-state polls keep WAIT. The keeper-validation test turns the safety backup off, since it does not test backups. |
||
|
Vito
|
0652beba90 |
ci: run the required workflows for the merge queue (#382)
* ci: run the required workflows for the merge queue main no longer requires a branch to be up to date before it merges, so a PR no longer re-runs CI after every other merge. The Tests, Code Quality and Viewer.js workflows also run on merge_group, so the merge queue can test each PR against the latest main once, in order. The tracked protection record now says strict: false. * test(history): give waits that span a database restore their own ceiling The merge queue's Windows py3.12 job failed test_restore_or_generation_rotation_between_transactions_discards_evidence (restore=True): the restore it waits for, a full verify/migrate/backup/ replace cycle, took over the 10 s BLOCK_SECONDS on a busy runner. The two waits that span a restore now use RESTORE_SECONDS (90 s), which stays under the suite's 120 s hang watchdog, so a real hang still fails with its thread dump. Behavioral waits keep BLOCK_SECONDS. * test(history): drain the writer before every bounded close or dispose The merge queue's Windows py3.12 job failed test_saturated_start_cannot_capture_outputs_or_retain_previous_roots: dispose(2) ran while the writer still worked through 1,024 queued commands on a busy runner. Like #370, the test now drains first; the 2 s bound is not what it tests. Four more closes that follow a released block get the same drain, so the next slow runner can't fail them one at a time. * ci(comfyui): run the real-host job in the pinned Playwright image The real ComfyUI compatibility job installed Chromium with `playwright install --with-deps`, which pulls the browser's system packages from Ubuntu's apt mirror on every run. Three runs (#349 run 40367579020, #378, #382 job 122287230985: "Fetched 19.7 MB in 12min 23s (26.5 kB/s)") spent 11-12 minutes there and were cancelled at the 24-minute limit after all 89 tests had passed. The job now runs inside mcr.microsoft.com/playwright:v1.64.1-noble, the same image as the frontend coverage and page workflow jobs, which ships Chromium and its libraries, so the job has no apt step at all. The whole job moves rather than only the browser part: the Playwright suites launch the managed ComfyUI host themselves (PM_TEST_PYTHON), so splitting the browser off would mean running the host on the runner and the browser in a container across a network boundary, a larger change than moving one job whose host is plain Python and pip wheels. Python follows the other image jobs: the image's own Python 3.12 with a user-space venv bootstrapped from a checksum-pinned pip wheel (toolcache setup-python builds do not run in the container). A step fails the job when the image Python differs from pythonVersion in tools/vendor/comfyui-runtime.json, so the pinned host stays honest; torch, torchvision and ComfyUI's requirements install from the same pins as before. pip and npm download caches replace setup-python's and setup-node's. tests/vendor/browser-image.test.mjs now checks this job like the other browser jobs: image tag equals @playwright/test in the lockfile, no browser install or apt, no setup-python/setup-node, checksum-pinned pip, Node 24, and the runtime Python check. The 24-minute limit is unchanged. |
||
|
Vito
|
b1e7abd380 |
fix(threads): stop and join the prompt cleanup thread, the image worker and the observer (#302, #370) (#372)
* fix(threads): stop and join the prompt cleanup thread, the image worker and the observer (#302, #370) - PromptTracker: the cleanup thread waits on a stop event instead of time.sleep; new shutdown(timeout=2) joins it and returns whether it ended. - ImageGenerationHandler.shutdown(timeout=5) now joins the worker and returns a bool; new close() is final: schedule() refuses and a sealed run's history re-check is dropped, so no worker restarts afterwards. - ImageMonitor.stop_monitoring(timeout=5) closes the handler first, then stops and joins the watchdog observer within one deadline. - Every test that builds a real tracker, handler or monitor ends with its final stop and an assertion, before its folder and databases go. - #370: the wrapper/session history test drains before dispose(); the held-write bound test uses 30 s ceilings instead of tight wall clocks. * test(history): drain the writer before asserting a bounded close it does not test (#370) * fix(threads): a later stop waits again for a worker or observer an earlier stop left running (#302) Review findings on #372: - ImageGenerationHandler keeps the workers a stop signalled that had not ended; every later shutdown/close joins them again, so a second call no longer returns True while the first worker still runs. - ImageMonitor.stop_monitoring keeps an observer or handler it left running and stops and joins it again on the next call. - A closed handler takes no awaiting history path, and a re-check that raced close() is not kept due. - Tests: second stops of a busy worker and a live observer, the closed handler's re-check paths, the handle test asserts the worker ended and stops before it asserts, and a no-new-thread check instead of set equality. * test(browser): the System reset restores every settings value, safety_backup included The frontend coverage shard of #372 failed in workspace-system-tagging: the review read "No safety backup (the policy is off)". The backups spec saves safety_backup=false, and /__test__/system/reset restored config.json but not PromptManagerConfig.SAFETY_BACKUP (nor INFINITE_SCROLL), so a later spec on the same worker server saw the policy off. The reset now captures and restores every non-tagging value POST /settings can change. |
||
|
Vito
|
25e7ae158b |
test(history): isolate restore-rotation subtests and wait on the rotation event (#300)
On a slow Windows runner the restore that the rotation test injects between the writer's transactions outlived wait_for's 3 s worker deadline. The subtest failed, and because every subtest shared the process-wide context budget and left its writer running, the still-held restore cascaded: the next subtest saw the in-flight 621-byte payload as charged, and the late restore rotated the epoch under the last subtest's writer. Each subtest now owns its writer and budget and closes them before the next starts, and the handoff waits on an event set once the rotation returns, with the existing BLOCK_SECONDS ceiling, before the short pending wait. A new test holds a restore between transactions with events and proves the payload is charged only while in flight, released afterwards, and that the writer re-probes and accepts new evidence. |
||
|
Vito
|
5ded3cfd39 |
test(history): prove bounded close with events, not wall-clock bounds (#261)
The real-clock subtest of the bounded-close test asserted close(0.02) returned in under 0.2 s; a slow macOS runner took 0.26 s. The failed subtest then skipped its release and join, so its worker stayed blocked holding a charge on the process-wide SHARED_CONTEXT_BUDGET, and the next subtest saw 621 != 0. Not a production race: close(2) only returns True once the worker has exited and released its leases. Prove boundedness with events instead: close() must return False while the in-flight write is still blocked (it has not left its fake write). Keep the exact upper bound only under the manual clock, where it is deterministic. Release and join each subtest's worker in a finally so a failure cannot leak a charge into the next subtest. The commit-gate test gets the same treatment: submit() and close(0) must return before the blocked invalidation and commit finish, instead of a 0.1 s bound. |
||
|
Vito
|
4ef236aaf7 |
feat(history): persist exact run output references (#214)
* feat(history): define bounded output reference contracts * feat(database): migrate execution output evidence to schema v3 * feat(history): persist immutable run output references * test(history): isolate mocked node gallery monitor * feat(history): capture bounded output descriptors with shared leases * fix(history): authenticate payload owners and release failed captures * feat(history): attach output evidence to exact finalized attempts * test(history): verify run output evidence on real ComfyUI * fix(history): release idle worker payload references * docs(history): clarify defensive lease cleanup * test(history): align output fixtures with native path styles * test(history): separate restore completion from writer drain * test(history): bound batch drains by progress and total time |