The round-4 fused crop loops re-derived each frame's detection image with
one single-row converter call per frame - N host sync points where the
round-3 crop did one batch. The grouped form converts the chunk's gathered
rows in one elementwise pass (bitwise the per-row rows) and is bounded by
the caller's chunk; the paste-fallback row form stays. Pins guard the
conversion count. The paste stage itself is proven byte-equivalent to
32e6628 (dispatch, cores, tails); timed CPU stubs show no superlinear
term in either tree - the real-box burn needs the owner's log to attribute
(the device-mode verdict under real VRAM pressure is the only paste-stage
quantity this diff can move).
- R1: the face helper is constructed on the pack device (was the offload
device); load_device() explicitly places the detector and the parser with
a logged CPU fallback, offload() mirrors the restore, the YOLO .device
skew is closed (P-D1..P-D5).
- R2: detection streams per max_clip_length group (no full-clip convert-in
retention), crop is fused into a chunk-bounded restore loop with
preallocated staging, the paste tail keeps one contractual float32
buffer, the helper is released after every run; golden fixtures pin the
outputs bitwise (P-M1..P-M7).
- The diagnosis: the OOM was O(frames) whole-clip host buffers, not the
faces (the detector/landmark path was already GPU-placed);
max_clip_length never bounded convert-in or the paste tail.
- Gate: 523 passed, 6 skipped, 7 deselected, 0 failed (baseline 505).
Two owner-reported bugs, both verified as gaps that never closed (the old
code on master had them too).
Cancel: the node never polled ComfyUI's interrupt flag, so the Cancel
button had no effect. A new src/keep_interrupt.py wraps
comfy.model_management.throw_exception_if_processing_interrupted() (lazy
import, a no-op outside ComfyUI); it is called at every long-loop
boundary - detection, crop, restore chunks, paste frames, and the paste
module's group/chunk loops. InterruptProcessingException is a
BaseException, so it escapes the pipeline untouched.
Multi-face: the pipeline collapsed to one face per frame - the detector
call passed only_keep_largest=True (the vendored selector reduces to the
largest face) and the processor then took all_landmarks_5[0]. With
only_center_face=False the new branches keep every detected face:
detection returns all faces, the crops/restore/paste flatten to the
ragged per-frame face count, the device canvas composites several ROIs
per row, and the vendored fallback hands the helper all of a frame's
faces. The near-black tail follows the real vendored reference (one
convert per image after a forward face loop). only_center_face=True and
the aligned path keep their bodies byte-for-byte.
44 new pins: cancel (the exception escapes the sequence and the node, no
work after it, unchanged output without a cancel), multi-face image and
video (all faces restored and pasted, faces appearing/disappearing,
None-affine and black-face drops, per-row near-black tail against the
vendored composite), and the paste core (face-to-frame map, row
compositing, chunk boundaries that never split a frame's faces).
Verified on the CPU device only - no model loads, no GPU runs. Gate: 505
passed, 6 skipped, 7 deselected, 0 failed (461 baseline + 44 new pins).
The round-1 device canvas disengaged past its fixed 1 GiB budget, so at
final_upscale_factor=4 (~44 MB/frame at 720p) typical clips fell back to
the host assembly at 16x the pixels. The engagement is now three-way:
- device_canvas_budget: a share of the free VRAM (0.25) under the hard
cap, the cap when free memory is unavailable - the mirror of
paste_chunk_size.
- device_output_mode: full | chunked | host. The chunked mode assembles
frame groups on the device (the same group size the full mode derives
internally, so the batch compositions coincide) and downloads the
per-group convert-outs into one preallocated host IMAGE buffer - the
device footprint stays bounded for any clip length.
- comfy_convert_out: the BGR->RGB float convert-out runs on the device
in budget-sized sub-chunks (the out-of-place chain peaks at ~9x the
canvas bytes of transient), so the budget bounds what it gates.
- a "host" verdict on a CUDA device is logged - the silent fallback
that hid the regression must not be silent again.
Also: the paste loop's NaN-landmark short-circuit never recorded its
frame in fallback_indices - in a device-output mode the tail's indexed
assignment length-mismatched (pinned red before the fix; the branch is
defensive today, the production interpolation fills NaN landmarks).
8 new bit-exactness pins (budget, selector, convert-out sub-chunking,
factor-4-scale, chunked assembly incl. group-boundary independence,
sentinels, end-to-end modes); the instrument gains the factor-4-scale
output-mode scenario. Verified on the CPU device only - no GPU run (the
CUDA before/after stays owner-gated). Gate: 461 passed, 6 skipped,
7 deselected, 0 failed.
The instrument now times the same paste_faces_batch call in the host and
the device output mode, one session, one fixture, and asserts the two
are bit-identical. On the CPU fixture (8 frames, 360x640, stub nets):
the output assembly's host<->device traffic is gone (1.06 MB each way
becomes 0, the ROI-sized staging recorded as roi_staged_bytes), wall
-8.8%, process CPU -8.7%. The delta dilutes the CUDA effect - on the
owner's GPU the kernel work stays on the GPU and the removed portion is
pure host time - so the CUDA measurement of the same comparison stays
the owner-gated V2.
A self-review of the device-output canvas found the rare near-black
branch calling the numpy _background_max_outside with a device tensor -
it happened to work through the numpy/torch API overlap, but it made an
undocumented per-frame device read inside the branch and would break on
a torch-only API change. The branch is rare by construction, so this one
frame is downloaded explicitly, exactly like the full-canvas fallback
next to it.
Also adds the device-output pin of the near-black branch itself
(test_the_device_canvas_keeps_the_near_black_full_canvas_fallback): the
rare path now runs in the canvas mode in the gate instead of only in
production.
The paste-stage attribution (tools/prof_paste_host_cpu.py, the CPU-device
stub run) showed the batch always takes the device path - the vendored
per-frame fallback never runs in the default configuration - and that the
remaining host cost of the step is the output assembly: the per-frame ROI
download, the whole-canvas host copy, the whole-stack np.stack and the
float math of the convert-out, all of which exist only because the output
stack is assembled in host memory.
F1 keeps the stack on the device: batched_bg_frames(return_device=True),
paste_faces_batch(device_output=True) scatters the blended ROI patches
into one uint8 device canvas, and the paste step assembles the batch with
one indexed copy, one stacked upload of the vendored-fallback frames and
a single bgr_u8_batch_to_comfy on the device. Nothing is downloaded or
copied per frame. The mode is off on a CPU device, over the 1 GiB
PASTE_DEVICE_STACK_MAX_BYTES budget and when nothing is eligible, so the
host path - and every existing pin - runs byte-for-byte as before; the
bytes that would have travelled are kept as roi_staged_bytes.
The new pin test_the_device_output_stack_is_bit_identical_to_the_host_stack
pins the canvas against the host stack on a CPU device. Gate: 451 passed,
6 skipped, 7 deselected.
The owner's route decision (the tolerance-accepted GPU resample):
resize_lanczos_u8 implements PIL's three-lobe Lanczos with the
scale-aware support and the drop-and-renormalize edge rule as dense
axis matrices + matmul, chunked by LANCZOS_CHUNK_PIXELS pixels.
batched_bg_frames keeps the result of _tiled_scale_batch on the device,
resizes with the new op when CUDA and downloads once at the end; a CPU
device keeps the per-frame cv2 loop (pixel-identical there - the
equivalence pin still holds exactly).
Pinned in test_image_op_equivalence.py against PIL (the semantic
reference: mean <= 0.3, max <= 12) and against cv2 (the accepted drift:
upscale mean <= 1.0 / max <= 8, downscale mean <= 8.5 / max <= 54),
plus chunk transparency and a GPU-vs-CPU pin; the profiler wraps the
op as p6_bg_resize.
The paste stage's largest remaining host-side item was the gray flag: one numpy variance pass per frame (41 ms of process CPU per frame, 4.7 cores). On a CUDA device the three channel-difference variances now run on the device in float64 (the vendored int16 differences, chunked to bound the transient), and only the frames whose result lands on the threshold fall back to the host computation - so the flags are the host path's flags, not a tolerance.
A CPU device and a caller without a device keep the numpy computation unchanged. Pinned in tests/test_is_gray_device.py: random, flat and saturated frames, a constructed borderline frame that actually hits the threshold, the CPU device, the empty batch, a bad shape, and both processor call sites.
The probe now measures the paste stage's sub-stages (the wall time, the process CPU time and the core count of every step) and counts the host-device copies of the paste, so a change of the paste can be judged on the CPU load it leaves, not on the wall time alone.
The paste stage spent its host time on three full-canvas passes per frame. The face warp now runs on the device with cv2's own fixed-point convention (the matrix inversion of warpAffine, the 1/1024 pre-scale, the 1/32 tap grid, the (+512)>>10 rounding), the mask warp reuses the one batched grid sliced to the face ROI, and the blend and the quantizing tail run inside that ROI. The frame-global maximum branch is kept: a near-black frame falls back to the full-canvas path, which stays bit-exact by construction.
Measured over 48 frames at 720p on CUDA: the paste call's process CPU time falls 20% (38.74 -> 30.92 ms/frame), the host-device transfers shrink 6.66x (the blended canvas download by 85%), and the result is bit-identical to the full-canvas path on the smooth and the noise corpus, at interior and edge-touching ROIs.
Times every stage of process_image_sequence on synthetic frames with a same-session control clone of the base implementation, plus micro benches for the parse chunk sweep and the KEEP-stage allocator question. The evidence behind the redesign ranking and the before/after tables of the internal design review.
process_image_sequence converts the batch once, detects per frame while capturing the frames the detector saw (the read_image products, including the min(h,w)<512 upscale), estimates each affine once with the vendored call, crops with cv2.warpAffine (decision D2: the uint8-bound warp stays bit-exact; the batched grid_sample helper stays parked and pinned), normalizes the whole crop batch on the device, keeps the KEEP chunk contract byte-identical, converts the restored faces with one batched tensor2img, pastes through the batched paste with the vendored fallback, and stacks the output in one transfer. The paste canvas is the detection frame, so small frames keep the base's upscaled canvas (the round-1 QA defect F1, fixed and pinned). A failed affine degrades to the neutral crop with a warning instead of crashing like the base. Progress still reports num_frames * 4 in the four families. Pins: the read-once/affine-once/chunk-contract AST guards, the progress contract, and the new end-to-end base-vs-shipped suite (tests/test_gpu_sequence_e2e.py) with the real vendored helper - measured drift 0/255 on the vendored branch, 1/255 on the batched branch.
New src/keep_gpu_paste.py: the chunked ParseNet forward (chunk cap 8, the measured VRAM sweet spot), the batched parse-mask postprocess, the batched blend with the vendored quantizing tail, one tiled_scale call for the whole background group, is_gray without the PIL round trip, and VRAM-aware chunk sizes from model_management.get_free_memory. The frames the batched paste cannot handle (draw_box, face upsampler, grayscale, no parse net, no face) keep the vendored paste_faces_to_input_image. Equivalence against the real vendored paste: <=1/255 on the parse mask, blend and tail bit-exact.
New src/keep_gpu_ops.py: affine capture with the vendored cv2 calls, batched crop/inverse warps (grid_sample, in-bounds compositing of the gray border, uint8 requantization), a batched form of the vendored tensor2img chain, the ParseNet colormap/blur/postprocess, and the batched ComfyUI converters in keep_image_utils.py (one host transfer per batch; the /255 reproduces img2tensor's double rounding bit for bit). Every op carries a measured-tolerance pin in tests/test_gpu_batch_equivalence.py; the uint8-bound warps measure up to 8/255 against cv2's fixed-point sampling on adversarial content - the numbers that drive the tolerance decision D2.
The host's requirements.txt provides torch, torchvision, numpy, einops, pyyaml, Pillow, scipy, tqdm and requests; the pack now declares only opencv-python-headless - the vendored architectures and the four LANCZOS4 resizes need cv2, and no GUI call exists anywhere in the pack or the vendored tree, so the headless wheel suffices (no libGL/Qt in a server install). requests was a phantom: only the vendored basicsr download helper imports it. The dependency guards move to the host-guarantee policy (closure coverage = declared + host-provided, pinned to the host's requirement lines, exhaustive in both directions).
download_weight's progress flag becomes real: one guarded comfy.utils.ProgressBar step per completed download (a failed one stays at 0), lazy import so the engines load without a server. The torch.hub transport stays bar-free - its own bar is untagged stderr noise.
The channel conversions, the INTER_LINEAR resizes, the GRAY2BGR lift and the temporal landmark smoothing (the former gaussian_filter1d) become torch-native helpers; tensors keep their device and the host copy happens once, as uint8. The four LANCZOS4 resizes keep cv2 (torch has no lanczos) - that is why OpenCV stays declared. Equivalence is pinned in tests/test_image_op_equivalence.py: conversions exact, bilinear resize within 1/255 of cv2 over a 20-geometry matrix, smoothing within 1e-10 (float64 measured ~4e-14, float32 bit-equal).
The loader of ComfyUI logs one line per yaml entry. The host already printed that block at its startup, so the pack raises the level of the root logger while it asks the host to read the file again - the block is not printed a second time.
ComfyUI reads the default extra_model_paths.yaml and every file of --extra-model-paths-config (main.py:142-144). The pack now follows the same list, so a user who supplies the extra roots only on the command line gets the layout convention as well and the checkpoint is found where it is instead of being downloaded again.
The hash mismatch path deleted the file it found - and since the extra model roots are resolved now, that file can live in the original directory of the user. The download is atomic now (the transport writes into a temporary file that is only put in place after it was verified), a file outside the primary root of the installation is never deleted or overwritten, and the co-location of the face weights keeps a file of the user as a backup instead of overwriting it.
Per-defect regression guards (the extra model roots, the logger of ComfyUI, the KEEP tag, the routed console output, the selective download) plus the host modules app and utils in the lazy-import guard.
The loader resolved every facedetection entry of the manifest, so all four detectors were fetched. The selection now drives the download set: the chosen detector and the parsing net the helper needs, co-located in one root because the vendored helper builds its weight paths from a single root.
The download target was always the primary root, so a checkpoint that already existed in the original model directory was downloaded again into the dev instance. The extra roots of extra_model_paths.yaml are now registered for the folders of the pack (the explicit keys through the loader of ComfyUI itself, the layout convention base_path/models/folder), and a missing weight is downloaded next to the other weights of the pack.
The pack owned a private stdlib handler, so every line bypassed the coloured console of ComfyUI, and the call sites carried a literal KEEP prefix. The logger now adopts the host console (no handler and propagate=True when ComfyUI is present, one private handler when it is not), the tag lives in a single filter, and the console writes of the pack (the tqdm bars of the processor, the prints of the vendored facelib) reach it through a thread-scoped redirect.
The tree fragment is now matched as a path segment, so an identifier that
merely carries the same letters (load_device, __device__) cannot trip the
check, and the POSIX home roots are checked outside the vendored upstream
tree. Two self-tests prove the tripwire catches a POSIX home path and does
not fire on device identifiers.
The tracked files still pointed at the development documents, which now live
in the untracked .dev tree: the usage headings promised the embedded
interpreter while the examples are interpreter agnostic, and the provenance
notes named a local document. The headings are reworded and the provenance
notes name the tool that produces the record instead.
The test asserts on the stub that records the progress calls. With
KEEP_NO_COMFY_MOCKS=1 the node uses the real comfy_api.latest API, whose
execution recorder has no calls attribute, so the test failed in the
mocks-off configuration (17 passed, 1 failed). It now skips there, with the
reason naming the configuration. The gate and the headless tier are
unchanged and green.
A tracked file must not name a local drive, a local tree or the local
interpreter, and the internal documents must not be in the index. The test
scans the tracked file list, builds its forbidden fragments from parts so it
does not trip itself, and skips binary files.
The tools named a local drive, a local interpreter and local ComfyUI trees.
They now take the tree from the command line or from the environment only -
a machine-local default would make a scan run against the wrong tree - and
the usage examples are interpreter agnostic. The exported example workflow
no longer carries the absolute path of the machine it was exported on.
The audit artifacts and the plan moved to the local .dev/docs tree, so the
references in the code, the tools and the tests follow them there. The
license file names the tool that produces the record instead of a path.
The development documents (the modernization plan, the phase reviews, the
audit artifacts, the status reports) are local working material: they live
under .dev/docs and must never be committed. docs/ leaves the index and
.gitignore keeps both trees out; the .dev pattern is anchored without a
trailing slash so it matches the .dev symlink itself.
The headless tier runs with the ComfyUI mocks switched off
(KEEP_NO_COMFY_MOCKS=1), but test_the_mock_exposes_no_symbol_the_real_api_lacks
read the mock out of sys.modules - where it is not installed in that
configuration - and died with KeyError: 'comfy_api.latest'.
The mock construction is now a module-level function (build_comfy_api_mocks)
that _install_comfy_mocks uses, so the test can build the mock itself when
the mocks are not installed. The comparison against the real ComfyUI API now
runs in the headless tier as well.
The development artifacts under docs/ are not shipped with the pack and may
be absent from a working tree (the docs folder is gitignored). The tests
that assert their content now skip with a message naming the missing
artifact instead of failing. The README no longer points at an audit file
that is not part of the package.
ComfyUI runs a V3 node's validate_inputs during the prompt validation, where
the links are not resolved yet: a connected pack arrives as None (after the
list unwrapping) or as the (None,) marker of a missing link. Both nodes
rejected that unresolved pack, so every prompt failed with the same error
once per input.
- validate_inputs now accepts an unresolved pack and only refuses a value
that is present but of the wrong type.
- execute raises the actionable error for a pack that is really wrong
(the plan already requires execute to raise instead of returning None).
- the shared message lives in src/keep_model_loader.keep_model_pack_error.
- tests cover the unresolved pack (None and (None,)) and the raising execute.
Verified against the real ComfyUI validation path: execution.get_input_data
with a linked pack returns (None,) and validate_inputs now returns True.
The five review findings of 2026-10-24 for phase P6:
1. Gate 2 (`ruff check .`) was RED on the P6 tree: F841 - the local `size`
of `_anchor_in` in tools/render_workflow_preview.py:95 was assigned but
never used, so the lint job of .github/workflows/tests.yml would have
gone red on this branch. The unused local is gone; the render output is
unchanged (the regenerated png has the same sha256 as the committed
one). The gate was recorded as "not runnable" although ruff IS
installed on this machine (0.15.22; the CI pin 0.8.6 reproduces it via
`uvx ruff@0.8.6`) - the embedded env is only one of the environments.
The gate records (audit D6, the progress doc, the status doc, the plan)
now state the truth.
2. The per-node parameter tables of the README were not checked by any
test, although the README calls the widget names, the defaults, the
ranges and the order compatibility guarantees. tools/docs_tables.py
renders what a V3 schema says (documented_parameters) and
test_the_parameter_tables_of_the_readme_match_the_node_schemas compares
the README with it. The check is mutation-checked: a wrong
`max_clip_length` default/range, a wrong `final_upscale_factor` range,
a wrong `only_center_face` default and two swapped widget rows each
fail the test (the values in the README are correct - the gap was the
coverage).
3. tools/docs_tables.py carried REPO_FILE_EXTENSIONS and
EXTERNAL_PATH_ROOTS without using them while the test re-defined them.
The test imports both from tools/docs_tables.py now - one copy, no
drift between the generator and the checker.
4. The rendered preview of the sequence workflow was never shown in the
README (only the shipped workflow's preview was). The README shows
example_workflows/keep-sequence-workflow.png in the Example workflows
section, captioned as a rendered graph diagram and not a screenshot.
5. The committed progress doc called its own review file untracked and
"stays uncommitted" although it is committed (0066001, 185c289). The
gate-7 row names `.dev` as the only untracked path now.
Checks (run on 2026-10-24): ruff check . -> All checks passed! (0.15.22)
and uvx ruff@0.8.6 check . -> All checks passed! (the version CI pins);
pytest -q -m "unit or mock_integration or headless" -> 262 passed, 6
skipped, 5 deselected; pytest -q tests/test_structure.py -> 21 passed;
pytest -q tests/test_example_workflows.py tests/test_docs_claims.py
tests/test_examples_dir.py -> 29 passed; pytest -q -m gpu_acceptance -rs
-> 5 skipped (opt-in); the headless tier against the real ComfyUI source
-> 4 passed, 1 failed (the pre-existing D5 mock-fidelity failure - P6
touched neither conftest.py nor that test).
The five review findings of 2026-10-23 for phase P5:
1. The P5.5 fallback branch (a checkpoint without flownet.* keys) died with
a bare RuntimeError at the strict load: the arch always owns the flownet
(keep_arch.py:882), so the state dict of the net always carries the
flownet.* keys - and the GMFlow weights the arch had just loaded were
discarded. The flownet tensors of the built net (the trained ones - the
arch was built with flownet_path) are now merged into the checkpoint
state dict before load_state_dict(..., strict=True). The load stays
strict (an unexpected key still raises) and the trained flow weights
survive. The stub arches of the tests behave like nn.Module now - a
no-op recorder had hidden the defect.
2. P5.4's acceptance is no longer unmet: the GPU acceptance run was
executed (real weights, real ComfyUI modules, RTX 4070 Ti SUPER) and is
green (5 passed, exit code 0; the run log is in the audit doc section 5).
KEEP_MODEL_DIR now really decides where the weights come from - it is the
models root of the run (folder_paths.models_dir), the harness fixture
no longer isolates the gpu_acceptance tier, and the run needs
KEEP_NO_COMFY_MOCKS=1 so the real ComfyUI modules are in charge.
3. The "the flow weights are not random" assertion of the acceptance test
could not distinguish random weights from trained ones. It now compares
the flownet tensors with the tensors of their source (the KEEP checkpoint
or the GMFlow file) bit for bit.
4. The superseded download helper load_file_from_url_comfy is gone
(src/keep_paths.py only resolves paths; the download lives in
src/keep_download.py since P5.2). The stale docstrings, the stale
comment in the vendored facelib and the two tests that called the dead
helper are fixed.
5. The risk list mentioned a pending deletion of assets/keep-preview.png
that does not exist - the asset is committed and present (93279 bytes).
The rule in the implementation workflow now states the truth.
Checks: the gate (pytest -q -m "unit or mock_integration") is green (234
passed, 1 skipped, 1 xfailed); the opt-in acceptance run is green (5 passed);
the acceptance tier still skips cleanly without its environment variables.
P5.0-P5.5 are implemented and committed. This commit adds the phase audit (docs/audit/2026-10-23-p5-weights.md, deviations D1-D8), the working-state review (docs/2026-10-23-p5-weights-progress.md) and marks the P5 row in the plan's progress table as done.
WildAi