The owner's live profile on the 30 s clip (17 -> 32 -> ~52 GB) is now
attributed by measurement (a per-stage RAM sampler outside the tree:
psutil RSS/commit + StageBar marks, 128-frame CPU fixture, x6.07):
- the paste stage's gradual per-tick growth is the round-18 output
buffer's lazy page commit (16 x 19.44 MB per 16-frame group) - the
contract floor being paid, not a leak;
- the first stage's +15 GB is the caller-owned input IMAGE materializing
into the working set as the detection loop reads it (dRSS +2.3 GB,
dCommit +0 B on the raw-input run) - not pack-owned;
- the static audit found no whole-clip float32 copy of the input in the
pack (the only advanced-indexing gather of the input is the
max_clip_length-bounded :1132).
Two pack-owned O(N) items remained and are removed:
- FIX 1: the mixed-case faces np.stack gather (~611 MB owner-scale) is
replaced by LazyFaceSource - the caller's face stack served per group
(a view when contiguous, a group-sized copy when scattered) through a
faces_provider seam mirroring frame_provider; the all-eligible
pass-through gate keeps handing the raw stack.
- FIX 2: the fallback tail converts per gray group (_flush_fallback_group,
the indexed write) - one group's stack + transient (<= 20 x 4.86 MB),
independent of the fallback count, instead of a whole-fallback stack
plus its ~10x conversion transient; the device-canvas and host tails
keep their whole-stack form.
Gate: 6 failed, 580 passed, 16 skipped, 7 deselected - the 6 are the
owner's own asset/example edits, unchanged; 11 new pins (byte identity,
no-copy censuses, the cross-group boundary, the one-whole-clip-buffer
count).
The paste core owned a float32 buffer and the processor then allocated a
second whole-clip one and copied the eligible rows into it while the
first was still alive. On the owner's clip (777 x 1500 x 1080 x 3,
float32 = 15.10 GB per buffer) the mixed case - any face-less frame -
peaked at input + two such buffers (~45.3 GB); the all-eligible case
aliased and was already at the ~30.2 GB contract floor.
- paste_faces_batch_chunked gains keyword-only out= / out_rows=: the
processor's final buffer and the destination row of each canvas row.
The group convert-out writes by INDEX (in-place index_put_, not the
read-side advanced-indexing copy) - a scattered destination stays
scattered. Both None keep today's behaviour exactly.
- comfy_convert_out gains out_rows= with the same indexed write plus the
loud-failure validation (pair, length, range, uniqueness, dtype,
canvas, capacity).
- both processor paste twins allocate the final whole-clip buffer BEFORE
the paste and hand it to the core; the scatter is deleted. The
all-eligible case keeps its alias - no copy is reintroduced.
- draw_box now lands on the final rows of the final buffer.
Measured (a torch.empty census over both paths, base vs this tree): the
mixed case goes from 2 canvas-shaped float32 buffers to 1 in both the
single-face and the multiface path; the all-eligible case stays at 1; the
returned IMAGE is that buffer (weakref identity).
Gate: 569 passed, 16 skipped, 7 deselected, 6 failed - the 6 are the
owner's own asset/example edits, unchanged; the 7 new pins are the
indexed-write byte identity (both paths), the validation contract, the
mixed-case census (both paths) and the draw_box rows.
The final stage materialized the clip twice as uint8 BGR: the whole-clip
stack (~3.78 GB on the owner's 777-frame 1500x1080 clip) and the
eligible-gather copy of it (up to another ~3.78 GB). Both are gone:
- BgrFrameSource (src/keep_image_utils.py) wraps the caller's float32
IMAGE tensor and converts rows on demand; the conversion is elementwise,
so a row is byte-identical to the same row of the whole-clip conversion.
- paste_faces_batch_chunked (src/keep_gpu_paste.py) asks the background
per group through one provider protocol; the numpy-array path is served
by NumpyFrameProvider and keeps the old zero-copy slicing.
- both paste twins (src/keep_processor.py) take the source; the eligible
rows are a lazy view, so the gather copy is gone and the rows that are
not pasted are never converted.
- the fallback rows arrive in groups of their own, and their gray flags
are computed with their group (the whole-clip gray pass is gone too);
the eligible rows' flags were never read.
The float32 output buffer stays (it is the node's returned IMAGE) and the
float32 input stays (the caller holds it). The full and host paste modes
still materialize the eligible stack - the disclosed limitation.
Gate: 561 passed, 16 skipped, 7 deselected, 6 failed - the 6 are the
owner's own asset/example edits, unchanged; the 9 new pins are the byte
identity, the laziness, the group-sized conversions and the gray flags.
Implements .dev/docs/keep-fix5-implement1.txt against the 9e6c50f tree:
Phase 1 - the paste pipeline: the face upscaler joins the batched core
(the _batched_paste/_batched_paste_multiface aborts are deleted), mode
is forced chunked with a bg model, face_upscale_model threads through
the paste chunk bodies (parse masks stay on the 512 faces, the ROI
grid_sample warps the upscaled faces), upscale_faces_batch_gpu (B=1,
FP16 autocast on CUDA), and _tiled_scale_and_resize_batch replaces
_tiled_scale_batch + the round-12 bg grouping (B=1 FP16 loop, one
bicubic-antialias resize straight to the paste canvas; the no-model
branch keeps _resize_bg_frames). bg_chunk_size and its pins are removed
as dead machinery. parse_soft_masks and _run_upscaler run under
autocast(enabled=(device.type == 'cuda')).
Phase 2 - facelib: RetinaFace caches priors by (h, w, device) and
detect_faces filters by confidence and runs torchvision.ops.nms on the
device before the single CPU transfer (output format preserved);
read_image gains check_gray=True, only the three sequence call sites
pass False.
Phase 3 - archs: SDPA in CrossAttention._attention, AttnBlock and both
gmflow attention forms; get_flow pinned to float32 with autocast
disabled; pos_emb cast to query_emb.dtype; keep_net under autocast at
both sequence call sites.
Phase 4 - the vendored basicsr pruning: data/, losses/, models/,
train.py, test.py, 23 unused archs, 7 unused utils deleted (78 files);
DCNv2Pack removed from arch_util; the three __init__.py rewrites;
enumeration manifests regenerated. metrics/ and ops/ stay on disk
byte-identical, off the import path. 4.2 skipped (explicitly optional).
Autocast is CUDA-only everywhere, so the CPU gate keeps exercising the
FP32 paths; no model is loaded. Gate: 556 passed, 0 failed (embedded
python; 557 -> 556 = the dead bg_chunk_size pins out, new pins in).
GPU throughput and the FP16/SDPA numerics stay owner-measured.
Round-11 upscaler-pipeline audit (owner task: check whether the bg/face
upscaler inputs are applied correctly or erroneous, and how they compose
with final_upscale_factor):
- VERDICT: the two upscaler models are quality operators only - their
output scales are always absorbed by resizes (batched_bg_frames /
vendored paste / aligned path); final_upscale_factor alone fixes the
output geometry (the paste canvas x factor), exactly as the node
tooltip states. Slowdown with x4 models is the documented design
(a face upscaler disqualifies the batched paste; bg model runs per
fallback frame; the canvas is model-widened x factor).
- CONFIRMED ERROR (the one real bug found, reproduced CPU-only in three
configurations): the single-face no-face/affine-None short-circuit
appended the raw _run_upscaler output without the factor/canvas
resize, so a bg model with scale != factor or a sub-512 clip mixed row
sizes and crashed the convert-out (np.stack ValueError). The
multiface sibling already resized (round 10); this applies the same
paste_canvas_hw + LANCZOS4 resize to the single-face site.
- The twin size check (:1352) is dead code (all_restored_faces_cv2 is
preallocated to num_input_frames) - no fix required.
- One pin's paste stub corrected to the real 512x512 canvas with a
dated docstring; no fixture re-capture needed (only rows that crashed
the run change pixels).
Gate: 555 passed, 6 skipped, 7 deselected, 0 failed (baseline 555).
Implements the owner's .dev/docs/keep-fix3.txt precisely:
- Fix 1.1: a frame with no detected face (affine None) short-circuits
the fallback paste to the upscaled background - no vendored
paste_faces_to_input_image on a face-less frame; paste_bar threaded
into _batched_paste; the canvas/chunk buffers alias when every frame
is batched (no copy); progress.update(1) added to the skipped rows.
- Fix 1.2: the grayscale eligibility filter is gone - the device blend
pastes a gray frame with the same tensor logic; _batched_paste gains
stage_bar and forwards it into all three paste modes.
- Fix 2.1: _paste_chunk_roi is one vectorized GPU tail - the faces and
the parse masks warp to the full canvas in one warp_affine_batch call
each (grid_sample), one-tensor alpha blend, the vendored quantizing
tail over the whole-canvas maximum; the per-face ROI machinery
(warp_face_roi, fixed-point terms, ROI patch transfers) is gone from
the single-face path. warp_face_roi stays live for the multi-face
sibling _paste_chunk_faces. Pixel change owner-authorized; the
retired transfer counters are disclosed in the dated docstring.
- Fix 2.2: device_output_mode - CUDA never returns "host": full while
the whole batch's canvas fits the budget, else always chunked (the
OOM handler is the net, the owner-authorized trade); non-CUDA keeps
"host".
- Fix 2.3: paste_faces_batch_chunked ticks the stage bar per completed
GPU group; paste_faces_batch does the same per chunk (the full/host
modes) - nobody ticks twice, the bar totals num_input_frames exactly
on every path; the multi-face groups get no tick (the multiface
stage owns its bar).
The single-face sequence golden is re-captured (owner-authorized
pixels); the paste equivalence and cadence pins are reworked to the
measured behavior with dated docstrings.
Gate: 555 passed, 6 skipped, 7 deselected, 0 failed (baseline 552).
The paste-mode verdict (device_output_mode) reads the free VRAM through
ComfyUI's get_free_memory(..., torch_free_too=True), which counts what
the torch cache can give back. Round 6 (Opt 6) deleted the per-chunk
_soft_empty_cache() of the restore loop, so cached restore blocks shrank
the paste budget (the min of the 1 GiB cap and the free share) and the
verdict flipped from full/chunked to host: the clip took the host-CPU
assembly - the owner's "Pasting faces and finalizing frames" stall.
One release before the verdict in both sequence paths restores the
pre-Opt-6 budget once per run, without re-paying the per-chunk sync cost
Opt 6 removed. The cadence pin records the measured twice-per-run clean
cadence (pre-paste, then the finally's); the OOM cadence is unchanged
and pinned; a new pin proves the release precedes the paste stage.
Gate: 535 passed, 6 skipped, 7 deselected, 0 failed (baseline 534 plus
the new ordering pin).
Opt 1 - the fused crop loops warp on the device (crop_faces_batch per
source-shape group) instead of one cv2.warpAffine per row.
Opt 2 - warp_face_roi uploads its four terms once and composes
x_full/y_full on the device; the fixed-point math is unchanged (bitwise).
Opt 3 - bgr_u8_batch_to_normalized runs in float32 end to end and
_gaussian_smooth convolves in float32; the float64 transient is gone.
Opt 4 - the Lanczos axis matrices are cached per (in_size, out_size); the
cache holds device-free numpy, so one entry serves both devices.
Opt 5 - the aligned single-image path restores on the device
(_restore_aligned_torch): no detection, no vendored img2tensor tail, no
vendored paste. The io schema is untouched.
Opt 6 - the chunk loops no longer call _soft_empty_cache(); the release
stays in the finally blocks and in the OOM handler.
The two sequence golden fixtures are re-captured (the owner's documented
override of the capture tool's do-not-rerun warning) - Opt 1 and Opt 3
change pixels by design. The image fixtures are unchanged.
Tier-1 gate: 534 passed, 6 skipped, 7 deselected, 0 failed.
The round-4 fused crop loops re-derived each frame's detection image with
one single-row converter call per frame - N host sync points where the
round-3 crop did one batch. The grouped form converts the chunk's gathered
rows in one elementwise pass (bitwise the per-row rows) and is bounded by
the caller's chunk; the paste-fallback row form stays. Pins guard the
conversion count. The paste stage itself is proven byte-equivalent to
32e6628 (dispatch, cores, tails); timed CPU stubs show no superlinear
term in either tree - the real-box burn needs the owner's log to attribute
(the device-mode verdict under real VRAM pressure is the only paste-stage
quantity this diff can move).
- R1: the face helper is constructed on the pack device (was the offload
device); load_device() explicitly places the detector and the parser with
a logged CPU fallback, offload() mirrors the restore, the YOLO .device
skew is closed (P-D1..P-D5).
- R2: detection streams per max_clip_length group (no full-clip convert-in
retention), crop is fused into a chunk-bounded restore loop with
preallocated staging, the paste tail keeps one contractual float32
buffer, the helper is released after every run; golden fixtures pin the
outputs bitwise (P-M1..P-M7).
- The diagnosis: the OOM was O(frames) whole-clip host buffers, not the
faces (the detector/landmark path was already GPU-placed);
max_clip_length never bounded convert-in or the paste tail.
- Gate: 523 passed, 6 skipped, 7 deselected, 0 failed (baseline 505).
process_image_sequence converts the batch once, detects per frame while capturing the frames the detector saw (the read_image products, including the min(h,w)<512 upscale), estimates each affine once with the vendored call, crops with cv2.warpAffine (decision D2: the uint8-bound warp stays bit-exact; the batched grid_sample helper stays parked and pinned), normalizes the whole crop batch on the device, keeps the KEEP chunk contract byte-identical, converts the restored faces with one batched tensor2img, pastes through the batched paste with the vendored fallback, and stacks the output in one transfer. The paste canvas is the detection frame, so small frames keep the base's upscaled canvas (the round-1 QA defect F1, fixed and pinned). A failed affine degrades to the neutral crop with a warning instead of crashing like the base. Progress still reports num_frames * 4 in the four families. Pins: the read-once/affine-once/chunk-contract AST guards, the progress contract, and the new end-to-end base-vs-shipped suite (tests/test_gpu_sequence_e2e.py) with the real vendored helper - measured drift 0/255 on the vendored branch, 1/255 on the batched branch.
The pack owned a private stdlib handler, so every line bypassed the coloured console of ComfyUI, and the call sites carried a literal KEEP prefix. The logger now adopts the host console (no handler and propagate=True when ComfyUI is present, one private handler when it is not), the tag lives in a single filter, and the console writes of the pack (the tqdm bars of the processor, the prints of the vendored facelib) reach it through a thread-scoped redirect.
WildAi