Implements .dev/docs/keep-fix5-implement1.txt against the 9e6c50f tree:
Phase 1 - the paste pipeline: the face upscaler joins the batched core
(the _batched_paste/_batched_paste_multiface aborts are deleted), mode
is forced chunked with a bg model, face_upscale_model threads through
the paste chunk bodies (parse masks stay on the 512 faces, the ROI
grid_sample warps the upscaled faces), upscale_faces_batch_gpu (B=1,
FP16 autocast on CUDA), and _tiled_scale_and_resize_batch replaces
_tiled_scale_batch + the round-12 bg grouping (B=1 FP16 loop, one
bicubic-antialias resize straight to the paste canvas; the no-model
branch keeps _resize_bg_frames). bg_chunk_size and its pins are removed
as dead machinery. parse_soft_masks and _run_upscaler run under
autocast(enabled=(device.type == 'cuda')).
Phase 2 - facelib: RetinaFace caches priors by (h, w, device) and
detect_faces filters by confidence and runs torchvision.ops.nms on the
device before the single CPU transfer (output format preserved);
read_image gains check_gray=True, only the three sequence call sites
pass False.
Phase 3 - archs: SDPA in CrossAttention._attention, AttnBlock and both
gmflow attention forms; get_flow pinned to float32 with autocast
disabled; pos_emb cast to query_emb.dtype; keep_net under autocast at
both sequence call sites.
Phase 4 - the vendored basicsr pruning: data/, losses/, models/,
train.py, test.py, 23 unused archs, 7 unused utils deleted (78 files);
DCNv2Pack removed from arch_util; the three __init__.py rewrites;
enumeration manifests regenerated. metrics/ and ops/ stay on disk
byte-identical, off the import path. 4.2 skipped (explicitly optional).
Autocast is CUDA-only everywhere, so the CPU gate keeps exercising the
FP32 paths; no model is loaded. Gate: 556 passed, 0 failed (embedded
python; 557 -> 556 = the dead bg_chunk_size pins out, new pins in).
GPU throughput and the FP16/SDPA numerics stay owner-measured.
Round-11 upscaler-pipeline audit (owner task: check whether the bg/face
upscaler inputs are applied correctly or erroneous, and how they compose
with final_upscale_factor):
- VERDICT: the two upscaler models are quality operators only - their
output scales are always absorbed by resizes (batched_bg_frames /
vendored paste / aligned path); final_upscale_factor alone fixes the
output geometry (the paste canvas x factor), exactly as the node
tooltip states. Slowdown with x4 models is the documented design
(a face upscaler disqualifies the batched paste; bg model runs per
fallback frame; the canvas is model-widened x factor).
- CONFIRMED ERROR (the one real bug found, reproduced CPU-only in three
configurations): the single-face no-face/affine-None short-circuit
appended the raw _run_upscaler output without the factor/canvas
resize, so a bg model with scale != factor or a sub-512 clip mixed row
sizes and crashed the convert-out (np.stack ValueError). The
multiface sibling already resized (round 10); this applies the same
paste_canvas_hw + LANCZOS4 resize to the single-face site.
- The twin size check (:1352) is dead code (all_restored_faces_cv2 is
preallocated to num_input_frames) - no fix required.
- One pin's paste stub corrected to the real 512x512 canvas with a
dated docstring; no fixture re-capture needed (only rows that crashed
the run change pixels).
Gate: 555 passed, 6 skipped, 7 deselected, 0 failed (baseline 555).
Implements the owner's .dev/docs/keep-fix3.txt precisely:
- Fix 1.1: a frame with no detected face (affine None) short-circuits
the fallback paste to the upscaled background - no vendored
paste_faces_to_input_image on a face-less frame; paste_bar threaded
into _batched_paste; the canvas/chunk buffers alias when every frame
is batched (no copy); progress.update(1) added to the skipped rows.
- Fix 1.2: the grayscale eligibility filter is gone - the device blend
pastes a gray frame with the same tensor logic; _batched_paste gains
stage_bar and forwards it into all three paste modes.
- Fix 2.1: _paste_chunk_roi is one vectorized GPU tail - the faces and
the parse masks warp to the full canvas in one warp_affine_batch call
each (grid_sample), one-tensor alpha blend, the vendored quantizing
tail over the whole-canvas maximum; the per-face ROI machinery
(warp_face_roi, fixed-point terms, ROI patch transfers) is gone from
the single-face path. warp_face_roi stays live for the multi-face
sibling _paste_chunk_faces. Pixel change owner-authorized; the
retired transfer counters are disclosed in the dated docstring.
- Fix 2.2: device_output_mode - CUDA never returns "host": full while
the whole batch's canvas fits the budget, else always chunked (the
OOM handler is the net, the owner-authorized trade); non-CUDA keeps
"host".
- Fix 2.3: paste_faces_batch_chunked ticks the stage bar per completed
GPU group; paste_faces_batch does the same per chunk (the full/host
modes) - nobody ticks twice, the bar totals num_input_frames exactly
on every path; the multi-face groups get no tick (the multiface
stage owns its bar).
The single-face sequence golden is re-captured (owner-authorized
pixels); the paste equivalence and cadence pins are reworked to the
measured behavior with dated docstrings.
Gate: 555 passed, 6 skipped, 7 deselected, 0 failed (baseline 552).
The paste-mode verdict (device_output_mode) reads the free VRAM through
ComfyUI's get_free_memory(..., torch_free_too=True), which counts what
the torch cache can give back. Round 6 (Opt 6) deleted the per-chunk
_soft_empty_cache() of the restore loop, so cached restore blocks shrank
the paste budget (the min of the 1 GiB cap and the free share) and the
verdict flipped from full/chunked to host: the clip took the host-CPU
assembly - the owner's "Pasting faces and finalizing frames" stall.
One release before the verdict in both sequence paths restores the
pre-Opt-6 budget once per run, without re-paying the per-chunk sync cost
Opt 6 removed. The cadence pin records the measured twice-per-run clean
cadence (pre-paste, then the finally's); the OOM cadence is unchanged
and pinned; a new pin proves the release precedes the paste stage.
Gate: 535 passed, 6 skipped, 7 deselected, 0 failed (baseline 534 plus
the new ordering pin).
Opt 1 - the fused crop loops warp on the device (crop_faces_batch per
source-shape group) instead of one cv2.warpAffine per row.
Opt 2 - warp_face_roi uploads its four terms once and composes
x_full/y_full on the device; the fixed-point math is unchanged (bitwise).
Opt 3 - bgr_u8_batch_to_normalized runs in float32 end to end and
_gaussian_smooth convolves in float32; the float64 transient is gone.
Opt 4 - the Lanczos axis matrices are cached per (in_size, out_size); the
cache holds device-free numpy, so one entry serves both devices.
Opt 5 - the aligned single-image path restores on the device
(_restore_aligned_torch): no detection, no vendored img2tensor tail, no
vendored paste. The io schema is untouched.
Opt 6 - the chunk loops no longer call _soft_empty_cache(); the release
stays in the finally blocks and in the OOM handler.
The two sequence golden fixtures are re-captured (the owner's documented
override of the capture tool's do-not-rerun warning) - Opt 1 and Opt 3
change pixels by design. The image fixtures are unchanged.
Tier-1 gate: 534 passed, 6 skipped, 7 deselected, 0 failed.
The round-4 fused crop loops re-derived each frame's detection image with
one single-row converter call per frame - N host sync points where the
round-3 crop did one batch. The grouped form converts the chunk's gathered
rows in one elementwise pass (bitwise the per-row rows) and is bounded by
the caller's chunk; the paste-fallback row form stays. Pins guard the
conversion count. The paste stage itself is proven byte-equivalent to
32e6628 (dispatch, cores, tails); timed CPU stubs show no superlinear
term in either tree - the real-box burn needs the owner's log to attribute
(the device-mode verdict under real VRAM pressure is the only paste-stage
quantity this diff can move).
- R1: the face helper is constructed on the pack device (was the offload
device); load_device() explicitly places the detector and the parser with
a logged CPU fallback, offload() mirrors the restore, the YOLO .device
skew is closed (P-D1..P-D5).
- R2: detection streams per max_clip_length group (no full-clip convert-in
retention), crop is fused into a chunk-bounded restore loop with
preallocated staging, the paste tail keeps one contractual float32
buffer, the helper is released after every run; golden fixtures pin the
outputs bitwise (P-M1..P-M7).
- The diagnosis: the OOM was O(frames) whole-clip host buffers, not the
faces (the detector/landmark path was already GPU-placed);
max_clip_length never bounded convert-in or the paste tail.
- Gate: 523 passed, 6 skipped, 7 deselected, 0 failed (baseline 505).
process_image_sequence converts the batch once, detects per frame while capturing the frames the detector saw (the read_image products, including the min(h,w)<512 upscale), estimates each affine once with the vendored call, crops with cv2.warpAffine (decision D2: the uint8-bound warp stays bit-exact; the batched grid_sample helper stays parked and pinned), normalizes the whole crop batch on the device, keeps the KEEP chunk contract byte-identical, converts the restored faces with one batched tensor2img, pastes through the batched paste with the vendored fallback, and stacks the output in one transfer. The paste canvas is the detection frame, so small frames keep the base's upscaled canvas (the round-1 QA defect F1, fixed and pinned). A failed affine degrades to the neutral crop with a warning instead of crashing like the base. Progress still reports num_frames * 4 in the four families. Pins: the read-once/affine-once/chunk-contract AST guards, the progress contract, and the new end-to-end base-vs-shipped suite (tests/test_gpu_sequence_e2e.py) with the real vendored helper - measured drift 0/255 on the vendored branch, 1/255 on the batched branch.
The pack owned a private stdlib handler, so every line bypassed the coloured console of ComfyUI, and the call sites carried a literal KEEP prefix. The logger now adopts the host console (no handler and propagate=True when ComfyUI is present, one private handler when it is not), the tag lives in a single filter, and the console writes of the pack (the tqdm bars of the processor, the prints of the vendored facelib) reach it through a thread-scoped redirect.
WildAi