The package installed its own handler, Formatter and level on a
module-level logger, then logged through that logger. ComfyUI core already
installs app.logger.ColoredFormatter on the ROOT logger, so a bare
logging.<level>() call whose message starts with "[ComfyUI-VibeVoice] " is
tagged and coloured for free. This removes the duplicate machinery rather
than extending it -- no vv_logging module, Logger subclass, LoggerAdapter,
vendored ANSI table, new handler, setLevel or propagate.
Three hazards that motivated deleting the setup block rather than moving it:
- propagate=False severed pytest's caplog; the tests only passed because
__init__.py's `if "pytest" in sys.modules` guard skipped the block.
Bare root calls propagate by default.
- logger.setLevel(INFO) on the package logger pinned every descendant, so
ComfyUI's --verbose never applied to this node. Deleting the block fixes it.
- The level must stay a literal at the call site; no env-var switch was
added. modules/diagnostics.py gates diagnostic CONTENT and is untouched.
Every call site (235 across 34 files) is now a bare root call with the
prefix. The vendored src/vibevoice tree used transformers.utils.logging,
which has no module-level info/warning/debug; those files now import stdlib
logging, so a new call site there would fail loudly instead of silently.
Level audit against ERROR=failure / WARNING=degradation / INFO=user-facing
progress / DEBUG=internals. Deleted as noise: the resample notice (44.1 kHz
reference audio is the normal case) and the two "Successfully loaded
external VibeVoice" confirmations, which duplicated the patcher's load line.
Promoted to WARNING: a mixed-naming GGUF, which is resolved by heuristic
majority vote and silently aliases the rest. Demoted to DEBUG: the four
SageAttention kernel-selection lines, model discovery, shard counts,
per-retry download attempts, attention-mode confirmation and the
save_pretrained notice. Kept at INFO: generation complete, transcription
results, model downloads and load starts.
Judgement calls recorded in docs/2026-10-01-vv-logging-cleanup-design.md.
The two I am least sure of: the ungated memory-census profile line was
demoted rather than gated (adding a gate would not be presentation-only),
and patcher.py's "Loading VibeVoice models for..." was kept at INFO against
the ask's example list because it is the only load-start line the package has.
48 new tests in tests/test_logging_idiom.py plus tests/test_audio_utils.py:
byte-exact ColoredFormatter rendering, no-leftover-machinery, AST prefix and
two-direction level policy, and the 44.1 kHz resample regression (the call
still fires with (44100, 24000); nothing is logged). 37 caplog.at_level
pins that named a module logger were stripped -- they only lowered a named
logger and left root at WARNING, so the record was discarded before capture.
Gate: 1924 passed, 30 skipped, 0 failed (1876 before this change). Twelve
deliberate mutations -- re-deleting a message, re-leveling, re-adding
setLevel and propagate=False, stripping a prefix, moving the resample out
of its branch -- were each caught by at least one test.
Not run: ComfyUI was never launched and no checkpoint was loaded.
The [vvcensus], [vvrss], [vvpull], [vvload], [vvdtype], 'Load diagnostics:'
and 'GGUF forward diagnostics:' lines exist to make a slow load or a slow
autoregressive step explainable. None of them produce audio, and all of
them default to ON, so a normal run prints several of them.
Gate them at the six choke points that already existed (report_census,
RssSampler.report, _install_vbar_observer, _log_conv_dtype_mismatch,
_log_load_diagnostics, log_gguf_forward_counters, plus the two emit sites
in generation.py and base_loader.py) rather than commenting out 18 call
sites: the formatting helpers stay importable and unconditional, so a
probe or a test can still ask for the string directly.
New modules/diagnostics.py owns the switch. VIBEVOICE_DIAGNOSTICS=1 turns
everything on; VIBEVOICE_RAM_CENSUS and VIBEVOICE_VBAR_OBSERVER keep
their existing per-family meaning. Production leaves all three unset.
Tests that assert the CONTENT of a line now set the env var explicitly,
and three tests pin the new silent-by-default contract (census, gguf
counters, vbar observer).
Loading problem closed for every model type. Live, 2026-09-30 (RTX 4070 Ti
SUPER 16GB), 1.5B bf16 and 7B fp8 both load straight into VRAM with
+0.69 GB / +1.45 GB of machine RAM and no sustained SSD traffic.
What was wrong
--------------
The per-tensor placement used `Tensor.to(cuda)` on a memory-mapped view. That
is a HOST-side read: the copy engine faults every page in through the CPU, at
0.55 GB/s and +2.18 GB of machine RAM per 2 GB (report 2026-09-30 section F6).
Every route paid it, on every model type.
The 7B fp8 load had looked SSD-free only because pass 1's `safe_open` was
committing the whole file as private memory first, so those host-side copies
were served from RAM. The 19.7 -> 28 -> 20 GB spike was the receipt for that.
What changed
------------
* base_loader.place_tensor_on_device(): one placement helper for every route.
With an aimdo mapping (main.py sets aimdo_enabled at startup) it uses core's
own `read_tensor_file_slice_into` to DMA the file byte range straight into a
preallocated CUDA tensor -- the same primitive core already uses to page
weights into VRAM, measured at 2.6 GB/s with the page cache left clean.
Falls back to `.to()` when core declines or when the aimdo native library
raises; a one-time `[vvload]` log line confirms the DMA is live.
* One streaming assign for all routes. `_stream_apply_dense` gained
`target_device`, and a new `_stream_apply_dense_safetensors` is the dense
twin of `_stream_apply_safetensors`. The TTS and ASR dense branches no longer
build a whole CPU state dict and call `model.to(cuda)`; GGUF and the internal
official-model loader joined the same path.
* `read_safetensors_tensors_by_name()`: pass-1 dequant scales are read from the
byte ranges the safetensors header already records, instead of `safe_open`.
That drops a genuine ~1x-file private commit.
* select_patcher_class() always returns the standard ModelPatcher, so core can
full-load instead of trapping weights in VBAR for an autoregressive model.
Tests
-----
5 new tests for DMA routing and both fallbacks; load-path tests updated to the
new read seam. Targeted run: 373 passed / 34 failed, all 34 pre-existing and
asserting the ModelPatcherDynamic/vbar architecture this change removes
(26 in test_dynamic_patcher_selection.py). Full suite not run, per user rule.
Loading filled host RAM 20->40GB while streaming from SSD and never released
it; inference then re-served the whole model per AR step. Teardown against
core's aimdo/DynamicVRAM machinery, measured on the real 9.47GB 7B fp8 and
5.16GB 1.5B bf16 checkpoints, found four loader defects plus one that was
never ours to fix.
Defects fixed (each with before/after numbers):
* Quant pass 1 pre-read each dequant-at-load layer's scale with
safetensors.safe_open + get_tensor. On Windows one such call commits ~1x
file size as private, untouched memory, pinned for the lifetime of the
returned tensor: +9,050MB on the 7B fp8, invisible to both the working-set
counter and the storage census. Now maps the file once through core's
aimdo arm and clones only the tiny scales. Whole load: 1.51x -> 0.13x file
size; retained after free 11.3GB -> 96MB.
* Quant families were excluded from core's dynamic patcher by an inherited
"quant streams natively" stop condition. Because a legacy patcher has
nowhere to page from, those routes cloned every tensor into host RAM and
fully H2D'd it. select_patcher_class now follows core's availability rule
for every family, with gguf_block still excluded for a measured reason
(GGUFTensor.from_reader_tensor clones the reader's view).
* replace_linears_for_quant built its resident modules outside the meta
context: 8.08GB of never-written host allocation per 7B fp8 load.
* Dormant weight_function/bias_function double application: core already
applies them inside cast_bias_weight (ops.py:431-438).
Exonerated with numbers, not assumed: the dense read path is core's own
(13.5MB private for a 5.16GB file), and core's file->VRAM paging read is
cache-clean (+0.13GB per 2GB). Only host-side view reads reproduce the
reported 1:1 RAM at 0.55GB/s signature.
Inference: reproduced headlessly that when the tree does not fit in the
VRAM that is actually free, every forward re-reads every weight from the
checkpoint file (3072/3072 file reads, 21ms -> 125ms per step). Our wrappers
are protocol-correct; residency is stable at 100% with headroom. [vvpull]
now reports resident vs reread with bytes and free VRAM so one live line
settles it, behind VIBEVOICE_VBAR_OBSERVER=0.
Not changed after being tried and reverted: deriving fast_disk from the
checkpoint. It measured as a no-op on the dev host but converted RAM-speed
pinned re-reads into disk-speed reads live, which made loading dramatically
slower. Reverted in full; see the report for the probe-design lesson.
Adds host-RAM instrumentation ([vvrss] with machine-level start/end, [vvcensus]
storage census, tts-generate bracket), four standalone probes, and ~4000
lines of tests pinning the invariants above.
Loading
- GGUF install is now two passes: a metadata pass that decides each
tensor's disposition, then an install pass that assigns residents in
place and streams dense tensors. The whole checkpoint is no longer
buffered in a dict alongside the model being built.
- dequantize_reader_tensor takes a target dtype, so dequant-at-load
writes straight into the destination parameter and the fp32
intermediate is never allocated.
- A bundle whose heavy fields were released is rebuilt from its recorded
source_path instead of failing the consumer.
- Host memory is released after install.
Numerics
- Q8_0 dequant computes in fp32 so the result is rounded once, at the
final cast. The activation-dtype path was reverted: it rounded twice
and moved stored weights.
- Removed a redundant weight-sized copy from the dequant kernel.
Bitwise-identical, ~1.16x.
- Precision gates are bitwise rather than tolerance-based.
Docs and tests
- README condensed; changelog moved to CHANGELOG.md.
- Third-party project references removed from source comments.
- Tests no longer assert README prose; the e2e smoke contract follows
the developer script to its new location and skips when absent.
- Version guard reads CHANGELOG.md.
Garbled, parameter-insensitive speech came from a randomized EOS head:
5.3 re-runs _initialize_weights over acoustic_connector and
tts_eos_classifier because the vendored _init_weights override had no
_is_hf_initialized guard. Guard it; only checkpoint-absent weights are
initialized now.
- MockCacheLayer exposes both the 4.x and 5.x cache APIs, so the
prefilled voice prompt is visible to 5.3's mask builder.
- _ensure_cache_has_layers covers the container: offload/prefetch,
batch ops, crop, and a copyable lazy prefetch stream.
- max_new_tokens is a combined text+speech budget, not latents.
- cfg_scale floor of 1.5 for the realtime family.
- voice presets resolve against every registered TTS root.
- sage excluded from the realtime path: it ignores the attention mask.
- bound transformers to >=5.3.0,<5.4, the measured line.
Config auto-detection:
- config_detect: read the checkpoint embedding shape as an architecture
fingerprint (header-only for safetensors, reuses the open GGUF reader);
7B=[152064,3584], 1.5B=[151936,1536]; orientation-agnostic for
shape-reversed GGUF files
- Auto-detect is the new default config_name; resolves the family before
any heavy load, or fails fast with an actionable error (.bin/.pt and
unknown families cannot be fingerprinted)
- an explicit config_name that contradicts the weights self-corrects to
the detected family with one WARNING (reconcile_config)
- loader: friendly shape pre-check in _apply_state_dict names the
offending tensors and hints at config_name instead of torch's raw
size-mismatch RuntimeError
Dropdown dedup:
- VibeVoice-Large removed from config_name options (duplicate of 7B);
kept as a legacy alias so saved workflows still load (normalize at node
+ loader entry; validate_inputs(**kwargs) override makes core skip its
combo-membership check)
- node resolves Auto-detect before computing the cache identity so the
request key matches the consumer's bundle-derived key (no per-run churn)
Console noise:
- demote ~30 internal INFO logs to DEBUG across loaders/patcher/registry
- drop two stray tie_weights prints; tied lm_head.weight no longer warned
as missing (expected under tie_word_embeddings)
Tests: 1060 passed / 5 pre-existing failures / 4 skipped
WildAi