veda_timing (loader, method veda) logs the time in gather, score, select, kernel and scatter
for a run; off by default and it changes no number. The run line now names the size and tile
plan of each stage, not only the last. Cheap speed-ups that keep the numbers exactly: the
list width of kept tiles comes from the layout instead of a device read, the head groups and
the per-row pattern stay on the device, and the predictor's projections are copied from pinned
memory.
It was drawn at the bottom of the model page, far from the field it fills, and the
selector's hint pointed at 'the row below'. The row now follows the selector directly and
the hint no longer points anywhere.
Learned tile-sparse attention (Veda, arXiv 2605.30325): tiles, tile plans, predictor
scoring and selection in plain torch (tested on CPU, including a real tiny H3 with a
stand-in kernel), an INT8 Triton kernel derived from SageAttention (untested on a GPU),
installed through the same override and per-step wrapper as SLA. Runs at the predictor's
trained budget; LoRA, MXFP8, Two stages, dense steps and the attention backend compose.
Settings > Models offers the official Hugging Face download of the predictor on a click,
with its licence beside it; taggers and this share one downloader (core/fetch.py).
A few-step H3 file whose metadata keeps explicit head sizes but not the head
count was built with one head and failed on the head biases. The count is now
taken from the head bias and patch projection shapes (shapes only, any device
or dtype), with a test through the real loader at every weight dtype.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tw9fjUsYDkyEtLCrzqv7jo
Skip fewer attention blocks on early steps and more on late ones, in a straight
line up to sla_sparsity. Off at 0, untested on H3. Credit GSRS (arXiv 2610.21129).
tools/test_env.sh and tools/cpuforce.py set up the suite on a GPU-less machine.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0176SHndmQj7T1VtWAWZiNcq
Loader-level attention option: mean-pool Q and a smoothed K into
blocks, score them with one small matmul, attend only the top
1-sparsity fraction of key blocks per query block, via a vendored
Triton kernel. Runs alongside the chosen attention backend rather than
replacing it (one override slot; the backend becomes SLA's dense
fall-through for the text refiner, masked calls, and trailing dense
steps), and refuses cleanly -- naming why -- on anything that is not
MiniMax H3 or a machine without CUDA+Triton.
modules/loaders/sla_block_map.py and sla_kernel.py are near-verbatim
ports of v4's vendored files (no FunPack-specific imports, so nothing
to adapt); sla_attention.py is the loader-facing glue (H3 gate,
override composition, per-step wrapper), same content as v4's with a
"not a module" note added to match gguf_support.py's convention.
Wired into FunPackDiffusionModelLoader as six new optional widgets,
following v4's loaders.py wiring but expressed in v5's io.Schema style.
The Triton kernel itself is unexercisable on this Mac (no CUDA) --
matches how real generation is verified in this project (rented GPU,
not local). Ported the CPU-testable half of v4's test suite (the
override contract, the H3 gate, the per-step wrapper); the three
kernel-correctness tests carry the same CUDA+Triton skip guard v4 used.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The port plan calls for every module under modules/ to declare a
CATEGORY so Settings/Engine panels can group by it. Only alg, sharpen,
appearance, and full_control had one; the other 12 (loaders, empty
latent, media load, minimax_h3, output decode, sampling modifiers/
sampler, system pipeline) are plain infrastructure with no user-facing
settings dial, so they land in "system" except output_decode (post,
matching its STAGE) and the two sampling modules (sampling). Verified
against core/tests (548 passed) -- category validation and the
registry's field-mapping catch a typo here, per the earlier fixture-gap
fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
GGUF: port v4's gguf_support.py (backend selection, content-based .gguf
detection, dequantize fallback) and wire it into both the diffusion-model
and CLIP loaders.
Pipeline editor: reopening the Models-and-pipeline settings window was
silently discarding the whole edited pipeline (a loaded preset, every
loader file, every sampler setting) because GET /api/pipeline is
deliberately stateless server-side and boot.js never fed its own
already-saved slots back into a freshly remounted editor -- found live
while walking through the MiniMax-H3 setup flow. Fixed by re-validating
the saved slots through check() on reopen instead of always replaying
the raw stateless default.
Also fixes, found via the extensive_testing adversarial loop over both
of the above: a stale file-listing cache in the pipeline window (no way
to see a newly-added model file without a full page reload), and a
chain of races/leaks in its own refresh/commit error-banner handling
that surfaced across five review rounds -- a rejection from a superseded
request overwriting a fresher success, a failure banner never clearing
on success, banners bleeding across groups on navigation, and a
commit()'s refusal landing on the wrong view after the user backs out
before it answers.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
[Log] Info: FunPack Diffusion Model Loader | flux.safetensors loaded, attention sage
[Log] Alert: MiniMaxH3ImageToVideo node | no "first_image" input is connected
[Log] Warning: FunPack LoRA Loader | style.safetensors is empty and will not load
[Log] Critical: torch | cannot allocate 14 GB (3 GB free); the run will not start
The difference between the middle two is the one that matters. Alert is a
choice you may not have made on purpose. Warning is something that will NOT
do what its name says -- the level v4 needed and did not have, which is how
560 handlers came to swallow a failure and carry on, so a feature could be
switched on, do nothing, and never say so.
Info is not decoration either: "loaded on GPU using sage attention" is how
an attention setting that did nothing gets noticed months before the
pictures look wrong enough to investigate.
Every call site moved -- there is no note() left to be vague through -- and
the loaders now say what they actually loaded rather than only what they
were asked for. Records are kept in a bounded ring as well as printed, so a
long rental session cannot turn the log into a memory leak, and served at
/funpack/api/log so a person can read it somewhere other than a terminal
full of ComfyUI's own output. There is a page for it.
Also dropped the preflight this was traded for: min and max are already
declared and already enforced at both doors, and a second protection layer
around a check that exists is ceremony.
Two mistakes worth keeping: a log line reached into the model's shape to
describe it and could have failed the load it was only narrating; and both
new pages passed `children` to an element that takes `rows`, which the kit
accepted and silently dropped. It throws on an unknown element and not on an
unknown prop.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
set_model_compute_dtype sets force_cast_weights on purpose: it is what
casts each layer's weights to match. Clearing it afterwards left the model
casting its INPUT to the chosen dtype while the weights stayed as loaded,
so the first Linear of the first sampling step raised "mat1 and mat2 must
have the same dtype".
Picking any compute dtype other than default therefore broke sampling on a
full-VRAM load. It survives on constrained hardware only because the
low-VRAM path forces the cast back on by itself, which is why nobody hit it.
Straight port from v4, where the same line has shipped since 3837f8f. The
branch had no test at all; it now has one that fails against the old line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ported from v4, minus two mechanisms that are no longer needed. The LoRA
loader takes one LoRA and chains, so the FlexibleOptionalInputType dict
subclass that lied to ComfyUI's validator has nothing left to do. The CLIP
loader uses numbered slots like core's DualCLIPLoader instead of a JSON
array in a string widget.
Autogrow looked like the right answer for the encoder list and is not: it
sets force_input on any widget input, so a list of combos renders as sockets
and the file picker stops being a picker. Every core use of it is a
connection type.
GGUF and SLA are deliberately not here. Both are capabilities rather than
loader options, and both belong in modules of their own.
The tests follow what the loaders PRODUCE rather than stopping at the call:
the attention override is carried into comfy's real dispatch and checked
against every backend this machine registers, because an override that only
fails at the first attention call fails at sampling, not at load.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
