Vyacheslav-TomashevskiyandVyacheslav-Tomashevskiy
035a13bc25
The randomized Hadamard transform needs its sign flips applied before the transform. Applied afterwards they cannot spread anything: absmax and a codebook symmetric about zero are both invariant under a sign flip, so the reconstruction comes back bit-for-bit identical. The documented decorrelation step was therefore a no-op, and any block carrying an offset or low-frequency content stayed collapsed into a single Hadamard coefficient - a constant block round-tripped at cosine 0.611 with a relative L2 error of 1.27, i.e. worse than returning zeros. Also make the test suite runnable (it errored out entirely, because pytest imports the repository root __init__.py as a top-level module and the relative imports failed) and stop the attention patch from raising AttributeError on ComfyUI's attn2 path, which does not coerce a None context the way attn1 does. Co-authored-by: Vyacheslav-Tomashevskiy <Vyacheslav-Tomashevskiy@users.noreply.github.com>
24 lines
827 B
Python
24 lines
827 B
Python
# SPDX-License-Identifier: MIT
|
|
"""
|
|
ComfyUI-TurboQuant: TQ3 KV Cache Compression
|
|
|
|
Reduces attention KV cache VRAM by ~4.5x using 3-bit Lloyd-Max
|
|
quantization with Fast Walsh-Hadamard Transform decorrelation.
|
|
|
|
Nodes:
|
|
- TurboQuant KV Patch: Patches any model for compressed KV cache
|
|
- TurboQuant Info: Shows compression stats
|
|
"""
|
|
|
|
try:
|
|
# Normal case: loaded as a package from ComfyUI/custom_nodes.
|
|
from .turboquant_nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
|
|
except ImportError:
|
|
# Loaded as a top-level module (pytest imports this file as "__init__"
|
|
# because the repository root is a package, and flat loaders do the same).
|
|
from turboquant_nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
|
|
|
|
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
|
|
|
|
WEB_DIRECTORY = None
|