CthomasdesignandClaude Opus 4.8
1db2179abe
Groundwork for image-grounded conditioning (rung B). Vendors mlx-vlm's pure-MLX Qwen3-VL vision tower into krea2_engine/vision/ (self-contained: inlined SDPA helper, local VisionConfig) and adds torch-free image preprocessing (smart-resize + CLIP normalize + patch flatten). Both validated against the PyTorch reference (tools/test_vision_tower.py): - tower vs transformers Qwen3VLVisionModel: merged embeds + all 3 deepstack features match to max|Δ|~2e-4, cosine 1.0 (bf16 rounding). - preprocessing vs transformers image processor: max|Δ|~2e-7, grid exact. Loads the visual.* weights already present in the Krea-2-Turbo text encoder (315 tensors, 0 missing). Not yet wired into any node — the grounded-encode integration (M-RoPE, deepstack injection, 12-layer tap) is the next step. mlx-vlm is MIT; attribution added to NOTICE. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
7 lines
196 B
Python
7 lines
196 B
Python
"""Vendored Qwen3-VL vision tower (from Blaizzy/mlx-vlm, MIT — see ../../NOTICE)."""
|
|
|
|
from .config import VisionConfig
|
|
from .vision import VisionModel
|
|
|
|
__all__ = ["VisionConfig", "VisionModel"]
|