Claude
b193b6b0b1
Ports the sharpest-frame selection from scanpe_experiments/h3_image.py, which takes it from IF-Edit (arXiv:2511.19435): decoded frames carry different amounts of residual motion blur, so score by Laplacian variance and keep the crispest instead of a fixed index. Pairs with running MiniMax-H3 as an image generator: the VAE's minimum clip is 5 pixel frames (2 latents), decode yields frames of uneven sharpness, and the best one is not always at the same index. The node documents a limitation found while testing it, because it is easy to misuse: Laplacian variance is a BLUR heuristic, not a quality metric -- artifacts are high-frequency too. On a real H3 decode it picked the frame with the WORST PSNR of its block (19.72 dB against 23.19 for its neighbour) because that frame was the most artifacted. It is sound when the candidates are the same content differing only in blur, which is the IF-Edit setting; it is meaningless for choosing among frames from different moments. Also relevant and not obvious: the "last frame of each decoded block is the valid one" rule (decoded[:, :, -1] in the upstream example) held for the first block here (34.49 dB against 26.80 for its neighbour) but inverted on the second. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
63 lines
2.9 KiB
Python
63 lines
2.9 KiB
Python
"""Escolhe o frame mais nitido de um lote, por variancia do Laplaciano.
|
|
|
|
Vem do IF-Edit (arXiv:2511.19435): frames decodificados carregam quantidades diferentes de
|
|
borrao residual, entao pontuar e ficar com o mais crocante bate pegar um indice fixo.
|
|
|
|
Util em particular com o MiniMax-H3 usado como gerador de imagem: o clipe minimo do VAE
|
|
(5 frames = 2 latentes) sai do decode com nitidez desigual, e o melhor frame nao e sempre
|
|
o mesmo indice.
|
|
|
|
CUIDADO -- e uma heuristica de BORRAO, nao uma metrica de qualidade. Variancia do
|
|
Laplaciano mede energia de alta frequencia, e artefato tambem e alta frequencia. Medido
|
|
num decode do H3: entre 4 frames do mesmo bloco, o Laplaciano escolheu o de PIOR PSNR
|
|
(19.72 dB contra 23.19 do vizinho), porque ele era o mais artefatado. Use quando os
|
|
candidatos forem o MESMO conteudo diferindo so em borrao -- que e o caso do IF-Edit e do
|
|
clipe estatico gerado. Nao use para escolher entre frames de momentos diferentes.
|
|
"""
|
|
|
|
import torch
|
|
|
|
|
|
def _laplacian_variance(img: torch.Tensor) -> float:
|
|
"""img: [H, W, C] em [0,1]. Variancia do Laplaciano 4-vizinhos sobre a luminancia."""
|
|
weights = torch.tensor([0.299, 0.587, 0.114], device=img.device, dtype=torch.float32)
|
|
g = (img[..., :3].float() * weights).sum(-1)
|
|
lap = (
|
|
-4.0 * g[1:-1, 1:-1]
|
|
+ g[:-2, 1:-1] + g[2:, 1:-1] + g[1:-1, :-2] + g[1:-1, 2:]
|
|
)
|
|
return float(lap.var())
|
|
|
|
|
|
class SharpestFrame:
|
|
"""Recebe um lote de imagens e devolve a mais nitida, com o indice e as notas."""
|
|
|
|
@classmethod
|
|
def INPUT_TYPES(cls):
|
|
return {
|
|
"required": {
|
|
"images": ("IMAGE", {"tooltip": "Lote de frames que sejam o MESMO conteudo diferindo em borrao (ex.: clipe estatico do H3 em modo imagem). Para frames de momentos diferentes isso nao faz sentido."}),
|
|
},
|
|
"optional": {
|
|
"skip_first": ("INT", {"default": 0, "min": 0, "max": 64, "tooltip": "Ignora os N primeiros frames antes de pontuar. No H3 os primeiros frames de cada bloco decodificado costumam ser preenchimento temporal."}),
|
|
},
|
|
}
|
|
|
|
RETURN_TYPES = ("IMAGE", "INT", "STRING")
|
|
RETURN_NAMES = ("image", "index", "scores")
|
|
FUNCTION = "pick"
|
|
CATEGORY = "BFS/image"
|
|
|
|
def pick(self, images, skip_first=0):
|
|
total = images.shape[0]
|
|
start = min(skip_first, max(total - 1, 0))
|
|
scores = [_laplacian_variance(images[i]) for i in range(start, total)]
|
|
best = start + max(range(len(scores)), key=scores.__getitem__)
|
|
texto = " | ".join(f"{start + i}:{s:.5f}{' <-' if start + i == best else ''}" for i, s in enumerate(scores))
|
|
print(f"[BFSNodes] SharpestFrame: {total} frames, escolhido {best} -> {texto}")
|
|
return (images[best:best + 1], best, texto)
|
|
|
|
|
|
NODE_CLASS_MAPPINGS = {"SharpestFrame": SharpestFrame}
|
|
NODE_DISPLAY_NAME_MAPPINGS = {"SharpestFrame": "Sharpest Frame / Laplacian (BFS)"}
|