Merge dev into main - Release 3.4.0
This commit is contained in:
digital-garbage
@@ -1,5 +1,69 @@
|
||||
# Changelog
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [3.4.0] - 2026-07-24
|
||||
|
||||
### Added
|
||||
- **Easy Gen** (the simplified single-scene UI) reaches near feature-parity with the Cutting
|
||||
Room for the workflows it's meant for:
|
||||
- Shortcut/transition JSON library **import & export** in Settings ▸ Shortcuts (merge or
|
||||
replace) — Easy Gen previously had no way to load a shortcut/transition library at all.
|
||||
- The full **Engine settings** panel (Studio refinement/adjustments/sampler algorithm,
|
||||
Chain Sampler continuity/timing/guidance/decode/experimental, Best-FaceID identity
|
||||
transfer) — restored via the same shared frontend module the Cutting Room uses.
|
||||
- A **Log** button (ComfyUI backend log, same as the Cutting Room's).
|
||||
- An explicit **Save** button, and a **Gallery** picker to reuse already-uploaded media as
|
||||
the i2v anchor without re-uploading — with a one-click **continuity identity pin**
|
||||
button right on each thumbnail, mirroring the Cutting Room's Media Bin.
|
||||
- An **Export…** button to download the project as a `.funpack_project.json` file (e.g. to
|
||||
move a project off a rental GPU box) — the backend route already existed but had no UI.
|
||||
|
||||
### Changed
|
||||
- **Loop temporal style now works alongside guides.** Scenes carrying appended guide frames
|
||||
(carry-i2v-guides, mid-scene guide, JoyAI memory, custom guide stacks) no longer disable
|
||||
the loop roll: only the content region rolls while the pinned guide tail (and the audio
|
||||
memory tail) stays canonical, so guides keep informing every position of the cycle without
|
||||
ever being rendered into it. The i2v anchor was always supported - it rides the denoise
|
||||
mask, which rolls in step with the latent.
|
||||
- **Easy Gen's Studio always runs in Prompt-only mode.** Easy Gen has no rating UI, so every
|
||||
Engine setting that's a no-op without a trained refinement key/rated history (refinement
|
||||
key, value guidance, steer mode, reference injection, embed/score/output guidance, taste
|
||||
retrieval, DynaShift, sampler-panel velocity bias/rescue) is hidden there and enforced at
|
||||
the pipeline level, not just the UI — every Easy Gen generate call forces Studio's `mode`
|
||||
to "Prompt only" and the rating-gated Chain Sampler knobs off via node overrides, so a
|
||||
project configured with these on in the Cutting Room still generates plainly from Easy
|
||||
Gen. A note points to the Cutting Room or the ComfyUI graph for the full learned refiner.
|
||||
- The "expose to editor" eye buttons in Models & Pipeline are hidden under Easy Gen — there's
|
||||
no per-scene inspector for an exposed control to ever surface in, so they were inert
|
||||
clutter that could look actionable and silently do nothing.
|
||||
|
||||
### Fixed
|
||||
- The shortcut autocomplete menu could open off-screen below the prompt field (Easy Gen's
|
||||
prompt box sits in a footer bar at the bottom of the viewport) with no way to see or click
|
||||
a suggestion. Now flips above the field when there isn't room below.
|
||||
- A node's **bypass** toggle could silently do nothing: when the pass-through mapping was
|
||||
ambiguous, or the bypassed node's own matching input wasn't wired to anything, the failure
|
||||
was only ever recorded in a report field no frontend code read or displayed — the node
|
||||
stayed fully active with no visible error. Generation now blocks with a clear reason
|
||||
instead of a silent no-op.
|
||||
- Easy Gen could silently revert Models & Pipeline edits (bypass, exposed widgets, LoRA
|
||||
settings): any later project save sent a stale in-memory copy of the models config back to
|
||||
the server, clobbering whatever was just changed there independently.
|
||||
- Easy Gen's preview restarted playback (ignoring a pause) on every background health poll
|
||||
or prompt keystroke, since each one rebuilt the video element from scratch.
|
||||
- A "fetching process...aborted by the user agent" unhandled-rejection error banner could
|
||||
appear when switching between Gallery/generated media in Easy Gen.
|
||||
- Downloaded generation results always suggested "result.mp4" as the filename regardless of
|
||||
the actual render (no `Content-Disposition` on the `/result` route) — the underlying file
|
||||
was already uniquely named per run and already lived in ComfyUI's temp dir, just wasn't
|
||||
surfaced to the browser's save dialog.
|
||||
- Easy Gen's Generation settings showed a "Seed" field always reading "1" and looking fixed.
|
||||
It was bound to the Project's legacy `seed` field, which nothing in generation has ever
|
||||
read (the real seed resolution only reads `sampler_inputs.seed`, already correctly random
|
||||
when unset) — removed the dead field in favor of the working one under Engine ▸ Timing &
|
||||
Seed, which already says so.
|
||||
|
||||
## [3.3.0] - 2026-07-15
|
||||
|
||||
### Added
|
||||
|
||||
@@ -15,9 +15,20 @@
|
||||
|
||||
## What it looks like
|
||||
|
||||
FunPack 3.x ships the **Cutting Room** - a full non-linear video editor living in your browser, inside ComfyUI. A real timeline with per-scene prompts, ratings and audio lanes, a program monitor, a media bin, and one-click generation driving FunPack Studio + the Scene Chain Sampler - no graph wiring required.
|
||||
Opening `/funpack/` inside ComfyUI lands you on the **Hub** - a small picker between FunPack's two browser UIs, plus at-a-glance version/update status.
|
||||
|
||||
|
||||
|
||||
|
||||
### Two UIs, one engine
|
||||
|
||||
Both UIs drive the exact same generation engine (FunPack Studio + the LTXAV Scene Chain Sampler) and the exact same project files - a project opened in one opens unchanged in the other. Pick based on how much control you want in front of you:
|
||||
|
||||
- **Cutting Room** - the full multi-scene non-linear editor: a real timeline with per-scene prompts, ratings and audio lanes, a program monitor, a media bin, and every Engine/Studio knob a click away. This is where rating-driven learning (refinement keys, value guidance, taste steering) lives.
|
||||
- **Easy Gen** - one prompt, one Generate button, one big preview. Built for quick single-scene (or simple multi-scene via prompt splits) generations without the full NLE around it - advanced settings are still there, just tucked behind an "Advanced settings" link, and rating-dependent controls are hidden entirely since there's no rating UI to feed them (Studio always runs in a plain Prompt-only mode there).
|
||||
|
||||
| Cutting Room - timeline, preview monitor, media bin, and scene inspector | Easy Gen - one prompt, one Generate button, one big preview |
|
||||
| --- | --- |
|
||||
|  |  |
|
||||
|
||||
| The Composer - global prompt, shortcuts, split markers, `$variables` | Engine Settings - Studio + Chain Sampler knobs without touching a graph |
|
||||
| --- | --- |
|
||||
@@ -56,6 +67,13 @@ Montage** trailer builder, a **Temp files** browser, **reconnect-after-reload**
|
||||
generations, and a Blackwell (sm_120) guide-scene crash fix — see the [CHANGELOG](CHANGELOG.md)
|
||||
for the full list.
|
||||
|
||||
**3.4.0** brings **Easy Gen** to near feature-parity with the Cutting Room for the workflows
|
||||
it targets: shortcut/transition library import/export, the full Engine settings panel
|
||||
(including Best-FaceID), a Gallery picker with a one-click continuity pin, Save/Export buttons,
|
||||
and several bugfixes (a bypass toggle that could silently do nothing, a state-sync bug that
|
||||
could revert Models & Pipeline edits, and more) — see the [CHANGELOG](CHANGELOG.md) for the
|
||||
full list.
|
||||
|
||||
See [`docs/MovieEditor.md`](docs/MovieEditor.md) for complete Cutting Room documentation.
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -26,10 +26,18 @@ if __package__:
|
||||
from . import batch_training # noqa: F401 registers /funpack/batch/* routes
|
||||
except Exception as _e:
|
||||
print(f"[FunPack] batch_training routes unavailable: {_e}")
|
||||
try:
|
||||
from . import hub # noqa: F401 registers /funpack/ (the UI picker)
|
||||
except Exception as _e:
|
||||
print(f"[FunPack] Hub route unavailable: {_e}")
|
||||
try:
|
||||
from .movie_editor import server as _movie_editor_server # noqa: F401 registers /funpack/movie/* routes
|
||||
except Exception as _e:
|
||||
print(f"[FunPack] Movie Editor routes unavailable: {_e}")
|
||||
try:
|
||||
from . import easy_gen # noqa: F401 registers /funpack/easy/* routes
|
||||
except Exception as _e:
|
||||
print(f"[FunPack] Easy Gen routes unavailable: {_e}")
|
||||
else:
|
||||
# Standalone tests may not have the full ComfyUI/CUDA runtime loaded.
|
||||
from conditioning import (
|
||||
|
||||
+21
-2
@@ -6484,10 +6484,10 @@ class FunPackVideoRefinerV2(FunPackVideoRefiner):
|
||||
"default": "",
|
||||
"tooltip": "Optional user feedback describing what was specifically wrong with the previous output (e.g. 'he was supposed to hold her hand, not her head'). Has highest priority in the advisor system prompt.",
|
||||
}),
|
||||
"temporal_style": (["natural", "auto", "accelerate", "decelerate", "loop", "freeze", "pulse"], {
|
||||
"temporal_style": (["natural", "auto", "accelerate", "decelerate", "loop", "freeze", "pulse", "rapid_start", "rapid_end", "rapid_start_end"], {
|
||||
"default": "natural",
|
||||
"label": "Temporal Style",
|
||||
"tooltip": "Controls how the model perceives motion timing via frame_rate RoPE manipulation. natural=no change, auto=per-scene director picks motion energy from each scene's prompt (needs the Scene Chain Sampler), accelerate=faster motion, decelerate=heavier motion, loop=circular temporal coords, freeze=highly compressed time, pulse=repeated ease-down motion holds per scene (needs the Scene Chain Sampler).",
|
||||
"tooltip": "Controls how the model perceives motion timing via frame_rate RoPE manipulation. natural=no change, auto=per-scene director picks motion energy from each scene's prompt (needs the Scene Chain Sampler), accelerate=faster motion, decelerate=heavier motion, loop=circular temporal coords, freeze=highly compressed time, pulse=repeated ease-down motion holds per scene (needs the Scene Chain Sampler), rapid_start=speeds up just the beginning of each scene then settles to natural, rapid_end=natural then speeds up just the ending, rapid_start_end=speeds up both ends with natural motion in the middle.",
|
||||
}),
|
||||
"split_by_transitions": ("BOOLEAN", {
|
||||
"default": False,
|
||||
@@ -13355,6 +13355,24 @@ class FunPackVideoRefinerV2(FunPackVideoRefiner):
|
||||
out.append([cond, meta] if isinstance(entry, list) else (cond, meta))
|
||||
return out
|
||||
|
||||
def _v2_apply_rapid_temporal(self, conditioning_list, temporal_style):
|
||||
"""temporal_style in (rapid_start, rapid_end, rapid_start_end): tag each scene
|
||||
entry so the Scene Chain Sampler installs a rapid-edge ease temporal wrapper
|
||||
per scene (mirrors _v2_apply_pulse_temporal)."""
|
||||
style = str(temporal_style or "natural").strip().lower()
|
||||
if style not in ("rapid_start", "rapid_end", "rapid_start_end"):
|
||||
return conditioning_list
|
||||
out = []
|
||||
for entry in conditioning_list or []:
|
||||
if not (isinstance(entry, (list, tuple)) and len(entry) >= 2 and isinstance(entry[1], dict)):
|
||||
out.append(entry)
|
||||
continue
|
||||
cond, meta = entry[0], dict(entry[1])
|
||||
meta["funpack_temporal_mode"] = style
|
||||
meta["funpack_temporal_label"] = style
|
||||
out.append([cond, meta] if isinstance(entry, list) else (cond, meta))
|
||||
return out
|
||||
|
||||
def _v2_apply_auto_temporal(self, conditioning_list, temporal_style, fallback_text=""):
|
||||
"""temporal_style="auto": bake a per-scene frame_rate multiplier into each
|
||||
conditioning entry's metadata so the Scene Chain Sampler can apply per-scene motion
|
||||
@@ -13452,6 +13470,7 @@ class FunPackVideoRefinerV2(FunPackVideoRefiner):
|
||||
_v2_apply_scene_refinement_keys); scenes without bound keys keep the project key."""
|
||||
mode = str(steer_mode or "relative").lower()
|
||||
out = self._v2_apply_pulse_temporal(conditioning_list, temporal_style)
|
||||
out = self._v2_apply_rapid_temporal(out, temporal_style)
|
||||
out = self._v2_apply_auto_temporal(out, temporal_style, fallback_text=temporal_fallback_text)
|
||||
out = self._v2_apply_bounded_attention(out, clip, fallback_text=temporal_fallback_text, encode_cache=encode_cache)
|
||||
if mode in ("relative", "both"):
|
||||
|
||||
+505
@@ -0,0 +1,505 @@
|
||||
"""Segmented detailing: ADetailer-style region refine for the Scene Chain sampler.
|
||||
|
||||
After a scene finishes denoising, CLIPSeg (text-prompted segmentation, the same
|
||||
model family face-swap stacks use for occlusion masking) locates user-named
|
||||
regions ("hands", "feet", ...) on a few decoded keyframes. The matched region is
|
||||
cut out of the video latent as a spatiotemporal tube, pushed through Lightricks'
|
||||
own trained LatentUpsampler (2x spatial, latent space - the official two-stage
|
||||
pipeline's stage-2 model), re-noised to the official stage-2 re-entry sigma and
|
||||
re-denoised for the tail schedule, then downscaled back to its original latent
|
||||
size and pasted through the feathered CLIPSeg soft mask. Final resolution never
|
||||
changes: the upsample exists only so the model can resolve structure (fingers,
|
||||
edges) at a higher working resolution before the crop returns to the grid.
|
||||
|
||||
Boundary law: this is a visual-behavior op, so it lives on the Chain Sampler.
|
||||
Audio is never modified - the refine pass carries the scene's audio stream so
|
||||
joint attention sees a legal AV latent, but its audio output is discarded.
|
||||
|
||||
No-op guarantee: empty targets, no upsampler, or no detection above threshold
|
||||
returns the input latent object untouched (bit-identical run).
|
||||
"""
|
||||
|
||||
import math
|
||||
|
||||
import torch
|
||||
import torch.nn.functional as F
|
||||
|
||||
# Official LTX-2.3 two-stage distilled workflow, stage-2 ManualSigmas: re-noise the
|
||||
# upsampled latent to 0.85 and run the tail of the 8-step distilled schedule.
|
||||
# Lightricks designed this re-entry point to SHARPEN already-correct structure at
|
||||
# higher resolution, not to REPAIR a malformed region — our use case (a mangled
|
||||
# hand) needs more genuine freedom to redraw than their use case (a clean shot
|
||||
# that just needs finer detail). 0.85 is the built-in default (matches the
|
||||
# official recipe / prior behavior exactly), but it is a real knob
|
||||
# (detail_denoise on the sampler) precisely because the right value depends on
|
||||
# how wrong the source region already is: too high and the region drifts from
|
||||
# its surroundings; too low and the tail just smooths the upsampled
|
||||
# interpolation back down without adding anything (upscaled-looking, not
|
||||
# detailed - the failure mode this constant existed to describe from day one).
|
||||
DEFAULT_RENOISE_SIGMA = 0.85
|
||||
# Shape of the tail preserved as ratios of the entry sigma, not fixed values, so
|
||||
# raising/lowering the entry point scales the whole tail schedule consistently
|
||||
# instead of only moving its first step.
|
||||
_STAGE2_RATIOS = (1.0, 0.725 / 0.85, 0.421875 / 0.85, 0.0)
|
||||
|
||||
|
||||
def stage2_sigmas(renoise_sigma=DEFAULT_RENOISE_SIGMA):
|
||||
"""The re-denoise tail schedule for a given re-entry sigma.
|
||||
|
||||
stage2_sigmas(0.85) reproduces the exact official STAGE2_SIGMAS values.
|
||||
"""
|
||||
r = max(0.0, min(1.0, float(renoise_sigma)))
|
||||
return tuple(r * ratio for ratio in _STAGE2_RATIOS)
|
||||
|
||||
|
||||
# Kept for anyone importing the old name directly (tests, prior callers) - identical
|
||||
# to stage2_sigmas() at the default re-entry point.
|
||||
STAGE2_SIGMAS = stage2_sigmas(DEFAULT_RENOISE_SIGMA)
|
||||
|
||||
# CLIPSeg heat above this (post-sigmoid) counts as "the named thing is here".
|
||||
DEFAULT_THRESHOLD = 0.35
|
||||
|
||||
# Default ceiling on tube area, as a fraction of the frame - purely a COST guardrail
|
||||
# (cost ~= 4x area fraction x tail steps, so a big tube starts to rival a second full
|
||||
# render) exposed as detail_max_area so the user decides the cost/quality trade-off,
|
||||
# not this constant. Never silently override what the user asked to detail.
|
||||
MAX_TUBE_AREA = 0.35
|
||||
|
||||
# Latent cells of context padding around the thresholded region, and the minimum
|
||||
# tube edge. Padding gives the refine pass surrounding context to blend against;
|
||||
# the minimum keeps the crop a legal latent for conv stacks and attention.
|
||||
TUBE_PAD = 2
|
||||
MIN_TUBE_EDGE = 4
|
||||
|
||||
# Keyframes decoded per scene for detection. Single latent frames decode cheaply
|
||||
# through the causal VAE (treated as stills) - enough for a temporal-union tube.
|
||||
DETECTION_FRAMES = 3
|
||||
|
||||
# Cond keys that tie a scene's conditioning to the full-frame latent layout
|
||||
# (i2v anchors / guide keyframes). The crop is its own small latent with no
|
||||
# appended guide frames, so these must not travel into the refine pass.
|
||||
_LAYOUT_COND_KEYS = ("keyframe_idxs", "guiding_latent", "concat_latent_image", "concat_mask")
|
||||
|
||||
_CLIPSEG = None # (processor, model), lazy - loaded on first enabled run only
|
||||
|
||||
|
||||
def _log(debug, msg):
|
||||
if debug:
|
||||
print(f"[FunPackDetail] {msg}")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Models
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Official Lightricks stage-2 spatial upscaler — the exact file the two-stage
|
||||
# distilled workflows use. ~1 GB, fetched once into models/latent_upscale_models
|
||||
# when nothing suitable is installed.
|
||||
DEFAULT_UPSAMPLER_REPO = "Lightricks/LTX-2.3"
|
||||
DEFAULT_UPSAMPLER_FILE = "ltx-2.3-spatial-upscaler-x2-1.1.safetensors"
|
||||
|
||||
|
||||
def resolve_upsampler_name(model_name=None):
|
||||
"""Turn the node's detail_upsampler widget value into a loadable filename.
|
||||
|
||||
Explicit filename -> returned as-is. "auto" (and the legacy "None" default)
|
||||
-> prefer an installed spatial upscaler (highest version wins), else any
|
||||
installed latent upscale model, else download the official file from HF.
|
||||
Raises with a user-actionable message when nothing can be obtained — callers
|
||||
surface it, they must NOT silently skip.
|
||||
"""
|
||||
import os
|
||||
|
||||
import folder_paths
|
||||
|
||||
name = str(model_name or "").strip()
|
||||
if name and name not in ("None", "auto"):
|
||||
return name
|
||||
files = folder_paths.get_filename_list("latent_upscale_models")
|
||||
if DEFAULT_UPSAMPLER_FILE in files:
|
||||
return DEFAULT_UPSAMPLER_FILE
|
||||
spatial = [f for f in files if "spatial" in f.lower() and "upscal" in f.lower()]
|
||||
if spatial:
|
||||
# Highest embedded version wins (…-1.1 over …-1.0); plain lexicographic would
|
||||
# rank "ltx2_3_…" above "ltx-2.3-…-1.1" ('2' sorts after '-').
|
||||
import re
|
||||
return max(spatial, key=lambda f: ([int(n) for n in re.findall(r"\d+", f)], f))
|
||||
if files:
|
||||
return files[0]
|
||||
try:
|
||||
from huggingface_hub import hf_hub_download
|
||||
except ImportError as exc:
|
||||
raise RuntimeError(
|
||||
"no model in models/latent_upscale_models and huggingface_hub is not "
|
||||
f"installed — pip install huggingface_hub, or download {DEFAULT_UPSAMPLER_FILE} "
|
||||
f"from {DEFAULT_UPSAMPLER_REPO} into that folder manually") from exc
|
||||
dest_dir = folder_paths.get_folder_paths("latent_upscale_models")[0]
|
||||
os.makedirs(dest_dir, exist_ok=True)
|
||||
print(f"[FunPackDetail] no latent upscale model installed — downloading "
|
||||
f"{DEFAULT_UPSAMPLER_FILE} (~1 GB) from {DEFAULT_UPSAMPLER_REPO} into {dest_dir} …")
|
||||
try:
|
||||
hf_hub_download(repo_id=DEFAULT_UPSAMPLER_REPO, filename=DEFAULT_UPSAMPLER_FILE,
|
||||
local_dir=dest_dir)
|
||||
except Exception as exc:
|
||||
raise RuntimeError(
|
||||
f"auto-download of {DEFAULT_UPSAMPLER_FILE} failed ({exc}) — if the repo is "
|
||||
"gated, accept the license on huggingface.co and set HF_TOKEN, or place the "
|
||||
f"file in {dest_dir} manually") from exc
|
||||
print(f"[FunPackDetail] download complete: {DEFAULT_UPSAMPLER_FILE}")
|
||||
return DEFAULT_UPSAMPLER_FILE
|
||||
|
||||
|
||||
def load_latent_upsampler(model_name):
|
||||
"""Load Lightricks' LatentUpsampler from models/latent_upscale_models.
|
||||
|
||||
Mirrors the LTX branch of comfy's LatentUpscaleModelLoader (safe_load +
|
||||
config from safetensors metadata); the architecture itself ships in comfy
|
||||
core (comfy.ldm.lightricks.latent_upsampler), so there is nothing to port.
|
||||
"""
|
||||
import json
|
||||
|
||||
import comfy.model_management
|
||||
import comfy.utils
|
||||
import folder_paths
|
||||
from comfy.ldm.lightricks.latent_upsampler import LatentUpsampler
|
||||
|
||||
model_path = folder_paths.get_full_path_or_raise("latent_upscale_models", model_name)
|
||||
sd, metadata = comfy.utils.load_torch_file(model_path, safe_load=True, return_metadata=True)
|
||||
if "post_upsample_res_blocks.0.conv2.bias" not in sd:
|
||||
raise ValueError(
|
||||
f"{model_name} is not an LTX latent upsampler (expected post_upsample_res_blocks keys).")
|
||||
config = json.loads(metadata["config"])
|
||||
model = LatentUpsampler.from_config(config).to(
|
||||
dtype=comfy.model_management.vae_dtype(allowed_dtypes=[torch.bfloat16, torch.float32]))
|
||||
model.load_state_dict(sd)
|
||||
model.eval()
|
||||
return model
|
||||
|
||||
|
||||
def _get_clipseg():
|
||||
"""CLIPSeg processor+model, loaded once and kept on CPU.
|
||||
|
||||
CPU on purpose: ~150M params, a handful of 352px frames per scene - the cost
|
||||
is milliseconds against a video denoise, and it never competes with the
|
||||
diffusion model for VRAM. Loaded straight from transformers (which comfy
|
||||
already ships); no GPL face-swap code involved.
|
||||
"""
|
||||
global _CLIPSEG
|
||||
if _CLIPSEG is None:
|
||||
from transformers import CLIPSegForImageSegmentation, CLIPSegProcessor
|
||||
|
||||
name = "CIDAS/clipseg-rd64-refined"
|
||||
processor = CLIPSegProcessor.from_pretrained(name)
|
||||
model = CLIPSegForImageSegmentation.from_pretrained(name)
|
||||
model.eval()
|
||||
_CLIPSEG = (processor, model)
|
||||
return _CLIPSEG
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Detection
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def parse_targets(detail_targets):
|
||||
"""Comma-separated user text -> clean prompt list ('' -> [])."""
|
||||
if not detail_targets:
|
||||
return []
|
||||
return [t.strip() for t in str(detail_targets).split(",") if t.strip()]
|
||||
|
||||
|
||||
def _decode_detection_frames(vae, video, count, debug=False):
|
||||
"""Decode `count` evenly spaced single latent frames to [H, W, C] float images.
|
||||
|
||||
Each latent frame is decoded alone as [B, C, 1, h, w]: the causal VAE treats
|
||||
a length-1 clip as a still, which is exactly good enough for segmentation and
|
||||
avoids decoding the whole scene just to find a region.
|
||||
"""
|
||||
frames = []
|
||||
f = int(video.shape[2])
|
||||
idxs = sorted({min(f - 1, round(i * (f - 1) / max(1, count - 1))) for i in range(max(1, count))})
|
||||
for i in idxs:
|
||||
try:
|
||||
decoded = vae.decode(video[:, :, i:i + 1])
|
||||
except Exception as exc:
|
||||
_log(debug, f"keyframe {i} decode failed ({exc}); skipping")
|
||||
continue
|
||||
img = decoded
|
||||
if img.dim() == 5: # [B, T, H, W, C]
|
||||
img = img[0, 0]
|
||||
elif img.dim() == 4: # [B, H, W, C]
|
||||
img = img[0]
|
||||
frames.append((i, img.detach().float().clamp(0.0, 1.0).cpu()))
|
||||
return frames
|
||||
|
||||
|
||||
def _clipseg_heat(frames, prompts, debug=False):
|
||||
"""Max-over-frames, max-over-prompts CLIPSeg heat map.
|
||||
|
||||
Returns [H, W] in [0, 1] at the first frame's pixel size, or None when
|
||||
nothing was decodable. Max (not mean) across frames on purpose: the tube
|
||||
must cover the region wherever it appears during the scene.
|
||||
"""
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
if not frames or not prompts:
|
||||
return None
|
||||
processor, model = _get_clipseg()
|
||||
h, w = frames[0][1].shape[0], frames[0][1].shape[1]
|
||||
heat = None
|
||||
with torch.no_grad():
|
||||
for _, img in frames:
|
||||
pil = Image.fromarray((img.numpy() * 255.0).astype(np.uint8))
|
||||
# padding=True is mandatory once len(prompts) > 1: CLIPSeg's tokenizer
|
||||
# otherwise returns one variable-length id list per prompt and torch can't
|
||||
# stack them into a batch ("excessive nesting... type list where int is
|
||||
# expected"). truncation=True guards the rare very-long user-typed target.
|
||||
inputs = processor(
|
||||
text=list(prompts), images=[pil] * len(prompts), return_tensors="pt",
|
||||
padding=True, truncation=True)
|
||||
logits = model(**inputs).logits # [P, 352, 352] (or [352, 352] for one prompt)
|
||||
if logits.dim() == 2:
|
||||
logits = logits[None]
|
||||
frame_heat = torch.sigmoid(logits).amax(dim=0)[None, None] # union over prompts
|
||||
frame_heat = F.interpolate(frame_heat, size=(h, w), mode="bilinear", align_corners=False)[0, 0]
|
||||
heat = frame_heat if heat is None else torch.maximum(heat, frame_heat)
|
||||
return heat
|
||||
|
||||
|
||||
def find_tube(heat, latent_h, latent_w, threshold=DEFAULT_THRESHOLD,
|
||||
area_cap=MAX_TUBE_AREA, debug=False, diag=None):
|
||||
"""Threshold pixel-space heat into a latent-space tube box + soft paste mask.
|
||||
|
||||
Returns (y0, y1, x0, x1, mask) in latent coords - mask is [1, 1, 1, th, tw]
|
||||
matching the tube, feathered from the CLIPSeg heat itself so the paste
|
||||
follows the actual silhouette instead of the rectangle. None when nothing
|
||||
crossed the threshold or the tube breached the area cap.
|
||||
|
||||
`diag`, when given a dict, is filled with why a None came back (max_heat
|
||||
always; reason + area on a miss) so a caller can report something more
|
||||
useful than "nothing found" - CLIPSeg's raw sigmoid score for a real,
|
||||
correctly-named region is often well under a naive 0.5, so a silent miss
|
||||
is indistinguishable from "the threshold is simply too high" without it.
|
||||
"""
|
||||
if heat is None:
|
||||
return None
|
||||
# Heat -> latent grid first; box math happens where the crop happens.
|
||||
lat_heat = F.interpolate(heat[None, None], size=(latent_h, latent_w),
|
||||
mode="bilinear", align_corners=False)[0, 0]
|
||||
max_heat = float(lat_heat.max())
|
||||
if diag is not None:
|
||||
diag["max_heat"] = max_heat
|
||||
hot = lat_heat >= threshold
|
||||
if not bool(hot.any()):
|
||||
_log(debug, f"no region above threshold {threshold} (max heat seen: {max_heat:.3f})")
|
||||
if diag is not None:
|
||||
diag["reason"] = "below_threshold"
|
||||
return None
|
||||
ys, xs = torch.where(hot)
|
||||
y0 = max(0, int(ys.min()) - TUBE_PAD)
|
||||
y1 = min(latent_h, int(ys.max()) + 1 + TUBE_PAD)
|
||||
x0 = max(0, int(xs.min()) - TUBE_PAD)
|
||||
x1 = min(latent_w, int(xs.max()) + 1 + TUBE_PAD)
|
||||
# Enforce a minimum edge (grow symmetrically, clamped to the frame).
|
||||
while (y1 - y0) < min(MIN_TUBE_EDGE, latent_h):
|
||||
y0, y1 = max(0, y0 - 1), min(latent_h, y1 + 1)
|
||||
while (x1 - x0) < min(MIN_TUBE_EDGE, latent_w):
|
||||
x0, x1 = max(0, x0 - 1), min(latent_w, x1 + 1)
|
||||
area = (y1 - y0) * (x1 - x0) / float(latent_h * latent_w)
|
||||
if area > area_cap:
|
||||
_log(debug, f"tube area {area:.0%} exceeds cap {area_cap:.0%}; refusing "
|
||||
"(a region that large is a re-render, not a detail pass)")
|
||||
if diag is not None:
|
||||
diag["reason"] = "area_cap"
|
||||
diag["area"] = area
|
||||
return None
|
||||
# Soft mask: the heat inside the tube, floored at 0 outside the threshold and
|
||||
# lightly blurred so the seam never lands on a hard edge.
|
||||
m = lat_heat[y0:y1, x0:x1].clone()
|
||||
m = ((m - threshold) / max(1e-6, 1.0 - threshold)).clamp(0.0, 1.0)
|
||||
m = F.avg_pool2d(m[None, None], kernel_size=3, stride=1, padding=1)[0, 0]
|
||||
return y0, y1, x0, x1, m[None, None, None]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Refine
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _strip_layout_conds(conds):
|
||||
"""Copy conditioning minus full-frame layout keys (anchors/guides).
|
||||
|
||||
The crop refine is its own small denoise: text still applies, but keyframe
|
||||
indices and guide latents describe positions in the full-frame latent and
|
||||
would fight (or crash) the crop-sized pass.
|
||||
"""
|
||||
out = []
|
||||
for entry in conds or []:
|
||||
if isinstance(entry, (list, tuple)) and len(entry) == 2 and isinstance(entry[1], dict):
|
||||
meta = {k: v for k, v in entry[1].items() if k not in _LAYOUT_COND_KEYS}
|
||||
out.append([entry[0], meta])
|
||||
else:
|
||||
out.append(entry)
|
||||
return out
|
||||
|
||||
|
||||
def _per_channel_stats(vae):
|
||||
return getattr(getattr(vae, "first_stage_model", None), "per_channel_statistics", None)
|
||||
|
||||
|
||||
def _run_upsampler(upsampler, video_crop, vae, debug=False):
|
||||
"""un_normalize -> LatentUpsampler(2x spatial) -> normalize, on the crop.
|
||||
|
||||
Same recipe as comfy's LTXVLatentUpsampler node: the model was trained on
|
||||
raw (denormalized) VAE latents, so the trip through per_channel_statistics
|
||||
is mandatory, not cosmetic.
|
||||
"""
|
||||
import comfy.model_management
|
||||
|
||||
device = comfy.model_management.get_torch_device()
|
||||
model_dtype = next(upsampler.parameters()).dtype
|
||||
in_dtype, in_device = video_crop.dtype, video_crop.device
|
||||
stats = _per_channel_stats(vae)
|
||||
try:
|
||||
upsampler.to(device)
|
||||
x = video_crop.to(device=device, dtype=model_dtype)
|
||||
if stats is not None:
|
||||
x = stats.un_normalize(x)
|
||||
with torch.no_grad():
|
||||
x = upsampler(x)
|
||||
if stats is not None:
|
||||
x = stats.normalize(x)
|
||||
finally:
|
||||
upsampler.cpu()
|
||||
return x.to(device=in_device, dtype=in_dtype)
|
||||
|
||||
|
||||
def _downscale_to(video, h, w):
|
||||
"""Area-downscale [B, C, F, H, W] spatially to (h, w); frames independent."""
|
||||
b, c, f, hh, ww = video.shape
|
||||
x = video.permute(0, 2, 1, 3, 4).reshape(b * f, c, hh, ww)
|
||||
x = F.interpolate(x.float(), size=(h, w), mode="area")
|
||||
return x.reshape(b, f, c, h, w).permute(0, 2, 1, 3, 4).to(dtype=video.dtype)
|
||||
|
||||
|
||||
def detail_refine_scene(chain, model, vae, sampler, positive, negative, latent,
|
||||
detail_targets, upsampler, seed, cfg,
|
||||
threshold=DEFAULT_THRESHOLD, strength=1.0, area_cap=MAX_TUBE_AREA,
|
||||
renoise_sigma=DEFAULT_RENOISE_SIGMA, mode="repair", debug=False):
|
||||
"""Run the segmented-detail pass on one finished scene latent.
|
||||
|
||||
`mode`:
|
||||
- "sharpen": crop -> upsampler forward -> downscale -> paste. NO video-model
|
||||
calls at all - the trained upsampler net (small ResBlock/PixelShuffle CNN,
|
||||
not naive interpolation) is itself doing the work, so this is close to
|
||||
free next to a diffusion step. It can genuinely sharpen a region that's
|
||||
blurry/under-resolved but already correctly shaped. It CANNOT fix wrong
|
||||
structure (an extra finger stays an extra finger) - a super-res net adds
|
||||
detail consistent with its input, it doesn't invent new correct content.
|
||||
- "repair" (default): the above PLUS a `renoise_sigma`-entry, 3-step
|
||||
re-denoise through the actual video model - genuinely expensive (3 extra
|
||||
forwards on the crop) because only the generative model itself can decide
|
||||
the region should look structurally different, not just sharper.
|
||||
|
||||
Returns (latent, note): the refined latent dict plus a run_mechanisms note.
|
||||
Two different "nothing happened" shapes, both same-object no-ops:
|
||||
- (latent, None): the pass is effectively OFF (no targets, no upsampler,
|
||||
strength 0) - not worth a note on every scene.
|
||||
- (latent, "segmented_detail(no match: ...)"): the pass RAN and found
|
||||
nothing - always reported, with the max CLIPSeg score seen, because a
|
||||
miss is otherwise indistinguishable from "nothing to detail here" when
|
||||
it's actually just the threshold being wrong for this content.
|
||||
`area_cap` is a COST guardrail, not a content judgment: the user named the
|
||||
target, so a region crossing it is reported as a skip with the actual size
|
||||
(never silently reinterpreted as "not found") and area_cap itself is a
|
||||
user-facing knob (detail_max_area) - raise it, don't route around it here.
|
||||
`chain` is the FunPackLTXAVSceneChainSampler instance (reuses _latent_tensors
|
||||
and _sample_chunk so the refine denoise goes through the exact same path as
|
||||
the scene it is polishing).
|
||||
"""
|
||||
targets = parse_targets(detail_targets)
|
||||
if not targets or upsampler is None or strength <= 0:
|
||||
return latent, None
|
||||
|
||||
tensors = chain._latent_tensors(latent)
|
||||
video = tensors[0]
|
||||
if video.dim() != 5:
|
||||
_log(debug, f"unexpected video latent rank {video.dim()}; skipping")
|
||||
return latent, None
|
||||
b, c, f, lat_h, lat_w = video.shape
|
||||
target_str = ", ".join(targets)
|
||||
import comfy.nested_tensor
|
||||
|
||||
# 1) Detect on a few decoded keyframes.
|
||||
frames = _decode_detection_frames(vae, video, DETECTION_FRAMES, debug=debug)
|
||||
heat = _clipseg_heat(frames, targets, debug=debug)
|
||||
if heat is None:
|
||||
return latent, f"segmented_detail(no match: '{target_str}' - no keyframe decoded)"
|
||||
diag = {}
|
||||
tube = find_tube(heat, lat_h, lat_w, threshold=threshold, area_cap=area_cap,
|
||||
debug=debug, diag=diag)
|
||||
if tube is None:
|
||||
if diag.get("reason") == "area_cap":
|
||||
note = (f"segmented_detail(no match: '{target_str}' region covers "
|
||||
f"{diag['area']:.0%} of frame > {area_cap:.0%} cap - raise "
|
||||
"detail_max_area if you want it detailed at this size anyway)")
|
||||
else:
|
||||
note = (f"segmented_detail(no match: '{target_str}', max CLIPSeg score "
|
||||
f"{diag.get('max_heat', 0.0):.2f} < threshold {threshold:.2f} - "
|
||||
"try a lower detail_threshold or a different target word)")
|
||||
return latent, note
|
||||
y0, y1, x0, x1, mask = tube
|
||||
_log(debug, f"tube y[{y0}:{y1}] x[{x0}:{x1}] of {lat_h}x{lat_w} for {targets}")
|
||||
|
||||
# 2) Crop -> 2x latent upsample (Lightricks stage-2 model).
|
||||
crop = video[:, :, :, y0:y1, x0:x1]
|
||||
crop_up = _run_upsampler(upsampler, crop, vae, debug=debug)
|
||||
|
||||
if mode == "sharpen":
|
||||
# No re-denoise, no video-model forward at all: crop_up IS the result. The
|
||||
# trained upsampler is itself a learned super-resolution net (ResBlocks +
|
||||
# PixelShuffle), not naive interpolation, so it genuinely hallucinates
|
||||
# plausible fine detail consistent with the crop's EXISTING structure -
|
||||
# that's exactly the "cheap, on-the-fly sharpen" the feature was first
|
||||
# pitched as. What it cannot do: invent correct structure the crop never
|
||||
# had (an extra finger stays an extra finger, just a sharper one) - that
|
||||
# needs the model to actually reconsider the region, which is "repair".
|
||||
refined_crop = crop_up
|
||||
cost_note = "sharpen: upsampler-only, no diffusion"
|
||||
else:
|
||||
# 3) Re-denoise the crop over a stage-2 tail shaped like the official two-
|
||||
# stage workflow's, but re-entering at `renoise_sigma` (default 0.85, their
|
||||
# exact value) rather than a hardcoded one: how much genuine freedom the
|
||||
# model needs to fix this crop depends on how wrong it already is, which
|
||||
# only the person looking at the result can judge. CONST noise_scaling
|
||||
# (sigma*noise + (1-sigma)*x) re-noises the clean crop to that sigma
|
||||
# automatically. The scene's audio stream rides along so joint attention
|
||||
# sees a legal AV latent, but its refined output is discarded below - audio
|
||||
# is protected by construction, not by masking. This is the expensive path
|
||||
# (3 extra video-model forwards on the crop): actually changing WRONG
|
||||
# structure requires the generative model to redo the region, no shortcut.
|
||||
crop_latent = {"samples": crop_up}
|
||||
if len(tensors) > 1:
|
||||
crop_latent["samples"] = comfy.nested_tensor.NestedTensor(
|
||||
[crop_up] + [t.detach().clone() for t in tensors[1:]])
|
||||
tail_sigmas = torch.tensor(stage2_sigmas(renoise_sigma), dtype=torch.float32)
|
||||
refined = chain._sample_chunk(
|
||||
model, sampler, tail_sigmas, int(seed) + 7777, cfg,
|
||||
_strip_layout_conds(positive), _strip_layout_conds(negative), crop_latent)
|
||||
refined_crop = chain._latent_tensors(refined)[0]
|
||||
cost_note = f"repair: renoise={renoise_sigma:.2f}"
|
||||
|
||||
# 4) Back to grid size, feathered paste through the CLIPSeg silhouette.
|
||||
refined_down = _downscale_to(refined_crop, y1 - y0, x1 - x0)
|
||||
m = (mask.to(device=video.device, dtype=video.dtype) * float(strength)).clamp(0.0, 1.0)
|
||||
out_video = video.clone()
|
||||
region = out_video[:, :, :, y0:y1, x0:x1]
|
||||
out_video[:, :, :, y0:y1, x0:x1] = region + (refined_down.to(region.device) - region) * m
|
||||
|
||||
out = dict(latent)
|
||||
if len(tensors) > 1:
|
||||
out["samples"] = comfy.nested_tensor.NestedTensor([out_video] + list(tensors[1:]))
|
||||
else:
|
||||
out["samples"] = out_video
|
||||
note = (f"segmented_detail({cost_note}, '{target_str}', tube={y1 - y0}x{x1 - x0}"
|
||||
f"/{lat_h}x{lat_w}, s={strength})")
|
||||
return out, note
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 38 KiB |
@@ -0,0 +1 @@
|
||||
from . import server # noqa: F401 registers /funpack/easy/* routes
|
||||
@@ -0,0 +1,319 @@
|
||||
// Thin backend client — FORKED from movie_editor/frontend/api.js on purpose:
|
||||
// Easy Gen is served from /funpack/easy/ but there is no /funpack/easy/api/*
|
||||
// backend (see easy_gen/server.py) — Easy Gen reuses the Movie Editor's
|
||||
// project/generate/media/models API as-is, so BASE must stay pinned to
|
||||
// /funpack/movie regardless of the page's own URL. Keep every other method
|
||||
// here in sync with the Editor's api.js; only this constant differs.
|
||||
(function () {
|
||||
const BASE = "/funpack/movie";
|
||||
const API = (p) => `${BASE}/api${p}`;
|
||||
const FUNPACK = "/funpack";
|
||||
|
||||
async function funpackFetch(method, path, body) {
|
||||
const opts = { method, headers: {}, cache: "no-store" };
|
||||
if (body !== undefined) {
|
||||
opts.headers["Content-Type"] = "application/json";
|
||||
opts.body = JSON.stringify(body);
|
||||
}
|
||||
const res = await fetch(path, opts);
|
||||
if (!res.ok) {
|
||||
let payload = null;
|
||||
try { payload = await res.json(); } catch (_) {}
|
||||
throw new Error(readApiError(res, payload));
|
||||
}
|
||||
return res;
|
||||
}
|
||||
|
||||
function readApiError(res, payload) {
|
||||
if (payload && typeof payload === "object") {
|
||||
const detail = payload.detail;
|
||||
if (typeof detail === "string" && detail.trim()) return detail.trim();
|
||||
if (Array.isArray(detail)) {
|
||||
const parts = detail.map((item) => {
|
||||
if (item && typeof item === "object" && item.msg) return String(item.msg);
|
||||
return JSON.stringify(item);
|
||||
}).filter(Boolean);
|
||||
if (parts.length) return parts.join("; ");
|
||||
}
|
||||
if (payload.error) return String(payload.error);
|
||||
if (payload.parse_errors && typeof payload.parse_errors === "object") {
|
||||
return Object.entries(payload.parse_errors).map(([k, v]) => `${k}: ${v}`).join("; ");
|
||||
}
|
||||
}
|
||||
const status = res && res.status ? `HTTP ${res.status}` : "";
|
||||
const statusText = (res && res.statusText ? res.statusText : "").trim();
|
||||
return statusText || status || "Request failed";
|
||||
}
|
||||
|
||||
async function j(method, url, body) {
|
||||
const opts = { method, headers: {} };
|
||||
if (body !== undefined) {
|
||||
opts.headers["Content-Type"] = "application/json";
|
||||
opts.body = JSON.stringify(body);
|
||||
}
|
||||
const res = await fetch(url, opts);
|
||||
if (!res.ok) {
|
||||
let payload = null;
|
||||
try { payload = await res.json(); } catch (_) {}
|
||||
throw new Error(readApiError(res, payload));
|
||||
}
|
||||
return res.status === 204 ? null : res.json();
|
||||
}
|
||||
|
||||
const ClientAPI = {
|
||||
health: () => j("GET", API("/health")),
|
||||
|
||||
// projects
|
||||
listProjects: () => j("GET", API("/projects")),
|
||||
createProject: (name) => j("POST", API("/projects"), { name }),
|
||||
getProject: (id) => j("GET", API(`/projects/${id}`)),
|
||||
saveProject: (id, data) => j("PUT", API(`/projects/${id}`), data),
|
||||
deleteProject: (id) => j("DELETE", API(`/projects/${id}`)),
|
||||
downloadProjectUrl: (id) => API(`/projects/${id}/download`),
|
||||
importProject: (data) => j("POST", API("/projects/import"), data),
|
||||
|
||||
// timeline preview
|
||||
preview: (id, includeExcluded, forGeneration = true) =>
|
||||
j("GET", API(`/projects/${id}/preview?include_excluded=${includeExcluded ? "true" : "false"}&for_generation=${forGeneration ? "true" : "false"}`)),
|
||||
parsePrompt: (id, prompt) => j("POST", API(`/projects/${id}/parse`), { prompt }),
|
||||
|
||||
// libraries
|
||||
transitions: () => j("GET", API("/library/transitions")),
|
||||
saveTransition: (item) => j("POST", API("/library/transitions"), item),
|
||||
deleteTransition: (name) => j("DELETE", API(`/library/transitions/${encodeURIComponent(name)}`)),
|
||||
exportTransitionsUrl: () => API("/library/transitions/export"),
|
||||
importTransitions: (data, mode) => j("POST", API(`/library/transitions/import?mode=${mode || "merge"}`), data),
|
||||
clearTransitions: () => j("POST", API("/library/transitions/clear"), {}),
|
||||
shortcuts: () => j("GET", API("/library/shortcuts")),
|
||||
suggestionStats: () => j("GET", API("/library/suggestion_stats")),
|
||||
saveCategory: (payload) => j("POST", API("/library/categories"), payload),
|
||||
saveShortcut: (item) => j("POST", API("/library/shortcuts"), item),
|
||||
deleteShortcut: (name) => j("DELETE", API(`/library/shortcuts/${encodeURIComponent(name)}`)),
|
||||
exportShortcutsUrl: () => API("/library/shortcuts/export"),
|
||||
importShortcuts: (data, mode) => j("POST", API(`/library/shortcuts/import?mode=${mode || "merge"}`), data),
|
||||
clearShortcuts: () => j("POST", API("/library/shortcuts/clear"), {}),
|
||||
revolverSettings: () => j("GET", API("/library/revolver")),
|
||||
setRevolverSettings: (payload) => j("POST", API("/library/revolver"), payload),
|
||||
|
||||
// FunPack file manager (Composer ▸ Files)
|
||||
listFiles: () => j("GET", API("/files")),
|
||||
deleteFile: (group, name) => j("DELETE", API(`/files/${encodeURIComponent(group)}/${encodeURIComponent(name)}`)),
|
||||
clearFiles: (group) => j("POST", API(`/files/${encodeURIComponent(group)}/clear`), {}),
|
||||
nleLibrary: () => j("GET", API("/library/nle")),
|
||||
|
||||
// media bin
|
||||
listMedia: () => j("GET", API("/media")),
|
||||
mediaUrl: (id) => API(`/media/${encodeURIComponent(id)}`),
|
||||
deleteMedia: (id) => j("DELETE", API(`/media/${encodeURIComponent(id)}`)),
|
||||
renameMedia: (id, name) => j("PATCH", API(`/media/${encodeURIComponent(id)}`), { name }),
|
||||
importClipToMediaBin: (clip, name) => j("POST", API("/media/import-clip"), { clip, name: name || null }),
|
||||
async uploadMedia(file) {
|
||||
const fd = new FormData(); fd.append("file", file, file.name);
|
||||
const res = await fetch(API("/media"), { method: "POST", body: fd });
|
||||
if (!res.ok) throw new Error((await res.json().catch(() => ({}))).detail || res.statusText);
|
||||
return res.json();
|
||||
},
|
||||
|
||||
// models / node slots
|
||||
nodeRoles: () => j("GET", API("/node-roles")),
|
||||
nodeCandidates: (role, refresh) => j("GET", API(`/node-candidates/${role}${refresh ? "?refresh=true" : ""}`)),
|
||||
allNodes: () => j("GET", API("/all-nodes")),
|
||||
nodeSpec: (cls) => j("GET", API(`/node/${encodeURIComponent(cls)}`)),
|
||||
pipelinePorts: () => j("GET", API("/pipeline-ports")),
|
||||
pipelineDeps: () => j("GET", API("/pipeline-deps")),
|
||||
pipelineDepsInstall: (packIds) => j("POST", API("/pipeline-deps/install"), { pack_ids: packIds }),
|
||||
pipelineDepsInstallManager: () => j("POST", API("/pipeline-deps/install"), { install_manager: true }),
|
||||
pipelineDepsInstallStatus: (jobId) => j("GET", API(`/pipeline-deps/install/${encodeURIComponent(jobId)}`)),
|
||||
pipelineDepsInstallCancel: (jobId) => j("POST", API(`/pipeline-deps/install/${encodeURIComponent(jobId)}/cancel`), {}),
|
||||
imageTargets: (pid) => j("GET", API("/image-targets" + (pid ? `?pid=${encodeURIComponent(pid)}` : ""))),
|
||||
// Models config is per-project; pass the project id. Falls back to the global
|
||||
// default route (used as the seed/template) when no project is given.
|
||||
coreGraph: (pid) => j("GET", API("/core-graph" + (pid ? `?pid=${encodeURIComponent(pid)}` : ""))),
|
||||
getModels: (pid) => j("GET", API(pid ? `/projects/${pid}/models` : "/models")),
|
||||
saveModels: (pid, data) => j("PUT", API(pid ? `/projects/${pid}/models` : "/models"), data),
|
||||
refreshModels: () => j("POST", API("/models/refresh")),
|
||||
parseWorkflow: (workflow) => j("POST", API("/workflow/parse"), { workflow }),
|
||||
applyWorkflow: (pid, workflow, bindings) =>
|
||||
j("POST", API(`/projects/${pid}/workflow/apply`), { workflow, bindings }),
|
||||
restart: () => j("POST", API("/restart")),
|
||||
gitStatus: () => j("GET", API("/git/status")),
|
||||
gitUpdate: (branch) => j("POST", API("/git/update"), branch ? { branch } : {}),
|
||||
gitCheckout: (branch) => j("POST", API("/git/checkout"), { branch }),
|
||||
|
||||
// generate (a single scene, or an explicit run of scene ids = one chain request)
|
||||
generate: (id, onlyScene, sceneIds, resetSession, nodeOverrides) =>
|
||||
j("POST", API(`/projects/${id}/generate`), { only_scene: onlyScene || null, scene_ids: sceneIds || null, reset_session: !!resetSession, node_overrides: nodeOverrides || null }),
|
||||
status: (id, promptId) => j("GET", API(`/projects/${id}/status/${promptId}`)),
|
||||
progress: () => j("GET", API("/progress")),
|
||||
// The editor's own in-flight generation, recovered from ComfyUI's queue (survives a UI reload).
|
||||
active: () => j("GET", API("/active")),
|
||||
ratingLabels: () => j("GET", API("/rating-labels")),
|
||||
log: (limit) => j("GET", API("/log" + (limit ? `?limit=${limit}` : ""))),
|
||||
interrupt: () => j("POST", API("/interrupt")),
|
||||
|
||||
// ComfyUI temp folder browser (scene previews & other transient outputs, wiped on restart).
|
||||
listTemp: () => j("GET", API("/temp")),
|
||||
// Served by ComfyUI's own same-origin /view endpoint — no editor route needed.
|
||||
tempFileUrl: (f) => "/view?" + new URLSearchParams({
|
||||
filename: f.filename, subfolder: f.subfolder || "", type: "temp", t: String(f.mtime || Date.now()),
|
||||
}).toString(),
|
||||
async downloadTempFile(f) {
|
||||
const res = await fetch(this.tempFileUrl(f), { cache: "no-store" });
|
||||
if (!res.ok) throw new Error(`Could not fetch ${f.filename} (HTTP ${res.status})`);
|
||||
const blob = await res.blob();
|
||||
const url = URL.createObjectURL(blob);
|
||||
const link = document.createElement("a");
|
||||
link.href = url;
|
||||
link.download = f.filename;
|
||||
document.body.appendChild(link);
|
||||
link.click();
|
||||
link.remove();
|
||||
URL.revokeObjectURL(url);
|
||||
},
|
||||
renderFinal: (id, clips) => j("POST", API(`/projects/${id}/render`), { clips }),
|
||||
renderFinalStatus: (id, jobId) => j("GET", API(`/projects/${id}/render/${encodeURIComponent(jobId)}`)),
|
||||
exportClip: (id, clip) => j("POST", API(`/projects/${id}/export-clip`), { clip }),
|
||||
exportClipsCombined: (id, clips) => j("POST", API(`/projects/${id}/export-clips`), { clips }),
|
||||
exportClipsStatus: (id, jobId) => j("GET", API(`/projects/${id}/export-clips/${encodeURIComponent(jobId)}`)),
|
||||
resultUrl: (id, m) =>
|
||||
API(`/projects/${id}/result?filename=${encodeURIComponent(m.filename)}`) +
|
||||
`&subfolder=${encodeURIComponent(m.subfolder || "")}&type=${encodeURIComponent(m.type || "output")}`,
|
||||
previewSegmentUrl: (id, sceneId, spec) => {
|
||||
let u = API(`/projects/${id}/preview-segment/${encodeURIComponent(sceneId)}`);
|
||||
const m = spec?.media;
|
||||
if (m?.filename) {
|
||||
const q = new URLSearchParams({
|
||||
filename: m.filename,
|
||||
subfolder: m.subfolder || "",
|
||||
type: m.type || "output",
|
||||
render_in: String(spec.renderIn != null ? spec.renderIn : 0),
|
||||
});
|
||||
// dur is in the URL for two reasons: ghosts have no scene server-side (the trim
|
||||
// window must travel in the query), and it versions the browser cache — segments
|
||||
// are cached for an hour, so a timeline trim must produce a different URL.
|
||||
if (spec.dur != null) q.set("dur", String(spec.dur));
|
||||
u += "?" + q.toString();
|
||||
}
|
||||
return u;
|
||||
},
|
||||
|
||||
// FunPack refinement keys (ComfyUI root routes, same as Studio / Refinement Key Loader)
|
||||
refinementKeys: async () => {
|
||||
const res = await funpackFetch("GET", `${FUNPACK}/refinement_keys?cache_bust=${Date.now()}`);
|
||||
return res.json();
|
||||
},
|
||||
importRefinementKey: async (data, opts = {}) => {
|
||||
const ow = opts.overwrite ? "&overwrite=true" : "";
|
||||
const json = JSON.stringify(data);
|
||||
const bytes = new TextEncoder().encode(json);
|
||||
// A 409 means a key of that name already exists — surface it as a tagged
|
||||
// error so the caller can ask the user to overwrite (then retry).
|
||||
const finish = async (res) => {
|
||||
if (res.status === 409) {
|
||||
let p = null; try { p = await res.json(); } catch (_) {}
|
||||
const err = new Error((p && p.error) || "Refinement key already exists.");
|
||||
err.exists = true; err.key = (p && p.key) || "";
|
||||
throw err;
|
||||
}
|
||||
if (!res.ok) {
|
||||
let p = null; try { p = await res.json(); } catch (_) {}
|
||||
throw new Error(readApiError(res, p));
|
||||
}
|
||||
return res.json();
|
||||
};
|
||||
// Reverse proxies on Vast.ai / Runpod cap the request body well below
|
||||
// ComfyUI's 100 MB, so a one-shot POST of a large key gets HTTP 413 at the
|
||||
// proxy. Keep a fast single POST for small keys; stream big ones in chunks
|
||||
// (each far under any proxy limit) so they can never be rejected.
|
||||
const CHUNK = 256 * 1024;
|
||||
if (bytes.length <= CHUNK) {
|
||||
const res = await fetch(`${FUNPACK}/refinement_keys/import?_=${Date.now()}${ow}`, {
|
||||
method: "POST", cache: "no-store",
|
||||
headers: { "Content-Type": "application/json" }, body: json,
|
||||
});
|
||||
if (res.status !== 413) return finish(res); // 413 → fall through to chunked
|
||||
}
|
||||
const uploadId = `${Date.now()}_${Math.random().toString(36).slice(2)}`;
|
||||
const postChunk = async (path, body) => {
|
||||
const res = await fetch(path, {
|
||||
method: "POST", cache: "no-store",
|
||||
headers: { "Content-Type": "application/octet-stream" }, body,
|
||||
});
|
||||
if (!res.ok) {
|
||||
let payload = null;
|
||||
try { payload = await res.json(); } catch (_) {}
|
||||
throw new Error(readApiError(res, payload));
|
||||
}
|
||||
return res;
|
||||
};
|
||||
let index = 0;
|
||||
for (let off = 0; off < bytes.length; off += CHUNK, index++) {
|
||||
// Slice by bytes (not string chars) so a multi-byte char is never split
|
||||
// across the seam where the two halves get re-encoded independently.
|
||||
await postChunk(
|
||||
`${FUNPACK}/refinement_keys/import_chunk?upload_id=${encodeURIComponent(uploadId)}&index=${index}`,
|
||||
bytes.slice(off, off + CHUNK),
|
||||
);
|
||||
}
|
||||
return finish(await fetch(
|
||||
`${FUNPACK}/refinement_keys/import_finalize?upload_id=${encodeURIComponent(uploadId)}${ow}`,
|
||||
{ method: "POST", cache: "no-store" },
|
||||
));
|
||||
},
|
||||
async exportRefinementKeyFile(key) {
|
||||
const res = await fetch(
|
||||
`${FUNPACK}/refinement_keys/export?key=${encodeURIComponent(key)}&cache_bust=${Date.now()}`,
|
||||
{ cache: "no-store" },
|
||||
);
|
||||
if (!res.ok) {
|
||||
let payload = null;
|
||||
try { payload = await res.json(); } catch (_) {}
|
||||
throw new Error(readApiError(res, payload));
|
||||
}
|
||||
const blob = await res.blob();
|
||||
const url = URL.createObjectURL(blob);
|
||||
const link = document.createElement("a");
|
||||
link.href = url;
|
||||
link.download = `${key}.json`;
|
||||
document.body.appendChild(link);
|
||||
link.click();
|
||||
link.remove();
|
||||
URL.revokeObjectURL(url);
|
||||
},
|
||||
|
||||
// Atomically delete a refinement key and ALL its sidecars (value function,
|
||||
// blessed attention / K-V banks, creativity latent, velocity store). Deleting
|
||||
// only the visible <key>.json orphans those, and they keep steering future runs
|
||||
// across restarts — the backend sweep is what makes "key deleted" mean it.
|
||||
async deleteRefinementKey(key) {
|
||||
const res = await funpackFetch(
|
||||
"POST", `${FUNPACK}/refinement_keys/delete?key=${encodeURIComponent(key)}&_=${Date.now()}`,
|
||||
);
|
||||
if (!res.ok) {
|
||||
let payload = null;
|
||||
try { payload = await res.json(); } catch (_) {}
|
||||
throw new Error(readApiError(res, payload));
|
||||
}
|
||||
return res.json();
|
||||
},
|
||||
|
||||
// The keyless Absolute "global taste" store — learns from every rated generation
|
||||
// across all prompts, invisible in the key list, applied only in absolute/both
|
||||
// steer mode, and otherwise only wiped by Session Reset.
|
||||
async absoluteStoreInfo() {
|
||||
const res = await funpackFetch("GET", `${FUNPACK}/refinement_keys/absolute?_=${Date.now()}`);
|
||||
return res.json();
|
||||
},
|
||||
async clearAbsoluteStore() {
|
||||
const res = await funpackFetch("POST", `${FUNPACK}/refinement_keys/clear_absolute?_=${Date.now()}`);
|
||||
if (!res.ok) {
|
||||
let payload = null;
|
||||
try { payload = await res.json(); } catch (_) {}
|
||||
throw new Error(readApiError(res, payload));
|
||||
}
|
||||
return res.json();
|
||||
},
|
||||
};
|
||||
|
||||
window.MovieEditorAPI = ClientAPI;
|
||||
})();
|
||||
@@ -0,0 +1,199 @@
|
||||
// Easy Gen entrypoint: wires the big-preview / prompt / upload / generate bar
|
||||
// to the shared easy_store.js + preview.js, and opens the "FunPack Easy Gen"
|
||||
// project picker on boot. No re-attach-on-reload support yet (the Editor's
|
||||
// /api/active flow) — a reload while a run is in flight loses the progress
|
||||
// view, though the run itself still completes server-side.
|
||||
(function () {
|
||||
const S = window.Store;
|
||||
const API = window.MovieEditorAPI;
|
||||
|
||||
const LAST_PROJECT_KEY = "funpack_easy_last_project";
|
||||
|
||||
const nameEl = document.getElementById("easy-project-name");
|
||||
const healthEl = document.getElementById("easy-health-chip");
|
||||
const promptEl = document.getElementById("easy-prompt");
|
||||
const uploadBtn = document.getElementById("easy-upload-btn");
|
||||
const uploadInput = document.getElementById("easy-upload-input");
|
||||
const galleryBtn = document.getElementById("easy-gallery-btn");
|
||||
const generateBtn = document.getElementById("easy-generate-btn");
|
||||
const advancedBtn = document.getElementById("easy-advanced-btn");
|
||||
const projectsBtn = document.getElementById("easy-projects-btn");
|
||||
const saveBtn = document.getElementById("easy-save-btn");
|
||||
const exportBtn = document.getElementById("easy-export-btn");
|
||||
|
||||
const preview = window.EasyPreview.create(document.getElementById("easy-preview"));
|
||||
|
||||
let saveTimer = null;
|
||||
function scheduleSave() {
|
||||
if (saveTimer) clearTimeout(saveTimer);
|
||||
saveTimer = setTimeout(() => { S.save().catch(() => {}); }, 800);
|
||||
}
|
||||
|
||||
let promptFocused = false;
|
||||
promptEl.addEventListener("focus", () => { promptFocused = true; });
|
||||
promptEl.addEventListener("blur", () => { promptFocused = false; });
|
||||
promptEl.addEventListener("input", () => {
|
||||
S.scheduleGlobalPromptApply(promptEl.value);
|
||||
});
|
||||
// Self-contained: reads Store.getEditorSetting("autocomplete")/get().shortcuts
|
||||
// and no-ops until a project + shortcut library are loaded.
|
||||
window.ShortcutAutocomplete?.attach(promptEl);
|
||||
|
||||
function removeUpload() {
|
||||
S.setSceneMedia(null, null);
|
||||
scheduleSave();
|
||||
}
|
||||
preview.setOnClearUpload(removeUpload);
|
||||
|
||||
let lastProjectId = null;
|
||||
function render(st) {
|
||||
const p = st.project;
|
||||
nameEl.textContent = p ? p.name : "No project";
|
||||
generateBtn.disabled = !p;
|
||||
uploadBtn.disabled = !p;
|
||||
galleryBtn.disabled = !p;
|
||||
saveBtn.disabled = !p;
|
||||
exportBtn.disabled = !p;
|
||||
if (p && !promptFocused) {
|
||||
const text = p.global_prompt || "";
|
||||
if (promptEl.value !== text) promptEl.value = text;
|
||||
}
|
||||
const source = p?.scenes[0]?.source;
|
||||
const attached = source && source.type !== "empty" && source.media_ref;
|
||||
uploadBtn.textContent = attached ? "⬆ Uploaded ✕" : "⬆ Upload";
|
||||
preview.setUpload(attached ? API.mediaUrl(source.media_ref) : null, source?.type === "v2v" ? "video" : "image");
|
||||
|
||||
const pid = p ? p.id : null;
|
||||
if (pid !== lastProjectId) {
|
||||
lastProjectId = pid;
|
||||
preview.stopPolling();
|
||||
preview.showEmpty();
|
||||
generateBtn.textContent = "Generate";
|
||||
}
|
||||
}
|
||||
S.subscribe(render);
|
||||
|
||||
uploadBtn.onclick = () => {
|
||||
const st = S.get();
|
||||
const attached = st.project && st.project.scenes[0]?.source?.type !== "empty";
|
||||
if (attached) {
|
||||
removeUpload();
|
||||
return;
|
||||
}
|
||||
uploadInput.click();
|
||||
};
|
||||
|
||||
uploadInput.onchange = async () => {
|
||||
const file = uploadInput.files?.[0];
|
||||
uploadInput.value = "";
|
||||
if (!file) return;
|
||||
uploadBtn.disabled = true;
|
||||
uploadBtn.textContent = "Uploading…";
|
||||
try {
|
||||
const res = await API.uploadMedia(file);
|
||||
const media = res.media?.[0]; // POST /media wraps the result: { media: [ {id, ...} ] }
|
||||
if (!media) throw new Error("Upload succeeded but returned no media entry.");
|
||||
const kind = file.type.startsWith("video") ? "video" : "image";
|
||||
S.setSceneMedia(media.id, kind);
|
||||
await S.save();
|
||||
S.loadMedia();
|
||||
} catch (e) {
|
||||
alert("Upload failed: " + (e.message || e));
|
||||
} finally {
|
||||
render(S.get());
|
||||
}
|
||||
};
|
||||
|
||||
// Easy Gen has no rating UI, so Studio must never apply learned/rated steering here —
|
||||
// forced at the graph level (not just hidden in Engine settings) so a project edited in
|
||||
// the Cutting Room with these on still generates plainly when run from Easy Gen.
|
||||
// "studio"/"sampler" are the built-in pipeline's fixed graph keys (see backend/builder.py).
|
||||
const EASY_MODE_OVERRIDES = [
|
||||
{ node: "studio", input: "mode", value: "Prompt only" },
|
||||
{ node: "sampler", input: "embed_guidance", value: false },
|
||||
{ node: "sampler", input: "score_slider", value: false },
|
||||
{ node: "sampler", input: "output_guidance", value: false },
|
||||
{ node: "sampler", input: "dynashift", value: false },
|
||||
{ node: "sampler", input: "taste_nearest_prompt", value: false },
|
||||
];
|
||||
|
||||
generateBtn.onclick = async () => {
|
||||
const st = S.get();
|
||||
if (!st.project) return;
|
||||
if (!(st.project.global_prompt || "").trim()) {
|
||||
alert("Type a prompt first.");
|
||||
return;
|
||||
}
|
||||
generateBtn.disabled = true;
|
||||
generateBtn.textContent = "Generating…";
|
||||
try {
|
||||
await S.save();
|
||||
const resetSession = S.takeResetSessionFlag();
|
||||
const res = await API.generate(st.project.id, null, null, resetSession, EASY_MODE_OVERRIDES);
|
||||
preview.watch(st.project.id, res.prompt_id, {
|
||||
onDone: () => { generateBtn.disabled = false; generateBtn.textContent = "Generate"; },
|
||||
});
|
||||
} catch (e) {
|
||||
preview.showError("Generate failed: " + (e.message || e));
|
||||
generateBtn.disabled = false;
|
||||
generateBtn.textContent = "Generate";
|
||||
}
|
||||
};
|
||||
|
||||
advancedBtn.onclick = () => window.SettingsWindow.open("generation");
|
||||
projectsBtn.onclick = () => window.ProjectMenu.open({ dismissable: !!S.get().project });
|
||||
galleryBtn.onclick = () => window.EasyGallery.open();
|
||||
|
||||
// "Save" persists into FunPack's own project store on THIS machine — the same mechanism
|
||||
// as the Editor's autosave. It doesn't help moving a project off a rental GPU box. This
|
||||
// downloads the actual .funpack_project.json file (re-importable via "Load Project
|
||||
// File…" on another machine), same as the Editor's File ▸ "Save Project File…".
|
||||
exportBtn.onclick = async () => {
|
||||
const st = S.get();
|
||||
if (!st.project) return;
|
||||
try { await S.save(); } catch (_) { /* export the last-saved version anyway */ }
|
||||
const a = document.createElement("a");
|
||||
a.href = API.downloadProjectUrl(st.project.id);
|
||||
a.download = "";
|
||||
document.body.appendChild(a); a.click(); document.body.removeChild(a);
|
||||
};
|
||||
|
||||
let saveResetTimer = null;
|
||||
saveBtn.onclick = async () => {
|
||||
if (saveResetTimer) clearTimeout(saveResetTimer);
|
||||
saveBtn.disabled = true;
|
||||
saveBtn.textContent = "Saving…";
|
||||
try {
|
||||
await S.save();
|
||||
saveBtn.textContent = "Saved ✓";
|
||||
} catch (e) {
|
||||
saveBtn.textContent = "Save failed";
|
||||
alert("Save failed: " + (e.message || e));
|
||||
} finally {
|
||||
saveResetTimer = setTimeout(() => {
|
||||
saveBtn.textContent = "Save";
|
||||
saveBtn.disabled = !S.get().project;
|
||||
}, 1200);
|
||||
}
|
||||
};
|
||||
|
||||
async function boot() {
|
||||
S.refreshHealth();
|
||||
S.loadLibraries();
|
||||
S.loadMedia();
|
||||
setInterval(() => S.refreshHealth(), 15000);
|
||||
S.subscribe((st) => {
|
||||
healthEl.textContent = st.health?.ok ? "● online" : "● offline";
|
||||
healthEl.classList.toggle("bad", !(st.health && st.health.ok));
|
||||
if (st.project) localStorage.setItem(LAST_PROJECT_KEY, st.project.id);
|
||||
});
|
||||
|
||||
const lastId = localStorage.getItem(LAST_PROJECT_KEY);
|
||||
if (lastId) {
|
||||
try { await S.loadProject(lastId); return; } catch (_) { /* fall through to picker */ }
|
||||
}
|
||||
window.ProjectMenu.open({ dismissable: false });
|
||||
}
|
||||
|
||||
boot();
|
||||
})();
|
||||
@@ -0,0 +1,98 @@
|
||||
/* Easy Gen layout on top of the Editor's shared styles.css (design tokens,
|
||||
.btn/.modal/.sw-*/.welcome-* rules) — big preview, prompt bar, nothing else. */
|
||||
|
||||
html, body.easy-body {
|
||||
margin: 0; height: 100%; background: var(--ink-0); color: var(--text);
|
||||
font-family: var(--font-ui);
|
||||
}
|
||||
|
||||
.easy-app {
|
||||
height: 100vh; display: flex; flex-direction: column;
|
||||
}
|
||||
|
||||
.easy-top {
|
||||
height: var(--menubar-h); flex: none; display: flex; align-items: center; gap: 14px;
|
||||
padding: 0 16px; border-bottom: 1px solid var(--line); background: var(--ink-1);
|
||||
}
|
||||
.easy-project-name { color: var(--muted); font-size: 13px; }
|
||||
.easy-top .easy-health-chip { margin-left: auto; }
|
||||
a.brand { text-decoration: none; color: inherit; }
|
||||
a.brand:hover .brand-sub { color: var(--amber-2); }
|
||||
|
||||
.easy-preview {
|
||||
flex: 1; min-height: 0; display: flex; align-items: center; justify-content: center;
|
||||
background: var(--ink-0); position: relative; padding: 24px;
|
||||
}
|
||||
.easy-preview-inner {
|
||||
width: 100%; height: 100%; display: flex; align-items: center; justify-content: center;
|
||||
}
|
||||
.easy-preview-inner img, .easy-preview-inner video {
|
||||
max-width: 100%; max-height: 100%; border-radius: var(--radius); box-shadow: var(--shadow);
|
||||
background: #000;
|
||||
}
|
||||
.easy-preview-empty {
|
||||
color: var(--faint); font-size: 14px; font-family: var(--font-display);
|
||||
}
|
||||
.easy-preview-error { color: var(--danger); max-width: 60ch; text-align: center; white-space: pre-wrap; }
|
||||
|
||||
.easy-preview-close {
|
||||
position: absolute; top: 34px; right: 34px; width: 28px; height: 28px; padding: 0;
|
||||
border-radius: 50%; background: rgba(12,11,9,.72); border: 1px solid var(--line);
|
||||
color: var(--text); font-size: 13px; cursor: pointer; display: flex; align-items: center; justify-content: center;
|
||||
backdrop-filter: blur(2px); transition: border-color .12s, color .12s;
|
||||
}
|
||||
.easy-preview-close[hidden] { display: none; } /* class display:flex above would otherwise beat [hidden] */
|
||||
.easy-preview-close:hover { border-color: var(--danger); color: var(--danger); }
|
||||
.easy-preview-badge {
|
||||
position: absolute; top: 34px; left: 34px; padding: 4px 10px; border-radius: 999px;
|
||||
background: rgba(12,11,9,.72); border: 1px solid var(--line); color: var(--muted);
|
||||
font-size: 11.5px; letter-spacing: .3px; backdrop-filter: blur(2px);
|
||||
}
|
||||
|
||||
.easy-progress {
|
||||
position: absolute; left: 24px; right: 24px; bottom: 24px;
|
||||
background: var(--ink-1); border: 1px solid var(--line); border-radius: var(--radius);
|
||||
padding: 10px 14px; box-shadow: var(--shadow);
|
||||
}
|
||||
.easy-progress-bar {
|
||||
height: 6px; border-radius: 4px; background: linear-gradient(90deg, var(--amber), var(--amber-2));
|
||||
width: 0%; transition: width .2s ease; margin-bottom: 6px;
|
||||
}
|
||||
.easy-progress-label { font-size: 12px; color: var(--muted); font-family: var(--font-mono); }
|
||||
|
||||
.easy-bar {
|
||||
flex: none; display: flex; align-items: flex-end; gap: 12px;
|
||||
padding: 14px 16px; border-top: 1px solid var(--line); background: var(--ink-1);
|
||||
}
|
||||
.easy-upload-btn, .easy-gallery-btn { flex: none; align-self: stretch; padding: 0 16px; }
|
||||
.easy-prompt {
|
||||
flex: 1; resize: none; background: var(--ink-2); border: 1px solid var(--line);
|
||||
border-radius: var(--radius); color: var(--text); font: 14px/1.4 var(--font-ui);
|
||||
padding: 10px 12px;
|
||||
}
|
||||
.easy-prompt:focus { outline: none; border-color: var(--amber); }
|
||||
.easy-actions { flex: none; display: flex; flex-direction: column; align-items: stretch; gap: 6px; }
|
||||
.easy-generate-btn { padding: 10px 26px; font-size: 14px; }
|
||||
.easy-advanced-btn { align-self: center; }
|
||||
|
||||
.es-io-controls { display: flex; align-items: center; gap: 8px; }
|
||||
.es-io-mode { background: var(--ink-2); border: 1px solid var(--line); color: var(--text);
|
||||
border-radius: 6px; font-size: 12px; padding: 4px 6px; }
|
||||
|
||||
.gal-grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(140px, 1fr)); gap: 12px; }
|
||||
.gal-cell { cursor: pointer; border: 1px solid var(--line); border-radius: 10px; overflow: hidden;
|
||||
background: var(--ink-2); transition: border-color .15s; }
|
||||
.gal-cell:hover { border-color: var(--amber); }
|
||||
.gal-thumb { position: relative; aspect-ratio: 1 / 1; background: var(--ink-0); display: flex; align-items: center; justify-content: center; }
|
||||
.gal-thumb img, .gal-thumb video { width: 100%; height: 100%; object-fit: cover; }
|
||||
.gal-name { padding: 6px 8px; font-size: 11.5px; color: var(--muted); white-space: nowrap;
|
||||
overflow: hidden; text-overflow: ellipsis; }
|
||||
|
||||
.gal-pin-btn { position: absolute; top: 4px; right: 4px; z-index: 2; width: 22px; height: 22px;
|
||||
border-radius: 6px; background: rgba(10, 9, 7, .72); border: 1px solid var(--line);
|
||||
filter: grayscale(1); font-size: 12px; cursor: pointer; display: flex; align-items: center;
|
||||
justify-content: center; padding: 0; }
|
||||
.gal-pin-btn:hover { filter: none; }
|
||||
.gal-pin-btn.active { filter: none; background: var(--amber-soft); border-color: var(--amber); }
|
||||
.gal-pin-badge { position: absolute; top: 4px; left: 4px; z-index: 2; font-size: 10.5px;
|
||||
background: rgba(10, 9, 7, .72); border-radius: 4px; padding: 1px 4px; line-height: 1.4; }
|
||||
@@ -0,0 +1,116 @@
|
||||
// "Gallery" picker: choose an already-uploaded media-bin file as the i2v anchor,
|
||||
// without re-uploading it. Standalone modal (same modal-overlay/.modal shell as
|
||||
// project_menu.js) since Easy Gen has no Media Bin panel of its own.
|
||||
(function () {
|
||||
const { el, clear } = window.dom;
|
||||
const API = window.MovieEditorAPI;
|
||||
const S = window.Store;
|
||||
let overlay = null;
|
||||
|
||||
function close() {
|
||||
if (overlay) { overlay.remove(); overlay = null; }
|
||||
}
|
||||
|
||||
// Continuity pin, right on the gallery card — same setting as Engine settings ▸
|
||||
// Continuity ▸ Identity pin, just quicker to reach (mirrors mediabrowser.js's card
|
||||
// pin button in the Editor). Images only, chain-sampler pipelines only.
|
||||
const pinEls = []; // { id, btn, badge } for every image cell in the currently-open grid
|
||||
|
||||
function refreshPins() {
|
||||
const cs = (S.get().project || {}).continuity_settings || {};
|
||||
pinEls.forEach(({ id, btn, badge }) => {
|
||||
const isPin = cs.identity_pin_ref === id;
|
||||
btn.classList.toggle("active", isPin);
|
||||
btn.title = isPin
|
||||
? "Continuity pin — click to unpin"
|
||||
: "Pin as identity guide for every scene (continuity pin)"
|
||||
+ (cs.auto_enabled === false ? " — also turns Auto continuity back on" : "");
|
||||
badge.hidden = !isPin;
|
||||
});
|
||||
}
|
||||
|
||||
function togglePin(m) {
|
||||
const cs = (S.get().project || {}).continuity_settings || {};
|
||||
const isPin = cs.identity_pin_ref === m.id;
|
||||
const patch = { ...cs, identity_pin_ref: isPin ? null : m.id };
|
||||
if (!isPin && patch.auto_enabled === false) patch.auto_enabled = true;
|
||||
S.patchProject({ continuity_settings: patch });
|
||||
refreshPins();
|
||||
}
|
||||
|
||||
function cellFor(m) {
|
||||
const cell = el("div", "gal-cell");
|
||||
cell.title = m.name || m.id;
|
||||
const thumb = el("div", "gal-thumb");
|
||||
const url = API.mediaUrl(m.id);
|
||||
if (m.kind === "video") {
|
||||
const v = el("video");
|
||||
v.src = url; v.muted = true; v.preload = "metadata";
|
||||
thumb.append(v);
|
||||
} else {
|
||||
const img = el("img");
|
||||
img.src = url; img.loading = "lazy";
|
||||
thumb.append(img);
|
||||
}
|
||||
if (m.kind === "image" && window.PipelineCaps?.usesChainSampler(S.get())) {
|
||||
const btn = el("button", "gal-pin-btn", "📌");
|
||||
btn.type = "button";
|
||||
btn.onclick = (e) => { e.stopPropagation(); togglePin(m); };
|
||||
thumb.append(btn);
|
||||
const badge = el("span", "gal-pin-badge", "📌");
|
||||
badge.hidden = true;
|
||||
thumb.append(badge);
|
||||
pinEls.push({ id: m.id, btn, badge });
|
||||
}
|
||||
cell.append(thumb);
|
||||
cell.append(el("div", "gal-name", m.name || m.id));
|
||||
cell.onclick = async () => {
|
||||
S.setSceneMedia(m.id, m.kind === "video" ? "video" : "image");
|
||||
await S.save();
|
||||
close();
|
||||
};
|
||||
return cell;
|
||||
}
|
||||
|
||||
async function open() {
|
||||
if (!S.get().project) return;
|
||||
close();
|
||||
pinEls.length = 0;
|
||||
overlay = el("div", "modal-overlay");
|
||||
const box = el("div", "modal modal-wide");
|
||||
const head = el("div", "modal-head");
|
||||
head.append(el("div", "modal-title", "Gallery — choose an anchor"));
|
||||
const x = el("button", "btn ghost tiny", "✕");
|
||||
x.onclick = close;
|
||||
head.append(el("div", "modal-head-right"), x);
|
||||
box.append(head);
|
||||
|
||||
const content = el("div", "modal-content gal-grid");
|
||||
content.append(el("div", "sw-hint", "Loading…"));
|
||||
box.append(content);
|
||||
|
||||
overlay.append(box);
|
||||
overlay.addEventListener("click", (e) => { if (e.target === overlay) close(); });
|
||||
document.addEventListener("keydown", function onKey(e) {
|
||||
if (e.key === "Escape") { close(); document.removeEventListener("keydown", onKey); }
|
||||
});
|
||||
document.body.append(overlay);
|
||||
|
||||
try {
|
||||
const r = await API.listMedia();
|
||||
const items = (r.media || []).filter((m) => m.kind === "image" || m.kind === "video");
|
||||
clear(content);
|
||||
if (!items.length) {
|
||||
content.append(el("div", "sw-hint", "No uploaded media yet — use Upload first."));
|
||||
return;
|
||||
}
|
||||
items.forEach((m) => content.append(cellFor(m)));
|
||||
refreshPins();
|
||||
} catch (e) {
|
||||
clear(content);
|
||||
content.append(el("div", "sw-hint", "Could not load media: " + (e.message || e)));
|
||||
}
|
||||
}
|
||||
|
||||
window.EasyGallery = { open, close };
|
||||
})();
|
||||
@@ -0,0 +1,104 @@
|
||||
// window.FunPackMaintenance shim — ports the five actions menubar.js exposes
|
||||
// for the Settings window's built-in "Refinement & Taste" / "Updates & ComfyUI"
|
||||
// sections (settings_window.js), without pulling in the other ~300 lines of
|
||||
// menubar.js (menu rendering, keyboard shortcuts, timeline-specific state).
|
||||
// Same API calls, same confirm/alert flows as the Editor.
|
||||
(function () {
|
||||
const API = window.MovieEditorAPI;
|
||||
|
||||
function projectRefinementKey() {
|
||||
return window.Store?.get().project?.refinement_key || "";
|
||||
}
|
||||
|
||||
const refinementKeyImportInput = (function () {
|
||||
const inp = document.createElement("input");
|
||||
inp.type = "file"; inp.accept = ".json,application/json"; inp.style.display = "none";
|
||||
inp.onchange = async () => {
|
||||
const file = inp.files?.[0];
|
||||
inp.value = "";
|
||||
if (!file) return;
|
||||
try {
|
||||
const data = JSON.parse(await file.text());
|
||||
let res;
|
||||
try {
|
||||
res = await API.importRefinementKey(data);
|
||||
} catch (e) {
|
||||
if (!e.exists) throw e;
|
||||
if (!confirm(`A refinement key named "${e.key}" already exists. Overwrite it?`)) return;
|
||||
res = await API.importRefinementKey(data, { overwrite: true });
|
||||
}
|
||||
alert(`Imported refinement key "${res.key}".`);
|
||||
} catch (e) {
|
||||
alert("Refinement key import failed: " + (e.message || e));
|
||||
}
|
||||
};
|
||||
document.body.appendChild(inp);
|
||||
return inp;
|
||||
})();
|
||||
|
||||
async function exportRefinementKey() {
|
||||
let payload;
|
||||
try { payload = await API.refinementKeys(); }
|
||||
catch (e) { alert("Could not list refinement keys: " + (e.message || e)); return; }
|
||||
const names = (payload.keys || []).filter((k) => k && k !== "-None-");
|
||||
if (!names.length) { alert("No refinement keys on disk yet."); return; }
|
||||
const cur = projectRefinementKey();
|
||||
if (!window.SlotPicker) return;
|
||||
window.SlotPicker.open({
|
||||
title: "Export refinement key",
|
||||
options: names.map((k) => ({ value: k, label: k, hint: k === cur ? "project session" : "" })),
|
||||
onPick: async (key) => {
|
||||
try { await API.exportRefinementKeyFile(key); }
|
||||
catch (e) { alert("Refinement key export failed: " + (e.message || e)); }
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
async function deleteRefinementKey() {
|
||||
let payload;
|
||||
try { payload = await API.refinementKeys(); }
|
||||
catch (e) { alert("Could not list refinement keys: " + (e.message || e)); return; }
|
||||
const names = (payload.keys || []).filter((k) => k && k !== "-None-");
|
||||
if (!names.length) { alert("No refinement keys on disk yet."); return; }
|
||||
const cur = projectRefinementKey();
|
||||
if (!window.SlotPicker) return;
|
||||
window.SlotPicker.open({
|
||||
title: "Delete refinement key",
|
||||
options: names.map((k) => ({ value: k, label: k, hint: k === cur ? "project session" : "" })),
|
||||
onPick: async (key) => {
|
||||
if (!confirm(`Delete refinement key "${key}"?\n\nThis removes its learned state AND all sidecars (value function, blessed attention/K-V banks, creativity latent, velocity memory). This cannot be undone.`)) {
|
||||
return;
|
||||
}
|
||||
try { await API.deleteRefinementKey(key); alert(`Deleted "${key}".`); }
|
||||
catch (e) { alert("Delete failed: " + (e.message || e)); }
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
async function clearGlobalTaste() {
|
||||
let info;
|
||||
try { info = await API.absoluteStoreInfo(); }
|
||||
catch (e) { alert("Could not read the global-taste store: " + (e.message || e)); return; }
|
||||
if (!info.exists) { alert("The Absolute global-taste store is already empty."); return; }
|
||||
if (!confirm(`Clear the Absolute global-taste store?\n\nIt has pooled ${info.total_iterations} rated generation(s) (${info.liked_count} liked / ${info.bad_count} disliked directions) across all prompts. It learns from every rated run and is applied only in absolute/both steer mode. This cannot be undone.`)) {
|
||||
return;
|
||||
}
|
||||
try { await API.clearAbsoluteStore(); alert("Cleared the Absolute global-taste store."); }
|
||||
catch (e) { alert("Clear global taste failed: " + (e.message || e)); }
|
||||
}
|
||||
|
||||
async function restartComfy() {
|
||||
if (!confirm("Restart ComfyUI now?\n\nThe server will be down for ~10-40s and any running generation will be lost. This page reloads automatically when it's back.")) return;
|
||||
const msg = window.FunPackRestart?.showOverlay?.("Restarting ComfyUI…\nThis page will reload when it's back.");
|
||||
try { await API.restart(); } catch (_) { /* connection drops as it execv's - expected */ }
|
||||
window.FunPackRestart?.waitForReload?.(msg, Date.now());
|
||||
}
|
||||
|
||||
window.FunPackMaintenance = {
|
||||
exportRefinementKey,
|
||||
importRefinementKeyFile: () => refinementKeyImportInput.click(),
|
||||
deleteRefinementKey,
|
||||
clearGlobalTaste,
|
||||
restartComfy,
|
||||
};
|
||||
})();
|
||||
@@ -0,0 +1,90 @@
|
||||
// Easy Gen's own slimmed "Generation" Settings section — modeled on the
|
||||
// field(label, control, hint) row pattern in movie_editor/frontend/engine_settings.js,
|
||||
// but limited to plain Project-level fields (no scene-chain/continuity/overlay
|
||||
// knobs, which only make sense with multiple scenes in the full Editor).
|
||||
(function () {
|
||||
const { el } = window.dom;
|
||||
|
||||
function field(labelText, control, hint) {
|
||||
const row = el("div", "sw-row eng-field");
|
||||
const main = el("div", "sw-row-main");
|
||||
main.append(el("div", "sw-row-title", labelText));
|
||||
if (hint) main.append(el("div", "sw-row-hint", hint));
|
||||
row.append(main, control);
|
||||
return row;
|
||||
}
|
||||
|
||||
function numberInput(value, onChange, opts = {}) {
|
||||
const inp = el("input", "sw-input");
|
||||
inp.type = "number";
|
||||
if (opts.min != null) inp.min = opts.min;
|
||||
if (opts.step != null) inp.step = opts.step;
|
||||
inp.value = value;
|
||||
inp.style.width = "90px";
|
||||
inp.onchange = () => onChange(Number(inp.value));
|
||||
return inp;
|
||||
}
|
||||
|
||||
function textInput(value, onChange, placeholder) {
|
||||
const inp = el("input", "sw-input");
|
||||
inp.type = "text";
|
||||
inp.value = value || "";
|
||||
if (placeholder) inp.placeholder = placeholder;
|
||||
inp.style.width = "220px";
|
||||
inp.onchange = () => onChange(inp.value);
|
||||
return inp;
|
||||
}
|
||||
|
||||
function mount(body) {
|
||||
const S = window.Store;
|
||||
const wrap = el("div", "sw-stack");
|
||||
body.append(wrap);
|
||||
|
||||
function render() {
|
||||
body.innerHTML = "";
|
||||
body.append(wrap);
|
||||
wrap.innerHTML = "";
|
||||
const st = S.get();
|
||||
const p = st.project;
|
||||
if (!p) {
|
||||
wrap.append(el("div", "sw-hint", "Open or create a project first."));
|
||||
return;
|
||||
}
|
||||
const commit = async (mut) => { mut(p); await S.save(); };
|
||||
|
||||
wrap.append(el("div", "sw-rows-label", "Output"));
|
||||
const out = el("div", "sw-rows");
|
||||
out.append(field("Width", numberInput(p.width, (v) => commit((pr) => pr.width = v), { min: 64, step: 16 })));
|
||||
out.append(field("Height", numberInput(p.height, (v) => commit((pr) => pr.height = v), { min: 64, step: 16 })));
|
||||
out.append(field("Frames", numberInput(p.num_frames_per_scene, (v) => commit((pr) => pr.num_frames_per_scene = v), { min: 9, step: 8 }),
|
||||
"Length of the generated clip, in frames."));
|
||||
out.append(field("Frame rate", numberInput(p.frame_rate, (v) => commit((pr) => pr.frame_rate = v), { min: 1 })));
|
||||
wrap.append(out);
|
||||
|
||||
wrap.append(el("div", "sw-rows-label", "Prompt"));
|
||||
const pr = el("div", "sw-rows");
|
||||
pr.append(field("Negative prompt", textInput(p.negative_prompt, (v) => commit((proj) => proj.negative_prompt = v)),
|
||||
"Passed to the negative conditioning for every run."));
|
||||
wrap.append(pr);
|
||||
|
||||
wrap.append(el("div", "sw-hint",
|
||||
"Studio/Chain Sampler/continuity controls, including Seed, live under Engine ▸ Timing "
|
||||
+ "& Seed (empty = a new random seed every Generate). Studio always runs in Prompt-only "
|
||||
+ "mode here — no refinement key, since Easy Gen has no rating UI to train one. For the "
|
||||
+ "full learned refiner, use the Cutting Room or the ComfyUI node graph."));
|
||||
}
|
||||
|
||||
const unsub = S.subscribe(render);
|
||||
render();
|
||||
return () => unsub();
|
||||
}
|
||||
|
||||
window.SettingsWindow.register({
|
||||
id: "generation", group: "Generation", order: 1, title: "Generation",
|
||||
subtitle: "Output size, length, and negative prompt for this project.",
|
||||
keywords: "width height frames frame rate negative prompt generation seed engine",
|
||||
iconBg: "linear-gradient(180deg,#ffb64d,#e07f1f)",
|
||||
icon: '<svg viewBox="0 0 16 16" width="13" height="13"><path d="M9.2 1.3 3 9h4.1l-1 5.7L12.9 7H8.5l.7-5.7z" fill="#fff"/></svg>',
|
||||
mount,
|
||||
});
|
||||
})();
|
||||
@@ -0,0 +1,156 @@
|
||||
// Easy Gen's "Shortcuts" settings section: prompt autocomplete + anchor toggles
|
||||
// (same per-browser settings/localStorage key as the Editor's editor_settings.js,
|
||||
// via Store.getEditorSetting/setEditorSetting) plus the shortcut revolver
|
||||
// (server-side, shared with the Editor and with generation regardless of which
|
||||
// UI queues the run). Deliberately NOT the whole of editor_settings.js — this
|
||||
// skips "Shortcut ideas" (not wanted here) and the "Anchor as guide" custom-node
|
||||
// bypass section (Easy Gen never creates anchor_guide scenes, so it would be
|
||||
// dead UI). revolverSection() below is intentionally a copy of
|
||||
// editor_settings.js's — it's small and purely API-driven (no Store deps beyond
|
||||
// dom/API), so duplicating it here keeps Easy Gen's settings free of the
|
||||
// Editor-only sections above without forcing a shared-module refactor.
|
||||
(function () {
|
||||
const { el } = window.dom;
|
||||
const S = window.Store;
|
||||
const API = window.MovieEditorAPI;
|
||||
|
||||
function toggleRow(label, hint, key) {
|
||||
const row = el("div", "es-row");
|
||||
const lbl = el("label", "chk es-toggle");
|
||||
const cb = el("input"); cb.type = "checkbox"; cb.checked = !!S.getEditorSetting(key);
|
||||
cb.onchange = () => S.setEditorSetting(key, cb.checked);
|
||||
lbl.append(cb, el("span", null, label));
|
||||
row.append(lbl);
|
||||
if (hint) row.append(el("div", "es-hint", hint));
|
||||
return row;
|
||||
}
|
||||
|
||||
function revolverSection() {
|
||||
const wrap = el("div", "es-section");
|
||||
wrap.append(el("div", "es-section-title", "Shortcut revolver"));
|
||||
wrap.append(el("div", "es-hint",
|
||||
"For shortcuts with several replacements: remember which replacement fired and don't reuse it "
|
||||
+ "until the WHOLE set has fired. Off = seeded random pick (may repeat). Saved with your shortcut "
|
||||
+ "library — applies to every browser and to generation."));
|
||||
|
||||
const main = el("label", "chk es-toggle");
|
||||
const mainCb = el("input"); mainCb.type = "checkbox"; mainCb.disabled = true;
|
||||
main.append(mainCb, el("span", null, "Shortcut revolver"));
|
||||
wrap.append(main);
|
||||
|
||||
const sub = el("div", "es-subfields");
|
||||
const rand = el("label", "chk es-toggle");
|
||||
const randCb = el("input"); randCb.type = "checkbox"; randCb.disabled = true;
|
||||
rand.append(randCb, el("span", null, "Random"));
|
||||
sub.append(rand);
|
||||
sub.append(el("div", "es-hint",
|
||||
"Unchecked: replacements fire in order — first, second, … last, then the cycle restarts. "
|
||||
+ "Checked: any order, but still no repeats until the set is exhausted. Changing either "
|
||||
+ "option restarts all cycles."));
|
||||
wrap.append(sub);
|
||||
|
||||
const apply = (s) => {
|
||||
mainCb.checked = !!(s && s.enabled);
|
||||
randCb.checked = !!(s && s.random);
|
||||
mainCb.disabled = randCb.disabled = false;
|
||||
sub.style.display = mainCb.checked ? "" : "none";
|
||||
};
|
||||
|
||||
const save = () => {
|
||||
mainCb.disabled = randCb.disabled = true;
|
||||
API.setRevolverSettings({ enabled: mainCb.checked, random: randCb.checked })
|
||||
.then(apply)
|
||||
.catch(() => { API.revolverSettings().then(apply).catch(() => {}); });
|
||||
};
|
||||
mainCb.onchange = save;
|
||||
randCb.onchange = save;
|
||||
|
||||
sub.style.display = "none";
|
||||
API.revolverSettings().then(apply).catch(() => {
|
||||
wrap.append(el("div", "es-hint", "Could not load revolver settings."));
|
||||
});
|
||||
return wrap;
|
||||
}
|
||||
|
||||
// Import/export the shortcut and transition ("prompt split") JSON libraries — one
|
||||
// shared FunPack library, same files the ComfyUI node graph and the Movie Editor
|
||||
// read/write. Easy Gen had no UI for this at all, so a fresh install or a browser
|
||||
// with an empty library had no way to load a shortcuts/transitions file.
|
||||
function ioRow(label, hint, onImport, exportUrl) {
|
||||
const row = el("div", "es-row");
|
||||
row.append(el("div", "es-toggle", label));
|
||||
if (hint) row.append(el("div", "es-hint", hint));
|
||||
|
||||
const modeSel = el("select", "es-io-mode");
|
||||
const optMerge = el("option", null, "Merge"); optMerge.value = "merge";
|
||||
const optReplace = el("option", null, "Replace all"); optReplace.value = "replace";
|
||||
modeSel.append(optMerge, optReplace);
|
||||
|
||||
const fileInput = el("input");
|
||||
fileInput.type = "file"; fileInput.accept = ".json,application/json"; fileInput.style.display = "none";
|
||||
fileInput.onchange = async () => {
|
||||
const file = fileInput.files?.[0];
|
||||
fileInput.value = "";
|
||||
if (!file) return;
|
||||
const mode = modeSel.value;
|
||||
if (mode === "replace"
|
||||
&& !confirm(`This replaces the ENTIRE ${label.toLowerCase()} library with the imported file. Continue?`)) return;
|
||||
try {
|
||||
const n = await onImport(file, mode);
|
||||
alert(`Imported ${n} ${label.toLowerCase()}.`);
|
||||
} catch (e) { alert("Import failed: " + (e.message || e)); }
|
||||
};
|
||||
|
||||
const importBtn = el("button", "btn ghost tiny", "Import…");
|
||||
importBtn.type = "button";
|
||||
importBtn.onclick = () => fileInput.click();
|
||||
|
||||
const exportBtn = el("button", "btn ghost tiny", "Export");
|
||||
exportBtn.type = "button";
|
||||
exportBtn.onclick = () => window.open(exportUrl(), "_blank");
|
||||
|
||||
const controls = el("div", "es-io-controls");
|
||||
controls.append(modeSel, importBtn, exportBtn, fileInput);
|
||||
row.append(controls);
|
||||
return row;
|
||||
}
|
||||
|
||||
function librarySection() {
|
||||
const wrap = el("div", "es-section");
|
||||
wrap.append(el("div", "es-section-title", "Shortcut & transition libraries"));
|
||||
wrap.append(el("div", "es-hint",
|
||||
"Load a shortcuts/transitions JSON file exported from the ComfyUI node graph or the Movie "
|
||||
+ "Editor, or export what's currently loaded. Merge overlays onto the existing library "
|
||||
+ "(same trigger name overwrites); Replace all wipes it first."));
|
||||
wrap.append(ioRow("Shortcuts", "Prompt-expansion triggers.",
|
||||
(file, mode) => S.importShortcuts(file, mode), () => API.exportShortcutsUrl()));
|
||||
wrap.append(ioRow("Transitions", "Split/cut trigger words used to break the global prompt into scenes.",
|
||||
(file, mode) => S.importTransitions(file, mode), () => API.exportTransitionsUrl()));
|
||||
return wrap;
|
||||
}
|
||||
|
||||
function mount(body) {
|
||||
const content = el("div", "es-content");
|
||||
content.append(toggleRow(
|
||||
"Prompt autocomplete",
|
||||
"Suggest matching shortcuts while you type. Shows the trigger, its prompt, and category.",
|
||||
"autocomplete"));
|
||||
content.append(toggleRow(
|
||||
"Use anchor",
|
||||
"Text before the first split trigger (\"cut\", etc.) is a shared anchor prepended to every "
|
||||
+ "scene. Turn off to make that leading text the first scene instead.",
|
||||
"anchorEnabled"));
|
||||
content.append(revolverSection());
|
||||
content.append(librarySection());
|
||||
body.append(content);
|
||||
}
|
||||
|
||||
window.SettingsWindow.register({
|
||||
id: "shortcuts", group: "Generation", order: 2, title: "Shortcuts",
|
||||
subtitle: "Autocomplete, anchor text, the shortcut revolver, and library import/export — shared with the Editor.",
|
||||
keywords: "autocomplete anchor shortcuts split cut revolver random replacements cycle import export json library transitions",
|
||||
iconBg: "linear-gradient(180deg,#6aa9ff,#3b6fd9)",
|
||||
icon: '<svg viewBox="0 0 16 16" width="13" height="13" fill="none" stroke="#fff" stroke-width="1.5" stroke-linecap="round"><path d="M2 4.5h12M2 8h12M2 11.5h12"/><circle cx="6" cy="4.5" r="1.7" fill="#fff" stroke="none"/><circle cx="11" cy="8" r="1.7" fill="#fff" stroke="none"/><circle cx="5" cy="11.5" r="1.7" fill="#fff" stroke="none"/></svg>',
|
||||
mount,
|
||||
});
|
||||
})();
|
||||
@@ -0,0 +1,368 @@
|
||||
// Minimal state container for Easy Gen — NOT the Editor's store.js (that's a
|
||||
// 4000+ line timeline/composer/overlay engine tightly coupled to the full NLE
|
||||
// DOM). This is a small pub-sub wrapper around the same /funpack/movie/api/*
|
||||
// endpoints, but it DOES support the same one-prompt-splits-into-N-scenes
|
||||
// flow as the Editor's Composer — the user only ever sees one textarea, the
|
||||
// resulting scenes are never surfaced in any per-scene UI, and Generate always
|
||||
// runs the whole project as one chain (no scene_ids filter), so N scenes come
|
||||
// back as one stitched video with no manual cut/trim ability, by design.
|
||||
//
|
||||
// Exposed as window.Store (same global name the Editor uses) purely so
|
||||
// reused sections — models.js, autocomplete.js, and the built-in "Refinement
|
||||
// & Taste" / "Updates & ComfyUI" sections baked into settings_window.js —
|
||||
// work unmodified: they all read window.Store.get().project/.shortcuts/etc.
|
||||
// and call window.Store.subscribe(fn) without needing to know which app they're in.
|
||||
(function () {
|
||||
const API = window.MovieEditorAPI;
|
||||
|
||||
const state = {
|
||||
project: null,
|
||||
health: null,
|
||||
resetSessionArmed: false,
|
||||
shortcuts: [], // prompt shortcut library (for autocomplete)
|
||||
transitions: [], // transition trigger library (for split-marker inference)
|
||||
mediaBin: [], // uploaded assets [{id,name,kind,...}] — feeds engine_settings.js's MediaPicker
|
||||
};
|
||||
const listeners = new Set();
|
||||
|
||||
function get() { return state; }
|
||||
|
||||
function notify() { listeners.forEach((fn) => { try { fn(state); } catch (_) {} }); }
|
||||
|
||||
function subscribe(fn) {
|
||||
listeners.add(fn);
|
||||
return () => listeners.delete(fn);
|
||||
}
|
||||
|
||||
function set(patch) {
|
||||
Object.assign(state, patch);
|
||||
notify();
|
||||
}
|
||||
|
||||
async function refreshHealth() {
|
||||
try { set({ health: await API.health() }); } catch (_) { set({ health: { ok: false } }); }
|
||||
}
|
||||
|
||||
// ── per-browser editor settings ──────────────────────────────────────────
|
||||
// Same localStorage key/shape as movie_editor/frontend/store.js so toggling
|
||||
// "Prompt autocomplete" or "Use anchor" is shared between the Editor and
|
||||
// Easy Gen on the same browser (matches the Editor's own cross-entry-point
|
||||
// behavior — these are per-browser prompt-editing prefs, not project data).
|
||||
const EDITOR_SETTINGS_KEY = "funpack_editor_settings";
|
||||
const EDITOR_SETTINGS_DEFAULTS = { autocomplete: true, anchorEnabled: true };
|
||||
function _loadEditorSettings() {
|
||||
try {
|
||||
const raw = JSON.parse(localStorage.getItem(EDITOR_SETTINGS_KEY) || "{}");
|
||||
return { ...EDITOR_SETTINGS_DEFAULTS, ...(raw && typeof raw === "object" ? raw : {}) };
|
||||
} catch (_) { return { ...EDITOR_SETTINGS_DEFAULTS }; }
|
||||
}
|
||||
let _editorSettings = _loadEditorSettings();
|
||||
function getEditorSetting(key) { return _editorSettings[key]; }
|
||||
function setEditorSetting(key, val) {
|
||||
if (_editorSettings[key] === val) return;
|
||||
_editorSettings = { ..._editorSettings, [key]: val };
|
||||
try { localStorage.setItem(EDITOR_SETTINGS_KEY, JSON.stringify(_editorSettings)); } catch (_) {}
|
||||
notify();
|
||||
if (key === "anchorEnabled" && state.project && (state.project.global_prompt || "").trim()) {
|
||||
applyGlobalPrompt(state.project.global_prompt);
|
||||
}
|
||||
}
|
||||
|
||||
async function loadLibraries() {
|
||||
try { const r = await API.shortcuts(); set({ shortcuts: r.shortcuts || [] }); } catch (_) {}
|
||||
try { const r = await API.transitions(); set({ transitions: r.transitions || r || [] }); } catch (_) {}
|
||||
}
|
||||
|
||||
async function loadMedia() {
|
||||
try { set({ mediaBin: (await API.listMedia()).media || [] }); } catch (_) { set({ mediaBin: [] }); }
|
||||
}
|
||||
|
||||
// Import/export the same shortcut/transition JSON libraries the ComfyUI node graph
|
||||
// and the Movie Editor read and write (one shared FunPack library, no Easy Gen copy).
|
||||
// mode: "merge" (overlay onto the existing library) | "replace" (wipe first).
|
||||
async function importShortcuts(file, mode) {
|
||||
const data = JSON.parse(await file.text());
|
||||
const r = await API.importShortcuts(data, mode);
|
||||
set({ shortcuts: r.shortcuts || state.shortcuts });
|
||||
return r.imported;
|
||||
}
|
||||
async function importTransitions(file, mode) {
|
||||
const data = JSON.parse(await file.text());
|
||||
const r = await API.importTransitions(data, mode);
|
||||
set({ transitions: r.transitions || state.transitions });
|
||||
return r.imported;
|
||||
}
|
||||
|
||||
function emptyScene() {
|
||||
return { text: "", source: { type: "empty" } };
|
||||
}
|
||||
|
||||
async function listProjects() {
|
||||
return (await API.listProjects()).projects || [];
|
||||
}
|
||||
|
||||
async function createProject(name) {
|
||||
const p = await API.createProject(name || "Untitled");
|
||||
p.scenes = [emptyScene()];
|
||||
const saved = await API.saveProject(p.id, p);
|
||||
set({ project: saved });
|
||||
return saved;
|
||||
}
|
||||
|
||||
// Fallback for projects saved before the global-prompt split existed, or
|
||||
// edited scene-by-scene in the Editor without ever using its Composer:
|
||||
// reconstruct a verbatim prompt from anchor + scene texts + transition
|
||||
// markers so the textarea shows something sensible instead of going blank.
|
||||
function buildPromptFallback(p) {
|
||||
const parts = [];
|
||||
const anchor = (p.anchor || "").trim();
|
||||
if (anchor) parts.push(anchor);
|
||||
(p.scenes || []).forEach((s, i) => {
|
||||
if (i > 0) parts.push((p.scenes[i - 1].transition_to_next || "cut").trim());
|
||||
const t = (s.text || "").trim();
|
||||
if (t) parts.push(t);
|
||||
});
|
||||
return parts.filter(Boolean).join(" ");
|
||||
}
|
||||
|
||||
async function loadProject(id) {
|
||||
const p = await API.getProject(id);
|
||||
if (!p.scenes || !p.scenes.length) p.scenes = [emptyScene()];
|
||||
if (!(p.global_prompt || "").trim()) p.global_prompt = buildPromptFallback(p);
|
||||
set({ project: p });
|
||||
return p;
|
||||
}
|
||||
|
||||
async function deleteProject(id) {
|
||||
await API.deleteProject(id);
|
||||
if (state.project && state.project.id === id) set({ project: null });
|
||||
}
|
||||
|
||||
// ── global prompt → N scenes (mirrors the Editor's Composer Apply, minus
|
||||
// the timeline/ghost/history machinery Easy Gen has no use for) ──────────
|
||||
|
||||
// Ported from movie_editor/frontend/store.js's _normalizeGlobalPromptParse:
|
||||
// folds the parser's "anchor" back into Scene 1 when the anchor setting is off.
|
||||
function _normalizeGlobalPromptParse(trimmed, v) {
|
||||
v = v || {};
|
||||
const text = String(trimmed || "").trim();
|
||||
const anchor = String(v.anchor || "").trim();
|
||||
const scenes = Array.isArray(v.scenes) ? v.scenes : [];
|
||||
const hasSceneText = scenes.some((s) => String(s.text || "").trim());
|
||||
|
||||
if (!hasSceneText && text) return { anchor: "", scenes: [{ text }] };
|
||||
if (!hasSceneText && anchor) return { anchor: "", scenes: [{ text: anchor }] };
|
||||
if (!_editorSettings.anchorEnabled && anchor) {
|
||||
return { anchor: "", scenes: [{ text: anchor }, ...scenes] };
|
||||
}
|
||||
return { anchor: v.anchor || "", scenes };
|
||||
}
|
||||
|
||||
// Ported from movie_editor/frontend/store.js's _inferSplitMarkersFromPrompt:
|
||||
// finds which transition trigger (if any) sits in the raw gap text between
|
||||
// two consecutive verbatim scene chunks, so the chain sampler gets the same
|
||||
// seam behavior ("cut", etc.) the Editor would have produced.
|
||||
function _inferSplitMarkers(scenes, fullPrompt) {
|
||||
if (!fullPrompt || !scenes.length) return;
|
||||
const markers = (state.transitions || []).filter((m) => m.enabled !== false && (m.trigger || m.name));
|
||||
let pos = 0;
|
||||
for (let i = 0; i < scenes.length; i++) {
|
||||
const t = scenes[i].text || "";
|
||||
if (!t) continue;
|
||||
const idx = fullPrompt.indexOf(t, pos);
|
||||
if (idx < 0) continue;
|
||||
const endOfScene = idx + t.length;
|
||||
if (i < scenes.length - 1) {
|
||||
const nextT = scenes[i + 1].text || "";
|
||||
const nextIdx = nextT ? fullPrompt.indexOf(nextT, endOfScene) : endOfScene;
|
||||
const gapEnd = nextIdx >= 0 ? nextIdx : fullPrompt.length;
|
||||
const gap = fullPrompt.slice(endOfScene, gapEnd);
|
||||
for (const m of markers) {
|
||||
const trig = m.trigger || m.name;
|
||||
if (trig && gap.includes(trig)) { scenes[i].transition_to_next = trig; break; }
|
||||
}
|
||||
}
|
||||
pos = endOfScene;
|
||||
}
|
||||
}
|
||||
|
||||
// Sequence guard: if the user keeps typing while a parse is in flight, an
|
||||
// older (now-stale) response must not overwrite scenes built from newer text.
|
||||
let _applySeq = 0;
|
||||
|
||||
// Splits `text` into project.anchor + project.scenes via the same backend
|
||||
// parser the Editor's Composer uses (POST /projects/{id}/parse, lossless —
|
||||
// no shortcut expansion, no scene UI). Scene 0 keeps whatever upload/guide
|
||||
// it already had; new scenes 1..N-1 default to "carry" (continue the chain),
|
||||
// matching how the Editor treats anchorless continuation scenes.
|
||||
async function applyGlobalPrompt(text) {
|
||||
if (!state.project) return false;
|
||||
const trimmed = String(text || "").trim();
|
||||
state.project.global_prompt = trimmed;
|
||||
if (!trimmed) {
|
||||
const prevSource = state.project.scenes[0]?.source || { type: "empty" };
|
||||
state.project.scenes = [{ text: "", source: prevSource }];
|
||||
state.project.anchor = "";
|
||||
notify();
|
||||
return true;
|
||||
}
|
||||
const seq = ++_applySeq;
|
||||
let res;
|
||||
try { res = await API.parsePrompt(state.project.id, trimmed); }
|
||||
catch (e) { console.warn("Prompt parse failed:", e); return false; }
|
||||
if (seq !== _applySeq) return false; // superseded by a newer keystroke
|
||||
if (!state.project || state.project.global_prompt !== trimmed) return false; // project changed underneath us
|
||||
const v = _normalizeGlobalPromptParse(trimmed, res.parsed_verbatim || res.parsed_raw || res.parsed || {});
|
||||
if (!(v.scenes || []).length) return false;
|
||||
const prevSource = state.project.scenes[0]?.source || { type: "empty" };
|
||||
state.project.anchor = v.anchor || "";
|
||||
state.project.scenes = v.scenes.map((s, i) => ({
|
||||
text: String(s.text || "").trim(),
|
||||
transition_to_next: "",
|
||||
source: i === 0 ? prevSource : { type: "carry" },
|
||||
}));
|
||||
_inferSplitMarkers(state.project.scenes, trimmed);
|
||||
notify();
|
||||
return true;
|
||||
}
|
||||
|
||||
let _applyTimer = null;
|
||||
function scheduleGlobalPromptApply(text) {
|
||||
if (_applyTimer) clearTimeout(_applyTimer);
|
||||
_applyTimer = setTimeout(() => { applyGlobalPrompt(text).then(() => save().catch(() => {})); }, 500);
|
||||
}
|
||||
|
||||
// kind: "image" | "video" | null (null clears the upload → pure t2v).
|
||||
function setSceneMedia(mediaId, kind) {
|
||||
if (!state.project) return;
|
||||
state.project.scenes[0].source = mediaId
|
||||
? { type: kind === "video" ? "v2v" : "image", media_ref: mediaId }
|
||||
: { type: "empty" };
|
||||
notify();
|
||||
}
|
||||
|
||||
// Two overlapping save() calls (e.g. a debounced auto-save from an earlier
|
||||
// edit still in flight when a later action calls save() directly) can land
|
||||
// on the server out of order — whichever PUT is written to disk last wins,
|
||||
// even if it was the older/now-stale one, silently reverting a newer edit.
|
||||
// Serializing through one chain means a save only starts once the previous
|
||||
// one has FULLY completed, and reads state.project live at that point, so
|
||||
// it always picks up whatever changed while it waited.
|
||||
let saveQueue = Promise.resolve();
|
||||
function save() {
|
||||
const run = saveQueue.then(async () => {
|
||||
if (!state.project) return null;
|
||||
// Models & Pipeline (models.js) saves bypass/expose/widget edits straight to the
|
||||
// server via its own PUT /projects/{id}/models call — it never touches state.project
|
||||
// here. Without refetching first, this save would send project.models as it was when
|
||||
// the project was last loaded, silently reverting whatever was just changed there
|
||||
// (same class of bug the Editor's store.js fixed by syncing state.models into
|
||||
// state.project.models before every autosave commit).
|
||||
try { state.project.models = await API.getModels(state.project.id); } catch (_) {}
|
||||
const saved = await API.saveProject(state.project.id, state.project);
|
||||
set({ project: saved });
|
||||
return saved;
|
||||
});
|
||||
saveQueue = run.catch(() => {}); // keep the queue alive even if this save failed
|
||||
return run;
|
||||
}
|
||||
|
||||
// ── Engine settings (Studio / Chain Sampler / continuity) ──────────────────
|
||||
// Minimal re-implementation of movie_editor/frontend/store.js's project-patch
|
||||
// helpers, just enough for the shared engine_settings.js (loaded via the same
|
||||
// frontend-fallback mechanism as autocomplete.js/suggestions.js) to run
|
||||
// unmodified against Easy Gen's project. No undo/history, no global-prompt
|
||||
// resync — Easy Gen's scenes come from the split parser, not manual editing.
|
||||
let _engineSaveTimer = null;
|
||||
function _scheduleEngineSave(ms) {
|
||||
if (_engineSaveTimer) clearTimeout(_engineSaveTimer);
|
||||
_engineSaveTimer = setTimeout(() => { _engineSaveTimer = null; save().catch(() => {}); }, ms);
|
||||
}
|
||||
|
||||
function patchProject(patch) {
|
||||
if (!state.project) return;
|
||||
Object.assign(state.project, patch);
|
||||
notify();
|
||||
save().catch(() => {});
|
||||
}
|
||||
function patchProjectQuiet(patch) {
|
||||
if (!state.project) return;
|
||||
Object.assign(state.project, patch);
|
||||
notify();
|
||||
_scheduleEngineSave(600);
|
||||
}
|
||||
|
||||
function setConditioningSlot(slotId) { patchProject({ conditioning_slot: slotId || "funpack" }); }
|
||||
function setSamplerSlot(slotId) { patchProject({ sampler_slot: slotId || "funpack" }); }
|
||||
function setSamplerInput(name, value) {
|
||||
if (!state.project) return;
|
||||
patchProjectQuiet({ sampler_inputs: { ...(state.project.sampler_inputs || {}), [name]: value } });
|
||||
}
|
||||
function setSamplerInputNow(name, value) {
|
||||
if (!state.project) return;
|
||||
patchProject({ sampler_inputs: { ...(state.project.sampler_inputs || {}), [name]: value } });
|
||||
}
|
||||
function unsetSamplerInput(name) {
|
||||
if (!state.project) return;
|
||||
const prev = { ...(state.project.sampler_inputs || {}) };
|
||||
delete prev[name];
|
||||
patchProjectQuiet({ sampler_inputs: prev });
|
||||
}
|
||||
function setStudioInput(name, value) {
|
||||
if (!state.project) return;
|
||||
patchProjectQuiet({ studio_inputs: { ...(state.project.studio_inputs || {}), [name]: value } });
|
||||
}
|
||||
function setStudioInputNow(name, value) {
|
||||
if (!state.project) return;
|
||||
patchProject({ studio_inputs: { ...(state.project.studio_inputs || {}), [name]: value } });
|
||||
}
|
||||
|
||||
const ENGINE_PRESETS = {
|
||||
draft: { label: "Fast draft", studio_inputs: { "refiner.split_by_transitions": false }, sampler_inputs: { steps: 20 } },
|
||||
quality: { label: "Quality", studio_inputs: { "refiner.split_by_transitions": true }, sampler_inputs: { steps: 40 } },
|
||||
continuity: {
|
||||
label: "Continuity-heavy",
|
||||
guide_settings: { stack_enabled: true, auto_guides: true, identity_pins: true },
|
||||
continuity_settings: { enabled: true },
|
||||
sampler_inputs: { steps: 36 },
|
||||
},
|
||||
};
|
||||
function applyEnginePreset(key) {
|
||||
const preset = ENGINE_PRESETS[key];
|
||||
if (!preset || !state.project) return;
|
||||
const patch = {};
|
||||
if (preset.studio_inputs) patch.studio_inputs = { ...(state.project.studio_inputs || {}), ...preset.studio_inputs };
|
||||
if (preset.sampler_inputs) patch.sampler_inputs = { ...(state.project.sampler_inputs || {}), ...preset.sampler_inputs };
|
||||
if (preset.guide_settings) patch.guide_settings = { ...(state.project.guide_settings || {}), ...preset.guide_settings };
|
||||
if (preset.continuity_settings) patch.continuity_settings = { ...(state.project.continuity_settings || {}), ...preset.continuity_settings };
|
||||
patchProject(patch);
|
||||
}
|
||||
|
||||
function resetStudioSession() {
|
||||
set({ resetSessionArmed: !state.resetSessionArmed });
|
||||
}
|
||||
|
||||
// Consumed by the Generate flow: reset_session applies to the NEXT run only.
|
||||
function takeResetSessionFlag() {
|
||||
const armed = state.resetSessionArmed;
|
||||
if (armed) set({ resetSessionArmed: false });
|
||||
return armed;
|
||||
}
|
||||
|
||||
window.Store = {
|
||||
get, subscribe, set,
|
||||
refreshHealth,
|
||||
getEditorSetting, setEditorSetting,
|
||||
loadLibraries, importShortcuts, importTransitions,
|
||||
loadMedia,
|
||||
listProjects, createProject, loadProject, deleteProject,
|
||||
applyGlobalPrompt, scheduleGlobalPromptApply,
|
||||
setSceneMedia, save,
|
||||
patchProject, patchProjectQuiet,
|
||||
setConditioningSlot, setSamplerSlot,
|
||||
setSamplerInput, setSamplerInputNow, unsetSamplerInput,
|
||||
setStudioInput, setStudioInputNow,
|
||||
ENGINE_PRESETS, applyEnginePreset,
|
||||
resetStudioSession, takeResetSessionFlag,
|
||||
};
|
||||
})();
|
||||
@@ -0,0 +1,105 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>FunPack · Easy Gen</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
|
||||
<link href="https://fonts.googleapis.com/css2?family=Bricolage+Grotesque:opsz,wght@12..96,500;12..96,700;12..96,800&family=IBM+Plex+Mono:wght@400;500;600&family=IBM+Plex+Sans:wght@400;500;600;700&display=swap" rel="stylesheet" />
|
||||
<link rel="stylesheet" href="styles.css" />
|
||||
<link rel="stylesheet" href="easy.css" />
|
||||
<script>
|
||||
(function () {
|
||||
// A single failed shared script (e.g. dom.js) cascades into near-identical
|
||||
// errors from every script that depends on it — keep the first few DISTINCT
|
||||
// messages instead of overwriting, so the root cause (usually the first one)
|
||||
// stays visible instead of being clobbered by the last script in load order.
|
||||
var seen = [];
|
||||
function show(msg) {
|
||||
if (seen.indexOf(msg) !== -1) return;
|
||||
seen.push(msg);
|
||||
if (seen.length > 5) return;
|
||||
var b = document.getElementById("boot-error");
|
||||
if (!b) {
|
||||
b = document.createElement("div"); b.id = "boot-error";
|
||||
b.style.cssText = "position:fixed;z-index:99999;left:0;right:0;top:0;padding:10px 34px 10px 14px;" +
|
||||
"background:#3a1320;color:#ffb3c1;font:12px/1.5 monospace;border-bottom:1px solid #e0566b;white-space:pre-wrap;max-height:60vh;overflow:auto";
|
||||
var x = document.createElement("button");
|
||||
x.textContent = "✕"; x.title = "Dismiss";
|
||||
x.style.cssText = "position:absolute;top:6px;right:8px;background:transparent;border:1px solid #e0566b;" +
|
||||
"color:#ffb3c1;border-radius:6px;cursor:pointer;font:12px monospace;padding:1px 7px;line-height:1.4";
|
||||
x.onclick = function () { b.remove(); };
|
||||
b.appendChild(x);
|
||||
(document.body || document.documentElement).appendChild(b);
|
||||
}
|
||||
var t = b.querySelector(".boot-error-text");
|
||||
if (!t) { t = document.createElement("span"); t.className = "boot-error-text"; b.insertBefore(t, b.firstChild); }
|
||||
t.textContent = "Easy Gen error(s) — " + seen.join("\n---\n")
|
||||
+ "\n(restart ComfyUI + hard-reload Cmd/Ctrl+Shift+R; if it persists, send me this text)";
|
||||
}
|
||||
window.addEventListener("error", function (e) {
|
||||
if (e && e.target && e.target !== window && e.target.nodeName) return;
|
||||
show((e.message || "load error") + (e.filename ? " @ " + e.filename.split("/").pop() + ":" + e.lineno : ""));
|
||||
}, true);
|
||||
window.addEventListener("unhandledrejection", function (e) {
|
||||
show("promise: " + ((e.reason && (e.reason.message || e.reason)) || "rejected"));
|
||||
});
|
||||
})();
|
||||
</script>
|
||||
</head>
|
||||
<body class="easy-body">
|
||||
<div class="easy-app" id="easy-app">
|
||||
<header class="easy-top">
|
||||
<a class="brand" href="/funpack/" title="FunPack Hub"><span class="brand-mark">◉</span> FunPack <span class="brand-sub">Easy Gen</span></a>
|
||||
<div class="easy-project-name" id="easy-project-name">No project</div>
|
||||
<span class="health-chip" id="easy-health-chip"></span>
|
||||
<div class="menubar-right">
|
||||
<button class="btn ghost tiny" id="easy-save-btn" type="button">Save</button>
|
||||
<button class="btn ghost tiny" id="easy-export-btn" type="button"
|
||||
title="Download this project as a .funpack_project.json file — e.g. to move it off a rental machine">Export…</button>
|
||||
<button class="btn ghost tiny" id="easy-projects-btn" type="button">Projects…</button>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<main class="easy-preview" id="easy-preview"></main>
|
||||
|
||||
<footer class="easy-bar">
|
||||
<button class="btn ghost easy-upload-btn" id="easy-upload-btn" type="button"
|
||||
title="Upload an image or video to guide generation">⬆ Upload</button>
|
||||
<button class="btn ghost easy-gallery-btn" id="easy-gallery-btn" type="button"
|
||||
title="Choose from previously uploaded media">🖼 Gallery</button>
|
||||
<textarea class="easy-prompt" id="easy-prompt" rows="2"
|
||||
placeholder="Describe what you want to generate…"></textarea>
|
||||
<div class="easy-actions">
|
||||
<button class="btn primary easy-generate-btn" id="easy-generate-btn" type="button">Generate</button>
|
||||
<button class="btn ghost tiny easy-advanced-btn" id="easy-advanced-btn" type="button">Advanced settings</button>
|
||||
</div>
|
||||
</footer>
|
||||
<input type="file" id="easy-upload-input" accept="image/*,video/*" hidden />
|
||||
</div>
|
||||
|
||||
<script>window.FunPackAppName = "Easy Gen";</script>
|
||||
<script src="api.js"></script>
|
||||
<script src="dom.js"></script>
|
||||
<script src="settings_window.js"></script>
|
||||
<script src="git_actions.js"></script>
|
||||
<script src="restart_ui.js"></script>
|
||||
<script src="slot_picker.js"></script>
|
||||
<script src="pipeline_caps.js"></script>
|
||||
<script src="models.js"></script>
|
||||
<script src="easy_store.js"></script>
|
||||
<script src="easy_maintenance.js"></script>
|
||||
<script src="easy_settings.js"></script>
|
||||
<script src="media_picker.js"></script>
|
||||
<script src="sampler_panel.js"></script>
|
||||
<script src="engine_settings.js"></script>
|
||||
<script src="easy_shortcuts_settings.js"></script>
|
||||
<script src="autocomplete.js"></script>
|
||||
<script src="project_menu.js"></script>
|
||||
<script src="easy_gallery.js"></script>
|
||||
<script src="log.js"></script>
|
||||
<script src="preview.js"></script>
|
||||
<script src="app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,196 @@
|
||||
// Big preview pane: plain <video>/<img>, not the Editor's timeline-oriented
|
||||
// player.js. Polls /api/progress (step count) while a run is in flight and
|
||||
// /api/projects/{id}/status/{prompt_id} for the final result.
|
||||
//
|
||||
// Display priority: generated result > uploaded source > empty message. The
|
||||
// corner ✕ closes whichever of those two media layers is currently showing —
|
||||
// closing a generated result reveals the uploaded source underneath (if any),
|
||||
// closing the uploaded source clears the upload itself (via onClearUpload).
|
||||
(function () {
|
||||
const { el, clear } = window.dom;
|
||||
const API = window.MovieEditorAPI;
|
||||
|
||||
function create(root) {
|
||||
const inner = el("div", "easy-preview-inner");
|
||||
const closeBtn = el("button", "easy-preview-close", "✕");
|
||||
closeBtn.type = "button";
|
||||
closeBtn.title = "Close";
|
||||
closeBtn.hidden = true;
|
||||
const badge = el("div", "easy-preview-badge", "Uploaded source");
|
||||
badge.hidden = true;
|
||||
const progressWrap = el("div", "easy-progress");
|
||||
progressWrap.hidden = true;
|
||||
const bar = el("div", "easy-progress-bar");
|
||||
const label = el("div", "easy-progress-label");
|
||||
progressWrap.append(bar, label);
|
||||
root.append(inner, closeBtn, badge, progressWrap);
|
||||
|
||||
let pollTimer = null;
|
||||
let progressTimer = null;
|
||||
|
||||
// generated/upload hold the last-drawn payload so the ✕ handler knows what
|
||||
// layer it's dismissing; emptyMsg is what to fall back to once both are gone.
|
||||
let generated = null; // { media, projectId } | null
|
||||
let upload = null; // { url, kind } | null
|
||||
let errorMsg = null;
|
||||
let emptyMsg = "No generation yet";
|
||||
let onClearUpload = null;
|
||||
|
||||
function stopPolling() {
|
||||
if (pollTimer) clearTimeout(pollTimer);
|
||||
if (progressTimer) clearInterval(progressTimer);
|
||||
pollTimer = progressTimer = null;
|
||||
}
|
||||
|
||||
function mediaEl(url, kind, opts = {}) {
|
||||
if (kind === "video") {
|
||||
const video = el("video");
|
||||
video.src = url;
|
||||
video.controls = true;
|
||||
// Not the `autoplay` attribute: that makes the browser call .play() internally, and
|
||||
// if this element gets replaced/removed before the fetch starts playing (e.g. the
|
||||
// user immediately picks a different Gallery item), that internal promise rejects
|
||||
// with "fetching process...aborted by the user agent" — surfacing as an unhandled
|
||||
// rejection banner, since we never had a handle on it to catch. Call .play()
|
||||
// ourselves so we own the promise.
|
||||
if (opts.autoplay) {
|
||||
video.loop = true;
|
||||
video.play().catch(() => {});
|
||||
}
|
||||
return video;
|
||||
}
|
||||
const img = el("img");
|
||||
img.src = url;
|
||||
return img;
|
||||
}
|
||||
|
||||
// The store notifies on every health poll / prompt keystroke, and each of those calls
|
||||
// setUpload()/render() again — without this, draw() rebuilt the <video> from scratch
|
||||
// every time (new element = autoplay/loop reapplied), silently resuming anything the
|
||||
// user had paused. Skip the rebuild when the content actually shown hasn't changed.
|
||||
let _lastSig = null;
|
||||
function _sig() {
|
||||
if (errorMsg) return "error:" + errorMsg;
|
||||
if (generated) return "generated:" + generated.projectId + ":" + generated.media.filename;
|
||||
if (upload) return "upload:" + upload.url + ":" + upload.kind;
|
||||
return "empty:" + emptyMsg;
|
||||
}
|
||||
|
||||
function draw() {
|
||||
const sig = _sig();
|
||||
if (sig === _lastSig) return;
|
||||
_lastSig = sig;
|
||||
clear(inner);
|
||||
badge.hidden = true;
|
||||
if (errorMsg) {
|
||||
closeBtn.hidden = true;
|
||||
inner.append(el("div", "easy-preview-empty easy-preview-error", errorMsg));
|
||||
} else if (generated) {
|
||||
closeBtn.hidden = false;
|
||||
const url = API.resultUrl(generated.projectId, generated.media);
|
||||
inner.append(mediaEl(url, generated.media.kind === "images" ? "image" : "video", { autoplay: true }));
|
||||
} else if (upload) {
|
||||
closeBtn.hidden = false;
|
||||
badge.hidden = false;
|
||||
inner.append(mediaEl(upload.url, upload.kind));
|
||||
} else {
|
||||
closeBtn.hidden = true;
|
||||
inner.append(el("div", "easy-preview-empty", emptyMsg));
|
||||
}
|
||||
}
|
||||
|
||||
closeBtn.onclick = () => {
|
||||
if (generated) {
|
||||
generated = null;
|
||||
draw();
|
||||
} else if (upload) {
|
||||
upload = null;
|
||||
draw();
|
||||
if (onClearUpload) onClearUpload();
|
||||
}
|
||||
};
|
||||
|
||||
function setUpload(url, kind) {
|
||||
upload = url ? { url, kind } : null;
|
||||
draw();
|
||||
}
|
||||
|
||||
function setGenerated(media, projectId) {
|
||||
generated = media ? { media, projectId } : null;
|
||||
draw();
|
||||
}
|
||||
|
||||
function showEmpty(msg) {
|
||||
errorMsg = null;
|
||||
generated = null;
|
||||
emptyMsg = msg || "No generation yet";
|
||||
draw();
|
||||
}
|
||||
|
||||
function showError(msg) {
|
||||
errorMsg = msg;
|
||||
draw();
|
||||
}
|
||||
|
||||
async function watch(projectId, promptId, { onDone } = {}) {
|
||||
stopPolling();
|
||||
errorMsg = null;
|
||||
progressWrap.hidden = false;
|
||||
label.textContent = "Starting…";
|
||||
bar.style.width = "0%";
|
||||
|
||||
progressTimer = setInterval(async () => {
|
||||
try {
|
||||
const pr = await API.progress();
|
||||
if (pr.max > 0) {
|
||||
const pct = Math.min(100, Math.round((pr.value / pr.max) * 100));
|
||||
bar.style.width = pct + "%";
|
||||
label.textContent = `Sampling… ${pr.value}/${pr.max}`;
|
||||
}
|
||||
} catch (_) { /* transient */ }
|
||||
}, 700);
|
||||
|
||||
const poll = async () => {
|
||||
let res;
|
||||
try {
|
||||
res = await API.status(projectId, promptId);
|
||||
} catch (e) {
|
||||
stopPolling();
|
||||
progressWrap.hidden = true;
|
||||
showError("Status check failed: " + (e.message || e));
|
||||
if (onDone) onDone(false);
|
||||
return;
|
||||
}
|
||||
if (res.state === "completed") {
|
||||
stopPolling();
|
||||
progressWrap.hidden = true;
|
||||
const media = res.media && res.media.length ? res.media[res.media.length - 1] : null;
|
||||
if (media) setGenerated(media, projectId); else showEmpty("Generation finished but produced no media.");
|
||||
if (onDone) onDone(true, res);
|
||||
return;
|
||||
}
|
||||
if (res.state === "error") {
|
||||
stopPolling();
|
||||
progressWrap.hidden = true;
|
||||
showError(res.error || "Generation failed.");
|
||||
if (onDone) onDone(false, res);
|
||||
return;
|
||||
}
|
||||
pollTimer = setTimeout(poll, 1200);
|
||||
};
|
||||
poll();
|
||||
}
|
||||
|
||||
return {
|
||||
setUpload,
|
||||
setGenerated,
|
||||
setOnClearUpload(fn) { onClearUpload = fn; },
|
||||
showEmpty,
|
||||
showError,
|
||||
watch,
|
||||
stopPolling,
|
||||
};
|
||||
}
|
||||
|
||||
window.EasyPreview = { create };
|
||||
})();
|
||||
@@ -0,0 +1,104 @@
|
||||
// "FunPack Easy Gen" project picker — same /funpack/movie/api/projects* calls
|
||||
// as the Editor's welcome.js / menubar.js "Open recent" menu, reimplemented as
|
||||
// a small standalone modal (welcome.js/menubar.js are wired into the full
|
||||
// Editor chrome and not reusable in isolation).
|
||||
(function () {
|
||||
const { el, clear } = window.dom;
|
||||
const API = window.MovieEditorAPI;
|
||||
const S = window.Store;
|
||||
let overlay = null;
|
||||
|
||||
const loadFileInput = (function () {
|
||||
const inp = document.createElement("input");
|
||||
inp.type = "file";
|
||||
inp.accept = ".json,.funpack_project.json,application/json";
|
||||
inp.style.display = "none";
|
||||
inp.onchange = async () => {
|
||||
const file = inp.files?.[0];
|
||||
inp.value = "";
|
||||
if (!file) return;
|
||||
try {
|
||||
const data = JSON.parse(await file.text());
|
||||
const p = await API.importProject(data);
|
||||
S.set({ project: p });
|
||||
close();
|
||||
} catch (e) {
|
||||
alert("Could not load project file: " + (e.message || e));
|
||||
}
|
||||
};
|
||||
document.body.appendChild(inp);
|
||||
return inp;
|
||||
})();
|
||||
|
||||
function close() {
|
||||
if (overlay) { overlay.remove(); overlay = null; }
|
||||
}
|
||||
|
||||
async function open({ dismissable } = {}) {
|
||||
close();
|
||||
overlay = el("div", "modal-overlay");
|
||||
if (!dismissable) overlay.style.pointerEvents = "auto";
|
||||
const box = el("div", "modal modal-wide");
|
||||
const head = el("div", "modal-head");
|
||||
head.append(el("div", "modal-title", "FunPack Easy Gen"));
|
||||
if (dismissable) {
|
||||
const x = el("button", "btn ghost tiny", "✕");
|
||||
x.onclick = close;
|
||||
head.append(el("div", "modal-head-right"), x);
|
||||
}
|
||||
box.append(head);
|
||||
|
||||
const content = el("div", "modal-content");
|
||||
box.append(content);
|
||||
|
||||
const actions = el("div", "welcome-actions");
|
||||
const mkBtn = (label, cls, hint, fn) => {
|
||||
const b = el("button", "welcome-btn" + (cls ? " " + cls : ""));
|
||||
b.type = "button";
|
||||
b.append(el("span", "welcome-btn-label", label));
|
||||
if (hint) b.append(el("span", "welcome-btn-hint", hint));
|
||||
b.onclick = fn;
|
||||
return b;
|
||||
};
|
||||
actions.append(
|
||||
mkBtn("New Generation", "primary", "Start with a blank prompt", async () => {
|
||||
await S.createProject("Untitled");
|
||||
close();
|
||||
}),
|
||||
mkBtn("Load Project File…", "", "Import a .funpack_project.json", () => loadFileInput.click()),
|
||||
);
|
||||
content.append(actions);
|
||||
|
||||
content.append(el("div", "sw-rows-label", "Recent"));
|
||||
const recentWrap = el("div", "sw-rows");
|
||||
content.append(recentWrap);
|
||||
try {
|
||||
const list = await S.listProjects();
|
||||
clear(recentWrap);
|
||||
if (!list.length) {
|
||||
recentWrap.append(el("div", "sw-hint", "No projects yet."));
|
||||
}
|
||||
list.slice(0, 12).forEach((p) => {
|
||||
const row = el("div", "sw-row");
|
||||
const main = el("div", "sw-row-main");
|
||||
main.append(el("div", "sw-row-title", p.name));
|
||||
main.append(el("div", "sw-row-hint", `${p.scene_count ?? 1} scene(s)`));
|
||||
row.append(main);
|
||||
const openBtn = el("button", "btn ghost tiny", "Open");
|
||||
openBtn.type = "button";
|
||||
openBtn.onclick = async () => { await S.loadProject(p.id); close(); };
|
||||
row.append(openBtn);
|
||||
recentWrap.append(row);
|
||||
});
|
||||
} catch (e) {
|
||||
clear(recentWrap);
|
||||
recentWrap.append(el("div", "sw-hint", "Could not load project list: " + (e.message || e)));
|
||||
}
|
||||
|
||||
overlay.append(box);
|
||||
if (dismissable) overlay.addEventListener("click", (e) => { if (e.target === overlay) close(); });
|
||||
document.body.append(overlay);
|
||||
}
|
||||
|
||||
window.ProjectMenu = { open, close };
|
||||
})();
|
||||
@@ -0,0 +1,74 @@
|
||||
"""FunPack Easy Gen — simplified single-scene generation UI.
|
||||
|
||||
UI: GET /funpack/easy/ (static assets under /funpack/easy/<file>)
|
||||
|
||||
No new backend API on purpose: this frontend talks to the exact same
|
||||
/funpack/movie/api/* endpoints the Movie Editor uses (same Project files,
|
||||
same on-disk storage), so a project created here opens unchanged in the
|
||||
full Editor and vice versa. The only job of this module is serving Easy
|
||||
Gen's own frontend files, falling back to the Editor's frontend directory
|
||||
for shared scripts/styles (api.js excluded — see easy_gen/frontend/api.js)
|
||||
so those stay a single source of truth instead of being copy-pasted.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import mimetypes
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
try:
|
||||
from aiohttp import web
|
||||
from server import PromptServer
|
||||
except Exception: # pragma: no cover - only available inside ComfyUI
|
||||
web = None
|
||||
PromptServer = None
|
||||
|
||||
from ..movie_editor.backend import config as movie_config
|
||||
|
||||
UI_PREFIX = "/funpack/easy"
|
||||
FRONTEND_DIR = Path(__file__).resolve().parent / "frontend"
|
||||
|
||||
|
||||
def _resolve(tail: str) -> Optional[Path]:
|
||||
rel = (tail or "index.html").lstrip("/")
|
||||
for root in (FRONTEND_DIR, movie_config.FRONTEND_DIR):
|
||||
root = root.resolve()
|
||||
target = (root / rel).resolve()
|
||||
if root not in target.parents and target != root:
|
||||
continue # path traversal guard
|
||||
if target.is_dir():
|
||||
target = target / "index.html"
|
||||
if target.is_file():
|
||||
return target
|
||||
return None
|
||||
|
||||
|
||||
def _serve_static(tail: str) -> "web.Response":
|
||||
target = _resolve(tail)
|
||||
if target is None:
|
||||
raise web.HTTPNotFound()
|
||||
ctype = mimetypes.guess_type(str(target))[0] or "application/octet-stream"
|
||||
if target.suffix == ".js":
|
||||
ctype = "text/javascript" # be explicit; some platforms guess text/plain
|
||||
return web.Response(
|
||||
body=target.read_bytes(), content_type=ctype,
|
||||
headers={"Cache-Control": "no-store, max-age=0"},
|
||||
)
|
||||
|
||||
|
||||
if web is not None and PromptServer is not None:
|
||||
routes = PromptServer.instance.routes
|
||||
|
||||
@routes.get(UI_PREFIX)
|
||||
async def _root_redirect(_req):
|
||||
raise web.HTTPFound(UI_PREFIX + "/")
|
||||
|
||||
@routes.get(UI_PREFIX + "/")
|
||||
async def _index(_req):
|
||||
return _serve_static("index.html")
|
||||
|
||||
@routes.get(UI_PREFIX + "/{tail:.*}")
|
||||
async def _static(req):
|
||||
return _serve_static(req.match_info["tail"])
|
||||
|
||||
print(f"[FunPack] Easy Gen available at {movie_config.comfy_display_url()}{UI_PREFIX}/")
|
||||
@@ -0,0 +1,253 @@
|
||||
"""FunPack Hub — a small landing page at /funpack/ linking to the two FunPack
|
||||
UIs (Movie Editor "Cutting Room" and "Easy Gen"), plus an About block (version/
|
||||
branch/commit) and an Update/Restart action. Talks to the Movie Editor's
|
||||
existing /funpack/movie/api/{health,git/status,git/update,restart} endpoints —
|
||||
no new backend API, same git_update.py logic the Editor's own "Updates &
|
||||
ComfyUI" settings section uses. Kept as a single self-contained inline page
|
||||
(no shared JS files) since its surface is a handful of fetch calls, much
|
||||
smaller than pulling in api.js/dom.js/git_actions.js/restart_ui.js would need.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
try:
|
||||
from aiohttp import web
|
||||
from server import PromptServer
|
||||
except Exception: # pragma: no cover - only available inside ComfyUI
|
||||
web = None
|
||||
PromptServer = None
|
||||
|
||||
from .movie_editor.backend import config as movie_config
|
||||
|
||||
HUB_PREFIX = "/funpack"
|
||||
|
||||
_PAGE = """<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>FunPack</title>
|
||||
<style>
|
||||
:root {
|
||||
--ink-0: #0c0b09; --ink-1: #131210; --ink-2: #1a1814; --line: #2a261e;
|
||||
--text: #ece7db; --muted: #918a7a; --faint: #645e51;
|
||||
--amber: #f3a93c; --amber-2: #ffc36b; --teal: #57d0c6; --danger: #e8607a; --good: #79c479;
|
||||
--radius: 12px; --shadow: 0 10px 30px -12px rgba(0,0,0,.7);
|
||||
}
|
||||
* { box-sizing: border-box; }
|
||||
html, body {
|
||||
margin: 0; min-height: 100%; background: var(--ink-0); color: var(--text);
|
||||
font-family: "IBM Plex Sans", system-ui, sans-serif;
|
||||
display: flex; align-items: center; justify-content: center; padding: 32px 16px;
|
||||
}
|
||||
.hub { text-align: center; max-width: 640px; width: 100%; }
|
||||
.brand { font-weight: 800; font-size: 28px; letter-spacing: .2px; margin-bottom: 6px; }
|
||||
.brand-mark { color: var(--amber); margin-right: 8px; }
|
||||
.lead { color: var(--muted); margin: 0 0 20px; font-size: 14px; }
|
||||
|
||||
.about {
|
||||
display: inline-flex; align-items: center; gap: 14px; flex-wrap: wrap; justify-content: center;
|
||||
background: var(--ink-1); border: 1px solid var(--line); border-radius: var(--radius);
|
||||
padding: 10px 16px; margin-bottom: 28px; font-size: 12.5px; color: var(--muted);
|
||||
font-family: "IBM Plex Mono", ui-monospace, monospace;
|
||||
}
|
||||
.about .led { width: 7px; height: 7px; border-radius: 50%; background: var(--faint); display: inline-block; margin-right: 5px; }
|
||||
.about .led.ok { background: var(--good); box-shadow: 0 0 8px var(--good); }
|
||||
.about .led.bad { background: var(--danger); box-shadow: 0 0 8px var(--danger); }
|
||||
.about b { color: var(--text); font-weight: 600; }
|
||||
.about .sep { color: var(--line); }
|
||||
|
||||
.update-row { margin-bottom: 28px; display: flex; align-items: center; justify-content: center; gap: 10px; flex-wrap: wrap; }
|
||||
.btn {
|
||||
background: var(--ink-2); border: 1px solid var(--line); border-radius: 7px; padding: 7px 14px;
|
||||
cursor: pointer; font-size: 13px; color: var(--text); transition: border-color .12s, background .12s;
|
||||
}
|
||||
.btn:hover:not(:disabled) { border-color: var(--amber); }
|
||||
.btn:disabled { opacity: .45; cursor: default; }
|
||||
.btn.primary { background: linear-gradient(#f3a93c, #e0972a); color: #1a1206; border-color: #f3a93c; font-weight: 700; }
|
||||
.btn.primary:hover:not(:disabled) { filter: brightness(1.08); }
|
||||
.btn.ghost { background: transparent; color: var(--muted); }
|
||||
.btn.ghost:hover:not(:disabled) { color: var(--amber-2); background: var(--ink-3); }
|
||||
.update-hint { color: var(--faint); font-size: 12px; }
|
||||
|
||||
.cards { display: flex; gap: 18px; flex-wrap: wrap; justify-content: center; }
|
||||
a.card {
|
||||
display: block; width: 260px; padding: 22px 20px; text-align: left;
|
||||
background: var(--ink-1); border: 1px solid var(--line); border-radius: var(--radius);
|
||||
text-decoration: none; color: inherit; box-shadow: var(--shadow);
|
||||
transition: border-color .12s, transform .12s;
|
||||
}
|
||||
a.card:hover { border-color: var(--amber); transform: translateY(-2px); }
|
||||
.card-title { font-weight: 700; font-size: 17px; margin-bottom: 6px; }
|
||||
.card-title.easy { color: var(--amber-2); }
|
||||
.card-title.editor { color: var(--teal); }
|
||||
.card-sub { color: var(--faint); font-size: 12.5px; line-height: 1.5; }
|
||||
|
||||
.restart-overlay {
|
||||
position: fixed; inset: 0; z-index: 500; background: rgba(6,5,4,.75); backdrop-filter: blur(3px);
|
||||
display: flex; align-items: center; justify-content: center;
|
||||
}
|
||||
.restart-card {
|
||||
background: #16140f; border: 1px solid var(--line); border-radius: 14px; padding: 26px 30px;
|
||||
box-shadow: var(--shadow); text-align: center; max-width: 360px;
|
||||
}
|
||||
.restart-spin {
|
||||
width: 26px; height: 26px; margin: 0 auto 14px; border-radius: 50%;
|
||||
border: 3px solid var(--line); border-top-color: var(--amber); animation: spin .8s linear infinite;
|
||||
}
|
||||
@keyframes spin { to { transform: rotate(360deg); } }
|
||||
.restart-msg { color: var(--muted); font-size: 13px; white-space: pre-wrap; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="hub">
|
||||
<div class="brand"><span class="brand-mark">◉</span>FunPack</div>
|
||||
<p class="lead">Pick a UI.</p>
|
||||
|
||||
<div class="about" id="about">Loading…</div>
|
||||
<div class="update-row" id="update-row"></div>
|
||||
|
||||
<div class="cards">
|
||||
<a class="card" href="/funpack/easy/">
|
||||
<div class="card-title easy">Easy Gen</div>
|
||||
<div class="card-sub">One prompt, one Generate button, one big preview. Advanced settings tucked away.</div>
|
||||
</a>
|
||||
<a class="card" href="/funpack/movie/">
|
||||
<div class="card-title editor">Cutting Room</div>
|
||||
<div class="card-sub">The full multi-scene NLE — timeline, composer, overlays, audio.</div>
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
(function () {
|
||||
const API = (p) => "/funpack/movie/api" + p;
|
||||
const aboutEl = document.getElementById("about");
|
||||
const updateEl = document.getElementById("update-row");
|
||||
|
||||
async function j(method, url, body) {
|
||||
const opts = { method, headers: {} };
|
||||
if (body !== undefined) { opts.headers["Content-Type"] = "application/json"; opts.body = JSON.stringify(body); }
|
||||
const res = await fetch(url, opts);
|
||||
if (!res.ok) {
|
||||
let payload = null; try { payload = await res.json(); } catch (_) {}
|
||||
throw new Error((payload && payload.detail) || res.statusText || "Request failed");
|
||||
}
|
||||
return res.status === 204 ? null : res.json();
|
||||
}
|
||||
|
||||
function led(ok) { return `<span class="led ${ok ? "ok" : "bad"}"></span>`; }
|
||||
|
||||
function renderAbout(health, git) {
|
||||
const parts = [];
|
||||
parts.push(`${led(!!(health && health.ok))}${health && health.ok ? "online" : "offline"}`);
|
||||
if (git && git.ok) {
|
||||
parts.push(`<b>v${git.version || "?"}</b>`);
|
||||
parts.push(`${git.branch}@${git.commit}${git.dirty ? " (local changes)" : ""}`);
|
||||
} else {
|
||||
parts.push(git && git.version ? `<b>v${git.version}</b>` : "");
|
||||
parts.push("git unavailable");
|
||||
}
|
||||
aboutEl.innerHTML = parts.filter(Boolean).join(' <span class="sep">·</span> ');
|
||||
}
|
||||
|
||||
function renderUpdateRow(git) {
|
||||
updateEl.innerHTML = "";
|
||||
if (!git || !git.ok) return;
|
||||
const updateBtn = document.createElement("button");
|
||||
updateBtn.className = "btn primary";
|
||||
updateBtn.disabled = !!git.dirty;
|
||||
updateBtn.textContent = git.behind > 0 ? `${git.behind}↓ Update` : "⬇⟳ Update";
|
||||
updateBtn.onclick = doUpdate;
|
||||
updateEl.appendChild(updateBtn);
|
||||
|
||||
const restartBtn = document.createElement("button");
|
||||
restartBtn.className = "btn ghost";
|
||||
restartBtn.textContent = "⟳ Restart ComfyUI";
|
||||
restartBtn.onclick = doRestart;
|
||||
updateEl.appendChild(restartBtn);
|
||||
|
||||
const hint = document.createElement("span");
|
||||
hint.className = "update-hint";
|
||||
hint.textContent = git.dirty
|
||||
? "Local changes — commit or stash first"
|
||||
: (git.behind > 0 ? `${git.behind} commit(s) behind origin/${git.branch}` : `origin/${git.branch} up to date`);
|
||||
updateEl.appendChild(hint);
|
||||
}
|
||||
|
||||
function showOverlay(message) {
|
||||
document.querySelector(".restart-overlay")?.remove();
|
||||
const ov = document.createElement("div"); ov.className = "restart-overlay";
|
||||
const card = document.createElement("div"); card.className = "restart-card";
|
||||
const spin = document.createElement("div"); spin.className = "restart-spin";
|
||||
const msg = document.createElement("div"); msg.className = "restart-msg"; msg.textContent = message;
|
||||
card.append(spin, msg); ov.append(card); document.body.append(ov);
|
||||
return msg;
|
||||
}
|
||||
|
||||
function waitForReload(msgEl, startMs) {
|
||||
const start = startMs || Date.now();
|
||||
const tick = async () => {
|
||||
try {
|
||||
const h = await j("GET", API("/health"));
|
||||
if (h && h.ok) { location.reload(); return; }
|
||||
} catch (_) { /* still down */ }
|
||||
if (Date.now() - start > 90000) {
|
||||
msgEl.textContent = "Still waiting on ComfyUI…\\nIt may need a manual restart - check the console.";
|
||||
}
|
||||
setTimeout(tick, 2000);
|
||||
};
|
||||
setTimeout(tick, 3500);
|
||||
}
|
||||
|
||||
async function doUpdate() {
|
||||
let git;
|
||||
try { git = await j("GET", API("/git/status")); } catch (e) { alert("Could not read git status: " + e.message); return; }
|
||||
if (!git.ok) { alert(git.detail || "Git unavailable for this install."); return; }
|
||||
if (git.dirty) { alert("Local changes detected in the FunPack folder.\\nCommit or stash them before updating."); return; }
|
||||
const behind = git.behind > 0 ? `\\n\\n${git.behind} commit(s) available on origin/${git.branch}.` : "";
|
||||
if (!confirm(`Pull latest "${git.branch}" from origin and restart ComfyUI?\\n\\nAny running generation will be lost.${behind}`)) return;
|
||||
const msg = showOverlay(`Pulling origin/${git.branch}…\\nComfyUI will restart when the pull finishes.`);
|
||||
try {
|
||||
const res = await j("POST", API("/git/update"), {});
|
||||
msg.textContent = res.updated ? `Updated ${res.before} → ${res.after}.\\nRestarting ComfyUI…` : "Already up to date.\\nRestarting ComfyUI…";
|
||||
} catch (e) {
|
||||
document.querySelector(".restart-overlay")?.remove();
|
||||
alert("Update failed: " + e.message);
|
||||
return;
|
||||
}
|
||||
waitForReload(msg, Date.now());
|
||||
}
|
||||
|
||||
async function doRestart() {
|
||||
if (!confirm("Restart ComfyUI now?\\n\\nThe server will be down for ~10-40s and any running generation will be lost. This page reloads automatically when it's back.")) return;
|
||||
const msg = showOverlay("Restarting ComfyUI…\\nThis page will reload when it's back.");
|
||||
try { await j("POST", API("/restart"), {}); } catch (_) { /* connection drops as it execv's - expected */ }
|
||||
waitForReload(msg, Date.now());
|
||||
}
|
||||
|
||||
async function boot() {
|
||||
let health = null, git = null;
|
||||
try { health = await j("GET", API("/health")); } catch (_) {}
|
||||
try { git = await j("GET", API("/git/status")); } catch (_) {}
|
||||
renderAbout(health, git);
|
||||
renderUpdateRow(git);
|
||||
}
|
||||
boot();
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
if web is not None and PromptServer is not None:
|
||||
routes = PromptServer.instance.routes
|
||||
|
||||
@routes.get(HUB_PREFIX)
|
||||
async def _hub_redirect(_req):
|
||||
raise web.HTTPFound(HUB_PREFIX + "/")
|
||||
|
||||
@routes.get(HUB_PREFIX + "/")
|
||||
async def _hub_index(_req):
|
||||
return web.Response(body=_PAGE, content_type="text/html", headers={"Cache-Control": "no-store, max-age=0"})
|
||||
|
||||
print(f"[FunPack] Hub available at {movie_config.comfy_display_url()}{HUB_PREFIX}/")
|
||||
+167
-40
@@ -6,7 +6,8 @@ Technique 2: Per-block attention temperature
|
||||
|
||||
Technique 3: Temporal RoPE style
|
||||
Manipulate frame_rate in the model's positional embedding to change motion character.
|
||||
Styles: natural / accelerate / decelerate / loop / freeze / pulse
|
||||
Styles: natural / accelerate / decelerate / loop / freeze / pulse /
|
||||
rapid_start / rapid_end / rapid_start_end
|
||||
|
||||
Technique 4: Denoise creativity mask
|
||||
Spatial noise weighting derived from latent variance: high-variance regions get more freedom.
|
||||
@@ -25,7 +26,10 @@ from hashlib import md5
|
||||
|
||||
import torch
|
||||
|
||||
TEMPORAL_STYLES = ["natural", "auto", "accelerate", "decelerate", "loop", "freeze", "pulse"]
|
||||
TEMPORAL_STYLES = [
|
||||
"natural", "auto", "accelerate", "decelerate", "loop", "freeze", "pulse",
|
||||
"rapid_start", "rapid_end", "rapid_start_end",
|
||||
]
|
||||
|
||||
# frame_rate multiplier fed to the LTX RoPE per style. >1 => model assumes more
|
||||
# frames cover the same wall-clock time => smaller inter-frame deltas => smoother /
|
||||
@@ -45,6 +49,12 @@ PULSE_SEGMENT_COUNT = 3
|
||||
PULSE_PEAK_MULT = 0.88
|
||||
PULSE_FLOOR_MULT = 1.65
|
||||
|
||||
# Rapid start/end: punchy (low) frame_rate mult held over the first/last slice of the
|
||||
# scene's denoise progress, easing to natural (1.0) elsewhere. Same progress-from-sigma
|
||||
# mechanism as pulse, just shaped as a single ramp instead of repeated segments.
|
||||
RAPID_MULT = 0.65
|
||||
RAPID_FRACTION = 0.35 # portion of denoise progress the rapid ramp covers at each edge
|
||||
|
||||
|
||||
# Keyword -> motion-energy intent for the "auto" director. The classifier scans a
|
||||
# scene's prompt and emits a continuous frame_rate multiplier (plus a loop flag).
|
||||
@@ -118,6 +128,22 @@ def pulse_mult_for_progress(progress, segment_count=None, peak_mult=None, floor_
|
||||
return peak + (floor - peak) * _ease_down(local_t)
|
||||
|
||||
|
||||
def rapid_mult_for_progress(progress, mode, fraction=None, mult=None):
|
||||
"""Map denoise progress [0,1] to a frame_rate multiplier for the rapid_start /
|
||||
rapid_end / rapid_start_end styles: `mult` (punchy) held over the first and/or
|
||||
last `fraction` of progress, easing to natural (1.0) toward the middle."""
|
||||
frac = RAPID_FRACTION if fraction is None else max(1e-3, min(0.5, float(fraction)))
|
||||
rapid = RAPID_MULT if mult is None else float(mult)
|
||||
progress = max(0.0, min(1.0, float(progress)))
|
||||
start = mode in ("rapid_start", "rapid_start_end")
|
||||
end = mode in ("rapid_end", "rapid_start_end")
|
||||
if start and progress < frac:
|
||||
return rapid + (1.0 - rapid) * _ease_down(progress / frac)
|
||||
if end and progress > 1.0 - frac:
|
||||
return rapid + (1.0 - rapid) * _ease_down((1.0 - progress) / frac)
|
||||
return 1.0
|
||||
|
||||
|
||||
def _scale_frame_rate_in_args(args, mult):
|
||||
"""Return a shallow-copied args dict with frame_rate scaled by mult (no-op on failure)."""
|
||||
c = args.get("c")
|
||||
@@ -188,6 +214,33 @@ def make_pulse_temporal_wrapper(old_wrapper, segment_count=None, peak_mult=None,
|
||||
|
||||
return _pulse_wrapper
|
||||
|
||||
|
||||
def make_rapid_temporal_wrapper(old_wrapper, mode, fraction=None, mult=None):
|
||||
"""Build a model_function_wrapper for rapid_start / rapid_end / rapid_start_end:
|
||||
punchy frame_rate mult over the first/last slice of denoise progress (derived from
|
||||
sigma, same as pulse), natural (1.0, i.e. no-op) elsewhere. A fresh wrapper should
|
||||
be installed per scene so sigma bounds reset between scenes."""
|
||||
frac = RAPID_FRACTION if fraction is None else fraction
|
||||
rapid = RAPID_MULT if mult is None else mult
|
||||
sigma_start = [None]
|
||||
|
||||
def _rapid_wrapper(apply_fn, args, _old=old_wrapper, _mode=mode, _frac=frac, _rapid=rapid):
|
||||
ts = args.get("timestep")
|
||||
try:
|
||||
sigma = float(ts.max().item()) if ts is not None else 1.0
|
||||
except Exception:
|
||||
sigma = 1.0
|
||||
if sigma_start[0] is None:
|
||||
sigma_start[0] = max(sigma, 1e-6)
|
||||
progress = max(0.0, min(1.0, 1.0 - sigma / sigma_start[0]))
|
||||
mult_here = rapid_mult_for_progress(progress, _mode, _frac, _rapid)
|
||||
args = _scale_frame_rate_in_args(args, mult_here)
|
||||
if _old is not None:
|
||||
return _old(apply_fn, args)
|
||||
return apply_fn(args["input"], args["timestep"], **args.get("c", {}))
|
||||
|
||||
return _rapid_wrapper
|
||||
|
||||
# --- Loop temporal style: Mobius-style latent roll (arXiv:2502.20307) ---------------
|
||||
# Seamless looping without training or extra forward passes: on eligible denoise steps
|
||||
# the latent is cyclically rolled along its temporal dim before the forward and the
|
||||
@@ -198,11 +251,15 @@ def make_pulse_temporal_wrapper(old_wrapper, segment_count=None, peak_mult=None,
|
||||
#
|
||||
# Roll starts below the near-noise plateau: content is ~pure noise up there (nothing to
|
||||
# smooth) and the plateau step-cache reuses base forwards, which a per-step roll would
|
||||
# invalidate. Guides pin absolute frame positions (keyframe_idxs / attention entries
|
||||
# reference appended frames inside the same T range), so calls carrying them are left
|
||||
# canonical rather than rolled.
|
||||
# invalidate. Appended guide frames (carry_i2v_guides / mid-scene guide / JoyAI memory /
|
||||
# custom stacks) live at the TAIL of the same T range, pinned to absolute positions by
|
||||
# keyframe_idxs — so only the content region [0, T - tail) rolls and the tail (plus
|
||||
# keyframe_idxs and attention entries, which reference tail tokens) stays canonical.
|
||||
# Guides keep informing every rolled position (their influence smears across the cycle),
|
||||
# which is exactly what identity/style references should do in a loop; they are never
|
||||
# rendered as frames themselves.
|
||||
LOOP_ROLL_MAX_SIGMA = 0.95
|
||||
LOOP_ROLL_MIN_FRAMES = 4 # below this many latent frames there is nothing to loop
|
||||
LOOP_ROLL_MIN_FRAMES = 4 # below this many CONTENT latent frames there is nothing to loop
|
||||
|
||||
|
||||
def _van_der_corput(n):
|
||||
@@ -242,37 +299,94 @@ def _loop_stream_shapes(args):
|
||||
return shapes
|
||||
|
||||
|
||||
def _loop_roll_packed(x, shapes, frac, direction):
|
||||
"""Cyclically roll every stream's temporal dim (dim 2) of a packed [B,1,N] tensor by
|
||||
round(frac * T_stream) frames (video and audio each by their own frame count, so the
|
||||
two stay time-aligned). direction=+1 rolls, -1 unrolls. Returns a new tensor."""
|
||||
def _loop_content_shift(frac, t, tail):
|
||||
"""Frame shift for a stream with `t` total frames of which the last `tail` are pinned
|
||||
guide frames. 0 when the content region is too small to roll."""
|
||||
tc = int(t) - max(0, int(tail))
|
||||
if tc < 2:
|
||||
return 0, tc
|
||||
return int(round(frac * tc)) % tc, tc
|
||||
|
||||
|
||||
def _loop_roll_stream(stream, frac, direction, tail):
|
||||
"""Roll only the content region [0, T-tail) of one stream's temporal dim (dim 2);
|
||||
the pinned guide tail stays canonical. Returns the input unchanged when there is
|
||||
nothing to roll."""
|
||||
shift, tc = _loop_content_shift(frac, stream.shape[2], tail)
|
||||
if not shift:
|
||||
return stream
|
||||
out = stream.clone()
|
||||
out[:, :, :tc] = torch.roll(stream[:, :, :tc], shifts=direction * shift, dims=2)
|
||||
return out
|
||||
|
||||
|
||||
def _loop_roll_packed(x, shapes, frac, direction, tails=None):
|
||||
"""Cyclically roll every stream's content region of a packed [B,1,N] tensor by
|
||||
round(frac * T_content) frames (video and audio each by their own content length, so
|
||||
the two stay time-aligned). `tails` gives per-stream pinned tail-frame counts aligned
|
||||
with `shapes` (appended guides / audio memory — left canonical). direction=+1 rolls,
|
||||
-1 unrolls. Returns a new tensor."""
|
||||
import math as _math
|
||||
if x.ndim == 5: # unpacked single stream
|
||||
t = x.shape[2]
|
||||
shift = int(round(frac * t)) % t
|
||||
return torch.roll(x, shifts=direction * shift, dims=2) if shift else x
|
||||
return _loop_roll_stream(x, frac, direction, (tails or [0])[0])
|
||||
out = x.clone()
|
||||
off = 0
|
||||
for dims in shapes:
|
||||
for i, dims in enumerate(shapes):
|
||||
sz = _math.prod(dims[1:])
|
||||
if len(dims) >= 3 and dims[2] >= 2:
|
||||
t = dims[2]
|
||||
shift = int(round(frac * t)) % t
|
||||
if shift:
|
||||
stream = x[..., off:off + sz].reshape([x.shape[0]] + list(dims[1:]))
|
||||
out[..., off:off + sz] = torch.roll(
|
||||
stream, shifts=direction * shift, dims=2).reshape(x.shape[0], 1, sz)
|
||||
stream = x[..., off:off + sz].reshape([x.shape[0]] + list(dims[1:]))
|
||||
rolled = _loop_roll_stream(stream, frac, direction, (tails or [])[i] if tails else 0)
|
||||
if rolled is not stream:
|
||||
out[..., off:off + sz] = rolled.reshape(x.shape[0], 1, sz)
|
||||
off += sz
|
||||
return out
|
||||
|
||||
|
||||
def _loop_roll_mask(mask, frac, direction):
|
||||
"""Roll a denoise mask ([B,1,T,H,W] video / [B,C,T,F] audio) in step with its stream."""
|
||||
def _loop_roll_mask(mask, frac, direction, tail=0):
|
||||
"""Roll a denoise mask ([B,1,T,H,W] video / [B,C,T,F] audio) in step with its stream,
|
||||
leaving the pinned tail region canonical like the stream itself."""
|
||||
if not isinstance(mask, torch.Tensor) or mask.ndim < 3 or mask.shape[2] < 2:
|
||||
return mask
|
||||
t = mask.shape[2]
|
||||
shift = int(round(frac * t)) % t
|
||||
return torch.roll(mask, shifts=direction * shift, dims=2) if shift else mask
|
||||
return _loop_roll_stream(mask, frac, direction, tail)
|
||||
|
||||
|
||||
def _loop_video_tail_frames(c, video_shape):
|
||||
"""Number of appended guide latent frames pinned at the video stream's tail. 0 when
|
||||
the call carries no guides; None when a guide context exists but can't be parsed
|
||||
(caller then leaves the call canonical rather than risking a corrupting roll).
|
||||
keyframe_idxs is per-TOKEN — frames = tokens / (H*W), same math as comfy's
|
||||
get_keyframe_idxs."""
|
||||
kf = c.get("keyframe_idxs")
|
||||
if kf is None:
|
||||
# Attention entries without keyframe_idxs shouldn't happen; treat as unparseable.
|
||||
return None if c.get("guide_attention_entries") else 0
|
||||
try:
|
||||
if len(video_shape) != 5 or not hasattr(kf, "shape") or len(kf.shape) < 3:
|
||||
return None
|
||||
tokens_per_frame = int(video_shape[-2]) * int(video_shape[-1])
|
||||
n = int(kf.shape[2]) // max(1, tokens_per_frame)
|
||||
return n if 0 < n < int(video_shape[2]) else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _loop_audio_tail_frames(mask):
|
||||
"""Appended audio-memory frames at the tail of the audio stream = the trailing run of
|
||||
all-zero frames in its denoise mask (JoyAI paired audio memory rides mask=0; genuine
|
||||
audio content is never fully masked). 0 when there is no mask or no zero tail."""
|
||||
if not isinstance(mask, torch.Tensor) or mask.ndim < 3 or mask.shape[2] < 2:
|
||||
return 0
|
||||
try:
|
||||
other_dims = [d for d in range(mask.ndim) if d != 2]
|
||||
per_frame = mask.float().abs().amax(dim=other_dims) # [T]
|
||||
tail = 0
|
||||
for v in reversed(per_frame.tolist()):
|
||||
if v > 1e-4:
|
||||
break
|
||||
tail += 1
|
||||
return tail if tail < mask.shape[2] else 0
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
|
||||
def make_loop_temporal_wrapper(old_wrapper):
|
||||
@@ -299,36 +413,46 @@ def make_loop_temporal_wrapper(old_wrapper):
|
||||
state["last_sigma"] = sigma
|
||||
|
||||
c = args.get("c") or {}
|
||||
if (
|
||||
sigma > LOOP_ROLL_MAX_SIGMA
|
||||
or c.get("keyframe_idxs") is not None
|
||||
or c.get("guide_attention_entries") is not None
|
||||
):
|
||||
if sigma > LOOP_ROLL_MAX_SIGMA:
|
||||
return _call(args)
|
||||
shapes = _loop_stream_shapes(args)
|
||||
if not shapes:
|
||||
return _call(args)
|
||||
# Gate on the VIDEO stream's frame count (the 5-dim shape); audio rides along.
|
||||
t_video = max((s[2] for s in shapes if len(s) == 5),
|
||||
default=max((s[2] for s in shapes if len(s) >= 3), default=0))
|
||||
if t_video < LOOP_ROLL_MIN_FRAMES:
|
||||
# Per-stream pinned tails: appended guide frames (video, counted from the
|
||||
# per-token keyframe_idxs) and appended audio memory (trailing mask=0 run).
|
||||
# Only the content region in front of each tail rolls; the tail — and the
|
||||
# keyframe_idxs / attention entries that reference it — stays canonical.
|
||||
video_shape = next((s for s in shapes if len(s) == 5), shapes[0])
|
||||
v_tail = _loop_video_tail_frames(c, video_shape)
|
||||
if v_tail is None:
|
||||
return _call(args) # guide context we can't parse: never risk a bad roll
|
||||
tails = [
|
||||
(v_tail if len(s) == 5 else _loop_audio_tail_frames(c.get("audio_denoise_mask")))
|
||||
for s in shapes
|
||||
]
|
||||
t_content = int(video_shape[2]) - v_tail if len(video_shape) >= 3 else 0
|
||||
if t_content < LOOP_ROLL_MIN_FRAMES:
|
||||
return _call(args)
|
||||
|
||||
state["count"] += 1
|
||||
frac = _van_der_corput(state["count"])
|
||||
if not state["logged"]:
|
||||
state["logged"] = True
|
||||
print(f"[FunPack] loop temporal style: Mobius latent roll active (T={t_video})")
|
||||
tail_note = f", {v_tail} guide frame(s) pinned" if v_tail else ""
|
||||
print(f"[FunPack] loop temporal style: Mobius latent roll active (T={t_content}{tail_note})")
|
||||
try:
|
||||
new_c = dict(c)
|
||||
for key in ("denoise_mask", "audio_denoise_mask"):
|
||||
if isinstance(new_c.get(key), torch.Tensor):
|
||||
new_c[key] = _loop_roll_mask(new_c[key], frac, 1)
|
||||
if isinstance(new_c.get("denoise_mask"), torch.Tensor):
|
||||
new_c["denoise_mask"] = _loop_roll_mask(new_c["denoise_mask"], frac, 1, tail=v_tail)
|
||||
if isinstance(new_c.get("audio_denoise_mask"), torch.Tensor):
|
||||
new_c["audio_denoise_mask"] = _loop_roll_mask(
|
||||
new_c["audio_denoise_mask"], frac, 1,
|
||||
tail=_loop_audio_tail_frames(new_c["audio_denoise_mask"]))
|
||||
rolled = dict(args)
|
||||
rolled["input"] = _loop_roll_packed(args["input"], shapes, frac, 1)
|
||||
rolled["input"] = _loop_roll_packed(args["input"], shapes, frac, 1, tails=tails)
|
||||
rolled["c"] = new_c
|
||||
out = _call(rolled)
|
||||
return _loop_roll_packed(out, shapes, frac, -1)
|
||||
return _loop_roll_packed(out, shapes, frac, -1, tails=tails)
|
||||
except Exception as e:
|
||||
if state.get("roll_error") is None:
|
||||
state["roll_error"] = True
|
||||
@@ -1366,6 +1490,9 @@ def build_enhancements(model, rating_profile, temporal_style, refinement_key, re
|
||||
elif temporal_style == "loop":
|
||||
model.model_options["model_function_wrapper"] = make_loop_temporal_wrapper(
|
||||
model.model_options.get("model_function_wrapper"))
|
||||
elif temporal_style in ("rapid_start", "rapid_end", "rapid_start_end"):
|
||||
model.model_options["model_function_wrapper"] = make_rapid_temporal_wrapper(
|
||||
model.model_options.get("model_function_wrapper"), temporal_style)
|
||||
elif temporal_style not in ("natural", "auto"):
|
||||
mult = TEMPORAL_STYLE_MULT.get(temporal_style, 1.0)
|
||||
wrapper = make_temporal_wrapper(model.model_options.get("model_function_wrapper"), mult)
|
||||
|
||||
@@ -866,8 +866,10 @@ def _autowire(graph, slots, slot_node_id, slot_def, object_info, producers, repo
|
||||
def _apply_bypass(graph, slots, slot_node_id, slot_def, report):
|
||||
"""Drop each bypassed slot's node, rewiring its consumers to whatever already feeds its
|
||||
matching-type input (same idea as ComfyUI's node bypass) — only when that mapping is
|
||||
unambiguous (exactly one connection_input per output type). Otherwise leaves the node
|
||||
wired normally and reports why, rather than guessing. Runs after auto-wire so the
|
||||
unambiguous (exactly one connection_input per output type) AND that input actually has
|
||||
something wired to pass through. Otherwise blocks generation with a clear reason rather
|
||||
than silently leaving the node active or dropping a consumer's input — a bypass a user
|
||||
explicitly asked for must never be silently ignored. Runs after auto-wire so the
|
||||
passthrough source is already resolved to a concrete value/link.
|
||||
"""
|
||||
for s in slots:
|
||||
@@ -890,9 +892,16 @@ def _apply_bypass(graph, slots, slot_node_id, slot_def, report):
|
||||
break
|
||||
passthrough[i] = graph[sid]["inputs"].get(names[0])
|
||||
if not ok:
|
||||
report["unsatisfied"].append(
|
||||
# Silently leaving the node active would mean a user who explicitly bypassed it
|
||||
# (e.g. to skip an i2v preprocessing node) gets generation output as if they
|
||||
# hadn't — with no visible sign why. Block instead: bypass either does what was
|
||||
# asked or the run stops with a clear reason, never a silent no-op.
|
||||
msg = (
|
||||
f"{s.get('node_class')}: bypass needs exactly one input matching each output's "
|
||||
f"type to pass through — this node doesn't have one, so it stays active.")
|
||||
f"type to pass through — this node doesn't have one, so it can't be safely "
|
||||
f"bypassed. Remove it or rewire it with a single matching input per output type.")
|
||||
report["unsatisfied"].append(msg)
|
||||
report["blocking"].append(msg)
|
||||
continue
|
||||
for nid, ndata in graph.items():
|
||||
if nid == sid:
|
||||
@@ -903,6 +912,14 @@ def _apply_bypass(graph, slots, slot_node_id, slot_def, report):
|
||||
if replacement is not None:
|
||||
ndata["inputs"][inp_name] = replacement
|
||||
else:
|
||||
# The bypassed node's own matching input was never wired — passing
|
||||
# nothing through would silently drop {nid}.{inp_name} instead of
|
||||
# restoring the original source. Block rather than guess.
|
||||
msg = (
|
||||
f"{s.get('node_class')}: bypass can't pass a value through to "
|
||||
f"{nid}.{inp_name} — its own matching input isn't wired to anything.")
|
||||
report["unsatisfied"].append(msg)
|
||||
report["blocking"].append(msg)
|
||||
del ndata["inputs"][inp_name]
|
||||
del graph[sid]
|
||||
report["wired"].append(f"{s.get('node_class')} bypassed (pass-through)")
|
||||
|
||||
@@ -53,6 +53,30 @@ def comfy_base_url() -> str:
|
||||
return f"http://127.0.0.1:{port}" # always loopback for self-calls
|
||||
|
||||
|
||||
def comfy_display_url() -> str:
|
||||
"""Best-effort http://host:port for STARTUP LOG PRINTS (full clickable UI
|
||||
URLs) — not for internal calls, see comfy_base_url() above. Routes are
|
||||
registered while custom nodes load, before PromptServer binds a socket, so
|
||||
the port/host here come from comfy.cli_args (already parsed at process
|
||||
start), not the live server instance.
|
||||
"""
|
||||
override = os.environ.get("FUNPACK_COMFY_URL")
|
||||
if override:
|
||||
return override.rstrip("/")
|
||||
host, port = None, None
|
||||
try:
|
||||
from comfy.cli_args import args
|
||||
host = getattr(args, "listen", None)
|
||||
port = getattr(args, "port", None)
|
||||
except Exception:
|
||||
pass
|
||||
host = (host or "127.0.0.1").split(",")[0].strip()
|
||||
if host in ("0.0.0.0", "::", ""):
|
||||
host = "127.0.0.1" # not directly browsable; show the loopback equivalent
|
||||
port = int(port or 8188)
|
||||
return f"http://{host}:{port}"
|
||||
|
||||
|
||||
def ensure_dirs() -> None:
|
||||
PROJECTS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
MEDIA_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@@ -142,8 +142,24 @@
|
||||
if (!menu) return;
|
||||
if (!ta.isConnected) { close(); return; }
|
||||
const r = ta.getBoundingClientRect();
|
||||
// Flip above the field when there isn't room below (e.g. Easy Gen's prompt box
|
||||
// sits in a footer bar at the bottom of the viewport) — otherwise the menu opens
|
||||
// off-screen under the field with no way to see or click any suggestion.
|
||||
const GAP = 2;
|
||||
const menuH = menu.offsetHeight || 240;
|
||||
const spaceBelow = window.innerHeight - r.bottom;
|
||||
const spaceAbove = r.top;
|
||||
const openAbove = spaceBelow < menuH + GAP && spaceAbove > spaceBelow;
|
||||
menu.style.left = r.left + "px";
|
||||
menu.style.top = (r.bottom + 2) + "px";
|
||||
if (openAbove) {
|
||||
menu.style.top = "";
|
||||
menu.style.bottom = (window.innerHeight - r.top + GAP) + "px";
|
||||
menu.style.maxHeight = Math.max(120, Math.min(240, spaceAbove - GAP * 2)) + "px";
|
||||
} else {
|
||||
menu.style.bottom = "";
|
||||
menu.style.top = (r.bottom + GAP) + "px";
|
||||
menu.style.maxHeight = Math.max(120, Math.min(240, spaceBelow - GAP * 2)) + "px";
|
||||
}
|
||||
menu.style.minWidth = Math.min(r.width, 420) + "px";
|
||||
menu.style.maxWidth = Math.max(r.width, 320) + "px";
|
||||
}
|
||||
|
||||
@@ -217,8 +217,40 @@
|
||||
|
||||
// Variables ($name) — collapsible, same toggle pattern as Settings ▸ Built-in pipeline.
|
||||
wrap.append(composeVariables());
|
||||
// Segmented detailing targets — only while the Chain Sampler feature is enabled
|
||||
// (Engine ▸ Chain Sampler ▸ Experimental), so prompt-craft and "what gets detailed"
|
||||
// live side by side without adding a permanently-visible field.
|
||||
const detailBlock = composeDetailTargets(st);
|
||||
if (detailBlock) wrap.append(detailBlock);
|
||||
return wrap;
|
||||
}
|
||||
|
||||
// ── segmented detailing targets (shown only when the feature is on) ─────────────
|
||||
function composeDetailTargets(st) {
|
||||
const p = st.project;
|
||||
if (!p) return null;
|
||||
const PC = window.PipelineCaps;
|
||||
if (PC && PC.usesChainSampler && !PC.usesChainSampler(st)) return null;
|
||||
const si = p.sampler_inputs || {};
|
||||
if (!si.segmented_detailing) return null;
|
||||
const box = el("div", "compose-detail");
|
||||
const row = el("div", "compose-head");
|
||||
row.append(el("div", "lib-form-title", "Segmented detailing"));
|
||||
box.append(row);
|
||||
const input = el("input", "lib-in");
|
||||
input.type = "text";
|
||||
input.placeholder = "hands, feet";
|
||||
input.value = si.detail_targets != null ? String(si.detail_targets) : "hands";
|
||||
// Quiet while typing (no repaint under the caret), commit on blur/Enter —
|
||||
// same contract as the Engine Settings text knob, same store key, so the
|
||||
// two fields can never disagree.
|
||||
input.oninput = () => S.setSamplerInput("detail_targets", input.value);
|
||||
input.onchange = () => S.setSamplerInputNow("detail_targets", input.value);
|
||||
box.append(input);
|
||||
box.append(el("div", "insp-hint",
|
||||
"Comma-separated regions to detail (CLIPSeg text queries — plain words work best). Strength and the upsampler model live in Engine ▸ Chain Sampler ▸ Experimental."));
|
||||
return box;
|
||||
}
|
||||
let composeTextarea = null;
|
||||
let varsOpen = false; // Variables panel expanded?
|
||||
let varHintEl = null; // undeclared / cycle hint line under the prompt
|
||||
@@ -834,6 +866,12 @@
|
||||
const fp = JSON.stringify({
|
||||
s: st.shortcuts?.length, t: st.transitions?.length, pid: st.project?.id,
|
||||
c: (st.shortcutCategories || []).reduce((n, x) => n + 1 + (x.sub_categories?.length || 0), 0),
|
||||
// Segmented detailing block visibility: toggling it in Engine Settings must
|
||||
// repaint an open Composer or the targets field appears/disappears only on the
|
||||
// next unrelated repaint ("store right, UI stale"). The targets TEXT is not
|
||||
// fingerprinted — typing goes through quiet patches and the field itself is
|
||||
// the source of those edits.
|
||||
sd: !!(st.project?.sampler_inputs?.segmented_detailing),
|
||||
});
|
||||
if (fp === lastFp) return;
|
||||
lastFp = fp;
|
||||
|
||||
@@ -7,6 +7,20 @@
|
||||
const S = window.Store;
|
||||
const API = window.MovieEditorAPI;
|
||||
|
||||
// Easy Gen has no rating UI at all (by design — see easy_gen/frontend/), so every
|
||||
// setting that is a no-op without a trained refinement key / rated history is hidden
|
||||
// there, not just made harder to find. window.FunPackAppName is the same discriminator
|
||||
// settings_window.js already uses to relabel the About section for a different frontend
|
||||
// sharing this file; Editor leaves it unset.
|
||||
const EASY = !!window.FunPackAppName;
|
||||
const RATING_GATED_KNOBS = new Set([
|
||||
"embed_guidance", "embed_guidance_source", "embed_guidance_strength",
|
||||
"score_slider", "score_slider_strength", "taste_nearest_prompt",
|
||||
"output_guidance", "output_guidance_strength",
|
||||
"dynashift", "dynashift_strength", "dynashift_threshold",
|
||||
]);
|
||||
const RATING_GATED_STUDIO = new Set(["reference_injection", "value_guidance", "steer_mode", "absolute_strength"]);
|
||||
|
||||
let _mounted = null; // { scroller (pane, set per render), content (shell root) }
|
||||
let unsub = null;
|
||||
let _editing = false;
|
||||
@@ -58,7 +72,7 @@
|
||||
{ name: "absolute_strength", label: "Absolute strength", kind: "float", default: 0.6, min: 0, max: 1, step: 0.05,
|
||||
dependsOn: "steer_mode", dependsVals: ["absolute", "both"] },
|
||||
{ name: "temporal_style", label: "Temporal style", kind: "combo",
|
||||
choices: ["natural", "auto", "accelerate", "decelerate", "loop", "freeze", "pulse"], default: "natural" },
|
||||
choices: ["natural", "auto", "accelerate", "decelerate", "loop", "freeze", "pulse", "rapid_start", "rapid_end", "rapid_start_end"], default: "natural" },
|
||||
{ name: "split_transition_placement", label: "Transition placement", kind: "combo",
|
||||
choices: ["start", "end", "silent"], default: "start" },
|
||||
];
|
||||
@@ -75,10 +89,11 @@
|
||||
const { rf } = parseStudioSettings(p);
|
||||
let n = 0;
|
||||
[...STUDIO_REFINER_ESSENTIALS, ...STUDIO_REFINER_ADVANCED].forEach((f) => {
|
||||
if (EASY && RATING_GATED_STUDIO.has(f.name)) return;
|
||||
const cur = rf[f.name] != null ? rf[f.name] : f.default;
|
||||
if (cur !== f.default) n++;
|
||||
});
|
||||
if ((p.refinement_key || "default") !== "default") n++;
|
||||
if (!EASY && (p.refinement_key || "default") !== "default") n++;
|
||||
return n;
|
||||
}
|
||||
|
||||
@@ -196,6 +211,21 @@
|
||||
hint: "EXPERIMENTAL speed: the near-pure-noise early steps carry almost no signal, so the transformer output barely changes across them. Computes it once at the top of the plateau and reuses it for the rest, skipping ~3-4 of 8 transformer passes. Deterministic given seed (safe in Batch Training) but an approximation — A/B it before trusting on finals. Much of wall-clock time is outside the sampler, so total speedup is smaller than the forward count suggests. UNVALIDATED LIVE." },
|
||||
{ name: "plateau_cache_threshold", label: "Plateau threshold (sigma)", kind: "float", default: 0.975, min: 0.5, max: 0.999, step: 0.005, dependsOn: "plateau_cache",
|
||||
hint: "Steps with sigma at or above this count as the reusable plateau. Higher = fewer steps cached (safer); lower = more cached (faster, more approximation). 0.975 catches the documented noise plateau while leaving structure formation fully computed." },
|
||||
{ name: "segmented_detailing", label: "Segmented detailing (region refine)", kind: "bool", default: false,
|
||||
hint: "EXPERIMENTAL ADetailer-for-video: after each scene renders, CLIPSeg finds the regions named below (hands, feet, …), cuts them out of the latent as a tube, refines them at 2× working resolution through Lightricks' latent upsampler (3 extra steps on the crop only), and pastes them back through a feathered silhouette. Final resolution never changes. Cost ≈ 4 × region area × 3 steps (hands ~+15%); regions over 35% of the frame are refused. The upsampler model below is found — or downloaded (~1 GB, once) — automatically; skips are reported in the scene report. UNVALIDATED LIVE." },
|
||||
{ name: "detail_targets", label: "Detail targets", kind: "text", default: "hands", dependsOn: "segmented_detailing", placeholder: "hands, feet",
|
||||
hint: "Comma-separated regions to detail, in plain words. Each becomes a CLIPSeg text query — malformed anatomy still matches its name. Also editable from Composer ▸ Compose while detailing is on." },
|
||||
{ name: "detail_strength", label: "Detail strength", kind: "float", default: 1.0, min: 0, max: 1, step: 0.05, dependsOn: "segmented_detailing",
|
||||
hint: "Blend of the refined region at paste-back. 1.0 = full replacement inside the silhouette; 0 disables the pass." },
|
||||
{ name: "detail_threshold", label: "Detail match threshold", kind: "float", default: 0.35, min: 0.05, max: 0.9, step: 0.05, dependsOn: "segmented_detailing",
|
||||
hint: "CLIPSeg match confidence required before a region counts as found. Its raw score for a real region is often well under 0.5 — if the scene report shows 'no match: max CLIPSeg score X < threshold', lower this toward X rather than assuming nothing is there." },
|
||||
{ name: "detail_max_area", label: "Detail max area", kind: "float", default: 0.35, min: 0.05, max: 1.0, step: 0.05, dependsOn: "segmented_detailing",
|
||||
hint: "Ceiling on how much of the frame the region may cover before it's refused, as a fraction of the frame. Cost-only guardrail (a bigger region costs more, roughly 4× its area × 3 steps) — never a judgment call about whether it's worth detailing. If the scene report shows a region refused at some %, raise this above that % to detail it anyway. 1.0 = no cap." },
|
||||
{ name: "detail_mode", label: "Detail mode", kind: "combo", choices: ["repair", "sharpen"], default: "repair", dependsOn: "segmented_detailing",
|
||||
hint: "'repair' (default): upsamples the crop, then re-denoises it through the video model — can genuinely fix wrong structure (bad anatomy) but costs real compute (~4× region area × 3 steps). 'sharpen': stops after the upsampler's own pass — no video-model calls at all, close to free — good for a region that's blurry/under-resolved but already correctly shaped; it cannot fix wrong structure (an extra finger stays an extra finger, just sharper)." },
|
||||
{ name: "detail_denoise", label: "Detail re-noise strength", kind: "float", default: 0.85, min: 0.3, max: 0.99, step: 0.05,
|
||||
deps: [{ name: "segmented_detailing" }, { name: "detail_mode", value: "repair" }],
|
||||
hint: "How much noise the crop gets re-noised to before its 3-step refine (0.85 is the official LTX 2.3 recipe's own value). Higher = more freedom to genuinely reconstruct the region (fix bad anatomy), risking drift from the surrounding frame; lower = closer to a plain upscale — looks 'detailed' as interpolation but doesn't actually repair it. If the result looks upscaled but not corrected, raise this. Only used in 'repair' mode." },
|
||||
];
|
||||
const SAMPLER_KNOB_MAP = Object.fromEntries(SAMPLER_KNOBS.map((k) => [k.name, k]));
|
||||
|
||||
@@ -241,10 +271,23 @@
|
||||
S.patchProject({ guide_settings: { ...(p.guide_settings || {}), ...patch } });
|
||||
}
|
||||
|
||||
function _depSatisfied(name, value, si) {
|
||||
const depVal = si[name] != null ? si[name] : SAMPLER_KNOB_MAP[name]?.default;
|
||||
return value !== undefined ? depVal === value : !!depVal;
|
||||
}
|
||||
|
||||
function knobVisible(k, si) {
|
||||
if (k.dependsOn) {
|
||||
const depVal = si[k.dependsOn] != null ? si[k.dependsOn] : SAMPLER_KNOB_MAP[k.dependsOn]?.default;
|
||||
if (!depVal) return false;
|
||||
if (EASY && RATING_GATED_KNOBS.has(k.name)) return false;
|
||||
// dependsOn/dependsValue: single condition (dependsValue absent = plain truthy
|
||||
// gate, as every existing boolean dependsOn already relies on).
|
||||
if (k.dependsOn && !_depSatisfied(k.dependsOn, k.dependsValue, si)) return false;
|
||||
// deps: an AND'd list, for knobs gated by more than one condition (e.g. the
|
||||
// feature toggle AND a mode combo equaling a specific option) — dependsOn
|
||||
// alone can't express two conditions without one silently overriding the other.
|
||||
if (Array.isArray(k.deps)) {
|
||||
for (const d of k.deps) {
|
||||
if (!_depSatisfied(d.name, d.value, si)) return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
@@ -271,6 +314,14 @@
|
||||
ctrl = el("select"); ctrl.dataset.k = "si-" + k.name;
|
||||
(k.choices || []).forEach((c) => { const o = el("option", null, c); o.value = c; if (c === val) o.selected = true; ctrl.append(o); });
|
||||
ctrl.onchange = () => S.setSamplerInputNow(k.name, ctrl.value);
|
||||
} else if (k.kind === "text") {
|
||||
ctrl = el("input"); ctrl.type = "text";
|
||||
ctrl.value = val != null ? String(val) : "";
|
||||
if (k.placeholder) ctrl.placeholder = k.placeholder;
|
||||
ctrl.dataset.k = "si-" + k.name;
|
||||
// Quiet while typing (no repaint under the caret), commit on blur/Enter.
|
||||
ctrl.oninput = () => S.setSamplerInput(k.name, ctrl.value);
|
||||
ctrl.onchange = () => S.setSamplerInputNow(k.name, ctrl.value);
|
||||
} else {
|
||||
ctrl = el("input"); ctrl.type = "number";
|
||||
if (k.step != null) ctrl.step = String(k.step);
|
||||
@@ -300,13 +351,14 @@
|
||||
chain_timing: ["frame_overlap", "transition_duration", "use_same_seed"],
|
||||
chain_guidance: ["cfg", "embed_guidance", "embed_guidance_source", "embed_guidance_strength", "score_slider", "score_slider_strength", "taste_nearest_prompt", "output_guidance", "output_guidance_strength", "dynashift", "dynashift_strength", "dynashift_threshold"],
|
||||
chain_decode: ["decode_noise_scale", "decode_timestep", "decode_tile_size"],
|
||||
chain_experimental: ["plateau_cache", "plateau_cache_threshold", "mid_scene_guide", "mid_scene_guide_strength", "joyai_memory", "joyai_memory_size", "joyai_fix_frames", "joyai_frame_select", "joyai_memory_strength", "joyai_audio_memory", "v2a_grad_scale", "alg_blur_guides", "alg_guide_blur_strength", "alg_guide_blur_sigma_threshold", "bounded_attention_enabled", "identity_transfer_enabled", "source_id", "phase_scale", "id_strength", "arcface_mode", "debug_log"],
|
||||
chain_experimental: ["plateau_cache", "plateau_cache_threshold", "segmented_detailing", "detail_targets", "detail_strength", "detail_threshold", "detail_max_area", "detail_mode", "detail_denoise", "mid_scene_guide", "mid_scene_guide_strength", "joyai_memory", "joyai_memory_size", "joyai_fix_frames", "joyai_frame_select", "joyai_memory_strength", "joyai_audio_memory", "v2a_grad_scale", "alg_blur_guides", "alg_guide_blur_strength", "alg_guide_blur_sigma_threshold", "bounded_attention_enabled", "identity_transfer_enabled", "source_id", "phase_scale", "id_strength", "arcface_mode", "debug_log"],
|
||||
};
|
||||
|
||||
function countChainView(p, id) {
|
||||
const si = p.sampler_inputs || {};
|
||||
let n = 0;
|
||||
(CHAIN_VIEW_KNOBS[id] || []).forEach((name) => {
|
||||
if (EASY && RATING_GATED_KNOBS.has(name)) return;
|
||||
const k = SAMPLER_KNOB_MAP[name];
|
||||
if (k && si[name] != null && si[name] !== k.default) n++;
|
||||
});
|
||||
@@ -396,22 +448,35 @@
|
||||
const p = st.project;
|
||||
const { rf } = parseStudioSettings(p);
|
||||
|
||||
// Refinement key — project-level (feeds Studio / Chain Sampler / SaveRefinementLatent).
|
||||
// "default" uses the keyless store; a custom name trains/loads its own key. Shortcuts
|
||||
// bound to a non-default key layer per-scene training on top of this.
|
||||
const gKey = group(pane, "Session");
|
||||
const keyCtrl = el("input"); keyCtrl.type = "text"; keyCtrl.dataset.k = "refinement_key";
|
||||
keyCtrl.placeholder = "default"; keyCtrl.value = p.refinement_key || "default";
|
||||
keyCtrl.onchange = () => S.patchProject({ refinement_key: (keyCtrl.value || "").trim() || "default" });
|
||||
gKey.append(field("Refinement key", keyCtrl, "Named learning session — \"default\" is the keyless store."));
|
||||
if (!EASY) {
|
||||
// Refinement key — project-level (feeds Studio / Chain Sampler / SaveRefinementLatent).
|
||||
// "default" uses the keyless store; a custom name trains/loads its own key. Shortcuts
|
||||
// bound to a non-default key layer per-scene training on top of this.
|
||||
const gKey = group(pane, "Session");
|
||||
const keyCtrl = el("input"); keyCtrl.type = "text"; keyCtrl.dataset.k = "refinement_key";
|
||||
keyCtrl.placeholder = "default"; keyCtrl.value = p.refinement_key || "default";
|
||||
keyCtrl.onchange = () => S.patchProject({ refinement_key: (keyCtrl.value || "").trim() || "default" });
|
||||
gKey.append(field("Refinement key", keyCtrl, "Named learning session — \"default\" is the keyless store."));
|
||||
}
|
||||
|
||||
const gEss = group(pane, "Essentials");
|
||||
STUDIO_REFINER_ESSENTIALS.forEach((f) => renderStudioRefinerBool(gEss, rf, f));
|
||||
STUDIO_REFINER_ESSENTIALS.filter((f) => !EASY || !RATING_GATED_STUDIO.has(f.name))
|
||||
.forEach((f) => renderStudioRefinerBool(gEss, rf, f));
|
||||
|
||||
const gAdv = group(pane, "Refinement");
|
||||
STUDIO_REFINER_ADVANCED.forEach((f) => renderStudioRefinerField(gAdv, rf, f));
|
||||
const gAdv = group(pane, EASY ? "Prompt shaping" : "Refinement");
|
||||
STUDIO_REFINER_ADVANCED.filter((f) => !EASY || !RATING_GATED_STUDIO.has(f.name))
|
||||
.forEach((f) => renderStudioRefinerField(gAdv, rf, f));
|
||||
|
||||
pane.append(hintEl("Scene text and transitions come from the timeline. Advisor, LoRA, and batch training remain in the ComfyUI Studio popup on the graph."));
|
||||
if (EASY) {
|
||||
pane.append(hintEl(
|
||||
"Studio runs in Prompt-only mode from Easy Gen — it shapes and splits the prompt "
|
||||
+ "and passes conditioning through unchanged. Rating-dependent controls (refinement "
|
||||
+ "key, value guidance, steer mode, reference injection) aren't shown here since "
|
||||
+ "there's no rating UI in Easy Gen to feed them. For the full learned refiner, use "
|
||||
+ "the Cutting Room (Movie Editor) or the ComfyUI node graph directly."));
|
||||
} else {
|
||||
pane.append(hintEl("Scene text and transitions come from the timeline. Advisor, LoRA, and batch training remain in the ComfyUI Studio popup on the graph."));
|
||||
}
|
||||
}
|
||||
|
||||
function renderStudioAdjust(pane, st) {
|
||||
@@ -576,6 +641,47 @@
|
||||
renderKnobList(g, st, CHAIN_VIEW_KNOBS[id]);
|
||||
}
|
||||
|
||||
// ── Segmented detailing: the upsampler model list is live (models/latent_upscale_models
|
||||
// on the server), so like the ArcFace projector below it can't be a static SAMPLER_KNOBS
|
||||
// combo. The chain sampler node's own spec already carries the choices (with "None").
|
||||
let _detailUpsamplerChoices = null;
|
||||
|
||||
async function ensureDetailUpsamplerChoices() {
|
||||
if (_detailUpsamplerChoices) return _detailUpsamplerChoices;
|
||||
try {
|
||||
const spec = await API.nodeSpec("FunPackLTXAVSceneChainSampler");
|
||||
const w = (spec?.inputs || []).find((i) => i.name === "detail_upsampler");
|
||||
_detailUpsamplerChoices = (w && w.choices) || ["None"];
|
||||
} catch (_) {
|
||||
_detailUpsamplerChoices = ["None"];
|
||||
}
|
||||
render();
|
||||
return _detailUpsamplerChoices;
|
||||
}
|
||||
|
||||
function renderDetailUpsampler(pane, si) {
|
||||
if (!si.segmented_detailing) return;
|
||||
const g = group(pane, "Segmented detailing: upsampler model");
|
||||
if (!_detailUpsamplerChoices) {
|
||||
g.append(hintEl("Loading upsampler list…"));
|
||||
ensureDetailUpsamplerChoices();
|
||||
return;
|
||||
}
|
||||
const sel = el("select"); sel.dataset.k = "si-detail_upsampler";
|
||||
const cur = si.detail_upsampler && si.detail_upsampler !== "None" ? si.detail_upsampler : "auto";
|
||||
_detailUpsamplerChoices.forEach((c) => {
|
||||
const o = el("option", null, c); o.value = c;
|
||||
if (c === cur) o.selected = true;
|
||||
sel.append(o);
|
||||
});
|
||||
sel.onchange = () => S.setSamplerInputNow("detail_upsampler", sel.value);
|
||||
g.append(field("Latent upsampler", sel,
|
||||
"The LTX 2.3 spatial upsampler from models/latent_upscale_models (the official two-stage workflows use the same file). 'auto' picks the newest installed spatial upscaler — or downloads the official one (~1 GB, once) when the folder is empty."));
|
||||
if (_detailUpsamplerChoices.length <= 1) {
|
||||
g.append(hintEl("Nothing installed in models/latent_upscale_models yet — the first detailed run downloads the official upsampler automatically (watch the ComfyUI console)."));
|
||||
}
|
||||
}
|
||||
|
||||
// ── Best-FaceID ArcFace projector: the one identity-transfer field that needs a
|
||||
// live (server-fetched) choice list, so it can't live in the static SAMPLER_KNOBS combo
|
||||
// shape. Reuses the same loras folder listing LoraLoaderModelOnly exposes.
|
||||
@@ -598,6 +704,7 @@
|
||||
renderChainKnobsView(pane, st, "chain_experimental", "Experimental",
|
||||
"Research techniques — off by default; expect quality/overhead trade-offs.");
|
||||
const si = st.project.sampler_inputs || {};
|
||||
renderDetailUpsampler(pane, si);
|
||||
if (!si.identity_transfer_enabled) return;
|
||||
const g = group(pane, "Best-FaceID: ArcFace projector");
|
||||
if (!_loraChoices) {
|
||||
@@ -636,7 +743,9 @@
|
||||
case "studio_sampler": return renderStudioSampler(pane, st);
|
||||
case "chain_continuity": return renderChainContinuity(pane, st);
|
||||
case "chain_timing": return renderChainTiming(pane, st);
|
||||
case "chain_guidance": return renderChainKnobsView(pane, st, "chain_guidance", "Guidance");
|
||||
case "chain_guidance": return renderChainKnobsView(pane, st, "chain_guidance", "Guidance",
|
||||
EASY ? "Rating-dependent guidance (embed guidance, score slider, output guidance, taste retrieval, "
|
||||
+ "DynaShift) is hidden here — use the Cutting Room or ComfyUI graph for those." : null);
|
||||
case "chain_decode": return renderChainKnobsView(pane, st, "chain_decode", "Decode");
|
||||
case "chain_experimental": return renderChainExperimental(pane, st);
|
||||
default: return renderOverview(pane, st);
|
||||
|
||||
@@ -4,6 +4,10 @@
|
||||
(function () {
|
||||
const { el, clear } = window.dom;
|
||||
const API = window.MovieEditorAPI;
|
||||
// Easy Gen has no per-scene inspector / main editor window for an exposed widget or
|
||||
// bypass toggle to actually surface in — the eye buttons below would just be inert
|
||||
// clutter (or worse, look actionable and do nothing visible), so they're hidden there.
|
||||
const EASY = !!window.FunPackAppName;
|
||||
|
||||
let roles = []; // [{key,label,category}]
|
||||
let ports = []; // pipeline connection points [{id,type,label}]
|
||||
@@ -439,7 +443,7 @@
|
||||
byp.title = "Skip this node's effect (pass its inputs straight through) without losing its configuration.";
|
||||
byp.onclick = async (e) => { e.stopPropagation(); slot.bypassed = !slot.bypassed; await persist(); render(); };
|
||||
acts.append(byp);
|
||||
acts.append(bypassEyeButton(slot));
|
||||
if (!EASY) acts.append(bypassEyeButton(slot));
|
||||
const rm = el("button", "btn ghost tiny danger", "remove");
|
||||
rm.onclick = async (e) => {
|
||||
e.stopPropagation();
|
||||
@@ -465,7 +469,7 @@
|
||||
const l2 = linkOf(slot.id, spec.name); if (l2) applyLinkValue(l2, v); // keep group in sync
|
||||
await persist();
|
||||
});
|
||||
f.classList.add("with-eye");
|
||||
if (!EASY || iw || lk || linkMode) f.classList.add("with-eye");
|
||||
if (iw) {
|
||||
// Wired from another node's output: the widget value is replaced by the
|
||||
// connection at generation — lock the field and say where it comes from
|
||||
@@ -490,7 +494,7 @@
|
||||
render();
|
||||
};
|
||||
f.append(chk);
|
||||
} else {
|
||||
} else if (!EASY) {
|
||||
f.append(eyeButton(slot, spec));
|
||||
}
|
||||
grid.append(f);
|
||||
|
||||
@@ -6,6 +6,10 @@
|
||||
(function () {
|
||||
const { el } = window.dom;
|
||||
|
||||
// Easy Gen has no rating UI — velocity bias / rescue are no-ops without a rated
|
||||
// velocity bank, so they're hidden there (see engine_settings.js for the same gate).
|
||||
const EASY = !!window.FunPackAppName;
|
||||
|
||||
const SAMPLER_TYPES = ["Hybrid Euler 2S", "Distilled Flow", "KSampler"];
|
||||
const VELOCITY_BIAS_MODES = ["off", "capture", "apply", "capture_and_apply"];
|
||||
const MOTION_PULSE_MODES = ["off", "balanced", "aggressive", "custom"];
|
||||
@@ -125,6 +129,11 @@
|
||||
// ── velocity-bias / rescue block (shared by Hybrid and Distilled) ──
|
||||
|
||||
function renderVelocityBlock(c, dk, sub, save, saveNow) {
|
||||
if (EASY) {
|
||||
hint(c, "Velocity bias / rescue mode hidden — both are no-ops without a rated velocity bank, "
|
||||
+ "which Easy Gen has no UI for. Use the Cutting Room or ComfyUI graph instead.");
|
||||
return;
|
||||
}
|
||||
const vbm = selCtrl(VELOCITY_BIAS_MODES, sub.velocity_bias_mode || "off", dk + "-vbm",
|
||||
(v) => { sub.velocity_bias_mode = v; saveNow(); });
|
||||
row(c, "velocity bias mode", vbm);
|
||||
|
||||
@@ -197,7 +197,10 @@
|
||||
const git = G ? G.get() : null;
|
||||
wrap.append(el("div", "sw-about-mark", "◉"));
|
||||
wrap.append(el("div", "sw-about-name", "FunPack"));
|
||||
wrap.append(el("div", "sw-about-sub", "Cutting Room"));
|
||||
// window.FunPackAppName lets a different frontend sharing this file (e.g. Easy
|
||||
// Gen) relabel the app name without forking the whole section; Editor leaves it
|
||||
// unset and keeps "Cutting Room".
|
||||
wrap.append(el("div", "sw-about-sub", window.FunPackAppName || "Cutting Room"));
|
||||
|
||||
const facts = el("div", "sw-about-facts");
|
||||
const fact = (k, v) => {
|
||||
|
||||
+20
-4
@@ -1948,9 +1948,19 @@ if web is not None and PromptServer is not None:
|
||||
if path and os.path.isfile(path):
|
||||
import mimetypes as _mt
|
||||
ctype = _mt.guess_type(path)[0] or "application/octet-stream"
|
||||
# The render's own filename is already unique per run (builder.py's per-run
|
||||
# uuid filename_prefix) — but with no Content-Disposition, "Save Video As" falls
|
||||
# back to whatever the browser derives from the URL path, which has no filename
|
||||
# here (it's in the query string) and defaults to a generic name like "result.mp4"
|
||||
# every time. Surface the real name so downloads don't collide/need renaming
|
||||
# (Easy Gen has no export button, so this native browser save is the only path out).
|
||||
dl_name = req.query.get("filename", "") or os.path.basename(path)
|
||||
if ctype.startswith("video/"):
|
||||
path = await _faststart_path(path)
|
||||
return web.FileResponse(path, headers={"Content-Type": ctype})
|
||||
return web.FileResponse(path, headers={
|
||||
"Content-Type": ctype,
|
||||
"Content-Disposition": f'inline; filename="{dl_name}"',
|
||||
})
|
||||
# Fallback (custom output dirs the resolver doesn't know): loopback /view fetch.
|
||||
try:
|
||||
data, ctype = await bridge.fetch_view(
|
||||
@@ -1984,8 +1994,14 @@ if web is not None and PromptServer is not None:
|
||||
f.write(data)
|
||||
os.replace(tmp, spool) # atomic: no reader ever sees a partial spool
|
||||
path = await _faststart_path(spool)
|
||||
return web.FileResponse(path, headers={"Content-Type": ctype.split(";")[0]})
|
||||
return web.Response(body=data, content_type=ctype.split(";")[0])
|
||||
return web.FileResponse(path, headers={
|
||||
"Content-Type": ctype.split(";")[0],
|
||||
"Content-Disposition": f'inline; filename="{fname}"',
|
||||
})
|
||||
return web.Response(
|
||||
body=data, content_type=ctype.split(";")[0],
|
||||
headers={"Content-Disposition": f'inline; filename="{fname}"'},
|
||||
)
|
||||
|
||||
# --- API: final render (async job — ffmpeg can exceed Cloudflare tunnel timeouts) ---
|
||||
import asyncio
|
||||
@@ -2417,4 +2433,4 @@ if web is not None and PromptServer is not None:
|
||||
async def _static(req):
|
||||
return _serve_static(req.match_info["tail"])
|
||||
|
||||
print(f"[FunPack] Movie Editor available at {UI_PREFIX}/")
|
||||
print(f"[FunPack] Movie Editor available at {config.comfy_display_url()}{UI_PREFIX}/")
|
||||
|
||||
@@ -344,12 +344,14 @@ def test_non_bypassed_lora_stays_in_graph():
|
||||
assert graph["studio"]["inputs"]["clip"] == ["slot_lora", 1]
|
||||
|
||||
|
||||
def test_bypass_on_node_with_no_matching_input_leaves_it_active_and_reports():
|
||||
# VaeLoader has no connection_input at all, so there's nothing to pass an output through —
|
||||
# bypass must be a documented no-op here, not a crash or a silently dropped node.
|
||||
def test_bypass_on_node_with_no_matching_input_blocks_generation():
|
||||
# VaeLoader has no connection_input at all, so there's nothing to pass an output through.
|
||||
# A bypass a user explicitly asked for must never be silently ignored — generation blocks
|
||||
# with a clear reason instead of quietly leaving the node active.
|
||||
models = {"full_control": True, "slots": [
|
||||
{"id": "v", "node_class": "VaeLoader", "bypassed": True, "inputs": {}, "wires": {}},
|
||||
]}
|
||||
graph, report = builder.build(OI, models, PARAMS)
|
||||
assert "slot_v" in graph
|
||||
assert any("bypass needs exactly one input" in u for u in report["unsatisfied"])
|
||||
assert any("bypass needs exactly one input" in b for b in report["blocking"])
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
[project]
|
||||
name = "ComfyUI-FunPack" # Unique identifier for your node. Immutable after creation.
|
||||
description = "An AI movie studio inside ComfyUI: the Cutting Room browser-based video editor with timeline, per-scene prompts and one-click montage generation, plus a self-improving engine that learns from your ratings. Includes the full node stack (Studio, Refiner V2, LTXAV Scene Chain Sampler) for classic graph workflows."
|
||||
version = "3.3.0" # Custom Node version. Must be semantically versioned.
|
||||
version = "3.4.0" # Custom Node version. Must be semantically versioned.
|
||||
license = { text = "GNU GPL v3" }
|
||||
dependencies = [] # Filled in from requirements.txt
|
||||
|
||||
|
||||
+102
-7
@@ -2450,6 +2450,38 @@ class FunPackLTXAVSceneChainSampler:
|
||||
"default": False,
|
||||
"tooltip": "EXPERIMENTAL: source embed_guidance / score_slider from the taste direction learned on the prompts NEAREST this scene's prompt, instead of the single global liked-direction average. On every liked rating the Refiner records (prompt fingerprint -> that run's liked direction); with this on, each scene retrieves the similarity-weighted direction of its closest matches (a forest prompt pulls what worked on forests, not the mean across all prompts). Non-parametric retrieval — no extra model forward, just a cosine lookup + vector mean, and it can't collapse into a spurious attractor the way a value function can. Falls back to the global liked direction when no rated prompt is close enough (or the index is empty). Only affects embed_guidance / score_slider; needs refinement_key_input (or embed_guidance_source=absolute). UNVALIDATED LIVE.",
|
||||
}),
|
||||
"segmented_detailing": ("BOOLEAN", {
|
||||
"default": False,
|
||||
"tooltip": "EXPERIMENTAL ADetailer-for-video: after each scene finishes denoising, CLIPSeg (text-prompted segmentation) locates the regions named in detail_targets on a few decoded keyframes; the matched region is cut out of the video latent as a spatiotemporal tube, pushed through Lightricks' trained latent upsampler (2x spatial — the official two-stage pipeline's stage-2 model), then either used directly (detail_mode='sharpen', near-free) or re-noised + re-denoised for a 3-step tail (detail_mode='repair', default — costs ~4x tube area fraction x 3 steps, hands ~+15%). Downscaled back to its ORIGINAL latent size and pasted through the feathered CLIPSeg silhouette either way. Final resolution never changes. Tubes over detail_max_area (default 35%) are refused as a cost guard, not a content judgment. Audio untouched by construction. detail_upsampler 'auto' finds or downloads the official Lightricks upsampler (~1 GB, once); skips are reported loudly in console + scene report. UNVALIDATED LIVE.",
|
||||
}),
|
||||
"detail_targets": ("STRING", {
|
||||
"default": "hands",
|
||||
"tooltip": "Comma-separated regions to detail, in plain words ('hands', 'hands, feet', 'face'). Each becomes a CLIPSeg text query; matched regions merge into one tube per scene. CLIPSeg matches broad CLIP semantics, so malformed anatomy still lights up for its name.",
|
||||
}),
|
||||
"detail_upsampler": (cls._detail_upsampler_choices(), {
|
||||
"default": "auto",
|
||||
"tooltip": "Latent upsampler checkpoint from models/latent_upscale_models (the LTX 2.3 spatial upsampler used by the official two-stage workflows). 'auto' picks the newest installed spatial upscaler, or downloads the official file (~1 GB, once) when the folder is empty. Pick a file explicitly to pin it.",
|
||||
}),
|
||||
"detail_strength": ("FLOAT", {
|
||||
"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.05,
|
||||
"tooltip": "Blend of the refined region into the frame at paste-back (through the feathered CLIPSeg mask). 1.0 = full replacement inside the silhouette; 0 disables the pass entirely.",
|
||||
}),
|
||||
"detail_threshold": ("FLOAT", {
|
||||
"default": 0.35, "min": 0.05, "max": 0.9, "step": 0.05,
|
||||
"tooltip": "CLIPSeg match confidence (post-sigmoid) required before a region counts as found. CLIPSeg's raw score for a real, correctly-named region is often well under 0.5 — if the scene report shows 'no match: max CLIPSeg score X < threshold', lower this toward X (or just below it) rather than assuming nothing is there. Lower = more permissive (more false positives on unrelated regions); higher = stricter.",
|
||||
}),
|
||||
"detail_max_area": ("FLOAT", {
|
||||
"default": 0.35, "min": 0.05, "max": 1.0, "step": 0.05,
|
||||
"tooltip": "Ceiling on how much of the frame the detected region may cover before the pass refuses it, as a fraction of frame area. This is a COST guardrail only (cost ~= 4x area x 3 steps, so a large region starts to rival a second full render), never a judgment about whether the region is worth detailing — if the scene report shows a region refused at some %, raise this above that % to detail it anyway (up to 1.0 = no cap, full-frame allowed).",
|
||||
}),
|
||||
"detail_denoise": ("FLOAT", {
|
||||
"default": 0.85, "min": 0.3, "max": 0.99, "step": 0.05,
|
||||
"tooltip": "Only used in 'repair' mode. How much noise the crop is re-noised to before the 3-step refine tail (the official LTX 2.3 two-stage recipe's own value, 0.85, is the default). Higher = more freedom for the model to genuinely reconstruct the region (fix bad anatomy) at the cost of possibly drifting from the surrounding frame; lower = closer to a plain upscale (looks 'detailed' as interpolation, but doesn't actually repair the region — if that's what you're seeing, raise this).",
|
||||
}),
|
||||
"detail_mode": (["repair", "sharpen"], {
|
||||
"default": "repair",
|
||||
"tooltip": "'repair' (default): upsample the crop, then re-denoise it through the video model for 3 extra steps — can genuinely fix wrong structure (bad anatomy) but costs real compute (~4x region area x 3 steps). 'sharpen': stop after the upsampler's own forward pass — no video-model calls at all, close to free — good for a region that's blurry/under-resolved but already correctly shaped; it CANNOT fix wrong structure (an extra finger stays an extra finger, just sharper), since a super-resolution net only adds detail consistent with what's already there.",
|
||||
}),
|
||||
},
|
||||
"hidden": {
|
||||
"unique_id": "UNIQUE_ID",
|
||||
@@ -2474,6 +2506,17 @@ class FunPackLTXAVSceneChainSampler:
|
||||
except Exception:
|
||||
return ["None"]
|
||||
|
||||
@classmethod
|
||||
def _detail_upsampler_choices(cls):
|
||||
# "auto": prefer an installed spatial upscaler, else download the official
|
||||
# Lightricks file on first use. ("None" from older saved workflows/projects is
|
||||
# treated as auto too — the enable toggle is the only off switch.)
|
||||
try:
|
||||
import folder_paths
|
||||
return ["auto"] + folder_paths.get_filename_list("latent_upscale_models")
|
||||
except Exception:
|
||||
return ["auto"]
|
||||
|
||||
def _is_nested(self, samples):
|
||||
return bool(getattr(samples, "is_nested", False))
|
||||
|
||||
@@ -4637,6 +4680,9 @@ class FunPackLTXAVSceneChainSampler:
|
||||
carry_overlap_through_anchor=False,
|
||||
plateau_cache=False, plateau_cache_threshold=0.975,
|
||||
taste_nearest_prompt=False,
|
||||
segmented_detailing=False, detail_targets="hands",
|
||||
detail_upsampler="None", detail_strength=1.0, detail_threshold=0.35,
|
||||
detail_max_area=0.35, detail_denoise=0.85, detail_mode="repair",
|
||||
unique_id=None, prompt=None):
|
||||
if not isinstance(positive, list) or not positive:
|
||||
raise ValueError("positive conditioning must contain at least one scene entry.")
|
||||
@@ -4825,6 +4871,8 @@ class FunPackLTXAVSceneChainSampler:
|
||||
scene_runs: list = []
|
||||
joyai_bank = _JoyAIMemoryBank(joyai_memory_size, joyai_fix_frames) if joyai_memory else None
|
||||
_identity_overlap_state: dict = {}
|
||||
_detail_upsampler_model = None # lazy: resolved+loaded at the first detailed scene
|
||||
_detail_disabled_reason = None # set on resolve/load failure: don't retry per scene
|
||||
|
||||
for scene_index, scene_cond in enumerate(scene_conditionings):
|
||||
scene_positive = [scene_cond]
|
||||
@@ -4953,10 +5001,10 @@ class FunPackLTXAVSceneChainSampler:
|
||||
# Loop temporal style (auto director's funpack_temporal_loop): Mobius latent
|
||||
# roll. Installed BELOW the guidance wrappers (embed guidance / slider /
|
||||
# dynashift / output guidance) so they see canonical-orientation inputs and
|
||||
# predictions — the roll exists only around the base forward. Calls that
|
||||
# carry guides (keyframe_idxs pins appended frames to absolute positions)
|
||||
# are left canonical by the wrapper itself, so on guide-carrying scenes the
|
||||
# roll is inert rather than corrupting.
|
||||
# predictions — the roll exists only around the base forward. Appended guide
|
||||
# frames stay pinned: the wrapper rolls only the content region in front of
|
||||
# the guide tail (counted from keyframe_idxs / the audio mask), so guides
|
||||
# keep informing the whole cycle without ever being dragged into it.
|
||||
if self._scene_temporal_loop(scene_cond):
|
||||
try:
|
||||
from .ltx_enhancements import make_loop_temporal_wrapper
|
||||
@@ -4967,7 +5015,7 @@ class FunPackLTXAVSceneChainSampler:
|
||||
make_loop_temporal_wrapper(_loop_base), _loop_base)
|
||||
_loop_pinned = guide_tail > 0 or carried > 0 or carried_guide_frames > 0
|
||||
run_mechanisms.append(
|
||||
"temporal_loop_roll(inert: guides pin frames)" if _loop_pinned
|
||||
"temporal_loop_roll(content-only: guide tail pinned)" if _loop_pinned
|
||||
else "temporal_loop_roll")
|
||||
# taste_nearest_prompt: swap the single global liked direction for the one
|
||||
# learned on this scene's closest rated prompts (resolved from the ORIGINAL
|
||||
@@ -5001,8 +5049,9 @@ class FunPackLTXAVSceneChainSampler:
|
||||
# corrects whatever prediction those already produced, not the raw base one.
|
||||
run_mechanisms.append(f"output_guidance({output_guidance_strength})")
|
||||
self._build_output_guidance_wrapper(model, _output_value_fn, output_guidance_strength)
|
||||
# Per-scene temporal style (auto / pulse): layer a frame_rate wrapper on top
|
||||
# of whatever is installed (e.g. embed guidance).
|
||||
# Per-scene temporal style (auto / pulse / rapid_start / rapid_end /
|
||||
# rapid_start_end): layer a frame_rate wrapper on top of whatever is
|
||||
# installed (e.g. embed guidance).
|
||||
_scene_mode = self._scene_temporal_mode(scene_cond)
|
||||
_scene_mult = self._scene_temporal_mult(scene_cond)
|
||||
_cur_wrapper = model.model_options.get("model_function_wrapper")
|
||||
@@ -5014,6 +5063,14 @@ class FunPackLTXAVSceneChainSampler:
|
||||
_tw = make_pulse_temporal_wrapper(_cur_wrapper)
|
||||
if _tw is not None:
|
||||
model.model_options["model_function_wrapper"] = _tag_scene_wrapper(_tw, _cur_wrapper)
|
||||
elif _scene_mode in ("rapid_start", "rapid_end", "rapid_start_end"):
|
||||
try:
|
||||
from .ltx_enhancements import make_rapid_temporal_wrapper
|
||||
except ImportError:
|
||||
from ltx_enhancements import make_rapid_temporal_wrapper
|
||||
_tw = make_rapid_temporal_wrapper(_cur_wrapper, _scene_mode)
|
||||
if _tw is not None:
|
||||
model.model_options["model_function_wrapper"] = _tag_scene_wrapper(_tw, _cur_wrapper)
|
||||
elif _scene_mult is not None and abs(_scene_mult - 1.0) >= 1e-3:
|
||||
try:
|
||||
from .ltx_enhancements import make_temporal_wrapper
|
||||
@@ -5074,6 +5131,44 @@ class FunPackLTXAVSceneChainSampler:
|
||||
sampled = self._crop_video_tail(sampled, guide_tail)
|
||||
if audio_tail > 0:
|
||||
sampled = self._crop_audio_tail(sampled, audio_tail)
|
||||
# Segmented detailing runs on the clean, fully-cropped scene (guide/audio
|
||||
# tails gone, overlap still present — the tube is spatial, so carried head
|
||||
# frames detail together with the rest) and BEFORE the JoyAI harvest, so
|
||||
# cross-shot memory banks the improved frames. Every skip is LOUD (console +
|
||||
# scene report): with the toggle on, silence must mean "ran and found nothing".
|
||||
if segmented_detailing and detail_strength > 0 and not _detail_disabled_reason:
|
||||
try:
|
||||
try:
|
||||
from . import detailing as _detailing
|
||||
except ImportError:
|
||||
import detailing as _detailing
|
||||
if _detail_upsampler_model is None:
|
||||
_resolved = _detailing.resolve_upsampler_name(detail_upsampler)
|
||||
_detail_upsampler_model = _detailing.load_latent_upsampler(_resolved)
|
||||
print(f"[FunPackSceneChain] segmented detailing upsampler: {_resolved}")
|
||||
_t_detail0 = _time.perf_counter()
|
||||
sampled, _detail_note = _detailing.detail_refine_scene(
|
||||
self, model, vae, sampler, scene_positive, scene_negative, sampled,
|
||||
detail_targets, _detail_upsampler_model, scene_seed, cfg,
|
||||
threshold=detail_threshold, strength=detail_strength,
|
||||
area_cap=detail_max_area, renoise_sigma=detail_denoise,
|
||||
mode=detail_mode, debug=debug_log)
|
||||
# detail_refine_scene always returns a diagnostic note when it actually
|
||||
# ran detection (miss or hit) — None only means it never attempted
|
||||
# detection at all (which can't happen on this branch).
|
||||
if _detail_note:
|
||||
run_mechanisms.append(_detail_note)
|
||||
_phase_sampling += _time.perf_counter() - _t_detail0
|
||||
except Exception as _detail_exc:
|
||||
# A failed detail pass must never cost the scene itself — but it must
|
||||
# never fail silently either. Model resolution/load failures disable
|
||||
# the pass for the rest of the run (no per-scene download retries).
|
||||
if _detail_upsampler_model is None:
|
||||
_detail_disabled_reason = str(_detail_exc)
|
||||
print(f"[FunPackSceneChain] SEGMENTED DETAILING SKIPPED (scene kept as-is): {_detail_exc}")
|
||||
run_mechanisms.append(f"segmented_detail(SKIPPED: {_detail_exc})")
|
||||
elif segmented_detailing and _detail_disabled_reason:
|
||||
run_mechanisms.append("segmented_detail(SKIPPED: upsampler unavailable, see first scene)")
|
||||
if joyai_bank is not None:
|
||||
# Harvest from the clean, fully-cropped scene so injected memory tails never re-enter
|
||||
# the bank. Scene 0 seeds the pinned anchor (num_fix); later scenes roll in. The audio
|
||||
|
||||
@@ -0,0 +1,456 @@
|
||||
"""Unit tests for segmented detailing (detailing.py).
|
||||
|
||||
Covers the pure pieces (target parsing, tube geometry + area cap, layout-cond
|
||||
stripping, spatial downscale) and the detail_refine_scene contract: bit-identical
|
||||
no-op when disabled / nothing detected, and the crop -> refine -> feathered
|
||||
paste round trip with the model-touching stages stubbed out.
|
||||
"""
|
||||
import sys
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import torch
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
|
||||
sys.modules.setdefault("comfy", types.ModuleType("comfy"))
|
||||
_nested = types.ModuleType("comfy.nested_tensor")
|
||||
|
||||
|
||||
class _FakeNested:
|
||||
is_nested = True
|
||||
|
||||
def __init__(self, tensors):
|
||||
self._tensors = list(tensors)
|
||||
|
||||
def unbind(self):
|
||||
return list(self._tensors)
|
||||
|
||||
|
||||
_nested.NestedTensor = _FakeNested
|
||||
sys.modules.setdefault("comfy.nested_tensor", _nested)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _nested_stub(monkeypatch):
|
||||
"""Re-install the usable NestedTensor stub around every test.
|
||||
|
||||
Other test modules stub comfy.nested_tensor.NestedTensor = object on the SAME
|
||||
shared sys.modules entry at collection time (whichever file is collected last
|
||||
wins), and stubbed submodules never run real import machinery, so the parent
|
||||
attribute must be bound by hand too."""
|
||||
monkeypatch.setattr(sys.modules["comfy.nested_tensor"], "NestedTensor",
|
||||
_FakeNested, raising=False)
|
||||
monkeypatch.setattr(sys.modules["comfy"], "nested_tensor",
|
||||
sys.modules["comfy.nested_tensor"], raising=False)
|
||||
|
||||
import detailing # noqa: E402
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# parse_targets
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_parse_targets_splits_and_strips():
|
||||
assert detailing.parse_targets(" hands , feet,,") == ["hands", "feet"]
|
||||
assert detailing.parse_targets("") == []
|
||||
assert detailing.parse_targets(None) == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# stage2_sigmas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_stage2_sigmas_default_matches_official_recipe():
|
||||
# Must reproduce the exact official two-stage workflow values unless the
|
||||
# user raises/lowers detail_denoise — no behavior change for existing runs.
|
||||
assert torch.allclose(torch.tensor(detailing.stage2_sigmas(0.85)),
|
||||
torch.tensor(detailing.STAGE2_SIGMAS))
|
||||
assert detailing.stage2_sigmas(0.85)[-1] == 0.0
|
||||
|
||||
def test_stage2_sigmas_scales_with_renoise_sigma():
|
||||
# Raising the re-entry point should raise every step in the tail proportionally
|
||||
# (more genuine freedom to reconstruct, not just a bigger first jump).
|
||||
low = detailing.stage2_sigmas(0.5)
|
||||
high = detailing.stage2_sigmas(0.95)
|
||||
assert high[0] > low[0] > 0
|
||||
assert high[1] > low[1]
|
||||
assert low[-1] == high[-1] == 0.0
|
||||
# Shape (ratios between steps) preserved regardless of entry point.
|
||||
assert abs(low[1] / low[0] - high[1] / high[0]) < 1e-9
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# find_tube
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _heat_with_hot_square(h=64, w=64, y=(20, 30), x=(40, 50), value=0.9):
|
||||
heat = torch.zeros(h, w)
|
||||
heat[y[0]:y[1], x[0]:x[1]] = value
|
||||
return heat
|
||||
|
||||
def test_find_tube_boxes_hot_region_with_padding():
|
||||
heat = _heat_with_hot_square()
|
||||
tube = detailing.find_tube(heat, 64, 64, threshold=0.35)
|
||||
assert tube is not None
|
||||
y0, y1, x0, x1, mask = tube
|
||||
# Hot square plus TUBE_PAD on each side (interpolation is identity at same size).
|
||||
assert y0 <= 20 - detailing.TUBE_PAD + 1 and y1 >= 30 + detailing.TUBE_PAD - 1
|
||||
assert x0 <= 40 - detailing.TUBE_PAD + 1 and x1 >= 50 + detailing.TUBE_PAD - 1
|
||||
assert mask.shape == (1, 1, 1, y1 - y0, x1 - x0)
|
||||
assert float(mask.max()) <= 1.0 and float(mask.min()) >= 0.0
|
||||
# Mask peaks inside the hot region, feathers to ~0 at the padded border.
|
||||
assert float(mask[..., (y1 - y0) // 2, (x1 - x0) // 2]) > 0.5
|
||||
assert float(mask[..., 0, 0]) < 0.2
|
||||
|
||||
def test_find_tube_none_below_threshold():
|
||||
heat = torch.full((64, 64), 0.1)
|
||||
assert detailing.find_tube(heat, 64, 64, threshold=0.35) is None
|
||||
assert detailing.find_tube(None, 64, 64) is None
|
||||
|
||||
def test_find_tube_refuses_oversized_region():
|
||||
heat = torch.full((64, 64), 0.9) # whole frame hot -> a re-render, not a detail pass
|
||||
assert detailing.find_tube(heat, 64, 64, threshold=0.35) is None
|
||||
|
||||
def test_find_tube_diag_reports_max_heat_and_reason():
|
||||
diag = {}
|
||||
heat = torch.full((64, 64), 0.1)
|
||||
assert detailing.find_tube(heat, 64, 64, threshold=0.35, diag=diag) is None
|
||||
assert diag["reason"] == "below_threshold"
|
||||
assert abs(diag["max_heat"] - 0.1) < 1e-6
|
||||
|
||||
diag2 = {}
|
||||
heat2 = torch.full((64, 64), 0.9) # whole frame -> area cap, not threshold
|
||||
assert detailing.find_tube(heat2, 64, 64, threshold=0.35, diag=diag2) is None
|
||||
assert diag2["reason"] == "area_cap"
|
||||
assert diag2["area"] > detailing.MAX_TUBE_AREA
|
||||
|
||||
def test_find_tube_enforces_minimum_edge():
|
||||
heat = _heat_with_hot_square(y=(31, 32), x=(31, 32)) # single hot cell
|
||||
tube = detailing.find_tube(heat, 64, 64, threshold=0.35)
|
||||
assert tube is not None
|
||||
y0, y1, x0, x1, _ = tube
|
||||
assert (y1 - y0) >= detailing.MIN_TUBE_EDGE
|
||||
assert (x1 - x0) >= detailing.MIN_TUBE_EDGE
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _clipseg_heat
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_clipseg_heat_passes_padding_and_truncation(monkeypatch):
|
||||
"""Regression: a multi-prompt call without padding=True crashes CLIPSeg's
|
||||
tokenizer ("excessive nesting... type list where int is expected") the moment
|
||||
detail_targets has more than one entry — caught live on a real rental run.
|
||||
truncation=True is the same call, cheap to require alongside it."""
|
||||
calls = []
|
||||
|
||||
class _FakeInputs(dict):
|
||||
def to(self, *a, **k):
|
||||
return self
|
||||
|
||||
class _FakeProcessor:
|
||||
def __call__(self, text, images, return_tensors, **kwargs):
|
||||
calls.append(kwargs)
|
||||
return _FakeInputs()
|
||||
|
||||
class _FakeModel:
|
||||
def __call__(self, **inputs):
|
||||
n = 2
|
||||
return types.SimpleNamespace(logits=torch.zeros(n, 8, 8))
|
||||
|
||||
def eval(self):
|
||||
return self
|
||||
|
||||
monkeypatch.setattr(detailing, "_get_clipseg", lambda: (_FakeProcessor(), _FakeModel()))
|
||||
frames = [(0, torch.zeros(16, 16, 3))]
|
||||
detailing._clipseg_heat(frames, ["hands", "feet"])
|
||||
assert calls and calls[0].get("padding") is True
|
||||
assert calls[0].get("truncation") is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# resolve_upsampler_name
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _patch_folder_paths(monkeypatch, files, folder="/models/latent_upscale_models"):
|
||||
fp = sys.modules["folder_paths"]
|
||||
monkeypatch.setattr(fp, "get_filename_list", lambda kind: list(files), raising=False)
|
||||
monkeypatch.setattr(fp, "get_folder_paths", lambda kind: [folder], raising=False)
|
||||
|
||||
def test_resolve_explicit_name_passes_through(monkeypatch):
|
||||
_patch_folder_paths(monkeypatch, [])
|
||||
assert detailing.resolve_upsampler_name("my_upscaler.safetensors") == "my_upscaler.safetensors"
|
||||
|
||||
def test_resolve_auto_prefers_newest_spatial_upscaler(monkeypatch):
|
||||
_patch_folder_paths(monkeypatch, [
|
||||
"other_model.safetensors",
|
||||
"ltx-2.3-spatial-upscaler-x2-1.0.safetensors",
|
||||
"ltx-2.3-spatial-upscaler-x2-1.1.safetensors",
|
||||
])
|
||||
for alias in ("auto", "None", "", None):
|
||||
assert detailing.resolve_upsampler_name(alias) == "ltx-2.3-spatial-upscaler-x2-1.1.safetensors"
|
||||
|
||||
def test_resolve_auto_falls_back_to_any_installed(monkeypatch):
|
||||
_patch_folder_paths(monkeypatch, ["something_else.safetensors"])
|
||||
assert detailing.resolve_upsampler_name("auto") == "something_else.safetensors"
|
||||
|
||||
def test_resolve_auto_downloads_when_folder_empty(monkeypatch, tmp_path):
|
||||
_patch_folder_paths(monkeypatch, [], folder=str(tmp_path / "lup"))
|
||||
calls = {}
|
||||
hub = types.ModuleType("huggingface_hub")
|
||||
hub.hf_hub_download = lambda repo_id, filename, local_dir: calls.update(
|
||||
repo=repo_id, file=filename, dest=local_dir)
|
||||
monkeypatch.setitem(sys.modules, "huggingface_hub", hub)
|
||||
name = detailing.resolve_upsampler_name("auto")
|
||||
assert name == detailing.DEFAULT_UPSAMPLER_FILE
|
||||
assert calls["repo"] == detailing.DEFAULT_UPSAMPLER_REPO
|
||||
assert calls["file"] == detailing.DEFAULT_UPSAMPLER_FILE
|
||||
|
||||
def test_resolve_download_failure_raises_actionable_error(monkeypatch, tmp_path):
|
||||
_patch_folder_paths(monkeypatch, [], folder=str(tmp_path / "lup"))
|
||||
hub = types.ModuleType("huggingface_hub")
|
||||
def _boom(**kw):
|
||||
raise OSError("401 gated")
|
||||
hub.hf_hub_download = _boom
|
||||
monkeypatch.setitem(sys.modules, "huggingface_hub", hub)
|
||||
with pytest.raises(RuntimeError, match="manually"):
|
||||
detailing.resolve_upsampler_name("auto")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _strip_layout_conds / _downscale_to
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_strip_layout_conds_removes_guide_keys_only():
|
||||
cond = [[torch.zeros(1, 4), {"keyframe_idxs": "kf", "guiding_latent": "g",
|
||||
"pooled_output": "keep"}]]
|
||||
out = detailing._strip_layout_conds(cond)
|
||||
assert out[0][1] == {"pooled_output": "keep"}
|
||||
# Original untouched (the scene's own conds must not lose their guides).
|
||||
assert "keyframe_idxs" in cond[0][1]
|
||||
|
||||
def test_downscale_to_shape_and_content():
|
||||
video = torch.arange(2 * 3 * 4 * 8 * 8, dtype=torch.float32).reshape(2, 3, 4, 8, 8)
|
||||
down = detailing._downscale_to(video, 4, 4)
|
||||
assert down.shape == (2, 3, 4, 4, 4)
|
||||
# Area downscale of a constant tensor is exact.
|
||||
const = torch.full((1, 2, 2, 6, 6), 3.5)
|
||||
assert torch.allclose(detailing._downscale_to(const, 3, 3), torch.full((1, 2, 2, 3, 3), 3.5))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# detail_refine_scene
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class _FakeChain:
|
||||
"""Just enough of FunPackLTXAVSceneChainSampler for detail_refine_scene."""
|
||||
|
||||
def __init__(self):
|
||||
self.chunk_calls = []
|
||||
|
||||
def _latent_tensors(self, latent):
|
||||
samples = latent["samples"]
|
||||
if getattr(samples, "is_nested", False):
|
||||
return list(samples.unbind())
|
||||
return [samples]
|
||||
|
||||
def _sample_chunk(self, model, sampler, sigmas, seed, cfg, positive, negative, latent,
|
||||
**kwargs):
|
||||
self.chunk_calls.append({"sigmas": sigmas, "seed": seed,
|
||||
"positive": positive, "latent": latent})
|
||||
return latent # identity "refine": returns the crop unchanged
|
||||
|
||||
|
||||
def _scene_latent(nested=False):
|
||||
video = torch.randn(1, 8, 5, 16, 24)
|
||||
if not nested:
|
||||
return {"samples": video}, video
|
||||
audio = torch.randn(1, 4, 40)
|
||||
return {"samples": _FakeNested([video, audio])}, video
|
||||
|
||||
def test_refine_noop_without_targets_or_upsampler():
|
||||
chain = _FakeChain()
|
||||
latent, _ = _scene_latent()
|
||||
out, note = detailing.detail_refine_scene(
|
||||
chain, None, None, None, [], [], latent, "", object(), 0, 1.0)
|
||||
assert out is latent and note is None # same object: bit-identical guarantee
|
||||
out, note = detailing.detail_refine_scene(
|
||||
chain, None, None, None, [], [], latent, "hands", None, 0, 1.0)
|
||||
assert out is latent and note is None
|
||||
assert chain.chunk_calls == []
|
||||
|
||||
def test_refine_noop_when_nothing_decodable(monkeypatch):
|
||||
"""No keyframe could be decoded: same-object no-op, but WITH a diagnostic note —
|
||||
a miss must never look identical to "the feature is off" once it actually ran."""
|
||||
chain = _FakeChain()
|
||||
latent, _ = _scene_latent()
|
||||
monkeypatch.setattr(detailing, "_decode_detection_frames", lambda *a, **k: [])
|
||||
monkeypatch.setattr(detailing, "_clipseg_heat", lambda *a, **k: None)
|
||||
out, note = detailing.detail_refine_scene(
|
||||
chain, None, None, None, [], [], latent, "hands", object(), 0, 1.0)
|
||||
assert out is latent
|
||||
assert note is not None and "no match" in note
|
||||
assert chain.chunk_calls == []
|
||||
|
||||
def test_refine_reports_max_heat_when_below_threshold(monkeypatch):
|
||||
"""Regression: a real live run hit CLIPSeg-found-nothing with zero visibility into
|
||||
WHY (was it a bad prompt, or just a threshold too strict for CLIPSeg's raw score?).
|
||||
The note must carry the actual max score so the fix (lower detail_threshold) is
|
||||
obvious without re-running with debug_log."""
|
||||
chain = _FakeChain()
|
||||
latent, _ = _scene_latent()
|
||||
monkeypatch.setattr(detailing, "_decode_detection_frames",
|
||||
lambda *a, **k: [(0, torch.zeros(16, 16, 3))])
|
||||
monkeypatch.setattr(detailing, "_clipseg_heat", lambda *a, **k: torch.full((16, 16), 0.18))
|
||||
out, note = detailing.detail_refine_scene(
|
||||
chain, None, None, None, [], [], latent, "hands", object(), 0, 1.0, threshold=0.35)
|
||||
assert out is latent
|
||||
assert "0.18" in note and "0.35" in note
|
||||
assert chain.chunk_calls == []
|
||||
|
||||
def test_refine_reports_area_cap_reason(monkeypatch):
|
||||
chain = _FakeChain()
|
||||
latent, _ = _scene_latent()
|
||||
monkeypatch.setattr(detailing, "_decode_detection_frames",
|
||||
lambda *a, **k: [(0, torch.zeros(160, 240, 3))])
|
||||
monkeypatch.setattr(detailing, "_clipseg_heat", lambda *a, **k: torch.full((160, 240), 0.9))
|
||||
out, note = detailing.detail_refine_scene(
|
||||
chain, None, None, None, [], [], latent, "hands", object(), 0, 1.0)
|
||||
assert out is latent
|
||||
assert "cap" in note.lower()
|
||||
assert chain.chunk_calls == []
|
||||
|
||||
def test_refine_area_cap_is_a_cost_knob_not_a_judgment(monkeypatch):
|
||||
"""The cap must never override what the user explicitly asked to detail — it's a
|
||||
cost guardrail the user can raise, not FunPack deciding a big region 'isn't worth
|
||||
detailing'. Same whole-frame region: refused at the default cap, allowed once the
|
||||
user raises detail_max_area (mirrors area_cap, threaded from the sampler node)."""
|
||||
chain = _FakeChain()
|
||||
latent, video = _scene_latent()
|
||||
monkeypatch.setattr(detailing, "_decode_detection_frames",
|
||||
lambda *a, **k: [(0, torch.zeros(160, 240, 3))])
|
||||
monkeypatch.setattr(detailing, "_clipseg_heat", lambda *a, **k: torch.full((160, 240), 0.9))
|
||||
monkeypatch.setattr(
|
||||
detailing, "_run_upsampler",
|
||||
lambda upsampler, crop, vae, debug=False: torch.nn.functional.interpolate(
|
||||
crop, scale_factor=(1, 2, 2), mode="nearest"))
|
||||
out_default, note_default = detailing.detail_refine_scene(
|
||||
chain, "m", "v", "s", [], [], latent, "hands", object(), 0, 1.0)
|
||||
assert out_default is latent and "cap" in note_default.lower()
|
||||
out_raised, note_raised = detailing.detail_refine_scene(
|
||||
chain, "m", "v", "s", [], [], latent, "hands", object(), 0, 1.0, area_cap=1.0)
|
||||
assert note_raised is not None and "segmented_detail(" in note_raised and "cap" not in note_raised.lower()
|
||||
assert out_raised is not latent
|
||||
|
||||
def _patch_detection_and_upsampler(monkeypatch):
|
||||
"""Hot square in pixel space; upsampler = exact 2x nearest (model-free)."""
|
||||
heat = torch.zeros(160, 240)
|
||||
heat[40:80, 120:180] = 0.9
|
||||
monkeypatch.setattr(detailing, "_decode_detection_frames",
|
||||
lambda *a, **k: [(0, torch.zeros(160, 240, 3))])
|
||||
monkeypatch.setattr(detailing, "_clipseg_heat", lambda *a, **k: heat)
|
||||
monkeypatch.setattr(
|
||||
detailing, "_run_upsampler",
|
||||
lambda upsampler, crop, vae, debug=False: torch.nn.functional.interpolate(
|
||||
crop, scale_factor=(1, 2, 2), mode="nearest"))
|
||||
return heat
|
||||
|
||||
def test_refine_passes_renoise_sigma_to_the_tail_schedule(monkeypatch):
|
||||
chain = _FakeChain()
|
||||
latent, _ = _scene_latent()
|
||||
_patch_detection_and_upsampler(monkeypatch)
|
||||
detailing.detail_refine_scene(
|
||||
chain, "model", "vae", "sampler", [], [], latent, "hands", object(),
|
||||
seed=0, cfg=1.0, renoise_sigma=0.95)
|
||||
sigmas = chain.chunk_calls[0]["sigmas"]
|
||||
assert abs(float(sigmas[0]) - 0.95) < 1e-6
|
||||
assert torch.allclose(sigmas, torch.tensor(detailing.stage2_sigmas(0.95)))
|
||||
|
||||
def test_refine_sharpen_mode_skips_the_diffusion_tail_entirely(monkeypatch):
|
||||
"""The whole point of sharpen mode: zero video-model forwards. It must use the
|
||||
upsampler's own output directly, not merely a fast-looking diffusion call."""
|
||||
chain = _FakeChain()
|
||||
latent, video = _scene_latent()
|
||||
_patch_detection_and_upsampler(monkeypatch)
|
||||
out, note = detailing.detail_refine_scene(
|
||||
chain, "model", "vae", "sampler", [], [], latent, "hands", object(),
|
||||
seed=0, cfg=1.0, mode="sharpen")
|
||||
assert note is not None and "sharpen" in note
|
||||
assert chain.chunk_calls == [] # no _sample_chunk call at all
|
||||
assert out["samples"].shape == video.shape
|
||||
diff = (out["samples"] - video).abs().amax(dim=(0, 1, 2))
|
||||
assert not diff[:2].any() and not diff[:, :2].any() # far corners still untouched
|
||||
|
||||
def test_refine_repair_mode_is_still_the_default(monkeypatch):
|
||||
chain = _FakeChain()
|
||||
latent, _ = _scene_latent()
|
||||
_patch_detection_and_upsampler(monkeypatch)
|
||||
_, note = detailing.detail_refine_scene(
|
||||
chain, "model", "vae", "sampler", [], [], latent, "hands", object(), 0, 1.0)
|
||||
assert "repair" in note
|
||||
assert len(chain.chunk_calls) == 1
|
||||
|
||||
def test_refine_round_trip_touches_only_the_tube(monkeypatch):
|
||||
chain = _FakeChain()
|
||||
latent, video = _scene_latent()
|
||||
_patch_detection_and_upsampler(monkeypatch)
|
||||
pos = [[torch.zeros(1, 4), {"keyframe_idxs": "kf", "pooled_output": "keep"}]]
|
||||
out, note = detailing.detail_refine_scene(
|
||||
chain, "model", "vae", "sampler", pos, [], latent, "hands", object(),
|
||||
seed=11, cfg=1.0)
|
||||
assert note is not None and "segmented_detail" in note
|
||||
assert len(chain.chunk_calls) == 1
|
||||
call = chain.chunk_calls[0]
|
||||
# Official stage-2 tail schedule and a detail-offset seed.
|
||||
assert torch.allclose(call["sigmas"], torch.tensor(detailing.STAGE2_SIGMAS))
|
||||
assert call["seed"] == 11 + 7777
|
||||
# Layout conds stripped for the crop pass.
|
||||
assert "keyframe_idxs" not in call["positive"][0][1]
|
||||
assert call["positive"][0][1]["pooled_output"] == "keep"
|
||||
# The crop passed to the refine is 2x the tube size.
|
||||
crop = call["latent"]["samples"]
|
||||
out_video = out["samples"]
|
||||
assert out_video.shape == video.shape
|
||||
# With an identity refine, upsample(2x)->downscale(area) reproduces the crop, so
|
||||
# the paste changes (nearly) nothing — but crucially everything OUTSIDE the tube
|
||||
# must be exactly untouched regardless of what the refine did.
|
||||
diff = (out_video - video).abs().amax(dim=(0, 1, 2)) # [H, W]
|
||||
hot = diff > 1e-6
|
||||
assert not hot[:2].any() and not hot[:, :2].any() # far corners untouched
|
||||
|
||||
def test_refine_preserves_audio_stream(monkeypatch):
|
||||
chain = _FakeChain()
|
||||
latent, video = _scene_latent(nested=True)
|
||||
audio = chain._latent_tensors(latent)[1]
|
||||
_patch_detection_and_upsampler(monkeypatch)
|
||||
out, note = detailing.detail_refine_scene(
|
||||
chain, "model", "vae", "sampler", [], [], latent, "hands", object(),
|
||||
seed=0, cfg=1.0)
|
||||
assert note is not None
|
||||
out_tensors = chain._latent_tensors(out)
|
||||
assert len(out_tensors) == 2
|
||||
# Audio comes back as the SAME tensor the scene finished with: protected by
|
||||
# construction, not by a lossy round trip.
|
||||
assert out_tensors[1] is audio
|
||||
# The crop pass carried an audio stream so joint attention saw a legal AV latent.
|
||||
crop_samples = chain.chunk_calls[0]["latent"]["samples"]
|
||||
assert getattr(crop_samples, "is_nested", False)
|
||||
|
||||
def test_refine_strength_scales_paste(monkeypatch):
|
||||
chain = _FakeChain()
|
||||
latent, video = _scene_latent()
|
||||
_patch_detection_and_upsampler(monkeypatch)
|
||||
# Non-identity refine: chunk returns a shifted crop so the paste has signal.
|
||||
def _shifted_chunk(model, sampler, sigmas, seed, cfg, positive, negative, crop_latent, **kw):
|
||||
return {"samples": crop_latent["samples"] + 10.0}
|
||||
chain._sample_chunk = _shifted_chunk
|
||||
out_full, _ = detailing.detail_refine_scene(
|
||||
chain, "m", "v", "s", [], [], latent, "hands", object(), 0, 1.0, strength=1.0)
|
||||
out_half, _ = detailing.detail_refine_scene(
|
||||
chain, "m", "v", "s", [], [], latent, "hands", object(), 0, 1.0, strength=0.5)
|
||||
d_full = (out_full["samples"] - video).abs().sum()
|
||||
d_half = (out_half["samples"] - video).abs().sum()
|
||||
assert d_full > 0
|
||||
assert abs(float(d_half / d_full) - 0.5) < 1e-3
|
||||
@@ -94,7 +94,7 @@ def test_wrapper_rolls_in_unrolls_out():
|
||||
assert torch.equal(seen["input"][..., : video.numel()].reshape(VSHAPE), torch.roll(video, 4, dims=2))
|
||||
|
||||
|
||||
def test_wrapper_gates_plateau_guides_and_tiny_clips():
|
||||
def test_wrapper_gates_plateau_unparseable_guides_and_tiny_clips():
|
||||
video, _, packed = _packed()
|
||||
seen = {}
|
||||
|
||||
@@ -105,8 +105,10 @@ def test_wrapper_gates_plateau_guides_and_tiny_clips():
|
||||
w = make_loop_temporal_wrapper(None)
|
||||
w(fake_apply, _args(0.99, packed)) # near-noise plateau
|
||||
assert torch.equal(seen["input"], packed)
|
||||
# malformed keyframe_idxs (too few dims to count tokens): canonical, never a blind roll
|
||||
w(fake_apply, _args(0.725, packed, extra_c={"keyframe_idxs": torch.zeros(1, 1)}))
|
||||
assert torch.equal(seen["input"], packed)
|
||||
# attention entries without keyframe_idxs: guide context we can't size -> canonical
|
||||
w(fake_apply, _args(0.725, packed, extra_c={"guide_attention_entries": [object()]}))
|
||||
assert torch.equal(seen["input"], packed)
|
||||
|
||||
@@ -118,6 +120,76 @@ def test_wrapper_gates_plateau_guides_and_tiny_clips():
|
||||
assert torch.equal(seen["input"], tiny)
|
||||
|
||||
|
||||
def _keyframe_idxs_for(n_frames, h=4, w=6):
|
||||
# per-token layout: [B, coords, n_tokens, start/end] — the wrapper only reads shape[2]
|
||||
return torch.zeros(1, 1, n_frames * h * w, 2)
|
||||
|
||||
|
||||
def test_wrapper_content_only_roll_with_guide_tail():
|
||||
"""Guide frames appended at the video tail stay pinned; only content rolls."""
|
||||
v_tail = 2
|
||||
video, audio, packed = _packed() # video T=16 -> content 14; audio T=32, no tail
|
||||
mask = torch.rand(1, 1, 16, 4, 6)
|
||||
seen = {}
|
||||
|
||||
def fake_apply(x, ts, **c):
|
||||
seen["input"], seen["mask"] = x, c.get("denoise_mask")
|
||||
return x * 1.0
|
||||
|
||||
w = make_loop_temporal_wrapper(None)
|
||||
out = w(fake_apply, _args(0.909, packed, mask,
|
||||
extra_c={"keyframe_idxs": _keyframe_idxs_for(v_tail)}))
|
||||
v_in = seen["input"][..., : video.numel()].reshape(VSHAPE)
|
||||
# first eligible step: frac 0.5 of the 14 content frames -> shift 7
|
||||
assert torch.equal(v_in[:, :, :14], torch.roll(video[:, :, :14], 7, dims=2))
|
||||
assert torch.equal(v_in[:, :, 14:], video[:, :, 14:]) # guide tail untouched
|
||||
assert torch.equal(seen["mask"][:, :, :14], torch.roll(mask[:, :, :14], 7, dims=2))
|
||||
assert torch.equal(seen["mask"][:, :, 14:], mask[:, :, 14:]) # tail mask untouched
|
||||
# audio has no tail: rolls fully by its own content length (0.5 * 32 = 16)
|
||||
a_in = seen["input"][..., video.numel():].reshape(ASHAPE)
|
||||
assert torch.equal(a_in, torch.roll(audio, 16, dims=2))
|
||||
assert torch.equal(out, packed) # unrolled on the way out
|
||||
|
||||
|
||||
def test_wrapper_audio_memory_tail_stays_pinned():
|
||||
"""A trailing mask=0 run on the audio stream (JoyAI memory) is treated as pinned."""
|
||||
a_tail = 4
|
||||
video, audio, packed = _packed()
|
||||
amask = torch.ones(1, 1, 32, 16)
|
||||
amask[:, :, -a_tail:] = 0.0
|
||||
seen = {}
|
||||
|
||||
def fake_apply(x, ts, **c):
|
||||
seen["input"], seen["amask"] = x, c.get("audio_denoise_mask")
|
||||
return x * 1.0
|
||||
|
||||
w = make_loop_temporal_wrapper(None)
|
||||
out = w(fake_apply, _args(0.909, packed, extra_c={"audio_denoise_mask": amask}))
|
||||
a_in = seen["input"][..., video.numel():].reshape(ASHAPE)
|
||||
# audio content 28 frames -> frac 0.5 -> shift 14; tail canonical
|
||||
assert torch.equal(a_in[:, :, :28], torch.roll(audio[:, :, :28], 14, dims=2))
|
||||
assert torch.equal(a_in[:, :, 28:], audio[:, :, 28:])
|
||||
assert torch.equal(seen["amask"], amask) # all-ones content rolls invisibly; zeros pinned
|
||||
assert torch.equal(out, packed)
|
||||
|
||||
|
||||
def test_wrapper_guide_tail_reduces_content_gate():
|
||||
"""A clip whose content region (T - tail) is under the minimum stays canonical."""
|
||||
shape = (1, 128, 5, 4, 6)
|
||||
x = torch.rand(1, 1, math.prod(shape[1:]))
|
||||
seen = {}
|
||||
|
||||
def fake_apply(x_, ts, **c):
|
||||
seen["input"] = x_
|
||||
return x_ * 1.0
|
||||
|
||||
w = make_loop_temporal_wrapper(None)
|
||||
w(fake_apply, {"input": x, "timestep": torch.tensor([0.7]),
|
||||
"c": {"latent_shapes": [torch.Size(shape)],
|
||||
"keyframe_idxs": _keyframe_idxs_for(2)}}) # content 3 < 4
|
||||
assert torch.equal(seen["input"], x)
|
||||
|
||||
|
||||
def test_wrapper_resets_shift_sequence_on_new_scene():
|
||||
video, _, packed = _packed()
|
||||
seen = {}
|
||||
|
||||
@@ -8,7 +8,7 @@ const LORA_TYPES = ["general", "action", "style", "quality", "character"];
|
||||
const ADVISOR_DTYPES = ["bfloat16", "float16", "float32"];
|
||||
const REFINER_MODES = ["Refine", "Prompt only", "Learning"];
|
||||
const ADVISOR_MODES = ["Off", "Only diagnostics", "Only prompt", "Full"];
|
||||
const TEMPORAL_STYLES = ["natural", "auto", "accelerate", "decelerate", "loop", "freeze", "pulse"];
|
||||
const TEMPORAL_STYLES = ["natural", "auto", "accelerate", "decelerate", "loop", "freeze", "pulse", "rapid_start", "rapid_end", "rapid_start_end"];
|
||||
const CATEGORY_ORDER = ["action", "camera", "subject", "appearance", "environment", "style", "quality", "details"];
|
||||
const TABS = ["Session", "Shortcuts", "Split markers", "Refiner", "Advisor", "LoRA", "Sampler", "Adjustments", "Timeline"];
|
||||
const SAMPLER_TYPES = ["Hybrid Euler 2S", "Distilled Flow", "KSampler"];
|
||||
|
||||
Reference in New Issue
Block a user