[refactor] use existing workflow namespace for InterleaveThinker
This commit is contained in:
@@ -8,7 +8,7 @@ Current working location:
|
||||
|
||||
- Directory: `/home/toolbox/FastVideo`
|
||||
- Branch: `interleavethinker-fastvideo`
|
||||
- Latest observed branch head: `91d8fb85` (`[misc] format InterleaveThinker workflow helpers`)
|
||||
- Latest observed branch head: `58256b12` (`[docs] record InterleaveThinker workflow migration`)
|
||||
|
||||
This file is the canonical handoff for the InterleaveThinker integration work.
|
||||
It intentionally summarizes older execution logs; use git history for the full
|
||||
@@ -48,8 +48,9 @@ append-only detail if needed.
|
||||
- RL algorithms live in `fastvideo/train/methods/rl`.
|
||||
- Reward parsing/scoring lives under `fastvideo/train/methods/rl/rewards`.
|
||||
- Interleaved inference helpers live under
|
||||
`fastvideo/workflows/interleave_thinker`; the old entrypoints namespace was
|
||||
removed because the standalone CLI/API surface was removed.
|
||||
`fastvideo/workflow/interleave_thinker`; use the pre-existing singular
|
||||
`fastvideo/workflow` namespace, not a parallel `fastvideo/workflows`
|
||||
package.
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -74,7 +75,7 @@ Out of scope unless explicitly re-opened:
|
||||
|
||||
Implemented and retained surfaces:
|
||||
|
||||
- `fastvideo.workflows.interleave_thinker`
|
||||
- `fastvideo.workflow.interleave_thinker`
|
||||
- schema objects, generator backend translation, orchestrator, provider
|
||||
adapters, config/runner helpers, prompt-set evaluation, and trace metrics.
|
||||
- Standalone command registration and standalone server modules were removed.
|
||||
@@ -107,12 +108,14 @@ Removed by the API/CLI cleanup:
|
||||
|
||||
Namespace integration status:
|
||||
|
||||
- Completed. The reusable helper layer moved from the old entrypoints package
|
||||
to `fastvideo/workflows/interleave_thinker`.
|
||||
- Internal imports, tests, examples, and docs now use the workflow namespace.
|
||||
- The old `fastvideo.entrypoints.interleave` package was deleted rather than
|
||||
kept as a compatibility shim. This branch has not merged, so preserving the
|
||||
old public path was not required.
|
||||
- Correction in progress. The reusable helper layer should live under the
|
||||
pre-existing singular `fastvideo/workflow/interleave_thinker` package, not
|
||||
under a new parallel `fastvideo/workflows` package.
|
||||
- Internal imports, tests, examples, docs, and this handoff should use
|
||||
`fastvideo.workflow.interleave_thinker`.
|
||||
- The old `fastvideo.entrypoints.interleave` package remains deleted rather
|
||||
than kept as a compatibility shim. This branch has not merged, so preserving
|
||||
the old public path is not required.
|
||||
- Do not scatter the helper code into unrelated core modules unless a genuinely
|
||||
generic abstraction emerges. The planner -> generator/edit -> critic loop is
|
||||
InterleaveThinker-specific workflow code, not `VideoGenerator`, training
|
||||
@@ -163,10 +166,13 @@ Namespace integration status:
|
||||
library/training integration. Commits: `555a600f`, `704e5667`.
|
||||
- Handoff instructions were condensed and updated with standing Modal approval
|
||||
and the no-local-tests rule. Commit: `d2c53951`.
|
||||
- Namespace integration moved the helper package to
|
||||
- Namespace integration first moved the helper package to
|
||||
`fastvideo.workflows.interleave_thinker`, renamed stale tests, updated docs
|
||||
and examples, and deleted the old entrypoints package. Commits: `11d55fb5`,
|
||||
`91d8fb85`.
|
||||
- Follow-up correction requested by the user: move the helper package into the
|
||||
pre-existing singular `fastvideo.workflow.interleave_thinker` namespace and
|
||||
remove the parallel `fastvideo.workflows` package.
|
||||
|
||||
## Validation Evidence
|
||||
|
||||
@@ -269,9 +275,9 @@ Broad-suite status:
|
||||
before editing.
|
||||
2. Read the relevant per-directory `AGENTS.md` before touching files under
|
||||
`fastvideo/`, `examples/`, `docs/`, `scripts/`, or tests.
|
||||
3. The API cleanup and namespace migration are complete. Continue with review
|
||||
preparation, PR decomposition, reward-backend hardening, or broad-suite
|
||||
environment repair.
|
||||
3. Complete the namespace correction from `fastvideo.workflows` to the existing
|
||||
`fastvideo.workflow` package, then continue with review preparation, PR
|
||||
decomposition, reward-backend hardening, or broad-suite environment repair.
|
||||
4. Validate only on Modal. Local syntax-only commands such as `git diff --check`
|
||||
are acceptable, but no local pytest or other local test execution should be
|
||||
used.
|
||||
|
||||
@@ -44,7 +44,7 @@ Training examples live under:
|
||||
|
||||
### Orchestration Helpers
|
||||
|
||||
The Python helper layer under `fastvideo.workflows.interleave_thinker` is intentionally
|
||||
The Python helper layer under `fastvideo.workflow.interleave_thinker` is intentionally
|
||||
not registered as a CLI or server contract. It provides reusable dataclasses,
|
||||
provider adapters, image-backend adapters, trace serialization, prompt-set
|
||||
execution helpers, and saved-trace metrics for tests, examples, and downstream
|
||||
@@ -61,9 +61,9 @@ The runnable example is:
|
||||
| Planner and critic actors | `fastvideo/train/models/interleave_thinker/` | Wrap Transformers Qwen3-VL checkpoints. They are training actors, not diffusion pipeline components. |
|
||||
| RL/SFT algorithms | `fastvideo/train/methods/` | Own loss, reward aggregation, advantage computation, KL, and optimizer cadence. |
|
||||
| Rewards and API clients | `fastvideo/train/methods/rl/rewards/` | Offline reward aggregation is separate from network-backed Gemini/Nano Banana clients. |
|
||||
| Image generation/editing helpers | `fastvideo/workflows/interleave_thinker/generator.py` | Presents a small image backend protocol for FastVideo, Nano Banana, and fake backends. |
|
||||
| Runtime orchestration helpers | `fastvideo/workflows/interleave_thinker/` | Plans steps, calls generator/edit backends, calls critic providers, records traces. |
|
||||
| Evaluation helpers | `fastvideo/workflows/interleave_thinker/evaluation.py` and `trace_eval.py` | Prompt-set execution and saved-trace reporting remain outside training methods. |
|
||||
| Image generation/editing helpers | `fastvideo/workflow/interleave_thinker/generator.py` | Presents a small image backend protocol for FastVideo, Nano Banana, and fake backends. |
|
||||
| Runtime orchestration helpers | `fastvideo/workflow/interleave_thinker/` | Plans steps, calls generator/edit backends, calls critic providers, records traces. |
|
||||
| Evaluation helpers | `fastvideo/workflow/interleave_thinker/evaluation.py` and `trace_eval.py` | Prompt-set execution and saved-trace reporting remain outside training methods. |
|
||||
|
||||
This keeps the Qwen actor implementation reusable by SFT, planner GRPO, critic
|
||||
GRPO, and inference providers without coupling those paths to a specific
|
||||
|
||||
@@ -36,7 +36,7 @@ The RL reward config can use closed-source Google models through lazy wrappers:
|
||||
|
||||
- `fastvideo.train.methods.rl.rewards.GeminiNanoBananaEditScorer` generates
|
||||
edits with Nano Banana and scores them with Gemini.
|
||||
- `fastvideo.workflows.interleave_thinker.generator.NanoBananaImageGeneratorBackend`
|
||||
- `fastvideo.workflow.interleave_thinker.generator.NanoBananaImageGeneratorBackend`
|
||||
implements the same image backend protocol as the local FastVideo generator.
|
||||
|
||||
Install the optional SDK and provide a key only when using these API backends:
|
||||
|
||||
@@ -19,7 +19,7 @@ from fastvideo.api.schema import (
|
||||
OffloadConfig,
|
||||
PipelineSelection,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker import (
|
||||
from fastvideo.workflow.interleave_thinker import (
|
||||
AcceptAllCritic,
|
||||
FastVideoImageGeneratorBackend,
|
||||
InterleaveOrchestrator,
|
||||
|
||||
@@ -22,11 +22,11 @@ import time
|
||||
import uuid
|
||||
from typing import Any
|
||||
|
||||
from fastvideo.workflows.interleave_thinker.generator import (
|
||||
from fastvideo.workflow.interleave_thinker.generator import (
|
||||
NanoBananaImageGeneratorBackend,
|
||||
encode_file_to_base64,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.schema import (
|
||||
from fastvideo.workflow.interleave_thinker.schema import (
|
||||
InterleaveEditRequest as InterleaveAPIEditRequest, )
|
||||
from fastvideo.train.methods.rl.rewards.interleave_thinker import (
|
||||
InterleaveThinkerEditRequest,
|
||||
|
||||
+9
-9
@@ -1,7 +1,7 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""InterleaveThinker workflow helpers for FastVideo."""
|
||||
|
||||
from fastvideo.workflows.interleave_thinker.config import (
|
||||
from fastvideo.workflow.interleave_thinker.config import (
|
||||
InterleaveCriticConfig,
|
||||
InterleaveImageBackendConfig,
|
||||
InterleavePlannerConfig,
|
||||
@@ -10,7 +10,7 @@ from fastvideo.workflows.interleave_thinker.config import (
|
||||
load_interleave_run_config,
|
||||
resolve_interleave_instruction,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.evaluation import (
|
||||
from fastvideo.workflow.interleave_thinker.evaluation import (
|
||||
InterleavePromptItem,
|
||||
InterleavePromptResult,
|
||||
InterleavePromptSetSummary,
|
||||
@@ -20,26 +20,26 @@ from fastvideo.workflows.interleave_thinker.evaluation import (
|
||||
run_interleave_prompt_set_config,
|
||||
save_prompt_set_summary,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.generator import (
|
||||
from fastvideo.workflow.interleave_thinker.generator import (
|
||||
FastVideoImageGeneratorBackend,
|
||||
ImageGeneratorBackend,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.orchestrator import (
|
||||
from fastvideo.workflow.interleave_thinker.orchestrator import (
|
||||
AcceptAllCritic,
|
||||
CriticProvider,
|
||||
InterleaveOrchestrator,
|
||||
PlannerProvider,
|
||||
SinglePromptPlanner,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.providers import (
|
||||
from fastvideo.workflow.interleave_thinker.providers import (
|
||||
InterleaveThinkerCriticProvider,
|
||||
InterleaveThinkerPlannerProvider,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.runner import (
|
||||
from fastvideo.workflow.interleave_thinker.runner import (
|
||||
InterleaveRunResult,
|
||||
run_interleave_config,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.schema import (
|
||||
from fastvideo.workflow.interleave_thinker.schema import (
|
||||
CriticDecision,
|
||||
CriticInput,
|
||||
GeneratedImage,
|
||||
@@ -50,11 +50,11 @@ from fastvideo.workflows.interleave_thinker.schema import (
|
||||
PlannedInterleaveStep,
|
||||
PlannerInput,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.trace import (
|
||||
from fastvideo.workflow.interleave_thinker.trace import (
|
||||
save_trace,
|
||||
trace_to_dict,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.trace_eval import (
|
||||
from fastvideo.workflow.interleave_thinker.trace_eval import (
|
||||
InterleaveTraceEvaluationSummary,
|
||||
InterleaveTraceMetrics,
|
||||
discover_interleave_trace_paths,
|
||||
+5
-5
@@ -11,14 +11,14 @@ from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastvideo.workflows.interleave_thinker.generator import ImageGeneratorBackend
|
||||
from fastvideo.workflows.interleave_thinker.runner import (
|
||||
from fastvideo.workflow.interleave_thinker.generator import ImageGeneratorBackend
|
||||
from fastvideo.workflow.interleave_thinker.runner import (
|
||||
build_critic,
|
||||
build_image_backend,
|
||||
build_planner,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.schema import InterleaveTrace
|
||||
from fastvideo.workflows.interleave_thinker.trace import save_trace
|
||||
from fastvideo.workflow.interleave_thinker.schema import InterleaveTrace
|
||||
from fastvideo.workflow.interleave_thinker.trace import save_trace
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -208,7 +208,7 @@ def _build_prompt_set_orchestrator(
|
||||
config: Any,
|
||||
image_backend: ImageGeneratorBackend,
|
||||
) -> Any:
|
||||
from fastvideo.workflows.interleave_thinker.orchestrator import InterleaveOrchestrator
|
||||
from fastvideo.workflow.interleave_thinker.orchestrator import InterleaveOrchestrator
|
||||
|
||||
return InterleaveOrchestrator(
|
||||
planner=build_planner(config.planner),
|
||||
+1
-1
@@ -19,7 +19,7 @@ from fastvideo.api.compat import (
|
||||
)
|
||||
from fastvideo.api.results import GenerationResult
|
||||
from fastvideo.api.schema import GenerationRequest
|
||||
from fastvideo.workflows.interleave_thinker.schema import (
|
||||
from fastvideo.workflow.interleave_thinker.schema import (
|
||||
GeneratedImage,
|
||||
InterleaveEditRequest,
|
||||
)
|
||||
+2
-2
@@ -6,11 +6,11 @@ from __future__ import annotations
|
||||
from collections.abc import Sequence
|
||||
from typing import Any, Protocol
|
||||
|
||||
from fastvideo.workflows.interleave_thinker.generator import (
|
||||
from fastvideo.workflow.interleave_thinker.generator import (
|
||||
ImageGeneratorBackend,
|
||||
encode_file_to_base64,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.schema import (
|
||||
from fastvideo.workflow.interleave_thinker.schema import (
|
||||
CriticDecision,
|
||||
CriticInput,
|
||||
GeneratedImage,
|
||||
+1
-1
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from fastvideo.workflows.interleave_thinker.schema import (
|
||||
from fastvideo.workflow.interleave_thinker.schema import (
|
||||
CriticDecision,
|
||||
CriticInput,
|
||||
PlannedInterleaveStep,
|
||||
+6
-6
@@ -8,31 +8,31 @@ from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastvideo.workflows.interleave_thinker.config import (
|
||||
from fastvideo.workflow.interleave_thinker.config import (
|
||||
InterleaveCriticConfig,
|
||||
InterleaveImageBackendConfig,
|
||||
InterleavePlannerConfig,
|
||||
InterleaveRunConfig,
|
||||
resolve_interleave_instruction,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.generator import (
|
||||
from fastvideo.workflow.interleave_thinker.generator import (
|
||||
FastVideoImageGeneratorBackend,
|
||||
ImageGeneratorBackend,
|
||||
NanoBananaImageGeneratorBackend,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.orchestrator import (
|
||||
from fastvideo.workflow.interleave_thinker.orchestrator import (
|
||||
AcceptAllCritic,
|
||||
CriticProvider,
|
||||
InterleaveOrchestrator,
|
||||
PlannerProvider,
|
||||
SinglePromptPlanner,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.providers import (
|
||||
from fastvideo.workflow.interleave_thinker.providers import (
|
||||
InterleaveThinkerCriticProvider,
|
||||
InterleaveThinkerPlannerProvider,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.schema import InterleaveTrace
|
||||
from fastvideo.workflows.interleave_thinker.trace import save_trace
|
||||
from fastvideo.workflow.interleave_thinker.schema import InterleaveTrace
|
||||
from fastvideo.workflow.interleave_thinker.trace import save_trace
|
||||
from fastvideo.train.models.interleave_thinker import (
|
||||
InterleaveThinkerCriticModel,
|
||||
InterleaveThinkerPlannerModel,
|
||||
+1
-1
@@ -7,7 +7,7 @@ import json
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastvideo.workflows.interleave_thinker.schema import (
|
||||
from fastvideo.workflow.interleave_thinker.schema import (
|
||||
CriticDecision,
|
||||
GeneratedImage,
|
||||
InterleaveAttempt,
|
||||
@@ -1,2 +0,0 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Reusable workflow helpers built on top of FastVideo components."""
|
||||
@@ -5,12 +5,12 @@ from __future__ import annotations
|
||||
import base64
|
||||
from pathlib import Path
|
||||
|
||||
from fastvideo.workflows.interleave_thinker.orchestrator import InterleaveOrchestrator
|
||||
from fastvideo.workflows.interleave_thinker.providers import (
|
||||
from fastvideo.workflow.interleave_thinker.orchestrator import InterleaveOrchestrator
|
||||
from fastvideo.workflow.interleave_thinker.providers import (
|
||||
InterleaveThinkerCriticProvider,
|
||||
InterleaveThinkerPlannerProvider,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.schema import (
|
||||
from fastvideo.workflow.interleave_thinker.schema import (
|
||||
CriticInput,
|
||||
GeneratedImage,
|
||||
PlannedInterleaveStep,
|
||||
|
||||
@@ -6,10 +6,10 @@ import types
|
||||
from PIL import Image
|
||||
import pytest
|
||||
|
||||
from fastvideo.workflows.interleave_thinker.generator import (
|
||||
from fastvideo.workflow.interleave_thinker.generator import (
|
||||
NanoBananaImageGeneratorBackend,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.schema import InterleaveEditRequest
|
||||
from fastvideo.workflow.interleave_thinker.schema import InterleaveEditRequest
|
||||
from fastvideo.train.methods.rl.rewards.interleave_api import (
|
||||
GeminiNanoBananaEditScorer,
|
||||
)
|
||||
|
||||
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from fastvideo.workflows.interleave_thinker import (
|
||||
from fastvideo.workflow.interleave_thinker import (
|
||||
discover_interleave_trace_paths,
|
||||
evaluate_interleave_traces,
|
||||
interleave_trace_evaluation_to_dict,
|
||||
|
||||
@@ -10,17 +10,17 @@ from fastvideo.api.compat import (
|
||||
legacy_generate_call_to_request,
|
||||
)
|
||||
from fastvideo.api.results import GenerationResult
|
||||
from fastvideo.workflows.interleave_thinker.generator import (
|
||||
from fastvideo.workflow.interleave_thinker.generator import (
|
||||
FastVideoImageGeneratorBackend,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.orchestrator import InterleaveOrchestrator
|
||||
from fastvideo.workflows.interleave_thinker.schema import (
|
||||
from fastvideo.workflow.interleave_thinker.orchestrator import InterleaveOrchestrator
|
||||
from fastvideo.workflow.interleave_thinker.schema import (
|
||||
CriticDecision,
|
||||
GeneratedImage,
|
||||
InterleaveEditRequest,
|
||||
PlannedInterleaveStep,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.trace import (
|
||||
from fastvideo.workflow.interleave_thinker.trace import (
|
||||
save_trace,
|
||||
trace_to_dict,
|
||||
)
|
||||
|
||||
@@ -7,7 +7,7 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from fastvideo.workflows.interleave_thinker import (
|
||||
from fastvideo.workflow.interleave_thinker import (
|
||||
GeneratedImage,
|
||||
InterleaveEditRequest,
|
||||
InterleavePromptItem,
|
||||
@@ -17,8 +17,8 @@ from fastvideo.workflows.interleave_thinker import (
|
||||
run_interleave_prompt_set,
|
||||
run_interleave_config,
|
||||
)
|
||||
from fastvideo.workflows.interleave_thinker.orchestrator import SinglePromptPlanner
|
||||
from fastvideo.workflows.interleave_thinker.schema import PlannerInput
|
||||
from fastvideo.workflow.interleave_thinker.orchestrator import SinglePromptPlanner
|
||||
from fastvideo.workflow.interleave_thinker.schema import PlannerInput
|
||||
|
||||
|
||||
def test_interleave_run_config_loads_prompt_and_request_defaults(tmp_path: Path) -> None:
|
||||
|
||||
Reference in New Issue
Block a user