Add Gemini Omni Flash support and fix stale Veo model listings

- Merge Omni Flash into the existing "gemini" provider: one dropdown
  entry, model list now includes both Veo and Omni Flash ids, each
  revealing its own correct settings (resolution/duration/etc).
- Omni Flash runs through litellm's request/response Interactions API
  (litellm.interactions.acreate) instead of the polling
  avideo_generation/avideo_status/avideo_content trio Veo and the other
  providers use, since it has no status-polling endpoint. Text-to-video
  only for now; image/video-conditioned input and conversational
  editing aren't implemented yet.
- Fix progress reporting for the Omni path: it now ticks up by
  elapsed/timeout while waiting on the single request instead of
  sitting at 0% and jumping straight to done.
- Fix live Veo model listing to filter out models Google's API no
  longer serves for generation (checks supported_actions for
  predictLongRunning) rather than trusting name-matching alone — root
  cause of a live 404 on veo-3.1-fast-generate-001, which is also
  dropped from the static fallback list.
- Bump litellm 1.92.0 -> 1.100.0.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVFBqa77Zo9t4ps8wt7YB2
This commit is contained in:
Andy Ratsirarson
2026-09-07 20:44:24 -07:00
co-authored by Claude Sonnet 5
parent 6691b570a7
commit 3ea918e84c
14 changed files with 383 additions and 51 deletions
+7 -3
View File
@@ -14,14 +14,16 @@ BYOK (bring-your-own-key) ComfyUI nodes for generating video through external pr
| Provider | `provider` value | Models | Aspect ratios | Resolutions | Duration |
|---|---|---|---|---|---|
| Google Gemini (Veo) | `gemini` | Fetched live from your account via `google-genai`; falls back to a built-in list (`veo-3.1-fast-generate-preview`, `veo-3.1-fast-generate-001`, `veo-3.1-generate-preview`, `veo-3.1-generate-001`, `veo-3.1-lite-generate-preview`, `veo-2.0-generate-001`) if that fails | 16:9, 9:16 | 1K, 2K | 4–8s |
| Google Gemini (Veo + Omni Flash) | `gemini` | Fetched live from your account via `google-genai`; falls back to a built-in list of Veo models (`veo-3.1-fast-generate-preview`, `veo-3.1-fast-generate-001`, `veo-3.1-generate-preview`, `veo-3.1-generate-001`, `veo-3.1-lite-generate-preview`, `veo-2.0-generate-001`) plus Omni Flash models (`gemini-omni-1.1-flash`, `gemini-omni-flash-preview`) if that fails | Veo: 16:9, 9:16 · Omni: 16:9, 9:16 | Veo: 1K, 2K · Omni: 360p, 720p, 1080p, 4k | Veo: 4–8s · Omni: 3–10s |
| OpenAI (Sora) | `openai` | `sora-2`, `sora-2-pro`, `sora-2-pro-high-res` | 16:9, 9:16 | 1K, 2K | 4–12s* |
| Azure OpenAI (Sora) | `azure` | `sora-2`, `sora-2-pro`, `sora-2-pro-high-res` | 16:9, 9:16 | 1K, 2K | 4–12s* |
| RunwayML | `runwayml` | `gen3a_turbo`, `gen4_turbo`, `gen4_aleph`, `gen4_image`, `gen4_image_turbo` | 16:9, 9:16 | 1K, 2K | 5s or 10s |
\* OpenAI/Azure's upper duration bound isn't published by the provider; 12s is a conservative default rather than a confirmed hard limit.
Gemini's model list is fetched live from `google.genai.Client().models.list()` at node-registration time when a Gemini API key is available on the ComfyUI server, so new Veo releases show up automatically without an update to this package.
Gemini's model list is fetched live from `google.genai.Client().models.list()` at node-registration time when a Gemini API key is available on the ComfyUI server, so new Veo/Omni releases show up automatically without an update to this package. The single `gemini` provider's `model` dropdown lists both families together — picking a Veo model reveals Veo's settings, picking an Omni Flash model reveals Omni's.
**Omni Flash is a different API shape than Veo, under the hood.** It runs through litellm's request/response Interactions API (`litellm.interactions.acreate`) instead of the polling `avideo_generation`/`avideo_status`/`avideo_content` calls Veo and the other providers use, so there's no in-progress polling for it — the node just waits on the single call. Only one-shot text-to-video is wired up today; Omni Flash's headline features — image/video-conditioned generation and multi-turn conversational editing — aren't implemented (see [Roadmap](#roadmap)), so its models don't appear in the `image`-source `provider` dropdown. Very large outputs that Google delivers by URI instead of inline base64 aren't downloaded yet and will raise a clear error rather than silently failing.
**Image-to-video** (a reference image conditioning the result) is currently supported for **Gemini/Veo only**. The other providers are text-to-video only for now — see [Roadmap](#roadmap). The models, aspect ratios, resolutions, and durations in the table above apply to both text- and image-to-video.
@@ -29,7 +31,7 @@ Gemini's model list is fetched live from `google.genai.Client().models.list()` a
- ComfyUI with the V3 node API (`comfy_api.latest`).
- Python dependencies (installed automatically, see `requirements.txt`):
- `litellm==1.92.0`
- `litellm==1.100.0`
- `google-genai==1.47.0`
- An API key from at least one of the providers above.
@@ -94,6 +96,8 @@ This currently supports exactly **one** reference image per generation. It does
- **Image generation** — a separate `APIImageGenerate` node, tracked in [issue #2](https://github.com/andy-ratsirarson/ComfyUI_ExternalAPI/issues/2).
- **Image-to-video** — implemented for Gemini/Veo (set `source` to `image`), single reference image only. Support for the other providers (OpenAI/Azure Sora, RunwayML) is future work.
- **Multi-image references** — RunwayML supports referencing several images at once (with in-prompt tagging), and Gemini/Veo 3.1 separately supports up to 3 reference images plus last-frame interpolation and video extension. None of this richer, multi-image capability is implemented yet — today's image-to-video is single-image only for both providers that support it at all.
- **Gemini Omni Flash: image/video-conditioned generation and conversational editing** — Omni Flash accepts image and video inputs and supports iteratively refining a result across turns via `previous_interaction_id`. Only one-shot text-to-video is wired up today (`gemini_omni` provider, `source: text`); feeding it a reference image/video and chaining edits is not yet implemented.
- **Gemini Omni Flash: large (uri-delivered) outputs** — Google delivers big videos by URI instead of inline base64; downloading those isn't implemented yet, so such a response raises a clear error instead of returning a video.
## Development
+1 -1
View File
@@ -20,7 +20,7 @@ license = "MIT"
# "Environment :: GPU :: Apple Metal", # Apple Metal support
# ]
dependencies = ["litellm==1.92.0", "google-genai==1.47.0"]
dependencies = ["litellm==1.100.0", "google-genai==1.47.0"]
[project.optional-dependencies]
dev = ["pytest>=7.0"]
+1 -1
View File
@@ -1,2 +1,2 @@
litellm==1.92.0
litellm==1.100.0
google-genai==1.47.0
+1 -1
View File
@@ -1,4 +1,4 @@
from src.models.openai import _SoraModel
from .openai import _SoraModel
class AzureModel(_SoraModel):
+33 -5
View File
@@ -12,13 +12,29 @@ class Model:
PROVIDER: str = None
VIDEO_SETTINGS = None
@classmethod
def uses_interactions_api(cls, model_id: str | None = None) -> bool:
"""True if this model must be called through litellm's request/response
Interactions API (litellm.interactions.acreate) instead of the polling
avideo_generation/avideo_status/avideo_content trio. A provider class
can cover models that need either (e.g. Gemini's Veo vs. Omni Flash),
so this is decided per model_id rather than being a fixed class flag."""
return False
@classmethod
def list_models(cls) -> list[str]:
"""Return this provider's available model ids for text-to-video."""
raise NotImplementedError(f"list_models not implemented for {cls.__name__}.")
@classmethod
def video_settings_inputs(cls):
def image_to_video_model_ids(cls) -> list[str]:
"""Model ids from list_models() that also support image-to-video.
Defaults to all of them; override when only some do (e.g. Gemini's
Veo models support it, but its Omni Flash models don't yet)."""
return cls.list_models()
@classmethod
def video_settings_inputs(cls, model_id: str | None = None):
"""Build the io.Input list revealed once a specific model is selected."""
if cls.VIDEO_SETTINGS is None:
raise NotImplementedError(f"{cls.__name__} does not support video generation.")
@@ -36,7 +52,7 @@ class Model:
raise NotImplementedError(f"video_kwargs not implemented for {cls.__name__}.")
@classmethod
def image_to_video_settings_inputs(cls):
def image_to_video_settings_inputs(cls, model_id: str | None = None):
"""Build the io.Input list revealed once a specific model is selected,
for the image-to-video ('image' source) path. Distinct from
video_settings_inputs() because providers can genuinely differ here
@@ -50,6 +66,13 @@ class Model:
expects for image-to-video."""
raise NotImplementedError(f"image_to_video_kwargs not implemented for {cls.__name__}.")
@classmethod
def extract_video_bytes(cls, response) -> bytes:
"""USES_INTERACTIONS_API providers only: pull the generated video's raw
bytes out of the litellm InteractionsAPIResponse returned by
litellm.interactions.acreate()."""
raise NotImplementedError(f"extract_video_bytes not implemented for {cls.__name__}.")
@staticmethod
def image_tensor_to_bytesio(image_tensor):
"""Convert a ComfyUI IMAGE tensor ([B, H, W, C] float32 in [0, 1]) into
@@ -74,15 +97,20 @@ class Model:
"""Build this provider's DynamicCombo.Option: a nested 'model' combo where
each model reveals this provider's settings for the given source mode
("text" = text-to-video, "image" = image-to-video)."""
build_inputs = cls.video_settings_inputs if mode == "text" else cls.image_to_video_settings_inputs
if mode == "text":
model_ids = cls.list_models()
build_inputs = cls.video_settings_inputs
else:
model_ids = cls.image_to_video_model_ids()
build_inputs = cls.image_to_video_settings_inputs
return io.DynamicCombo.Option(
cls.PROVIDER,
[
io.DynamicCombo.Input(
"model",
options=[
io.DynamicCombo.Option(model_id, build_inputs())
for model_id in cls.list_models()
io.DynamicCombo.Option(model_id, build_inputs(model_id))
for model_id in model_ids
],
),
],
+118 -27
View File
@@ -1,34 +1,75 @@
import base64
from comfy_api.latest import io
from src.models.base import Model
from src.settings.base import Resolutions, Sizes
from src.settings.video import VideoSettings, reference_image_input
from .base import Model
from ..settings.base import Resolutions, Sizes
from ..settings.video import VideoSettings, reference_image_input
STATIC_MODELS = [
# Veo (text/image-to-video, polled via avideo_generation) and Omni Flash
# (text-to-video only so far, via litellm's request/response Interactions
# API) are both under the single "gemini" provider dropdown entry, and
# dispatched on per model_id below — they share Google's account/API key but
# are otherwise unrelated APIs with different parameters and response shapes.
VEO_STATIC_MODELS = [
"veo-3.1-fast-generate-preview",
"veo-3.1-fast-generate-001",
"veo-3.1-generate-preview",
"veo-3.1-generate-001",
"veo-3.1-lite-generate-preview",
"veo-2.0-generate-001",
]
RESOLUTION_MAP = {Resolutions._1K: "720p", Resolutions._2K: "1080p"}
# The REST method Veo generation calls (litellm's avideo_generation ->
# POST {model}:predictLongRunning). Google's model list reports, per model,
# which methods it actually supports right now — model names alone aren't
# reliable, since Google retires/moves specific ids off this method without
# renaming them (e.g. veo-3.1-fast-generate-001 404s here as of 2026-09 while
# still resolving as a real model name).
VEO_GENERATION_ACTION = "predictLongRunning"
OMNI_STATIC_MODELS = [
"gemini-omni-1.1-flash",
"gemini-omni-flash-preview",
]
VEO_RESOLUTION_MAP = {Resolutions._1K: "720p", Resolutions._2K: "1080p"}
OMNI_RESOLUTION_LABELS = {
Resolutions._360: "360p",
Resolutions._720: "720p",
Resolutions._1080: "1080p",
Resolutions._4K: "4k",
}
VEO_VIDEO_SETTINGS = VideoSettings(
SIZE=[Sizes._16_9, Sizes._9_16],
RESOLUTIONS=[Resolutions._1K, Resolutions._2K],
MIN_SECONDS=4,
MAX_SECONDS=8,
)
OMNI_VIDEO_SETTINGS = VideoSettings(
SIZE=[Sizes._16_9, Sizes._9_16],
RESOLUTIONS=[Resolutions._360, Resolutions._720, Resolutions._1080, Resolutions._4K],
MIN_SECONDS=3,
MAX_SECONDS=10,
)
def _is_omni(model_id: str) -> bool:
return "omni" in model_id.lower()
class GeminiModel(Model):
"""Google Veo text-to-video via litellm's 'gemini/<model_id>' provider."""
"""Google Gemini video generation via litellm's 'gemini/<model_id>' provider:
Veo (text/image-to-video) and Omni Flash (text-to-video only, so far)."""
PROVIDER = "gemini"
VIDEO_SETTINGS = VideoSettings(
SIZE=[Sizes._16_9, Sizes._9_16],
RESOLUTIONS=[Resolutions._1K, Resolutions._2K],
MIN_SECONDS=4,
MAX_SECONDS=8,
)
_models_cache = None
@classmethod
def uses_interactions_api(cls, model_id: str | None = None) -> bool:
return _is_omni(model_id or "")
@classmethod
def list_models(cls) -> list[str]:
if cls._models_cache is not None:
@@ -43,13 +84,31 @@ class GeminiModel(Model):
if name.startswith("models/"):
name = name[len("models/"):]
if "veo" in name.lower():
# Only offer it if Google's own listing says it still
# supports video generation. A model can resolve as a
# real name yet 404 on the actual generate call once
# Google retires it from this method (unknown/missing
# supported_actions is treated as "don't know" rather
# than excluded, so a leaner API response can't hide
# every model).
actions = getattr(model, "supported_actions", None)
if actions and VEO_GENERATION_ACTION not in actions:
continue
models.append(name)
cls._models_cache = models or list(STATIC_MODELS)
elif "omni" in name.lower():
models.append(name)
cls._models_cache = models or (list(VEO_STATIC_MODELS) + list(OMNI_STATIC_MODELS))
except Exception:
cls._models_cache = list(STATIC_MODELS)
cls._models_cache = list(VEO_STATIC_MODELS) + list(OMNI_STATIC_MODELS)
return cls._models_cache
@classmethod
def image_to_video_model_ids(cls) -> list[str]:
# Image-to-video is Veo-only for now; Omni Flash's image/video
# conditioning isn't implemented yet (see README roadmap).
return [model_id for model_id in cls.list_models() if not _is_omni(model_id)]
@staticmethod
def _negative_prompt_input():
return io.String.Input(
@@ -61,25 +120,32 @@ class GeminiModel(Model):
)
@classmethod
def video_settings_inputs(cls):
inputs = super().video_settings_inputs()
inputs.append(
def video_settings_inputs(cls, model_id: str | None = None):
if _is_omni(model_id or ""):
return [
OMNI_VIDEO_SETTINGS.size_input(),
OMNI_VIDEO_SETTINGS.resolution_input(),
OMNI_VIDEO_SETTINGS.seconds_input(),
]
return [
VEO_VIDEO_SETTINGS.size_input(),
VEO_VIDEO_SETTINGS.resolution_input(),
VEO_VIDEO_SETTINGS.seconds_input(),
io.Combo.Input(
"person_generation",
options=["allow_all", "allow_adult"],
default="allow_all",
tooltip="Whether people may be generated in the video.",
)
)
inputs.append(cls._negative_prompt_input())
return inputs
),
cls._negative_prompt_input(),
]
@classmethod
def image_to_video_settings_inputs(cls):
def image_to_video_settings_inputs(cls, model_id: str | None = None):
return [
cls.VIDEO_SETTINGS.size_input(),
cls.VIDEO_SETTINGS.resolution_input(),
cls.VIDEO_SETTINGS.seconds_input(),
VEO_VIDEO_SETTINGS.size_input(),
VEO_VIDEO_SETTINGS.resolution_input(),
VEO_VIDEO_SETTINGS.seconds_input(),
reference_image_input(),
io.Combo.Input(
"person_generation",
@@ -92,9 +158,17 @@ class GeminiModel(Model):
@classmethod
def video_kwargs(cls, model_id: str, settings: dict) -> dict:
if _is_omni(model_id):
return {
"response_format": {
"type": "video",
"aspect_ratio": settings["size"],
"resolution": OMNI_RESOLUTION_LABELS[settings["resolution"]],
}
}
kwargs = {
"aspectRatio": settings["size"],
"resolution": RESOLUTION_MAP[settings["resolution"]],
"resolution": VEO_RESOLUTION_MAP[settings["resolution"]],
"durationSeconds": settings["seconds"],
"personGeneration": settings["person_generation"],
}
@@ -110,3 +184,20 @@ class GeminiModel(Model):
kwargs = cls.video_kwargs(model_id, settings)
kwargs["input_reference"] = cls.image_tensor_to_bytesio(reference_image)
return kwargs
@classmethod
def extract_video_bytes(cls, response) -> bytes:
for step in getattr(response, "steps", None) or []:
if step.get("type") != "model_output":
continue
for item in step.get("content") or []:
if item.get("type") != "video":
continue
data = item.get("data")
if data:
return base64.b64decode(data)
raise RuntimeError(
"Gemini Omni returned the video by reference (uri) instead of inline "
"base64 data; downloading a uri-delivered video isn't supported yet."
)
raise RuntimeError("Gemini Omni response contained no video output.")
+3 -3
View File
@@ -1,6 +1,6 @@
from src.models.base import Model
from src.settings.base import Resolutions, Sizes
from src.settings.video import VideoSettings, dimensions_string
from .base import Model
from ..settings.base import Resolutions, Sizes
from ..settings.video import VideoSettings, dimensions_string
class _SoraModel(Model):
+4 -4
View File
@@ -1,8 +1,8 @@
from comfy_api.latest import io
from src.models.base import Model
from src.settings.base import Resolutions, Sizes
from src.settings.video import VideoSettings, dimensions_string
from .base import Model
from ..settings.base import Resolutions, Sizes
from ..settings.video import VideoSettings, dimensions_string
class RunwayMLModel(Model):
@@ -20,7 +20,7 @@ class RunwayMLModel(Model):
return ["gen3a_turbo", "gen4_turbo", "gen4_aleph", "gen4_image", "gen4_image_turbo"]
@classmethod
def video_settings_inputs(cls):
def video_settings_inputs(cls, model_id: str | None = None):
return [
cls.VIDEO_SETTINGS.size_input(),
cls.VIDEO_SETTINGS.resolution_input(),
+25 -4
View File
@@ -3,11 +3,12 @@ from io import BytesIO
from comfy_api.latest import ComfyAPI, InputImpl, io
from litellm import avideo_content, avideo_generation, avideo_status
from litellm.interactions import acreate as acreate_interaction
from src.models.azure import AzureModel
from src.models.gemini import GeminiModel
from src.models.openai import OpenAIModel
from src.models.runwayml import RunwayMLModel
from .models.azure import AzureModel
from .models.gemini import GeminiModel
from .models.openai import OpenAIModel
from .models.runwayml import RunwayMLModel
api = ComfyAPI()
@@ -126,6 +127,26 @@ class APIVideoGenerate(io.ComfyNode):
extra_kwargs = provider_cls.image_to_video_kwargs(model_id, settings)
model = f"{provider_name}/{model_id}"
if provider_cls.uses_interactions_api(model_id):
# The Interactions API is a single request/response call with no
# status-polling endpoint, so there's no real progress fraction to
# report. Tick the bar up by elapsed/timeout while we wait on it,
# the same heuristic the polling path below uses between polls,
# instead of sitting at 0% and then jumping straight to done.
request = asyncio.ensure_future(
acreate_interaction(model=model, input=prompt, timeout=timeout, **auth_kwargs, **extra_kwargs)
)
elapsed = 0.0
await api.execution.set_progress(0.0, timeout, node_id=node_id)
while not request.done():
await asyncio.wait([request], timeout=poll_interval)
elapsed = min(elapsed + poll_interval, timeout)
await api.execution.set_progress(elapsed, timeout, node_id=node_id)
response = await request
await api.execution.set_progress(timeout, timeout, node_id=node_id)
video_bytes = provider_cls.extract_video_bytes(response)
return io.NodeOutput(InputImpl.VideoFromFile(BytesIO(video_bytes)))
op = await avideo_generation(
model=model, prompt=prompt, timeout=timeout, **auth_kwargs, **extra_kwargs
)
+1
View File
@@ -25,6 +25,7 @@ class Resolutions(str, Enum):
once; every model's RESOLUTIONS list references these members instead of
a raw string."""
_360 = "360"
_480 = "480"
_720 = "720"
_1080 = "1080"
+1 -1
View File
@@ -1,6 +1,6 @@
from comfy_api.latest import io
from src.settings.base import Resolutions, Settings, Sizes
from .base import Resolutions, Settings, Sizes
IMAGE_SIZES = [
Sizes._21_9,
+1 -1
View File
@@ -1,6 +1,6 @@
from comfy_api.latest import io
from src.settings.base import Resolutions, Settings, Sizes
from .base import Resolutions, Settings, Sizes
VIDEO_SIZES = [Sizes._21_9, Sizes._16_9, Sizes._4_3, Sizes._1_1, Sizes._3_4, Sizes._9_16]
VIDEO_RESOLUTIONS = [Resolutions._1K, Resolutions._2K, Resolutions._4K]
+88
View File
@@ -0,0 +1,88 @@
import sys
import types
import pytest
from src.models.gemini import GeminiModel
class FakeGenaiModel:
def __init__(self, name, supported_actions=None):
self.name = name
self.supported_actions = supported_actions
def _install_fake_genai(models):
fake_models = types.SimpleNamespace(list=lambda: models)
fake_client_instance = types.SimpleNamespace(models=fake_models)
fake_genai_module = types.SimpleNamespace(Client=lambda: fake_client_instance)
google_module = types.ModuleType("google")
google_module.genai = fake_genai_module
sys.modules["google"] = google_module
sys.modules["google.genai"] = fake_genai_module
@pytest.fixture(autouse=True)
def _reset_cache():
GeminiModel._models_cache = None
yield
GeminiModel._models_cache = None
sys.modules.pop("google", None)
sys.modules.pop("google.genai", None)
def test_excludes_veo_model_missing_predict_long_running():
_install_fake_genai(
[
FakeGenaiModel("models/veo-3.1-fast-generate-001", supported_actions=["generateContent"]),
FakeGenaiModel("models/veo-3.1-fast-generate-preview", supported_actions=["predictLongRunning"]),
]
)
models = GeminiModel.list_models()
assert "veo-3.1-fast-generate-001" not in models
assert "veo-3.1-fast-generate-preview" in models
def test_keeps_veo_model_with_unknown_supported_actions():
_install_fake_genai(
[
FakeGenaiModel("models/veo-3.1-generate-preview", supported_actions=None),
]
)
models = GeminiModel.list_models()
assert "veo-3.1-generate-preview" in models
def test_omni_models_not_filtered_by_supported_actions():
_install_fake_genai(
[
FakeGenaiModel("models/gemini-omni-flash-preview", supported_actions=["generateContent"]),
]
)
models = GeminiModel.list_models()
assert "gemini-omni-flash-preview" in models
def test_falls_back_to_static_models_when_genai_list_fails():
def _raise():
raise RuntimeError("no api key")
fake_models = types.SimpleNamespace(list=_raise)
fake_client_instance = types.SimpleNamespace(models=fake_models)
fake_genai_module = types.SimpleNamespace(Client=lambda: fake_client_instance)
google_module = types.ModuleType("google")
google_module.genai = fake_genai_module
sys.modules["google"] = google_module
sys.modules["google.genai"] = fake_genai_module
models = GeminiModel.list_models()
assert "veo-3.1-fast-generate-001" not in models
assert "veo-3.1-fast-generate-preview" in models
assert "gemini-omni-1.1-flash" in models
+99
View File
@@ -1,4 +1,5 @@
import asyncio
import base64
import types
from unittest.mock import AsyncMock, patch
@@ -197,12 +198,28 @@ def test_schema_builds_nested_source_provider_model_combo():
field_names = {i.name for i in model_option.inputs}
assert {"size", "resolution", "seconds"} <= field_names
gemini_text_option = next(o for o in provider_input.options if o.key == "gemini")
gemini_text_model_input = gemini_text_option.inputs[0]
gemini_text_model_ids = {o.key for o in gemini_text_model_input.options}
assert "veo-3.1-fast-generate-preview" in gemini_text_model_ids
assert "gemini-omni-flash-preview" in gemini_text_model_ids
veo_option = next(o for o in gemini_text_model_input.options if o.key == "veo-3.1-fast-generate-preview")
veo_field_names = {i.name for i in veo_option.inputs}
assert "person_generation" in veo_field_names
omni_option = next(o for o in gemini_text_model_input.options if o.key == "gemini-omni-flash-preview")
omni_field_names = {i.name for i in omni_option.inputs}
assert "person_generation" not in omni_field_names
image_provider_input = image_option.inputs[0]
assert image_provider_input.name == "provider"
assert [o.key for o in image_provider_input.options] == ["gemini"]
gemini_model_input = image_provider_input.options[0].inputs[0]
assert gemini_model_input.name == "model"
gemini_image_model_ids = {o.key for o in gemini_model_input.options}
assert "gemini-omni-flash-preview" not in gemini_image_model_ids
for model_option in gemini_model_input.options:
field_names = {i.name for i in model_option.inputs}
assert {"size", "resolution", "seconds", "reference_image", "person_generation"} <= field_names
@@ -282,6 +299,88 @@ def test_gemini_image_to_video_requires_reference_image():
asyncio.run(run())
class FakeInteractionResponse:
def __init__(self, steps):
self.steps = steps
def _gemini_omni_source(model="gemini-omni-flash-preview", **overrides):
settings = {"model": model, "size": "16:9", "resolution": "720", "seconds": 5}
settings.update(overrides)
return {"source": "text", "provider": {"provider": "gemini", "model": settings}}
def test_gemini_omni_uses_interactions_api_not_polling():
async def run():
interaction_mock = AsyncMock(
return_value=FakeInteractionResponse(
steps=[
{"type": "user_input", "content": [{"type": "text", "text": "hi"}]},
{
"type": "model_output",
"content": [
{
"type": "video",
"data": base64.b64encode(b"fake-mp4-bytes").decode(),
"mime_type": "video/mp4",
}
],
},
]
)
)
gen_mock = AsyncMock()
with patch.object(nodes, "acreate_interaction", interaction_mock), patch.object(
nodes, "avideo_generation", gen_mock
):
result = await nodes.APIVideoGenerate.execute(
prompt="a marble rolling down a track",
source=_gemini_omni_source(),
)
gen_mock.assert_not_awaited()
interaction_mock.assert_awaited_once()
kwargs = interaction_mock.call_args.kwargs
assert kwargs["model"] == "gemini/gemini-omni-flash-preview"
assert kwargs["input"] == "a marble rolling down a track"
assert kwargs["response_format"] == {
"type": "video",
"aspect_ratio": "16:9",
"resolution": "720p",
}
assert "api_key" not in kwargs
video = result.args[0]
assert isinstance(video, nodes.InputImpl.VideoFromFile)
assert video.file_obj.getvalue() == b"fake-mp4-bytes"
asyncio.run(run())
def test_gemini_omni_uri_delivery_raises_clear_error():
async def run():
interaction_mock = AsyncMock(
return_value=FakeInteractionResponse(
steps=[
{
"type": "model_output",
"content": [{"type": "video", "uri": "https://example.com/video.mp4"}],
}
]
)
)
with patch.object(nodes, "acreate_interaction", interaction_mock):
with pytest.raises(RuntimeError, match="uri"):
await nodes.APIVideoGenerate.execute(
prompt="anything",
source=_gemini_omni_source(),
)
asyncio.run(run())
def test_azure_provider_string_uses_azure_prefix():
async def run():
gen_mock = AsyncMock(return_value=FakeVideoObject("completed"))