vantang-ComfyUI-MediaHub/docs/CURRENT_STATE.md

Current State

Last updated: 2026-06-13

Product Status

• Published: Comfy Registry node mediahub-cloud is active and version 0.1.0 was accepted on 2026-06-13. The version API initially reported NodeVersionStatusPending while Registry processing completed.
• GitHub Release v0.1.0 is published and points to the Registry source commit d9c1e3841097575ffc2ae159790c52625ef79ce9.
• ComfyUI-Manager registration PR https://github.com/Comfy-Org/ComfyUI-Manager/pull/2986 is open.
• Implemented: MiniMax CN Text-to-Image, Image-to-Image, Text-to-Video, Image-to-Video (I2V/FL2V/S2V); Agnes Image 2.1 T2I/I2I; Agnes Video 2.0 T2V/I2V/multi-image/keyframes; Seed Volcengine Seedream T2I/I2I, Seedance T2V/I2V (first_frame, first_last_frame); Wan Bailian CN T2I/I2I (wan2.7-image), T2V/I2V (wan2.7-t2v/i2v), R2V and VideoEdit adapters ready; HappyHorse Bailian CN T2V/I2V (happyhorse-1.0), R2V and VideoEdit adapters ready.
• Not implemented: Agnes Image 2.0, MiniMax Global, OpenAI Compatible, Gemini, Kling, secure local credential storage, complete Seed/Wan/HappyHorse live verification across all advertised operations, Wan/HappyHorse R2V and VideoEdit config nodes, UI workflow examples for R2V and VideoEdit.
• Architecture: provider-specific config nodes feed provider-neutral image and video execution nodes through stable request/job contracts.
• Media-family isolation: image and video own separate result contracts, services, typed requests, execution nodes, provider operations, workflows, and tests. Future audio/music must be added as another independent vertical slice.
• Old MediaHub task nodes and old workflows are intentionally unsupported.
• MiniMax CN image endpoint: https://api.minimaxi.com/v1/image_generation (shared by T2I and I2I).
• MiniMax CN video endpoints:
  • Submit: https://api.minimaxi.com/v1/video_generation
  • Query: https://api.minimaxi.com/v1/query/video_generation
  • File: https://api.minimaxi.com/v1/files/retrieve
• Seed Volcengine image endpoint: https://ark.cn-beijing.volces.com/api/v3/images/generations (shared by T2I and I2I, synchronous). Auth: ARK_API_KEY Bearer token.
• Seed Volcengine video endpoints:
  • Create task: https://ark.cn-beijing.volces.com/api/v3/contents/generations/tasks
  • Query task: https://ark.cn-beijing.volces.com/api/v3/contents/generations/tasks/{id}
  • Auth: ARK_API_KEY Bearer token.
  • Video URL returned directly in query response content.video_url.
• Wan Bailian CN / HappyHorse Bailian CN image endpoint: https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation (synchronous, shared by T2I and I2I). Auth: DASHSCOPE_API_KEY Bearer token.
• Wan Bailian CN / HappyHorse Bailian CN video endpoints:
  • Create task: https://dashscope.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis (header X-DashScope-Async: enable)
  • Query task: https://dashscope.aliyuncs.com/api/v1/tasks/{task_id}
  • Auth: DASHSCOPE_API_KEY Bearer token.
  • Video URL in query response output.video_url.
  • All video models (wan2.7-t2v, wan2.7-i2v, wan2.7-r2v, wan2.7-videoedit, happyhorse-1.0-t2v, happyhorse-1.0-i2v, happyhorse-1.0-r2v, happyhorse-1.0-video-edit) share the same endpoints.

Current UI Contract

• User-facing workflows use three left-to-right regions: user-owned prompt and source-media inputs; a portable MediaHub block containing API Key, provider configuration, and execution; and Preview/Save outputs.
• MediaHub API Key uses one compact row containing the masked password input and visibility toggle.
• Provider config nodes are grouped under MediaHub/Provider/<Provider>/<Media Family>; provider-neutral Image, Video, and Inputs remain at the MediaHub top level.
• API-format fixtures are deterministic smoke-test inputs, not visual workflow templates.

Text to Image (T2I)

• Prompt is a required STRING connection on MediaHub Text to Image.
• The user example connects ComfyUI's native PrimitiveStringMultiline node.
• API key is a required STRING connection from MediaHub API Key.
• Dynamic provider parameters are preserved in serialization while unavailable fields are visually collapsed.
• Dynamic visibility and node height are recalculated after workflow configuration, not only when a new node is created.
• The MiniMax CN node exposes every currently documented T2I request field: model, prompt, style, aspect_ratio, width, height, response_format, seed, n, prompt_optimizer, and aigc_watermark.
• use_seed defaults to false for normal random-generation behavior. The API smoke fixture deliberately keeps a fixed seed for deterministic validation.

Image to Image (I2I)

• A separate MediaHub Image to Image node, not a mode inside T2I.
• Uses the same MiniMax CN endpoint as T2I with the subject_reference parameter for character-portrait reference images.
• The subject_image input accepts a ComfyUI IMAGE tensor. Each image in the batch becomes a separate subject_reference entry ({"type": "character", "image_file": "data:image/jpeg;base64,..."}).
• The API supports multiple character references per request (verified from official documentation: subject_reference is an array).
• Source image requirements: portrait photo, < 10 MB, JPG/JPEG/PNG.
• Exposes the same generation parameters as T2I (model, aspect_ratio, seed, n, style, etc.) except size_mode/width/height (I2I always uses aspect_ratio; width/height are hidden in the UI but preserved in serialization).
• use_seed defaults to false; implements the same IS_CHANGED cache contract as T2I.
• Dynamic widgets: seed toggles with use_seed; use_style and style_type/style_weight toggle with model selection (image-01-live).

Text to Video (T2V)

• Three composable nodes forming a pipeline: MediaHubT2VSubmit → MediaHubVideoQueryStatus → MediaHubRetrieveVideo (renamed from MediaHubT2VQueryStatus / MediaHubT2VRetrieveVideo; legacy aliases preserved). The video output connects directly to ComfyUI's built-in SaveVideo node for saving with video preview.
• Uses MiniMax CN's async video generation API:
  • Submit: POST /v1/video_generation returns task_id.
  • Poll: GET /v1/query/video_generation?task_id=... returns status and file_id on success.
  • Download: GET /v1/files/retrieve?file_id=... returns download_url.
• MediaHubT2VSubmit: validates and submits the generation request. Outputs task_id and raw_json.
• MediaHubVideoQueryStatus: the "status indicator". Polls until the task succeeds or fails, then outputs file_id, video_info, and raw_json. This node acts as the trigger — downstream nodes only execute once the video is ready.
• MediaHubRetrieveVideo: downloads the MP4 by file_id, wraps it as a standard VIDEO object via InputImpl.VideoFromFile. Output connects directly to ComfyUI's built-in SaveVideo (which provides video preview).
• All three nodes share the api_key input from a single MediaHubAPIKey node. The provider flows from MediaHubT2VSubmit to both downstream nodes — QueryStatus and RetrieveVideo follow the Submit node's provider choice and cannot be independently changed.
• Supported models: MiniMax-Hailuo-2.3, MiniMax-Hailuo-02, T2V-01-Director, T2V-01.
• Parameter constraints are model-dependent:
  • Hailuo-2.3/02: 768P (6s or 10s) or 1080P (6s only).
  • T2V-01-Director/01: 720P only, 6s only.
• fast_pretreatment is only available for Hailuo-2.3 and Hailuo-02.
• Camera control instructions ([Pan right], [Zoom in], etc.) are supported in prompts for Hailuo-2.3, Hailuo-02, and T2V-01-Director.
• No use_seed or seed input (the T2V API does not expose a seed parameter). IS_CHANGED always returns NaN for all T2V nodes.
• Dynamic widgets on MediaHubT2VSubmit: resolution and duration combo values update based on model selection; fast_pretreatment is visible only for Hailuo models.

Image to Video (I2V)

• A single MediaHub I2V Submit node with three modes: i2v, fl2v, s2v.
• Reuses the video pipeline: MediaHubI2VSubmit → MediaHubVideoQueryStatus → MediaHubRetrieveVideo → SaveVideo.
• The provider and task_id outputs connect to the existing MediaHubVideoQueryStatus and MediaHubRetrieveVideo nodes.
• Image inputs are per-mode and all optional:
  • first_frame_image: used by i2v and fl2v modes.
  • last_frame_image: used by fl2v mode only.
  • subject_image: used by s2v mode only.
  • Python validation raises if a required image for the selected mode is None.

I2V mode (Image-to-Video)

• first_frame_image: optional IMAGE input (required for i2v mode), converted to JPEG base64 data URL.
• prompt: optional text (max 2000 chars), supports camera control instructions.
• Models: MiniMax-Hailuo-2.3, MiniMax-Hailuo-2.3-Fast, MiniMax-Hailuo-02, I2V-01-Director, I2V-01-live, I2V-01.
• Resolution/duration constraints are model-dependent:
  • Hailuo-2.3/2.3-Fast: 768P (6s or 10s), 1080P (6s only).
  • Hailuo-02: 512P (6s or 10s), 768P (6s or 10s), 1080P (6s only).
  • I2V-01-Director/01-live/01: 720P (6s only).
• fast_pretreatment is available for Hailuo-2.3, Hailuo-2.3-Fast, and Hailuo-02.

FL2V mode (First-Last Frame)

• first_frame_image + last_frame_image: both optional IMAGE inputs (required for fl2v mode).
• prompt: optional text.
• Model is locked to MiniMax-Hailuo-02.
• Resolution: 768P (6s or 10s), 1080P (6s only). 512P is not supported for FL2V.
• Video dimensions follow the first frame image. Last frame is cropped to match.

S2V mode (Subject Reference)

• subject_image: optional IMAGE input (required for s2v mode, batch images become multiple refs). Does NOT need first_frame_image.
• prompt: required text (max 2000 chars).
• Model is locked to S2V-01.
• No resolution/duration/fast_pretreatment parameters.
• Subject reference uses {"type": "character", "image": [data_urls]} format (note: image array, not image_file string like I2I).

Frontend behavior

• mode changes update: model combo, resolution combo, duration combo, fast_pretreatment visibility, and image input visibility.
• In I2V mode: model/resolution/duration are dynamically constrained.
• In FL2V mode: model widget is hidden (locked to Hailuo-02).
• In S2V mode: model, resolution, duration, and fast_pretreatment are all hidden.
• All hidden widgets use computeSize + non-drawing hidden type to preserve serialization.

Verification Status

Release-candidate verification on 2026-06-13:

• Comfy Registry publish workflow run 27470314727 completed successfully; Registry API reports mediahub-cloud as active and exposes the 0.1.0 version package while its version status is initially pending.
• The published CDN archive downloaded successfully, contained the expected 55 files, and contained no credentials, tests, scripts, internal docs, or output media.
• A user-run sample with a real provider API key completed image generation successfully. This confirms the sampled live image path and credential flow; it does not claim live verification for every provider or video operation.
• 221 pytest tests passed locally and in GitHub Actions on Python 3.10 and Python 3.12.
• Python compilation, JavaScript syntax, workflow JSON validation, tracked-file credential scanning, Registry metadata validation, and package inspection passed.
• A restarted ComfyUI 0.24.0 backend accepted every API workflow fixture.

Verified during the provider-isolation and Agnes integration on 2026-06-13:

• 120 pytest tests passed before the workflow-layout follow-up.
• Provider-isolation architecture tests passed.
• Python compilation, JavaScript syntax, and all example/fixture JSON checks passed.
• Agnes Image 2.1 live T2I succeeded.
• Agnes Image 2.1 live I2I with Data URI succeeded only with top-level image; the copied example's extra_body.image returned HTTP 502.
• Agnes Video 2.0 live T2V reached completed; the final URL field was remixed_from_video_id.
• Agnes Video 2.0 live I2V accepted a ComfyUI-style Data URI input and entered in_progress.
• ComfyUI backend was restarted, loaded the isolated provider architecture, and scripts/smoke_comfyui.py confirmed all eight API workflows were accepted.

Verified during the workflow-layout and media-family-isolation follow-up on 2026-06-13:

• 124 pytest tests passed.
• All eight user-facing workflows use the canonical input/config/output layout.
• All sample connections flow left to right through a portable middle integration block.
• The API Key node is compact and keeps its password field and visibility control on one row.
• A restarted ComfyUI backend exposed provider config nodes under MediaHub/Provider/<Provider>/<Media Family> and accepted all eight API workflow structures.
• Agnes image and video modules no longer import each other; shared Agnes errors live in mediahub/providers/agnes/common.py.
• Architecture tests prohibit sibling image/video provider-module imports.

Verified on the user's running ComfyUI environment on 2026-06-13 (T2I):

• 23 pytest tests passed.

• Python compilation, JavaScript syntax checks, the UI workflow JSON check, and the API prompt fixture JSON check passed.

• ComfyUI backend was restarted and loaded the new INPUT_TYPES.

• scripts/smoke_comfyui.py passed against http://127.0.0.1:8000.

• examples/minimax_cn_t2i_preview.json was actually imported into the ComfyUI frontend and queued.

• Both API and UI workflow smoke tests reached only the expected empty-key error:

  ValueError: MiniMax CN API key is required
  

• No parameter-validation or positional widget-alignment errors remained.

• The live API Key input was confirmed to render as an HTML password input.

Verified after the latest prompt-decoupling and dynamic-layout changes:

• MiniMax CN's current official OpenAPI document was checked again. The node exposes every documented request property.
• 24 pytest tests, JavaScript syntax, Python compilation, and both workflow JSON checks passed.
• A fresh ComfyUI 0.24.0 backend loaded the plugin.
• scripts/smoke_comfyui.py passed against the fresh backend and /prompt reached only the expected empty-key execution error.
• The user workflow was imported into the actual ComfyUI frontend.
• The prompt is connected from native PrimitiveStringMultiline.
• image-01 aspect-ratio and custom-width/height states resize correctly.
• image-01-live exposes use_style, and only exposes style_type and style_weight after the style switch is enabled.
• No dynamic parameter renders outside the T2I node boundary.

Verified after I2I implementation:

• MiniMax CN I2I API documentation was researched. The endpoint, models, auth, and response format are identical to T2I. The additional parameter is subject_reference (array of character reference objects).

• 49 pytest tests passed (up from 25).

• JavaScript syntax, Python compilation, and all four workflow JSON checks passed.

• ComfyUI backend was restarted and loaded MediaHubImageToImage.

• scripts/smoke_comfyui.py validates both T2I and I2I workflows and both reached only the expected empty-key error:

  ValueError: MiniMax CN API key is required
  

  The I2I error at nodes.py:257 confirms the I2I execution path was exercised.

• The I2I node accepts a ComfyUI IMAGE input, converts each image to a JPEG base64 data URL, and builds the subject_reference array.

• Multiple images in a batch become multiple character references.

Verified after T2V pipeline implementation (2026-06-13):

• MiniMax CN T2V API documentation was researched. The submit, query, and file retrieval endpoints are documented. All model/resolution/duration constraints are validated.
• T2V is decomposed into three composable nodes: MediaHubT2VSubmit, MediaHubVideoQueryStatus (status indicator / trigger), and MediaHubRetrieveVideo.
• 98 pytest tests passed (up from 49).
• JavaScript syntax, Python compilation, and all six workflow JSON checks passed.
• Submit node exposes every documented T2V request field: model, prompt, duration, resolution, prompt_optimizer, fast_pretreatment, and aigc_watermark.
• Frontend dynamically constrains resolution and duration combo values based on model selection. fast_pretreatment is only visible for Hailuo models.
• Each T2V node's contract and API interaction is covered by unit tests with mocked HTTP.
• T2V smoke fixture (three-node pipeline) and UI workflow were validated.

Verified after I2V pipeline implementation (2026-06-13):

• MiniMax CN I2V, FL2V, and S2V API documentation was researched. All three modes use the same POST /v1/video_generation endpoint and the same async query/retrieve flow as T2V.

• 134 pytest tests passed (up from 98). I2V tests cover all three modes, model/resolution/duration constraints, fast_pretreatment, validation errors, and the submit_i2v_task HTTP interaction.

• JavaScript syntax, Python compilation, and all workflow JSON checks passed.

• ComfyUI backend was restarted and loaded MediaHubI2VSubmit.

• scripts/smoke_comfyui.py validates T2I, I2I, T2V, and I2V workflows and all four reached only the expected empty-key error:

  ValueError: MiniMax CN API key is required
  

  The I2V error at nodes.py:623 confirms the I2V execution path was exercised.

• examples/minimax_cn_i2v_preview.json UI workflow created with LoadImage → I2V Submit → VideoQueryStatus → RetrieveVideo → SaveVideo pipeline. Default mode is i2v with MiniMax-Hailuo-2.3 at 768P/6s. Only first_frame_image is connected; last_frame_image and subject_image are optional and mode-dependent.

• MediaHubT2VQueryStatus renamed to MediaHubVideoQueryStatus; MediaHubT2VRetrieveVideo renamed to MediaHubRetrieveVideo. Legacy aliases preserved in NODE_CLASS_MAPPINGS and NODE_DISPLAY_NAME_MAPPINGS for backward-compatible workflow loading.

• T2V and I2V API fixtures now include SaveVideo output nodes (required by ComfyUI 0.24.0's prompt_no_outputs validation).

• The I2V node outputs ("provider", "task_id", "raw_json") matching the T2V Submit contract, so it reuses MediaHubVideoQueryStatus and MediaHubRetrieveVideo without modification.

The remaining unverified boundary is a paid MiniMax CN request using a real API key for T2I, I2I, T2V, and I2V.

Verified during Seed Volcengine provider implementation (2026-06-13):

• Seed Volcengine provider implemented: mediahub/providers/seed_volcengine/ with image.py (Seedream T2I/I2I) and video.py (Seedance T2V/I2V).
• Provider registers 9 operations: image.t2i.build, image.i2i.build, image.generate, video.t2v.build, video.i2v.build, video.submit, video.query, video.state, video.final_url.
• 4 config nodes added: SeedVolcengineT2IConfig, SeedVolcengineI2IConfig, SeedVolcengineT2VConfig, SeedVolcengineI2VConfig.
• JS dynamic widgets: image nodes constrain size by model; video nodes constrain resolution and duration by model; generate_audio visibility toggles by model; I2V last_frame_image toggles by mode.
• 170 pytest tests passed (up from 124). 46 new Seed Volcengine tests cover payload construction, validation, HTTP interaction, state mapping, and URL extraction.
• Architecture isolation tests passed (no cross-provider imports, no comfy imports in providers, no image/video cross-imports).
• Python compilation and JavaScript syntax checks passed.
• All workflow JSON files validated.
• Seed Volcengine uses ARK_API_KEY (Bearer token), not AK/SK.
• Seedream image API is synchronous (no async polling needed).
• Seedance video API query returns content.video_url directly (no separate file retrieval step needed). The generic resolve_video() handles this through the video.final_url registry operation.
• Video status mapping: succeeded → completed, failed/expired/ cancelled → failed, everything else → pending.

Remaining unverified boundaries:

• Paid Seed Volcengine live verification with a real ARK_API_KEY.

Verified during release-metadata preparation (2026-06-13):

• The immutable Registry node ID is planned as mediahub-cloud; Registry publisher vantang is active.
• MIT license, Registry metadata, English README, Chinese README, contribution guide, generated logo/banner assets, and .comfyignore were prepared.
• Seed Volcengine I2V now declares mode-dependent last_frame_image as an optional ComfyUI input; runtime validation still requires it for first_last_frame.
• 221 pytest tests passed.
• Python compilation, JavaScript syntax, and all workflow JSON checks passed.
• A restarted ComfyUI 0.24.0 backend loaded the current provider set, including Bailian CN.
• scripts/smoke_comfyui.py confirmed all API workflow structures were accepted by the restarted backend.

Random Generation Cache Behavior

Confirmed from the live ComfyUI log:

• A normal MiniMax request took about 20 seconds.
• Re-queuing the identical workflow with use_seed=false completed in 0.00s, proving ComfyUI returned its cached node output and did not make a second provider request.

Both T2I and I2I nodes implement ComfyUI's IS_CHANGED contract:

• use_seed=false: returns a non-cacheable fingerprint so every Queue action sends a fresh MiniMax request.
• use_seed=true: permits normal ComfyUI caching when all inputs remain unchanged.

The T2V node always returns NaN from IS_CHANGED because the T2V API does not expose a seed parameter. Every Queue action triggers a new video generation request.

Known Unrelated Environment Errors

ComfyUI startup currently logs:

AttributeError: _ARRAY_API not found

This is caused by numpy 2.4.1 with old opencv-python-headless 4.7.0.72. It occurs before custom-node imports and is not caused by MediaHub.

ComfyUI Manager also logs network failures reaching Registry/GitHub. These affect Manager metadata refresh, not MediaHub node loading.

Handoff Checklist

The Seed Volcengine provider implementation is complete. The next bounded task depends on the user's direction. Possible next tasks:

1. Seed Volcengine live verification: restart ComfyUI, run smoke test, create UI workflow examples, and test with a real ARK_API_KEY.
2. Seed Volcengine UI workflow examples (T2I, I2I, T2V, I2V) and API smoke fixtures.
3. Paid live verification of MiniMax CN T2I, I2I, T2V, and I2V with a real API key.
4. MiniMax Global provider (separate endpoint, keys, and possible field differences).
5. Secure local credential storage.
6. Jimeng provider (separate AK/SK auth, different API structure).