ttulttul-ComfyUI-Modal/docs/ARCHITECTURE.md

Architecture: Module Ownership

This page maps each subsystem to the module that owns it. The README covers how to use the extension. LEARNINGS.md records why things work the way they do.

The prompt interception layer is organized as a compatibility aggregator over focused modules: planning values, graph analysis, component planning, prompt rewriting, execution scheduling, queue lifecycle, diagnostics, and HTTP route domains each have a dedicated owner. Route handlers share an immutable RouteContext, while api_intercept.py retains only registration and compatibility re-exports. Host-side remote execution follows the same ownership model: deployment loading, runtime compatibility checks, and auto-deploy state live in remote/modal_deployment.py; proactive warmup, snapshot profiles, and local-gap keepalive state live in remote/modal_warmup.py; mapped or implicit-batch lane scheduling lives in remote/mapped_execution.py; and CPU ModelStager state lives in remote/modal_llm_profile_staging.py. Adjacent invocation and staging callbacks are injected by the remote/modal_app.py orchestrator. Blocking Modal executors are constructed lazily on first use, and importing the host orchestrator no longer constructs a vestigial Modal app or volume. Cloud helpers receive tail-created Modal dictionaries and volumes through cloud_runtime_context.py, active request cleanup through cloud_execution_control.py, and timestamped phase reporting through cloud_runtime_logging.py, allowing focused cloud subsystems to move without reaching back into entry-module globals; pre-deploy app collision checks live in cloud_app_guard.py, image filters and worker construction policy live in cloud_image_env.py, durable invocation replay and live-canary coordination live in cloud_durable_invocation.py, session-bridge storage and replay live in cloud_session_bridge.py, headless ComfyUI server compatibility plus progress tracing live in cloud_prompt_server_shims.py, ComfyUI initialization, custom-node loading, asset path translation, compatibility patches, and warm loader caches live in cloud_comfy_bootstrap.py, distributed node-output key construction, serialization, restore, and write-back live in cloud_node_output_cache.py, prompt schema validation and normalization live in cloud_prompt_validation.py, reusable executor management and local subgraph execution live in cloud_prompt_execution.py, mapped/phased cloud execution lives in cloud_mapped_execution.py, bounded progress/result delivery plus abandoned invocation cleanup live in cloud_streaming.py, mounted-volume visibility and reload recovery live in cloud_volume_reload.py, and snapshot, loader, LLM, and request prewarming live in cloud_prewarm.py. Container placement and snapshot-restore diagnostics (Modal container startup kind=boot|snapshot-created|restore ...) live in cloud_container_placement.py. The stable comfyui_modal_sync_cloud.py entrypoint imports only the hooks and operations it executes; legacy reads of extracted private names resolve lazily to their owning module. Root modules support both package imports used by ComfyUI and flat imports used by the Modal container; tests/test_import_shapes.py guards both layouts and the stable cloud surface. The integration tests mirror those ownership boundaries in focused files and keep shared doubles and builders in non-collected *_test_support.py modules.

Execution scheduling keeps provider-neutral orchestration in execution_scheduling.py while resource estimation and history signatures live in execution_resource_estimates.py, credential-safe plan rendering and its R2 usage cache live in execution_plan_reporting.py, and runtime backend construction plus assignment stamping live in execution_assignment_runtime.py.

Prompt rewriting keeps proxy construction in prompt_rewrite.py, while prompt_payload_building.py owns the ordered, hybrid, and single-phase payload builders; downstream frontier discovery, parallel local-branch release, local-gap keepalive, and speculative affinity prewarm decoration live in prompt_affinity_planning.py.

Remote graph analysis keeps transportability and component partitioning in remote_graph_analysis.py, while recursive saved-workflow traversal and workflow-path-to-queued-prompt resolution live in workflow_prompt_mapping.py.

The stable api_intercept.py entrypoint now limits itself to compatibility exports, route context construction, and route registration. Queue-time selection, capacity planning coordination, bounded per-environment asset preparation, and proxy rewrite orchestration live in prompt_interception.py.

The host-side Modal orchestrator delegates prompt cancellation state and propagation to remote/modal_interrupts.py, and streamed remote event decoding plus local UI forwarding to remote/payload_stream.py. remote/modal_app.py retains invocation, recovery, materialization, and public entrypoint coordination.

Resident LLM inference uses the same ownership boundaries: llm_types.py defines backend-neutral values, llm_inputs.py normalizes multimodal and file inputs, vllm_instrumentation.py owns process-wide vLLM execution policy and Triton telemetry, and llm_backend_transformers.py, llm_backend_llamacpp.py, and llm_backend_vllm.py isolate backend implementations. modal_llm_runtime.py remains the model-residency coordinator, while local_llm_runtime.py consumes shared types and input preparation without importing private helpers from the Modal backend. CPU-side staging adapts the general cross-process ownership lock in snapshot_lease.py to LLM-specific progress events.

Asset synchronization is composed from focused collaborators: sync_protocols.py holds provider-neutral contracts and values, sync_backends.py owns local and Modal persistence adapters, sync_hashing.py owns the persistent content-hash cache, sync_custom_nodes.py builds and transfers custom-node archives and manifests, and sync_r2_transfer.py owns worker preflight plus idle-gated R2 read-through and write-back. sync_engine.py coordinates those collaborators and retains compatibility entrypoints.

Proxy execution separates provider routing (remote_executor_router.py), run-scoped payload normalization and cache-safe registries (proxy_payloads.py), and dynamic v3 proxy class construction (proxy_node_factory.py). modal_executor_node.py retains workflow concurrency gates and the five static ComfyUI node classes, then injects those residual execution hooks into the factory.