58 lines
2.1 KiB
Markdown
58 lines
2.1 KiB
Markdown
# ComfyUI-STARFlow
|
|
|
|
ComfyUI nodes for Apple's STARFlow text-to-image model and STARFlow-V text/image-to-video model.
|
|
|
|
## Installation
|
|
|
|
Clone this repository into `ComfyUI/custom_nodes/ComfyUI-STARFlow`, install `requirements.txt` in ComfyUI's Python environment, and restart ComfyUI.
|
|
|
|
## STARFlow text-to-image
|
|
|
|
### Models
|
|
|
|
- Checkpoint: place `starflow_3B_t2i_256x256.safetensors` in `ComfyUI/models/starflow/`.
|
|
- Text encoder: place FLAN-T5-XL encoder weights in `ComfyUI/models/text_encoders/`.
|
|
- VAE: use `sd-vae-ft-ema.safetensors` with the core `VAELoader`. This model was trained with `stabilityai/sd-vae-ft-ema`.
|
|
|
|
### Nodes
|
|
|
|
- `STARFlowCheckpointLoader`
|
|
- `STARFlowT5TextEncode`
|
|
- `STARFlowSampler`
|
|
- `STARFlowVAEDecode`
|
|
|
|
Example: `workflows/starflow_basic_t2i.json`.
|
|
|
|
## STARFlow-V text-to-video and image-to-video
|
|
|
|
### Models
|
|
|
|
- Checkpoint: place `starflow-v_7B_t2v_caus_480p_v3.safetensors` in `ComfyUI/models/starflow_v/`.
|
|
- Text encoder: place a complete local `google/flan-t5-xl` directory in `ComfyUI/models/text_encoders/flan-t5-xl/`.
|
|
- VAE: use a Wan2.2 TI2V VAE with the core `VAELoader`.
|
|
|
|
The checkpoint loader accepts the shipped single safetensors file directly. Legacy `.pth` and sharded safetensors checkpoints remain supported. `scripts/convert_checkpoint.py` converts a tensor-only `.pth` checkpoint to verified safetensors shards.
|
|
|
|
### Nodes
|
|
|
|
- `STARFlowVCheckpointLoader`
|
|
- `STARFlowVTextEncoderLoader`
|
|
- `STARFlowVTextEncode`
|
|
- `STARFlowVSampler`
|
|
- `STARFlowVDecode`
|
|
|
|
Examples:
|
|
|
|
- `workflows/starflow_v_t2v.json`
|
|
- `workflows/starflow_v_i2v.json`
|
|
|
|
The video examples use the released inference profile: 848x480 (the nearest 16-aligned 16:9 size), 81 frames at 16 FPS, CFG 3.5, Jacobi block size 32, threshold 0.001, and the learnable denoiser enabled. Width and height must be divisible by 16, and frame counts must follow `4n+1`.
|
|
|
|
## Model repository
|
|
|
|
Converted checkpoints and checksums are published at https://huggingface.co/AkaneTendo25/starflow.
|
|
|
|
## License
|
|
|
|
See `LICENSE`, `LICENSE_MODEL`, and `ACKNOWLEDGMENTS`. The implementation is based on Apple's `ml-starflow` release.
|