diff --git a/README.ja.md b/README.ja.md new file mode 100644 index 0000000..cb96d4c --- /dev/null +++ b/README.ja.md @@ -0,0 +1,61 @@ +# ComfyUI-FramePackWrapper_PlusOne + +[ComfyUI-FramePackWrapper_PlusOne](https://github.com/tori29umai0123/ComfyUI-FramePackWrapper_PlusOne) は、[ComfyUI-FramePackWrapper](https://github.com/kijai/ComfyUI-FramePackWrapper)および[ComfyUI-FramePackWrapper_Plus](https://github.com/ShmuelRonen/ComfyUI-FramePackWrapper_Plus)から派生した、FramePackの1フレーム推論ノード(kisekaeichi対応)を含むフォークです。 + +本リポジトリは、 @tori29umai0123 氏の[依頼を受けて](https://x.com/tori29umai/status/1928692381735432320)公開用にフォークしました。 + +## 機能 + +- **1フレーム推論**: 基本的な1フレーム推論および、kisekaeichi方式に対応しています。技術的詳細は[musubi-tunerのドキュメント](https://github.com/kohya-ss/musubi-tuner/blob/main/docs/framepack_1f.md)を参照してください。 +- **F1サンプラー対応**: より高品質で時間的一貫性の高い動画生成のための改良されたF1方式を採用 +- **LoRA統合**: 適切な重み付けと融合オプションを備えたHunyuanVideo LoRAの完全サポート +- **タイムスタンプ付きプロンプト**: 特定のタイムスタンプでプロンプトを変更できる動的動画の作成 +- **柔軟な入力オプション**: 参照画像と空の潜在空間の両方で完全なクリエイティブコントロールが可能 +- **解像度制御**: 最適な動画サイズのための自動バケット検出 +- **ブレンド制御**: タイムスタンプ間の異なるプロンプトのスムーズな遷移 + +### Not yet supported + +- 1フレーム推論のうち、f-mc (one frame multi-control) 方式は未対応です。 + +## インストール + +1. このリポジトリをComfyUIのcustom_nodesフォルダにクローンします: +```bash +cd ComfyUI/custom_nodes +git clone https://github.com/xhiroga/ComfyUI-FramePackWrapper_PlusOne.git +``` + +2. 必要な依存パッケージをインストールします: +```bash +pip install -r requirements.txt +``` + +3. 必要なモデルファイルをダウンロードして、modelsフォルダに配置します: +- FramePackI2V_HY: [HuggingFace Link](https://huggingface.co/lllyasviel/FramePackI2V_HY) +- FramePack_F1_I2V_HY: [HuggingFace Link](https://huggingface.co/lllyasviel/FramePack_F1_I2V_HY_20250503) + +## モデルファイル + +### メインモデルオプション +- [FramePackI2V_HY_fp8_e4m3fn.safetensors](https://huggingface.co/Kijai/HunyuanVideo_comfy/blob/main/FramePackI2V_HY_fp8_e4m3fn.safetensors) - Optimized fp8 version (smaller file size) +- [FramePackI2V_HY_bf16.safetensors](https://huggingface.co/Kijai/HunyuanVideo_comfy/blob/main/FramePackI2V_HY_bf16.safetensors) - BF16 version (better quality) + +### 必要なコンポーネント +- **CLIP Vision**: [sigclip_vision_384](https://huggingface.co/Comfy-Org/sigclip_vision_384/tree/main) +- **Text Encoder and VAE**: [HunyuanVideo_repackaged](https://huggingface.co/Comfy-Org/HunyuanVideo_repackaged/tree/main/split_files) + +## Usage + +[example_workflows](./example_workflows)を参照ください。 + +## License + +[MIT License](LICENSE) + +## クレジット + +- [FramePack](https://github.com/lllyasviel/Fooocus-FramePack): @lllyasviel 氏によるオリジナルの実装です。 +- [ComfyUI-FramePackWrapper](https://github.com/kijai/ComfyUI-FramePackWrapper): @kijai 氏によるオリジナルの実装です。 +- [ComfyUI-FramePackWrapper_Plus](https://github.com/ShmuelRonen/ComfyUI-FramePackWrapper_Plus): @ShmuelRonen 氏によるF1対応のフォークです。 +- [ComfyUI-FramePackWrapper_PlusOne](https://github.com/tori29umai0123/ComfyUI-FramePackWrapper_PlusOne): @tori29umai0123 氏による1フレーム推論対応のフォークです。 diff --git a/README.md b/README.md index 594acb7..021ef1b 100644 --- a/README.md +++ b/README.md @@ -1,18 +1,14 @@ # ComfyUI-FramePackWrapper_PlusOne -[ComfyUI-FramePackWrapper_PlusOne](https://github.com/tori29umai0123/ComfyUI-FramePackWrapper_PlusOne) is a fork by @tori29umai0123 derived from [ComfyUI-FramePackWrapper](https://github.com/kijai/ComfyUI-FramePackWrapper) and [ComfyUI-FramePackWrapper_Plus](https://github.com/ShmuelRonen/ComfyUI-FramePackWrapper_Plus/tree/main), which includes a 1-frame inference node for FramePack (compatible with kisekaeichi). This repository was forked for public release at the request of @tori29umai0123. +**[日本語](./README.ja.md)** -
-日本語 +[ComfyUI-FramePackWrapper_PlusOne](https://github.com/tori29umai0123/ComfyUI-FramePackWrapper_PlusOne) is a fork derived from [ComfyUI-FramePackWrapper](https://github.com/kijai/ComfyUI-FramePackWrapper) and [ComfyUI-FramePackWrapper_Plus](https://github.com/ShmuelRonen/ComfyUI-FramePackWrapper_Plus), containing FramePack's single-frame inference node (with kisekaeichi support). -[ComfyUI-FramePackWrapper_PlusOne](https://github.com/tori29umai0123/ComfyUI-FramePackWrapper_PlusOne) は、[ComfyUI-FramePackWrapper](https://github.com/kijai/ComfyUI-FramePackWrapper)および[ComfyUI-FramePackWrapper_Plus](https://github.com/ShmuelRonen/ComfyUI-FramePackWrapper_Plus/tree/main)から派生した、 @tori29umai0123 氏によるFramePackの1フレーム推論ノード(kisekaeichi対応)を含むフォーク -です。本リポジトリは、 @tori29umai0123 氏の依頼を受けて公開用にフォークしました。 -
- -An improved wrapper for the FramePack project that allows the creation of videos of any length based on reference images and LoRAs with F1 sampler. +This repository was forked for public release at the request of @tori29umai0123 [as requested here](https://x.com/tori29umai/status/1928692381735432320). ## Features +- **1-Frame Inference**: Supports basic single frame inference and the kisekaeichi method. For technical details, please refer to the [musubi-tuner documentation](https://github.com/kohya-ss/musubi-tuner/blob/main/docs/framepack_1f.md). - **F1 Sampler Support**: Uses the improved F1 video generation method for higher quality and better temporal coherence - **LoRA Integration**: Full support for HunyuanVideo LoRAs with proper weight handling and fusion options - **Timestamped Prompts**: Create dynamic videos with changing prompts at specific timestamps @@ -20,12 +16,16 @@ An improved wrapper for the FramePack project that allows the creation of videos - **Resolution Control**: Automatic bucket finding for optimal video dimensions - **Blend Control**: Smooth transitions between different prompts at timestamps +### Not yet supported + +- 1-Frame Inference: f-mc (one frame multi-control) is not supported yet. + ## Installation 1. Clone this repository into your ComfyUI custom_nodes folder: ```bash cd ComfyUI/custom_nodes -git clone https://github.com/tori29umai0123/ComfyUI-FramePackWrapper_PlusOne.git +git clone https://github.com/xhiroga/ComfyUI-FramePackWrapper_PlusOne.git ``` 2. Install the required dependencies: @@ -49,96 +49,7 @@ pip install -r requirements.txt ## Usage -### Basic Workflow - -1. Load the HunyuanVideo model with your preferred settings -2. (Optional) Add LoRAs with the FramePackLoraSelect node -3. Prepare your input image or empty latent -4. Set up CLIP Vision encoding for image embeddings -5. Create timestamped prompts with FramePackTimestampedTextEncode -6. Generate your video with FramePackSampler (F1) - -### Example Workflow - -![image](https://github.com/user-attachments/assets/e7ba12b5-41ef-484b-a796-801b701628a5) - -### Timestamped Prompts - -Use the following format for timestamped prompts: -``` -[0s: A beautiful landscape, mountains in the background] -[5s-10s: Camera pans to reveal a lake, reflections of clouds] -[10s: A boat appears on the horizon, sailing slowly] -``` - -- `[Xs: prompt]`: Starts at X seconds and continues until the next timestamp -- `[Xs-Ys: prompt]`: Active from X seconds to Y seconds - -### LoRA Usage - -1. Place your HunyuanVideo LoRAs in the `ComfyUI/models/loras` folder -2. Use the FramePackLoraSelect node to add them to your workflow -3. Adjust strength as desired (typically 0.5-1.2) -4. Set fuse_lora to false for flexibility or true for performance - -## Node Reference - -### FramePackSampler (F1) -The main generation node using the F1 sampling technique. - -**Inputs:** -- `model`: The loaded FramePack model -- `positive_timed_data`: Timestamped positive prompts -- `negative`: Negative prompt conditioning -- `start_latent`: Initial latent for generation -- `start_image_embeds`: CLIP Vision embeddings for start image -- `end_latent`: (Optional) End latent for transitions -- `end_image_embeds`: (Optional) CLIP Vision embeddings for end image -- `initial_samples`: (Optional) For video-to-video generation -- Various sampling parameters (steps, CFG, guidance scale, etc.) - -### FramePackTimestampedTextEncode -Encodes text prompts with timestamps for timed conditioning. - -**Inputs:** -- `clip`: CLIP text model -- `text`: Text prompt with timestamps -- `negative_text`: Negative prompt -- `total_second_length`: Video duration in seconds -- `latent_window_size`: Processing window size -- `prompt_blend_sections`: Number of sections to blend prompts - -### FramePackLoraSelect -Selects and configures LoRA models. - -**Inputs:** -- `lora`: LoRA model selection -- `strength`: LoRA strength (0.0-2.0) -- `fuse_lora`: Whether to fuse the LoRA weights into the base model -- `prev_lora`: (Optional) For chaining multiple LoRAs - -### LoadFramePackModel / DownloadAndLoadFramePackModel -Loads the FramePack model with various precision options. - -## Advanced Tips - -1. **Resolution Control**: Use the FramePackFindNearestBucket node to optimize dimensions -2. **Memory Management**: Adjust gpu_memory_preservation for large models -3. **Blending Prompts**: Set prompt_blend_sections > 0 for smooth transitions -4. **Multiple LoRAs**: Chain several LoRAs together for combined effects -5. **Empty Latents**: Use an Empty Latent Image node when starting from scratch - -## Troubleshooting - -- **CUDA Out of Memory**: Reduce resolution, decrease latent_window_size, or increase gpu_memory_preservation -- **LoRA Loading Issues**: Ensure LoRAs are in the correct format (safetensors) -- **Video Artifacts**: Try increasing steps or adjusting CFG/guidance_scale - -## Acknowledgements - -- Based on the original [ComfyUI-FramePackWrapper](https://github.com/kijai/ComfyUI-FramePackWrapper) by kijai -- Uses models from [FramePack](https://github.com/lllyasviel/Fooocus-FramePack) by lllyasviel -- Special thanks to the ComfyUI and Stable Diffusion communities +See [example_workflows](./example_workflows). ## License @@ -146,12 +57,7 @@ Loads the FramePack model with various precision options. ## Credits -This project is an extension of the original [ComfyUI-FramePackWrapper](https://github.com/kijai/ComfyUI-FramePackWrapper) created by kijai. The original wrapper provided the foundation for working with FramePack models in ComfyUI. - -ComfyUI-FramePackWrapper_Plus builds upon that foundation by adding support for: -- F1 sampler for improved temporal coherence -- LoRA integration for customized generation -- Timestamped prompts for dynamic video creation -- Additional workflow improvements and optimizations - -Special thanks to kijai for the original implementation that made this extension possible. +- [FramePack](https://github.com/lllyasviel/Fooocus-FramePack): @lllyasviel's original implementation. +- [ComfyUI-FramePackWrapper](https://github.com/kijai/ComfyUI-FramePackWrapper): @kijai's original implementation. +- [ComfyUI-FramePackWrapper_Plus](https://github.com/ShmuelRonen/ComfyUI-FramePackWrapper_Plus): @ShmuelRonen's F1-supported fork. +- [ComfyUI-FramePackWrapper_PlusOne](https://github.com/tori29umai0123/ComfyUI-FramePackWrapper_PlusOne): @tori29umai0123's 1-frame inference-supported fork.