first version finished
This commit is contained in:
@@ -1,354 +1,85 @@
|
||||
# ComfyUI-Grounding
|
||||
|
||||
ComfyUI custom nodes for open-vocabulary object detection using state-of-the-art grounding models.
|
||||
**Grounding for dummies, simplest workflow**
|
||||
|
||||
## Features
|
||||

|
||||
|
||||
- **GroundingDINO**: Open-vocabulary object detection with natural language prompts
|
||||
- **YOLO-World**: Real-time open-vocabulary detection
|
||||
- **Batch Processing**: Process multiple images at once for improved efficiency
|
||||
- **Referring Expression Comprehension (REC)**: Use full text as label when no separators present
|
||||
- **Flexible Architecture**: Support for both local and HuggingFace implementations
|
||||
- **Utility Nodes**: Bounding box visualization and mask conversion
|
||||
🎯 **3 Nodes Total** - Loader → Detector → Visualizer
|
||||
🚀 **5 Model Families** - GroundingDINO, MM-GroundingDINO, OWLv2, Florence-2, YOLO-World
|
||||
💾 **Smart Caching** - Instant reload
|
||||
📦 **Batch Processing** - Multiple images at once
|
||||
🎭 **Built-in Masks** - No separate BboxToMask node needed
|
||||
|
||||
## Supported Models
|
||||
## Quick Start
|
||||
|
||||
### GroundingDINO (Original)
|
||||
- GroundingDINO-T (SwinT backbone, 694MB)
|
||||
- GroundingDINO-B (SwinB backbone, 938MB)
|
||||
|
||||
### MM-GroundingDINO (Multimodal)
|
||||
**Tiny Models (Swin-T backbone):**
|
||||
- MM-GroundingDINO-Tiny (O365+GoldG, 50.4 mAP)
|
||||
- MM-GroundingDINO-Tiny (O365+GoldG+GRIT, 50.5 mAP)
|
||||
- MM-GroundingDINO-Tiny (O365+GoldG+V3Det, 50.6 mAP)
|
||||
- MM-GroundingDINO-Tiny (O365+GoldG+GRIT+V3Det, 50.4 mAP)
|
||||
|
||||
**Base Models (Swin-B backbone):**
|
||||
- MM-GroundingDINO-Base (O365+GoldG+V3Det, 52.5 mAP)
|
||||
- MM-GroundingDINO-Base (All datasets, 59.5 mAP) ⭐ Recommended
|
||||
|
||||
**Large Models (Swin-L backbone):**
|
||||
- MM-GroundingDINO-Large (O365v2+OIv6+GoldG, 53.0 mAP)
|
||||
- MM-GroundingDINO-Large (All datasets, 60.3 mAP) ⭐ Best accuracy
|
||||
|
||||
### YOLO-World
|
||||
- YOLOv8s-World (small)
|
||||
- YOLOv8m-World (medium)
|
||||
- YOLOv8l-World (large)
|
||||
- YOLOv8x-World (extra large)
|
||||
|
||||
## Installation
|
||||
|
||||
1. Navigate to your ComfyUI custom nodes directory:
|
||||
```bash
|
||||
cd ComfyUI/custom_nodes/
|
||||
```
|
||||
|
||||
2. Clone or copy this repository:
|
||||
```bash
|
||||
git clone <repository-url> ComfyUI-Grounding
|
||||
```
|
||||
|
||||
3. Install dependencies:
|
||||
```bash
|
||||
cd ComfyUI-Grounding
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Nodes
|
||||
## The Nodes
|
||||
|
||||
### 1. GroundingDINOModelLoader
|
||||
Loads a GroundingDINO or MM-GroundingDINO model for object detection.
|
||||
### 1. Grounding Model Loader
|
||||
Load any of 10+ models from a single dropdown.
|
||||
|
||||
**Inputs:**
|
||||
- `model_name`: Select from available GroundingDINO/MM-GroundingDINO models
|
||||
- `keep_in_memory`: Cache model in memory for faster reuse (default: True)
|
||||
1. GroundingDINO: SwinT OGC (694MB) - IDEA-Research/grounding-dino-tiny
|
||||
2. GroundingDINO: SwinB (938MB) - IDEA-Research/grounding-dino-base
|
||||
3. MM-GroundingDINO: Tiny O365+GoldG (50.4 mAP) - openmmlab-community/mm_grounding_dino_tiny_o365v1_goldg
|
||||
4. MM-GroundingDINO: Tiny O365+GoldG+GRIT (50.5 mAP) - openmmlab-community/mm_grounding_dino_tiny_o365v1_goldg_grit
|
||||
5. MM-GroundingDINO: Tiny O365+GoldG+V3Det (50.6 mAP) - openmmlab-community/mm_grounding_dino_tiny_o365v1_goldg_v3det
|
||||
6. MM-GroundingDINO: Base O365+GoldG+V3Det (52.5 mAP) - openmmlab-community/mm_grounding_dino_base_o365v1_goldg_v3det
|
||||
7. MM-GroundingDINO: Base All Datasets (59.5 mAP) - openmmlab-community/mm_grounding_dino_base_all
|
||||
8. MM-GroundingDINO: Large O365v2+OIv6+GoldG (53.0 mAP) - openmmlab-community/mm_grounding_dino_large_o365v2_oiv6_goldg
|
||||
9. MM-GroundingDINO: Large All Datasets (60.3 mAP) - openmmlab-community/mm_grounding_dino_large_all
|
||||
10. OWLv2: Base Patch16 - google/owlv2-base-patch16
|
||||
11. OWLv2: Large Patch14 - google/owlv2-large-patch14
|
||||
12. OWLv2: Base Patch16 Ensemble - google/owlv2-base-patch16-ensemble
|
||||
13. OWLv2: Large Patch14 Ensemble - google/owlv2-large-patch14-ensemble ⭐
|
||||
14. Florence-2: Base (0.23B params) - microsoft/Florence-2-base ⚡ Best Balance
|
||||
15. Florence-2: Large (0.77B params) - microsoft/Florence-2-large ⭐
|
||||
16. YOLO-World: v8s (Small) - yolov8s-worldv2.pt ⚡ Fastest
|
||||
17. YOLO-World: v8m (Medium) - yolov8m-worldv2.pt
|
||||
18. YOLO-World: v8l (Large) - yolov8l-worldv2.pt
|
||||
19. YOLO-World: v8x (Extra Large) - yolov8x-worldv2.pt
|
||||
|
||||
**Outputs:**
|
||||
- `model`: Loaded GroundingDINO model
|
||||
|
||||
**Features:**
|
||||
- ✅ Model caching - loads instantly on second use
|
||||
- ✅ Local model storage in `ComfyUI-Grounding/Models/`
|
||||
- ✅ Supports both original and MM-GroundingDINO variants
|
||||
|
||||
### 2. GroundingDINODetector
|
||||
Detects objects in images using text prompts.
|
||||
|
||||
**Inputs:**
|
||||
- `model`: GroundingDINO model (from loader)
|
||||
- `image`: Input image (supports batches)
|
||||
- `prompt`: Text description of objects to detect (e.g., "person . car . dog .")
|
||||
- `box_threshold`: Confidence threshold for bounding boxes (0.0-1.0)
|
||||
- `text_threshold`: Confidence threshold for text matching (0.0-1.0)
|
||||
- `single_box_mode`: (Optional) Return only highest-scoring detection for REC
|
||||
- `output_format`: (Optional) "list_only" (default, SAM2-compatible) or "dict_with_data"
|
||||
|
||||
**Outputs:**
|
||||
- `bboxes`: Detected bounding boxes (format depends on output_format setting)
|
||||
- `list_only`: List of numpy arrays `[[boxes_img0], [boxes_img1], ...]` - SAM2-compatible
|
||||
- `dict_with_data`: Dict `{"boxes": [...], "labels": [...], "scores": [...]}`
|
||||
- `annotated_image`: Image with drawn bounding boxes (batched)
|
||||
- `labels`: String with detected object labels
|
||||
|
||||
**Features:**
|
||||
- ✅ Batch processing - process multiple images at once
|
||||
- ✅ Automatic REC mode - when prompt has no separators (. or ,), uses full text as label
|
||||
- ✅ Single box mode - returns only highest-scoring detection for referring expressions
|
||||
- ✅ Flexible output format - choose between SAM2-compatible list or full dict with metadata
|
||||
|
||||
### 3. YOLOWorldModelLoader
|
||||
Loads a YOLO-World model for real-time detection.
|
||||
|
||||
**Inputs:**
|
||||
- `model_name`: Select from available YOLO-World models
|
||||
- `keep_in_memory`: Cache model in memory for faster reuse (default: True)
|
||||
|
||||
**Outputs:**
|
||||
- `model`: Loaded YOLO-World model
|
||||
|
||||
**Features:**
|
||||
- ✅ Model caching - loads instantly on second use
|
||||
- ✅ Local model storage in `ComfyUI-Grounding/Models/`
|
||||
- ✅ Auto-downloads models on first use
|
||||
|
||||
### 4. YOLOWorldDetector
|
||||
Detects objects using YOLO-World with custom classes.
|
||||
|
||||
**Inputs:**
|
||||
- `model`: YOLO-World model (from loader)
|
||||
- `image`: Input image (supports batches)
|
||||
- `classes`: Comma-separated list of classes (e.g., "person, car, dog")
|
||||
- `confidence_threshold`: Minimum confidence for detections (0.0-1.0)
|
||||
- `single_box_mode`: (Optional) Return only highest-scoring detection for REC
|
||||
- `output_format`: (Optional) "list_only" (default, SAM2-compatible) or "dict_with_data"
|
||||
|
||||
**Outputs:**
|
||||
- `bboxes`: Detected bounding boxes (format depends on output_format setting)
|
||||
- `list_only`: List of numpy arrays `[[boxes_img0], [boxes_img1], ...]` - SAM2-compatible
|
||||
- `dict_with_data`: Dict `{"boxes": [...], "labels": [...], "scores": [...]}`
|
||||
- `annotated_image`: Image with drawn bounding boxes (batched)
|
||||
- `labels`: String with detected object labels
|
||||
|
||||
**Features:**
|
||||
- ✅ Batch processing - process multiple images at once
|
||||
- ✅ Automatic REC mode - when classes has no comma, uses full text as label
|
||||
- ✅ Single box mode - returns only highest-scoring detection for referring expressions
|
||||
- ✅ Flexible output format - choose between SAM2-compatible list or full dict with metadata
|
||||
|
||||
### 5. BboxVisualizer
|
||||
Draws bounding boxes on images.
|
||||
|
||||
**Inputs:**
|
||||
- `image`: Input image
|
||||
- `bboxes`: Bounding box data
|
||||
- `line_width`: Thickness of bounding box lines (optional)
|
||||
|
||||
**Outputs:**
|
||||
- `annotated_image`: Image with drawn bounding boxes
|
||||
|
||||
### 5. UnifiedDetector ⭐ NEW
|
||||
Universal detector that works with BOTH GroundingDINO and YOLO-World models.
|
||||
|
||||
**Inputs:**
|
||||
- `model`: Any model (GroundingDINO or YOLO-World)
|
||||
- `image`: Input image
|
||||
- `prompt`: Text prompt (supports both formats)
|
||||
- For GroundingDINO: "cat . dog . car ."
|
||||
- For YOLO-World: "cat, dog, car"
|
||||
- `threshold`: Confidence threshold (0.0-1.0)
|
||||
- `text_threshold`: Text matching threshold (optional, GroundingDINO only)
|
||||
|
||||
**Outputs:**
|
||||
- `bboxes`: Detected bounding boxes with labels and scores
|
||||
- `annotated_image`: Image with drawn bounding boxes
|
||||
- `labels`: String with detected object labels
|
||||
|
||||
**Features:**
|
||||
- ✅ Auto-detects model type
|
||||
- ✅ Single node for all detection models
|
||||
- ✅ Simplified workflow
|
||||
|
||||
### 6. BboxToMask
|
||||
Converts bounding boxes to binary masks.
|
||||
|
||||
**Inputs:**
|
||||
- `image`: Input image (for dimensions)
|
||||
- `bboxes`: Bounding box data
|
||||
- `combine_masks`: Whether to combine all boxes into one mask (optional)
|
||||
|
||||
**Outputs:**
|
||||
- `masks`: Binary mask(s) for detected regions
|
||||
### 2. Grounding Detector
|
||||
Universal detector for all models.
|
||||
|
||||
## Key Features
|
||||
|
||||
### 🚀 Model Caching
|
||||
Models are cached in memory by default for instant reuse:
|
||||
- First load: Downloads and initializes model
|
||||
- Second load: Instant loading from cache
|
||||
- Toggle `keep_in_memory=False` to clear cache
|
||||
- Overrides standard text label splitting. It splits only at ".", otherwise label is what you write
|
||||
- Enable `single_box_mode` for single detection
|
||||
- First load: Downloads model
|
||||
- Second load: Instant from cache
|
||||
- Models stored in `ComfyUI-Grounding/models/`
|
||||
- Process multiple images in one pass
|
||||
- All nodes support batches
|
||||
|
||||
### 📁 Local Model Storage
|
||||
All models are downloaded to the plugin's local Models directory:
|
||||
- Location: `ComfyUI/custom_nodes/ComfyUI-Grounding/Models/`
|
||||
- Auto-created on first use with a `models_go_here.txt` placeholder
|
||||
- GroundingDINO/MM-GroundingDINO: Stored in HuggingFace transformers cache format
|
||||
- YOLO-World: Stored as `.pt` files (yolov8s-world.pt, etc.)
|
||||
## Florence-2 Attention Modes
|
||||
|
||||
### 🎯 UnifiedDetector
|
||||
One detector for all models - automatically detects whether you're using GroundingDINO or YOLO-World and routes to the appropriate inference method.
|
||||
- `eager` - Most compatible (default)
|
||||
- `sdpa` - Faster on PyTorch 2.0+
|
||||
- `flash_attention_2` - Fastest on A100/H100
|
||||
|
||||
### 📦 Batch Processing
|
||||
All detector nodes support batch processing:
|
||||
- Process multiple images in a single forward pass
|
||||
- Input images as a batch tensor (B, H, W, C)
|
||||
- Returns batched bounding boxes with lists of detections for each image
|
||||
- Annotated images are returned as a single batch tensor
|
||||
### 2. Bounding Box Visualizer
|
||||
|
||||
### 🎯 Referring Expression Comprehension (REC)
|
||||
Automatic REC mode for natural language queries:
|
||||
- **GroundingDINO**: If prompt has no periods (`.`) or commas (`,`), uses full text as label
|
||||
- Example: `"main object in the center of the picture"` → all detections labeled with full text
|
||||
- With separators: `"person . car . dog ."` → uses standard noun phrase extraction
|
||||
- **YOLO-World**: If classes has no comma (`,`), uses full text as label
|
||||
- Example: `"red car in the parking lot"` → all detections labeled with full text
|
||||
- With separators: `"person, car, dog"` → uses standard class mapping
|
||||
- **Single Box Mode**: Enable `single_box_mode` to return only the highest-scoring detection (perfect for REC tasks)
|
||||
|
||||
### 📋 Output Formats
|
||||
Detector nodes support two output formats for bounding boxes:
|
||||
|
||||
**1. `list_only` (Default - SAM2 Compatible)**
|
||||
- Returns: List of numpy arrays, one per image in batch
|
||||
- Format: `[[bbox1, bbox2], [bbox3], ...]` where each bbox is `[x1, y1, x2, y2]`
|
||||
- Use when: Connecting to SAM2 segmentation nodes
|
||||
- Example:
|
||||
```python
|
||||
# For 2 images with 2 and 1 detections respectively
|
||||
[
|
||||
array([[10, 20, 100, 200], [50, 60, 150, 250]]), # Image 0: 2 boxes
|
||||
array([[30, 40, 120, 180]]) # Image 1: 1 box
|
||||
]
|
||||
```
|
||||
|
||||
**2. `dict_with_data` (Full Metadata)**
|
||||
- Returns: Dictionary with boxes, labels, and scores
|
||||
- Format: `{"boxes": [...], "labels": [...], "scores": [...]}`
|
||||
- Use when: Need label and confidence information for downstream processing
|
||||
- NOT compatible with SAM2 (dict iteration gives keys, not values)
|
||||
- Example:
|
||||
```python
|
||||
{
|
||||
"boxes": [array([...]), array([...])], # Boxes per image
|
||||
"labels": [["cat", "dog"], ["bird"]], # Labels per image
|
||||
"scores": [array([0.9, 0.8]), array([0.7])] # Scores per image
|
||||
}
|
||||
```
|
||||
|
||||
**Recommendation:** Use `list_only` (default) unless you specifically need label/score metadata for your workflow.
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Example 1: Basic Object Detection with GroundingDINO
|
||||
|
||||
1. Add `GroundingDINOModelLoader` node
|
||||
2. Select model (e.g., "GroundingDINO_SwinT_OGC (694MB)")
|
||||
3. Add `GroundingDINODetector` node
|
||||
4. Connect model output to detector
|
||||
5. Load an image and connect to detector
|
||||
6. Set prompt: "person . car . bicycle ."
|
||||
7. Adjust thresholds as needed
|
||||
8. View annotated image output
|
||||
|
||||
### Example 2: YOLO-World Detection
|
||||
|
||||
1. Add `YOLOWorldModelLoader` node
|
||||
2. Select model (e.g., "yolov8s-world")
|
||||
3. Add `YOLOWorldDetector` node
|
||||
4. Connect model and image
|
||||
5. Set classes: "cat, dog, bird, horse"
|
||||
6. Adjust confidence threshold
|
||||
7. View results
|
||||
|
||||
### Example 3: Using UnifiedDetector (Recommended)
|
||||
|
||||
1. Add any model loader (GroundingDINO or YOLO-World)
|
||||
2. Load your model
|
||||
3. Add `UnifiedDetector` node
|
||||
4. Connect model and image
|
||||
5. Set prompt (works with both formats!)
|
||||
6. The node automatically detects model type and uses the right inference
|
||||
|
||||
### Example 4: MM-GroundingDINO for High Accuracy
|
||||
|
||||
1. Add `GroundingDINOModelLoader` node
|
||||
2. Select "MM-GroundingDINO-Base (All datasets, 59.5 mAP)"
|
||||
3. Enable `keep_in_memory=True` (default)
|
||||
4. Add `GroundingDINODetector` or `UnifiedDetector`
|
||||
5. Set prompt: "specific object . another object ."
|
||||
6. Enjoy state-of-the-art detection accuracy!
|
||||
|
||||
### Example 5: Convert Detections to Masks
|
||||
|
||||
1. Use any detector node (GroundingDINO or YOLO-World)
|
||||
2. Add `BboxToMask` node
|
||||
3. Connect image and bboxes outputs
|
||||
4. Choose whether to combine masks
|
||||
5. Use masks for segmentation, inpainting, etc.
|
||||
|
||||
## Model Downloads
|
||||
|
||||
Models will be automatically downloaded on first use:
|
||||
- **All models**: Stored in `ComfyUI/custom_nodes/ComfyUI-Grounding/Models/`
|
||||
- **GroundingDINO/MM-GroundingDINO**: HuggingFace transformers cache format
|
||||
- **YOLO-World**: Direct `.pt` files (e.g., yolov8s-world.pt)
|
||||
Re-draw bboxes on images with custom line width. Optional since detector already returns annotated images.
|
||||
|
||||
## Tips
|
||||
|
||||
- **GroundingDINO prompts**: End each object with a period (e.g., "cat . dog . car .")
|
||||
- **YOLO-World classes**: Use comma-separated values (e.g., "cat, dog, car")
|
||||
- **Threshold tuning**: Start with 0.3 and adjust based on results
|
||||
- Lower threshold = more detections (including false positives)
|
||||
- Higher threshold = fewer, more confident detections
|
||||
- **Performance**: YOLO-World is faster but GroundingDINO may be more accurate
|
||||
|
||||
## Integration with SAM2
|
||||
|
||||
These nodes work seamlessly with SAM2 for instance segmentation:
|
||||
|
||||
1. Use GroundingDINO or YOLO-World to detect objects
|
||||
2. Convert bboxes to masks using `BboxToMask`
|
||||
3. Use masks with SAM2 for precise segmentation
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### ImportError: transformers not found
|
||||
```bash
|
||||
pip install transformers timm
|
||||
```
|
||||
|
||||
### ImportError: ultralytics not found
|
||||
```bash
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
### Models not downloading
|
||||
Check your internet connection and ensure you have write permissions to the ComfyUI models directory.
|
||||
|
||||
### CUDA out of memory
|
||||
Try using a smaller model variant (e.g., GroundingDINO-T instead of -B, or yolov8s instead of yolov8x).
|
||||
|
||||
## License
|
||||
|
||||
MIT License - See LICENSE file for details
|
||||
**Prompt Format:**
|
||||
- Use periods for multiple labels → `"cat . dog ."`
|
||||
- Use commas to keep a single label → `"cat, dog"`
|
||||
|
||||
## Credits
|
||||
|
||||
- GroundingDINO: [IDEA-Research](https://github.com/IDEA-Research/GroundingDINO)
|
||||
- YOLO-World: [Ultralytics](https://github.com/ultralytics/ultralytics)
|
||||
- Transformers: [HuggingFace](https://huggingface.co/transformers)
|
||||
- [GroundingDINO](https://github.com/IDEA-Research/GroundingDINO) - IDEA-Research
|
||||
- [OWLv2](https://huggingface.co/google/owlv2-base-patch16) - Google Research
|
||||
- [Florence-2](https://huggingface.co/microsoft/Florence-2-base) - Microsoft Research
|
||||
- [YOLO-World](https://github.com/ultralytics/ultralytics) - Ultralytics
|
||||
|
||||
## Support
|
||||
## License
|
||||
|
||||
For issues and feature requests, please visit the GitHub repository.
|
||||
MIT License
|
||||
|
||||
+8
-20
@@ -1,36 +1,24 @@
|
||||
"""
|
||||
ComfyUI-Grounding: Object detection nodes for ComfyUI
|
||||
Supports GroundingDINO, MM-GroundingDINO, and YOLO-World models
|
||||
ComfyUI-Grounding: Simplified object detection nodes for ComfyUI
|
||||
Supports GroundingDINO, MM-GroundingDINO, OWLv2, Florence-2, and YOLO-World
|
||||
"""
|
||||
|
||||
from .nodes import (
|
||||
GroundingDINOModelLoader,
|
||||
GroundingDINODetector,
|
||||
YOLOWorldModelLoader,
|
||||
YOLOWorldDetector,
|
||||
UnifiedDetector,
|
||||
GroundingModelLoader,
|
||||
GroundingDetector,
|
||||
BboxVisualizer,
|
||||
BboxToMask
|
||||
)
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"GroundingDINOModelLoader": GroundingDINOModelLoader,
|
||||
"GroundingDINODetector": GroundingDINODetector,
|
||||
"YOLOWorldModelLoader": YOLOWorldModelLoader,
|
||||
"YOLOWorldDetector": YOLOWorldDetector,
|
||||
"UnifiedDetector": UnifiedDetector,
|
||||
"GroundingModelLoader": GroundingModelLoader,
|
||||
"GroundingDetector": GroundingDetector,
|
||||
"BboxVisualizer": BboxVisualizer,
|
||||
"BboxToMask": BboxToMask,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"GroundingDINOModelLoader": "Grounding DINO Model Loader",
|
||||
"GroundingDINODetector": "Grounding DINO Detector",
|
||||
"YOLOWorldModelLoader": "YOLO-World Model Loader",
|
||||
"YOLOWorldDetector": "YOLO-World Detector",
|
||||
"UnifiedDetector": "Unified Detector (Auto)",
|
||||
"GroundingModelLoader": "Grounding Model Loader",
|
||||
"GroundingDetector": "Grounding Detector",
|
||||
"BboxVisualizer": "Bounding Box Visualizer",
|
||||
"BboxToMask": "Bounding Box to Mask",
|
||||
}
|
||||
|
||||
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 471 KiB |
@@ -0,0 +1,8 @@
|
||||
# Ignore all files in this directory (model downloads)
|
||||
*
|
||||
|
||||
# But keep this .gitignore file
|
||||
!.gitignore
|
||||
|
||||
# And keep the placeholder file
|
||||
!models_go_here.txt
|
||||
+3
-4
@@ -1,15 +1,14 @@
|
||||
# Core dependencies for ComfyUI-Grounding
|
||||
|
||||
# For GroundingDINO (HuggingFace transformers implementation)
|
||||
# For GroundingDINO, MM-GroundingDINO, and OWLv2 (HuggingFace transformers)
|
||||
transformers>=4.30.0
|
||||
timm>=0.9.0
|
||||
|
||||
# For YOLO-World
|
||||
ultralytics>=8.2.0
|
||||
|
||||
# Image processing
|
||||
opencv-python>=4.8.0
|
||||
pillow>=10.0.0
|
||||
# Image processing (usually already installed with ComfyUI)
|
||||
# pillow>=10.0.0
|
||||
|
||||
# Optional: supervision for advanced visualization
|
||||
# supervision>=0.16.0
|
||||
|
||||
@@ -0,0 +1,353 @@
|
||||
{
|
||||
"id": "b29971c7-6074-4707-895e-9f2360e11e2d",
|
||||
"revision": 0,
|
||||
"last_node_id": 30,
|
||||
"last_link_id": 40,
|
||||
"nodes": [
|
||||
{
|
||||
"id": 29,
|
||||
"type": "LoadImage",
|
||||
"pos": [
|
||||
691.3480657963805,
|
||||
244.71705551823334
|
||||
],
|
||||
"size": [
|
||||
274.080078125,
|
||||
314
|
||||
],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "IMAGE",
|
||||
"type": "IMAGE",
|
||||
"links": [
|
||||
38,
|
||||
40
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "MASK",
|
||||
"type": "MASK",
|
||||
"links": null
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"cnr_id": "comfy-core",
|
||||
"ver": "0.3.65",
|
||||
"Node name for S&R": "LoadImage"
|
||||
},
|
||||
"widgets_values": [
|
||||
"example.png",
|
||||
"image"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"type": "MaskPreview",
|
||||
"pos": [
|
||||
1252.4602357504743,
|
||||
329.44736978995775
|
||||
],
|
||||
"size": [
|
||||
140,
|
||||
246.00000000000003
|
||||
],
|
||||
"flags": {},
|
||||
"order": 5,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "mask",
|
||||
"type": "MASK",
|
||||
"link": 37
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"properties": {
|
||||
"cnr_id": "comfy-core",
|
||||
"ver": "0.3.65",
|
||||
"Node name for S&R": "MaskPreview"
|
||||
},
|
||||
"widgets_values": []
|
||||
},
|
||||
{
|
||||
"id": 22,
|
||||
"type": "PreviewImage",
|
||||
"pos": [
|
||||
1937.3573937178255,
|
||||
228.10370816997101
|
||||
],
|
||||
"size": [
|
||||
140,
|
||||
246.00000000000003
|
||||
],
|
||||
"flags": {},
|
||||
"order": 6,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "images",
|
||||
"type": "IMAGE",
|
||||
"link": 32
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"properties": {
|
||||
"cnr_id": "comfy-core",
|
||||
"ver": "0.3.65",
|
||||
"Node name for S&R": "PreviewImage"
|
||||
},
|
||||
"widgets_values": []
|
||||
},
|
||||
{
|
||||
"id": 30,
|
||||
"type": "PreviewImage",
|
||||
"pos": [
|
||||
1664.5457193908678,
|
||||
-114.32244507189236
|
||||
],
|
||||
"size": [
|
||||
140,
|
||||
246
|
||||
],
|
||||
"flags": {},
|
||||
"order": 4,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "images",
|
||||
"type": "IMAGE",
|
||||
"link": 39
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"properties": {
|
||||
"cnr_id": "comfy-core",
|
||||
"ver": "0.3.65",
|
||||
"Node name for S&R": "PreviewImage"
|
||||
},
|
||||
"widgets_values": []
|
||||
},
|
||||
{
|
||||
"id": 21,
|
||||
"type": "BboxVisualizer",
|
||||
"pos": [
|
||||
1608.8053963516986,
|
||||
258.9104840919287
|
||||
],
|
||||
"size": [
|
||||
210,
|
||||
78
|
||||
],
|
||||
"flags": {},
|
||||
"order": 3,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "image",
|
||||
"type": "IMAGE",
|
||||
"link": 40
|
||||
},
|
||||
{
|
||||
"name": "bboxes",
|
||||
"type": "BBOX",
|
||||
"link": 31
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "annotated_image",
|
||||
"type": "IMAGE",
|
||||
"links": [
|
||||
32
|
||||
]
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"aux_id": "PozzettiAndrea/ComfyUI-Grounding",
|
||||
"ver": "c9c72642cd7affb18d5aa54f21ee7d3acf7412eb",
|
||||
"Node name for S&R": "BboxVisualizer"
|
||||
},
|
||||
"widgets_values": [
|
||||
3
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 18,
|
||||
"type": "GroundingModelLoader",
|
||||
"pos": [
|
||||
761.1567094157901,
|
||||
37.11377863503034
|
||||
],
|
||||
"size": [
|
||||
272.064453125,
|
||||
106
|
||||
],
|
||||
"flags": {},
|
||||
"order": 1,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "model",
|
||||
"type": "GROUNDING_MODEL",
|
||||
"links": [
|
||||
27
|
||||
]
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"aux_id": "PozzettiAndrea/ComfyUI-Grounding",
|
||||
"ver": "c9c72642cd7affb18d5aa54f21ee7d3acf7412eb",
|
||||
"Node name for S&R": "GroundingModelLoader"
|
||||
},
|
||||
"widgets_values": [
|
||||
"GroundingDINO: SwinB (938MB)",
|
||||
true,
|
||||
"eager"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 19,
|
||||
"type": "GroundingDetector",
|
||||
"pos": [
|
||||
1147.1057310247927,
|
||||
-143.3627616665786
|
||||
],
|
||||
"size": [
|
||||
400,
|
||||
268
|
||||
],
|
||||
"flags": {},
|
||||
"order": 2,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "model",
|
||||
"type": "GROUNDING_MODEL",
|
||||
"link": 27
|
||||
},
|
||||
{
|
||||
"name": "image",
|
||||
"type": "IMAGE",
|
||||
"link": 38
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "bboxes",
|
||||
"type": "BBOX",
|
||||
"links": [
|
||||
31
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "annotated_image",
|
||||
"type": "IMAGE",
|
||||
"links": [
|
||||
39
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "labels",
|
||||
"type": "STRING",
|
||||
"links": null
|
||||
},
|
||||
{
|
||||
"name": "masks",
|
||||
"type": "MASK",
|
||||
"links": [
|
||||
37
|
||||
]
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"aux_id": "PozzettiAndrea/ComfyUI-Grounding",
|
||||
"ver": "c9c72642cd7affb18d5aa54f21ee7d3acf7412eb",
|
||||
"Node name for S&R": "GroundingDetector"
|
||||
},
|
||||
"widgets_values": [
|
||||
"character",
|
||||
0.1,
|
||||
0.01,
|
||||
true,
|
||||
"dict_with_data",
|
||||
true
|
||||
]
|
||||
}
|
||||
],
|
||||
"links": [
|
||||
[
|
||||
27,
|
||||
18,
|
||||
0,
|
||||
19,
|
||||
0,
|
||||
"GROUNDING_MODEL"
|
||||
],
|
||||
[
|
||||
31,
|
||||
19,
|
||||
0,
|
||||
21,
|
||||
1,
|
||||
"BBOX"
|
||||
],
|
||||
[
|
||||
32,
|
||||
21,
|
||||
0,
|
||||
22,
|
||||
0,
|
||||
"IMAGE"
|
||||
],
|
||||
[
|
||||
37,
|
||||
19,
|
||||
3,
|
||||
28,
|
||||
0,
|
||||
"MASK"
|
||||
],
|
||||
[
|
||||
38,
|
||||
29,
|
||||
0,
|
||||
19,
|
||||
1,
|
||||
"IMAGE"
|
||||
],
|
||||
[
|
||||
39,
|
||||
19,
|
||||
1,
|
||||
30,
|
||||
0,
|
||||
"IMAGE"
|
||||
],
|
||||
[
|
||||
40,
|
||||
29,
|
||||
0,
|
||||
21,
|
||||
0,
|
||||
"IMAGE"
|
||||
]
|
||||
],
|
||||
"groups": [],
|
||||
"config": {},
|
||||
"extra": {
|
||||
"ds": {
|
||||
"scale": 0.8390550503631276,
|
||||
"offset": [
|
||||
-372.4858306734141,
|
||||
284.24946587737446
|
||||
]
|
||||
},
|
||||
"frontendVersion": "1.28.7"
|
||||
},
|
||||
"version": 0.4
|
||||
}
|
||||
Reference in New Issue
Block a user