303 lines
8.8 KiB
Markdown
303 lines
8.8 KiB
Markdown
# PDF Tools Custom Node - Installation Guide
|
|
|
|
## Overview
|
|
|
|
This is a comprehensive ComfyUI custom node package that provides powerful tools for:
|
|
- **PDF Extraction & Processing** - Extract images and text from PDFs
|
|
- **Media Downloading** - Download images/videos from Instagram, Reddit, Twitter, YouTube, etc.
|
|
- **AI-Powered Image Analysis** - Florence2 vision models for rectangle detection
|
|
- **Layout Analysis** - Detect document layouts and structures
|
|
- **Image Enhancement** - Modern image enhancement for better quality
|
|
|
|
## What's Included
|
|
|
|
### 1. **Gallery-dl Downloader Node**
|
|
Download media from 100+ websites including:
|
|
- Instagram (posts, stories, reels)
|
|
- Reddit (posts, galleries)
|
|
- Twitter/X (images, videos)
|
|
- Imgur, DeviantArt, Artstation, and more
|
|
|
|
Features:
|
|
- Browser cookie authentication
|
|
- File organization by type (images/videos/audio)
|
|
- Download archive to avoid duplicates
|
|
- Metadata extraction
|
|
|
|
### 2. **Yt-dlp Downloader Node**
|
|
Download videos and audio from:
|
|
- YouTube (videos, playlists, channels)
|
|
- TikTok, Twitch, Instagram videos
|
|
- 1000+ video platforms
|
|
|
|
Features:
|
|
- Format selection (quality presets)
|
|
- Audio extraction (MP3, FLAC, etc.)
|
|
- Subtitle download and embedding
|
|
- Playlist support
|
|
|
|
### 3. **PDF Extractor Nodes**
|
|
Multiple versions for extracting content from PDFs:
|
|
- Extract images with quality assessment
|
|
- OCR text recognition
|
|
- Spread detection (book scanning)
|
|
- Metadata preservation
|
|
- Multiple output formats
|
|
|
|
### 4. **Florence2 Rectangle Detector**
|
|
AI-powered image analysis using Microsoft's Florence2 model:
|
|
- Detect rectangular regions in images
|
|
- Caption generation
|
|
- Object detection
|
|
- Visual question answering
|
|
|
|
### 5. **Layout Parser Nodes**
|
|
Document layout analysis:
|
|
- Detect text blocks, figures, tables
|
|
- Enhanced OCR with multiple engines
|
|
- Computer vision-based layout detection
|
|
|
|
## Installation Steps
|
|
|
|
### Step 1: Navigate to Your Python Environment
|
|
|
|
```powershell
|
|
cd A:\Comfy25\ComfyUI_windows_portable
|
|
```
|
|
|
|
### Step 2: Install Core Requirements
|
|
|
|
Use the embedded Python to install packages:
|
|
|
|
```powershell
|
|
.\python_embeded\python.exe -m pip install -r custom_nodes\PDF_tools\requirements.txt
|
|
```
|
|
|
|
### Step 3: Install External Tools (Optional but Recommended)
|
|
|
|
#### 3a. Install gallery-dl (for web downloads)
|
|
```powershell
|
|
.\python_embeded\python.exe -m pip install gallery-dl
|
|
```
|
|
|
|
#### 3b. Install yt-dlp (for video downloads)
|
|
```powershell
|
|
.\python_embeded\python.exe -m pip install yt-dlp
|
|
```
|
|
|
|
#### 3c. Install FFmpeg (Required for yt-dlp audio extraction)
|
|
1. Download FFmpeg from: https://www.gyan.dev/ffmpeg/builds/
|
|
2. Extract to `C:\ffmpeg\` (or any location)
|
|
3. Add `C:\ffmpeg\bin` to your system PATH
|
|
4. Verify: `ffmpeg -version`
|
|
|
|
### Step 4: Verify Installation
|
|
|
|
Test that packages are installed:
|
|
|
|
```powershell
|
|
# Test gallery-dl
|
|
.\python_embeded\python.exe -m gallery_dl --version
|
|
|
|
# Test yt-dlp
|
|
.\python_embeded\python.exe -m yt_dlp --version
|
|
|
|
# Test PyMuPDF
|
|
.\python_embeded\python.exe -c "import fitz; print(f'PyMuPDF {fitz.__version__}')"
|
|
|
|
# Test transformers
|
|
.\python_embeded\python.exe -c "import transformers; print(f'Transformers {transformers.__version__}')"
|
|
```
|
|
|
|
### Step 5: Configure Authentication (Optional)
|
|
|
|
#### For Instagram/Reddit downloads:
|
|
|
|
1. **Export cookies from your browser:**
|
|
- Use browser extension: "Get cookies.txt LOCALLY" (Chrome/Firefox)
|
|
- Save as `configs/instagram_cookies.json` (Netscape format is fine)
|
|
|
|
2. **Or use browser cookies directly:**
|
|
- Set `use_browser_cookies: True` in the node
|
|
- Chrome requires admin privileges on Windows
|
|
- Firefox works without admin
|
|
|
|
#### For Reddit API (if using config file):
|
|
- See `Docs/reddit_app_creation_guide.py` for setup instructions
|
|
- Note: Reddit API requires OAuth and may have rate limits
|
|
|
|
### Step 6: Start ComfyUI
|
|
|
|
```powershell
|
|
# Start ComfyUI normally
|
|
.\run_nvidia_gpu.bat
|
|
|
|
# Or with admin privileges (for Chrome cookie access)
|
|
# Right-click run_nvidia_gpu.bat → "Run as administrator"
|
|
```
|
|
|
|
## Quick Test
|
|
|
|
### Test Gallery-dl Node:
|
|
|
|
1. Add "Gallery-dl Downloader" node to workflow
|
|
2. Set URL: `https://www.instagram.com/janaioannaa/` (or any public profile)
|
|
3. Set output_dir: `./test-output`
|
|
4. Run workflow
|
|
5. Check `test-output/instagram/janaioannaa/images/` for downloaded files
|
|
|
|
### Test Yt-dlp Node:
|
|
|
|
1. Add "Yt-dlp Downloader" node to workflow
|
|
2. Set URL: `https://www.youtube.com/watch?v=dQw4w9WgXcQ` (example)
|
|
3. Set output_dir: `./yt-output`
|
|
4. Run workflow
|
|
5. Check output directory for downloaded video
|
|
|
|
### Test PDF Extractor:
|
|
|
|
1. Add "PDF Extractor v08" node to workflow
|
|
2. Load a PDF file
|
|
3. Set output directory
|
|
4. Run to extract pages as images
|
|
|
|
## Common Issues & Solutions
|
|
|
|
### Issue: "gallery-dl: command not found"
|
|
**Solution:** Install with pip: `python_embeded\python.exe -m pip install gallery-dl`
|
|
|
|
### Issue: "yt-dlp: command not found"
|
|
**Solution:** Install with pip: `python_embeded\python.exe -m pip install yt-dlp`
|
|
|
|
### Issue: "FFmpeg not found" (yt-dlp)
|
|
**Solution:**
|
|
1. Download FFmpeg: https://www.gyan.dev/ffmpeg/builds/
|
|
2. Extract and add to PATH
|
|
3. Or place `ffmpeg.exe` in ComfyUI root directory
|
|
|
|
### Issue: "PyMuPDF not found"
|
|
**Solution:** `python_embeded\python.exe -m pip install PyMuPDF`
|
|
|
|
### Issue: Chrome cookies not accessible
|
|
**Solutions:**
|
|
1. Run ComfyUI as administrator (Windows security restriction)
|
|
2. Or use Firefox instead (doesn't require admin)
|
|
3. Or export cookies to file and use `cookie_file` parameter
|
|
|
|
### Issue: Instagram/Reddit downloads fail
|
|
**Solutions:**
|
|
1. Export cookies from logged-in browser session
|
|
2. Place in `configs/instagram_cookies.json`
|
|
3. Set `cookie_file` parameter in node
|
|
4. Make sure you're logged into the site in your browser
|
|
|
|
### Issue: "CUDA out of memory" (Florence2/AI models)
|
|
**Solutions:**
|
|
1. Close other GPU applications
|
|
2. Reduce batch size in node settings
|
|
3. Use smaller model variants
|
|
4. Enable model offloading in ComfyUI settings
|
|
|
|
### Issue: Transformers version conflicts
|
|
**Solution:**
|
|
```powershell
|
|
.\python_embeded\python.exe -m pip install transformers>=4.35.0 --upgrade
|
|
```
|
|
|
|
## Package Size Warnings
|
|
|
|
Some packages are large and optional:
|
|
|
|
- **Surya OCR**: ~1GB models (advanced OCR)
|
|
- **SAM2**: ~1-2GB models (segmentation)
|
|
- **Florence2 models**: ~500MB-2GB (vision models, auto-downloaded)
|
|
- **PaddleOCR**: ~500MB models (Chinese/English OCR)
|
|
- **EasyOCR**: ~1GB models (multi-language OCR)
|
|
|
|
These are **commented out** in requirements.txt - only install if needed.
|
|
|
|
## Minimal Installation
|
|
|
|
If you only want specific features:
|
|
|
|
### Just Gallery-dl (web downloads):
|
|
```powershell
|
|
.\python_embeded\python.exe -m pip install gallery-dl browser-cookie3 requests
|
|
```
|
|
|
|
### Just Yt-dlp (video downloads):
|
|
```powershell
|
|
.\python_embeded\python.exe -m pip install yt-dlp
|
|
```
|
|
|
|
### Just PDF extraction:
|
|
```powershell
|
|
.\python_embeded\python.exe -m pip install PyMuPDF Pillow numpy
|
|
```
|
|
|
|
### Just Florence2 (AI vision):
|
|
```powershell
|
|
.\python_embeded\python.exe -m pip install transformers safetensors accelerate timm
|
|
```
|
|
|
|
## Next Steps
|
|
|
|
1. **Review the documentation:**
|
|
- `Docs/gallery_dl_node_complete_guide.md` - Gallery-dl setup
|
|
- `Docs/yt_dlp_node_complete_guide.md` - Yt-dlp setup
|
|
- `Docs/SETUP_COMPLETE.md` - Authentication setup
|
|
|
|
2. **Test with example workflows:**
|
|
- Start with simple single-URL downloads
|
|
- Test authentication with your accounts
|
|
- Try batch downloads from files
|
|
|
|
3. **Configure for your needs:**
|
|
- Set up cookie files for authenticated sites
|
|
- Create custom config files for specific sites
|
|
- Organize download directories
|
|
|
|
## Getting Help
|
|
|
|
- Check the `Docs/` folder for detailed guides
|
|
- Review test scripts in `Docs/test_*.py` for examples
|
|
- Check ComfyUI console for debug output (nodes provide detailed status)
|
|
|
|
## System Requirements
|
|
|
|
- **OS**: Windows 10/11 (primary), Linux (should work)
|
|
- **GPU**: NVIDIA GPU with CUDA (for AI models, optional for downloaders)
|
|
- **RAM**: 8GB minimum, 16GB+ recommended for AI models
|
|
- **Storage**: 5-10GB for packages + models
|
|
- **Python**: 3.10+ (comes with ComfyUI portable)
|
|
|
|
## What's Working
|
|
|
|
✅ Gallery-dl downloads (Instagram, Reddit, Twitter, etc.)
|
|
✅ Yt-dlp downloads (YouTube, TikTok, etc.)
|
|
✅ PDF extraction with PyMuPDF
|
|
✅ Florence2 rectangle detection
|
|
✅ Browser cookie authentication
|
|
✅ File organization by type
|
|
✅ Download archives (no duplicates)
|
|
✅ Metadata extraction
|
|
✅ Debug output and error handling
|
|
|
|
## Known Limitations
|
|
|
|
⚠️ Reddit API may hang with old credentials (use browser cookies instead)
|
|
⚠️ Chrome cookies require admin privileges on Windows
|
|
⚠️ Some sites require valid login cookies
|
|
⚠️ Large models (Florence2, SAM2) require GPU memory
|
|
⚠️ Transformers versions may need updates
|
|
|
|
## Support & Updates
|
|
|
|
This is a custom node package. For issues:
|
|
1. Check the documentation in `Docs/`
|
|
2. Review test scripts for working examples
|
|
3. Check ComfyUI console for detailed error messages
|
|
4. Ensure all requirements are installed correctly
|
|
|
|
Happy downloading and processing! 🚀
|