# PDF Tools - ComfyUI Custom Node Package

A comprehensive ComfyUI custom node collection for PDF processing with AI-powered analysis.

## 📢 Important Notice: Package Split

The **download functionality has moved** to a separate package:

- **PDF Tools** (this package): PDF extraction, OCR, AI vision processing
- **[Download Tools](../download-tools/)** (new package): gallery-dl and yt-dlp downloaders

If you need the download nodes (yt-dlp, gallery-dl), please install the **download-tools** package separately:

```powershell
cd ComfyUI/custom_nodes/download-tools
.\install.ps1
```

Both packages work independently and can be installed together or separately.

---

## 🎉 Quick Start

### Just Moved to New Install?

1. **Check Installation:**
   ```powershell
   cd A:\Comfy25\ComfyUI_windows_portable
   powershell -ExecutionPolicy Bypass -File .\ComfyUI\custom_nodes\PDF_tools\check_install.ps1
   ```

2. **Install Dependencies:**
   - The check script will offer to install everything
   - Or manually run: `.\python_embeded\python.exe -m pip install -r .\ComfyUI\custom_nodes\PDF_tools\requirements.txt`

3. **Start ComfyUI:**
   - Run your normal startup script
   - Nodes will appear in the "Add Node" menu

## 📦 What's Included

### PDF Extraction Nodes
- **PDF Extractor v08** - Advanced PDF image extraction with quality assessment and spread detection
- **PDF Extractor v09** - Latest version with enhanced image processing
- **PDF Extractor v05-v07** - Previous versions for compatibility
- **Simple PDF Extractor** - Basic PDF extraction without advanced features

### OCR & Layout Analysis Nodes
- **Surya OCR Layout Node** - State-of-the-art multilingual OCR with layout detection
- **Surya Layout OCR Hybrid** - Combined layout analysis and text extraction
- **Basic Surya** / **Basic Surya v02** - Simplified Surya OCR interfaces
- **Enhanced Layout Parser v06** - Advanced document layout understanding
- **LayoutLMv3 Node** - Microsoft LayoutLMv3 for document AI

### AI Vision & Detection Nodes
- **Florence2 Rectangle Detector** - Microsoft Florence-2 for object detection and vision analysis
- **Florence2 Cropper Node** - Crop images based on Florence-2 detections
- **Eric Rectangle Detector** - Custom rectangle detection implementation

### Remote Processing Nodes
- **PaddleOCR VL Remote** - Remote PaddleOCR processing for Chinese/multilingual documents

### Downloader Nodes (Moved)
> **Note:** Download functionality has moved to the [download-tools](../download-tools/) package.
> Install it separately if you need gallery-dl or yt-dlp downloaders.

## 🚀 Features

✅ **PDF Extraction** - Extract images from PDFs with quality scoring and duplicate detection  
✅ **AI-Powered OCR** - Surya, PaddleOCR, LayoutLMv3 for accurate text extraction  
✅ **Layout Analysis** - Understand document structure (headers, paragraphs, tables, figures)  
✅ **Vision Models** - Florence-2 for object detection and image understanding  
✅ **Spread Detection** - Automatically detect two-page spreads in scanned documents  
✅ **Quality Assessment** - Evaluate image sharpness, contrast, and overall quality  
✅ **Batch Processing** - Process multiple PDFs or images efficiently  
✅ **GPU Acceleration** - Leverage CUDA for faster AI model inference  

## 📚 Documentation

- **[INSTALLATION_SUCCESS.md](INSTALLATION_SUCCESS.md)** - Installation complete? Read this!
- **[INSTALLATION_GUIDE.md](INSTALLATION_GUIDE.md)** - Detailed setup instructions
- **[CODE_OVERVIEW.md](CODE_OVERVIEW.md)** - Understanding the codebase
- **[requirements.txt](requirements.txt)** - All Python dependencies

### Detailed Guides (in Docs/)
- `gallery_dl_node_complete_guide.md` - Gallery-dl usage guide
- `yt_dlp_node_complete_guide.md` - Yt-dlp usage guide  
- `SETUP_COMPLETE.md` - Authentication configuration
- Test scripts for all features

## 🔧 Requirements

### Core Dependencies (Auto-installed)
- PyMuPDF (fitz) - PDF processing
- Pillow - Image processing
- numpy - Array operations
- opencv-python - Computer vision
- transformers - AI models

### Downloader Tools (Auto-installed)
- gallery-dl - Web media downloader
- yt-dlp - Video/audio downloader
- browser-cookie3 - Cookie extraction

### Optional
- FFmpeg - For yt-dlp audio extraction (download separately)
- Tesseract - For enhanced OCR (download separately)

## � Configuration Files

**Important:** This repository uses **template config files** to protect your personal credentials.

### First Time Setup
1. Copy template files: `configs/*.example` → remove `.example` extension
2. Edit with your credentials (usernames, passwords, API keys)
3. Your personal configs are automatically excluded from Git

See **[CONFIG_SETUP.md](CONFIG_SETUP.md)** for detailed instructions.

Your personal configuration files (without `.example`) will **never** be committed to GitHub! 🔒

## �💡 Quick Examples

### Download from Instagram
```
Node: Gallery-dl Downloader
URL: https://www.instagram.com/username/
Output: ./instagram-downloads
Enable: organize_files, use_download_archive
```

### Download from YouTube
```
Node: Yt-dlp Downloader
URL: https://www.youtube.com/watch?v=VIDEO_ID
Format: best (or audio-only for music)
Output: ./youtube-downloads
```

### Extract PDF Pages
```
Node: PDF Extractor v08
Input: Your PDF file
Output: ./pdf-output
Enable: quality_assessment, spread_detection
```

## 🔐 Authentication

Many sites (Instagram, Reddit) require authentication:

**Option 1: Browser Cookies**
- Log into site in browser
- Enable `use_browser_cookies` in node
- Select browser (Firefox, Chrome, etc.)

**Option 2: Export Cookies**
- Install browser extension "Get cookies.txt LOCALLY"
- Export cookies while logged in
- Save to `configs/instagram_cookies.json`
- Set `cookie_file` parameter

## 📁 Project Structure

```
PDF_tools/
├── nodes/              # ComfyUI node implementations
├── florence2_scripts/  # Florence2 AI vision models
├── sam2_scripts/       # SAM2 segmentation models  
├── configs/            # Authentication & config files
├── Docs/               # Comprehensive documentation
└── [test directories]  # Test output locations
```

## ⚙️ System Requirements

- Windows 10/11 (primary), Linux (should work)
- Python 3.10+ (included with ComfyUI)
- NVIDIA GPU with CUDA (recommended for AI models)
- 8GB+ RAM (16GB+ for AI models)
- 5-10GB storage for packages + models

## 🐛 Troubleshooting

### "gallery-dl not found"
✅ **Already installed!** Use: `.\python_embeded\python.exe -m gallery_dl`

### "Chrome cookies not accessible"  
💡 **Solution:** Run ComfyUI as admin, or use Firefox, or export cookies

### "Instagram/Reddit downloads fail"
💡 **Solution:** Export cookies from logged-in browser session

### "CUDA out of memory"
💡 **Solution:** Close other GPU apps, reduce batch size, use smaller models

See INSTALLATION_GUIDE.md for detailed troubleshooting.

## 🎯 Best Practices

1. **Use Download Archives** - Enable to avoid re-downloading
2. **Enable File Organization** - Keep downloads sorted by type
3. **Export Cookies for Private Content** - More reliable than browser cookies
4. **Start with Public URLs** - Test setup before authenticated downloads
5. **Monitor GPU Usage** - When using AI models

## 📝 Version Info

- **Gallery-dl:** 1.30.9 ✅
- **Yt-dlp:** 2025.9.5 ✅
- **PyMuPDF:** 1.26.4 ✅
- **Transformers:** 4.55.0 ✅
- **Torch:** 2.7.1+cu128 ✅

## 🤝 Contributing

This is a working codebase! If you extend it:
- Follow existing patterns
- Add comprehensive error handling
- Include debug output
- Update documentation
- Test with real data

## 📄 License

Copyright (c) 2025 Eric Hiss. All rights reserved.

This software is dual-licensed:

- **Non-Commercial Use:** Licensed under [Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0)](http://creativecommons.org/licenses/by-nc/4.0/)
- **Commercial Use:** Requires a separate commercial license. Contact: eric@historic.camera or eric@rollei.us

See [LICENSE.md](LICENSE.md) for full licensing terms.

### Third-Party Dependencies

This project relies on numerous open-source libraries, each with its own license. See [CREDITS.md](CREDITS.md) for a comprehensive list of all dependencies and their licenses.

**Important License Notes:**
- Some dependencies (gallery-dl, PyMuPDF) use GPL/AGPL licenses
- Commercial use requires compliance with all dependency licenses
- See CREDITS.md for detailed licensing information

## 👥 Contributing

Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on:
- Code style and standards
- Testing requirements
- Pull request process
- Development setup

## 📧 Contact

- **Author:** Eric Hiss
- **GitHub:** [EricRollei](https://github.com/EricRollei)
- **Email:** eric@historic.camera, eric@rollei.us

## 🙏 Acknowledgments

Special thanks to:
- The **ComfyUI** community for the amazing platform
- **Hugging Face** for hosting models and providing ML tools
- **Microsoft Research** for Florence-2 vision models
- **Meta AI** for SAM2 (Segment Anything Model 2)
- All open-source developers whose libraries make this project possible

See [CREDITS.md](CREDITS.md) for detailed acknowledgments.

## 🎉 Getting Started

1. ✅ **Check installation:** Run `check_install.ps1`
2. ✅ **Read:** [INSTALLATION_SUCCESS.md](INSTALLATION_SUCCESS.md)
3. ✅ **Start ComfyUI:** Nodes will appear automatically
4. ✅ **Test:** Try a simple download
5. ✅ **Configure:** Set up authentication if needed

---

**Ready to go!** All dependencies are installed. Read INSTALLATION_SUCCESS.md for next steps.

## 📚 Additional Documentation

- [LICENSE.md](LICENSE.md) - Full license terms
- [CREDITS.md](CREDITS.md) - Third-party libraries and acknowledgments
- [CONTRIBUTING.md](CONTRIBUTING.md) - Contribution guidelines