nodes/EricRollei-PDF-Tools
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
main
EricRollei-PDF-Tools/SURYA_OCR_NODE_GUIDE.md
T

6.7 KiB
Raw Permalink Blame History

Surya OCR Layout Node - Complete Guide

✅ Status: WORKING!

The Surya OCR Layout node has been successfully created and is fully functional!

📋 What This Node Does

This ComfyUI node provides:

  1. Layout Detection - Identifies different document elements:

    • Images
    • Text blocks
    • Tables
    • Headers/Footers
    • Titles
    • Captions
    • Lists
    • Forms
    • And more!

  2. Text Detection - Finds text regions in images

  3. OCR (Optical Character Recognition) - Extracts actual text content

  4. Visual Annotations - Creates annotated images with colored bounding boxes showing detected elements

🎯 Node Outputs

The node provides 7 outputs:

  1. annotated_image (IMAGE) - Input image with colored bounding boxes overlaid
  2. extracted_text (STRING) - All extracted text from OCR
  3. layout_json (STRING) - Layout detection data with element types and positions
  4. text_bboxes_json (STRING) - Bounding boxes for all text elements (for cropping)
  5. image_bboxes_json (STRING) - Bounding boxes for all detected images (for cropping)
  6. full_data_json (STRING) - Complete data from all detection/OCR operations
  7. status (STRING) - Summary message with statistics

🔧 Node Parameters

Required:

  • image - Input image to process
  • mode - Processing mode:
    • layout_only - Only detect document layout
    • ocr_only - Only perform text detection and OCR
    • layout_and_ocr - Do both (recommended)
  • confidence_threshold - Minimum confidence (0.0-1.0) for detections

Optional:

  • show_labels - Show element labels on annotated image (default: True)
  • batch_size - Batch size for processing (default: 2, lower if memory errors)
  • force_cpu_layout - IMPORTANT: Use CPU for layout detection (default: False)
    • Set to TRUE if you have a newer GPU (RTX 40XX, RTX Blackwell, etc.)
    • Avoids Flash Attention compatibility issues

⚠️ Important: Flash Attention Issue

Problem: Newer GPUs (RTX 4090, RTX Blackwell, etc.) have Flash Attention incompatibility issues.

Solution: Enable force_cpu_layout = True

This makes:

  • Layout detection run on CPU (slower but works)
  • OCR and text detection still run on GPU (fast!)

The performance impact is minimal since layout detection is a small part of the overall process.

🎨 Color Coding

The annotated image uses different colors for different element types:

  • Red - Images
  • Green - Tables
  • Blue - Text blocks
  • Magenta - Titles
  • Yellow - Section headers
  • Cyan - Lists
  • Orange - Captions
  • Purple - Footnotes
  • Pink - Formulas
  • Brown - Page headers
  • Gray - Page footers

📦 Installation & Setup

The node is located at:

a:\Comfy25\ComfyUI_windows_portable\ComfyUI\custom_nodes\PDF_tools\nodes\surya_ocr_layout_node.py

Requirements:

  • ✅ Surya OCR v0.17.0 (already installed)
  • ✅ PyTorch with CUDA support (already installed)
  • ✅ PIL/Pillow (already installed)

To Use in ComfyUI:

  1. Restart ComfyUI to load the new node
  2. Look for "Surya OCR & Layout Detection" in the node browser under text/ocr category
  3. Connect an image input
  4. Important: Set force_cpu_layout = True for newer GPUs
  5. Connect outputs to:
    • Image preview (to see annotated results)
    • Text display nodes
    • Crop nodes (using bbox data)
    • JSON parsers

💡 Usage Examples

Example 1: Extract Text and Layout from Document

[Load Image] 
   ↓
[Surya OCR & Layout Detection]
   mode: layout_and_ocr
   force_cpu_layout: True
   ↓
   ├→ annotated_image → [Preview Image]
   ├→ extracted_text → [Display Text]
   └→ layout_json → [JSON Parser]

Example 2: Find and Crop Images from Document

[Load Image]
   ↓
[Surya OCR & Layout Detection]
   mode: layout_only
   force_cpu_layout: True
   ↓
   image_bboxes_json → [Parse JSON] → [Crop Images]

Example 3: Extract Specific Text Blocks

[Load Image]
   ↓
[Surya OCR & Layout Detection]
   mode: layout_and_ocr
   force_cpu_layout: True
   ↓
   text_bboxes_json → [Filter by Type: "Title"] → [Crop Text] → [OCR]

🐛 Troubleshooting

Error: "CUDA error: the provided PTX was compiled with an unsupported toolchain"

Solution: Enable force_cpu_layout = True

Slow Performance

  • Layout on CPU is expected - It's a workaround for GPU incompatibility
  • OCR still uses GPU - This is the most intensive part and remains fast
  • Lower batch_size if running out of memory

No Text Detected

  • Try lowering confidence_threshold (default 0.5)
  • Make sure image has actual text content
  • Check image quality/resolution

No Layout Boxes Detected

  • Image might be blank or very simple
  • Try with a document/PDF page image
  • Lower confidence_threshold

📊 Performance Notes

On your system (RTX PRO 6000 Blackwell):

  • OCR (GPU): Very fast (~0.4 seconds per image)
  • Layout (CPU): ~2 seconds per image
  • Combined: Still very usable for most workflows

🔬 Technical Details

Why CPU for Layout?

The Flash Attention library used by Surya's layout detection was compiled with CUDA toolchain that doesn't support the very newest GPU architectures (Blackwell). This is a temporary limitation until Flash Attention releases updated binaries.

Architecture:

  • Foundation Model: Shared base model (GPU or CPU)
  • Detection Predictor: Text detection (GPU)
  • Recognition Predictor: OCR text extraction (GPU)
  • Layout Predictor: Document structure (CPU fallback available)

📝 JSON Output Format Examples

Layout JSON:

{
  "layout": [
    {
      "page": 0,
      "boxes": [
        {
          "label": "Text",
          "confidence": 0.95,
          "polygon": [[x1, y1], [x2, y2], ...],
          "position": 0
        }
      ]
    }
  ]
}

Image Bboxes JSON:

{
  "image_boxes": [
    {
      "page": 0,
      "polygon": [[x1, y1], [x2, y2], [x3, y3], [x4, y4]],
      "confidence": 0.98,
      "position": 1
    }
  ]
}

🎓 Next Steps

  1. Test with real documents - Try PDF pages, scanned documents, etc.
  2. Build workflows - Combine with crop, mask, and other processing nodes
  3. Compare OCR accuracy - Test against Tesseract or other OCR engines
  4. Create specialized workflows:
    • Extract all images from documents
    • Find and extract specific text blocks (headers, captions, etc.)
    • Table extraction and processing
    • Multi-page document processing

📞 Support

If you encounter issues:

  1. Check force_cpu_layout = True is enabled
  2. Verify surya-ocr is properly installed
  3. Check ComfyUI console for error messages
  4. Test with the provided test script first

Created: October 26, 2025
Version: 1.0
Compatible with: Surya OCR v0.17.0, ComfyUI

Reference in New Issue View Git Blame Copy Permalink