eljefe80-comfyui-mqtt-node/DECISIONS.md

Technical Decisions and Design Documentation

This document outlines the technical decisions made during the development of the ComfyUI MQTT Node.

Architecture Decisions

1. Node Design Pattern

Decision: Implement two separate nodes - one for binary image publishing and one for JSON metadata publishing.

Rationale:

• Separation of concerns: Binary and metadata have different use cases
• Binary images can be large (MBs), while metadata is small (bytes)
• Different consumers may want only metadata (for monitoring, analytics) vs full images
• Allows flexible workflow composition in ComfyUI
• Better performance when only metadata is needed

Alternatives Considered:

• Single node with a mode switch - rejected due to reduced clarity in workflows
• Three nodes (binary, metadata, combined) - rejected as over-engineered

2. MQTT Library Selection

Decision: Use paho-mqtt library.

Rationale:

• Official Eclipse MQTT client for Python
• Mature, well-maintained, and widely used
• Full MQTT 3.1.1 protocol support
• Built-in TLS/SSL support
• Comprehensive authentication options
• Synchronous API suitable for ComfyUI's execution model
• Active community and excellent documentation

Alternatives Considered:

• gmqtt (async) - rejected due to async complexity in ComfyUI's sync workflow
• hbmqtt - rejected due to less maintenance and smaller community
• Custom implementation - rejected as reinventing the wheel

3. Image Format Handling

Decision: Convert ComfyUI tensors to PIL Images, then encode as JPEG or PNG.

Rationale:

• ComfyUI uses PyTorch tensors in [B, H, W, C] format with float values [0, 1]
• PIL is standard for image manipulation in Python
• JPEG provides good compression for photographs
• PNG provides lossless compression and transparency support
• Easy conversion pipeline: Tensor → NumPy → PIL → Bytes

Implementation Details:

Tensor [B,H,W,C] float32 [0,1]
→ NumPy [H,W,C] uint8 [0,255]
→ PIL Image
→ Bytes (JPEG/PNG)

Alternatives Considered:

• Raw tensor bytes - rejected due to incompatibility with standard image viewers
• Base64 encoded in JSON - rejected due to size increase (~33% overhead)
• Only PNG - rejected to give users format choice

4. TLS/SSL Configuration

Decision: Use CERT_REQUIRED with TLSv1.2 minimum, no custom certificate support in v1.

Rationale:

• CERT_REQUIRED ensures broker identity verification
• TLSv1.2 is widely supported and secure (TLSv1.3 not all brokers)
• System certificate store is sufficient for most use cases
• Custom certificates add complexity for edge cases

Security Considerations:

• Certificate verification prevents MITM attacks
• TLS version selection balances security and compatibility
• Password field in ComfyUI is visible - documented as limitation

Future Enhancements:

• Add custom CA certificate path option
• Support for client certificates (mutual TLS)
• Consider encrypted credential storage

5. Authentication Model

Decision: Username/password authentication via optional string inputs.

Rationale:

• Most common MQTT authentication method
• Simple to configure
• Supported by all MQTT brokers
• Optional nature allows unauthenticated development/testing

Limitations:

• ComfyUI string inputs are visible in UI (not password fields)
• No support for token-based auth in v1
• No certificate-based client authentication in v1

Mitigation:

• Documented security consideration in README
• Recommended to use environment variables or config files (future)

6. QoS (Quality of Service) Configuration

Decision: Expose QoS as integer input (0-2) with default of 1.

Rationale:

• QoS affects delivery guarantees and performance
• QoS 1 (at least once) is good default - balance of reliability and performance
• Users may need QoS 0 for high-throughput scenarios
• Users may need QoS 2 for critical applications

QoS Levels:

• 0: Fire and forget (fastest, no confirmation)
• 1: Acknowledged delivery (balanced, default)
• 2: Exactly once (slowest, guaranteed)

7. Message Retain Flag

Decision: Expose retain flag as boolean option, default False.

Rationale:

• Retain is important for status topics where new subscribers need last value
• Not appropriate for stream data (generates, renders)
• User choice depending on use case
• False default prevents unintended broker storage

Use Cases:

• Retain=True: Status updates, configuration data
• Retain=False: Image streams, transient data

8. Connection Management

Decision: Create new connection per publish, disconnect immediately after.

Rationale:

• ComfyUI workflows are event-driven, not continuous
• Simple connection lifecycle - no state management needed
• No connection pooling complexity
• Automatic cleanup prevents resource leaks
• Suitable for batch/generation workflows

Performance Considerations:

• Connection overhead acceptable for per-image publishing
• MQTT handshake is fast (milliseconds)
• For high-frequency publishing, users can batch upstream

Alternatives Considered:

• Persistent connection - rejected due to ComfyUI lifecycle complexity
• Connection pooling - rejected as premature optimization

9. Error Handling Strategy

Decision: Catch all exceptions, return status string, log to console.

Rationale:

• Node should not crash ComfyUI workflow
• Users need feedback about success/failure
• Console logging for debugging
• Pass-through design allows workflow continuation even on publish failure

Error Information:

• Success: "✓ Published to {broker}:{port}/{topic}"
• Failure: "✗ MQTT Error: {exception message}"
• Both printed to console with [MQTT] prefix

10. Pass-Through Design

Decision: Always output the input image unchanged, plus status string.

Rationale:

• Non-destructive workflow integration
• Can chain with other nodes
• Can publish to multiple brokers
• Publishing is a side effect, not a transformation
• Status output enables conditional logic in workflows

11. Metadata Node JSON Format

Decision: Send structured JSON with image dimensions and optional custom data.

Rationale:

• JSON is standard for structured data in MQTT
• Dimensions useful for consumers to allocate buffers
• Custom data allows user extensibility
• Lightweight for monitoring/analytics

JSON Schema:

{
  "batch_size": int,
  "width": int,
  "height": int,
  "channels": int,
  "dtype": string,
  ...custom fields
}

12. Client ID Configuration

Decision: Allow custom client ID with sensible defaults.

Rationale:

• Some brokers track clients by ID
• Multiple ComfyUI instances need unique IDs
• Default is sufficient for single instance
• Advanced users can customize for management

Default IDs:

• Binary node: "comfyui_mqtt_client"
• Metadata node: "comfyui_mqtt_json_client"

Implementation Decisions

13. Python Package Structure

Decision: Standard Python package with __init__.py exposing NODE_CLASS_MAPPINGS.

Rationale:

• Follows ComfyUI custom node conventions
• Clean namespace management
• Easy to extend with additional modules
• Standard Python import behavior

File Structure:

comfyui-mqtt-node/
├── __init__.py          # Package entry, exports mappings
├── mqtt_node.py         # Node implementations
├── requirements.txt     # Dependencies
├── README.md           # User documentation
└── DECISIONS.md        # This file

14. Dependency Management

Decision: Minimal dependencies via requirements.txt.

Rationale:

• paho-mqtt: MQTT protocol
• Pillow: Image encoding (likely already in ComfyUI)
• numpy: Array handling (required by ComfyUI)
• No unnecessary dependencies

Version Constraints:

• Minimum versions specified for compatibility
• No maximum versions for flexibility
• All are stable, mature libraries

15. Type Annotations

Decision: No static type hints in initial version.

Rationale:

• ComfyUI uses string-based type system (INPUT_TYPES)
• Python type hints add verbosity
• Runtime behavior is clear from INPUT_TYPES definitions
• Can add later if codebase grows

16. Testing Strategy

Decision: Manual testing with Mosquitto, no automated tests in v1.

Rationale:

• MQTT requires external broker for testing
• Integration testing needs real MQTT server
• Manual testing sufficient for initial release
• README includes clear testing procedures

Future Enhancements:

• Unit tests with mocked MQTT client
• Integration tests with Docker Mosquitto
• CI/CD pipeline with automated testing

ComfyUI Integration Decisions

17. Category Placement

Decision: Place nodes in "image/mqtt" category.

Rationale:

• Nodes operate on images
• MQTT subcategory groups related functionality
• Logical placement in ComfyUI node browser
• Consistent with ComfyUI category conventions

18. Node Naming

Decision:

• Class: MQTTPublishImage, MQTTPublishImageJSON
• Display: "MQTT Publish Image", "MQTT Publish Image Metadata (JSON)"

Rationale:

• Class names are PascalCase, descriptive
• Display names are human-readable
• Clear distinction between binary and metadata variants
• Searchable in node browser ("mqtt", "publish", "image")

19. OUTPUT_NODE Flag

Decision: Set OUTPUT_NODE = True for both nodes.

Rationale:

• Publishing is a side effect/output action
• Indicates to ComfyUI that this node performs external operations
• Ensures execution even if outputs aren't consumed downstream
• Consistent with other output nodes (Save Image, etc.)

Security Considerations

20. Credential Handling

Current State:

• Username/password visible in ComfyUI UI
• No encryption of credentials in workflow JSON
• Transmitted over TLS when use_tls=True

Documented Risks:

• Credentials visible in screenshots/recordings
• Workflow files contain plaintext credentials
• Recommended to use TLS always with auth

Future Improvements:

• Environment variable support
• External config file
• Credential manager integration
• Masked password fields (if ComfyUI supports)

21. Certificate Validation

Decision: Always validate certificates when TLS enabled.

Rationale:

• Security over convenience
• Prevents MITM attacks
• Encourages proper broker setup
• System CA store handles most cases

User Impact:

• Must have valid certificates on broker
• Self-signed certificates require system installation
• Clear error messages for certificate issues

Performance Considerations

22. Image Encoding Performance

Decision: JPEG quality default of 95, PNG as alternative.

Rationale:

• Quality 95 balances size and quality
• Higher quality for AI-generated images
• PNG option for transparency/lossless needs
• User configurable for their use case

Benchmarks (estimated, 1024x1024 RGB):

• JPEG Q95: ~500-800 KB, fast encoding
• JPEG Q85: ~300-500 KB, fast encoding
• PNG: ~2-4 MB, slower encoding

23. Synchronous Operation

Decision: Block during publish with wait_for_publish().

Rationale:

• ComfyUI workflows are sequential
• Ensures message is sent before continuing
• Status reflects actual publish result
• Simpler than async callbacks

Latency (typical):

• Local broker: 5-20ms
• Remote broker: 50-200ms
• TLS adds: 20-50ms handshake

Future Enhancement Ideas

Potential Features for Future Versions

1. Batch Publishing: Send multiple images in single message or rapid succession
2. Compression: Optional gzip/zstd compression before sending
3. Image Downscaling: Publish preview/thumbnail alongside full image
4. Topic Templates: Dynamic topics with variables (timestamp, size, etc.)
5. Connection Pooling: Reuse connections across nodes
6. Custom CA Certificates: Load CA file for self-signed certificates
7. Client Certificates: Mutual TLS authentication
8. Last Will and Testament: MQTT LWT for connection status
9. Environment Variables: Credentials from env vars
10. Base64 Encoding: Option for embedding images in JSON
11. Binary Format Options: WebP, AVIF for better compression
12. Workflow Integration: Publish workflow JSON alongside image

Lessons Learned

Development Insights

1. ComfyUI Tensor Format: Understanding [B,H,W,C] float32 format crucial
2. PIL vs OpenCV: PIL chosen for simplicity and JPEG quality control
3. MQTT QoS: Must call wait_for_publish() for QoS 1/2 reliability
4. TLS Testing: Need valid certificates even for testing, can be frustrating
5. Error Messages: Clear status strings greatly improve user experience

Design Tradeoffs

1. Simplicity vs Features: Chose simpler v1, extensible architecture
2. Security vs Usability: TLS and auth optional for easy onboarding
3. Performance vs Reliability: QoS 1 default balances both
4. Flexibility vs Complexity: Exposed many options, kept sensible defaults

Version History

• v1.0 (2026-01-24): Initial release
  • MQTT Publish Image node
  • MQTT Publish Image Metadata (JSON) node
  • TLS support
  • Username/password authentication
  • JPEG/PNG format support
  • QoS 0/1/2 support
  • Retain flag support

References

• MQTT 3.1.1 Specification
• Paho MQTT Python Documentation
• ComfyUI Custom Nodes Guide
• MQTT Security Fundamentals

Contact

For questions about these decisions or suggestions for improvements, please open an issue or pull request.