From 4ab37a3a4cd53d162ea25db08af41ca75abb837b Mon Sep 17 00:00:00 2001 From: Christopher Connock Date: Sat, 11 Jul 2026 10:48:50 -0400 Subject: [PATCH] docs: design spec for the Photo (EXIF) node Co-Authored-By: Claude Fable 5 --- .../2026-07-11-photo-exif-node-design.md | 166 ++++++++++++++++++ 1 file changed, 166 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-11-photo-exif-node-design.md diff --git a/docs/superpowers/specs/2026-07-11-photo-exif-node-design.md b/docs/superpowers/specs/2026-07-11-photo-exif-node-design.md new file mode 100644 index 0000000..567b443 --- /dev/null +++ b/docs/superpowers/specs/2026-07-11-photo-exif-node-design.md @@ -0,0 +1,166 @@ +# Photo (EXIF) Node β€” Design + +- **Date:** 2026-07-11 +- **Status:** Approved (design), pending implementation plan +- **Component:** Sphere-Light-Render-ComfyUI custom nodes +- **Builds on:** `2026-07-08-input-driven-sphere-light-design.md` (graph-driven + inputs; since superseded by the client-side mechanism in `js/nodes.js` β€” + `connectedInputValue` + `hookSourceWidgets`) + +## Goal + +A new node, **πŸ“· Sphere Light β€” Photo (EXIF)**, that turns a photo into the +inputs the Sun nodes want. Upload a photo; the node reads its EXIF metadata and +exposes **latitude, longitude, city, heading, and the capture date/time** as +outputs that wire directly into πŸ”† Sun (City) / πŸ”† Sun (Coordinates). It also +outputs the photo itself as an `IMAGE`, so one upload feeds both the pixels and +the light direction (it can replace a Load Image node). + +End-to-end story: *photo in β†’ sphere lit the way the sun actually was when and +where the photo was taken.* + +## Decisions (from brainstorming) + +- **Browser parses EXIF; widgets are the data store.** The sphere nodes resolve + connected inputs client-side by reading a widget **by name** on the origin + node (`connectedInputValue` in `js/nodes.js`). So this node's parsed values + live in widgets named exactly `latitude`, `longitude`, `city`, `heading`, + `year`, `month`, `day`, `hour`, `minute` β€” and the existing driving/live + re-render mechanism works with **zero changes to the sphere nodes**. +- **Python is a pass-through** for the nine values (same "browser bakes values" + pattern as `render_b64`): widget values serialize into the prompt at queue + time, `execute()` returns them as outputs. No second EXIF parser in Python, + no new server routes (approach B/C rejected: duplicate parser buys nothing β€” + headless runs can't drive the sphere nodes anyway; a parse endpoint would + re-introduce the round-trip removed in `e53f000`). +- **The node needs the image *file*, not an `IMAGE` input** β€” tensors are + decoded pixels, EXIF is already gone. So it has its own Load-Image-style + upload widget. +- **Widgets stay editable.** A photo missing a tag (no compass heading is + common β€” phones only write `GPSImgDirection` when the compass was active) + can be corrected by hand right on the node; parsing only overwrites widgets + for tags actually present in the file. +- **Date/time included** (EXIF `DateTimeOriginal`), since sun position at + capture time is the point. **Image pass-through included.** + +## Node definition (Python, `__init__.py`) + +`SphereLightPhotoExifNode`, display name **πŸ“· Sphere Light β€” Photo (EXIF)**, +category `render/3d`. + +- **Inputs (all widgets):** + - `image` β€” file combo over the ComfyUI input directory with + `{"image_upload": True}` (standard Load Image pattern; the core frontend + supplies the upload button and on-node thumbnail). + - `latitude` FLOAT (βˆ’90…90, step 0.0001, default 0), `longitude` FLOAT + (βˆ’180…180, step 0.0001, default 0), `city` STRING (default ""), `heading` + FLOAT (0…360, step 0.01, default 0), `year`/`month`/`day`/`hour`/`minute` + INT (same ranges/defaults as the Sun nodes). +- **Outputs:** `IMAGE`, `latitude` FLOAT, `longitude` FLOAT, `city` STRING, + `heading` FLOAT, `year` INT, `month` INT, `day` INT, `hour` INT, `minute` + INT. +- **`execute()`:** load the file with PIL (`ImageOps.exif_transpose` so + orientation matches what the user saw), convert to the usual + `(1,H,W,3)` float32 tensor, and return it plus the nine widget values + unchanged. `IS_CHANGED` hashes the file (Load Image pattern). No MASK + output (YAGNI). + +## EXIF parser (`js/exif.js`, new β€” pure, dependency-free) + +`parseExif(arrayBuffer)` β†’ `{ lat, lng, heading, date }`, each `null`/absent +when the tag is missing. Internals: + +- **Container scan** to find the TIFF/EXIF payload: JPEG (APP1 segment tagged + `Exif\0\0`), PNG (`eXIf` chunk), WebP (RIFF `EXIF` chunk). Other/absent β†’ + "no EXIF". +- **TIFF/IFD reader:** endianness from `II`/`MM`; IFD0 β†’ GPS IFD pointer + (0x8825) and Exif IFD pointer (0x8769). + - GPS IFD: `GPSLatitude` (three rationals, DMS β†’ decimal) with + `GPSLatitudeRef` (`S` β†’ negative), same for longitude (`W` β†’ negative), + `GPSImgDirection` (rational, degrees from North β€” the repo's `heading` + definition already matches it per the README). + - Exif IFD: `DateTimeOriginal` (`YYYY:MM:DD HH:MM:SS`) β†’ `{year, month, + day, hour, minute}`. +- Every offset/count is bounds-checked; malformed data throws and the caller + treats it as "no EXIF" (the file arrives from the user's own disk, but a + truncated or repacked file must not wedge the node). + +Pure functions over an `ArrayBuffer` β€” unit-testable in Node with synthetic +byte fixtures, no DOM. + +## Node glue (`js/nodes.js`) + +A `setupPhotoExif(node)` branch in the existing `nodeCreated` extension: + +1. Add the same status line the Sun nodes use (`addStatus`). +2. Hook the `image` widget's callback. Parse **only when the image changes** + (upload or picking another file) β€” not at setup: widget values persist in + the workflow JSON, so re-parsing on reload would only serve to clobber + hand-corrected values. On change: + - Fetch the file from ComfyUI's `/view` endpoint (`filename`/`subfolder`/ + `type=input` derived from the widget value, handling the + `subfolder/name [type]` annotation format). + - `parseExif()` the bytes. + - For each tag present, set the matching widget **via its callback**, so + `hookSourceWidgets`' wrapper fires and any connected sphere node + re-renders live. Absent tags leave their widgets untouched. + - `city`: reverse-geocode with the existing `nearestCity(lat, lng)` and + write a string that `findCity` resolves back to the same record + (`City, RegionCode-or-Country`). + - Status reports what was found: + `πŸ“· 48.86, 2.35 near Paris Β· heading 214.50Β°`, with warnings for the + gaps: `⚠ no GPS data in this image`, `no heading tag`, + `no date/time tag`. + +## Data flow (driving the Sun nodes) + +Connect e.g. `heading β†’ heading`, `city β†’ city` (Sun City) or +`latitude/longitude β†’ latitude/longitude` (Sun Coords), plus the date/time +outputs. At queue time β€” and live, on photo change β€” the sphere node's +`connectedInputValue` follows the link to this node and finds the identically +named widget holding the parsed value. Nothing new to build on the consumer +side; this node just has to keep those widgets accurate. + +## Error handling + +- No/garbled EXIF, missing tags, unreadable file, fetch failure β†’ status-line + warning; widgets keep their current (hand-editable) values; never throws + into LiteGraph. +- Python `execute()` guards file loading the same way Load Image does; the + nine value outputs are pass-through and can't fail. +- Headless/API runs: outputs are the last browser-baked widget values β€” the + same documented limitation as every driven input in this repo. + +## Scope + +**In:** the new node class + registration; `js/exif.js`; the `setupPhotoExif` +glue; `tests/exif.test.js`; README section; `tools/test_comfy_load.py` +coverage. + +**Out:** HEIC parsing (browsers can't preview it and ComfyUI uploads rarely +carry it β€” revisit on demand); magnetic vs true north correction for +`GPSImgDirectionRef` (treated as-is, like EXIF viewers do); any change to the +three existing nodes; server-side EXIF parsing. + +## Testing + +- **`tests/exif.test.js` (node:test):** DMSβ†’decimal incl. S/W signs; rational + heading; `DateTimeOriginal` parsing; JPEG/PNG/WebP container location; + missing-tag and truncated/garbage buffers (no throw leaks); both + endiannesses. Fixtures are small hand-built byte arrays. +- **Python:** node registers and loads (`tools/test_comfy_load.py`). +- **End-to-end gate (Playwright on the local ComfyUI):** upload a real + GPS-tagged photo β†’ widgets fill, status shows the resolved city; wire + outputs into a Sun node β†’ sphere re-renders to match; photo without GPS β†’ + warning, widgets untouched. + +## Risks / trade-offs + +- **Hand-rolled EXIF parsing** β€” the format is stable and the needed subset is + small; bounded by tests over synthetic fixtures and real-photo verification. +- **Widget-name coupling** β€” driving depends on this node's widget names + matching the Sun nodes' input names; cheap to keep in sync, called out in + code comments on both sides. +- **`nearestCity` mismatch** β€” the nearest listed city can be a suburb, so the + `city` string may surprise; the status line shows exactly what was resolved, + and lat/lon outputs (Sun Coords) bypass the issue entirely.