The workflow storage migration rewrites old-form image rows (text inline)
to document ids, batch by batch:
- Each batch is one BEGIN IMMEDIATE transaction of at most 200 rows or
16 MB of text, found by a rowid range search and read one row at a time.
- Each text is stored or reused, then read back through the one accessor
before its row changes; a text that would not come back identical fails
and rolls back the whole batch.
- A value that is not text stays inline and is counted as protected.
- A crash or a stop leaves committed batches new form and the rest old
form; both read the same, and a rerun continues. No media file is opened.
- Unreferenced documents are swept at the end.
The maintenance preview gains a read-only workflow_storage plan (records to
rewrite, inline bytes where SQLite has octet_length, the Reclaim space and
older-version notes). reclaim_space sweeps, runs VACUUM and truncates the
WAL so the freed pages go back to the disk.
generated_images.workflow_data and prompt_metadata repeated the same large
JSON text for every image of a run. Each distinct text now lives once in
workflow_documents, keyed by the sha256 of its exact UTF-8 bytes and
zlib-compressed; image rows reference it through workflow_document_id and
prompt_document_id.
- New writes (link_image_to_prompt, pair ingestion for scan, monitor and
Needs review) store the new form. Old rows keep their inline text and
read the same until the migration rewrites them.
- Every reader goes through one accessor (read_stored_text,
resolve_image_row): gallery and legacy image lists, image by id, prompt
info for the inspector and metadata page, diagnostics. Payloads keep their
shape. Inline text wins when present; a document that fails its length or
hash check reads as unreadable, never as other text.
- No trigger (restore validators refuse them): a document goes in the same
transaction as its last reference (image and prompt deletes, missing-link
cleanup, duplicate merges, duplicate-file link merges, ingestion updates),
and a sweep at startup catches anything a path missed.
- verify_database_file validates the table, the id columns, their foreign
keys and indexes, and the stored values.
Vito Sansevero