feat(database): migrate old workflow rows in batches, then reclaim space

The workflow storage migration rewrites old-form image rows (text inline)
to document ids, batch by batch:

- Each batch is one BEGIN IMMEDIATE transaction of at most 200 rows or
  16 MB of text, found by a rowid range search and read one row at a time.
- Each text is stored or reused, then read back through the one accessor
  before its row changes; a text that would not come back identical fails
  and rolls back the whole batch.
- A value that is not text stays inline and is counted as protected.
- A crash or a stop leaves committed batches new form and the rest old
  form; both read the same, and a rerun continues. No media file is opened.
- Unreferenced documents are swept at the end.

The maintenance preview gains a read-only workflow_storage plan (records to
rewrite, inline bytes where SQLite has octet_length, the Reclaim space and
older-version notes). reclaim_space sweeps, runs VACUUM and truncates the
WAL so the freed pages go back to the disk.
This commit is contained in:
Vito Sansevero
2026-10-24 04:34:09 -07:00
parent b75c371430
commit 02b0e6813f
5 changed files with 789 additions and 4 deletions
+2
View File
@@ -25,6 +25,7 @@ from .execution_outputs import ExecutionOutputOperations
from .history_reads import HistoryReadOperations
from .health import HealthOperations, iter_consistency_issues
from .duplicate_copies import DuplicateCopyOperations
from .workflow_storage import WorkflowStorageOperations
from .workflow_documents import (
DocumentReader,
documents_of_rows,
@@ -346,6 +347,7 @@ class PromptDatabase(
HealthOperations,
MaintenanceOperations,
DuplicateCopyOperations,
WorkflowStorageOperations,
):
"""Database operations class for managing prompts."""