Review fixes: at settle a certain answer ends the wait only when the
lookup found a record or the answer is the inspector's or the recorded
saver's; the legacy floor waits for the re-check. A record without a
certain prompt ends the wait at once. Foreign writes need both
save_foreign_runs and monitoring_enabled, read again just before the
store. Dropped foreign files are logged outside the lock (evictions as
evictions, due ones too), and a drained re-check never drops a newer
reference.
The recorded run's stored prompt now links in one transaction with the file's
link rows, so a file linked to another prompt is a conflict instead of a
second link. The scan's stores run through survive_library_switch. A single
file asks history once (a one-file _BatchHistory), the scan passes
in_output=False and reads the file size only for files with prompt
metadata. A conflict row reads "Kept: already linked to a different
prompt". TESTING.md records the rules.
A foreign output (no positive PromptManager node) is decided by the shared
ladder with the MONITOR policy. Certain files are stored at settle with
RUN_MARKS through survive_switch; files only recorded history can decide
wait in their own bounded set and are re-checked once after the run's
outputs are sealed, with the sealed reference. The setting
GalleryConfig.SAVE_FOREIGN_RUNS is read for every file.
_ingest_scanned_file calls decide_file_prompt (scan policy) and stores the
result: pairs through ingest_resolved_pair and positives through
store_scanned_prompt, both with the scan marks. Files the default profile
leaves undecided now get the inspector's resolve, with the saver recorded
history names, before the legacy parser, so each branch of a shared-prefix
dataset run links its own prompt. Junk text is never stored, a file linked
to another prompt is a conflict instead of a second link, and a failed link
leaves no prompt behind. _BatchHistory reads the stored prompt and the
recorded saver from one recorded_runs pass. _parse_comfyui_prompt calls the
moved parse_comfyui_prompt. TESTING.md documents the decision.
database/scanned_prompts.py links a file to the prompt of one text, creating
it when new, inside one BEGIN IMMEDIATE transaction: a file linked to
another prompt is a conflict, a failed link leaves no prompt behind and two
writers store one prompt. pair_ingestion gains file_link_rows,
link_conflict and Marks/mark_created; ingest_resolved_pair takes optional
marks for the components it creates. recorded_runs and named_prompt in
run_attribution become public (#247).
The history route inspects a case-normalized target on Windows, while
history matches file names exactly, so the occurrence's reference names
the file.
Review follow-ups: the texts PromptManager recorded are looked up at the
sampler history's saver leads to, never at a sampler the unanchored walk
named; a file name naming a different saver than history anchors nothing;
outputs and texts are read in one transaction; "Recorded run" ranks below
every workflow-inferred source; unexpected lookup errors log at error level.
When a file's own graph leaves the prompt open, both inspect routes ask
recorded run history about the file (one run reported it and it was written
during that run). The save node that run reported it from anchors the walk
like a file-name match, so dataset branches that share one saver prefix
show their own prompt, labelled "Recorded run". Still open at one sampler
whose PromptManager text was recorded at execution (a language model's
output), the recorded text shows as "Execution recorded". Results the file
resolves alone never change; a failed lookup judges the file alone.
* fix(threads): stop and join the prompt cleanup thread, the image worker and the observer (#302, #370)
- PromptTracker: the cleanup thread waits on a stop event instead of
time.sleep; new shutdown(timeout=2) joins it and returns whether it ended.
- ImageGenerationHandler.shutdown(timeout=5) now joins the worker and
returns a bool; new close() is final: schedule() refuses and a sealed
run's history re-check is dropped, so no worker restarts afterwards.
- ImageMonitor.stop_monitoring(timeout=5) closes the handler first, then
stops and joins the watchdog observer within one deadline.
- Every test that builds a real tracker, handler or monitor ends with its
final stop and an assertion, before its folder and databases go.
- #370: the wrapper/session history test drains before dispose(); the
held-write bound test uses 30 s ceilings instead of tight wall clocks.
* test(history): drain the writer before asserting a bounded close it does not test (#370)
* fix(threads): a later stop waits again for a worker or observer an earlier stop left running (#302)
Review findings on #372:
- ImageGenerationHandler keeps the workers a stop signalled that had not
ended; every later shutdown/close joins them again, so a second call no
longer returns True while the first worker still runs.
- ImageMonitor.stop_monitoring keeps an observer or handler it left
running and stops and joins it again on the next call.
- A closed handler takes no awaiting history path, and a re-check that
raced close() is not kept due.
- Tests: second stops of a busy worker and a live observer, the closed
handler's re-check paths, the handle test asserts the worker ended and
stops before it asserts, and a no-new-thread check instead of set
equality.
* test(browser): the System reset restores every settings value, safety_backup included
The frontend coverage shard of #372 failed in workspace-system-tagging:
the review read "No safety backup (the policy is off)". The backups spec
saves safety_backup=false, and /__test__/system/reset restored config.json
but not PromptManagerConfig.SAFETY_BACKUP (nor INFINITE_SCROLL), so a
later spec on the same worker server saw the policy off. The reset now
captures and restores every non-tagging value POST /settings can change.
* fix(ingestion): link an image only to a prompt that provably produced it
An LLM fed a PromptManagerPair's prepend_text while its text editor was
empty. The node refused to save (an empty positive raised and logged a
warning), and the image monitor then linked the image to a stale tracker
entry or to the newest prompt in the database.
- Nodes: an empty text editor saves and tracks nothing (prepend and
append text are never the stored prompt), logs at info, and marks the
node's run key empty so no earlier entry can be used for its images.
- Tracker entries carry a run key: a digest of the prompt node and its
whole upstream graph, the identity ComfyUI's cache uses. The monitor
computes the same key from the image's embedded graph and uses a
queued or snapshot entry only when the keys match (a snapshot also
needs a known run with one entry, so a batch's last item is never
applied to every image).
- The recency fallback is gone; anything uncertain stays unlinked and is
logged at info.
- Recorded execution history attributes a file when exactly one run
reported it, the file was written during that run, and the run's
positive contexts name one stored prompt. The monitor and the output
scan use it as their last certain source.
- The scan and pair ingestion treat a resolved pair with an empty stored
positive as unresolved instead of raising.
- tools/review_image_links.py reviews selected links by these rules;
dry run by default, --apply changes only named ids after a backup.
Closes#342
* test(ingestion): real-host attribution case and the #342 testing contract
The managed host disables the image monitor, so the owned fixture pack
gains a route that starts and stops the plugin's monitor singleton for
the attribution case only. The case runs in both renderers: a typed Pair
image links, a Pair image whose positive is only linked prepend text
stays unlinked with nothing stored, an unknown node feeding text links
through its run key, and its cached re-run links again.
* fix(ingestion): take a queued entry only from the run the image appeared in
Dropping a key's older queue entries whenever the node ran again let a
later queued run, executing during the monitor's processing delay, take
the earlier image's entry. The snapshot taken when the image appears now
names the key's latest run, and the queue is popped by key and run:
same-key entries of earlier runs ahead of it are stale, entries of later
runs wait for their own images.
Also keeps the attribution case out of the stock real-host suite.
* test(ingestion): host lookups degrade to no proof
* fix(ingestion): address the #344 review (history re-check, keys, repair)
- Images without an embedded API graph: the scan now asks recorded
history before skipping a file with no prompt metadata, and the monitor
gives a keyless unlinked image one history-only re-check when its run's
outputs are recorded (a seal notification from the outputs write; at
most 1000 waiting images, each for an hour).
- Run keys: an image is unkeyable when any positive node has no key, so a
two-pass graph is never linked through the other pass; links follow
ComfyUI's is_link exactly.
- A failed save marks the node's run key like an empty editor, so an
older entry of that key can't stand in for this run.
- A placeholder is the latest entry: the image stays unlinked without
falling through to history, the same rule as a popped placeholder.
- An index on execution_run_outputs(filename, subfolder, root_key,
path_style) for the history lookup, added by migration and accepted by
the output schema check (any other index is still refused).
- Repair tool: unlink removes only the link record and only while the
image is still linked to the prompt it was judged on; every run prints
what --apply does.
- Real-host case proves the re-run's PromptManager node was cached.
* test(ingestion): real-host bare-output re-check and the review-round contract
A metadata-disabled save, held open behind the fixture gate after
SaveImage, waits unlinked while its run is open and links once the run's
outputs are recorded. The fixture's monitor route lists the waiting
files. TESTING.md covers the history re-check, the read index and what
the repair tool's --apply does.
* fix(ingestion): batch history lookups instead of adding an index
An index on execution_run_outputs would make the library unopenable by
older builds, whose schema check refuses any extra index there. The
migration and the schema-check allowance are reverted, so the history
schema is exactly main's. stored_run_prompts instead looks up a batch of
files with one parameterized filename IN (...) query per chunk of at
most 500 names and matches in memory; the scan computes it once per scan
batch.
* fix(ingestion): address the #344 re-review (seal race, linked roots)
- A sealed history re-check that can't be scheduled yet (the image's own
pass still pending, or the pending set full) is kept due and scheduled
when a pending image finishes, instead of being dropped.
- History matches a file through ComfyUI's spelling of its output folder
first, then with both sides resolved, so a symlinked or junction output
folder still attributes the scan's resolved paths; waiting files are
keyed by their resolved path.
- The scan makes its batch history lookup only when a file needs it.
- set_current_prompt pushes to the queue last, so marking a failed
execution never adds a second queue slot.
- A keyless image outside the output folder doesn't take a waiting slot.
Vito Sansevero