Commit Graph
46 Commits
Author SHA1 Message Date
DumiFlex 39529a669b feat(engine): migrate library v1->v2 (multi-tag) + bump schema version
engine/migrations/v1_to_v2.py: option sub_category->sub_categories[]; sub-category name slugify (whitespace/grammar chars->_, reserved suffixed, de-duped) cascaded to options + tag_groups; instance category_filter list->boolean-expr string + exclude_null flag; nested @{uuid:a,b,null} refs->@{uuid:a or b!null}. Idempotent. Two entry points share one core: envelope migrate_v1_to_v2 (import/export, registered in _CHAIN) + per-row migrate_module/bundle_v1_to_v2 (lazy/bulk DB path).

Register _CHAIN[1] + bump engine CURRENT_SCHEMA_VERSION 1->2. New SQL migration 016 marches stale DB rows 1->2 at boot via run_bulk_lazy + the per-row migrators (014's documented bump pattern). TS mirror (migrations.ts) + client CURRENT bump follow next (no cross-language version assertion couples them). Update version/entity-count assertions in payload-migration, migration-runner, and export-build tests.
2026-06-07 01:40:53 +03:00
DumiFlex bbd1a37bd8 feat(publish): writeback community origin to local row after publish
Publishing a local row to community never linked the row back to the
resulting post, so: no community pill, no update-available tracking,
and the re-publish version-bump flow was unreachable (origin slug was
always null -> publishMode could only ever be 'create').

Adds the writeback path:
- ModuleRepository / BundleRepository.set_community_origin(id, slug,
  version): stamps community_post_slug + community_version_number
  WITHOUT a version bump or fingerprint change -- origin metadata is
  not runtime-affecting, so a published row stays byte-identical to
  what consumers installed.
- wp_api: POST /wp/api/modules/{id}/community-origin + the bundles
  twin. Validates {post_slug: str, version_number: int}.
- api client: modules.setCommunityOrigin + bundles.setCommunityOrigin.
- main.ts host bridge: new optional `markPublished(localId, kind,
  slug, version)` method. Calls the right setCommunityOrigin, then
  refreshes both catalogs so the pill appears without a reload.
  Optional + try/catch-returns-false so a writeback miss never breaks
  the embed's publish-success path. No ABI bump -- adding a method is
  non-breaking; the embed guards on its presence.

Tests: set_community_origin stamps slug+version, does NOT bump
version/fingerprint, raises on missing row.

Pairs with the community embed commit that calls markPublished after
createPost / createVersion.
2026-06-06 06:48:43 +03:00
DumiFlex 395af63b1a feat(engine): content_rating column on library rows (migration 015)
Adds `content_rating TEXT NOT NULL DEFAULT 'safe'` to modules + bundles.
Defaults all pre-015 rows to 'safe'. Repository create/update accept a
new content_rating kwarg with validation against {'safe', 'nsfw'}.
Importer _insert_module / _insert_bundle pass entity.content_rating
through; _update_module / _update_bundle use COALESCE(?, content_rating)
so an install path that omits the key preserves the existing flag.

Why: user wants two NSFW behaviors:
1. Edit page toggle to mark a module/bundle as NSFW locally.
2. Community posts with content_rating='nsfw' auto-stamp NSFW on
   the local row at install time.

This is the engine half of that work -- the schema, the data
plumbing, and validation. The SPA half (IdentityCard toggle + pill
render on ModuleListView + install.ts plumbing) lands next.

Tests cover: migration head bump to 15, column existence on both
tables, safe default backfill, accepts nsfw, rejects invalid, update
toggle round-trip, update without content_rating preserves prior.
2026-06-06 05:22:02 +03:00
DumiFlex 73ca8a3f9a feat(engine): bulk lazy runner — eager iteration of lazy_migrate_row
Implements spec §4 Flow 4: bulk-at-boot is NOT a parallel forward-chain
implementation; it's eager iteration of lazy_migrate_row over every
stale row. Same original-first logic, same re-projection, zero
divergence. Eliminates the transient-wrong window where bulk could
write fabricated defaults over recoverable data in original_payload_json.

run_bulk_lazy returns the failure count; caller surfaces a dashboard
banner when > 0. STALE_PREDICATE catches both 'sv < current' AND
'tolerantly-installed row whose original now outranks the projection'.

Migration 014 invokes run_bulk_lazy at the end of up() — no-op pass
at v1, but validates the wiring is correct. Future bump migrations
(015 = v2, etc) will reuse the same helper.
2026-06-05 14:21:31 +03:00
DumiFlex 07fe3ab064 feat(engine): lazy_migrate_row — the ONE migration runtime
Implements spec §4 Flow 5 in full:
- No-op when row is at CURRENT with no future original
- NULL-drop optimization when row.sv == original.sv == CURRENT
- Recovery exact catch-up: promote original into payload (the §4 bug fix —
  write-back gate is 'base_v < CURRENT OR base IS original', not just
  'base_v < CURRENT'; otherwise exact catch-up silently no-ops)
- Still-drifted re-project: tolerantly strip original down to CURRENT
- Deferred strict-validate failure → tolerant_drift_status='broken'
- Standard forward chain otherwise

original_payload_json is inert storage: two writers only (initial
tolerant install + this NULL-drop). No other path rewrites it.
schema_migrated_at is separate from updated_at (structural migration
is not a user edit).
2026-06-05 13:46:49 +03:00
DumiFlex c46cbbf509 feat(engine): migration 014 - payload schema_version + verbatim-local mirror
Adds the four payload-versioning columns to modules + bundles per
the community-web spec sections 4-5:
- schema_version (NOT NULL DEFAULT 1)
- original_payload_json (NULL - only set during tolerant install)
- tolerant_drift_status ('none' | 'pending' | 'broken')
- schema_migrated_at (separate from updated_at - structural migration
  is not a user edit)

Backfill is implicit via the DEFAULT on the NOT NULL columns.
2026-06-05 12:50:54 +03:00
DumiFlex 9957702099 feat(community): track install origin + update-available indicator
Library rows installed from the community embed now carry the source
post slug + installed version_number in the engine sqlite. The SPA
uses both surfaces: a "community" pill next to the row name (so users
can tell at a glance which entries were authored locally vs pulled
from the embed), and a warn-coloured "v{N}" pill when the post has
shipped a newer version since install.

Engine:
- Migration 013 adds community_post_slug + community_version_number
  to modules + bundles. Both nullable; locally-authored rows stay
  NULL forever. Implemented as a .py migration with PRAGMA
  table_info guards so rerunning against an already-migrated DB
  (test fixtures rewind migrations to exercise older paths) is a
  clean no-op rather than "duplicate column name".
- _insert_module / _insert_bundle (importer.py) persist the fields
  when the entity dict carries them.
- _row_to_module / _row_to_bundle (repositories.py) surface them on
  every read path. Try/except guard for pre-013 DB shape so a
  connection opened mid-rollout does not crash.

Manager:
- installEnvelope grows optional `origin: { post_slug, version_number }`
  in InstallOptions. When set, stamps every module + bundle in the
  selection before the commit roundtrip.
- Host bridge in main.ts threads origin from `install(envelope, opts)`
  through to installEnvelope. ABI stays at 1 (additive).
- New communityUpdateStore walks the library snapshot, batch-fetches
  each unique community post (deduped, 8-wide concurrency), surfaces
  rows whose installed version is behind the post's
  latest_version_number. Network failures clear the store silently
  so an offline session does not render misleading "v2 available"
  pills.
- AppLayout kicks one check after the initial library fetch
  resolves. No background polling.
- ModuleListView renders two distinct pills per row: a static
  "community" indicator on origin-stamped rows, and a separate
  warn-coloured "v{N}" pill when the update store has an entry.

Migration tests updated for the new schema head + bundle/module
column sets.
2026-06-04 17:21:04 +03:00
DumiFlex 04d3795103 feat(api): expose db config get/put endpoints with pending-move 2026-06-03 02:16:51 +03:00
DumiFlex 747a84ac16 feat(engine): add user/global/root location path helpers 2026-06-03 02:09:20 +03:00
DumiFlex 52966e69d1 feat(engine): execute pending db moves at plugin load 2026-06-03 01:58:15 +03:00
DumiFlex c03fc070ba feat(engine): honor sidecar preference and add root db location 2026-06-03 01:54:05 +03:00
DumiFlex 069527a066 feat(engine): add sidecar db-config.json reader/writer 2026-06-03 01:45:39 +03:00
DumiFlex 5a62929f91 feat(engine): add run_migrations maintenance op 2026-06-03 00:03:17 +03:00
DumiFlex d39add1a44 feat(engine): add analyze maintenance op 2026-06-02 23:58:55 +03:00
DumiFlex 7f125756a3 feat(engine): add integrity_check maintenance op 2026-06-02 23:55:09 +03:00
DumiFlex 4abe679525 feat(engine): add vacuum maintenance op 2026-06-02 23:50:52 +03:00
DumiFlex 2b5f2b7af7 feat(engine): add gather_info DB introspection 2026-06-02 23:39:56 +03:00
DumiFlex 7e12626248 feat(engine): expose db path resolution source 2026-06-02 23:33:29 +03:00
DumiFlex 65080db689 feat(templates): add prompt template table, repository + REST API
Introduces a persisted prompt-template library: migration 012 creates the
templates table (name/description/category_id FK/tags/is_favorite/
template_string), TemplateRepository provides CRUD with partial-update
sentinel semantics, and wp_api/templates.py exposes list/get/create/update/
delete/favorite with FK violations mapped to 400. Bumps the migration-head
version assertions to 12.
2026-05-27 00:25:52 +03:00
DumiFlex 98730bc70b feat: polish branch — pair badges, bundle cascade, light theme, debug viewer, cleaner
Constraint pair badges:

- engine: via-nested carrier detection (computePairingsFull scans downstream wildcards' options for the target uuid; first match claims the slot as a CARRIER, sender + carrier share a PairingVia)

- PairBadge.vue: unified chip + Teleport popover for sender / direct / carrier / option surfaces. Carrier variant collapses inbound via-pairs into a single solid chip. Pair color hashed from senderRowKey+targetUuid so distinct senders avoid 8-slot palette collisions

- Threads viaOptionPairs from ContextWidget through ModuleEditModal -> WildcardInstanceModal -> PoolSection -> OptionRow for trailing chip in option value cells

Other polish:

- Bundle MISSING badge, push-to-library, cascade restore, fork-of-missing rebind, sidebar live counts, drift-store guard

- DebugViewer: RichTextPreview chips, library + preview-cache uuid resolve, constraint trace status 'never fired'

- Light theme: refchip/optionrow tokens, constraint matrix axes, kind colors, picker overlay

- Cleaner widget polish, autoheight, blocklist baseline, dangling_var post-strip

- Nested ref syntax #name segment, autocomplete dedupe by opts count + uuid, broken refs cached-name fallback

- Cascade: cleanup as opt-out checkbox, bundle delete scan + fixer, multi-root regression fix

- Migration 011 backfill ref names; test_migration_010 reset versions >=10 to coexist with 011
2026-05-26 11:21:05 +03:00
DumiFlex 9cab422e00 chore: rename to ComfyUI-Wildcard-Pipeline + add registry icon URL 2026-05-24 10:39:39 +03:00
DumiFlex c0d42aee5d feat(migrations): 010 backfill option ids and migrate exception schema 2026-05-24 06:27:02 +03:00
DumiFlex 93e644d466 feat(repo): backfill 8-hex ids for wildcard options on create/update 2026-05-24 06:23:43 +03:00
DumiFlex 913e3280e9 feat(engine): cascade_undo table migration 2026-05-24 00:55:40 +03:00
DumiFlex d0298fc6e3 fix(db): backfill NULL module fingerprints on next open
Migration 006 added the snapshot_fingerprint column but only populated
it on subsequent writes — every pre-006 row stayed NULL. The collision
detector correctly handles NULL as `exists-unknown` (amber EXISTING
badge), but a freshly exported-then-reimported untouched row still
showed EXISTING when content was byte-identical. Backfilling on first
open turns those into silent-skip (no badge), removing the visual noise.

Pure read+UPDATE — no schema change, no version bump on rows.
2026-05-23 22:00:31 +03:00
DumiFlex 17386f7990 feat(engine): atomic import commit + snapshot undo with 7-bucket dispatch
Adds engine/importer.py exposing commit_import / get_undo_entry /
undo_import. The committer applies a classified adds/replaces/renames
payload inside a single sqlite transaction, dispatching by `kind` to
the modules table (5 module types), bundles, or module_categories.
Module writes recompute snapshot_fingerprint; bundle writes recompute
payload_hash; categories use case-insensitive name-based merge
(matching the legacy import_bundle handler).

Every successful commit persists an import_undo row with enough
metadata (imported_records, replaced_snapshots, rename_map) to
reverse the commit. undo_import restores replaced rows bit-for-bit
(preserving original version + payload_hash + timestamps) and
deletes inserted rows, then evicts the undo entry.

Migration 007 creates the import_undo metadata table. Adds
17 importer tests covering adds across all 7 buckets, replace
snapshotting, rename with old→new mapping, collision rollback,
and undo restore/delete paths.
2026-05-22 21:52:11 +03:00
DumiFlex fb4ae039d0 feat(engine): backfill snapshot_fingerprint on module create/update 2026-05-22 17:50:11 +03:00
DumiFlex 6a0c0bf97b refactor: remove pipelines feature; add unified AllItems library view
Pipelines never paid off as a separate module kind, so peel out:

- Engine: drop pipeline_handler, remove "pipeline" from valid module
  types in repositories + schema parity.
- Manager: delete Pipelines/PipelineEditor views, PipelineSteps +
  PipelineStepPicker components and their tests. Router, sidebar,
  topbar, dashboard, TestRunner, ImportExport, resolver and
  bundleSelection repointed off the removed routes/types.
- wp_api: drop pipeline branch from modules.py + import_export.py
  (and its test).

In its place, add a unified library view:

- src/manager/views/AllItems.vue — modules + bundles in one filter
  panel with kind multi-select (OR across the 6 kinds). Reachable
  via the Dashboard "View all" link.
- ModuleListView grew the slots/props AllItems needs; BundleAddChild
  + Dashboard + ImportExport updated alongside.

Misc polish along the way: kind-icons / theme tokens / display-prefs
touch-ups, ContextWidget + DebugViewer + AssemblerHelper small fixes,
MockModuleCard + ModulePickerModal tweaks, _shared widget cleanup.
2026-05-17 20:50:26 +03:00
DumiFlex 78510e4e1e refactor(engine): resolve DB path via ComfyUI user dir with legacy fallback
Resolution order: WP_DB_PATH env > COMFYUI_USER_DIR env >
folder_paths.get_user_directory() > parents[4]/user path-traversal >
legacy ~/.comfyui. When the new ComfyUI-user location has no DB but
the legacy file exists, return legacy so existing installs keep data.
Tests short-circuit the chain via WP_DB_PATH.
2026-05-17 20:48:17 +03:00
DumiFlex a09a8e5833 feat(bundles): repository + CRUD API 2026-05-11 02:51:01 +03:00
DumiFlex 2f0e95db20 feat(db): migration 005 — bundles table for library-tracked groupings 2026-05-11 02:41:32 +03:00
DumiFlex 6921606570 feat(context): module picker + lock/internal + card indicators (#8)
Phase 5.5.4 (SPA library picker for ContextWidget) plus QA-driven polish.

* New ModulePickerModal with search, kind tabs, single + multi-pick, and
  compact filter UX (favorites/hide-added inline; category + tag chips
  behind popover with active-count badge).
* ModuleEditModal kind-aware body — wildcard option editor with
  enable/disable, weight overrides, mode picker (random/subset/pinned),
  sub-category multi-select.
* Lock + internal per-instance overrides ported and adapted: locked_seed
  derives a per-wildcard rng via sha256(seed:binding); internal hides
  bindings from the public PIPELINE_CONTEXT payload.
* Card indicators: modified dot (option-pool overrides), missing-from-
  library dot (polled hash check), in-card lock + internal toggles.
* RNG-faithful assembler preview via new /wp/api/preview/resolve; header
  shows "PREVIEW · 42". UNRESOLVED chips clickable to strip $var.
* Bulk collapse/expand toggle, focus toast w/ pulsing border overlay
  that tracks node bounds, friendly @name substitution in modal.
* Backend: new /wp/api/modules/import-from-workflow endpoint preserves
  uuid; engine emits per-module effective seeds via WP_Context UI
  payload for authoritative lock-toggle defaults.
2026-04-30 00:53:13 +03:00
DumiFlex 36e6e1159e fix(api): list /modules total reflects unpaginated row count
Dashboard count cards displayed "1" for every kind regardless of how
many rows the user actually had in the library. Root cause: the list
endpoint returned `total = len(items)` AFTER pagination, so any client
that passed `limit=1` for cheap polling (Dashboard does exactly this)
got `total=1` no matter what.

Adds `ModuleRepository.count(...)` that runs the same WHERE-clause
filters as `list()` but issues `SELECT COUNT(*)` with no LIMIT/OFFSET,
and reroutes both calls through a shared `_build_filter_clause` helper
so filter shape can never drift between the two paths.

Pinned by `test_list_total_ignores_limit` — seeds 3 wildcards, queries
`?type=wildcard&limit=1`, asserts items has 1 row but total is 3.
2026-04-28 23:01:27 +03:00
DumiFlex 759a71bbd9 refactor(db): preserve input order in get_by_uuids + document var-limit 2026-04-28 12:34:34 +03:00
DumiFlex d23e5b29f5 feat(db): add get_by_uuid + get_by_uuids indexed lookups 2026-04-28 12:29:30 +03:00
DumiFlex 3418796d9a feat(db): repository writes uuid + returns payload_hash on every row 2026-04-28 12:22:18 +03:00
DumiFlex d9c6ce758d test(db): expand migration 003 coverage + document partial-apply limit 2026-04-28 12:16:22 +03:00
DumiFlex 07e4df007c feat(db): add indexed uuid column to modules with backfill 2026-04-28 12:04:16 +03:00
DumiFlex 2d89a7391d feat(engine): scaffold combine/derivation/constraint/pipeline module types 2026-04-27 03:54:13 +03:00
DumiFlex 1ff08eed1c fix(api): atomic import + recursion guard + negative pagination + dry now_iso 2026-04-26 17:16:46 +03:00
DumiFlex ee428c83aa fix(db): unset sentinel for required fields + slug validation + delete consistency 2026-04-26 16:03:26 +03:00
DumiFlex ce6306dc74 feat(db): category repository with crud 2026-04-26 15:54:28 +03:00
DumiFlex 65d792ec61 fix(db): id entropy + like escaping + ms timestamps + sentinel typing
- Bump _gen_id suffix from token_hex(2) to token_hex(4) for 32-bit collision resistance
- Escape LIKE wildcards (%, _, \) in list(query=...) with ESCAPE clause
- Switch _now() and migrations.py to millisecond-precision ISO 8601 timestamps
- Add _Unset typed sentinel replacing object-typed ... default on update()
- Support offset without limit via LIMIT -1 OFFSET ? in list()
- Remove time.sleep() calls from module repository tests (ms precision makes them unnecessary)
- Add 3 new tests: percent/underscore literal search, offset-without-limit
2026-04-26 15:50:20 +03:00
DumiFlex d097f6e7c6 feat(db): module repository with crud and filtered list 2026-04-26 15:41:39 +03:00
DumiFlex 4195d300b9 fix(db): drop utcnow + atomic migrations + row factory test 2026-04-26 15:36:52 +03:00
DumiFlex cac9815026 feat(db): sqlite library foundation + migration runner 2026-04-26 15:26:59 +03:00