Commit Graph
163 Commits
Author SHA1 Message Date
Ewald 79cece3f01 fix: mypy type-ignore annotations in test_duplicates_service 2026-06-12 10:20:26 +02:00
Ewald 43a1fad285 chore: update dist, tests, and UI components 2026-06-12 10:19:33 +02:00
Ewald 5c31f29ecc Add workflow roots and fix video previews 2026-06-11 20:50:18 +02:00
Ewald 20a50040a6 Add workflow library scope 2026-06-09 10:28:25 +02:00
Ewald 4660c90b28 Fix CI event tests and bundle drift check 2026-06-03 15:05:49 +02:00
Ewald 99f6fe0ec3 test(types): satisfy mypy for StreamReader protocol stub 2026-06-02 17:22:07 +02:00
Ewald 8628122377 fix(ci): resolve ts typecheck and aiohttp test stub incompatibility 2026-06-02 17:21:42 +02:00
Ewald b255c25c49 refactor(filter): style popover neutre + fermeture via mjr:filters-changed 2026-06-02 16:58:15 +02:00
Ewald 31e5ae0d73 Fix asset grid refresh and media card handling 2026-06-02 16:25:20 +02:00
MajoorWaldi 99398aad5d Fix mojibake text and update default settings 2026-05-31 20:03:25 +02:00
Ewald a78f4572c8 Align frontend layout with ComfyUI conventions 2026-05-26 10:42:22 +02:00
Ewald 5bbf2364b2 fix(tests): align FORBIDDEN -> AUTH_REQUIRED for unauthenticated remote writes 2026-05-23 18:43:37 +02:00
Ewald 6587940862 refactor(app): enhance ComfyUI integration and settings sync - Implement two-way binding for ComfyUI settings in Majoor - Add new settings for auto-opening viewers and floating sidebar - Improve extension manager compatibility and dialog handling - Extend sidebar tab registration and controller discovery - Introduce graph canvas helpers for centering view and getting viewport bounds - Enhance minimap functionality with new graph canvas methods - Add settings sync with ComfyUI language 2026-05-23 18:35:21 +02:00
Ewald 37c65f984e fix(tests): add mypy None guards in test_fts_from_normalized + type-ignore in test_asset_updater 2026-05-22 15:03:38 +02:00
Ewald a363a77a6b feat(db): Phase C tag normalization — v18/v19 migrations, normalized FTS, readers v2, stop-write, drop legacy columns
- m018_fts_from_normalized_tags: replace FTS triggers to read from asset_tags+tags
  instead of the legacy JSON asset_metadata.tags column
- m019_drop_legacy_tag_columns: remove asset_metadata.tags and tags_text via
  create-copy-swap; includes invariant check (aborts if legacy tags not backfilled)
- registry.py: MIGRATIONS = [M017, M018, M019]
- schema_fts.py / schema_sql.py: FTS helpers and baseline triggers rewritten to
  source from normalized tables with asset_tags guard
- TagsRepository: normalized read path (get_tags_for_asset, list_assets_with_tag,
  search_assets_by_tags) + sync_from_legacy_row repair helper
- Readers v2: searcher, result_hydrator, metadata_helpers, updater, rating_tags_service,
  rename_service, stacks/service, duplicates/service migrated to asset_tags/tags JOIN
- Stop-Write: writes no longer update asset_metadata.tags / tags_text
- sqlite_facade: ahas_table helper; sqlite_recovery updated
- deps.py: TagsRepository wired into service container
- scripts/tag_drift_doctor.py: detects assets with legacy tags not backfilled
- ADR 0009, ARCHITECTURE_MAP, RATINGS_TAGS_COLLECTIONS, adr/README updated
- Tests: test_drop_legacy_tag_columns (2), test_fts_from_normalized (6),
  test_tags_dual_write, test_tags_normalized_readers + regression fixes across
  test_rating_tags_import, test_index_metadata_helpers, test_index_searcher,
  test_asset_updater, test_deps, test_schema_heal
- Full suite: 1687 passed, 2 skipped, 2 pre-existing failures (test_health_routes)
2026-05-22 15:00:27 +02:00
Ewald 9c19dfb6b5 feat(db): versioned migration runner + Repository pattern + normalized tags (Phase 3.2)
Introduces a forward-only, additive, idempotent migration framework on top of the existing legacy implicit-repair path (which remains the source of truth for schema versions <= 16). Versioned migrations live in mjr_am_backend/adapters/db/migrations/ and are tracked in a new schema_migrations bookkeeping table. The runner is invoked on every startup right after migrate_schema(); pending migrations are applied in version order, already-applied ones short-circuit.

First migration v17 normalize_tags creates a parallel tags + asset_tags relational layer (case-insensitive UNIQUE on tag name, cascade-delete on asset side, covering index on tag_id). Backfill reads asset_metadata.tags JSON safely (malformed rows skipped) and uses INSERT OR IGNORE everywhere for full idempotency. The legacy JSON column is NOT touched; existing v16 readers keep working.

Adds a Repository layer under mjr_am_backend/data/repositories/ to isolate raw SQL from services. First repository TagsRepository exposes ensure / get_by_name / list_all / attach / detach / list_for_asset / list_assets_with_tag / count on the new normalized tables, all returning Result.

Tests: 6 migration-runner tests (version validation, idempotency, bookkeeping, failure propagation, registry application) and 7 TagsRepository tests (CRUD, case-insensitivity, cascade-delete) - 13/13 passing. Existing test suite remains green (regressions limited to 2 pre-existing health_routes failures unrelated to this change).

Phase 3.2 closes the DB hardening item of the v3 improvement plan.
2026-05-21 23:15:55 +02:00
Ewald 625920846c feat(plugins): introduce Blueprint manifest format (Phase 3.3)
Adds a declarative blueprint.{yaml,json} manifest layer on top of the existing dynamic-module plugin loader. Each plugin can now ship as a self-contained package with a manifest describing name, version, author, type, entrypoint class, and capabilities. The legacy flat .py plugin format remains fully supported.

Manifest schema v1 with fields: schema_version, name (slug), version (semver), author, type (metadata_extractor), entrypoint.{module,class}, optional capabilities/tags/license/homepage/description/min_majoor_version. Path-traversal and module-name injection are rejected at parse time. YAML manifests require PyYAML (already shipped by ComfyUI core); JSON manifests need no extra deps.

Loader: PluginLoader.discover_plugins now treats subdirectories with a blueprint manifest as blueprint packages (taking precedence over plain Python-package scanning). New PluginLoader.blueprints property and get_stats[blueprints] field expose loaded manifests for listing endpoints.

Migrated bundled examples to blueprint layout: plugins/examples/wanvideo/, plugins/examples/custom_node_template/, plugins/examples/rgthree/ (YAML variant).

Tests: 11 new tests in tests/features/test_blueprint_plugins.py covering manifest parsing edge cases (bad name, bad semver, unknown type, path traversal, schema mismatch), YAML support, discovery skipping invalid manifests, and loader integration including a missing-entrypoint warning path.
2026-05-21 22:31:15 +02:00
Ewald 912ebb975d feat(api): add /api/v2/assets/* OpenAPI compat layer (Phase 3.1)
Read-only surface mirroring ComfyUI core OpenAPI assets spec under a /api/v2/ prefix to avoid collision with the existing /api/assets/* endpoints served by core.

Endpoints:

- GET  /api/v2/assets (list with name_contains, include/exclude_tags, sort, limit/offset)

- GET  /api/v2/assets/{id} (numeric ID, blake3:hex hash, or filename fallback)

- GET  /api/v2/assets/{id}/content (FileResponse, path-allowlist enforced)

- GET  /api/v2/assets/{id}/tags

- HEAD /api/v2/assets/hash/{hash} (existence check)

11 unit tests in tests/features/test_api_v2_assets.py covering payload shape, pagination, hash check (match/miss/malformed), 404 paths, and disallowed content paths.
2026-05-21 22:24:46 +02:00
Ewald 120c822c6f File path refacto 2026-05-21 21:07:16 +02:00
Ewald ddfe817b45 Polish asset manager UI and metadata helpers 2026-05-21 18:59:30 +02:00
Ewald b18cafcb3b Integrate Comfy core asset ingestion 2026-05-21 18:56:04 +02:00
Ewald 9876e36c22 chore: viewer, context menu, stacks, docs updates 2026-05-21 08:09:38 +02:00
Ewald 22648e5961 chore: commit remaining workspace changes 2026-05-17 21:21:49 +02:00
Ewald 4e7207a05d feat: update drag drop, docs, and build outputs 2026-05-15 17:13:04 +02:00
Ewald d174c1c141 test: fix mypy typing in asset operation service tests 2026-05-09 22:35:12 +02:00
Ewald 015c673d6d feat(viewer): add workflow graph map and viewer sync updates 2026-05-09 22:34:37 +02:00
Ewald 0bb9232ca7 Improve geninfo extraction and Happy Horse prompts 2026-05-06 23:14:48 +02:00
Ewald 8a7ab04558 Tighten remaining audit surfaces 2026-05-01 17:47:00 +02:00
Ewald 777d080494 Protect vector mutation routes 2026-05-01 17:36:12 +02:00
Ewald 0b72e50024 Harden mutation security defaults 2026-05-01 17:30:15 +02:00
Ewald 1ffd37f099 Fix mypy gate for search route tests 2026-05-01 04:36:42 +02:00
Ewald dbe876cff2 Fix asset manager pagination and preview stability 2026-05-01 04:35:04 +02:00
Ewald 80c0aad5e7 audit profond : corrections et améliorations 2026-04-30 08:42:27 +02:00
Ewald b9ab229555 feat: add VectorCaptionOnIndex setting (Florence-2 captions during indexing)
- Add AI.VectorCaptionOnIndex UI setting with rollback on API error
- Backend: persist/read vector_caption_on_index via AppSettings
- Expose set_vector_caption_on_index via health route and clientOps API
- Wire is_vector_caption_on_index_enabled() into config + env vars
- Update vector_indexer to honour the caption-on-index flag
- Add tests: settings backend, health routes, vector indexer video alignment
- Extend grid_scope_switch_cache test coverage
2026-04-25 12:12:04 +02:00
Ewald ALOEBOETOE 40945a91e6 test(arch): remove resolved WorkflowSidebar.js from KNOWN_FEATURE_VIOLATIONS
WorkflowSidebar.js no longer imports comfyApiBridge.js directly (logic
moved to WorkflowNodesTab). Sync the ADR 0007 ratchet set with the live
count so test_count_matches_known_violations_set passes.

Live count: 6 — KNOWN_FEATURE_VIOLATIONS now also has 6 entries.
2026-04-25 02:11:47 +02:00
Ewald 70a1a87c94 fix(security,arch): route WorkflowNodesTab through hostAdapter and reset prefs snapshot in tests
- Replace direct comfyApiBridge import in WorkflowNodesTab with getRawHostApp (ADR 0007).
- Reset security prefs snapshot in _reset_security_state_for_tests so tests that toggle MAJOOR_REQUIRE_AUTH are not influenced by an earlier ensure_security_bootstrap publishing require_auth=False.
- Call _reset_security_state_for_tests from _clear_write_auth_env helper for proper isolation.
- Rebuild frontend bundle.
2026-04-25 01:35:29 +02:00
Ewald b2ef3d0f80 fix(assets): improve grid reload and scroll behavior
- Adjusting the infinite scroll loading point to 4 card rows from the bottom
- Improving grid state restoration after reloads
- Enhancing the early fetch mechanism for faster initial loads
- Optimizing execution runtime to handle large numbers of assets efficiently
- Updating viewer and run button behavior for better integration with ComfyUI
- Fix various bugs and edge cases
2026-04-20 23:10:42 +02:00
Ewald 4e80b95bbc feat(geninfo): add api_node_extractor with 28+ comfy-core API providers
- New module mjr_am_backend/features/geninfo/api_node_extractor.py
- Covers all comfy_api_nodes/ providers: Alibaba Wan, Kling AI,
  Luma Dream Machine, MiniMax/Hailuo, Runway, Vidu, Google Veo,
  Black Forest Labs (Flux), xAI Grok, Ideogram, Recraft, Moonvalley,
  Pixverse, LTX Video, ByteDance Seedance/Seedream, Google Gemini,
  OpenAI/Sora, Stability AI, Reve, Bria, Magnific, Topaz, HitPaw,
  Wavespeed, Quiver, Sonilo, ElevenLabs
- _refine_engine_type(): classifies engine.type from class_type fragments
  (txt2img, txt2vid, img2vid, img2img, vid2vid, audio)
- _populate_api_node_output(): extracts duration, resolution->size, ratio
- ByteDance dotted-key pattern handled by _populate_seedance_output
- Hooked into payload_builder._build_no_sampler_result (first fallback)
2026-04-20 11:58:37 +02:00
Ewald 792b2a376e fix(graph_converter): improve text fallback handling for primitive strings
- Add special handling for primitive string nodes in text fallback
- Set text and value from non-empty primitive string nodes
- Update tests to cover new behavior
2026-04-15 15:12:39 +02:00
Ewald c024e71fc8 Fix runtime index override during DB bootstrap 2026-04-15 11:47:08 +02:00
Ewald c3f4cca73a Support string concatenation in geninfo prompt tracing 2026-04-15 11:12:19 +02:00
Ewald d72a31f4e3 Fix Florence vector health regression and sync backfill polling exports 2026-04-15 10:22:27 +02:00
Ewald 7b00cd9e50 Fix Florence-2 startup and refresh UI bundle 2026-04-15 09:44:21 +02:00
Ewald 379eb96df9 fix: allow index directory override to create non-existing path 2026-04-14 20:41:19 +02:00
MajoorWaldiandMajoorWaldi 895b49c0f5 fix: add missing count query response in test_search_global_fts_rows_and_search_fts mock
Agent-Logs-Url: https://github.com/MajoorWaldi/ComfyUI-Majoor-AssetsManager/sessions/031cf51d-799c-423b-9ed7-af675f137262

Co-authored-by: MajoorWaldi <129623145+MajoorWaldi@users.noreply.github.com>
2026-04-14 15:36:49 +00:00
Ewald 1493e76254 Fix viewer stepping and geninfo regressions 2026-04-14 17:19:13 +02:00
Ewald 8ae71eb31e Fix floating viewer playback and search responsiveness 2026-04-14 15:23:35 +02:00
Ewald f56d6284da fix: MFV popup dark theme, gen info panel, reset zoom + popout polish
- FloatingViewer.impl.js: add missing _resetMfvZoom() method (was called
  from floatingViewerLoader/Mode but never defined on the class)
- viewerShell.js: return genInfoTitle + genInfoTitleLeft from createViewerStageShell
- ViewerRuntime.js: destructure genInfoTitle + genInfoTitleLeft (were undefined
  => gen info body was never populated)
- floatingViewerPopout.js: inject document.documentElement.style.cssText as
  :root{...} in popup head so ComfyUI CSS vars are available; set body
  background to #0d0d0d; remove broken history.replaceState (Chrome blocks
  it from about:blank)
- __init__.py: remove unused FailureSeverity/StageStatus imports
2026-04-11 19:06:23 +02:00
Ewald 97649c9887 perf/test: audit backlog — lazy viewer imports, structured logging, perf budget + unit tests
- ViewerRuntime.js: convert 6 heavy static imports (abCompare, sideBySide,
  model3d, scopes, genInfo, frameExport) to dynamic lazy imports with
  _prewarmViewerModules(); all 8 call sites updated with optional chaining
- logging.js: add mjrDbg() export (gated by APP_CONFIG.DEBUG_VERBOSE_ERRORS);
  replace labeled console.debug() calls in bootstrap.js, useGridLoader.js,
  entry.js
- ttlCache.vitest.mjs: add 4 LRU edge-case tests (10k flood, re-set order,
  post-expiry pruning)
- grid_perf_budget.vitest.mjs: new CI perf-budget suite — 5 tests at 10k+
  assets (budget 400ms, actual ~40ms)
- backfill_jobs.py: new route module extracted from db_maintenance handler
- test_backfill_jobs.py: 61 direct unit tests covering all public functions
- test_db_corruption_recovery.py: new recovery scenario tests
- auth/fetch test suites: auth_refresh_race, client_auth_token, fetch_utils
- js_dist rebuilt

Tests: 393 JS passed, 1428 Python passed, 2 skipped
2026-04-11 11:56:01 +02:00
Ewald a2053d0cba refactor(v2): plan V2 complet — splits JS, tests frontend, durcissement securite
Chantier F - Typage Python
- ruff S101 déplacé en per-file-ignores (pyproject.toml)
- pyrightconfig.json : mode standard confirmé

Chantier G - Assertions & bare-except
- 16 blocs except documentés dans config.py
- --cov-branch activé, seuil coverage 60→68%

Chantier H - Cache settings
- Stratégie documentée dans settings.py
- _resolve_index_dir() logguée dans config.py

Chantier I - Split gros fichiers JS
- client.js 1509L → client.js 406L + clientAuth.js + clientOps.js + fetchUtils.js
- toolbar.js 927L → toolbar.js 458L + toolbarActions.js + toolbarControls.js
- FloatingViewer.js → FloatingViewer.impl.js + floatingViewer*(8 modules)
- Viewer_impl.js → ViewerRuntime.js + viewerShell.js + viewerThemeStyles.js + viewerOverlayDismiss.js
- model3dRenderer.js → model3dRenderer.impl.js + model3dCore.js + model3dLoaders.js + model3dSupport.js
- dialogs.js → dialogs.js + DialogTemplates.js

Chantier J - Tests frontend (Phases 1-2)
- use_drag_drop.vitest.mjs : 5 tests
- use_grid_state.vitest.mjs : 18 tests
- security_client_headers, security_deep_merge, security_runtime_state, runtime_store, dialog_templates
- Total: 80 fichiers test / 350 tests vitest

Chantier K - Pinia migration
- runtimeState.js entièrement migré vers Pinia
- getOptionalRuntimeStore.js + runtimeEnrichmentState.js ajoutés

Chantier L - Durcissement sécurité
- tests/security/test_rate_limit.py : 8 tests rate-limit (state machine complète)
- clientAuth.js : docstring sessionStorage security boundary
- three.js 0.183.2 audité : 0 CVE (Snyk)

Chantier M - Simplification config
- deps.py, tool_detect.py, utils.py nettoyés

docs: PLAN_REFACTO_V2_LONG_TERME.md ajouté et clôturé
.gitignore: coverage/ ajouté
2026-04-11 09:38:41 +02:00