Commit Graph
96 Commits
Author SHA1 Message Date
Ewald b2ef3d0f80 fix(assets): improve grid reload and scroll behavior
- Adjusting the infinite scroll loading point to 4 card rows from the bottom
- Improving grid state restoration after reloads
- Enhancing the early fetch mechanism for faster initial loads
- Optimizing execution runtime to handle large numbers of assets efficiently
- Updating viewer and run button behavior for better integration with ComfyUI
- Fix various bugs and edge cases
2026-04-20 23:10:42 +02:00
Ewald 4e80b95bbc feat(geninfo): add api_node_extractor with 28+ comfy-core API providers
- New module mjr_am_backend/features/geninfo/api_node_extractor.py
- Covers all comfy_api_nodes/ providers: Alibaba Wan, Kling AI,
  Luma Dream Machine, MiniMax/Hailuo, Runway, Vidu, Google Veo,
  Black Forest Labs (Flux), xAI Grok, Ideogram, Recraft, Moonvalley,
  Pixverse, LTX Video, ByteDance Seedance/Seedream, Google Gemini,
  OpenAI/Sora, Stability AI, Reve, Bria, Magnific, Topaz, HitPaw,
  Wavespeed, Quiver, Sonilo, ElevenLabs
- _refine_engine_type(): classifies engine.type from class_type fragments
  (txt2img, txt2vid, img2vid, img2img, vid2vid, audio)
- _populate_api_node_output(): extracts duration, resolution->size, ratio
- ByteDance dotted-key pattern handled by _populate_seedance_output
- Hooked into payload_builder._build_no_sampler_result (first fallback)
2026-04-20 11:58:37 +02:00
Ewald 792b2a376e fix(graph_converter): improve text fallback handling for primitive strings
- Add special handling for primitive string nodes in text fallback
- Set text and value from non-empty primitive string nodes
- Update tests to cover new behavior
2026-04-15 15:12:39 +02:00
Ewald c3f4cca73a Support string concatenation in geninfo prompt tracing 2026-04-15 11:12:19 +02:00
Ewald d72a31f4e3 Fix Florence vector health regression and sync backfill polling exports 2026-04-15 10:22:27 +02:00
Ewald 7b00cd9e50 Fix Florence-2 startup and refresh UI bundle 2026-04-15 09:44:21 +02:00
Ewald 379eb96df9 fix: allow index directory override to create non-existing path 2026-04-14 20:41:19 +02:00
MajoorWaldiandMajoorWaldi 895b49c0f5 fix: add missing count query response in test_search_global_fts_rows_and_search_fts mock
Agent-Logs-Url: https://github.com/MajoorWaldi/ComfyUI-Majoor-AssetsManager/sessions/031cf51d-799c-423b-9ed7-af675f137262

Co-authored-by: MajoorWaldi <129623145+MajoorWaldi@users.noreply.github.com>
2026-04-14 15:36:49 +00:00
Ewald 1493e76254 Fix viewer stepping and geninfo regressions 2026-04-14 17:19:13 +02:00
Ewald 8ae71eb31e Fix floating viewer playback and search responsiveness 2026-04-14 15:23:35 +02:00
Ewald 97649c9887 perf/test: audit backlog — lazy viewer imports, structured logging, perf budget + unit tests
- ViewerRuntime.js: convert 6 heavy static imports (abCompare, sideBySide,
  model3d, scopes, genInfo, frameExport) to dynamic lazy imports with
  _prewarmViewerModules(); all 8 call sites updated with optional chaining
- logging.js: add mjrDbg() export (gated by APP_CONFIG.DEBUG_VERBOSE_ERRORS);
  replace labeled console.debug() calls in bootstrap.js, useGridLoader.js,
  entry.js
- ttlCache.vitest.mjs: add 4 LRU edge-case tests (10k flood, re-set order,
  post-expiry pruning)
- grid_perf_budget.vitest.mjs: new CI perf-budget suite — 5 tests at 10k+
  assets (budget 400ms, actual ~40ms)
- backfill_jobs.py: new route module extracted from db_maintenance handler
- test_backfill_jobs.py: 61 direct unit tests covering all public functions
- test_db_corruption_recovery.py: new recovery scenario tests
- auth/fetch test suites: auth_refresh_race, client_auth_token, fetch_utils
- js_dist rebuilt

Tests: 393 JS passed, 1428 Python passed, 2 skipped
2026-04-11 11:56:01 +02:00
Ewald a2053d0cba refactor(v2): plan V2 complet — splits JS, tests frontend, durcissement securite
Chantier F - Typage Python
- ruff S101 déplacé en per-file-ignores (pyproject.toml)
- pyrightconfig.json : mode standard confirmé

Chantier G - Assertions & bare-except
- 16 blocs except documentés dans config.py
- --cov-branch activé, seuil coverage 60→68%

Chantier H - Cache settings
- Stratégie documentée dans settings.py
- _resolve_index_dir() logguée dans config.py

Chantier I - Split gros fichiers JS
- client.js 1509L → client.js 406L + clientAuth.js + clientOps.js + fetchUtils.js
- toolbar.js 927L → toolbar.js 458L + toolbarActions.js + toolbarControls.js
- FloatingViewer.js → FloatingViewer.impl.js + floatingViewer*(8 modules)
- Viewer_impl.js → ViewerRuntime.js + viewerShell.js + viewerThemeStyles.js + viewerOverlayDismiss.js
- model3dRenderer.js → model3dRenderer.impl.js + model3dCore.js + model3dLoaders.js + model3dSupport.js
- dialogs.js → dialogs.js + DialogTemplates.js

Chantier J - Tests frontend (Phases 1-2)
- use_drag_drop.vitest.mjs : 5 tests
- use_grid_state.vitest.mjs : 18 tests
- security_client_headers, security_deep_merge, security_runtime_state, runtime_store, dialog_templates
- Total: 80 fichiers test / 350 tests vitest

Chantier K - Pinia migration
- runtimeState.js entièrement migré vers Pinia
- getOptionalRuntimeStore.js + runtimeEnrichmentState.js ajoutés

Chantier L - Durcissement sécurité
- tests/security/test_rate_limit.py : 8 tests rate-limit (state machine complète)
- clientAuth.js : docstring sessionStorage security boundary
- three.js 0.183.2 audité : 0 CVE (Snyk)

Chantier M - Simplification config
- deps.py, tool_detect.py, utils.py nettoyés

docs: PLAN_REFACTO_V2_LONG_TERME.md ajouté et clôturé
.gitignore: coverage/ ajouté
2026-04-11 09:38:41 +02:00
Ewald d3fcc16980 fix(features): improve generation time parsing and handling
- Add support for various time units (ms, s, m, h) in generation time
- Enhance parsing logic to handle different input formats and invalid values
- Update database write and hydration processes to preserve generation time
- Add tests for new parsing capabilities and ensure generation time is maintained across quality upgrades
2026-04-10 16:08:14 +02:00
Ewald 847fccf087 refactor(mjr_am_backend): complete backend refactoring and update documentation
- Commit DB split, security, registry, and observability lots
- Create missing asset management sub-services
- Extend Vue panel tests and implement teardown
- Update dependency policy and ADRs
- Document refactoring progress and next steps
2026-04-10 06:25:52 +02:00
Ewald f9a72d2507 Refactor SQLite facade and introduce FTS schema management
- Delegated column validation and repair functions to transaction_manager.
- Introduced new schema management helpers for full-text search (FTS) in schema_fts.py.
- Added schema_sql.py for SQL constants and identifier safety helpers.
- Implemented vector embeddings schema repair in schema_vec.py.
- Updated tests to accommodate changes in function signatures and behavior.
2026-04-09 17:14:22 +02:00
Ewald 57dfbaddb7 refactor(mjr_am_backend): update architectural plan and status
- Review and update the master plan against the current state of the repo
- Formalize the end of the frontend migration in the master plan
- Add new modules and changes to the checklist
- Update the target structure and criteria for acceptance
- Add new sections for database splitting, handlers separation, and registry/security/observability clarification
- Update the summary and operational resume
2026-04-09 11:08:10 +02:00
Ewald f5917eef6e feat(execution): implement idle grace period and enhance asset handling
- Add EXECUTION_IDLE_GRACE_MS config to control post-execution delay
- Implement post-execution work scheduling to optimize resource handling
- Enhance asset upsert logic for live assets during stack finalization
- Add runtime status synchronization to backend
- Improve execution start/end handling with new job ID tracking
- Refactor live asset event processing to reduce complexity
- Optimize imports and event handling in various modules
2026-04-09 01:55:32 +02:00
Ewald 79f98217c5 feat(viewer): enhance video controls and asset navigation
- Add keyboard shortcuts for video in-point (I), out-point (O), and navigation (Home/End)
- Implement swipe gesture for navigating between assets in the viewer
- Add option to copy prompt to clipboard (Ctrl/Cmd+C)
- Improve stack group card functionality and representative selection
- Optimize image preloading for smoother navigation
- Enhance audio visualizer with WebGL support and mode switching
- Refine feed card appearance and interaction
2026-04-08 20:50:26 +02:00
MajoorWaldi 8b0b77051b feat(assets): implement grid snapshot caching
- Add grid snapshot caching to improve performance and reduce server requests
- Implement sessionStorage-based cache storage with size and TTL limitations
- Update grid loader to interact with the snapshot cache
- Modify asset upsert logic to work with cached snapshots
- Add functions to remember and retrieve snapshots
- Update CSS class names for grid skeleton layout
2026-04-08 13:48:20 +02:00
Ewald 93b9016221 feat(config): add output directory override handling and cleanup index directory settings 2026-04-07 17:34:20 +02:00
MajoorWaldi 9f9480e2f9 feat(settings): add index directory setting for Majoor
- Implement new API endpoints for getting and setting index directory
- Add corresponding frontend functionality in settings advanced section
- Introduce new toast messages for index directory-related actions
- Update i18n dictionary with new index directory-related translations
2026-04-07 16:39:13 +02:00
Ewald ba272531ef feat(viewer): enhance player bar manager with active video asset tracking
- Added `_activeVideoAssetId` to track the currently active video asset in `playerBarManager.js`.
- Updated sync logic to ensure controls are remounted only when the asset changes.
- Improved handling of video synchronization and FPS settings.

feat(sideBySide): sanitize prompts for display

- Introduced `sanitizePromptForDisplay` to clean up prompts before displaying.
- Updated `_extractGridGenInfo` to use the new sanitization function for prompts.

feat(sidebar): enhance workflow section with size options and stats

- Added size options for the minimap in `SidebarWorkflowSection.vue`.
- Implemented workflow statistics display, including nodes, links, and groups.
- Persisted size and label settings for the minimap.

fix(generation): prevent filepath prompts from being displayed

- Updated prompt handling in `generationSectionState.js` to sanitize prompts.
- Added tests to ensure filepath-like prompts do not affect UI behavior.

feat(gridLoader): implement adaptive paging for grid loader

- Introduced `getAdaptivePageLimit` to dynamically adjust page size based on empty append batches.
- Updated `useGridLoader` to utilize adaptive paging logic.

feat(vectorIndexer): add regex for path-like prompts

- Implemented `_looks_like_path_prompt` to identify and handle path-like prompts in `vector_indexer.py`.

test: add unit tests for prompt sanitization and grid loader

- Created tests for prompt sanitization logic in `generation_prompt_sanitization.vitest.mjs`.
- Added tests for adaptive paging behavior in `grid_loader_adaptive_paging.vitest.mjs`.
- Implemented tests for sidebar workflow section functionality in `sidebar_workflow_section.vitest.mjs`.
- Added tests for player bar FPS synchronization in `viewer_player_bar_fps_sync.vitest.mjs`.

docs: update user guide and add privacy/offline documentation

- Added a privacy and offline usage guide in `PRIVACY_OFFLINE.md`.
- Updated user guide to reference new privacy documentation.
2026-04-07 10:35:43 +02:00
MajoorWaldi 205b8c51b7 refactor(features/index): enhance Torch device management and model migration
- Add functions to resolve preferred Torch device and move models/tensors to device
- Update model loading and feature extraction processes to use specified device
- Improve error handling and compatibility with different Torch backends
2026-04-06 19:48:57 +02:00
MajoorWaldi fbd1d182e8 fix(dnd): include avif images in ComfyUI metadata stripping
- Updated the _STRIPPABLE_EXTS set to include the .avif extension
- This change ensures that AVIF images are considered for ComfyUI metadata stripping during the drag-and-drop operation
2026-04-06 18:28:58 +02:00
MajoorWaldi df96cac6ac feat(settings): add verbose logging options for route registration and startup
- Add new API endpoints for route logging settings
- Implement new settings in the advanced settings page
- Update installation instructions to reflect the correct WinGet package ID for ExifTool
2026-04-06 18:07:19 +02:00
MajoorWaldi 977179de90 feat(security): implement settings-first remote setup flow and improve documentation
- Add Recommended Remote LAN Setup option in Settings for trusted LAN configurations
- Implement loopback recovery bootstrap for local browser sessions
- Automatically rotate legacy token hashes to new plaintext tokens
- Update documentation to reflect new remote setup process and security features
- Add runtime write authorization status indicators in Assets Manager
- Improve client-side token handling and session management
- Update environment variables guide with new bootstrap capabilities
2026-04-06 16:59:18 +02:00
MajoorWaldi 4f8f6307df feat: add remote access security settings and enhance initial setup experience for ComfyUI users 2026-04-06 12:01:15 +02:00
MajoorWaldi 07262353f4 feat: add remote access write permissions and bootstrap token handling; enhance error messaging for write access issues 2026-04-06 10:26:23 +02:00
MajoorWaldi c860f38fbd Refactor executable path resolution in ExifTool and FFProbe; enhance vector search stats with coverage fields; improve DB backup naming and handling; add tests for new functionality 2026-04-06 09:41:58 +02:00
MajoorWaldi fbd35cfd0d feat: enhance executable resolution for ExifTool and FFprobe with candidate aliases and improved path handling 2026-04-06 08:38:06 +02:00
MajoorWaldi 99a429dffe feat: add LiveStreamTracker and entry module for enhanced video and image processing capabilities 2026-04-04 22:28:49 +02:00
Ewald 5adc81769d feat: ajouter la gestion des doublons d'actifs avec un bouton de pile de doublons et des améliorations de performance 2026-04-04 15:46:50 +02:00
Ewald d66fb7ae70 feat: ajouter la normalisation du temps de génération et des tests associés pour les balises d'actifs 2026-04-03 19:25:05 +02:00
MajoorWaldi 19d134ba45 feat: Add tests to verify stack finalization for prompt IDs during file indexing 2026-03-29 21:34:49 +02:00
MajoorWaldi 3cd7f0f334 feat: Improve cover asset handling with error propagation and new test cases 2026-03-29 20:46:19 +02:00
MajoorWaldi cce569f39a feat: Enhance asset management with watcher toggle and vector runtime improvements 2026-03-29 16:04:59 +02:00
MajoorWaldi c9e6092ea4 Corrige les conflits de merge dans health.py : fusionne les fonctions et supprime les marqueurs de conflit. 2026-03-29 15:37:11 +02:00
MajoorWaldi 5c17eb83d4 Refactor hybrid search and vector search handlers for improved query filtering and asset management
- Added `_build_asset_filter_query` and `_query_allowed_asset_ids` functions to streamline asset filtering in hybrid search.
- Enhanced `_filter_semantic_hits` to utilize new asset filtering methods.
- Refactored vector search functions to improve K-means clustering logic and asset filtering.
- Introduced helper functions for parsing and validating query filters in `query_sanitizer.py`.
- Improved viewer context resolution in `viewer.py` with better error handling and path validation.
- Updated schema healing tests to use helper functions for column validation.
- Enhanced test coverage for vector indexing and filesystem entries.
- Added new test images for filesystem entry tests.
2026-03-29 15:33:59 +02:00
Ewald 44c0c57f6e feat: Add asset management and real-time listener functionalities
- Implemented entry UI registration for the Assets Manager, including sidebar and bottom panel integration.
- Created real-time listeners for execution events, asset updates, and enrichment status.
- Developed execution runtime controller to manage job tracking, asset buffering, and indexing.
- Introduced lazy initialization for vector services to optimize resource usage.
- Added HTTP handlers for plugin management, enabling listing, enabling, and reloading of plugins.
2026-03-29 01:03:49 +01:00
MajoorWaldi 925b718e38 feat: Update vector backfill logic to use backfill_id and enhance stack management 2026-03-28 20:31:37 +01:00
MajoorWaldi 71ef418bdd feat: Implement sibling asset hiding logic for PNG files
- Added functions to manage sibling asset states and determine if PNG siblings should be hidden based on their context (video, audio, model3D).
- Refactored asset appending logic to utilize the new sibling hiding functionality.
- Updated the backend to prioritize image assets over video assets in the same stack group.
- Introduced a new execution asset buffer to manage job-related assets efficiently.
- Created tests for sibling hiding logic and execution asset buffer functionality.
2026-03-28 17:58:23 +01:00
MajoorWaldi 0124a8f6d5 feat: enhance asset management with job finalization logic, improve coverage thresholds, and update documentation for AI/vector features 2026-03-28 15:35:28 +01:00
MajoorWaldi 4411c72e4e Refactor test imports and clean up unused code across multiple test files
- Removed unnecessary imports from various test files to improve readability and maintainability.
- Added new settings for the Generated Feed display in the settingsFeed.js file.
- Introduced a new configuration file for Pyright type checking.
2026-03-28 13:26:28 +01:00
MajoorWaldi c33256be68 feat: Enhance asset management with stack finalization logic and cleanup of job ID references 2026-03-27 17:59:28 +01:00
MajoorWaldi 68bc7b74ba feat: Implement generated feed tab with asset management and selection handling
- Added `GeneratedFeedTab.js` for managing and displaying generated assets in a bottom panel.
- Implemented asset selection logic with support for shift and ctrl/meta key interactions.
- Integrated loading of recent assets and handling of pending assets.
- Enhanced UI with new styles for compact asset cards in the bottom panel.
- Introduced event handling for runtime status updates in `StatusDot.js`.
- Updated `floatingViewerManager.js` to utilize the last selected grid for viewer actions.
- Created unit tests for comfy API bridge and dialog functionalities.
- Added backend support for model path discovery and asset indexing with prompt correlation.
- Enhanced scan routes to emit progress and asset indexing events.
2026-03-27 17:15:53 +01:00
MajoorWaldi 881cbc6ded feat: Add generation time hydration logic and corresponding tests 2026-03-27 15:12:20 +01:00
MajoorWaldi e198c46e0d feat: Implement asset stacks service and API endpoints
- Added StacksService to manage asset stacks, including CRUD operations and automatic stacking based on job_id and workflow_hash.
- Created API routes for stacks, including listing, retrieving by job_id, managing stack members, setting cover assets, dissolving stacks, merging stacks, and triggering automatic stacking.
- Introduced new ColorSpaces and Volume classes in the three.js addon for enhanced color management and volume rendering capabilities.
2026-03-27 15:06:41 +01:00
MajoorWaldi cb171e826d feat: Add runtime bootstrap helpers and search request normalization
- Introduced runtime bootstrap helpers in `mjr_am_backend.features.runtime` for dependency initialization.
- Implemented `apply_startup_settings` to handle various startup configurations.
- Added search request normalization helpers in `mjr_am_backend.features.search` including `SearchRequestContext` and `parse_search_request` for better query handling.
- Created audit logging helpers in `mjr_am_backend.routes.core.audit_log` for state-changing route handlers.
- Developed a quality gate script in `scripts/run_quality_gate.py` to enforce code quality checks.
- Added comprehensive tests for audit routes, scan consistency, vendor routes, and code quality metrics.
2026-03-26 20:42:51 +01:00
MajoorWaldi c7f93f0eab Refactor and optimize various components
- Improved URL parameter handling in `appendAssetFilterQueryParams` for better readability and maintainability.
- Added error handling for performance marks in `startTimer` to prevent potential issues.
- Enhanced `setRuntimeStatePatch` to avoid modifying blocked keys in the runtime state.
- Introduced hard caps on recent file and execution start tracking to prevent memory growth.
- Reduced workflow cache size to limit memory usage.
- Implemented safe escaping for asset IDs in `findAssetElement` to mitigate XSS risks.
- Ensured proper grid reference invalidation during asset manager panel cleanup.
- Enhanced debounce utility with cancel and flush methods for better control.
- Updated SQLite facade to improve error handling and schema population.
- Simplified custom roots store error messages for clarity.
- Added validation for upload filenames to prevent reserved names and control characters.
- Improved path resolution logic to avoid race conditions with symlinks.
- Enhanced health check routes with raw socket peer IP validation for security.
- Added collision handling in staging file resolution to prevent infinite loops.
- Updated tests to reflect changes in query building and validation logic.
2026-03-21 16:20:26 +01:00
MajoorWaldi bd8b540160 feat(nodeStream): introduce Node Stream system with adapter registry
- Implement NodeStreamRegistry for managing adapters that handle ComfyUI node outputs.
- Create BaseAdapter interface for defining adapter structure and behavior.
- Add DefaultImageAdapter for standard image output handling.
- Implement KnownNodesAdapter for well-known ComfyUI nodes with specific output formats.
- Introduce VideoAdapter for handling video output from nodes.
- Establish NodeStream feature flag for controlled rollout.
- Add tests for NodeStreamController to ensure proper behavior when feature is disabled.
2026-03-17 22:30:56 +01:00