Commit Graph
31 Commits
Author SHA1 Message Date
Ewald b255c25c49 refactor(filter): style popover neutre + fermeture via mjr:filters-changed 2026-06-02 16:58:15 +02:00
Ewald 31e5ae0d73 Fix asset grid refresh and media card handling 2026-06-02 16:25:20 +02:00
Ewald a363a77a6b feat(db): Phase C tag normalization — v18/v19 migrations, normalized FTS, readers v2, stop-write, drop legacy columns
- m018_fts_from_normalized_tags: replace FTS triggers to read from asset_tags+tags
  instead of the legacy JSON asset_metadata.tags column
- m019_drop_legacy_tag_columns: remove asset_metadata.tags and tags_text via
  create-copy-swap; includes invariant check (aborts if legacy tags not backfilled)
- registry.py: MIGRATIONS = [M017, M018, M019]
- schema_fts.py / schema_sql.py: FTS helpers and baseline triggers rewritten to
  source from normalized tables with asset_tags guard
- TagsRepository: normalized read path (get_tags_for_asset, list_assets_with_tag,
  search_assets_by_tags) + sync_from_legacy_row repair helper
- Readers v2: searcher, result_hydrator, metadata_helpers, updater, rating_tags_service,
  rename_service, stacks/service, duplicates/service migrated to asset_tags/tags JOIN
- Stop-Write: writes no longer update asset_metadata.tags / tags_text
- sqlite_facade: ahas_table helper; sqlite_recovery updated
- deps.py: TagsRepository wired into service container
- scripts/tag_drift_doctor.py: detects assets with legacy tags not backfilled
- ADR 0009, ARCHITECTURE_MAP, RATINGS_TAGS_COLLECTIONS, adr/README updated
- Tests: test_drop_legacy_tag_columns (2), test_fts_from_normalized (6),
  test_tags_dual_write, test_tags_normalized_readers + regression fixes across
  test_rating_tags_import, test_index_metadata_helpers, test_index_searcher,
  test_asset_updater, test_deps, test_schema_heal
- Full suite: 1687 passed, 2 skipped, 2 pre-existing failures (test_health_routes)
2026-05-22 15:00:27 +02:00
Ewald b18cafcb3b Integrate Comfy core asset ingestion 2026-05-21 18:56:04 +02:00
Ewald 80c0aad5e7 audit profond : corrections et améliorations 2026-04-30 08:42:27 +02:00
Ewald c024e71fc8 Fix runtime index override during DB bootstrap 2026-04-15 11:47:08 +02:00
Ewald 57dfbaddb7 refactor(mjr_am_backend): update architectural plan and status
- Review and update the master plan against the current state of the repo
- Formalize the end of the frontend migration in the master plan
- Add new modules and changes to the checklist
- Update the target structure and criteria for acceptance
- Add new sections for database splitting, handlers separation, and registry/security/observability clarification
- Update the summary and operational resume
2026-04-09 11:08:10 +02:00
MajoorWaldi df96cac6ac feat(settings): add verbose logging options for route registration and startup
- Add new API endpoints for route logging settings
- Implement new settings in the advanced settings page
- Update installation instructions to reflect the correct WinGet package ID for ExifTool
2026-04-06 18:07:19 +02:00
MajoorWaldi 2fa41a0e76 feat: add useGridState and useVirtualGrid composables for asset management
- Implemented useGridState to manage grid state, including asset selection and loading states.
- Created useVirtualGrid to centralize virtualization helpers for improved performance.
- Added createVueApp for isolated Vue 3 + Pinia application instances.
- Introduced Vite configuration for building the ComfyUI-Majoor-AssetsManager with external dependencies.
2026-04-03 11:04:55 +02:00
MajoorWaldi 4411c72e4e Refactor test imports and clean up unused code across multiple test files
- Removed unnecessary imports from various test files to improve readability and maintainability.
- Added new settings for the Generated Feed display in the settingsFeed.js file.
- Introduced a new configuration file for Pyright type checking.
2026-03-28 13:26:28 +01:00
MajoorWaldi fb8a4d39a5 feat: Enhance database handling with new vector DB resolution and nested transaction support 2026-03-26 21:57:26 +01:00
MajoorWaldi cb171e826d feat: Add runtime bootstrap helpers and search request normalization
- Introduced runtime bootstrap helpers in `mjr_am_backend.features.runtime` for dependency initialization.
- Implemented `apply_startup_settings` to handle various startup configurations.
- Added search request normalization helpers in `mjr_am_backend.features.search` including `SearchRequestContext` and `parse_search_request` for better query handling.
- Created audit logging helpers in `mjr_am_backend.routes.core.audit_log` for state-changing route handlers.
- Developed a quality gate script in `scripts/run_quality_gate.py` to enforce code quality checks.
- Added comprehensive tests for audit routes, scan consistency, vendor routes, and code quality metrics.
2026-03-26 20:42:51 +01:00
MajoorWaldi 459b12ac39 feat(viewer): add 3D model viewing support
- Implement basic 3D model thumbnail rendering with fallback icon
- Update viewer mode selection logic to handle 3D models
- Add checks for 3D model interactions in the viewer
- Extend drag & drop support to include 3D model assets
2026-03-16 23:47:08 +01:00
MajoorWaldi 834ef1fef3 feat(docs): comprehensive AI features guide and new documentation
- Added new AI_FEATURES.md guide covering all AI-powered capabilities
- Updated DOCUMENTATION_INDEX.md to include new AI features guide
- Added AI Features section to README.md with overview and models used
- Improved backfill warning information in README.md
- Updated reset index / delete DB procedures in DB_MAINTENANCE.md
- Added configuration options for AI features in README.md
2026-03-06 12:43:46 +01:00
MajoorWaldi 08e07b5312 fix: replace silent catches with debug logging across multiple files
- Updated error handling in videoProcessor.js, videoProcessorWebGL.js, videoSync.js, viewerInstanceManager.js, and various utility files to log caught exceptions using console.debug.
- Introduced a new script to automate the replacement of silent catch blocks with logging statements.
- Enhanced safety checks in the database schema and transaction manager to prevent SQL injection vulnerabilities.
- Added regex patterns for validating column definitions and executable names in the ExifTool adapter.
- Improved error handling in health and filesystem routes to ensure better feedback on invalid inputs.
- Added tests to cover new error handling and validation logic.
2026-02-27 21:00:17 +01:00
MajoorWaldi e7c2443670 fix(security): improve API token handling and add server-side bootstrapping
- Move token from local to session storage on client side
- Set httpOnly cookie for server-side token on bootstrap/rotate
- Add token validation and auto-sanitization
- Implement retry logic for failed operations
- Add input validation for collection names
- Update toast durations and error handling
2026-02-27 10:09:44 +01:00
MajoorWaldi 9fa7ace7cc Add comprehensive tests for role_classifier, sampler_tracer, scan_diagnostics, scan_throttle, and extractors
- Introduced coverage tests for role_classifier.py, targeting various functions and edge cases.
- Added extra tests for sampler_tracer.py to cover uncovered branches and exception handling.
- Implemented tests for scan_diagnostics.py to validate error handling and unique filepath diagnosis.
- Created tests for scan_throttle.py to ensure proper functionality of throttling helpers.
- Developed additional coverage tests for extractors.py, focusing on various data handling scenarios.
2026-02-26 09:33:55 +01:00
MajoorWaldi 75261d636c Add comprehensive tests for SQLite facade, metadata extractors, and security helpers
- Introduced unit tests for SQLite facade helpers, including schema validation, query building, and async operations.
- Added integration tests for basic SQLite queries and transaction handling.
- Implemented additional tests for metadata extraction functions, focusing on dimension handling, EXIF data, and video metadata.
- Created tests for security core helpers, covering environment variable checks, IP validation, and CSRF protection.
- Included compatibility tests for route exports to ensure proper functionality.
2026-02-26 07:01:43 +01:00
Your Name fb1dabb134 feat: add JSON payload sanitization for non-finite floats and corresponding tests 2026-02-21 09:03:46 +01:00
Your Name 934f10b58a Refactor imports and improve type hints across multiple modules
- Organized and grouped import statements for better readability in health.py, metadata.py, releases.py, scan.py, search.py, version.py, viewer.py, and registry.py.
- Updated type hints to use the new union syntax (e.g., str | None) for better clarity in scan.py, settings.py, tool_detect.py, and result.py.
- Enhanced the handling of optional types in various functions to ensure more robust type checking.
- Removed unnecessary blank lines and improved formatting for consistency.
- Adjusted the logic in several test files to ensure compatibility with the updated import structure and type hints. (See <attachments> above for file contents. You may not need to search or read the file again.)
2026-02-20 19:13:15 +01:00
Your Name f38ab4209e Save changes 2026-02-19 22:44:26 +01:00
Ewald da847f35f9 Add regression tests for route security guards
- Implement tests to ensure file paths are properly allowlisted in custom view and folder info handlers.
- Verify that collections mutations enforce CSRF protection and write access guards.
- Check that settings mutations require write access for specific routes.
2026-02-18 14:37:13 +01:00
Your Name 7183c87400 Refactor filtersController to manage event listeners with lifecycle support
- Introduced `addManagedListener` function to handle event listeners with automatic cleanup.
- Updated event listeners for various filter controls to use the new management function.
- Added lifecycle signal support for event listeners to ensure proper disposal.
- Implemented cleanup logic to remove event listeners when no longer needed.

---

Update database schema handling to support async operations

- Changed `db.has_table` to `db.ahas_table` for async compatibility.
- Updated related database checks to use the new async method.

---

Refactor SQLite adapter to prioritize async operations

- Transitioned public API to expose async methods as primary.
- Removed synchronous methods and replaced them with async counterparts.
- Updated internal transaction handling to support async context management.
- Enhanced error handling and logging for async operations.

---

Add async support for FFProbe tool

- Implemented `aread` and `aread_batch` methods for asynchronous metadata reading.
- Utilized asyncio subprocess for non-blocking execution of ffprobe commands.

---

Enhance IndexScanner with I/O throttling

- Introduced `SCAN_IOPS_LIMIT` to control I/O operations during directory scans.
- Implemented pacing logic to manage scan I/O operations effectively.

---

Update MetadataService to utilize async reading methods

- Refactored methods to call async variants of exif and ffprobe reading.
- Improved error handling for async operations.

---

Implement path resolution checks in asset routes

- Added `_is_resolved_path_allowed` function to enforce allowed roots for file paths.
- Updated asset route handlers to validate resolved paths against allowed roots.

---

Refactor database maintenance routes for async handling

- Changed database close operations to use async `aclose` method.
- Updated error handling during database maintenance operations.

---

Update scan routes to utilize async database vacuuming

- Replaced synchronous database vacuum calls with async `avacuum` method.

---

Modify tests to support async database operations

- Updated test examples to use async context management for database interactions.
- Refactored integration tests to ensure proper async cleanup of database connections.
- Adjusted regression tests to count async transaction invocations correctly.
2026-02-16 01:05:48 +01:00
Ewald 00ee7806f7 feat: Add tool detection helpers for ExifTool and FFprobe
- Implemented caching for tool availability and versions.
- Added functions to check for ExifTool and FFprobe availability.
- Included version satisfaction checks against minimum required versions.
- Created utility functions for parsing boolean and float values from environment variables.
- Introduced shared utilities for error message sanitization and logging with emoji indicators.
- Established a Result pattern for error handling without exceptions.
- Added scan throttling helpers to manage background scans efficiently.
- Implemented time utilities for timestamps and performance measurement.
- Defined shared types, enums, and constants for consistent error handling and file classification.
- Created version management utilities to retrieve version information from pyproject.toml.
- Developed installation script for runtime dependencies.
- Added tests for ExifTool batch processing and various configuration scenarios.
- Implemented tests for version route aliases and runtime output root configuration.
2026-02-12 17:55:28 +01:00
Ewald 7d53165a5b feat: Enhance audio metadata extraction and TTS integration
- Added ExifTool and FFmpeg installation in CI workflow for audio processing.
- Implemented robust JSON parsing for workflow and prompt extraction in audio metadata.
- Introduced TTS-specific metadata extraction and fallback handling in geninfo parser.
- Updated sidebar components to display TTS-related metadata including voice, language, and parameters.
- Enhanced status dot feature to include database health checks and indexing status.
- Improved unit tests for ExifTool and FFProbe with mock responses.
- Added integration tests for TTS audio workflows to validate metadata extraction.
2026-02-09 16:17:38 +01:00
Ewald ALOEBOETOE 80800d1a9a Add features : Audio Extraction , and Improvements
Add features : Audio Extraction , and Improvements
2026-02-08 20:15:04 +01:00
Your Name 1646e2171b feat: Implement duplicate detection and similarity analysis features
- Added DuplicatesService for background duplicate analysis and similarity detection.
- Introduced endpoints for analyzing duplicates, checking status, and merging tags.
- Created utility functions for file hashing and perceptual hashing.
- Implemented watcher settings for runtime tuning of debounce and deduplication.
- Added database maintenance tools for resetting and deleting the database.
- Enhanced error handling with sanitization for client responses.
- Added tests for duplicates service and error handling.
- Updated ESLint configuration for better code quality checks.
- Introduced filename validation and logging utilities.
2026-02-08 15:11:17 +01:00
Ewald ALOEBOETOE 42da58b26f Format import statement in test_mvp.py 2026-02-07 17:36:44 +01:00
Ewald 640eccf422 chore: save workspace changes 2026-02-06 21:17:46 +01:00
Your Name f8898e964c Release 2.2.2: i18n support + viewer/minimap improvements 2026-01-30 03:37:28 +01:00
Your Name 8ed51fbf39 make assets manager docs, license, and code improvements 2026-01-24 19:28:33 +01:00