- Blend a phase baseline with a 120s throughput window; reset the phase
on pause and skip bursts so a resumed run is not estimated from the
folders it skipped.
- Expose pause-aware elapsed_seconds and add a compact status response
for the 2s progress poll.
- Save .download_progress.json from both download loops; the old save
sat behind an early return on the common image path.
- Treat blank aiohttp errors and raised exceptions as transient, and
bound gallery media at 25s (videos 180s) instead of the shared 300s
socket read.
- Add tracker, manager, progress and timeout tests plus the full-run
measurements.
Expose Auto-Organize on the Workflows page and give workflows their own
download path template, which both downloads and Auto-Organize use.
- Add a `workflow` default ({base_model}) to the backend and frontend path
template constants. Local workflows rarely carry CivitAI tags, so the
default stops at the base model instead of adding a "no tags" segment.
- Enable the bulk Auto-Organize action for workflows; the shared endpoint
and scanner move already handled them, only the UI gate was missing.
- Add the workflow row to Settings > Organization, rendered hidden and
toggled by the existing Workflow management master switch so the config
only appears while the feature is enabled.
- Preview workflow templates with a .json artifact name instead of a
weight-file name.
- Cover the default template, download routing, both auto-organize modes,
sidecar moves, and the settings gate with tests.
Preview URLs now carry a cache-busting `&v=<mtime_ns>` suffix, so the media
type could no longer be read from the end of the string. Model cards,
duplicates, recipe cards and modals, the showcase thumbnail and the ComfyUI
preview tooltip all fell into the image branch and rendered `<img>` for video
previews, which browsers cannot decode.
Move the query-aware video detection that already lived in the versions tab
into a shared mediaTypes helper and use it everywhere a server-provided
preview URL is inspected. The web UI and the ComfyUI widget bundle each keep a
copy because they are served from different mounts. Video previews that turn
out to be webm now also get the right `<source type>`.
Add regression coverage for versioned mp4/webm/image URLs.
Fixes#1168
The sidebar menus (folder tree, view options) and the page menus (card,
bulk selection, page-global) are independent components that only knew
how to close themselves, so right-clicking a folder left an open card
menu next to the folder menu, and right-clicking a card left the folder
menu behind.
Add a coordinator where every menu registers a closer; opening a menu
first closes all the others. BaseContextMenu takes part automatically,
the sidebar registers its two menus, and the global contextmenu handler
now uses the coordinator instead of hiding three specific menus by hand
(which also covers right-clicks outside the content area).
Menus are `position: fixed`, so only the window can clip them, but they
were placed at the raw cursor position with a naive flip: near the bottom
edge the menu opened below the window (nothing to scroll it into view),
and a tall menu that flipped up could land above the top edge. The
sidebar folder menu and the view-options menu did no viewport math at all,
so right-clicking a folder at the bottom of the tree showed no options.
Add a shared positionFixedMenu helper that picks the side with room,
clamps on both axes, and caps the height (scrolling internally) when the
menu is taller than the window. BaseContextMenu uses it and exposes a
prepareMenu hook so per-card entries are toggled before the menu is
measured; the Recipe/Lora/Checkpoint/Embedding/Other/Workflow menus used
to mutate the menu after positioning and so measured the previous
content. Submenus get the same treatment via positionSubmenu.
Fixes#1167
The workflow card menu reused the model-library copy ("Exclude Model",
"Delete Model", "Copy Model Filename"), and the shared delete dialog
hard-coded "Model:" in its info box.
- Workflow-specific labels for copy-filename / move / delete, and for the
delete confirm dialog (title, message, info-box label, failure alert)
- Move dialog title, root label and hint now go through i18n instead of
hard-coded English; bulk delete dialog switches to workflow copy
- Failed move / nothing-selected toasts use workflow wording
- "Copy Workflow Filename" now copies the file name; it used to click the
card's copy icon, which copies the workflow document (duplicate of
"Copy Workflow")
- Fix "Link to CivitAI" posting to /api/lm/loras/relink-civitai: the
mixin's prefix map was keyed by singular type and missed the plural
page type used by WorkflowContextMenu
Also drops the "Exclude Workflow" action: workflows, like recipes, have
no exclusion concept. The file has no consumer outside LM, and the
excluded view is only reachable from the global context menu, so
excluding a workflow was a one-way hide. The entry is removed from the
card menu, the global menu and the excluded-view banner; the backend
route stays available.
- Replace the multi-root folder_paths.workflows workflow library with a
scalar workflow_path setting (Settings > Library); remove the
extra_folder_paths.workflows path and the standalone Model Paths editor.
- Migrate pre-release settings.json files that still keep
folder_paths.workflows / extra_folder_paths.workflows on startup, so an
upgrade keeps the configured folder instead of silently losing it.
- Add "Create LM-managed workflow" to the ComfyUI Graph menu and the File
menu/command palette: it writes the serialized graph into the configured
folder, stamps extra.lm_source and registers the entry in the scanner
cache without forcing a rescan.
- Attach the decode image of the client's last executed prompt as the
workflow preview. The prompt id is sent with the create request and the
backend only honors it while it is still the registry's current prompt,
so a tab never borrows another tab's output; otherwise the card keeps the
placeholder preview.
- Detect library entries deleted/renamed while a graph still carries the
marker: a link-status endpoint clears the stale extra.lm_source (on graph
load, menu open and before create/update), and the write-back route
returns 404 instead of recreating the document.
- Collapse the workflows.noPaths standalone/ComfyUI split into a single
description/hint; the folder is edited in Settings > Library in both
modes, so the manual settings.json guidance and the standalone
open-settings-folder fallback are gone.
- Add the preview tensor conversion as a shared utility (reused by the
recipe widget save path) and cover create, preview gating, link status,
migration and menu injection with tests.
- Replace the browser confirm() with a generic #confirmationModal
(template + ModalManager registration + showConfirmationModal() helper
in modalUtils.js, promise-based, danger-styled confirm button).
- Show the loadingManager overlay while the restore request is in flight;
.json restores re-download from CivitAI and the wait previously looked
like a dead click. The overlay hides in a finally block.
New card context menu entry 'Restore Downloaded Version' discards local
edits made via the write-back flow:
- Container artifacts (.zip/images) were never modified — restore deletes
the derived .workflow.json sidecar and re-extracts from the pristine
archive, then re-syncs analysis metadata. Purely local, works for
self-hosted workflows too.
- Bare .json artifacts are rewritten in place by write-backs, so restore
re-downloads the recorded CivitAI version file (API-key auth, temp file
+ workflow validation + SHA256 check before the atomic replace).
The menu entry is disabled for local-only .json workflows that have no
restore source. On success the card is refreshed in place with the
shared 'Updated' cue.
The write-back result carries the new document hash; the LM page now
refreshes the card in place via virtualScroller.updateSingleItem, which
re-renders it with the shared 'Updated' indicator.
Add an Update from ComfyUI entry to the workflow card context menu,
enabled only when the workflow is the active graph in a connected
ComfyUI tab and carries unsaved edits.
ComfyUI side (workflow_writeback.js): the first graphChanged after a
load/write-back flips a dirty flag inside the extra.lm_source marker
(ChangeTracker only dispatches on real state changes); the active
graph's link state (file_path, sha256, dirty) is reported to
POST /api/lm/workflows/comfyui-state on load, tab switch, dirty flip
and write-back. A new lm_collect_workflow WS handler answers LM-page
write-back requests: it reuses the regular PUT content flow and posts
the outcome to /api/lm/workflows/collect-result.
Backend: an in-memory WorkflowLinkRegistry holds per-client link states
(pruned against live websocket sids) and pending collect futures;
POST /api/lm/workflows/update-from-comfy targets the tab holding the
requested file (dirty holders win) and awaits the collect result,
mapping no_changes/cancelled to 200, timeout to 504.
The card send/copy icons and the model modal's Send to ComfyUI button
only had lora/checkpoint/embedding branches; workflows fell through to
a silent no-op (modal) or lora-ish behaviour with wrong tooltips (card).
- shared helpers sendWorkflowToComfy / copyWorkflowToClipboard in
utils/workflowActions.js, reused by the context menu, card icons and
the model modal
- card send icon opens the workflow in the running ComfyUI graph; the
copy icon copies the workflow document JSON for paste-to-load
- correct tooltips (Open in ComfyUI / Copy workflow to clipboard)
- send entry points hidden in standalone mode (no graph to push into):
card icon and modal button, matching the context menu
The fallback produced WORK (first four letters), which reads like a verb
and is wider than the other badges; WF matches community shorthand and
the two-letter style of CN/CV/TE. Also give the workflow sub-type its
full display name so tooltips show Workflow instead of the raw key.
Add a Workflows page that manages ComfyUI workflow JSON files as a
first-class library alongside loras/checkpoints/embeddings:
- opt-in via the new 'Workflow Management' settings toggle (default off)
- dedicated workflow folder roots with sidecar/preview support, separate
from ComfyUI's own workflow storage; 'Send to ComfyUI' loads a workflow
straight into the graph
- workflow scanner parses the graph document (UI/API format sniffing,
node classes, model/civitai refs) and persists the analysis in the
SQLite model cache
- settings UI: workflow folder paths editor in the Library tab and the
standalone Model Paths page
Fixes from sandbox E2E verification:
- include the workflow scanner in startup initialization so the page no
longer hangs on 'initializing' when enabled without folders
- add the missing workflows entry to the frontend page-state map (page
init crashed and the sidebar fell back to the loras tree without it)
- project workflow analysis fields through the scanner cache entry and
persist them in the model cache (new columns with auto-migration)
- keep symlinked paths intact in the directory picker (abspath instead
of resolve), so stored library paths match what the user sees on disk
Also rebuilds the vue-widgets bundle to sync it with web/comfyui/utils.js.
Follow-up to #1152:
- Accept civitai.green and www-prefixed hosts in isSupportedCivitaiPageHost,
aligning with py/utils/civitai_utils.py and extensionReimportBridge.js, so
download URLs on those hosts resolve too
- Drop the duplicate civitaiModelByVersion entry from the LoRA-specific
endpoint map; the shared getApiEndpoints map now provides it for all types
- Report a distinct error when a linked fileId exists in the version but is
not a downloadable model weight file (new linkedFileNotWeightFile locale key)
Counts in the filter panel chips (base models, tags, model types), the All Types
dropdown and the Refresh scope rows were rendered as raw digits, so a library with
10k+ models turned every chip into a five digit pill that dominated the row it sat
in — and the Refresh menu grouped its numbers while the filter panel did not.
One formatter now renders all of them (`formatCount`): exact and locale-grouped
below 10,000, abbreviated to three significant digits above it, and abbreviated
only where the locale's Intl compact form is a real unit and narrower than the
plain number — German keeps 18.475 instead of the rounded 18.500, Russian keeps
18 475 instead of the wider 18,5 тыс.
Their hover text is always present and always exact ({name} — {count} models, and
... recipes on the Recipes page): an affordance that only exists above a threshold
is undiscoverable, and the tooltip is also what reveals a chip label that CSS has
ellipsized.
Chips gain a two-slot layout (the name ellipsizes, the count keeps a tabular
right-aligned slot) so a long tag name can no longer push the count out of the
chip; Refresh rows move from float:right to a flex row, which keeps the counts in
a right-hand column and lets a long root label ellipsize. The All Types group
tooltip moves onto its trigger so it stops leaking onto the menu options.
The two new strings ship as [TODO: Translate] placeholders in the nine locales;
the translation pass is the next commit.
- Global exclusion lists (node classes + title patterns) in settings,
applied at the shared filterRegistryNodes choke point; settings UI
section is hidden in standalone mode
- Per-node "Exclude from Send Targets" context-menu toggle persisted
in the workflow JSON (lm_exclude_send)
- Quick-exclude button on node selector items writes the comfy class
to the exclusion list
- Precedence: per-node exclusion > explicit marker > global lists
- interceptModeChange now supports multiple listeners; registry hooks
mode changes so mute/bypass refreshes candidates immediately
Help tooltips were CSS pseudo-elements on the info icon, so they were
descendants of the scrollable settings pane (.settings-content uses
overflow-y: auto) and got clipped at its top edge. The Active Library
deprecation notice made this obvious: the text grew to ~10 lines while
only ~5 fit above the icon, so the tooltip started mid-sentence at
"page. Deprecated:".
Render the tooltip once into <body> with position: fixed instead, so no
overflow ancestor can clip it, and position it from JS: above the trigger
when it fits, below otherwise, clamped to the viewport. It follows the
trigger on scroll/resize and hides on mouseout, Escape or touch. The
delegated listeners also cover help icons rendered later by the settings
manager, and the box may now use 320px instead of 220px, which keeps long
help texts to fewer lines.
- Return the ResourceNotFoundError detail from get_civitai_versions and
get_model_versions_status instead of a bare "Model not found", and let
the download modal display the server-provided message
- Anchor extract_civitai_image_id / extractCivitaiImageId to
^/images/<id> so API paths and partial numeric IDs are rejected, same
as the model-page parsing fix in #1140
Refreshing had no way to say "scan only this drive": a user with three external
drives had to spin all of them up for every refresh, and switching a drive off
made the next refresh treat its whole library as deleted (rows pruned from the
memory cache and the SQLite cache, preview_url stripped on the next scroll).
Backend (py/services/model_scanner.py, py/config.py):
* ReconcileScope(roots, folder) + _reconcile_cache(scope=...): files inside the
scope reconcile normally, everything outside is neither re-read nor removed.
The folder half is plumbing for the sidebar entry in the next change.
* Path-level pruning guard: cached entries under a path this walk could not
read are kept and reported instead of removed. Sources: a configured root that
is not reachable (drive switched off while LM runs), a directory os.walk
failed to enter (permissions / I/O error / Windows junction to an offline
drive), and a known first-level symlink whose target is gone
(Config.iter_path_mappings()).
* The recorded folder list is unioned instead of replaced whenever the scan did
not verify every root, so a scoped scan cannot empty the sidebar.
* _reconcile_cache returns a summary (added / removed / repaired /
scanned_roots / skipped_roots / unavailable_paths / kept_unreachable),
exposed as ModelScanner.last_reconcile_summary, returned by
BaseModelService.scan_models() and broadcast in the completed WS payload.
* _root_display_labels(): set-aware labels ("G: loras", "usb/loras") grown
leftwards with real parent segments until unique, shared by the walk-progress
line and the roots API.
* GET /scan accepts repeated `roots` (400 for unknown roots, 400 combined with
full_rebuild=true); GET /roots gains root_details (label / reachable / cached
count) while `roots` stays a plain path list for existing callers.
* serve_preview: a 404 no longer clears the cached preview_url when the file's
own directory is unreachable - browsing the grid with a drive off used to
strip preview references from the persistent cache.
Frontend:
* Refresh ▾ gains a "Scan one folder" section listing the page's roots with
their cached counts; offline roots stay clickable and explain themselves; rows
are wired by delegation (new static/js/components/controls/ScanScopeMenu.js).
* A scoped scan reports "Scanned <root>: N new, M removed"; a scan that kept
entries reports "<N> models kept: <paths> not reachable".
* registerAPI() now injects the two cross-page passthroughs (fetchModelRoots and
an argument-forwarding refreshModels) so a page facade cannot drop them: the
first version rendered an empty menu and would have run a full refresh.
* createToastElement whitelists toast types, so a wrong `type` argument degrades
to the info style instead of rendering an unstyled box.
Verified in a sandbox instance with three roots: a scoped scan walks only the
requested root (progress roots=0/1, 240 files); a full refresh with one root
offline reports kept_unreachable=60 and leaves all 420 models cached; /roots
reports the offline root with its cached count. 3699 passed, 7 skipped;
frontend 1488 passed (148 files); vue widgets 96 passed.
A regular Refresh walked every configured root sequentially, so a full
25 TB drive delayed the roots behind it, and the dialog sat on "Checking
for changes..." at 0 % for the whole walk with no way to tell it was
working. On a cold external drive the walk itself dominates the cost, so
the fix is to overlap the drives and to show what the walk is doing.
Backend (py/services/model_scanner.py):
* Extract the per-root walk into the synchronous _walk_root_for_reconcile()
worker and merge its results on the event loop afterwards, in configured
root order: which business path wins a file reachable through several
roots must not depend on the order the workers happened to finish in.
* Group roots by device (_root_device_key: drive letter on Windows, st_dev
on POSIX) and run one worker per device. Roots sharing a device stay
sequential, so directory claims and the overlap dedup (#871, #1041) keep
their configured-order semantics; different devices run in parallel.
* Track walk progress per root (_ReconcileWalkTracker), weighted by each
root's cached entry count because the real file count is only known once
the walk ends. The bar splits into walk (0-50 %) and new-file (50-99 %)
phases so it never jumps backwards, and the walk broadcasts files seen,
active roots and the ETA counters.
* Replace the Windows case-insensitive fallback -- a scan of every cached
path per miss, i.e. O(files x cached) -- with a lazily built lower-cased
index (_CachedPathLookups, also now guarding the realpath alias index for
worker threads), and lift the os.name == "nt" gate into the module-level
_CASE_INSENSITIVE_PATHS so the branch is testable off Windows.
* Excluded-model membership is a set lookup instead of a list scan.
Frontend:
* render the walk phase as "Checking for changes... <roots> (N files)" with
the ETA, and reset the ETA tracker when the stage changes: the per-file
rate of counting files says nothing about processing them.
* add common.scanProgress.walkFiles; the other locales keep the sanctioned
[TODO: Translate] placeholder until the translation pass.
Verified: no-change reconcile over 10k files/500 dirs 101 ms and 10k/5000
dirs 230 ms (was 93/229 ms, within noise); a two-device sandbox walk runs
both roots concurrently, names them in the progress messages and finishes
with added=15, removed=0; 3665 passed, 7 skipped; frontend 1456 passed,
vue widgets 96 passed.
The owner could not tell from the UI what "Buzz Price Tracking" enabled, what the
"Price alert threshold" number meant, or what "Price Alerts" was alerting about.
That was not a copy problem: the implementation exposed our mechanism (a page
scrape) and our SQL predicates as the user's concepts. Two concrete defects came
from the same root:
- the alert population included versions the user already owns (neither the event
generator nor the query filtered on is_in_library; in the owner's library 28 of
52 gated versions were already downloaded, so most "alerts" were about files
already on disk, which cannot become cheaper *for them*);
- a threshold-filtered state list lived in a notification surface, so an empty
panel had three indistinguishable causes and read as a broken feature.
The information model is now the version plus ownership: cost is shown only where
a decision exists. Owned -> nothing. Not owned and free -> nothing. Not owned and
gated -> the price when it is known, `Paid` without a number when it is not, and
early access keeps its countdown because "free on <date>" decides between waiting
and paying. The numeric threshold has no place in that model: every decision is
categorical (wait / pay / skip), so the setting, the comparison and the whole
alert-state machine are gone.
- both alert-state columns are removed from the schema rather than left dead; a
database created by an unreleased build has them dropped on open (native
ALTER TABLE ... DROP COLUMN, guarded), which is a no-op for everyone else
- gate events are emitted only for versions the user does not have, and the
price-drop event goes with the threshold it belonged to
- both alert endpoints, PriceAlertsHandler and the service-registry adapter field
it needed are removed: events already reach the UI through the refresh response
- the bell tab, panel, CSS, both entry points, the unread watermark and their
locales are removed; the setting keeps only the enable flag and the refresh
interval and is framed as plumbing
- "Price unavailable" is replaced by `Paid`: the gate is certain from the public
API, only the number is best-effort, and that is our plumbing, not the user's
problem
Verified against a copy of the owner's real database: 52 gated versions ->
28 owned (now silent) + 24 the feature is actually about; the drop migration ran
and both removed endpoints 404.
CivitAI's public API deliberately omits prices — paidAccess is trimmed to
{permanent, endsAt} because "pricing belongs to the purchase flow" — but the
public model page embeds the site's own model.getById result, including
paidAccess.terms, in its server-rendered payload. That is read anonymously
(no API key, no internal endpoint, no forged Origin), one request per gated
model, so only the ~2% of models that actually carry a gate pay for it.
- optional capture, off by default: price_tracking_enabled,
price_alert_threshold_buzz (0 = alert on "became free" only) and
price_check_ttl_hours; prices refresh on their own TTL and immediately when a
gate changes, and a failed fetch keeps the stored price instead of blanking it
- versions that stop carrying a gate are marked free (persisted gate_lapsed_at)
and gate transitions are reported as events on the refresh response, so a
version already in the library can announce that it became free
- price_alert_state plus a price_drop edge event; new
GET /api/lm/{type}/updates/price-alerts lists what is under the threshold
- versions tab shows the price (effective, with the list price struck through
and a Blue Buzz note) and a Free Now badge; an update check toasts the
transitions in one message
- the parser and the alerts query are unit-tested against a trimmed page
fixture, and every route definition is now asserted to resolve to a handler
Plan, verification notes and the deviations from it are in
docs/plans/paid-model-price-tracking.md.
Add OpenModelDB (openmodeldb.info) as a metadata and download source for
the existing upscaler model type.
Metadata:
- New OpenModelDBClient: fetches the site's bulk JSON dumps, caches them
on disk (24h TTL + ETag revalidation), and builds a local sha256 index
- New OpenModelDBModelMetadataProvider adapts catalogue entries to the
CivitAI-shaped version dict contract; registered in the fallback chain
behind the enable_openmodeldb_api setting (default on), gated to the
upscaler sub-type so other model types never trigger the dump download
- Persisted provenance uses metadata_source "openmodeldb" plus a nested
openmodeldb block (page URL, architecture, scale, license)
Images: paired-image LR/SR URLs are ephemeral imgdiff.net sessions, so
displayable images come from the site-hosted auto-generated thumbnails
(model-level cover leads images[], per-image thumbs for the rest); the
original comparison URL is kept in meta.comparisonUrl.
Downloads:
- New OpenModelDBSource (flat model ids, omdb: group prefix) with
resource filename derivation that recovers names hidden mid-path
(mediafire) or synthesizes {id}.{type} for folder links
- HTML-gateway mirrors (mediafire/mega/drive) are rejected with a clear
manual-download hint instead of silently saving an HTML page as .pth
- ModelSource base gains is_valid_source_id / default_subdir_parts /
resolve_download_url hooks so flat-id sources need no platform branches
UI: "View on OpenModelDB" link in the model modal (downloaded and
hash-enriched models), settings toggle next to the CivArchive one.
A repository is not a model identity: collection repos on Hugging Face
and ModelScope host many unrelated models, which were wrongly shown as
versions of each other.
- Hugging Face models no longer auto-group (the Hub exposes no
site-native model id)
- ModelScope models group by the site's native published-model id
(MuseInfo modelVersion.modelId), extracted during enrichment and
persisted on the sidecar as source_model_id/source_version_id;
unenriched models stay standalone instead of collapsing a whole repo
into one group
- TensorArt grouping unchanged (its URL id is already model-level)
- Frontend group-key derivation mirrors the new backend semantics
Add per-model-type filename templates ({model_name}, {version_name},
{base_model}, {author}, {first_tag}, {hash_short}, {original_name}) so
downloaded files get informative names instead of e.g. V1.safetensors.
Empty template keeps the current filename (opt-in, off by default).
- apply template automatically after downloads; rename conflicts keep
the original name and never fail the download
- record original_file_name in metadata on rename for traceability
- bulk apply via GET|POST /api/lm/{prefix}/apply-filename-template with
WebSocket progress, sharing the auto-organize lock
- settings UI lives in the new Organization tab with validation, live
preview, and per-type 'apply to library' actions
Standalone users previously had to hand-edit settings.json to configure
primary folder_paths. Add a standalone-only Model Paths section to the
settings modal:
- Backend exposes standalone_mode, folder_paths (with template placeholder
values filtered out) and a data-driven folder_path_schema derived from
OTHER_MODEL_FOLDER_SUBTYPES via GET /api/lm/settings
- The new section renders multi-path editors per model type from the
schema, with inline enable_other_models / sub-type controls so other
model types are configured without leaving the tab
- Persistent restart-required cues after a save: nav dot, inline notice
and a global banner (unique id per change so dismissals don't mute
future reminders)
- The missing-model-paths startup banner and the Other Models no-paths
empty state now deep-link into the new section instead of pointing at
settings.json
A ModelScope or Hugging Face download landed as a bare filename, hash and
source link; the model card stayed empty until the user ran "Enrich
Metadata with AI" by hand. But everything that makes a CivitAI download
useful — the display name, the description, the tags, the trigger words,
the example images, the preview — is already published by those sites'
public APIs, so asking for it at download time is deterministic work, not
model work.
Add `py/services/model_sources/hydration.py`, called by
`_save_source_metadata()` once the sidecar exists and the file is in the
scanner cache. It fetches the model card plus the site's card extras and
hands them to the same `PostProcessor` the AI skill uses, with an empty
`llm_output`, so the two paths cannot drift apart. What lands:
* `model_name` from the site's own display name (ModelScope's `Name`), so
the card stops showing the local filename — written only while the value
still equals the file stem, since once a user renames a model that
choice is theirs to keep
* `civitai.name` from the matched version's label (`showName`), which the
card renders as the version chip
* `civitai.description` / `modelDescription` from the author summary plus
the README as HTML
* `civitai.images` / `preview_url` from the per-file example images
* `civitai.trainedWords` from the per-file trigger words
* `base_model`, `tags` and `usage_tips` as before
Provenance stays honest: the pass records
`metadata_source = "source:<platform>"` rather than the skill's
`agent:enrich_hf_metadata`, and — because no provider ran — it no longer
stamps `llm_enriched_at`; that stamp is now conditional on the LLM
actually answering, which is what the field means. The five hand-rolled
`civitai` dict merges in the post-processor collapse into one
`_merge_civitai()` helper.
Two guards keep it safe. Only a model whose stored
`source_platform`/`source_url` match the repository being downloaded is
updated, so a local file that merely shares a name never receives another
model's card; and a file already on disk is topped up too, which
back-fills models downloaded before this existed. READMEs and detail
payloads describe the repository rather than the file, so a short-lived
process-wide `ModelSourceCache` (300 s, 32 entries) keeps a batch over one
repository to two HTTP requests. Every failure is logged and swallowed:
hydration can never fail a download.
Fix the hash policy while here. `_save_source_metadata()` went straight to
`MetadataManager.create_default_metadata()`, bypassing the per-type
factory on the owning scanner, so a checkpoint paid a full SHA256 inside
the download request — `CheckpointScanner`/`OtherScanner` deliberately
record `hash_status="pending"` with an empty `sha256` for their multi-GB
files. Metadata is now created through `scanner._create_default_metadata()`.
Hydration copes with the empty hash: `_matching_versions()` falls back to
the repository basename, which is exactly what the download just wrote.
Report both post-transfer stages, which advance no byte counter and so
read as a stall: the bar sat at 100% showing `0 B/s` for the seconds spent
hashing and fetching. `_report_phase()` broadcasts
`{"status": "metadata", "stage": "indexing" | "source", "platform": ...}`,
and `LoadingManager` names the stage in the status line (keeping the batch
position), retitles the item line, replaces the dead speed figure and runs
a sheen over the bar. `stage`/`platform` are machine-readable; the wording
is localised in the frontend.
Finally, `modelscope.ai` is its own catalogue rather than an alias of
`modelscope.cn` — `referall13/EM1` exists only on `.ai` and
`jj3550945163/Krea-2-LORA` only on `.cn` — so its URLs were rejected with
"Invalid model URL format". Register it as `ModelScopeIntlSource`
(`platform="modelscope-ai"`, `msai:` group prefix, its own default
download directory) and derive every URL either deployment builds from a
per-class `base_url`. `modelscope.com` stays an alias of `.cn`, which is
what it redirects to. The frontend source table, the link dialog hints and
the docs mirror the split.
Verified against the live APIs: both reported `.ai` repositories list
their files, read their READMEs and yield name / version / base model /
trigger words / example images. Backend 3092 passed; frontend 1259 JS +
91 Vue passed. The nine locales carry the new progress copy in the next
commit.
ModelScope became a linkable source, but downloading from it was impossible:
the URL picker only recognised huggingface.co, the file listing hit a
huggingface-only endpoint, the resolve URL was hardcoded, and the default
path template always wrote into a `huggingface/` directory.
Move the download knowledge into the providers so the handlers stay generic:
- `ModelSource` gains `list_files()`, `file_download_url()`,
`default_revision` and `default_subdir`. `HuggingFaceSource` keeps the Hub
tree API (`/api/models/{id}/tree/{rev}`, LFS-aware sizes, `main`).
`ModelScopeSource` uses `/api/v1/models/{id}/repo/files?Revision=master`
— which reports real byte sizes for LFS files, so no HEAD probe is needed,
and which only accepts `master` (an HF-imported repo still 404s on `main`)
— and downloads through `/models/{id}/resolve/{rev}/{path}`. That URL
redirects to a CDN target carrying a time-limited `auth_key`, so it is
rebuilt on every request and never cached, which is also what keeps
resumable Range requests working.
- `hf_handlers.py`/`HfHandler` become `model_source_handlers.py`/
`ModelSourceHandler` with `list_model_source_files` and
`download_model_source`. New routes `/api/lm/model-source-files` and
`/api/lm/download-model-source`; the old `/api/lm/hf-repo-files` and
`/api/lm/download-hf-model` paths stay as aliases, and a payload without
`platform` still means Hugging Face, so existing callers are unaffected.
- A downloaded sidecar now records `source_platform` + `source_url` (with the
`hf_url` alias only for Hugging Face) instead of always writing `hf_url`,
and `use_default_paths` files ModelScope downloads under
`modelscope/<owner>/<repo>`. The now-unused shared HF aiohttp session and
its shutdown hook are gone; providers open short-lived sessions.
- Frontend: `detectUrlType` returns the platform-neutral
`model-source-repo` / `model-source-file` plus an explicit `platform`, the
DownloadManager's `hf*` state and methods are renamed to `source*`, every
`source === 'huggingface'` check becomes `isExternalModelSource()`, and
batch groups are keyed by `platform:repo` so the same `owner/name` on two
sites renders as two groups. A bare `owner/name` still means Hugging Face.
- `is_valid_source_id()` centralises repo-id validation (exactly
`owner/name`, no traversal, no leading dot). This also fixes the old HF
download check that rejected any dot in the name, i.e. legitimate repos
such as `black-forest-labs/FLUX.1-dev`.
Verified against the live APIs: the example repo lists 8 weight files with
correct sizes, and a ranged GET of the built resolve URL returns 206 after
following the redirect to the CDN. Backend 2853 passed; frontend 1143 JS +
91 Vue passed. The nine locales carry the refreshed download copy in the
next commit.
A model file could only ever be linked to huggingface.co: `set_hf_url`
validated the URL with a huggingface-only regex, the agent fetched the card
from a hardcoded HF URL, and the readme processor built every relative image
path off `https://huggingface.co/{repo}/resolve/main`. ModelScope publishes the
same model-card convention (README.md + YAML frontmatter, often carrying
`base_model:` and `trigger_words:`) behind a public, key-less API, so the
enrichment pipeline could already serve it - it was the plumbing that was
HF-shaped, not the idea.
Make the external source a first-class, provider-driven concept:
- New `py/services/model_sources/` registry. A `ModelSource` owns URL
recognition (lenient for stored values, strict for user input), the
canonical page URL, model-card fetching, the asset base URL and the
capability flags. `HuggingFaceSource` is the previous logic relocated;
`ModelScopeSource` reads `/models/{o}/{n}/resolve/{master|main}/README.md`
and falls back to `/api/v1/models/{o}/{n}/repo`. `TensorArtSource` is
link-only on purpose: tensor.art answers plain HTTP clients with a
Cloudflare challenge and its internal API (ap-east-1.tensorart.cloud /
cn.tensorart.net) rejects every /v1/model/* route with "invalid
authorization header", so it declares supports_enrichment=False rather than
failing silently later.
- Metadata gains `source_platform` + `source_url`; `hf_url` stays as a
read/write alias, written only for Hugging Face, so existing sidecars,
cached rows and third-party consumers keep working. Normalisation runs at
the scanner, the persistent cache (both directions, plus two new columns
behind an ALTER migration) and the linking handler - which is what stops a
user who switches sources from leaving a stale `hf_url` on a ModelScope
model.
- The agent pipeline keys off the provider instead of `hf_url`: the fast-fail
gate now explains *why* a model is skipped (no source / unknown source /
source without a reachable card), the prompt context exposes
source_url/source_id/source_label/asset_base_url while still filling the
legacy hf_url/repo aliases, and the four README image extractors take a
base_url (defaulting to HF) so relative paths resolve against the right
site. Version grouping generalises to hf: / ms: / ta: keys.
- `POST /api/lm/set-hf-url` keeps its path and its legacy payload keys but
accepts `source_url`, validates against every provider and returns the
platform. `GET /api/lm/model-sources` lets the UI render the supported-site
list from the server.
- Frontend: a `modelSourceHelpers` mirror of the registry drives the link
dialog, the card/modal globe (branded "View on ModelScope/TensorArt"), the
version-group key and the enrichment gate; the versions tab no longer sends
ms:/ta: keys to the CivitAI API.
TensorArt stays in the list because provenance is worth keeping even when the
card is unreadable - the dialog says so plainly ("Sites that don't expose one
(currently TensorArt) can only be linked") and the context menu disables
enrichment with a matching tooltip, instead of the user getting
"Unsupported URL".
Verified against the real ModelScope API: jj3550945163/Krea-2-LORA returns a
1882-byte card whose frontmatter carries base_model/tags/trigger_words, and
relative images resolve to .../resolve/master/....
Tests: backend 2815 passed; frontend 1130 JS + 91 Vue passed; pytest
tests/i18n and a Jinja compile pass over templates/. The nine locales carry
[TODO: Translate] for the new strings, completed in the next commit.
get_download_path_template() fell back to "{base_model}/{first_tag}" for any
unconfigured model type, so other-model downloads were silently nested under an
arbitrary CivitAI tag even though the settings UI exposes no template row for
"other" and priority_tags has no "other" entry (making {first_tag} resolve to
tags[0]).
Add DEFAULT_DOWNLOAD_PATH_TEMPLATES with other -> "" so unconfigured and
unknown types resolve to a flat layout under the already sub_type-scoped
default_other_roots; explicit settings.json values still win. Mirror the flat
default in the frontend DEFAULT_PATH_TEMPLATES and stop the download/move
default-path previews from rendering "/undefined" or a dangling slash.
- The Other nav entry is hidden while the feature is off
(nav-item--hidden, toggled client-side after enabling) and now uses the
fa-shapes icon.
- Shared utils/otherModels.js helpers (enable through the settings API,
open the settings Library section) are reused by the disabled page, the
announcement banner and the download modal.
- BannerService registers a one-time dismissible "other-models-announcement"
banner while the feature is off; SettingsManager drops the banner and
updates the nav when the master switch flips.
- A disabled download routing answer now surfaces a showActionToast with an
"Enable Other Models" action.
- Settings UI: master toggle + five sub_type checkboxes whose default-root
selects are disabled when unchecked; i18n keys added to en.json and synced
(other locales keep TODO placeholders).
- Snapshot pre-rematch entry state (reconnectSnapshot) so rematched
entries can be undone via the existing restore flow
- Bulk missing-LoRA downloads mark unresolvable failures hash-invalid,
flipping those entries from download to reconnect candidacy
- Recipe modal always offers a reconnect action next to download for
missing LoRA entries
- Rematch runs collect an opt-in relaxed-matching choice (also reconnect
missing models by file name) via a pre-run options dialog on the
global, bulk and single-recipe entries
- L4 (filename-level) matches are listed in a results dialog with
per-entry undo
Recipes imported from CivitAI image URLs can contain 0 LoRAs: the backend
only sees the REST image API + EXIF, while the complete generation data
lives in the image page's internal trpc payload (see
docs/recipe-civitai-image-no-metadata.md). When the companion
lm-civitai-extension is installed with a valid license, re-import (single
and bulk) of CivitAI-image-sourced recipes is now delegated to the
extension via DOM CustomEvents; the extension scrapes the image page with
the user's session and calls back into the reimport endpoint with the
full metadata payload. Without the extension (or with an invalid license)
the native path runs unchanged.
- POST /api/lm/recipe/{id}/reimport accepts optional payload params
(image_url/name/resources/gen_params/base_model/tags); the payload path
reuses the import-remote engine with reimport semantics (user-edit
carryover, delete-after-save), and malformed/failed payloads fall back
to the legacy URL import. Response gains loras_count.
- The endpoint also accepts GET: the extension is GET-only by convention
(documented in AGENTS.md).
- New static/js/utils/extensionReimportBridge.js (probeExtension /
delegateReimport / getCivitaiImageInfo) wired into RecipeContextMenu
and BulkManager with silent native fallback.
- i18n: toast.recipes.reimportingViaExtension added and translated in
all 9 locales.
- Bind R=refresh, F=fetch metadata, D=download in PageControls via
eventManager (plain letters only, skipped while typing or when a
modal is open); triggers reuse the buttons' existing click handlers
- Show key-hint chips on the refresh/fetch/download/bulk toolbar
buttons; convert the bulk chip to a semantic <kbd>
- Redesign shortcut hints as a neutral theme-adaptive keycap:
--shortcut-* variables in base.css now derive from --text-muted
with a bottom-edge shadow, shared by the toolbar chips, the header
search cue, the help-modal cheat sheet, and onboarding key hints
- Add shared isTypingContext() helper to uiHelpers
- Add an Actions group (R/F/D) to the Shortcuts cheat-sheet tab
Verified with vitest (926 passing, incl. 6 new shortcut cases) and a
sandboxed E2E run in real Chrome (light/dark rendering, hover state,
'?' opening the Shortcuts tab, clean console)
Broadcast typed scan_progress messages over /ws/fetch-progress from the
manual refresh/rebuild paths of ModelScanner and RecipeScanner, and
render percent, processed/total, current file name and an EMA-smoothed
ETA in the loading overlay. Hardcoded refresh strings move to i18n
(common.scanProgress); WS connection failure falls back to the previous
static loading behavior.
The LoRA Manager page kept its active filters in localStorage, which the
ComfyUI-side autocomplete read directly. When the two run in different
browsers, origins, or the ComfyUI Desktop Electron shell, localStorage is
not shared and the active-filters search silently did nothing.
The manager page now mirrors its filter state to a server-side in-memory
store (PUT /api/lm/{prefix}/active-filters), pushed on every change via a
storage-listener hook and once on page load. The autocomplete widget sends
only use_active_filters=true, and the relative-paths endpoint injects the
stored filters into the search, with explicit query params taking
precedence.
With hardware acceleration disabled, Chrome rasterizes in software and a
full-viewport backdrop-filter forces a per-frame CPU blur over everything
behind the modal, freezing the whole browser.
Detect software rendering via the unmasked WebGL renderer string at app
startup and drop the backdrop blur in that case. Also route the download
modal's sticky toolbar through the shared --modal-backdrop-blur variable
instead of a hardcoded blur(8px).
- New OptimizationMode.DISPLAY (width=2400 for images, full quality for
videos) and getDisplayUrl(); the in-modal main viewer renders at most
~1200 CSS px wide, so full-size originals wasted 50-70% bandwidth
- Main viewer and adjacent prefetch use display URLs; the full-size
media viewer keeps using getShowcaseUrl for original quality
- Warm the HTTP cache for examples adjacent to the active one after
expand and on every navigation, so prev/next feels instant (images
only, deduped, low fetch priority)
- Add GALLERY_THUMBNAIL optimization mode (width=160) for the 72px
gallery strip instead of reusing the 450px card thumbnails
- Hint priorities: fetchpriority=high on the main media, low on
strip thumbnails
- Extract a shared BaseModelPicker (search, keyboard navigation,
filename-based suggestions, dynamic API models such as MiniMax H3
under 'Other (API)') used by both the single-model metadata modal
and the bulk base model modal
- Rework the bulk base model modal into a dedicated inline-list
layout: fixed modal size, sticky-free footer with app-standard
modal-actions/primary-btn/cancel-btn buttons, and an inline option
list that scrolls itself instead of an overlay dropdown covering
the footer
- Selecting an option in change mode now filters the list to the
selection instead of resetting and scroll-jumping to it
- Restore opaque sticky section headers in the bulk modal so scrolled
items no longer bleed through
Tag move-to-folder drags with a custom dataTransfer MIME type so card
preview-drop handlers skip them entirely (no highlight, no upload), and
mark the preview image non-draggable so the browser no longer synthesizes
a File payload when a drag starts on the image. Fixes card-on-card drops
and click-jitter self-drops replacing the preview with itself.
Track recipe modal opens in a separate stats file (never touching recipe
JSON/EXIF), expose a fire-and-forget POST endpoint, and add an 'opened'
sort that hides never-opened recipes as a true recently-opened view.
Includes i18n for all locales and backend/frontend tests.
Downloads failed with "No suitable file found in metadata" for models whose
only file uses newer CivitAI file types (e.g. 'Enhancement LoRA' for
Anima/AIR image-editing LoRAs) because the primary-file allowlist only
covered legacy types.
- unify the weights-type allowlist as MODEL_WEIGHT_FILE_TYPES
(py/utils/constants.py) and apply it across download, recipe and
metadata-refresh lookups
- mirror CivitAI's getPrimaryFile() semantics: prefer weights-type primary,
fall back to weights files, then trust CivitAI's primary flag (excluding
non-downloadable artifacts like Config/Archive/Workflow)
- mirror the allowlist in the frontend via shared isModelWeightFile() helper
- add regression tests for the Enhancement LoRA primary-file download,
primary-flag fallback and weights-over-non-weights-primary preference
Will Miao