A model on a switched-off drive lost its creator and license icons in the
modal even though "search by creator" found it: the list payload is
trimmed to id/modelId/name/trainedWords, and the only full-payload source
was the .metadata.json next to the model file.
- carry `creator` in the lightweight list payload so the card/modal
fallback keeps who published the model;
- add a cache fallback to GET /api/lm/{type}/metadata: when the sidecar
cannot be read (offline drive, unmounted share, missing/corrupt file),
rebuild the CivitAI payload from the cache entry and decode
`license_flags` back into the permission fields the icons render from.
The response reports `partial` and `offline` so callers can explain the
degradation instead of silently dropping fields;
- skip the stat() for paths outside every live root in alongside storage
so a dead share cannot stall the modal; centralized storage still reads
the real sidecar and serves the full payload while the drive is off;
- merge the fetched payload into the list data instead of replacing it
(a partial response must never drop cached fields), and show an
offline/partial notice plus a description-tab explanation.
A library with 81k models spent hours re-reading sidecars: force bypassed
the verified-folder skip (04b26b2e9), and the first run after upgrading
verified every folder that predated the completion markers.
- Keep the verified-folder skip under force unless the model is on the
failed list, which is exactly the set that can still have retryable
media. When no progress file was loaded, keep the historical full
re-check because the failed list cannot be trusted. Measured on 989
models: 8.0 ms/model -> 0.12 ms/model.
- Backfill completion markers once per examples root for folders that
already contain media (no sidecar reads), so the first run after
upgrading is fast too. Accepted trade-off: folders left partial by a
pre-upgrade interrupted run are trusted rather than audited
retroactively; interruptions after the upgrade are still repaired and a
per-model force repairs a folder. The showcase already falls back to
the remote image at runtime.
- Pass the cache entry a caller already holds to get_updated_model,
removing an O(N) scan per model (O(N^2) over a library-wide run).
- Throttle progress-file writes to one per 10 s at a ten-model boundary
and write atomically, instead of rewriting a ~6 MB file thousands of
times.
- Record the CDN-cache confound and the new measurements in the
throughput plan.
The global context menu's single "Download example images" entry always
posted force=true, i.e. the re-process-all path that ignores the previous
run's state and re-checks every model. The per-model and bulk menus
already expose both choices, so the global entry was the odd one out.
Turn it into a submenu with "Download Missing" (force=false, keeps the
progress-aware skip) and "Re-process All" (force=true), both still
library-wide. The labels reuse the per-model submenu keys, which are
already translated in every locale, so no new i18n keys or placeholders
are introduced.
- reject metadata writes when the backing model file is missing
- use the REST result when the auto-organize progress socket fails
- add regression coverage and a progress fallback notification
Refs #1126
The bootstrap's absolutize branch tested `url.indexOf('http') === 0`, so a
ws:// URL matched neither branch: it does not start with '/' (the relative
branch) and it does not start with 'http' (the absolute branch). It fell
through and was returned untouched, which meant the WebSocket shim below it
was bypassed for every real WebSocket.
baseModelApi.js builds exactly that shape:
new WebSocket(`${proto}${window.location.host}${WS_ENDPOINTS.fetchProgress}`)
so under a reverse-proxy prefix the socket dialled the host root instead of
the prefixed path. Behind SwarmUI (/ComfyBackendDirect/), the root is answered
by the front-end router:
ws://host:7801/ws/fetch-progress -> 302
ws://host:7801/ComfyBackendDirect/ws/fetch-progress -> 101
A 302 fails the upgrade immediately, so showWithProgress() rejected and
auto-organize reported an error after the move had already succeeded.
Accept ws:/wss: in the absolutize branch and compare hosts rather than
origins -- a ws: URL's origin serialises as ws://host, never http://host, so
the origin comparison would have rejected the very case being fixed.
Behaviour change is confined to same-origin ws/wss URLs; external and
http(s) cases are unchanged.
Co-authored-by: Kai <kai@tigersden.net>
The auto-organize exclusions, metadata refresh skip paths and both
send-to-workflow exclusion textareas only committed on Enter, so clicking
away or closing the modal silently discarded the edit while the rest of
the settings modal autosaves.
Commit on blur and keep Enter as an explicit commit; Enter now just blurs
the field so every path shares one commit handler, like the filename
template inputs. Also flush on Escape, because Chrome does not fire blur
when the focused field is hidden by the closing modal. Skip the post-save
value write-back while the field is focused so a quick re-entry is not
overwritten by the in-flight request.
The consolidation test asserted the raw POSIX value, but workflow_path is
normalized with os.path.abspath. On Windows that resolves the root-relative
'/workflows-a' against the current drive, so the Windows job failed with
'D:\workflows-a' != '/workflows-a'. Use the file's existing
_normalized_workflow_path() helper, as the other workflow_path tests do.
- Arm hover cycling only from the bare preview area; hovering the card
header or footer disarms the timer, so action buttons and tooltips
stay undisturbed
- Drop the activation delay from 1.5s to 0.8s now that chrome
interactions are protected structurally
- Revise cycle interval options to Fast (1.5s) / Normal (2s, default) /
Relaxed (3s), dropping the rarely useful 5s/8s choices
- Translate the updated cardPreview strings in all 9 locales
The fallback to config.loras_roots raised NameError because config was
never imported in that function. Use the same local import as the other
methods and add a test for the fallback.
- Shift+click "Send to ComfyUI" sends mode=replace, but mergeLoras keeps
the saved active flag of entries already in the list, so a listed but
disabled LoRA stayed off after the textarea was replaced.
- Call the shared enableListedLoras() after the merge, as the append path
does, so replace turns the target on and keeps its saved strength.
- Cover the path with the real mergeLoras in the send-to-comfy tests.
Based on the replace-mode coverage in willmiao/ComfyUI-Lora-Manager#1096.
Co-authored-by: Alexandre Zia <11000327+alexandrezia@users.noreply.github.com>
- Run a bounded model pipeline over the shared media limiter so model
preparation and failure bookkeeping overlap the previous model's tail.
New example_images_model_concurrency defaults to 2 (1 restores serial),
and metadata refreshes are serialized while media continues.
- Let the AIMD limiter probe one slot at a time above its base after a longer
success streak, up to example_images_download_max_concurrency (default 8).
A 429 still halves the limit below the base before recovery.
- Cache Civitai rendition -> blob redirects in Downloader.download_to_memory
for one hour (2048 entries, invalidated on 404), so retries skip the 301.
- Add scripts/benchmark_example_images.py, a reproducible old-vs-new A/B
harness that selects one cache/sidecar subset and runs serial, new-serial,
pipeline and aimd modes against the same temporary examples path.
- Document the measured results (19 models / 185 images, 3 repeats):
pipeline -23.4% vs new-serial, AIMD -22.2% vs pipeline, no 429s.
- Add an atomic .lm-examples-complete marker; only a folder with media and
the marker is skipped, so interrupted or legacy folders are re-checked by
the per-image existence path. Clear the marker before each pass.
- Treat processed/failed folders without a marker as pending so the
auto-download pre-check repairs them, and clear markers when provider
metadata is refreshed or custom images are imported.
- Bound stale-metadata refreshes at 90s so one hung provider cannot park the
serial run; count metadata-missing models as retryable.
- Record failure reasons in the progress file, expose failure_summary, and
render a no-media/unavailable/retry breakdown in the progress panel with a
warning completion toast.
- Add marker, partial-repair, timeout, failure-summary and UI tests.
`_DownloadProgress.snapshot()` converts the progress sets to lists, so their
order follows the process hash seed. The compact-status test compared them to
a fixed list and failed on CI whenever `{"a", "b"}` iterated as `["b", "a"]`.
Compare the contents instead.
Move the version panel out of Help -> Documentation into a dedicated
Settings -> About section. It shows the LoRA Manager, ComfyUI, ComfyUI
Frontend, Python, OS and browser versions, copies the block as Markdown
for issue reports, and links to the issue tracker, wiki, GitHub and
Discord.
Read the values from a new GET /api/lm/environment endpoint instead of
ComfyUI's /system_stats: Python and OS stay available in standalone mode,
while ComfyUI-only versions are omitted there instead of failing the
whole panel. Model Paths is inserted before About so About remains the
last settings section in standalone mode as well.
aiohttp already percent-decodes request.query values, so calling
urllib.parse.unquote() a second time turned a literal "%20" in a filename
into a space. Previews for files such as Female%20_nude.mp4 therefore
404ed, while renaming the file to use a real space masked the bug.
Use the decoded query value as-is and add regression tests for literal
"%20" and "%2F" filenames. Cached preview URLs need no migration: they
encode the percent sign once (%2520) and now resolve correctly.
deduplicate() step 4 matched retried queue rows only by (model_id,
model_version_id), so a retry for one file was deleted on the next backend
start whenever another file of the same version still had a failed history
row. Compare the file identity as steps 1-3 already do (#1058).
GET /downloads/history/retry now accepts start=true to run the retried
download on the backend. The companion extension omits the flag and keeps
draining the queue itself, so clients without it no longer leave retried rows
queued forever.
- Blend a phase baseline with a 120s throughput window; reset the phase
on pause and skip bursts so a resumed run is not estimated from the
folders it skipped.
- Expose pause-aware elapsed_seconds and add a compact status response
for the 2s progress poll.
- Save .download_progress.json from both download loops; the old save
sat behind an early return on the common image path.
- Treat blank aiohttp errors and raised exceptions as transient, and
bound gallery media at 25s (videos 180s) instead of the shared 300s
socket read.
- Add tracker, manager, progress and timeout tests plus the full-run
measurements.
- Add a shared MediaDownloadLimiter (default 6 concurrent) with a per-run
429 cooldown and AIMD recovery; drop the empirical inter-model delay.
- Request width-aware Civitai renditions (800px images, transcoded 450px
videos) and map Genur signed/direct URLs to c.genur.art _small.webp or
_large.webp.
- Return an ImageDownloadOutcome so offline/connect/timeout/5xx failures
stay transient instead of being written as downloadFailed; track
retryable_models and leave those models pending.
- Force retries flagged Genur images and clears downloadFailed/downloadError
only after a successful write, through the normal metadata save/cache path.
- Force bypasses the populated-folder guard; success is false when flagged
images were skipped, so models are no longer falsely marked recovered.
- Add settings, tests, and a design/measurement note.
Cards with a static image preview now crossfade through the model's
example images when the pointer rests on the card, giving multi-concept
LoRAs a richer at-a-glance signature without giving up the single fixed
preview that anchors visual memory of the library.
- Cycling activates only after a 1.5s dwell so ordinary hover actions
(action icons, tooltips) keep working; the card chrome fades out for
an unobstructed view while cycling and returns on mouse leave
- Example metadata and the first frame start preloading at hover time,
and each shown frame warms the next, so mode entry and interval ticks
never wait on the network
- The cover image (usually the preview itself) opens the rotation as
the anchor's stand-in; cycling starts from the second example and the
cover rejoins when the rotation wraps
- Video previews keep their autoplay-on-hover behavior; video entries
inside example lists are skipped, as are NSFW entries under SFW mode
and blurred cards until unblurred
- Honors prefers-reduced-motion, pauses in hidden tabs, and cleans up
when the virtual scroller recycles a card
- New settings: card_preview_cycle (default on) and
card_preview_cycle_interval (2/3/5/8s, default 3s)
Implements #1182
The "already added" cue was a single check, so a listed-but-disabled
LoRA looked identical to an enabled one, and picking either inserted a
duplicate tag that left the disabled entry off.
Show two states — a green "Added" pill and an amber "Off" pill — and
make the pick act on them: an added LoRA is enabled in place (reusing
the shared add-or-enable helper), an active one is left alone, and the
typed query is removed either way so no duplicate tag or stray search
text is left in the prompt.
Co-authored-by: smhc <6404304+smhc@users.noreply.github.com>
Node rows can show library titles (Settings > LoRA Manager > Node LoRA
Name Display), but the textarea autocomplete only ever searched file
names. In "Model Name" mode a title fragment that does not appear in the
file name therefore returned nothing, even though the LoRA was visible
on the node.
Title matching follows the existing label setting instead of adding a
new toggle: File Name mode (the default) sends the unchanged request and
gets the same results and ordering as before. Only Model Name mode asks
the endpoint for title matches, which are appended after every filename
match so a path search is never reordered or hidden. The dropdown then
shows the title as the primary row label with the file name as secondary
text, matching what the node rows display.
Also cue results that are already on the node. The set is read from the
node's LoRA list (falling back to the textarea tags), normalized like the
backend resolves names, so legacy bare-name entries are recognized too.
Co-authored-by: smhc <6404304+smhc@users.noreply.github.com>
"Send to ComfyUI" appended a tag for a LoRA the node already listed,
which left a disabled entry disabled and made the button look broken.
Route the append path through shared add-or-enable helpers: listed LoRAs
are switched on in place (strengths kept), unlisted ones still append.
Reused from willmiao/ComfyUI-Lora-Manager#1181.
Co-authored-by: smhc <6404304+smhc@users.noreply.github.com>
CivitAI Workflow archives can bundle only a PNG/WebP screenshot whose
metadata carries the graph, with no workflow JSON member. Zip candidate
discovery now falls back to image members (bounded reads, full-size
images first) and parses their metadata in memory, while the archive
keeps its original bytes and SHA256 identity for CivitAI matching.
The LoRAs page search ran difflib.SequenceMatcher over every token of every
cached entry. Matching entries exit at the substring check, so the cost was
paid entirely by the entries that do not match - exactly the state the list is
in while a word is still being typed. At 100k entries one non-matching query
took 1.7-3.2s, and the handler runs that work on the event loop.
Gate the matcher with two exact necessary conditions instead of a heuristic:
2*min(len_a,len_b)/(len_a+len_b) is an upper bound on the ratio, and so is
twice the character-multiset intersection (a matching block can only use
characters present in both strings). Pairs that fail either bound provably
cannot reach the threshold, so every result is unchanged.
A non-matching query now takes 0.28-0.36s at 100k entries (6-9x faster) and
54-72ms at 20k (was 0.32-0.62s). Add a Hypothesis equivalence test against the
previous implementation, a regression test proving obvious misses never reach
the matcher, and a library-scale benchmark.
Fixes#1179
TestRecipeFTSIndexPerformance asserts a 100ms bound on a ~1ms search. On shared CI runners that flakes: the same test measured 0.12s and 0.14s on two consecutive main pushes while the code around it did not change.
Mark the class with the existing performance marker (pytest.ini already runs with -m "not performance"), matching tests/performance/. Run them explicitly with pytest -m performance.
View all local versions answers "every local version of this model", but the
listing request still carried the active folder: versions moved to another
folder silently disappeared, and a group-by-model card's version count (counted
across the library) no longer matched the list it opened.
- suspend the folder filter when the view starts, remember it, and show the root
selection so the sidebar and the grid agree
- restore the folder when the filter is cleared; keep it suspended across a
page reload
- picking a folder while the filter is active now exits it instead of being
silently ignored (the view is library-wide)
- hand a suspended folder back to its own page when another model page clears
the stale filter
- backend contract unchanged: the client simply omits `folder` while the view
is active
Tests cover the query params, folder suspend/restore/foreign handoff and the
sidebar folder handoff.
Workflows are graph documents (.json/.zip/.png/.webp), so quantization has
no meaning for them, yet WorkflowScanner inherited the base scanner's
unconditional quantization pass: every workflow row was resolved to ''
("checked, none"), a meaningless "quantization": null landed in the
sidecar, and the one-time backfill logged "Quantization backfill: updated
10/10 models for workflow".
- Add ModelScanner.supports_quantization (default True); WorkflowScanner
overrides it to False.
- Gate the one-time backfill scheduling and the scan-time resolution on
that flag, and make QuantizationBackfillService a no-op for scanners
that do not support quantization.
- Cover both paths: the service leaves workflow rows in their "not
checked" (NULL) state, and _load_persisted_cache still schedules only
the autov3 pass.
The settings folder picker could only select existing directories, so
creating a model root meant leaving the UI to create the folder first.
- Add POST /api/lm/create-directory, backed by create_directory() in
py/utils/directory_browser.py: it creates one subfolder of the browsed
directory and rejects names that are not a single safe path segment
(separators, dot names, Windows-invalid characters and reserved names,
over-long names) plus duplicates, missing/non-directory parents and
permission errors, each with a stable error_code.
- Add a New Folder button and an inline name row to the picker; the
created folder is entered immediately so it can be selected with one
click. Esc dismisses the row before closing the modal, and the button
is disabled on the Windows drive-list level where there is no
directory to create in.
- Map the server error codes to localized picker messages.
- Cover the endpoint and the picker flow with backend and frontend tests.
Users see a trigger word in someone else's prompt and have no way to tell
whether the LoRA is already in their library, so they fall back to searching
the model sites again.
- Read trigger words from civitai.trainedWords through a small helper that
tolerates the shapes older sidecars contain (None, a bare string, mixed
lists). Provenance does not matter: CivitAI/CivArchive sync, HuggingFace
or ModelScope model cards, the AI enrichment skill and manual edits all
write the same field.
- Add an opt-in trigger_words search option to SearchStrategy, evaluated
last before the always-on hash fallback. Matching is case-insensitive
substring only: fuzzy would run the sequence matcher over every declared
word of every entry and cost seconds on large libraries.
- Expose it as a "Trigger Words" chip on the LoRAs page only, plumb the
option through page state and the list request, and parse the new
search_trigger_words query param.
- Cover the strategy (including AI-enriched HuggingFace entries and
malformed trainedWords), the param parsing, the chip-to-state wiring and
the request query string. The English key is synced with a
[TODO: Translate] placeholder in the other locales.
Expose Auto-Organize on the Workflows page and give workflows their own
download path template, which both downloads and Auto-Organize use.
- Add a `workflow` default ({base_model}) to the backend and frontend path
template constants. Local workflows rarely carry CivitAI tags, so the
default stops at the base model instead of adding a "no tags" segment.
- Enable the bulk Auto-Organize action for workflows; the shared endpoint
and scanner move already handled them, only the UI gate was missing.
- Add the workflow row to Settings > Organization, rendered hidden and
toggled by the existing Workflow management master switch so the config
only appears while the feature is enabled.
- Preview workflow templates with a .json artifact name instead of a
weight-file name.
- Cover the default template, download routing, both auto-organize modes,
sidecar moves, and the settings gate with tests.
A failed CivitAI lookup used to drop the entry that the fetch had just
refreshed: the on-demand hash was already computed and stored, but the
client only applied the response entry on success, so the card kept the
stale empty sha256 until a page reload.
- Return the refreshed entry in the error payload when the lookup fails.
- Apply the entry on both outcomes in refreshSingleModelMetadata; the
error toast is unchanged.
- Cover the failed-response path on the handler and the API client.
Checkpoints and other large models are scanned lazily: the cache entry has
an empty sha256 with hash_status="pending", so the modal's hash footnote
rendered nothing and the hash could not be copied for external lookups.
- Expose hash_status in the model list responses and pass it to the modal
through the card dataset.
- Render the pending / calculating / failed states in the hash footnote
with a manual trigger, then swap in the calculated hash with its copy
button; the underlying card and virtual-scroll entry stay in sync.
- Add POST /api/lm/{prefix}/calculate-hash, which reuses the scanner's
singleflight hash calculation and persists the SQLite cache so the hash
survives a restart without a rescan.
- Send version_name only for titles shared by several files and guard
non-dict civitai values, so the cycler payload stays small.
- Keep cached labels after the availability TTL expires instead of
flashing file names while a refresh is in flight.
- List every covered node in the setting tooltip.
- Cover the payload rules and the stale-cache labels with tests.
A node extractor exception propagated out of record_node_execution and
update_node_execution before _cache_node_metadata ran, so one broken
extractor skipped the node cache for that prompt and surfaced only as the
hook's generic "Error collecting metadata" line, without node context.
Catch extraction failures in the registry, log them with the node id and
class, and still cache whatever metadata was collected. When a failed
extractor wrote nothing at all, keep the previous cache entry instead of
clearing it.
Adds tests for partial extraction, an early failure, and update failures.
Will Miao