- Run a bounded model pipeline over the shared media limiter so model
preparation and failure bookkeeping overlap the previous model's tail.
New example_images_model_concurrency defaults to 2 (1 restores serial),
and metadata refreshes are serialized while media continues.
- Let the AIMD limiter probe one slot at a time above its base after a longer
success streak, up to example_images_download_max_concurrency (default 8).
A 429 still halves the limit below the base before recovery.
- Cache Civitai rendition -> blob redirects in Downloader.download_to_memory
for one hour (2048 entries, invalidated on 404), so retries skip the 301.
- Add scripts/benchmark_example_images.py, a reproducible old-vs-new A/B
harness that selects one cache/sidecar subset and runs serial, new-serial,
pipeline and aimd modes against the same temporary examples path.
- Document the measured results (19 models / 185 images, 3 repeats):
pipeline -23.4% vs new-serial, AIMD -22.2% vs pipeline, no 429s.
- Add an atomic .lm-examples-complete marker; only a folder with media and
the marker is skipped, so interrupted or legacy folders are re-checked by
the per-image existence path. Clear the marker before each pass.
- Treat processed/failed folders without a marker as pending so the
auto-download pre-check repairs them, and clear markers when provider
metadata is refreshed or custom images are imported.
- Bound stale-metadata refreshes at 90s so one hung provider cannot park the
serial run; count metadata-missing models as retryable.
- Record failure reasons in the progress file, expose failure_summary, and
render a no-media/unavailable/retry breakdown in the progress panel with a
warning completion toast.
- Add marker, partial-repair, timeout, failure-summary and UI tests.
- Blend a phase baseline with a 120s throughput window; reset the phase
on pause and skip bursts so a resumed run is not estimated from the
folders it skipped.
- Expose pause-aware elapsed_seconds and add a compact status response
for the 2s progress poll.
- Save .download_progress.json from both download loops; the old save
sat behind an early return on the common image path.
- Treat blank aiohttp errors and raised exceptions as transient, and
bound gallery media at 25s (videos 180s) instead of the shared 300s
socket read.
- Add tracker, manager, progress and timeout tests plus the full-run
measurements.
- Add a shared MediaDownloadLimiter (default 6 concurrent) with a per-run
429 cooldown and AIMD recovery; drop the empirical inter-model delay.
- Request width-aware Civitai renditions (800px images, transcoded 450px
videos) and map Genur signed/direct URLs to c.genur.art _small.webp or
_large.webp.
- Return an ImageDownloadOutcome so offline/connect/timeout/5xx failures
stay transient instead of being written as downloadFailed; track
retryable_models and leave those models pending.
- Force retries flagged Genur images and clears downloadFailed/downloadError
only after a successful write, through the normal metadata save/cache path.
- Force bypasses the populated-folder guard; success is false when flagged
images were skipped, so models are no longer falsely marked recovered.
- Add settings, tests, and a design/measurement note.
CivitAI Workflow archives can bundle only a PNG/WebP screenshot whose
metadata carries the graph, with no workflow JSON member. Zip candidate
discovery now falls back to image members (bounded reads, full-size
images first) and parses their metadata in memory, while the archive
keeps its original bytes and SHA256 identity for CivitAI matching.
The LoRAs page search ran difflib.SequenceMatcher over every token of every
cached entry. Matching entries exit at the substring check, so the cost was
paid entirely by the entries that do not match - exactly the state the list is
in while a word is still being typed. At 100k entries one non-matching query
took 1.7-3.2s, and the handler runs that work on the event loop.
Gate the matcher with two exact necessary conditions instead of a heuristic:
2*min(len_a,len_b)/(len_a+len_b) is an upper bound on the ratio, and so is
twice the character-multiset intersection (a matching block can only use
characters present in both strings). Pairs that fail either bound provably
cannot reach the threshold, so every result is unchanged.
A non-matching query now takes 0.28-0.36s at 100k entries (6-9x faster) and
54-72ms at 20k (was 0.32-0.62s). Add a Hypothesis equivalence test against the
previous implementation, a regression test proving obvious misses never reach
the matcher, and a library-scale benchmark.
Fixes#1179
The settings folder picker could only select existing directories, so
creating a model root meant leaving the UI to create the folder first.
- Add POST /api/lm/create-directory, backed by create_directory() in
py/utils/directory_browser.py: it creates one subfolder of the browsed
directory and rejects names that are not a single safe path segment
(separators, dot names, Windows-invalid characters and reserved names,
over-long names) plus duplicates, missing/non-directory parents and
permission errors, each with a stable error_code.
- Add a New Folder button and an inline name row to the picker; the
created folder is entered immediately so it can be selected with one
click. Esc dismisses the row before closing the modal, and the button
is disabled on the Windows drive-list level where there is no
directory to create in.
- Map the server error codes to localized picker messages.
- Cover the endpoint and the picker flow with backend and frontend tests.
Users see a trigger word in someone else's prompt and have no way to tell
whether the LoRA is already in their library, so they fall back to searching
the model sites again.
- Read trigger words from civitai.trainedWords through a small helper that
tolerates the shapes older sidecars contain (None, a bare string, mixed
lists). Provenance does not matter: CivitAI/CivArchive sync, HuggingFace
or ModelScope model cards, the AI enrichment skill and manual edits all
write the same field.
- Add an opt-in trigger_words search option to SearchStrategy, evaluated
last before the always-on hash fallback. Matching is case-insensitive
substring only: fuzzy would run the sequence matcher over every declared
word of every entry and cost seconds on large libraries.
- Expose it as a "Trigger Words" chip on the LoRAs page only, plumb the
option through page state and the list request, and parse the new
search_trigger_words query param.
- Cover the strategy (including AI-enriched HuggingFace entries and
malformed trainedWords), the param parsing, the chip-to-state wiring and
the request query string. The English key is synced with a
[TODO: Translate] placeholder in the other locales.
Expose Auto-Organize on the Workflows page and give workflows their own
download path template, which both downloads and Auto-Organize use.
- Add a `workflow` default ({base_model}) to the backend and frontend path
template constants. Local workflows rarely carry CivitAI tags, so the
default stops at the base model instead of adding a "no tags" segment.
- Enable the bulk Auto-Organize action for workflows; the shared endpoint
and scanner move already handled them, only the UI gate was missing.
- Add the workflow row to Settings > Organization, rendered hidden and
toggled by the existing Workflow management master switch so the config
only appears while the feature is enabled.
- Preview workflow templates with a .json artifact name instead of a
weight-file name.
- Cover the default template, download routing, both auto-organize modes,
sidecar moves, and the settings gate with tests.
Newly created workflow entries used to be hardcoded to "Unknown". The
base model is now resolved at creation time only (Create LM-managed
workflow and first-time scans); write-back never re-resolves it.
Resolution order:
1. generation-time hint: on create with a prompt context, the primary
checkpoint from the metadata collector (primary-sampler tracing plus
lm_marker_role marks) outranks static analysis when it resolves in
the local checkpoint library
2. static votes: majority base model of the referenced checkpoints in
the local library, falling back to the referenced LoRAs
3. CivitAI URN fallback for one-off user actions (never scans)
Workflow analysis now extracts structured loader model references from
both API-format and UI-format graphs, including loaders whose widget
values are serialized as arrays.
`tensor_to_png_bytes` imported torch unconditionally inside its try block,
so every run without torch (standalone, CI) hit ModuleNotFoundError, logged
it as an image-processing error and returned None. That silently disabled
workflow previews on standalone installs and failed two backend tests:
`test_tensor_to_png_bytes_*` and
`test_resolve_workflow_preview_requires_matching_prompt`.
Detect torch tensors behind an optional import instead; numpy arrays keep
converting as before when torch is absent.
- Replace the multi-root folder_paths.workflows workflow library with a
scalar workflow_path setting (Settings > Library); remove the
extra_folder_paths.workflows path and the standalone Model Paths editor.
- Migrate pre-release settings.json files that still keep
folder_paths.workflows / extra_folder_paths.workflows on startup, so an
upgrade keeps the configured folder instead of silently losing it.
- Add "Create LM-managed workflow" to the ComfyUI Graph menu and the File
menu/command palette: it writes the serialized graph into the configured
folder, stamps extra.lm_source and registers the entry in the scanner
cache without forcing a rescan.
- Attach the decode image of the client's last executed prompt as the
workflow preview. The prompt id is sent with the create request and the
backend only honors it while it is still the registry's current prompt,
so a tab never borrows another tab's output; otherwise the card keeps the
placeholder preview.
- Detect library entries deleted/renamed while a graph still carries the
marker: a link-status endpoint clears the stale extra.lm_source (on graph
load, menu open and before create/update), and the write-back route
returns 404 instead of recreating the document.
- Collapse the workflows.noPaths standalone/ComfyUI split into a single
description/hint; the folder is edited in Settings > Library in both
modes, so the manual settings.json guidance and the standalone
open-settings-folder fallback are gone.
- Add the preview tensor conversion as a shared utility (reused by the
recipe widget save path) and cover create, preview gating, link status,
migration and menu injection with tests.
Anima 2.9B/3.8B insert blocks between the original 28, so old LoRA keys land on the wrong layers and ComfyUI emits no missing-key warning. Add a model-family adapter that rewrites block indices in memory before the generic LoRA loader runs.
- 28->40, 40->52 and composed 28->52 mappings, with anchored kohya/generic key patterns
- Anima detection from image_model/llm_adapter and block count from blocks/num_blocks
- skip text encoder, llm_adapter and semantic connector keys; collision and newer-LoRA errors
- ComfyUI setting (default on) synced to the LM settings API; no base-model-specific node toggle
- docs, unit and loader integration tests
Reuses the manifest data from PR #1104; supersedes its node-level toggle design.
Library entries are the original artifact bytes (SHA256 = CivitAI identity);
container artifacts (.zip, .png, .webp) get their extracted workflow document
materialized as a *.workflow.json sidecar that follows rename/move/delete.
- workflow_extraction: json direct-read, zip candidate pick, png/webp metadata
- scanner whitelist extended; non-workflow files rejected; adjust_metadata
fills analysis fields for downloaded entries; image artifacts self-preview
- download_manager: workflow branch end-to-end, no zip unpacking, file
selection prefers primary then json/zip/image
- lifecycle: delete/rename/pending-delete carry the derived document
- send-to-comfy reads the derived document for container artifacts
- fix: force_refresh with no cache falls back to full initialization
(first scan before deferred background init no longer crashes)
Add a Workflows page that manages ComfyUI workflow JSON files as a
first-class library alongside loras/checkpoints/embeddings:
- opt-in via the new 'Workflow Management' settings toggle (default off)
- dedicated workflow folder roots with sidecar/preview support, separate
from ComfyUI's own workflow storage; 'Send to ComfyUI' loads a workflow
straight into the graph
- workflow scanner parses the graph document (UI/API format sniffing,
node classes, model/civitai refs) and persists the analysis in the
SQLite model cache
- settings UI: workflow folder paths editor in the Library tab and the
standalone Model Paths page
Fixes from sandbox E2E verification:
- include the workflow scanner in startup initialization so the page no
longer hangs on 'initializing' when enabled without folders
- add the missing workflows entry to the frontend page-state map (page
init crashed and the sidebar fell back to the loras tree without it)
- project workflow analysis fields through the scanner cache entry and
persist them in the model cache (new columns with auto-migration)
- keep symlinked paths intact in the directory picker (abspath instead
of resolve), so stored library paths match what the user sees on disk
Also rebuilds the vue-widgets bundle to sync it with web/comfyui/utils.js.
Downloads, imports, reconciles, metadata syncs and bulk updates all add
freshly allocated objects to caches that were frozen at startup, so
ComfyUI's post-prompt gc.collect() grows again within a running session
(issue #1161, follow-up to #1147).
- add_model_to_cache now stores the slim cache representation: license
flags are derived from the full source payload first, then the civitai
payload is slimmed to the scan-path shape. Fresh downloads previously
also ended up with default license flags.
- New schedule_refreeze(): debounced re-freeze with an idle guard that
reschedules while the ComfyUI prompt queue is busy, so the freeze never
lands mid-prompt. Wired into downloads/imports, reconcile, single-model
updates, deferred bulk updates and metadata sync.
- freeze_long_lived_objects() logs collect duration and the tracked-object
count at DEBUG level to tell 'more objects' apart from 'slower objects'.
A file whose directory entry reports zero bytes hashes to the SHA256 of an
empty byte string. That value is shared by every unread or still-being-written
file: the library collapses them into one bogus duplicate group and every
member resolves to the same polluted remote entry (#1162).
- _calculate_sha256_sync returns "" when the read yields no bytes, so no
caller can persist the placeholder as a real hash
- create_default_metadata refuses to create a sidecar for an empty file
- ModelScanner._process_model_file skips zero-byte files before loading or
creating metadata, leaving them unmanaged until they have content
- a file whose metadata cannot be created no longer aborts the scan via assert
Video preview downloads keep streaming to disk, but the saved file now
gets its extension sniffed from the actual bytes and renamed on
mismatch, so a future Civitai transcode format change cannot leave
e.g. WebM bytes in an .mp4 file. (#1155)
Civitai's width=450,optimized=true renditions are re-encoded (typically
WebP) regardless of the extension in the URL path, but the preview
download branches named the saved file after the URL suffix, leaving
WebP bytes in .jpeg files that Windows thumbnail pipelines cannot
decode. Detect the extension from the actual bytes/headers instead,
via a shared detect_media_extension() helper.
Move the Windows-safe chunked FileResponse out of PreviewHandler into a
reusable py/utils/file_responses.py module, with platform_file_response_class()
selecting the transport per platform. Static routes (add_static) can adopt it
later if they also need to bypass native sendfile on Windows.
Add a platform-independent test for ChunkedFileResponse so Linux CI exercises
the chunked transport; the merged Windows-only test never runs there.
CivitAI stores non-ASCII file names percent-encoded; the mini endpoint's
raw fileName (preferred since #1100) returned that encoded string, so it
landed on disk literally as "%E7%8A%AC..." and the path-budget
truncation could cut mid-escape. Decode to valid non-ASCII UTF-8 names
before the on-disk name is fixed; literal % (stored as %25) and invalid
escapes pass through unchanged.
The forward-slash paths fix changed the checkpoint/unet combo lists on
Windows from "dir\model.safetensors" to "dir/model.safetensors", so
workflows saved before it loaded with a value no option matches: the
front-end extension reset the widget to the first model, or ComfyUI
flagged it "value not in list" when the widget was wired.
Two layers:
* random_loader_control.js migrates the widget value to its
separator-normalized form when that form is a known model, before the
reset-to-first fallback and before the wired early return.
* The loader nodes declare VALIDATE_INPUTS for the model name input,
lifting ComfyUI's strict list check for it; the shared helper resolves
separator-normalized names against the cache, so API-submitted or
programmatic values validate too. get_checkpoint_info_absolute now
normalizes backslashes explicitly instead of via os.sep, which was a
no-op on Linux.
A full rebuild or library switch replaces the frozen cache with freshly
allocated, GC-tracked objects, so ComfyUI's post-prompt gc.collect() would
slow down again until the next restart. Schedule a debounced re-freeze
(10 s settle delay) after ModelScanner full rebuilds and after model/recipe
library-change re-initialization, so repeated rebuilds and multi-scanner
library switches freeze once, after follow-up work settles.
The async calculate_sha256 performed its full-file read+hash loop
synchronously, freezing the single aiohttp event loop for the whole
duration of a multi-GB checkpoint hash — every HTTP endpoint, websocket,
and the extension's progress polling hung with no error surfaced.
Move the blocking loop into run_in_executor so hashing occupies a worker
thread instead of the event loop. Fixes all call sites at once (download
finalize, checkpoint lazy hash, scanners, metadata manager).
85f79283 made _calculate_folder compare root identities, but sliced the
relative folder out of the identity itself -- and normalize_root_identity
folds case on Windows (or any drive-letter path), so every move recorded
the folder in lowercase. The sidebar then grew a lowercase twin branch
("Krea 2" vs "krea 2") on each move until the next disk walk healed it.
Slice from normalize_root_spelling instead, a case-preserving counterpart
that folds only separators and redundancies, with a length guard for the
rare Unicode case-fold that changes the string length.
Distinguish same-version quantized models (e.g. GGUF Q4_K_M vs Q8_0) on
cards, which previously showed only the identical CivitAI version name.
Detection is content-based and stored as a three-state sidecar field
(absent = unchecked, "" = checked-unavailable, value = canonical name),
mirroring the autov3 pipeline:
- GGUF: minimal pure-struct header parser reads general.file_type
- safetensors: dominant tensor dtype F8_* -> FP8
- CivitAI files[].metadata.fp == "fp8" fills in as a fallback only
- A one-time backfill service covers existing libraries; each file
costs only a few-hundred-byte header read
The footer renders the badge after the version name (suppressed in
grouped mode), gated by a new show_quantization_on_card setting that
follows the show_version_on_card toggle pattern.
ComfyUI runs a full gc.collect() after every prompt, and its cost scales
with the number of GC-tracked objects. With a large library the hydrated
model/recipe caches account for ~1M long-lived objects, so each post-prompt
collection held the GIL for seconds and froze the whole server.
After all scanner init tasks finish, run gc.collect() + gc.freeze() once to
move the long-lived objects into the permanent generation, which the
collector no longer scans. Matches the workaround the reporter verified
manually.
The per-component caps (150/100/50) do not add up to a guarantee: a deep
model root plus template folders plus a max-length stem still overflows
the Windows MAX_PATH limit, and raw download filenames carried no cap at
all. fit_stem_to_path_budget() now measures the actual destination —
the model path and the sidecar location resolved through sidecar_paths
(the centralized mirror tree is typically deeper) — and trims the stem
to fit, reserving room for companion suffixes (.metadata.json.bak,
previews, .aria2).
Applied at all four name/path decision points: bulk filename-template
rename, post-download template rename, the initial download save path
(file_info rewritten to the on-disk name, matching the #1100 raw-name
precedent), and an auto-organize pre-check that skips with an
actionable message instead of failing opaquely on Windows.
Download Root Rules: route downloads to a non-default root by ordered
base-model/tag rules per model type (loras/checkpoints/unet/embeddings),
with a rules editor in Settings > Organization, a default-root cue, and
the matched rule shown in the download modal. Translated in all locales.
Conflict note: settings_manager.py adopted the shared
utils.root_paths.normalize_root_identity from main (issue #1146).
Four places produced the host's separator (or a mixed path) where the app's
business form is forward-slashed, so on Windows the value could not be compared
with the one the rest of the app stores:
* `ModelScanner._calculate_folder` matched `file_path.startswith(root)` as raw
strings, so every forward-slashed path resolved to the library root and a
moved model was filed under the root instead of its folder. It now compares
root identities and slices the relative part off.
* `normalize_relative_folder` ran `os.path.normpath`, and ntpath folds forward
slashes back to backslashes: every folder parameter (scoped scan, create,
rename, delete) came back host-spelled and mismatched the stored folders.
* `_format_model_name_for_comfyui` returned `os.path.relpath` unchanged, so the
checkpoint/unet nodes and the metadata node emitted `unet\base.safetensors`
on Windows -- a name no ComfyUI widget list matches.
* `resolve_centralized_dir_for_dir` passed a component that spans two
directories ("<library>/<component>") whole to `os.path.join`, leaving a mixed
path ("...\comfyui/loras-1a2b3c4d\sub") that no native path equals.
Also hands the rename re-key callback business paths, like every other absolute
path the move service returns.
`with sqlite3.Connection` ends the transaction but keeps the file handle open,
so every cache class leaked one handle per instance until the garbage collector
happened to run. On Windows that makes the database file undeletable while the
process lives (WinError 32), which broke 30 cache tests in teardown and would
also block moving or clearing a cache at runtime.
All eight `with <connection>` sites now go through `closing_connection()`, and a
regression test asserts the schema init leaves no usable connection behind.
ComfyUI hands its model folders over in the host's own spelling --
`utils/extra_config.py` runs every configured path through `os.path.normpath()`,
which is all backslashes on Windows, and `main.py` registers `<output>/loras`
unconditionally for its save nodes. `Config` kept that spelling verbatim while
`_prepare_*_paths()` normalized the live root lists to forward slashes, and the
scanner compared the two as raw strings. On Windows every configured root
therefore looked like a root that was not in the live list: the Refresh menu
listed each directory twice (the twin showing "0 models"), the completion toast
claimed reachable roots were "currently inaccessible" and kept their whole cache
count as `kept_unreachable`, and the settings sync persisted host-registered
directories that were never created.
The duplication was not only cosmetic. `_UnreachablePaths.add()` normalizes the
prefix it is given, so the configured twin of a live root registered exactly the
live root's prefix as unreadable: in the prune phase every cached-but-missing
entry under it matched and was "kept", so models deleted from disk were never
removed from the cache again.
* `py/utils/root_paths.py` holds the single `normalize_root_identity()`
(separator style, redundant/trailing separators, and case where the filesystem
folds it: whole paths on Windows, drive letters everywhere). `config.py` and
`settings_manager.py` each carried a private copy and `model_scanner.py`
compared raw strings.
* `ModelScanner` compares root identities in `get_configured_model_roots()`,
`describe_model_roots()` (dedupe + `available`), the reconcile's
`configured_only` list and the `ReconcileScope.roots` membership test, so one
directory is one root whatever the host called it. `_normalized_root_prefix()`
is built on the same identity, and an empty prefix no longer matches every
path.
* A configured root that never made it into the live list is reported only while
it still holds cached entries -- in the menu and in `skipped_roots` /
`unavailable_paths` alike. A directory that neither exists nor was ever scanned
has nothing to keep, so `<output>/loras` stops being an unavailable root.
* The startup sync drops saved roots that neither exist nor hold cached entries
(`PersistentModelCache.roots_with_cached_entries()`, one scan per model type,
separator-agnostic and wildcard-escaped). A root named by the library's
`default_*_root` is kept even when empty: dropping it would let the next boot
"repair" the default onto another root. When the cache cannot be consulted,
every root is kept.
* `GET /scan?roots=` validates requested roots by identity too, so the browser
extension or a script passing the spelling settings.json holds selects the root
instead of answering 400 "Unknown model root(s)".
Verified: 8 of the new tests fail against the pre-fix source (checked with the
fix stashed); 3798 passed, 7 skipped, 11 deselected. The probe costs 71ms for a
100k-model library with seven missing roots (357ms when probing root by root).
The example-images download loop marked a model as failed in
.download_progress.json whenever its metadata had no civitai images or
processing raised any exception. Since failed_models is a permanent skip
list for non-force runs (and excluded from the pending pre-check), models
whose metadata simply had not been fetched yet — or that hit a transient
error — were never retried. Users had to delete the progress file to
unstick them, and the blacklist re-grew afterwards.
- Distinguish 'metadata not fetched yet' (no civitai payload) from
confirmed absences (civitai entry present but image-less, or the model
is known to be gone from CivitAI via civitai_deleted/from_civitai).
Only confirmed absences are marked failed; the rest stay pending.
Retrying pending models only re-reads local metadata, so no extra
provider API calls are introduced.
- Transient processing errors no longer add the model to failed_models.
- Add unblock_failed_example_image_models() to remove hashes from the
persisted failed list (library-scoped and legacy locations) and from
the live download manager's in-memory state.
- Hook it into MetadataSyncService.update_model_metadata so a successful
metadata fetch/relink immediately re-enables example image downloads
for that model — no TTL polling needed.
Add a download_root_rules setting: ordered per-type rules
({match_type: base_model, match: [...], root: path}) that replace the
default download root when 'use default paths' is on. The first matching
rule wins; unmatched downloads fall back to default_*_root. Rules are
consulted under the final download type, so a checkpoint routed to the
unet roots uses the unet rule set.
- settings_manager: normalize/get/set with root validation against the
configured roots (folder_paths + extra_folder_paths); invalid rules
are dropped with a warning
- download_manager: apply the resolved root in the use_default_paths
branches for lora/checkpoint(unet)/embedding downloads
- download routing handler: expose resolved_root so the download
confirmation modal preselects the rule-routed root
- settings UI: per-type rule editor in the Organization section with
base-model multi-select panels, root dropdowns, reorder and remove
- Return the ResourceNotFoundError detail from get_civitai_versions and
get_model_versions_status instead of a bare "Model not found", and let
the download modal display the server-provided message
- Anchor extract_civitai_image_id / extractCivitaiImageId to
^/images/<id> so API paths and partial numeric IDs are rejected, same
as the model-page parsing fix in #1140
The default recipes dir (<first lora root>/recipes) is auto-created by the
recipe scanner and recorded by the model-root walks as an empty folder, so
it showed up in the folder sidebar — and since recipe files are not model
weight files, the folder delete endpoint's model_count check let a
right-click delete wipe the whole recipe library via shutil.rmtree.
- scan walks (full scan, reconcile, all-folders backfill) no longer descend
into the effective recipes dir, and get_all_folders() filters it out of
snapshots persisted before the exclusion existed
- delete_folder/rename_folder refuse the recipes dir and any ancestor of
it with code "protected"
- the delete modal renders a protected state (confirm hidden, per-row
checkbox disabled) instead of a deletable 'no models' prediction
- RecipeScanner.recipes_dir now shares the side-effect-free resolver in
py/utils/recipes_paths.py
End-to-end verification against the live site found the price capture broken for
a whole class of users: the civitai page hosts are not interchangeable, and the
user's civitai_host preference was silently fatal. With civitai_host=civitai.red
the update DB held zero prices even with tracking enabled.
- civitai.red refuses non-browser HTTP clients outright (Cloudflare challenge,
403 for any User-Agent, aiohttp and httpx alike), while civitai.com and
civitai.green answer normally for anonymously visible models and 404 for
mature ones. An earlier manual check with curl passed on TLS fingerprint luck,
which is why this was missed.
- get_model_prices now tries the configured host first, then the others, and
takes the first parseable payload. The host that worked is remembered, and a
host that refuses outright is parked for 15 minutes so a library full of
mature models does not pay three requests each; a 404 is model-specific and
does not park the host. Links keep using the configured host, which is where
the user's own browser has clearance.
- Mature models still have no price source anywhere, so that is now stated
instead of silent: price_check_attempted_at separates "tried and unreadable"
from "never looked", gated versions show a muted "Price unavailable" badge,
and the alerts panel reports unavailableCount.
- Failures are logged at warning level, once per host per TTL, with the
per-host reason, instead of only at debug level.
- The recorded alternatives (internal tRPC with the user's API key, or an
extension-assisted fetch from the user's browser) and the strengthened
upstream ask for a public price field are documented in the plan.
CivitAI's public API deliberately omits prices — paidAccess is trimmed to
{permanent, endsAt} because "pricing belongs to the purchase flow" — but the
public model page embeds the site's own model.getById result, including
paidAccess.terms, in its server-rendered payload. That is read anonymously
(no API key, no internal endpoint, no forged Origin), one request per gated
model, so only the ~2% of models that actually carry a gate pay for it.
- optional capture, off by default: price_tracking_enabled,
price_alert_threshold_buzz (0 = alert on "became free" only) and
price_check_ttl_hours; prices refresh on their own TTL and immediately when a
gate changes, and a failed fetch keeps the stored price instead of blanking it
- versions that stop carrying a gate are marked free (persisted gate_lapsed_at)
and gate transitions are reported as events on the refresh response, so a
version already in the library can announce that it became free
- price_alert_state plus a price_drop edge event; new
GET /api/lm/{type}/updates/price-alerts lists what is under the threshold
- versions tab shows the price (effective, with the list price struck through
and a Blue Buzz note) and a Free Now badge; an update check toasts the
transitions in one message
- the parser and the alerts query are unit-tested against a trimmed page
fixture, and every route definition is now asserted to resolve to a handler
Plan, verification notes and the deviations from it are in
docs/plans/paid-model-price-tracking.md.
CivitAI only returns a non-null paidAccess for an *active* gate: a lapsed gate
stays in the database as a tombstone and is filtered out server-side, so
{"permanent": false, "endsAt": null} — a timed gate whose window end has not
been recorded yet — is still enforced. Verified live: on model 1802980 that
version reports canDownload: false while its lapsed siblings report true.
Both the update service and the download gate dropped that shape, so such
versions read as free and "Hide Early Access Updates" missed them — the class
of bug reported in #1060.
The interpretation now lives in py/utils/paid_access.py and is shared, so the
badge, the update filter and the download warning cannot disagree.
Cross-checked both baseModel lists against CivitAI's official
baseModelRecords (packages/civitai-shared src/basemodel.constants.ts):
- CHECKPOINT_BASE_MODELS gains SD 2.0/2.1 768, SD 2.1 Unclip, SDXL 0.9 /
1.0 LCM / Turbo / Distilled, Playground v2 and Stable Cascade
(unCLIP-style but CheckpointLoader-loaded).
- DIFFUSION_MODEL_BASE_MODELS gains SVD XT, LTXV 2.5, Flux 3 Video,
Wan Image 2.7, Wan Video 2.7 / 3.0, HiDream-O1, Boogu and the Ming
Image Design families. API-only (Kling/Sora/Veo/Imagen...), 3D and
audio baseModels are intentionally skipped.
- Pony V7 exclusion now backed by live-API evidence (model 1901521 is
AuraFlow-architecture shipping .gguf variants).
CivitAI has no model-level diffusion ModelType: DiT models are uploaded
as "Checkpoint" or "UNet", with only uploader-chosen file types to tell
them apart. model.type "unet" previously fell through type derivation
and failed with 'not supported for download'; it now goes through the
checkpoint branch in both the download manager and the download routing
endpoint, so the standard chain (file type -> baseModel lists -> unknown
default) applies.
CivitAI labels new DiT architectures (MiniMax H3, future Flux/Wan/Qwen
variants) as model.type "Checkpoint" with plain "Model" file entries,
so the DIFFUSION_MODEL_BASE_MODELS allowlist could never keep up and
such downloads were mis-routed to the checkpoint roots (e.g. model
2877206 / version 3374439). The set of true full-checkpoint families is
closed, so the baseModel fallback is inverted:
1. file type UNet/Diffusion Model -> unet (unchanged)
2. baseModel in DIFFUSION_MODEL_BASE_MODELS (now incl. MiniMax H3) -> unet
3. baseModel in new CHECKPOINT_BASE_MODELS (SD 1.x/2.x/3.x, SDXL, Pony,
Illustrious, NoobAI) -> checkpoint
4. unknown/empty baseModel -> new unknown_base_model_routing setting,
defaulting to diffusion models
The setting is exposed under Settings > Downloads, validated in
SettingsManager, and threaded into both the download manager and the
download routing endpoint so they keep agreeing.
ghosttigerllc-bit