- Run a bounded model pipeline over the shared media limiter so model
preparation and failure bookkeeping overlap the previous model's tail.
New example_images_model_concurrency defaults to 2 (1 restores serial),
and metadata refreshes are serialized while media continues.
- Let the AIMD limiter probe one slot at a time above its base after a longer
success streak, up to example_images_download_max_concurrency (default 8).
A 429 still halves the limit below the base before recovery.
- Cache Civitai rendition -> blob redirects in Downloader.download_to_memory
for one hour (2048 entries, invalidated on 404), so retries skip the 301.
- Add scripts/benchmark_example_images.py, a reproducible old-vs-new A/B
harness that selects one cache/sidecar subset and runs serial, new-serial,
pipeline and aimd modes against the same temporary examples path.
- Document the measured results (19 models / 185 images, 3 repeats):
pipeline -23.4% vs new-serial, AIMD -22.2% vs pipeline, no 429s.
Will Miao
587793e155