feat: make library builder reruns incremental

Adding assets to a catalogued library used to mean re-describing every asset:
the checkpoint is deleted on success, and its key hashed the file list, so any
change invalidated it. Cataloguing five new assets cost the same as the whole
library.

The manifest itself is now the durable record of what is described. With the
new update_existing input (default on), a rerun loads the existing manifest,
reuses those descriptions, and only sends genuinely new files to Claude.
Entries whose file no longer exists are dropped. Turning it off re-describes
everything.

The checkpoint key no longer hashes the file list either, so adding an asset
mid-recovery does not discard an interrupted run's progress.

Not applied to organise-into-a-copy, where manifest paths are post-rename and
cannot be matched against a scan of the untouched source.

Verified with a stubbed CLI across four reruns: fresh library describes all 6,
an unchanged rerun describes 0, adding 3 describes only those 3, and deleting
2 describes 0 while dropping both from the manifest.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

README.md

@@ -62,6 +62,92 @@ Chat with large language models directly inside your ComfyUI workflows, with opt
 | **SF Claude Code** | Send text and images to Claude through the Claude Code CLI on your own machine, using your existing Claude login instead of an API key. Image slots appear as you fill them. |
 | **SF Claude Asset Director** | Point Claude at an asset library and it chooses which references suit the brief and writes the image prompt to go with them. Outputs an IMAGE batch plus the prompt, ready to wire into any generation node. |
 
+| **SF Claude Library Builder** | Point Claude at a folder of game assets and it writes a searchable manifest describing each one. Can also reorganise and rename the files. Feeds **SF Claude Asset Director**. |
+
+#### The asset library workflow
+
+These two nodes are a pair. **Library Builder** catalogues your assets once; **Asset Director**
+then searches that catalogue on every run without ever opening an image.
+
+```
+   one time, when assets change          every generation
+┌──────────────────────────────┐   ┌──────────────────────────────────┐
+│ SF Claude Library Builder    │   │ SF Claude Asset Director         │
+│ opens every image,           │──►│ searches the manifest,           │──► images + prompt
+│ writes manifest.json         │   │ opens nothing                    │
+└──────────────────────────────┘   └──────────────────────────────────┘
+```
+
+That split is the point: cataloguing is slow (~5s per asset) but happens rarely, while choosing
+is instant and happens constantly.
+
+**The manifest schema** (`sf-asset-library/1`):
+
+```json
+{
+  "schema": "sf-asset-library/1",
+  "project": "Fantasy RPG",
+  "art_style": "Hand-painted 2D, saturated palette, thick outlines, 3/4 view",
+  "generated": "2026-10-08",
+  "assets": [
+    {
+      "file": "characters/heroes/knight_aldric.png",
+      "name": "Aldric the Knight",
+      "category": "character",
+      "subcategory": "hero",
+      "archetype": "tank",
+      "tags": ["armoured", "sword", "front view", "heroic"],
+      "desc": "Armoured human knight in silver plate with a red tabard, standing front-on."
+    }
+  ]
+}
+```
+
+It's shaped for fast searching rather than for humans: one **flat list** (cheaper to scan than a
+tree), the **same keys on every entry**, faceted fields (`category` / `subcategory` /
+`archetype` / `tags`) to narrow by, and a one-sentence `desc` carrying the meaning. The
+top-level **`art_style`** matters more than it looks — Asset Director reads it and keeps every
+prompt it writes consistent with it, so a library generates in one coherent style.
+
+**Library Builder modes:**
+
+| Mode | What it touches |
+|---|---|
+| **manifest only** (default) | Nothing. Reads the images, returns the JSON as text |
+| **organise into a copy** | Writes a tidy, renamed copy to `output_path`; originals untouched |
+| **organise in place** | Renames and moves your actual files |
+
+`write_manifest` is **off** by default, so the first run shows you the JSON without writing
+anything. Turn it on once you're happy, or paste the text in yourself.
+
+Start in **manifest only**. Reorganising is genuinely useful on a folder of `IMG_0021.png` and
+`final_v2 copy.png`, but it moves real files — try it on a copy first.
+
+**Large libraries are processed in batches, not one session.** `batch_size` (default 15) caps
+how many assets Claude handles per call — asking it to hold a whole large library open, compose
+one giant manifest, and write the file all in a single uninterrupted session is what used to
+blow up in turns and cost on libraries of 100+ assets, with nothing produced at the end. Each
+batch now only reads its own assets and replies with JSON; the node assembles and writes the
+manifest itself, so no file-writing tool is ever handed to Claude for this.
+
+**Reruns are incremental.** With `update_existing` on (the default), the node reads the manifest
+already in the library and only catalogues files it doesn't yet describe. So the normal workflow
+is: drop new assets into the folder, run the same node against the same path, and only the new
+ones cost anything. Entries whose file has since been deleted are dropped automatically. Turn
+`update_existing` off to re-describe everything from scratch.
+
+*(Not available in `organise into a copy` mode, where the manifest holds post-rename paths that
+can't be matched against the untouched source folder.)*
+
+Separately, a hidden checkpoint (`.sf_library_builder_checkpoint.json`, removed automatically on
+success) covers a run that dies partway. Rerunning **resumes** — finished batches are skipped,
+not re-paid for.
+
+If a batch's reply comes back **cut off** — the model ran past its reply length limit partway
+through describing a wordy set of assets — that batch is automatically **retried as two smaller
+halves**, repeatedly if needed, rather than failing the run. You'll see it in the log as
+`Batch 7/7: the reply was cut off… Retrying as two batches of 5 and 6`.
+
 #### Using SF Claude Asset Director
 
 Give it a folder of assets and a brief; it returns the references Claude chose and the prompt

py/sf_claude_asset_director.py

@@ -56,7 +56,9 @@ def build_request(system_prompt, instruction, files, manifest_text, max_images,
 
     if manifest_text:
         parts.append(
-            "Here is the asset library manifest describing what each file contains:\n\n"
+            "Here is the asset library manifest describing what each file contains. "
+            "Search it to choose; if it declares an `art_style`, the prompt you write "
+            "must stay consistent with it.\n\n"
             f"{manifest_text.strip()}"
         )
 
@@ -171,10 +173,11 @@ class SFClaudeAssetDirector:
                 "manifest": (
                     "STRING",
                     {
-                        "default": "assets.json",
+                        "default": "manifest.json",
                         "tooltip": (
                             "File inside the library describing each asset, used to choose "
-                            "between them. Any text or JSON format. Ignored if absent."
+                            "between them. Written by SF Claude Library Builder, though any "
+                            "text or JSON works. Ignored if absent."
                         ),
                     },
                 ),

py/sf_claude_library_builder.py

@@ -0,0 +1,646 @@
+# ABOUTME: Claude surveys a folder of game assets and writes a searchable manifest for it
+# ABOUTME: Processes assets in small batches with a resumable checkpoint, so a failure costs little
+
+import hashlib
+import json
+import os
+import subprocess
+import tempfile
+import time
+
+from .sf_claude_code import MODEL_CHOICES, find_claude_binary
+
+IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".webp", ".bmp", ".gif", ".tga"}
+
+MODE_MANIFEST = "manifest only (read-only)"
+MODE_COPY = "organise into a copy"
+MODE_IN_PLACE = "organise in place"
+MODES = [MODE_MANIFEST, MODE_COPY, MODE_IN_PLACE]
+
+SCHEMA_VERSION = "sf-asset-library/1"
+CHECKPOINT_NAME = ".sf_library_builder_checkpoint.json"
+
+ASSET_SCHEMA_SPEC = """{
+  "file": "<path relative to the library root, exactly as it exists on disk>",
+  "name": "<short human name>",
+  "category": "<character|environment|prop|ui|vfx|other>",
+  "subcategory": "<hero|enemy|npc|location|background|weapon|icon|...>",
+  "archetype": "<for characters: tank|healer|rogue|mage|boss|minion|... else omit>",
+  "tags": ["<lowercase keyword>", "..."],
+  "desc": "<one sentence: what it depicts, pose/view, dominant colours>"
+}"""
+
+DEFAULT_SYSTEM_PROMPT = """You are cataloguing a game asset library.
+
+Judge each asset on what it depicts and what role it would play in a game.
+Keep `desc` to one sentence that would let someone pick this asset without
+seeing it. Make `tags` lowercase, specific and useful for searching — subject,
+colour, pose, view angle, mood. Prefer a small shared vocabulary across the
+library over inventing a new word per asset."""
+
+
+def scan_assets(root):
+    """Image files under `root`, relative to it, in a stable sorted order."""
+    found = []
+    for folder, dirs, names in os.walk(root):
+        dirs[:] = [d for d in dirs if not d.startswith(".")]
+        for name in names:
+            if os.path.splitext(name)[1].lower() in IMAGE_EXTS:
+                found.append(os.path.relpath(os.path.join(folder, name), root))
+    return sorted(found)
+
+
+def chunk(items, size):
+    for i in range(0, len(items), size):
+        yield items[i : i + size]
+
+
+class BatchParseError(RuntimeError):
+    """Claude's reply was not usable JSON.
+
+    Nearly always a reply that ran past its length limit and stopped partway,
+    which a smaller batch fixes — so the caller retries by splitting rather than
+    giving up.
+    """
+
+
+def extract_json(text):
+    """The JSON object out of a reply that may be fenced or prefaced."""
+    start, end = text.find("{"), text.rfind("}")
+    if start == -1 or end <= start:
+        raise BatchParseError("the reply stopped before any complete JSON object")
+    body = text[start : end + 1]
+    try:
+        return json.loads(body)
+    except json.JSONDecodeError as e:
+        raise BatchParseError(f"the reply was cut off or malformed ({e})")
+
+
+def canonicalize(claude_path, expected):
+    """Map Claude's returned file path onto one of the paths we actually asked
+    about, tolerating a leading './' or a basename-only answer."""
+    cleaned = (claude_path or "").strip().lstrip("./")
+    if cleaned in expected:
+        return cleaned
+    base = os.path.basename(cleaned).lower()
+    for candidate in expected:
+        if os.path.basename(candidate).lower() == base:
+            return candidate
+    return cleaned
+
+
+def checkpoint_key(mode, manifest_name, system_prompt, project_notes):
+    """A fingerprint of this job's settings, so a checkpoint only resumes a
+    matching rerun.
+
+    Deliberately excludes the file list: adding or removing an asset should not
+    throw away a half-finished run, and entries are keyed by path anyway, so
+    stale ones are filtered out when the manifest is assembled.
+    """
+    payload = json.dumps(
+        {
+            "mode": mode,
+            "manifest_name": manifest_name,
+            "system_prompt": system_prompt,
+            "project_notes": project_notes,
+        },
+        sort_keys=True,
+    )
+    return hashlib.sha256(payload.encode("utf-8")).hexdigest()[:16]
+
+
+def load_existing_manifest(path):
+    """A previously written manifest, as (entries by file, project, art_style).
+
+    This is what makes a rerun incremental: anything already described here is
+    not sent to Claude again.
+    """
+    if not os.path.isfile(path):
+        return {}, None, None
+    try:
+        with open(path, encoding="utf-8") as f:
+            data = json.load(f)
+    except (OSError, json.JSONDecodeError):
+        return {}, None, None
+
+    entries = {}
+    for entry in data.get("assets") or []:
+        if isinstance(entry, dict) and entry.get("file"):
+            entries[entry["file"]] = entry
+    return entries, data.get("project"), data.get("art_style")
+
+
+def load_checkpoint(root, key):
+    path = os.path.join(root, CHECKPOINT_NAME)
+    if not os.path.isfile(path):
+        return None
+    try:
+        with open(path, encoding="utf-8") as f:
+            data = json.load(f)
+    except (OSError, json.JSONDecodeError):
+        return None
+    return data if data.get("key") == key else None
+
+
+def save_checkpoint(root, key, project, art_style, done_assets):
+    """Atomic write, so a crash mid-save can't corrupt the checkpoint."""
+    path = os.path.join(root, CHECKPOINT_NAME)
+    data = {"key": key, "project": project, "art_style": art_style, "done_assets": done_assets}
+    fd, tmp = tempfile.mkstemp(dir=root, prefix=".sf_checkpoint_")
+    try:
+        with os.fdopen(fd, "w", encoding="utf-8") as f:
+            json.dump(data, f)
+        os.replace(tmp, path)
+    except Exception:
+        if os.path.exists(tmp):
+            os.remove(tmp)
+        raise
+
+
+def clear_checkpoint(root):
+    path = os.path.join(root, CHECKPOINT_NAME)
+    if os.path.isfile(path):
+        os.remove(path)
+
+
+def build_batch_instruction(mode, batch_files, total_count, project_notes, established, out_dir):
+    """The task description for ONE batch call — scoped to just its own files."""
+    parts = [
+        f"You are cataloguing {len(batch_files)} file(s) from one batch of a "
+        f"{total_count}-file game asset library. You only need to handle the "
+        f"files listed below — nothing else exists from your point of view."
+    ]
+
+    if project_notes.strip():
+        parts.append(f"About this project:\n{project_notes.strip()}")
+
+    listing = "\n".join(f"  {name}" for name in batch_files)
+    parts.append(f"Your files for this batch:\n{listing}")
+
+    parts.append(
+        "Open each one with the Read tool before describing it. Work through them "
+        "steadily; do not guess from a filename."
+    )
+
+    if mode == MODE_MANIFEST:
+        parts.append("Do NOT move, rename, copy or delete anything. This is a survey only.")
+    else:
+        where = (
+            f"a new folder at `{out_dir}`, leaving the originals untouched"
+            if mode == MODE_COPY
+            else "the working directory itself"
+        )
+        parts.append(
+            f"Then reorganise ONLY these files into {where}:\n"
+            f"- Group by category, e.g. characters/heroes, characters/enemies, "
+            f"environments, props, ui, vfx.\n"
+            f"- Rename to lowercase_with_underscores, descriptive of the subject, "
+            f"no spaces, no version noise. Keep the original extension.\n"
+            f"- Never discard a file.\n"
+            f"- Use Bash (cp for a copy, mv in place) to do the moving.\n"
+            f"The `file` field of each asset must be its path AFTER reorganising, "
+            f"relative to the library root."
+        )
+
+    if established:
+        project, art_style = established
+        parts.append(
+            f"This library's project name and art style are already settled by an "
+            f"earlier batch — reuse them exactly, do not invent new ones:\n"
+            f'  project: "{project}"\n  art_style: "{art_style}"'
+        )
+        parts.append(
+            f"Reply with ONLY a JSON object of this shape — no commentary, no code "
+            f"fence, no other top-level keys:\n"
+            f'{{"assets": [{ASSET_SCHEMA_SPEC}]}}\n'
+            f"The assets array must contain exactly one entry per file listed above."
+        )
+    else:
+        parts.append(
+            f"You are establishing this library's project name and art style for "
+            f"the first time — later batches will reuse whatever you choose here, "
+            f"so make `art_style` a precise sentence covering the whole library's "
+            f"visual approach, not just these files.\n"
+            f"Reply with ONLY a JSON object of this shape — no commentary, no code "
+            f"fence:\n"
+            f'{{"project": "<short project name>", "art_style": "<one sentence>", '
+            f'"assets": [{ASSET_SCHEMA_SPEC}]}}\n'
+            f"The assets array must contain exactly one entry per file listed above."
+        )
+
+    return "\n\n".join(parts)
+
+
+class SFClaudeLibraryBuilder:
+    """
+    Surveys a folder of game assets and writes a manifest describing each one,
+    so SF Claude Asset Director can choose between them without opening images.
+
+    Processes assets in small batches rather than one session, because a single
+    session covering a large library can blow up in turns and cost with nothing
+    to show for it. A checkpoint records finished batches, so a failed or
+    interrupted run resumes instead of re-paying for work already done.
+
+    Read-only by default. It can also reorganise and rename the files, into a
+    copy or in place, when you ask it to.
+    """
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "library_path": (
+                    "STRING",
+                    {"multiline": False, "default": "", "tooltip": "Folder holding the assets to catalogue."},
+                ),
+                "mode": (
+                    MODES,
+                    {
+                        "default": MODE_MANIFEST,
+                        "tooltip": (
+                            "manifest only: nothing is moved or renamed. "
+                            "organise into a copy: originals untouched, tidy copy written "
+                            "to output_path. organise in place: renames and moves your "
+                            "actual files."
+                        ),
+                    },
+                ),
+                "output_path": (
+                    "STRING",
+                    {"multiline": False, "default": "", "tooltip": "Destination folder, for 'organise into a copy' only."},
+                ),
+                "project_notes": (
+                    "STRING",
+                    {
+                        "multiline": True,
+                        "default": "",
+                        "tooltip": (
+                            "What the game is, and anything Claude could not infer from "
+                            "the images — genre, art direction, naming you prefer."
+                        ),
+                    },
+                ),
+                "system_prompt": (
+                    "STRING",
+                    {
+                        "multiline": True,
+                        "default": DEFAULT_SYSTEM_PROMPT,
+                        "tooltip": "How to categorise and describe. Edit to suit your project.",
+                    },
+                ),
+                "manifest_name": (
+                    "STRING",
+                    {"default": "manifest.json", "tooltip": "Filename for the manifest, written at the library root."},
+                ),
+                "write_manifest": (
+                    "BOOLEAN",
+                    {
+                        "default": False,
+                        "tooltip": (
+                            "Off: the manifest is only returned as text for you to read "
+                            "and save yourself. On: it is written into the library."
+                        ),
+                    },
+                ),
+                "update_existing": (
+                    "BOOLEAN",
+                    {
+                        "default": True,
+                        "tooltip": (
+                            "On: reuse descriptions already in the manifest and only "
+                            "catalogue files that are new, so adding assets costs only "
+                            "those assets. Entries whose file has since been deleted are "
+                            "dropped. Off: describe every asset again from scratch."
+                        ),
+                    },
+                ),
+                "batch_size": (
+                    "INT",
+                    {
+                        "default": 15,
+                        "min": 3,
+                        "max": 50,
+                        "step": 1,
+                        "tooltip": (
+                            "Assets per Claude call. Keeps each session small enough to "
+                            "finish reliably — a failed batch only costs that batch, not "
+                            "the whole library, and a rerun resumes rather than restarts."
+                        ),
+                    },
+                ),
+                "model": (
+                    MODEL_CHOICES,
+                    {"default": "default", "tooltip": "Which Claude model does the cataloguing."},
+                ),
+                "timeout_seconds": (
+                    "INT",
+                    {
+                        "default": 300,
+                        "min": 60,
+                        "max": 1800,
+                        "step": 30,
+                        "tooltip": "Per BATCH, not per library. Budget roughly 5s per asset in the batch.",
+                    },
+                ),
+                "seed": (
+                    "INT",
+                    {
+                        "default": 0,
+                        "min": 0,
+                        "max": 0xFFFFFFFF,
+                        "control_after_generate": True,
+                        "tooltip": "Not sent to Claude. Change it to force a fresh run.",
+                    },
+                ),
+            }
+        }
+
+    RETURN_TYPES = ("STRING", "STRING")
+    RETURN_NAMES = ("manifest", "report")
+    FUNCTION = "build"
+    CATEGORY = "Stillfront/LLM"
+
+    DESCRIPTION = """
+Claude looks at every asset in a folder and writes a manifest describing each —
+category, archetype, tags and a one-line description — plus a project-level art
+style. **SF Claude Asset Director** then searches that manifest instead of
+opening images.
+
+Processes **batch_size** assets per Claude call rather than the whole library at
+once, and tracks progress in a hidden checkpoint file. If a run fails partway,
+rerunning with the same settings resumes from where it stopped instead of
+re-paying for batches already done.
+
+With **update_existing** on (the default), rerunning against a library that
+already has a manifest only catalogues files that are new — so adding assets
+costs just those assets. Entries whose file has since been deleted are dropped.
+
+Defaults to **manifest only**, which touches nothing: you get the JSON as text to
+read and save yourself. Switch `write_manifest` on to have it saved into the
+library, or change `mode` to let Claude reorganise and rename the files too.
+
+Budget about 5 seconds per asset — it opens every image.
+"""
+
+    def build(
+        self,
+        library_path,
+        mode,
+        output_path,
+        project_notes,
+        system_prompt,
+        manifest_name,
+        write_manifest,
+        update_existing,
+        batch_size,
+        model,
+        timeout_seconds,
+        seed,
+    ):
+        root = os.path.expanduser(library_path.strip())
+        if not root or not os.path.isdir(root):
+            raise ValueError(f"library_path is not a folder: {library_path!r}")
+
+        out_dir = os.path.expanduser(output_path.strip())
+        if mode == MODE_COPY:
+            if not out_dir:
+                raise ValueError("output_path is required for 'organise into a copy'.")
+            if os.path.abspath(out_dir).startswith(os.path.abspath(root) + os.sep):
+                raise ValueError("output_path must sit outside library_path.")
+            os.makedirs(out_dir, exist_ok=True)
+
+        binary = find_claude_binary()
+        if not binary:
+            raise RuntimeError(
+                "Claude Code CLI not found. Install it from https://claude.com/claude-code "
+                "and sign in, then restart ComfyUI."
+            )
+
+        assets = scan_assets(root)
+        if not assets:
+            raise ValueError(f"No image files found in {root}")
+
+        manifest_root = out_dir if mode == MODE_COPY else root
+        manifest_path = os.path.join(manifest_root, manifest_name)
+
+        done_by_file = {}
+        project = art_style = None
+
+        # An existing manifest is the durable record of what is already described,
+        # so a rerun only pays for assets that are genuinely new. Skipped for the
+        # copy mode, where the manifest holds post-rename paths that cannot be
+        # matched against a scan of the untouched source folder.
+        reused = 0
+        if update_existing and mode != MODE_COPY:
+            existing, prev_project, prev_style = load_existing_manifest(manifest_path)
+            if existing:
+                done_by_file.update(existing)
+                project, art_style = prev_project, prev_style
+                reused = sum(1 for f in assets if f in existing)
+                gone = len(existing) - reused
+                print(
+                    f"[SF Claude Library Builder] Found {manifest_name}: reusing "
+                    f"{reused} existing description(s)"
+                    + (f", dropping {gone} for deleted file(s)" if gone else "")
+                    + "."
+                )
+
+        # A checkpoint from an interrupted run is newer than the manifest, so it wins.
+        key = checkpoint_key(mode, manifest_name, system_prompt, project_notes)
+        checkpoint = load_checkpoint(root, key)
+        if checkpoint:
+            for entry in checkpoint.get("done_assets", []):
+                if entry.get("file"):
+                    done_by_file[entry["file"]] = entry
+            project = checkpoint.get("project") or project
+            art_style = checkpoint.get("art_style") or art_style
+            print(
+                f"[SF Claude Library Builder] Resuming an interrupted run — "
+                f"{len(done_by_file)}/{len(assets)} asset(s) already catalogued."
+            )
+
+        todo = sum(1 for f in assets if f not in done_by_file)
+        if todo == 0:
+            print("[SF Claude Library Builder] Nothing new to catalogue.")
+        else:
+            print(f"[SF Claude Library Builder] {todo} asset(s) to catalogue.")
+
+        # Claude only ever reads images and replies with JSON; the node writes the
+        # manifest itself, and the Write tool is never offered. Bash is needed
+        # only to move/rename files in the organise modes.
+        tools = "Read" if mode == MODE_MANIFEST else "Read,Bash"
+        batches = list(chunk(assets, max(1, batch_size)))
+
+        total_cost = 0.0
+        started = time.time()
+
+        def fail(message):
+            """Stop the run, leaving finished batches safely checkpointed."""
+            save_checkpoint(root, key, project, art_style, list(done_by_file.values()))
+            raise RuntimeError(
+                f"{message} {len(done_by_file)}/{len(assets)} asset(s) are saved in a "
+                f"checkpoint — rerun with the same settings to resume rather than "
+                f"start over."
+            )
+
+        def run_batch(files, label):
+            """One Claude call. Returns (parsed reply, cost, turns)."""
+            established = (project, art_style) if project and art_style else None
+            command = [
+                binary,
+                "-p",
+                build_batch_instruction(
+                    mode, files, len(assets), project_notes, established, out_dir
+                ),
+                "--output-format",
+                "json",
+                "--allowedTools",
+                tools,
+            ]
+            if system_prompt.strip():
+                command += ["--append-system-prompt", system_prompt.strip()]
+            if model != "default":
+                command += ["--model", model]
+
+            try:
+                completed = subprocess.run(
+                    command,
+                    cwd=root,
+                    capture_output=True,
+                    text=True,
+                    timeout=timeout_seconds,
+                    stdin=subprocess.DEVNULL,
+                )
+            except subprocess.TimeoutExpired:
+                fail(f"Batch {label} did not finish within {timeout_seconds}s.")
+
+            if completed.returncode != 0:
+                detail = (completed.stderr or completed.stdout or "").strip()[:500]
+                fail(f"Batch {label} failed (exit {completed.returncode}). {detail}")
+
+            try:
+                payload = json.loads(completed.stdout)
+            except json.JSONDecodeError:
+                fail(f"Batch {label}: unreadable response: {completed.stdout.strip()[:300]}")
+
+            if payload.get("is_error"):
+                fail(f"Batch {label} reported an error: {payload.get('result', 'unknown')}.")
+
+            # May raise BatchParseError, which catalogue() retries by splitting.
+            data = extract_json(payload.get("result", ""))
+            return data, payload.get("total_cost_usd") or 0.0, payload.get("num_turns", "?")
+
+        def catalogue(files, label):
+            """Catalogue `files`, halving and retrying if the reply comes back cut off.
+
+            Each piece that succeeds is merged and checkpointed immediately, so a
+            later failure never costs work already paid for.
+            """
+            nonlocal project, art_style, total_cost
+
+            try:
+                data, cost, turns = run_batch(files, label)
+            except BatchParseError as exc:
+                if len(files) == 1:
+                    fail(
+                        f"Batch {label}: {exc}. This is a single asset "
+                        f"({files[0]}), so it cannot be split any further."
+                    )
+                half = len(files) // 2
+                print(
+                    f"[SF Claude Library Builder] Batch {label}: {exc}. "
+                    f"Retrying as two batches of {half} and {len(files) - half}."
+                )
+                catalogue(files[:half], f"{label}a")
+                catalogue(files[half:], f"{label}b")
+                return
+
+            total_cost += cost
+            if project is None:
+                project = data.get("project", "Untitled project")
+                art_style = data.get("art_style", "")
+
+            expected = set(files)
+            for entry in data.get("assets") or []:
+                raw = entry.get("file")
+                if not raw:
+                    continue
+                entry["file"] = canonicalize(raw, expected)
+                done_by_file[entry["file"]] = entry
+
+            save_checkpoint(root, key, project, art_style, list(done_by_file.values()))
+
+            print(
+                f"[SF Claude Library Builder] Batch {label} done: {turns} turns, "
+                f"${cost:.2f}. Running total: {len(done_by_file)}/{len(assets)} "
+                f"asset(s), ${total_cost:.2f}."
+            )
+
+            skipped = [f for f in files if f not in done_by_file]
+            if skipped:
+                print(
+                    f"[SF Claude Library Builder] Batch {label}: {len(skipped)} file(s) "
+                    f"came back undescribed; a rerun will pick them up."
+                )
+
+        for batch_num, batch_files in enumerate(batches, start=1):
+            remaining = [f for f in batch_files if f not in done_by_file]
+            if not remaining:
+                continue  # fully covered by the checkpoint already
+
+            print(
+                f"[SF Claude Library Builder] Batch {batch_num}/{len(batches)} "
+                f"({len(remaining)} asset(s))..."
+            )
+            catalogue(remaining, f"{batch_num}/{len(batches)}")
+
+        elapsed = time.time() - started
+
+        # Assemble the final manifest in the library's original scan order.
+        final_assets = [done_by_file[f] for f in assets if f in done_by_file]
+        manifest_data = {
+            "schema": SCHEMA_VERSION,
+            "project": project or "Untitled project",
+            "art_style": art_style or "",
+            "generated": time.strftime("%Y-%m-%d"),
+            "assets": final_assets,
+        }
+        manifest_text = json.dumps(manifest_data, indent=2)
+
+        if write_manifest:
+            os.makedirs(manifest_root, exist_ok=True)
+            with open(manifest_path, "w", encoding="utf-8") as f:
+                f.write(manifest_text)
+
+        clear_checkpoint(root)
+
+        counts = {}
+        for entry in final_assets:
+            counts[entry.get("category", "?")] = counts.get(entry.get("category", "?"), 0) + 1
+
+        newly_done = max(0, len(final_assets) - reused)
+        report = "\n".join(
+            [
+                f"mode:        {mode}",
+                f"scanned:     {len(assets)} file(s) in {root}",
+                f"in manifest: {len(final_assets)} asset(s)",
+                f"reused:      {reused} already described, {newly_done} newly catalogued",
+                "categories:  " + (", ".join(f"{k} x{v}" for k, v in sorted(counts.items())) or "none"),
+                f"art style:   {manifest_data['art_style'] or '(not set)'}",
+                "manifest:    " + (manifest_path if write_manifest else "returned as text, not saved"),
+                f"took:        {elapsed:.0f}s, ${total_cost:.2f} total",
+            ]
+        )
+        print(
+            f"[SF Claude Library Builder] Done: {len(final_assets)}/{len(assets)} asset(s), "
+            f"{elapsed:.0f}s, ${total_cost:.2f}."
+        )
+
+        if len(final_assets) < len(assets):
+            report += f"\nWARNING:     {len(assets) - len(final_assets)} file(s) were not catalogued."
+
+        return (manifest_text, report)
+
+
+NODE_CLASS_MAPPINGS = {"SFClaudeLibraryBuilder": SFClaudeLibraryBuilder}
+NODE_DISPLAY_NAME_MAPPINGS = {"SFClaudeLibraryBuilder": "SF Claude Library Builder"}

pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "comfyui-sf-nodes"
 description = "Custom nodes for ComfyUI including LLM Chat with support for Claude, Gemini, and OpenAI models, WaveSpeed AI image/video generation, VertexAI integration, and workflow utilities"
-version = "1.2.0"
+version = "1.4.0"
 license = { file = "LICENSE" }
 dependencies = [
     "requests>=2.31.0",