Compare commits

...
134 Commits
Author SHA1 Message Date
DemonGatanjieuandClaude Opus 5.5 8a4c5d5c72 v1.62.1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 01:52:28 +08:00
DemonGatanjieuandClaude Opus 5.5 b9fd30ad72 Fix Anomalous sometimes not starting
ComfyUI imports every script under web/ on its own and at once, so any module
of the big import cycle can run first. ui_ai_workbench.js wrote to a const of
ui_ai_results.js as it loaded, which failed when ui_ai_results.js or
ui_ai_step_body.js was entered first, and the whole plugin went with it until
a reload. The value now lives in a module that imports nothing, and
tests/module_cycles.mjs refuses the pattern.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 01:52:28 +08:00
Claude e01524c80e v1.62.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 02:06:12 +00:00
DemonGatanjieuandClaude Opus 5.5 9c5075b3a3 Merge branch 'feature/1.61' into feature/1.62 (v1.61.1)
The delete-character change was already on 1.62; its CHANGELOG lines move to the v1.61.1 section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 21:35:57 +08:00
DemonGatanjieuandClaude Opus 5.5 be91723c57 v1.61.1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:59:28 +08:00
DemonGatanjieuandClaude Opus 5.5 f3100efde7 Delete character needs Anomalous_TTS interface 19
Interface 18 shipped in Anomalous_TTS v1.1.0 without the route, so the
button now waits for 19 (v1.1.1) instead of failing on a click.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:56:46 +08:00
DemonGatanjieuandClaude Opus 5.5 f1ae47f21a Delete a character from its card, to the Recycle Bin
Anomalous_TTS issue #2: there was no way to remove a character. Each card
now has an orange Delete character button (also on broken characters);
after a confirm, the node (interface 18, local only) moves the character's
folder to the Recycle Bin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:56:42 +08:00
DemonGatanjieuandClaude Opus 5.5 5767b4614c Delete character needs Anomalous_TTS interface 19
Interface 18 shipped in Anomalous_TTS v1.1.0 without the route, so the
button now waits for 19 (v1.1.1) instead of failing on a click.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:56:27 +08:00
DemonGatanjieuandClaude Opus 5.5 9a270d0a42 Delete a character from its card, to the Recycle Bin
Anomalous_TTS issue #2: there was no way to remove a character. Each card
now has an orange Delete character button (also on broken characters);
after a confirm, the node (interface 18, local only) moves the character's
folder to the Recycle Bin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:42:06 +08:00
DemonGatanjieuandClaude Opus 5.5 cae7fd2a4a Audio gallery becomes a view of Voices; the rail fits the number keys
With the AI workbench the rail had ten entries for nine number keys, so
the audio gallery had none. It is now the third tab of Voices (Characters,
Voice-over, Audio); 9 reopens Voices on the view used last, and a test
keeps the rail within nine entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:26:31 +08:00
DemonGatanjieuandClaude Opus 5.5 1d444f0669 Let the AI's Load Image take an output picture
new_workflow / add_node checked a LoadImage value against the input folder's files only, so
an AI could not build an img2img on a picture it had just made (and nothing at all with an
empty input folder). Names ComfyUI annotates ('x.png [output]') now pass; ComfyUI checks the
file. CHANGELOG also notes the rail labels fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 19:57:21 +08:00
DemonGatanjieuandClaude Opus 5.5 d3f249fd1a Rail shows 模型 / 图库 again, not the raw keys
ce35145 removed 'models' and 'gallery' as unused, but the rail (labelKey) and the page
titles (TITLE_KEYS) name them by key. A test now checks every such key in both languages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 19:35:26 +08:00
DemonGatanjieuandClaude Opus 5.5 c882d79eea A failure that named no target is resolved by a later success of its tool
279ad22 keys recovery by target, so a first try with missing or misnamed
arguments (a light model's {} call) recorded no target and a corrected retry
could never match it: the task stayed marked "a step failed". Such a failure is
now resolved by any later completed success of the same tool; a failure with a
target still needs that target's success.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 15:30:08 +08:00
DemonGatanjieu 279ad22a20 Honor completed task follow-ups and count only completed target retries as recovery 2026-10-27 03:58:05 +08:00
DemonGatanjieuandClaude Opus 5.5 39a0e50fc1 Keep what the AI shows right after done in that task
From a Flash run: after report(done) it showed a before and after and
asked whether the user is happy, and both went into a new task. A
panel, question or note within 5 minutes of the AI's own report(done)
now opens that task again (unless the user started or wrote to
another); a look still leaves it ended. "Just done" now means ended
by a report, not that the report is the last step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 01:20:30 +08:00
DemonGatanjieuandClaude Opus 5.5 1d3bb5ae2b Task list says what is true: recovered errors, looks after done, waiting tasks
From the user's first real run: the finished task showed as failed for
one wrong argument corrected on the next call, a look the AI took
right after report(done) made an empty one-step task, and a task the
user started 18 hours earlier still read "waiting for the AI" though
it had lapsed after 12 hours. Now only failures the AI did not get
past mark a task, a read-only call within 5 minutes of report(done)
goes into that task (it stays ended) unless the user started or wrote
to another, and a task the user starts waits until an AI takes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 21:58:50 +08:00
DemonGatanjieuandClaude Opus 5.5 1b03553946 Correct a needed argument only from a known other name
A value of the right type under an unknown name may mean something
else (set_prompt {style: "anime"} is no prompt), so the correction now
takes only the other names light models were seen to write in the tool
exams (name/workflow_name, file/filename/image_name, text/positive,
model/id, node/id); any other name is answered with what the tool takes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 21:34:04 +08:00
DemonGatanjieu db405a79f0 Keep corrected MCP arguments consistent and preserve answer editing state 2026-10-26 21:25:15 +08:00
DemonGatanjieuandClaude Opus 5.5 6289e895ee Take a misnamed needed argument as meant; second Flash exam
A call lacking its one needed argument but carrying one unknown
argument of that type is run with it (open_workflow {name} is
{workflow}), and the AI is told the right name. The timeline names
zoom_image steps; a test makes every tool have a name in both
languages. tool-exam.md records the second exam, all eight tasks
including the repaint one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 20:23:03 +08:00
DemonGatanjieuandClaude Opus 5.5 54b072bd91 Keep half-typed answers: no auto-pick over them, no wipe on redraw
A question the user began to answer is never answered with the
recommended option on time-out; the AI is told they began an answer
and ends its turn. On the page, what was half typed in a question's
line survives the question being drawn again (another question came,
the AI read an answer), with the cursor; that question stays first in
the strip; a narrow window lights the stage tab instead of taking it
from under the user's typing; Enter that picks an input-method word no
longer sends a stage answer line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 19:02:46 +08:00
DemonGatanjieu e29fd6f5f5 Fix task resumption and answer delivery at wait deadlines 2026-10-26 18:42:32 +08:00
DemonGatanjieuandClaude Opus 5.5 9633d3451a Make tools easier to get right on the first call, from a Flash exam
Gemini 3.8 Flash took seven exam tasks through Antigravity (107 calls, 19
failed, every task done). Nearly all failures were a tool's first use:
empty or guessed arguments. So:

- An argument error lists what the tool takes ("set_prompt takes: prompt
  (needed), role, node_id, mode"), and the next call is right.
- One argument name per thing: set_prompt's text is prompt (as in
  run_workflow), open_workflow's name is workflow (as in run_workflow).
- A wait over 55 s waits 55 s instead of failing; search_models takes a
  model type in the singular too.
- An unread reply older than two hours no longer draws a new chat's calls
  into its old task (a two-day-old mark took the first exam task).

The exam doc gets the setup, the Antigravity command line and the results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 16:24:58 +08:00
DemonGatanjieuandClaude Opus 5.5 0764683c52 Remove the daily tool set: AI apps get every tool
Antigravity, Claude Code, Cursor and VS Code load tool descriptions on
demand themselves; looking tools up through Anomalous (tool_help, use_tool)
only doubled that: in one Gemini 3.8 Flash run 18 of 29 calls went to it,
while full-set chats had none. tool_help, use_tool, action_catalog.DAILY,
ai_gates.tool_set and the Gates row go.

An app still holding an older list is now told apart by the list it fetched
last (a fingerprint per app in anomalous/ai_tool_lists.json), so the note
under the task header also covers an Anomalous update.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 15:01:28 +08:00
DemonGatanjieuandClaude Opus 5.5 cb523930da Guide the AI with fewer detours and guesses
- An AI app calling by an older tool list (tool_help or use_tool not offered,
  use_tool for a tool offered directly, a tool the list lacks) is told so, and
  the task header shows how to reconnect until the app fetches tools/list again.
- add_lora / set_model compare the base models on the model line
  (model_advice.py; the page reports the line) and add base_model_note.
- get_model gives author_examples (sampler, steps, CFG, size, a LoRA's weight
  from the Civitai record's example pictures) and the version name.
- run_workflow says ComfyUI's refusal as which node, what is wrong and the tool
  that fixes it (run_refusal.py; mcp_run.js passes the refusal as data).
- Gates "When I don't answer": wait for me, or go with the recommendation once
  the waiting time runs out (choices with a recommended option only), handed to
  the AI in the same call and shown on the stage and under Decisions.
- A tool budget test (count, size, description length, required arguments) and
  the exam tasks with their scorecard (docs/architecture/tool-exam.md,
  tools/tool_exam.mjs). The tool set's text recommends Full for Antigravity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 14:26:41 +08:00
DemonGatanjieuandClaude Opus 5.5 9f9a13b11b Keep CHANGELOG.md's CRLF line endings
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 03:55:16 +08:00
DemonGatanjieuandClaude Opus 5.5 342946141e Make the AI's questions and the waiting page say what to press
The user took an option for something to select and then pressed "skip"
to confirm; boxes shown open read as fields to fill.

- Questions (stage panels, the strip over the message box): options are
  buttons with an arrow under "One press sends it to the AI" and show the
  pressed one at once; "Let the AI decide" moves to the question's top
  right (the panel title for ask_user); a line of one's own is folded as
  "None of these, I'll say it…". The Gates card folds its reason the same.
- A waiting task is one page: the sentence for the AI leads and sticks
  under the task header while the page scrolls (the workbench measures the
  header); "Say what to do" and the gallery are optional folds that say
  how many pictures are in. The sentence has the AI do what the task says,
  asking only when it says nothing. The Get started / Put ready tabs and
  their words are gone.
- Gates window texts fixed (canvas covers building, lines are over the
  message box, "Ask me" named as on the button); run_workflow waits up to
  55 s by default, so a quick run comes back with its pictures.
- The UI map counts words the backend names (label_key) as used.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 03:54:20 +08:00
DemonGatanjieuandClaude Opus 5.5 6443ddf81b Let the AI agree on the repaint area with the user first
The user's marks are rough. propose_repaint_area takes the AI's area for
them (boxes or ellipses, grown and softened by pixels), draws both areas
tinted over the picture into the task's marks folder and shows them side
by side on a panel (block "area"): the AI's, the user's own, or marking
again, or words on what to change. The picked area is written into the
input folder like the user's own mask (ai_marks.write_inpaint, soft edges
kept) before the answer is recorded, so the AI reads inpaint_image and the
inpaint note with it. Building the repaint stays the AI's: the note names
the usual nodes, no template. Options Anomalous makes carry a page word
(label_key). The panel hint and guide say "Your call".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 02:02:50 +08:00
DemonGatanjieuandClaude Opus 5.5 6dbb73591f Keep the AI waiting for the user's answer; daily tools reach the rest
- The daily tool set now also offers tool_help and use_tool, and the
  directory marks the tools reached that way; show_panel joins the daily
  set. The "lazy" set is gone: a small model spent a third of its calls
  looking tools up and never built what it planned.
- Gates window "Waiting for your answer" (1, 3 or 10 minutes): an open
  question tells the AI to wait on with get_answers until then, and while
  the user types (POST /anomalous/ai_tasks/{id}/typing) until a minute after
  the last key. report and show_panel remind the AI to end each round with
  ask_user; a question written in a note is pointed to ask_user.
- "Skip this one" is "Your call": the AI decides and does not ask again.
- ComfyUI's own `$$` widgets (the canvas image preview) are no longer
  reported to the AI as the user's canvas change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 01:49:34 +08:00
DemonGatanjieuandClaude Opus 5.5 fd952fb9ce Go on in the task the user wrote to, not a new one
When the user wrote to an older task while the AI had moved on, the AI's
next call (once its own task ended or paused) started a new task, where
report(reply_to=...) could not find the message. Now an app with no open
task goes back to the newest task of its own holding a reply it has not
read, and opens it again; an open task it is busy with stays. Naming the
calling app moves to api/ai_callers.py to keep ai_tasks.py under its size.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 00:30:22 +08:00
DemonGatanjieuandClaude Opus 5.5 ea9d446151 Open a finished task again when the user writes to it
A message or marks on a task the AI had ended stayed "done": no line to
copy for the AI showed, and the AI's next call started a new task where
report(reply_to=...) could not find the marks. Writing to the task now
clears `ended`, so the copy line shows and the AI goes on in it; an older
ended task with an unseen reply also offers the line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 23:49:12 +08:00
DemonGatanjieu f543eb92c8 Merge branch 'main' into feature/1.62 2026-10-25 22:19:28 +08:00
DemonGatanjieuandClaude Opus 5.5 3679f42095 README: re-shoot the combo pictures on 1.61 (right-click → AMB → Save as combo, new combo page)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 22:19:23 +08:00
DemonGatanjieu 39f5c2380c Merge branch 'feature/1.61' into feature/1.62 (v1.61.0)
# Conflicts:
#	CHANGELOG.md
#	web/modules/locales.js
2026-10-25 22:04:35 +08:00
DemonGatanjieu 13698c2389 Merge branch 'feature/1.61' into feature/1.62 2026-10-25 19:27:37 +08:00
DemonGatanjieu bc9d52c4f3 Merge remote-tracking branch 'origin/feature/1.61' into HEAD 2026-10-25 15:05:06 +08:00
DemonGatanjieuandClaude Opus 5.5 ce35145e67 Remove the 32 screen words no code uses any more
Old toolbar, scan-wizard and workspace labels and a few hints, in both languages; each
checked against the web files, tests and the backend (no t() use, no key built in a
template, no error message used as a key). The UI map's scanner read a quoted string with
an escape ('\n') wrong and missed the key after it on the same line (delNote is used).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 14:45:14 +08:00
DemonGatanjieuandClaude Opus 5.5 b220fc8a31 Call a saved workflow a workflow on screen, not a recipe
- The 54 screen words that said 配方 / recipe for a saved workflow say 工作流 / workflow
  (the Workflows page, saving and exporting, the activity log, backups, the gallery's keep);
  配方包 / recipe package stays the .zip given to others. README and the MCP guide too.
- The Workflows page's empty guide drops its fallback texts: translate() returns the key
  when a word is missing, so they were never shown and still said 配方.
- The glossary retires 配方 and recipe; a retired name may list where it is not that name
  ("配方(配方包、搭配方案除外)"), and {placeholders} do not count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 13:52:08 +08:00
DemonGatanjieuandClaude Opus 5.5 375ba6ce26 A UI map: from a word on screen to its code, styles, route and backend function
- tools/ui_map.mjs find <words on screen | key | /anomalous/route | file.js>, read from the code
  each time (tools/ui_map_scan.mjs: the words, what each web file shows and calls, its CSS
  classes, imports, the backend's routes); build writes the whole map to the untracked
  docs/ui-map.md, with the routes no page calls and the words no file uses.
- docs/architecture/ui-flows.md: the hand-written part for the AI workbench (what each part
  is for, what a click does, why) and a glossary of one name per thing. The structure check
  fails when a word in it is no longer on screen, a file or route it names is gone, or a
  retired name is back on screen.
- One name each on the AI page: 素材 (not 材料), 留言 (not 消息), 节点结构 (the / command
  too), 连接 for the last AI, 副本 for the task's tab, 撤销; English "Node map".
- AGENTS.md: look a button up with the map before changing it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 13:29:04 +08:00
DemonGatanjieuandClaude Opus 5.5 0de22f8f46 AI workbench: how to change a marked picture, a quicker task list, "AI edits a copy", lit source rows
- Marker: "make a repaint mask" becomes How to change it (leave it to the AI, only the
  marked areas, the whole picture anew); the AI reads the choice first as the user's, the
  timeline row says it, the repaint brush picks "only the marked areas".
- Task list: folded, it stays laid out out of sight and slides in at once from its button
  or the page's left edge; redraws make anew only the rows that changed (sync_rows.js).
  A picked task shows at once from the copy read before or read ahead on hover
  (ai_task_cache.js), then what changed; activity entries follow instead of holding it up.
  Stage pictures load as thumbnails first, the file itself only when shown larger.
- Header: "Own tab" is "AI edits a copy ▾", a two-way menu with what each means (kept open
  across redraws), and "Show the copy".
- Materials: the Source list lights the hovered row and ticks the one shown (a later rule
  had hidden both), tooltips only for cut names, arrows and Enter (ui_ai_facet_menu.js).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 13:08:46 +08:00
DemonGatanjieu ae1d3b3a17 feat(ai): show stage content count and unread indicator 2026-10-25 00:42:24 +08:00
DemonGatanjieu 3b0857f4f1 fix(ai): make jump to latest responsive and uninterrupted 2026-10-25 00:32:22 +08:00
DemonGatanjieu d54c0a0791 feat(ai): collect task outputs in a results side panel 2026-10-24 21:08:36 +08:00
DemonGatanjieuandClaude Opus 5.5 3092e201ec Make the ways to the right column easy to see
Each panel's line in the timeline becomes a card: an icon of what it holds,
a line of what is in it, and a solid "Show on the right" button that turns
into "On the right now" while the panel is in view on the stage. A run's
"See it large on the right" uses the same button, and the blue text links
under steps become small buttons with icons.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:22:46 +08:00
DemonGatanjieuandClaude Opus 5.5 b9e864cca1 Open the stage when the AI puts a new panel on it
A new panel opens the side column on the stage at that panel, even after
the user closed it or picked another view; the AI changing a panel does not.
A switch in the stage's head turns it off (kept on this browser). In a
narrow window it lights the stage's tab instead of taking the timeline's
place; a question still brings the stage forward.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:14:47 +08:00
DemonGatanjieuandClaude Opus 5.5 a33d838d7f Fold the AI's working steps in a row into one timeline row
Three or more reads, canvas changes and decisions in a row, two of them not
reads, become one row: how many steps, chips of what they did, the last one
while the AI is at it, and undo for their canvas changes. It opens to the
steps as before, and a jump to one of them opens it. Words, notes,
questions, panels, runs, errors and waiting or running steps stay rows of
their own. The folded rows move to ui_ai_chain.js.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:08:43 +08:00
DemonGatanjieuandClaude Opus 5.5 24a9eb9ed9 Make ask_user wait for the answer even when told not to
The AI's question waits at least 30 s, so an answer the user gives in
Anomalous right away reaches the same call and the AI goes on without a word
in its chat (Gemini asked with wait_seconds 0, stopped, and the user had to
say "go on"). add_node's description points to add_nodes for a group of
nodes wired to each other.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 17:16:10 +08:00
DemonGatanjieuandClaude Opus 5.5 72e0132c14 Keep lines for the AI's chat over the message box
When the AI is not in a call, a run or a question, the AI workbench offers a
line to copy into the AI's chat, picked by what waits for it: the run that
ended, a reply in Anomalous it has not got yet, else "Go on", and a line
that picks the task up in a new chat. The lines under a finished run's step
move there. The Hand to AI sentence no longer names show_panel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 17:01:45 +08:00
DemonGatanjieuandClaude Opus 5.5 59916d1ea3 Offer an on-demand tool set: a few tools, and the directory for the rest
A third choice for "Tools for the AI" in the Gates window. With "On demand",
AI apps are offered only action_catalog.CORE plus two tools of
mcp_directory.py:
- CORE: begin_task, get_situation, run_workflow, run_status, report, ask_user,
  get_answers.
- tool_help: the description and arguments of up to three tools named in the
  directory.
- use_tool: calls any tool of the directory.

tools/list drops to 11 tools and about 9,000 characters (daily 22,200, full
40,700). With this set the directory begin_task brings names every tool and
says how to reach the ones not listed.

use_tool has no run of its own. mcp_server._call makes the call the tool it
names before anything else, so the user's gates, the timeline's step, the run
watch and the answer are all that tool's; the timeline never shows use_tool.
An unknown name, the card's own tools or wrong arguments come back as tool
errors that point to tool_help.

The default stays Full. Switching back to Daily or Full in the Gates window
undoes this without a revert, and this commit stands alone if it has to go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 16:11:41 +08:00
DemonGatanjieuandClaude Opus 5.5 9f0133e1b7 Give the AI a directory of the tools by what it wants to do
In the user's real tasks the AI spent many calls finding its way: reading the
canvas, searching node types, looking at combos and the activity log. The
directory gives it what ARCHITECTURE.md gives the code: each tool offered,
under its stage of the work, with its title. The stages are run, canvas,
models, prompts, combos, pictures, voices, the library, and the task and the
user. Seeing it, the AI can go straight to the tool it needs.

The directory is built from action_catalog's stages and the tool specs, so a
new tool shows up by itself, and a test checks that every stage has a heading.
It lists only the tools offered: about 1,100 characters for the daily set and
2,000 for the full one. begin_task's answer carries it, and so does
get_situation's while there is no task. The server's instructions and
begin_task's note point to it.

retext, the helper that rewrites a result's text from its data, moves from
ai_task_runs to mcp_tools, so both modules share it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 16:04:53 +08:00
DemonGatanjieuandClaude Opus 5.5 f7d8a69e75 Shorten the daily tools' descriptions and begin_task's rules
The descriptions of the daily set are rewritten without the repeats and
roundabout phrasing. Their rules and don'ts stay: no cropping or editing
pictures with code, no reading ComfyUI's folders, ask the user in the
workbench. So do the shared property texts: the query words, node ids, the
task id. tools/list for the daily set drops from about 27,300 to 22,200
characters, and the full one from 46,000 to 40,700. begin_task's note, read at
the start of every task, drops from about 1,900 to 900 characters. The
replacements were made with ast, so each description's adjacent literals
changed as one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 15:54:42 +08:00
DemonGatanjieuandClaude Opus 5.5 6246df480d Let the user tell the AI when its run is done instead of the AI polling
Waiting on runs was the biggest share of the AI's calls in the user's real
tasks (85 of 316 run_status calls, about ten per ten-minute run). A new Gates
setting, "Waiting for runs", defaults to "Tell AI when done" (ai_gates.run_wait,
stored beside the tool set in ai_tools.json).

In that mode:
- run_workflow still waits its 15 s, so a short run comes back done with its
  pictures.
- Otherwise the answer's note, `next` and the your_run sentence tell the AI to
  say the run is going and end its turn, not to wait with run_status.
  run_status still honours wait_seconds, so a model that polls anyway is no
  worse off than before.

When the run ends before the AI knows, the server marks the step `call_ai`. The
workbench then shows lines to copy into the AI's chat (ui_ai_call.js): "It's
done, take a look", "Again with another seed", "Go on", or "It failed, find out
why". The mark goes as soon as the AI is told, or when a newer run starts, and
the lines go with it. The AI's next call brings the result and the pictures.
"AI waits" keeps the old behaviour. The setting takes effect at once, because
the words are made for each answer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 15:45:39 +08:00
DemonGatanjieuandClaude Opus 5.5 a9928969f5 Show a finished run's pictures in the answer that tells the AI it ended
In the user's real tasks, nearly every run ended with get_image_info, then
show_panel, then report: one extra call just to fetch the picture. The answer
that first tells the AI its run ended now carries up to four thumbnails of the
run's pictures (pictures_attached). That answer can be run_status, run_workflow's
own wait, or any other tool's answer.

It happens once per run. status() and news() mark the run when they first tell
its end, and show_pictures, called for every answer in mcp_server, attaches the
pictures to the marked run and clears the mark. Runs told of before this change
carry no mark, so they never bring old pictures along. The done `next` now says
to report with the pictures and to use a panel only to compare or ask. tell()
and show_pictures share _retext.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 15:26:41 +08:00
DemonGatanjieuandClaude Opus 5.5 11ad97cb1f Add a group of nodes to the open canvas in one call
In one real task the AI built an inpaint branch with 9 add_node and 16
connect_nodes calls. add_nodes takes the whole group in ComfyUI's API form, the
format models know best:
- links inside the group, and to canvas nodes as "#12", with outputs by slot
  or name;
- a node's wire_to feeds canvas nodes the way add_node does.

All of it is checked before anything changes. The page lays the group out as
one block in a free spot (canvas_build.placedBlock) and applies it in one
transaction: one Ctrl+Z step, and a failure takes every new node out again
(applyWires now takes several nodes). The answer maps the AI's keys to node ids
and lists what is spliced and what is still unwired.

A splice now uses only an input that nothing else in the plan feeds. A node
whose latent comes from the group, or from wire_from, replaces the sampler's
empty latent instead (replace:true) rather than being fed twice. It is in the
daily tool set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 15:20:20 +08:00
DemonGatanjieuandClaude Opus 5.5 9b4e565826 Set any node's settings by name instead of growing the list of blanks
run_workflow's blanks were a fixed list (prompt, seed, size, image). Anything
else meant a set_values call per node, or yet another blank. Now `values`, on
run_workflow and on set_values without a node, sets any setting by its name, a
third-party node's too.
- The one node that has the setting gets it.
- When several nodes have it (a hires fix's two samplers), nothing changes and
  the nodes are listed, so the AI names one: {"12": {"denoise": 0.5}}.
- One Ctrl+Z step for all of them.

The remaining blanks are roles (which prompt box, which latent), not setting
names. `prompt` now stops when several positive prompt boxes are combined into
one sampler (regional prompts) instead of giving them all one text. A refiner's
own prompt still gets it.

The scalar check moves from mcp_canvas_edit to mcp_fill (flat_values), which
both use; set_values keeps its argument order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 15:13:07 +08:00
DemonGatanjieuandClaude Opus 5.5 4fcceffa1b Keep a hires fix's latent upscale out of run_workflow's size blank
width and height went to every node that outputs a LATENT and has a width
and a height. LatentUpscale has both, so a hires-fix workflow had its upscale
set to the start size and stopped upscaling. The size now goes only to a node
that starts a latent, one with no LATENT input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 14:59:05 +08:00
DemonGatanjieuandClaude Opus 5.5 7bbea78b1d Splice a new node in front of wired inputs and pair same-name sockets
GPT's auto-wiring (4afee83) was checked on the isolated test ComfyUI. A splice behind
a node, a new branch, the branch choice and the loop check each took one call. A
ControlNet still failed both ways, because its positive and negative have the same
type as a sampler's. A LoRA after a checkpoint wired only its MODEL line.

- Same-type sockets of the same name pair up (positive to positive).
- wire_to into an input that is already fed puts the new node in front: the old
  source now feeds the new node's matching input. A ControlNet before a KSampler
  is 6 links in one call; a LoRA before the sampler and both prompts is 5.
- insert_after splices every type a node passes through.
- add_node errors call the new node #new instead of #-1 and say how to pick sockets.
- frontend.md: the canvas_wiring paragraph moves from Verification to the
  mutation rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 14:48:37 +08:00
DemonGatanjieu 4afee8326c feat(mcp): add typed auto-wiring and compact workflow context 2026-10-24 09:53:22 +08:00
DemonGatanjieuandClaude Opus 5.5 7b43403243 Tell the AI the next step in the answers it acts on
A small model plans poorly but follows a line well. begin_task,
run_workflow and run_status, add_node, connect_nodes and remove_nodes,
and an answered ask_user now carry next: the one thing to do now (wait
again, look at the pictures and report, wire what is still unwired, find
the missing model). The run's per-state note becomes that next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 03:59:14 +08:00
DemonGatanjieuandClaude Opus 5.5 ca60ba36a2 Offer AI apps a daily set of tools, for a small model's everyday runs
Every chat began with 51 tools and about 43,000 characters of their
words, which a small model like Gemini Flash read half of before it
lost its way. The Gates window now offers either every tool or a daily
set of 19 (action_catalog.DAILY: runs of saved workflows with their
blanks, pictures, models and LoRAs, missing models, the task's own),
about 20,000 characters. tools/list follows the choice (ai_gates
tool_set, anomalous/ai_tools.json); calls are not limited by it, and an
app sees a change in a new chat. The window's rows share one builder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 03:55:28 +08:00
DemonGatanjieuandClaude Opus 5.5 ba45d611d2 Let run_workflow fill in a saved workflow: a daily run in one call
A daily run took six or seven calls (find the workflow, open it, read
the canvas, set the prompt, the seed...), each a decision a small model
could get wrong. run_workflow now takes workflow (a saved one, opened in
the task's tab the first time and reused after: the page marks the tab
with the workflow's file) and blanks: prompt, negative, seed, width,
height and image (image_node when there are several LoadImage nodes),
planned by mcp_fill.js and checked before anything changes; the rest
stays as saved. A run with blanks asks as the stricter of the GPU's and
the canvas's gate. get_situation sends an AI whose run failed on a
missing model to check_workflow_models instead of other choices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 03:48:57 +08:00
DemonGatanjieuandClaude Opus 5.5 68342ad61d Give an AI where things stand in one call, and what is still unwired
A small model lost its way wiring the canvas: describe_canvas says what
feeds each input but not what is still free or what fits it, so it
looked up every node type and kept the graph in mind, and long answers
were filed away by Antigravity for the AI to open.

get_situation (mcp_situation.py) gives the task (what the user asked
last, what waits for them, decisions, pictures, steps outside AMB), the
canvas in brief (mcp_canvas_brief.js: a line per node, the inputs a run
needs that nothing feeds yet with the outputs that fit, ComfyUI's
errors), the latest run and the next step, kept under 3,600 characters.
add_node, connect_nodes and remove_nodes answer with what is still
unwired. The connect sentence, the rules and the instructions point a
new chat to get_situation first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 03:37:49 +08:00
DemonGatanjieuandClaude Opus 5.5 142074a320 Show what changed outside Anomalous while an AI works
Anomalous sees only the calls made to it; an AI app's own commands and
scripts are invisible, and one rewrote a result six times. When the AI
calls, Anomalous now looks at the top of the output and input folders
for pictures no ComfyUI run saved and pictures rewritten in place since
its previous call (by the files' own times, bounded, never in the
background; new pictures wait while a prompt runs), shows them as an
"Outside AMB" step and tells the AI with its answer (ai_outside.py).
log_outside is the AI's own word on a command or script it ran.

The connect guide gains rules to paste into the AI app's own rules:
use AMB's tools, change pictures only by running workflows, ask and
explain in AMB, and say what was done outside it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 03:16:28 +08:00
DemonGatanjieuandClaude Opus 5.5 80ed034a24 Size a step's pictures to themselves instead of the whole row
One picture in a step filled the timeline's width as a wide grey box
around a small image. The cards now keep to about 300 px each, from
the left.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 02:56:36 +08:00
DemonGatanjieuandClaude Opus 5.5 8ea8c08dd3 Grey a panel's old progress bar and centre a wide task column
A progress block on an AI panel is the AI's own number and never moves.
An AI drew one at 12% during a run, came back after the run with a new
panel and left the old bar standing, which read as live progress. Once
the task goes past a bar without an update (a run ends, a later panel or
note, the task ends), the stage now draws it grey and says when it was
written; update_panel records when a panel's blocks changed. The tool
text tells the AI a run's real progress shows already.

run_status before any run answers state "none" instead of failing, so a
task no longer shows a failed step for asking.

In a wide window the task column keeps its content to about 880 px in
its middle, as chat apps do, instead of stretching across the window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 02:47:40 +08:00
DemonGatanjieuandClaude Opus 5.5 09136eabb5 Open AI tasks at their newest step and keep the grip off the scrollbar
A task now opens at its newest step instead of where the previous task
was scrolled to, and a timeline read at its foot follows new steps
whatever the task's state (before, only while a step was running).
Read further up, it stays put; a round button over the message box
leads back down, with the count of the AI's new steps on its corner.
It replaces the "new steps below" pill (ui_ai_newest.js).

The side column's grip lay over the timeline's scrollbar, so grabbing
the scrollbar often resized the column instead. It now has a gap of its
own beside the scrollbar, drawn as three dots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 02:13:31 +08:00
DemonGatanjieuandClaude Opus 5.5 0930d07d4f Give the AI a closer look and a run's previews, so it need not script pictures
In one hour Gemini ran 63 shell commands next to 23 AMB calls: checking files, cropping and
comparing parts of pictures, running OpenPose itself to see a skeleton, and painting pixels
with OpenCV (it overwrote an output picture six times).

- api/mcp_look.py zoom_image: part of a picture in the output, input or temp folder shown
  larger, the same part of a second picture beside it; read only.
- get_image_info reads the temp folder too; run_status lists a run's PreviewImage pictures
  (ai_task_runs.outputs_of(record, "temp"), kept as the step's previews, not its results).
- begin_task and the repaint note: AMB's tools cover looking; change pictures only by running
  workflows, never by editing picture files with code of one's own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 01:21:27 +08:00
DemonGatanjieuandClaude Opus 5.5 c6d4ac9aa4 Let the AI read a picture in ComfyUI's input folder
The inpaint picture a repaint mark writes is in the input folder, which get_image_info did not
read, so the AI checked it with a shell command (and the user had to allow it). get_image_info
with folder=input gives such a picture's size and the area LoadImage gives as MASK (where and
how much), optionally attached; the marks message says the picture is in place already.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 23:00:51 +08:00
DemonGatanjieuandClaude Opus 5.5 cc0578f893 Keep tool results small enough to stay in the AI's chat
Antigravity saves a tool result over about 4,000 characters to a file and has Gemini open it;
in one evening 16 of its 32 file reads were AMB results (describe_canvas 9 times). Results are
now compact JSON (mcp_tools.tool_text, also for attached answers and run status), the canvas
leaves out the page's own "$$" widgets, and recent_activity gives the last 10 by default.
begin_task also tells the AI that AMB's tools cover models, pictures, nodes, canvas and runs,
so it need not read ComfyUI's folders, source or routes (each asks the user to allow it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 22:37:33 +08:00
DemonGatanjieuandClaude Opus 5.5 945934e0cc Move the run line over the message box with the run's progress
A run's progress comes in alone and only moved the timeline's bar and the header's button, so
the strip over the message box stayed at the run's start ("nodes 0/29, 0%"). The strip keeps
its context and the progress fast path updates it too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 22:12:17 +08:00
DemonGatanjieuandClaude Opus 5.5 d834306a7a Keep the task's head to one line in a low window
When the AI page is lower than SHORT_PX (a window not maximised), the task's title, state,
time and steps, the task it goes on from and the tools share one line, the room around them
shrinks, the message box's hint shows only for a failure (its words become the input's
tooltip) and the waiting page's picture is smaller: about a fifth more room for the timeline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 21:53:38 +08:00
DemonGatanjieuandClaude Opus 5.5 3227a75c73 Add "Change type" for a model, and check copies at start, middle and end
- ui_model_retype.js: on a model's page, and from the toast after importing one model: where it
  is, what its header suggests (GET /anomalous/placement/look; picked and marked when it can
  tell, nothing guessed when it cannot), every type with its loader, where it goes, moved as
  Tidy moves (covers and notes along, never over a file). Kinds and the move's error words are
  shared (model_import.IMPORT_KINDS, model_placement.moveErrorText).
- model_import.move_into: a copy to another drive is compared with the original at its start,
  middle and end, besides its size, before the original goes.
- model_placement: the model a request names is resolved in one place (_source), which also
  refuses a negative root.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 21:44:59 +08:00
DemonGatanjieuandClaude Opus 5.5 dae3632f1d Never replace a file that takes a model's name during a move
move_into renamed with os.replace, which on Windows replaces a file that appeared after
free_path chose the name. _rename_new refuses a taken name (Windows' rename does by itself;
elsewhere a hard link of the file, or of a link itself), so the move fails and both files stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 21:33:57 +08:00
DemonGatanjieuandClaude Opus 5.5 a74f885c53 Show what the task needs now over the message box; take dropped and pasted pictures
- ui_ai_now.js: over the message box, a run's progress with stop, the AI's step still going, a
  gate to allow or refuse there, and the AI's open question with its options as buttons (own
  words or skip too; several picks and pictures lead to the stage). progressWords is shared
  with the run card; the panel's and gate's answers are exported for it.
- ui_ai_drop.js: pictures dropped on the box or pasted are kept in output/ai_inbox
  (POST /anomalous/ai_pictures/upload, ai_pictures.save_upload: read as a picture first,
  25 MB cap, never over a file) and attached; ComfyUI output pictures dragged in attach as they are.
- Long AI notes are folded to their first lines with "Show all".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 21:24:48 +08:00
DemonGatanjieuandClaude Opus 5.5 ecbebc049d Dock the message box at the foot of the task like a chat composer
The message box stays at the bottom of the task column however the timeline scrolls. Over the
input, what the message carries (ui_ai_tray.js): pictures with ✎ to mark up (sent on their own)
and ×, chips for what "@" names; gallery pictures are attached, not typed in as "@name". Beside
it the gallery and materials buttons with their counts; the input grows (Enter sends,
Shift+Enter breaks the line) and pictures alone may be sent. "New steps below" appears when the
AI adds steps while the user reads further up; one's own message scrolls into view. Pictures a
message carries show as small image cards with the usual tools. The ready page keeps the big
gallery and what is ready; the box below takes its pictures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 21:11:10 +08:00
DemonGatanjieuandClaude Opus 5.5 6dc0a6bf98 Make picks, hovers and empty views give clear feedback
From a hover audit of the AI workbench and the other pages: picked gallery pictures get a thick
glowing frame, tint and big tick; the picture button lights while open and counts the message's
pictures; weak hovers (materials rows, folded reads with a chevron, gate options, mode switch,
source menu, send and copy buttons, model tabs, set-up button, activity rows) are clear. The
Stage button opens an empty stage that says what goes there; "Hand to AI · 0" is disabled with
nothing picked; move_model's step names the file; a one-node change shows one "find"; the task
link chip keeps its size.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 20:50:23 +08:00
DemonGatanjieuandClaude Opus 5.5 3ab1fc7c29 Give a waiting task a page of its own for putting things ready
A waiting task now switches between Get started and Put ready at the top. Put ready shows the
gallery as a big searchable grid feeding the note below (click to add or drop a picture), the
note (the message box, moved in), the materials button and what is ready so far. The gallery
grid is split out of the message box's picker so both use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 20:26:10 +08:00
DemonGatanjieuandClaude Opus 5.5 bcb2577c2c Let the user put pictures and materials ready for a waiting task
A task waiting for an AI gets a "Put things ready" step: the message box (gallery pictures,
"@") moves into it beside the materials button, and what the user put ready is drawn there;
the AI gets it all when it takes the task. Pictures a message names show as thumbnails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 20:18:01 +08:00
DemonGatanjieuandClaude Opus 5.5 c41af60cd9 Tell the AI what loads a model, file model patches right, add gallery pictures to messages
- model_kind: Anima LLLite, Qwen-Image block-wise and Z-Image Fun ControlNets (and other
  ModelPatchLoader files) go to model_patches, not controlnet; import and Tidy follow.
- api/mcp_model_use.py: search_models / get_model say which node types load each model, or
  where a misfiled one belongs; move_model moves it there behind a new "files" gate (asks by
  default, never overwrites).
- An empty drop-down in add_node says ComfyUI found no file; the task tab switch is no longer
  logged as the user's canvas change.
- The folded task list stays while the pointer is in the box around its button and the list.
- The message box's picture button picks gallery pictures; "@" offers the gallery's newest and
  searches it as you type.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 19:50:57 +08:00
DemonGatanjieuandClaude Opus 5.5 6e376e7caf Make the side column's grip easy to catch
The whole gap left of the card catches the pointer, top to bottom; the
handle shows faintly at rest and the edge lights up when pointed at.
Focused, the arrow keys resize it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 16:41:42 +08:00
DemonGatanjieuandClaude Opus 5.5 030e955332 Add a simple view of the run map
Per sampler: the model and LoRAs feeding it, its prompts, where it starts
(an empty canvas, a picture, or the part before, upscaled) and where it is
saved, "same as part N" when nothing changed; every name finds its node.
Simple (the default) and Full switch in place, kept on this browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 16:34:43 +08:00
DemonGatanjieu 47bab22373 Fix task canvas ownership and exact message references 2026-10-23 16:15:12 +08:00
DemonGatanjieuandClaude Opus 5.5 ac2aa471e9 AI workbench: pictures deleted since say so
A task's pictures no longer in the output folder are marked gone when the
task is read and shown as "Picture deleted" (timeline, results strip,
stage panels and picks) instead of a broken picture; "@" no longer offers
them. The AI hears plainly that the user deleted one, and a tool reading a
gone picture says it may have been deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:56:44 +08:00
DemonGatanjieuandClaude Opus 5.5 572caf9cb1 Tasks remember the task they go on from
A new task an AI begins while on another, or one the user starts while
looking at a task, keeps it as its "from". Links under the title lead both
ways, the list marks follow-ups, the user can link one by hand or take the
link off (never a loop), and an AI beginning a follow-up hears what was
decided in the task before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:46:20 +08:00
DemonGatanjieuandClaude Opus 5.5 bcb80bd7d3 Key decisions stay in Anomalous
The AI records what was settled with log_decision (a card on the timeline:
what, why, what was left out, who decided, the steps it is about); the
user's own choices in Anomalous count as decisions too (panel answers,
gates, materials, the canvas mode, stops). A Decisions view in the side
column lists them all, links each to its step and notes work done since
the AI's last one; begin_task repeats them to an AI picking the task up.
A step's words move to ui_ai_step_body.js.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:39:07 +08:00
DemonGatanjieu 036817bf48 Merge remote-tracking branch 'origin/feature/1.62' into feature/1.62 2026-10-23 15:18:08 +08:00
DemonGatanjieuandClaude Opus 5.5 7c8c3f27ed AI workbench message box: @ to name things, / for commands
"@" lists the task's pictures, combos, models, workflows, prompts, node
packs and node types; what is named rides with the message and the AI gets
each in full (pictures attached with what made them). "/" lists commands
that act at once (stop, undo, side views, new task, canvas mode) or fill in
words to send. get_toolbox shares the lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:17:56 +08:00
Claude 3d41489f45 Merge branch 'feature/1.61' into feature/1.62 2026-10-22 21:46:41 +00:00
Claude 641ea9f93d Merge branch 'feature/1.61' into feature/1.62 2026-10-22 21:43:08 +00:00
DemonGatanjieuandClaude Opus 5.5 eb3c6fe48d Materials side panel: sources as a button and a searchable list
Instead of a strip to scroll sideways, one "Source: All" button opens the
list of sources (or types), with a search when it is long; a pick, Esc or a
click elsewhere closes it, Esc without closing the browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 03:14:59 +08:00
DemonGatanjieuandClaude Opus 5.5 68ec277b79 Materials: hand the AI a whole custom node pack
A pack shown in the nodes' second column has a bar to pick it whole
(packs, by folder). The AI gets its node types in get_toolbox and finds
them first; with "only" every node of the pack is allowed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 03:05:17 +08:00
DemonGatanjieuandClaude Opus 5.5 5410ff797c Materials picker in the side panel: one line per item, search and filters on one line
A short panel lists many: badge or small cover, name, where it is from and
the tick per line; the filters become icons beside the search. Switching
between the side panel and the window redraws as lines or cards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 02:57:18 +08:00
DemonGatanjieuandClaude Opus 5.5 0e4a60518d AI page scrollbars: thin, shown only while the pointer is over what scrolls
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 02:49:00 +08:00
DemonGatanjieuandClaude Opus 5.5 8ecd23a103 AI workbench: the side column as a resizable card, the task list folding from the title bar
The side column is a card of its own, narrower by default, its width set
by dragging its left edge (kept; double-click resets; the timeline keeps
its room). The task list folds from a title-bar button and peeks over the
page while pointed at. In the docked materials picker a card's detail opens
from its info button, not on hover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 02:23:55 +08:00
DemonGatanjieuandClaude Opus 5.5 d237f93b31 One view at a time in the AI workbench's side column, switched from the title bar
The stage, a run's node map at full size (pick the run; each run step has
"see it large") and the materials picker share the side column; the switch
sits in the window's title bar on the AI page, the shown one again hides the
column, and each view keeps its state. The task header stays on top while
the timeline scrolls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 01:28:11 +08:00
DemonGatanjieuandClaude Opus 5.5 4dcda6edf8 Show whether the AI has each answer, and let a question be skipped
Every answer, message, set of marks and handed picture shows a pill:
sent and waiting, then the AI has it with the time it did (delivered_at).
Panel questions can be skipped; the AI reads skipped: true.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 01:13:03 +08:00
DemonGatanjieuandClaude Opus 5.5 9abd596186 What the AI ran, a canvas tab of the task's own, and pictures straight to the AI
- Each run keeps its shape read from the prompt ComfyUI queued (ai_run_map.py): boxes by role
  with their settings, typed links, plumbing wired through, and what changed since the task's
  run before. The run step shows a one-line summary, the changes as chips and the boxes (top
  down, fitted to the column, each locating its node); show_run_map puts it on the stage.
- By default the AI works in a tab of the task's own (ai_task_canvas.py, mcp_task_tab.js): the
  open workflow is copied before its first change, the tab is brought back (a click on
  ComfyUI's tab) before later changes, reads and runs, and the user's tab stays untouched.
  Per-task switch in the header, default in the Gates window; the AI is told once.
- Output pictures can be handed to the task an AI is on (gallery card menu, the inspector's
  ✦), with the user's words: the AI gets them attached with what made them (ai_pictures.py).
  search_images(with_pictures) attaches the first six.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:56:01 +08:00
DemonGatanjieuandClaude Opus 5.5 049b265126 Run control: stop a run, see its console, time left, VRAM and size warnings; materials as a side panel
- A stop button on a running run (timeline, and beside the run-status button), and stop_run for
  the AI: only this run's prompts leave the queue or are interrupted; the step ends "cancelled"
  and a user's stop reaches the AI as a "stop" message (ai_run_control.py).
- The run's progress carries ComfyUI's console since it began (the bar's note, models loading),
  time left and per step, and VRAM; once queued, each sampler's working size is estimated from
  the prompt, with warnings for an upscaled picture sampled again, a size past the card and a
  huge decode, for the user and in the AI's run news (ai_run_watch.py, ui_ai_run_live.js).
- The materials picker opens as a side panel that leaves the page usable (the full-screen
  browser makes room), expands to the large window and back, and remembers the choice.
- Esc closes the run-status card only (data-anomalous-escape).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:23:08 +08:00
DemonGatanjieu b60ab63833 AI workbench: make status button icon-sized and clamp fixed popover when stage is open 2026-10-22 23:02:16 +08:00
DemonGatanjieu 0259b98af0 AI workbench: replace tall sticky execution banner with compact toolbar button and popover 2026-10-22 22:53:44 +08:00
DemonGatanjieu 183d6b7158 AI workbench: confirmed execution status header, bounded run_workflow wait, and track_run 2026-10-22 22:26:35 +08:00
DemonGatanjieuandClaude Opus 5.5 bb744a8c38 Repaint an area: the marker's brush makes an inpaint mask for the AI
The marker gains a repaint brush (size adjustable) and "make a repaint
mask": the page sends a mask (white where boxes, circles, lines and brush
strokes are) and the server writes the picture with that area as its
alpha to ComfyUI's input folder (anomalous_inpaint_<step>.png, a new file
each time), so LoadImage gives the picture and the area as MASK. The AI's
marks message carries the file and how to inpaint with it; the timeline
says the mask is ready. Canvas tools read ComfyUI's lists anew once when a
value is not among a drop-down's choices, so they take the new file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 21:27:07 +08:00
DemonGatanjieuandClaude Opus 5.5 353b5d9ed5 The AI replies in Anomalous: report(reply_to) and laid-out notes
report takes reply_to, the id of the user's message or marks it answers:
the two steps point at each other and the timeline links them both ways
("Reply to step 2" / "The AI replied · step 4"). The AI's notes are laid
out (headings, lists, bold, code) by ai_text_format.js with textContent
only. The notice that brings the user's marks and messages, the report
description and the server instructions tell the AI to answer in
Anomalous, and to ask which fix with ask_user.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 21:15:43 +08:00
DemonGatanjieuandClaude Opus 5.5 52bd4cbce2 The AI asks in Anomalous: ask_user, with a suggested option and a line of the user's own
ask_user shows a question in the AI workbench (options with an optional
recommended one, and unless other=false a line for an answer of the
user's own; no options: a line) and waits in the same call for the
answer, returning the choice or the text. Choices blocks gain
recommended and other; the step is named ask_user ("Asked you"). The
server instructions, begin_task and the workbench prompt tell the AI to
ask every question this way, never in its own chat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 20:51:03 +08:00
DemonGatanjieuandClaude Opus 5.5 355f95af48 AI workbench: a new look for the task list, the task header and a waiting task
The task column (now ui_ai_task_list.js) leads with a New task button, icon
buttons for Connect an AI and the gates, which AI came last, quick filters
by state with counts, and state icons; the current task has an accent bar.
A task's header uses icon buttons (Materials, Undo all, show the stage,
rename and remove as pencil and bin). A waiting task shows a picture, three
numbered steps joined by a line (what to do, the sentence with a large
Copy button, a faint preview of steps) and a connect card that opens the
guide in place; the empty page matches. One set of line icons for the AI
pages (ui_ai_icons.js). Illustrations made with ChatGPT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 20:37:57 +08:00
DemonGatanjieuandClaude Opus 5.5 f42a1f795e Materials window: ChatGPT illustrations for the empty selection and missing covers
The empty pane shows a toolbox with cards rising out of it, and a combo,
workflow or model without a cover shows a stand-in illustration of its
kind, all transparent WebP of a few dozen KB in web/assets (made with
ChatGPT for this window). The hand-drawn SVG picture they replace is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 20:20:29 +08:00
DemonGatanjieuandClaude Opus 5.5 fd9e415123 Materials window, redone: cards, sources, details, and no waiting
A large window: one search over every kind (/ jumps to it), kinds on the
left with picks and matches, a column by source (each custom node pack
with a badge of its own colour, ComfyUI's own, paid API nodes), prompt
origin or model type, a card grid drawn as it scrolls, the card in focus
in full on the right, and the picks below with thumbnails and a prefer /
only switch. A node card is the node in miniature, sockets in ComfyUI's
colours; combos, workflows and models show their covers; prompts their
text, including the prompts inside combos and workflows (combo:<name>,
workflow:<name>), which get_toolbox now resolves. Vector icons, badges
and the empty picture follow the theme.

It opens at once: node types come from the page's own definitions, the
rest in one answer (api/ai_toolbox_catalog.py) read ahead when a task
header is drawn and kept a minute, refreshed behind an open window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:46:31 +08:00
DemonGatanjieuandClaude Opus 5.5 622ae8be9a Materials for the AI: pick nodes, combos, workflows, prompts and models
A Materials button on each AI task opens a picker: every node type of this
ComfyUI grouped by source (built-in, each custom node pack from
/object_info's python_module, page-only helpers, paid API nodes last), plus
combos, saved workflows, prompts and models by the names the AI's tools
use. Picks go to the AI as a user_toolbox step (message kind toolbox), in
begin_task and in full from get_toolbox; "only" makes add_node and
new_workflow refuse other custom or paid node types (built-in stay
allowed). Favourites stand in for tasks without picks. find_nodes lists
picks first and marks paid API nodes. open_workflow opens a saved
workflow in a new tab; a workflow's page can hand it to the AI.

Checked on the isolated ComfyUI with copies of the user's 10 node packs:
all 1642 node types can be placed, and a Comfyroll workflow ran.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:19:58 +08:00
DemonGatanjieuandClaude Opus 5.5 854e63ba71 The AI builds workflows on the canvas the user sees
find_nodes looks up this ComfyUI's node types (inputs, outputs, choices)
from its own /object_info. new_workflow opens a new tab, blank or built and
laid out from a workflow in API form, never touching the open one;
add_node, connect_nodes, set_values and remove_nodes edit it. Everything is
checked against the nodes before the canvas changes (types, loops, settings,
choices with close matches); each is one Ctrl+Z step, logged as the AI's
and shown under its workbench step. Receipts now show rewiring ("wired":
what feeds an input, before and after). A run ComfyUI refuses reports
ComfyUI's own reason instead of "Prompt execution failed".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:56:23 +08:00
DemonGatanjieuandClaude Opus 5.5 cce04d9ec1 Styles: reload the AI flow sheet for the run percent
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:42:06 +08:00
DemonGatanjieuandClaude Opus 5.5 70234d293d Run percent, and the run's news in the AI's other answers
The bar under a run shows a percent like ComfyUI's tab title (the running
node's steps, else the nodes done). While a run goes, every other tool answer
carries your_run with one sentence on where it stands, and the next answer
after it ends says so once (run_status telling it counts), so the AI need not
look in the output folder or at the screen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:41:46 +08:00
DemonGatanjieu 249eac2dee fix(ai): track workflow results by exact submission IDs 2026-10-22 17:39:02 +08:00
DemonGatanjieu 6157585e3a Fix AI approval consumption, task input routing and run monitoring failures 2026-10-22 16:40:47 +08:00
DemonGatanjieuandClaude Opus 5.5 84d66bf932 Mark a picture for the AI: boxes, circles, arrows and lines, each with a note
- ui_ai_marker.js: ✎ on a picture of the AI page opens the marker (box, circle, arrow, pen; three
  colours; a note per mark and one for the whole picture). Send draws the marks onto the picture
  and hands both to the task; the timeline shows the marked picture and the notes.
- api/ai_marks.py: the PNG is checked (signature, size, dimensions) and kept with the task in
  anomalous/ai_marks/ (never the output folder; removed with the task); a user_marks step the AI
  gets as a message of kind "marks", where each mark is in fractions of the picture, with the
  marked picture attached as an image to the same answer (also from get_answers, whose tool now
  returns the answer and those pictures).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 16:08:52 +08:00
DemonGatanjieuandClaude Opus 5.5 57e70dee48 Run progress for the AI and the workbench: run_status, and a live bar under the run
- ai_task_runs.py reads where a run stands from ComfyUI's queue and progress registry (never
  changing them): queued and how many are ahead, or the node running, its steps and the nodes
  done. It is kept on the step in memory and sent as "progress" events that only move the
  step's bar; the run's step is saved only when something new is kept.
- run_status (read-only): the task's latest run and its progress, its pictures once done;
  wait_seconds (<= 55) waits for the end. run_workflow's answer tells the AI to use it instead
  of guessing from the time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 15:59:41 +08:00
DemonGatanjieuandClaude Opus 5.5 adcf5a9d15 Tests: a model note is cut only past the receipt's new 2000-character limit
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 15:51:56 +08:00
DemonGatanjieuandClaude Opus 5.5 e57858c93b Canvas receipts: a prompt change shows the tags taken out and put in, with their meanings
- ui_canvas_changes.js: a prompt written as tags lists the tags taken out (red, struck) and put
  in (green) first; "Show meanings" puts each one's translation under it (prompt_gloss.js);
  "Whole text" marks those tags in both texts and dims the rest. Other text marks the part that
  differs. A tag the log cut in half at either end is never called changed.
- activity_log.py: before/after values keep up to 2000 characters (was 400), room for a whole
  prompt; longer ones are still cut around the first difference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 15:47:44 +08:00
DemonGatanjieuandClaude Opus 5.5 4f09154a49 AI workbench: a waiting task asks what you want and gives one sentence to send
The card of a task waiting for an AI (ui_ai_connect.js waitingCard) is centred: a line for
what the AI should do (it renames the task), the sentence to send the AI with Copy, three small
steps, and the first-time connection folded under "AI not connected yet?". No message box and no
timeline line while nobody has taken the task.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 15:35:21 +08:00
DemonGatanjieuandClaude Opus 5.5 1dadd5b0f0 Hand to AI: a model, an image or a combo starts a task that carries it
- ui_ai_handoff.js: "Hand to AI" on a model's page, the image inspector (✦) and combo cards.
  The user says what they want (prefilled), Anomalous starts a waiting task with `about` (the
  model_id, the output image, the combo's name; checked in ai_tasks.start_task) and shows the
  sentence to send the AI; begin_task hands `about` to the AI so it needs no search.
- The AI windows sit above the image inspector they can open from; going to the AI page from
  there closes the inspector.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 15:05:35 +08:00
DemonGatanjieuandClaude Opus 5.5 ccbd0a28cf MCP: Anomalous's card in the AI app's conversation, and ready-made prompts
- api/mcp_apps.py + api/mcp_app/task_card.html (MCP Apps, io.modelcontextprotocol/ui): apps that
  support it (Claude, ChatGPT, VS Code…) show Anomalous's own card under begin_task, show_panel,
  report and run_workflow results: the task's steps, latest thumbnails (inline), the AI's open
  questions answered with a click (app_answer, plus a chat message so the AI goes on). The card
  reads through app_task_view; both tools are for the card only (visibility "app"), are never
  steps and never take the user's news. The card never answers a gate: an OK is only given in
  Anomalous, so no tool an AI can reach grants one.
- api/mcp_prompts.py: prompts/list and prompts/get, four ready-made requests (slash commands):
  workbench_task, pick_best, fix_missing_models, explain_canvas.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 14:55:48 +08:00
DemonGatanjieuandClaude Opus 5.5 cd9751ef14 AI workbench: the flow over a task, and gates that wait for your OK
- api/ai_gates.py: changing the canvas, using the GPU, downloading and scanning online can each
  go ahead, ask first or never (downloads ask by default; anomalous/ai_gates.json). A call that
  asks waits on its step (state "asking", what it would do listed) up to 50 s for Allow or No
  (with the user's reason for the AI); unanswered, the AI is told to wait and call again, the
  later answer reaches it as a message, and the same call after an Allow goes ahead.
- ui_ai_gate.js: the card under a waiting step and the Gates window; a toast while elsewhere.
- ui_ai_flow.js + ai_task_view.flowStages: the task's stages in order, each done, running,
  waiting or failed; a stage leads to its first step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 14:48:43 +08:00
DemonGatanjieuandClaude Opus 5.5 00958578fb AI workbench: one action catalog; your own changes join the AI's task; tick two pictures to compare
- api/action_catalog.py: every MCP tool's kind, stage of the work and gate in one place (a test
  keeps it and the tools in step); steps now carry their stage.
- api/ai_task_changes.py: changes the user makes in Anomalous while an AI works become steps of
  its task, reach the AI once with its next call (kind "change": keep it), and the activity
  entry names the task; the activity page links to it ("AI task ->").
- api/ai_task_runs.py: run watching moved out of ai_tasks.py (600-line rule); a step a restart
  left "running" is marked stopped when the record is read again.
- Pictures: inspector / compare with canvas / keep as tools on hover instead of three buttons
  under every picture; tick any two (any step or task) and compare from the bar at the foot of
  the page; the results strip shows thumbnails again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 14:38:37 +08:00
DemonGatanjieu 9b1d857e12 Integrate AI task results with gallery comparison and canvas receipts 2026-10-22 13:27:16 +08:00
DemonGatanjieu 97259ff8af AI workbench: record in-flight MCP steps and piggyback pending user answers/messages 2026-10-21 23:01:45 +08:00
DemonGatanjieu c924772344 Preserve chronological content in AI panel updates 2026-10-21 22:09:25 +08:00
DemonGatanjieu f694b40ab0 Add close and reopen controls for the AI stage 2026-10-21 21:57:01 +08:00
DemonGatanjieu 5c7f4b9cad Fix AI task handoff and stage bottom following 2026-10-21 19:36:40 +08:00
DemonGatanjieuandClaude Opus 5.5 6a256b7018 AI workbench: New task, Connect an AI, and the stage read top to bottom
New task: the user starts a task (POST /anomalous/ai_tasks) that waits for
an AI; the next app that calls takes it over, so its work starts there
instead of joining its last task. Connect an AI (ui_ai_connect.js): the
address, a sentence to send the AI so it adds the server itself and
starts on a waiting task, and each common app's way; beside the task list,
on the empty page and in a waiting task. The stage now lists the task's
panels oldest first with their step numbers, the newest at the bottom (a
new one scrolls there, or "New panel" says it came), and the strip of
results runs oldest to newest. The routes move to ai_tasks_routes.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 02:58:05 +08:00
DemonGatanjieuandClaude Opus 5.5 af627bff9d AI workbench: the stage, a column on the right for the AI's panels
Panels are shown on the stage beside the timeline (ui_ai_side.js), so they
stay in view however many steps follow; the timeline keeps one line per
panel that brings it there. The stage follows the newest panel until the
user goes back to another (pager or a panel's line). Too narrow for three
columns, the timeline and the stage become two tabs; a panel that asks
brings the stage forward. The task list can be folded away. What a step
made moves to ui_ai_results.js; the toast goes once the workbench opens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 00:21:49 +08:00
DemonGatanjieuandClaude Opus 5.5 b9c8685b40 AI workbench: redraw only the steps that changed; diagrams read in order
While the AI works the task's parts stay in the page and a step is drawn
again only when it changed, so a diagram or strip scrolled sideways no
longer jumps back and half-typed text in a panel or the message box stays.
Diagrams put a box that only feeds one further on right before it and sort
boxes in a column by what feeds them. The message box and the home list
move to ui_ai_message.js and ui_ai_recent.js (the workbench was over 600
lines).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 00:06:51 +08:00
DemonGatanjieuandClaude Opus 5.5 8fe9106e7b AI workbench: write to the AI from a task; tasks grouped by day
A box under a task's timeline sends the user's message (POST
/anomalous/ai_tasks/{id}/message), kept as a 'user' step; get_answers now
also brings the user's messages and, with no panel named, waits for one.
The step says whether the AI has seen it; when the task has ended the box
says to ask in the AI app. The task list is grouped by day with an icon
for where each task stands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 23:23:39 +08:00
DemonGatanjieuandClaude Opus 5.5 5ae61fb0ee AI workbench: steps numbered in order down the timeline
Each step's dot carries its number (1, 2, 3...) in its kind's colour, reads
folded into one numbered row, and while the AI works the next number
waits at the foot, so the order is plain at a glance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 23:05:50 +08:00
DemonGatanjieuandClaude Opus 5.5 f57a78e08c AI workbench: the stage - marks on pictures, before / after, picks, diagrams
New panel blocks, data only and drawn by Anomalous: marks on pictures
(circles or boxes at fractions of the picture, numbered, labels under it
that zoom there), a before / after slider, picks (pictures with the AI's
reason, Take this one as an answer, Put on the canvas through ComfyUI's
own file loading, drag onto the canvas) and diagrams (boxes and arrows laid
out by ai_stage_layout.js; a box naming a canvas node finds it). No SVG
or drawing code of the AI's is taken.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 22:54:04 +08:00
DemonGatanjieuandClaude Opus 5.5 0999e6f9c4 AI workbench: panels the AI draws from Anomalous's blocks, and asks you on
show_panel lays out a panel as a step of the AI's task from fixed blocks
(heading, text, pictures in a row, grid or side by side, the user's models
as cards with covers, table, list, params, progress, divider) and can ask:
buttons (one or several) and a line of text. api/ai_panels.py checks every
block (only these kinds, bounded plain text, pictures only from the output
folder, models only from the user's folders); nothing is run or read as
markup, and ui_ai_panel.js writes text only. Answers come in through
POST /anomalous/ai_tasks/{id}/answer and wait for get_answers (which can
wait up to 55 s); update_panel changes or closes a panel. A toast says
when a panel comes while the browser is elsewhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 22:35:38 +08:00
DemonGatanjieuandClaude Opus 5.5 8b549f463a AI workbench: every step an AI takes through MCP, kept, opened, undone
A new rail page keeps each MCP call as a step of a task (api/ai_tasks.py,
one file per task in the user folder, the newest 300): named by the AI
with the new begin_task tool, else one app's calls without a long pause.
A step keeps what it did and what it made (models, the combo, the canvas
change by its activity entry, the pictures a run made, watched in the
history until it ends, spoken audio, scans); the new report tool leaves
the AI's notes and pictures it made another way. The page shows the tasks
and one task's results strip and timeline with links (picture viewer,
model page, combo, find or undo on the canvas, audio), updates live, and
gives the rail a dot while you are elsewhere; Home lists the last tasks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 21:19:49 +08:00
232 changed files with 30158 additions and 454 deletions
+1 -1
View File
@@ -29,7 +29,7 @@
1. 查看 `git status` 和相关差异,保留已有工作;不覆盖、回退或提交他人的无关修改。
2. 阅读 [ARCHITECTURE.md](ARCHITECTURE.md),再按阅读地图打开与任务有关的专题,避免每次加载全部文档。
3. 搜索目标函数的调用方、相关状态、CSS 选择器及已有辅助函数,再确定修改位置。
3. 搜索目标函数的调用方、相关状态、CSS 选择器及已有辅助函数,再确定修改位置。界面上的按钮或文字,先用 `node tools/ui_map.mjs find <屏幕上的字>` 查它在哪个文件、样式在哪、调哪个接口、后端哪个函数;每块界面的用途和统一叫法见 [ui-flows.md](docs/architecture/ui-flows.md)。改了界面文字或流程,顺手更新那里(结构检查会指出对不上的地方)。
4. 简短说明本次行为变化和负责它的模块。一个修复优先落在能解释问题的最小责任范围内。
UX 请求默认允许调整布局、交互和文案,不代表允许改变文件存储位置、接口协议、工作流语义、模型识别依据或用户数据。
+338 -10
View File
@@ -17,6 +17,7 @@ and Current node lists the saved values. See the material contract below.
| When changing... | Read... |
| --- | --- |
| A button, a page or words on screen: where it is drawn, what it calls, which backend function answers | `node tools/ui_map.mjs find <the words on screen>` (read from the code), then [`docs/architecture/ui-flows.md`](docs/architecture/ui-flows.md) (what each part is for, why, one name per thing) |
| Python routes, storage, paths, metadata, covers, or scan state | [`docs/architecture/backend.md`](docs/architecture/backend.md) |
| Browser lifecycle, UI state, localization, media, or graph edits | [`docs/architecture/frontend.md`](docs/architecture/frontend.md) |
| Update-guide content/versioning or sidebar hover labels | [`docs/architecture/update-guide.md`](docs/architecture/update-guide.md) |
@@ -24,6 +25,7 @@ and Current node lists the saved values. See the material contract below.
| Saved prompts and node values (material files), image parameter details, keeping an output image | [`docs/architecture/material-library.md`](docs/architecture/material-library.md) |
| Model Check, provenance hashes, missing-model recovery, or model scanning | [`docs/architecture/model-resolution.md`](docs/architecture/model-resolution.md) |
| Why a current product boundary exists | [`docs/decisions/README.md`](docs/decisions/README.md) |
| The MCP tools AI apps get: adding one, its description or arguments | [`docs/architecture/tool-exam.md`](docs/architecture/tool-exam.md) (the tool budget, `tests/test_tool_budget.py`; the exam tasks and their scorecard, `node tools/tool_exam.mjs`) |
| Recurring implementation mistakes and post-mortems | [`docs/architecture/lessons.md`](docs/architecture/lessons.md) |
`README.md` is user-facing documentation and `CHANGELOG.md` is user-facing
@@ -69,6 +71,34 @@ covered by the `styles.css` manifest.
### Backend
- `api/ai_run_tracking.py` attaches exact external prompt IDs to one task, prevents duplicate
ownership, and returns bounded-wait execution receipts to the AI.
- `api/ai_run_watch.py` reads what a run shows beyond its node and steps: ComfyUI's console
since it began (`app.logger`'s buffer: the bar with its note, the models loading), the time
per step and left, the card's memory, and once per run the size each sampler works at from
the queued prompt, with warnings (an upscaled picture sampled again, a size past the card,
a huge decode). `api/ai_run_control.py` stops a run (`stop_run`, the workbench's stop
button): only its own prompts leave the queue or are interrupted; the step ends `cancelled`
and a user's stop reaches the AI as a `stop` message. `tests/test_ai_run_watch.py`.
- `api/ai_run_map.py` reads a run's shape from the prompt ComfyUI queued (plumbing wired
through; boxes with a role and facts, links with ComfyUI's types) and what changed since the
task's run before; `ai_task_runs.py` keeps both on the run step, `show_run_map` puts it on the
stage (a `run_map` block), `web/modules/ui_ai_run_map.js` words and draws it (Simple: told by
`web/modules/ui_ai_run_story.js`, one part per sampler with the model and LoRAs feeding it, its
prompts, where it starts and where it ends up; Full: top down, shrunk
to its column). `api/ai_task_canvas.py` keeps each task's own canvas tab (default "new", set in
the Gates window or per task): `mcp_bridge.ask_page` gives canvas actions the task's tab,
`web/modules/mcp_task_tab.js` brings it to the front by clicking ComfyUI's tab (the store
alone would not load its graph) or copies the open workflow into a new one before the first
change; `web/modules/ui_ai_task_canvas.js` is the header's chip saying which ("AI edits a
copy ▾", opening the two to pick from with what each means) and the copy's "Show the copy".
`ai_pictures.mark_gone` marks a task's pictures no longer in the output folder (`gone`) on its
`GET`, which `ui_ai_images.gonePicture` shows instead of a broken picture.
`api/ai_pictures.py` hands the AI output pictures (gallery card menu, the inspector's ✦): a
`user_pictures` step, delivered as a `pictures` message with the pictures attached and
what made each (read with ai_run_map); `search_images(with_pictures)` attaches the first
six. `tests/test_ai_run_map.py`.
- `api/config.py` owns configured paths and active model-folder types.
- `api/path_utils.py` owns containment, filename validation, atomic JSON writes, and resolving
a saved model value inside the model folders;
@@ -132,8 +162,298 @@ covered by the `styles.css` manifest.
canvas actions done by the open page, scans and Anomalous TTS speech through this
server's own routes. `api/mcp_bridge.py` hands a canvas action to the page over
ComfyUI's websocket and waits for its answer; `web/modules/mcp_bridge.js` is the page
side, doing it with Anomalous's canvas code (one Ctrl+Z step each, logged as the AI's).
side, doing it with Anomalous's canvas code (one Ctrl+Z step each, logged as the AI's; the
outcome carries the log entry's id as `_activity`, for the AI workbench's undo).
`api/mcp_look.py` is zoom_image, the AI's closer look: part of a picture in the output, input
or temp folder shown larger, the same part of a second picture beside it, read only (so an AI
need not crop or compare pictures with code of its own).
`api/mcp_model_use.py` offers search_models and get_model (mcp_tools.py finds the files)
with what loads each model: the node types whose drop-down lists it (from `/object_info`),
or, for a file whose header says it belongs in another folder (model_placement.py), where,
and `move_model` (behind the "files" gate) moves it there as Tidy does.
`api/model_advice.py` is what Anomalous knows about using a model well: get_model's
`author_examples` (sampler, steps, CFG, size and a LoRA's weight from the example pictures in
the Civitai record a scan keeps), and add_lora / set_model's `base_model_note` when a LoRA and
the main model on its line (`graph_splice.js` loraChain / modelLineStart, reported by the page)
are for different base models. `tests/test_model_advice.py`.
`api/mcp_canvas_edit.py` holds the tools that build a workflow where the user watches:
`find_nodes` reads this ComfyUI's node types from its own `/object_info` (kept two
minutes); `add_node`, `connect_nodes`, `set_values`, `remove_nodes` and `new_workflow` (a
new tab, blank or built from a workflow in ComfyUI's API form, never the open one) are
checked for shape there and done by `web/modules/mcp_canvas_edit.js` through the bridge's
`recorded`, after checking types, settings and choices against the nodes themselves;
`web/modules/canvas_build.js` is its pure part (type fit, value checks, a free spot, the
layered layout). `open_workflow` opens one of the user's saved workflows in a new tab.
`web/modules/canvas_wiring.js` plans unique concrete-type connections, selected-branch insertion,
splices, the wires of a group added in API form (add_nodes; `canvas_build.placedBlock` lays it
out as one block), conflict/cycle checks and rollback on application failure (`tests/canvas_wiring.mjs`).
`web/modules/mcp_canvas_brief.js` is the canvas in brief for an AI (pure): bounded path chains and node lines,
the inputs a run needs that nothing feeds yet with the outputs that fit them, ComfyUI's
errors; every canvas change answers with what is still unwired. `api/mcp_situation.py` is
get_situation: the task, that brief (describe_canvas with `brief`), the latest run and the
next step in one answer kept under Antigravity's size for answers it files away, for an AI
starting a new chat or lost (`tests/test_mcp_situation.py`, `tests/mcp_canvas_brief.mjs`).
`api/mcp_directory.py` is the directory of the tools by what an AI wants to do (as this file is
for the code): each tool under its stage (`action_catalog.TOOLS`) with its title, built from the
registry, in begin_task's answer and in get_situation's while there is no task
(`tests/test_mcp_directory.py`). AI apps are offered every tool (they load what they need
themselves: Antigravity reads a tool's file before its first call); the module also keeps which
list each app fetched last (`anomalous/ai_tool_lists.json`), so a call from an app still holding
the list of an older Anomalous is told so and the task header says how to reconnect it.
`api/mcp_fill.py` gives run_workflow its blanks, a daily run in one call: a saved workflow
opened in the task's tab once and reused (the page marks the tab with the workflow's file),
then the prompts, seed, size and the picture a LoadImage reads filled in by the page's
`fill_slots`, planned by `web/modules/mcp_fill.js` (pure). Any other setting goes by its name
(`values`, also set_values without a node): the one node that has it, else the AI names the
node; the fixed blanks are roles (which prompt, which latent), not a list that grows. With them a run asks as the stricter
of its gate and the canvas's (`action_catalog.ALSO_CANVAS`). `tests/test_mcp_fill.py`,
`tests/mcp_fill.mjs`.
`api/run_refusal.py` says ComfyUI's refusal of a run as what to do next: the page passes it on
as data (`web/modules/mcp_run.js` refusal), each problem becomes a line naming the node, what
is wrong and the tool that fixes it. `tests/test_run_refusal.py`, `tests/mcp_run.mjs`.
`api/ai_toolbox.py` keeps what the user hands the AI to work with in a task ("给 AI 的素材",
`web/modules/ui_ai_toolbox.js`, from the task header a side panel (on the AI page a view of
its side column, `setToolboxHost`; elsewhere docked on the right) that leaves
the page usable (kinds as tabs, one line per item (`rowCard`), a card's detail over them) or, expanded, a large
window: search, kinds, a column by source / origin / type, a card grid drawn as it scrolls,
a detail pane, the picks below; the last choice is kept in the browser),
by the names the AI's tools use (and whole custom node packs by folder, `packs`: get_toolbox
lists their node types, `chosen` counts them picked), with a mode: prefer, or only (add_node / new_workflow then
refuse other custom or paid node types). Node types come from the page's own definitions
(`web/modules/ai_toolbox_data.js`: LiteGraph's registered types and their `nodeData`, so no
request; page-only helpers go with their pack when their name says which); the rest in one
answer from `api/ai_toolbox_catalog.py` (combos, workflows with covers and prompts, prompts
saved or inside combos and workflows as `combo:<name>` / `workflow:<name>`, models with
covers), read ahead when a task header is drawn and kept a minute.
`web/modules/ui_ai_facet_menu.js` is the side panel's list of sources (or types) under its
"Source: All ▾" button: a search over the names, ↑ ↓ and Enter, the row under the pointer lit
and the one shown ticked.
`web/modules/ui_ai_toolbox_cards.js` draws the cards (a node in miniature with its sockets in
ComfyUI's colours, covers, prompt text) and the detail pane; `web/modules/ui_ai_toolbox_icons.js`
each pack's badge, and the illustrations (`web/assets/toolbox_*.webp`, made
with ChatGPT): the empty selection and the stand-in cover of a combo, workflow or model. Favourites (`anomalous/ai_toolbox.json`) stand in until the user picks for
a task. A pick is a `user_toolbox` step the AI gets as a message (kind `toolbox`); begin_task
names the picks, `get_toolbox` gives them in full, find_nodes lists them first.
A rewired input shows in the receipt as a `wired` change
(`activity_diff.js` keeps what feeds each input).
`web/modules/mcp_run.js` captures exact prompt IDs from one native queue request,
preserving native seed/widget callbacks and excluding later manual batches. Busy submissions
and unsupported host request IDs are refused before queueing; wrappers/listeners are released.
`api/mcp_prompts.py` serves `prompts/list` / `prompts/get`: ready-made requests (the AI app's
slash commands) that tell the AI how to do a common job so the user can follow it on the AI
workbench. `api/mcp_apps.py` serves the MCP Apps extension (`io.modelcontextprotocol/ui`):
`api/mcp_app/task_card.html`, Anomalous's own card that apps supporting it show in the
conversation under `begin_task`, `show_panel`, `report` and `run_workflow` results
(`resources/list` / `resources/read`, `_meta.ui.resourceUri` on those tools). The card reads
the task through `app_task_view` (steps, inline gallery thumbnails, open questions) and
answers the AI's questions through `app_answer`, tools marked for the card only
(`visibility: ["app"]`, never steps, never given the user's news). It never answers a gate:
an OK is only given in Anomalous, so no tool an AI can reach grants one.
User guide: `docs/guides/mcp.md`.
- The AI workbench (1.62). `api/ai_tasks.py` keeps every MCP call as a step of a task (one
file per task in the ComfyUI user folder, `anomalous/ai_tasks/`, the newest 300): a task is
named by the AI (`begin_task`) or is one app's calls without a long pause (20 minutes); a
step keeps the tool, a short summary, whether it worked and what it made (models, the combo,
the canvas change by its activity entry, the spoken audio, the scan; a run's pictures are
watched in ComfyUI's history until it ends, `api/ai_task_runs.py`, which also reads where it
stands from ComfyUI's queue and progress registry, never changing them: queued and how many
are ahead, or the node running, its steps, a percent like ComfyUI's tab title and the nodes
done; kept on the step in memory, sent to the page as `progress` events that only move the
step's bar, and given to the AI by `run_status`, which can wait for the run to end, and as
`your_run` with one sentence in the AI's other answers while it goes and once after it ends
(that first answer also carries up to four thumbnails of its pictures: `show_pictures`),
so the AI need not look for itself; the console's lines, time left, memory and the size
warnings come with it, `api/ai_run_watch.py`, and the page draws them with a stop button,
`web/modules/ui_ai_run_live.js`). A step a restart left
`"running"` is marked stopped when the record is read again. What each tool is, in one
place, is `api/action_catalog.py`: its kind (colours, words, counts), the stage of the work
it belongs to (model, combo, prompt, canvas, run, picture, voice, library: the flow over a
task) and its gate (canvas, gpu, download, network: what the user may want to approve); a
test keeps it and the MCP tools in step. `api/ai_task_changes.py` puts the user's own changes
(activity entries, `activity_log.add_listener`) made while an AI works into that AI's task
as `user_activity` steps, which the AI receives once with its next call (a message of kind
`change`); `activity_log.tag` names the task on the entry, and the activity page links to
it. `api/ai_gates.py` holds the user's gates: each gate is `auto`, `ask` or `never`
(`anomalous/ai_gates.json`, `/anomalous/ai_gates`); `mcp_server.py` asks it before running a
tool. It also keeps (`anomalous/ai_tools.json`) how long in all an AI waits for the user's answer (`answer_wait`: 1, 3 or 10 minutes): `api/ai_answer_wait.py`
tells an AI whose question is still open to wait on with get_answers until then, and while the
user types (the page posts `POST /anomalous/ai_tasks/{id}/typing`) until a minute after their last
key; report and show_panel remind it to end each round with ask_user. With `ask` the step waits in state `asking` (what it would do listed) for up to 50 s;
the user's Allow or No comes through `POST /anomalous/ai_tasks/{id}/gate`. Unanswered by
then, the AI is told to wait and call again; the later answer reaches it as a message of kind
`gate`, and the same call made again after an Allow goes ahead without asking twice.
`api/ai_marks.py` keeps the marks the user draws on an output picture for the AI
(`POST /anomalous/ai_tasks/{id}/marks`: the marked PNG, checked, kept in
`anomalous/ai_marks/<task>/`, never the output folder, gone with the task; the marks as
fractions with notes): a `user_marks` step the AI receives like a message (kind `marks`) with
the marked picture attached as an image to the same answer. `web/modules/ui_ai_marker.js` is
the marker window (box, circle, arrow, pen; three colours; a note per mark; a brush for the
area to repaint) opened from a picture's ✎ on the AI page, and the timeline row of marks sent.
The user also picks how to change it (`ai_marks.MODES`: leave it to the AI, only the marked
areas, the whole picture anew), the first line the AI reads, as the user's choice, and the
timeline row says it. For only the marked areas the page also sends a mask (white where boxes,
circles, lines and brush strokes are); the server writes the picture with that area as its alpha to ComfyUI's
input folder as `anomalous_inpaint_<step>.png` (a new file, never replacing one; kept when the
task goes, as workflows may load it), and the AI's marks message carries its name with how to
inpaint with it (LoadImage gives the MASK). The canvas tools read ComfyUI's lists anew once
when a value is not among a drop-down's choices, so they take such a new file.
`api/ai_repaint.py` lets the AI agree on that area with the user first (propose_repaint_area:
boxes or ellipses, grown and softened by pixels): both areas tinted over the picture into the
task's marks folder, side by side on a panel (block `area`, `ui_ai_panel.js`) with the AI's,
the user's own or marking again; the picked area is written like the user's mask
(`ai_marks.write_inpaint`, soft edges kept) before the answer is recorded, so the answer the AI
reads carries `inpaint_image` and the inpaint note. Building the repaint stays the AI's.
Options Anomalous makes name a page word (`label_key`). `tests/test_ai_repaint.py`.
`web/modules/ui_ai_gate.js` draws the card under a step that asks and the Gates window;
`ui_ai_flow.js` the flow over a task (its stages in order, each done, running, waiting or
failed; a stage leads to its first step). `ui_ai_handoff.js` is "Hand to AI" on a model's
page, an output image's inspector and a combo card: a task the user starts with `about`
(the model's `model_id`, the image, the combo's name; checked in `ai_tasks.start_task`),
which `begin_task` hands to the AI, and the sentence to paste into the AI app. It tells the page through the
`anomalous.ai_task` event; `api/ai_tasks_routes.py` serves `/anomalous/ai_tasks` (list, one
task, start, rename or remove, a panel's answer, a message). The user can start a task first
(New task): it waits, and the next app that calls takes it over, so its work starts there
instead of joining its last task (one waiting at a time, found again after a restart).
`begin_task` resumes a waiting or active task, including preliminary reads, and keeps a user's title;
`new_task=true` explicitly separates a new request. Ended or idle tasks are not resumed.
`mcp_server.py` starts an in-flight `"running"` step before each call and updates the same step in place when it finishes or fails (the app's name from its client
info or User-Agent); `api/mcp_task_tools.py` holds the AI's own tools for it, `begin_task`
and `report` (a note, with pictures from the output folder it made another way).
`web/modules/ui_ai_workbench.js` is the page (rail entry `ai`): the tasks, one task's
results strip and timeline with links (picture viewer, model page, combo, canvas locate and
undo, scan page, ▶ for audio), updated live; also the home page's "What the AI did lately"
and the rail's dot (`owner.railNews`). A task's parts stay in the page between redraws: a
step is drawn again only when it changed (`syncRows`), so a scrolled diagram or strip and
half-typed text survive the AI's steps coming in. `ui_ai_message.js` is the message box, made
once per task and docked at the foot of its column (in view while the timeline scrolls; a task
opens at its newest step, and `ui_ai_newest.js` is the button back to it over the box): a growing input, the gallery and materials buttons beside it, and
over it what the message carries (`ui_ai_tray.js`: pictures to mark up with
`ui_ai_marker.js` or leave out, chips of what "@" names), and over that what the task needs or
does now (`ui_ai_now.js`: a gate to allow, a run's progress with stop, the AI's open question
to answer there; with none of that, `ui_ai_call.js`'s lines to paste into the AI's chat once it
has stopped, which `ai_call_lines.js` picks by what waits for the AI: a run it was not told has
ended, a reply it has not got, else go on, and a line for a new chat; `tests/ai_call_lines.mjs`). `ui_ai_drop.js` takes pictures dropped on the box or pasted, kept in the
output folder's ai_inbox (`POST /anomalous/ai_pictures/upload`, `ai_pictures.save_upload`). `ui_ai_mention.js` holds those refs
and gives it "@" (the task's pictures, the gallery's newest and those its search finds,
combos, models, workflows, prompts, node packs and node types from the materials picker's
data) and "/" (`ui_ai_commands.js`: stop, undo, side views, new task, canvas mode, words to
fill in); `ui_ai_gallery_pick.js` is the gallery grid pictures are attached from.
`api/ai_message_refs.py` checks what the message names and gives each to the AI
with the message (in full, pictures attached; get_toolbox shares its `lookup`),
`tests/test_ai_message_refs.py`. `ui_ai_step_body.js` is what a timeline step says under its
title (the user's own steps with whether the AI has them, a decision's card, the AI's note, what
happened outside Anomalous). `api/ai_outside.py` is that last: when the AI calls (never in the
background) it looks at the top of the output and input folders, by the files' own times, for
pictures no ComfyUI run saved and pictures rewritten in place since the AI's previous call
(an `outside_files` step; the AI is told with its answer), and keeps `log_outside`, the AI's
own word on a command or script it ran; both are steps of kind `outside`.
`tests/test_ai_outside.py`.
`api/ai_decisions.py` keeps a task's key decisions: `log_decision` (the AI records what was
settled, why, what was left out, who decided, the steps it is about; a `decision` step), and
the user's own choices read from the task's steps (panel answers, gates, materials, the canvas
mode — `ai_task_canvas.set_mode` adds a `user_canvas` step — and stops); `GET` of a task adds
`decisions` and `decisions_gap` (runs and canvas changes since the AI's last one), begin_task
repeats them to an AI picking the task up. `ui_ai_decisions.js` draws the card and the side
column's Decisions view. `tests/test_ai_decisions.py`. `api/ai_callers.py` names who calls (the
app from its client info or User-Agent, a session, a task it names); an app whose task ended or
paused goes on in the task the user wrote to and it has not read (`ai_tasks._take_replied`,
which opens it again). `api/ai_task_links.py` keeps which task
a task goes on from (`from`: set when an AI begins a new task while on another, or the user
starts one while looking at a task; changed or taken off by the user, never a loop); `GET` of a
task adds `links` {from, next}, begin_task gives an AI the task it continues and what was
decided there (`continues`). `ui_ai_task_links.js` shows them under the task's title and picks
one to link. `tests/test_ai_task_links.py`. `ui_ai_recent.js` the home page's list, `ui_ai_results.js` what a
step made and where it leads, `ui_ai_shared.js` what the page's parts share. `ui_ai_outputs.js` is what the whole task made: a line over the
timeline counting it, and the side column's Results view (`ai_task_view.taskOutputs`: the
pictures by the step that brought them, the workflow, the combos, the newest picture's prompts,
the models used and download requests — `ai_tasks.results_of` names the downloads — and the sounds),
which keeps what made the newest picture as a recipe, a combo or a saved prompt through the
keep menu's own `keepPictureAs`; its button counts what came since it was last seen
(`stage.outputsSeen`). For a run the AI did not wait
for (`ai_gates.run_wait` "call") the server marks the ended run `call_ai` until the AI is told,
and the lines over the message box are about it meanwhile. `ui_ai_side.js` is the
stage, the column on the right where the AI's panels are shown (the timeline keeps a card per
panel, with what it holds from `ai_task_view.panelGist` and "Show on the right", which says
"on the right now" while the panel is in view there, an IntersectionObserver of the stage): like the timeline it reads from the top down, the task's panels in
the order they came with their step numbers, the newest at the bottom. Opening a task,
new/updated latest panels and late image loads follow the bottom while the reader is there;
reading earlier content pauses follow, with a "See latest" button to resume. Timeline links
instead locate the requested panel's start. The stage's close button hides it for the selected
task (`stage.hidden`); updates of a panel preserve that choice and its DOM/drafts, a new panel
reopens it (`bringStage`) unless the user turned "Open on new panels" off (localStorage
`anomalous_ai_stage_auto`; in a narrow window it lights the stage's tab instead). A timeline panel
link reopens it; selecting a different task resets it. The stage is one view of the side
column: `ui_ai_side_views.js` is the switch in the window title bar, AI page only (stage, a run's
node map at full size with a list of the runs, the decisions, the materials picker, the
results; the shown one again closes the column), one pane per view kept between switches
(each under `ui_ai_side.sideHead`), and the picker's dock there.
The task's header stays on top while the timeline scrolls. In a window too narrow
for three columns the timeline and the side column are two tabs (a panel that asks brings the stage
forward). The side column is a card whose left-edge grip sets its width (`--ai-side-w`, kept;
the timeline keeps `TIMELINE_MIN`). The task list folds from the title bar's list button
(`headerLeadTools`, `listToggle`); folded, the list stays laid out out of sight, and pointing at
the button (or moving left onto the page's left edge) slides it over the page at once.
`ui_ai_task_list.js` draws it (New task, Connect an AI
and the gates, which AI came last, quick filters by state, the tasks by day with a state
icon each), making anew only the rows that changed (`web/modules/sync_rows.js`, the timeline's
way too). `web/modules/ai_task_cache.js` keeps the tasks as last read: a task picked shows at
once from there (read ahead while the pointer rests on its row) and again, what changed, when the
server answers; the activity entries a task lacks follow. The stage's pictures, like the
timeline's, load the 512 px thumbnail first and the file itself only when shown larger. `ui_ai_icons.js` is the AI pages' one set of line icons and the icon button.
`ai_text_format.js` lays out what the AI writes (its notes and replies: headings, lists, bold,
code) with textContent only. A note may answer the user's message or marks (`report`'s
`reply_to`): the two steps point at each other, and the timeline links them both ways.
`ui_ai_connect.js` says how to connect an AI app (the address, a sentence to send the AI so
it sets itself up and starts on a waiting task, each common app's way), in a window from the
list's "Connect an AI", and as a card (`connectHint`) that opens the guide in place on the
empty page and under a task waiting for an AI. `ui_ai_ready.js` is that waiting task's one
page: the sentence to send the AI first, kept in view at the top while the page scrolls, then
folds to open for what may be put ready (what to do, a line naming the task; the gallery grid
of `ui_ai_gallery_pick.js` feeding the message box) and the user's own steps so far. The pictures (`web/assets/ai_*.webp`)
were made with ChatGPT. `ai_task_view.js` holds
its rules without DOM (folding reads and chains of the AI's work, tool words, counts, result addresses);
`ui_ai_chain.js` draws the folded rows: reads in a row, and a chain of reads, canvas changes and
decisions as one row of chips with undo for its canvas changes, opening to its steps.
`ask_user` (`api/mcp_task_tools.py`) is the AI's question in the workbench: a one-block panel
(choices with an optional `recommended` option and an `other` line, or an input) whose answer
it waits for in the same call; the server instructions and `begin_task` tell the AI to ask
this way rather than in its own chat.
Panels the AI draws (`show_panel`, `update_panel`): `api/ai_panels.py` checks every block
against a fixed set (heading, text, images from the output folder, models by id with their
covers, table, list, params, choices, input, progress, divider; plain strings of bounded
length, nothing run or read as markup) and keeps the panel as a step. `update_panel` preserves
the order already seen: new blocks append; omitted blocks remain; unchanged resubmitted
blocks match without their ids. Every new block gets a stable id, returned by the tools;
an explicit existing id edits that block in place without changing its type or position.
A uniquely labelled progress block also updates in place for clients that omit ids.
The cumulative limit is 30 blocks; overflow or invalid input rejects the entire update
before changing the panel, title or answers. Legacy records are not reordered on load;
blocks without ids receive handles when updated. New phases should use `show_panel`.
The user's answers come
in through `POST /anomalous/ai_tasks/{id}/answer` and wait there until the AI reads them
(`get_answers`, which can wait up to 55 s for one; Anomalous cannot call the AI). The user
can also write to the AI on a task (`POST /anomalous/ai_tasks/{id}/message`, a `user` step);
`get_answers` brings those messages too, and with no panel named waits for one; unread answers and user messages are also piggybacked once onto the next successful MCP tool response (`ai_panels.attach_pending`) so AI apps that do not poll `get_answers` still receive them immediately.
`web/modules/ui_ai_panel.js` draws a panel in the timeline (textContent only) and the toast
when the AI shows one while the user is elsewhere; a question can be skipped (`{skip: true}`,
the AI reads `skipped`). `web/modules/ui_ai_delivery.js` is the pill under each answer and
each thing the user gave the AI: sent and waiting, or the AI has it at `delivered_at` (set
when `ai_panels._collect` hands it over). The stage blocks are data the page lays
out, never drawings of the AI's own (no SVG is taken): marks on pictures (circles or boxes at
fractions of the picture, a label that zooms there), the slider (before and after over each
other), picks (pictures with a reason, "Take this one", "Put on the canvas" through ComfyUI's
own file loading, drag onto the canvas) and the diagram (boxes and arrows; a box naming a
canvas node finds it). `ui_ai_stage.js` draws them, `ai_stage_layout.js` holds where they go
without DOM (a diagram's layers by its longest chain of arrows, boxes, arrow ends, marks).
- `ui_ai_images.js` connects task outputs and panel pictures to the existing image inspector,
keep menu and `ui_param_compare.js`: tools over each output picture (shown on hover), and a
tick; two ticked pictures, from any step or task, are compared from the bar at the foot of
the AI page (a third tick replaces the oldest). Videos and temporary files get neither.
Thumbnails load initially, metadata only after an explicit action. `ui_canvas_changes.js`
renders the activity record's bounded before/after receipt for both the AI timeline and
`ui_activity.js`, with a caller-owned locate action. Missing records show unavailable;
receipts never reconstruct history from the current graph. The workbench retains only
its selected task's activity entries and ignores superseded refresh/navigation responses.
- `api/backup.py` is Settings → Backup (`/anomalous/backup/*`): one .zip of the user's stores
(recipes, combos, materials, parameters, optionally ComfyUI's saved workflows), the plugin
settings without the Civitai key, and each model's user layer and own cover; putting it
@@ -159,7 +479,10 @@ covered by the `styles.css` manifest.
and the scanned models, never replaces a file (" (2)").
- `api/model_placement.py` is the models page's Tidy check (`/anomalous/placement/*`): models
whose header names another type than their folder, identical files (by SHA-256), and moving
a model with its sidecars to where it belongs.
a model with its sidecars to where it belongs; `belongs()` checks one file for the AI, `look()`
says what one file's header suggests even when unsure. `web/modules/ui_model_retype.js` is
"Change type" for one model (its page's actions, the toast after an import): the types with
their loaders, the header's guess marked, where it goes, moved as Tidy moves.
- `api/model_usage.py` is Tidy's "not used lately" (`GET /anomalous/usage/check`, this computer
only, on a press): when each model and node pack was last used, from the run record, the
PNG outputs' prompts (image_search's cached records, which keep the node types too), FLAC /
@@ -203,7 +526,8 @@ covered by the `styles.css` manifest.
tells, shared with the standalone scanner (which runs as its own process, so the API
never imports `scraper.py`). `model_kind.py` reads a .safetensors / .gguf header (never
formats that can carry code) and says which models folder the file belongs in and its base
model; model import and the scanner's offline base model use it.
model (model patches such as Anima LLLite go to `model_patches`, not `controlnet`); model
import and the scanner's offline base model use it.
- `recycle_bin.py` is the only way the plugin deletes the user's files (models, covers,
outputs, recipes, materials, notes, audio): to the system Recycle Bin, together, or not
at all (`TrashUnavailable` on drives without one). `api/trash.py` is the API's import of it
@@ -370,14 +694,17 @@ covered by the `styles.css` manifest.
`planVoiceDrop` decides every character drop and names the reason for each refusal;
unlisted nodes are refused, never matched by widget name. Supporting a node
means adding one entry plus a test.
- The rail's Voices entry has two views, switched by `ui_voice_tabs.js` at the top of
each and sharing the character list: Characters (shell page `voices`) and Voice-over
(shell page `script`; `ui_shell_nav.js` keeps the rail on Voices and reopens the view
used last).
- The rail's Voices entry has three views, switched by `ui_voice_tabs.js` at the top of
each and sharing the list column: Characters (shell page `voices`), Voice-over (shell
page `script`) and Audio (shell page `audio-gallery`, `ui_audio_gallery.js`).
`ui_shell_nav.js` keeps the rail on Voices and reopens the view used last, from the
rail and from the 9 key; the rail stays within the nine number keys.
`ui_audio_studio.js` owns the Characters view: the character cards, preview playback,
tag copying, dragging a card header onto a node through the shared `bindMaterialDrag`
with `targetHint`/`rejectHint` telling the user what releasing does, and each card's
"Voice-over" button (`owner.openScript(group)`).
with `targetHint`/`rejectHint` telling the user what releasing does, each card's
"Voice-over" button (`owner.openScript(group)`), and its Delete character button
(node interface 19, local only): after a confirm, `removeGptSovitsCharacter` has the node
move the character's folder to the Recycle Bin.
`ui_script_page.js` owns the Voice-over view: it loads the characters and mounts the
script director, or shows the install steps or the way to import one when there is
nothing to voice yet. A character picked in the list speaks there; the audio
@@ -455,7 +782,8 @@ covered by the `styles.css` manifest.
recorded in each file, plus a section for unsaved temp previews that can be
copied into `output/audio/`.
- `ui_sidebar.js` assembles the browser window (rail, list column, header, page panels;
`remote_view.js` adds the header's "view only" chip when opened from another device)
`remote_view.js` adds the header's "view only" chip when opened from another device;
`headerTools` is a page's own slot in the header, shown on that page: the AI page's side switch)
and puts the model folder column in the list column (`ui_folder_list.js`: the folders of the
type the chips show, counts with subfolders, 📂 to open one in Explorer). `ui_shell_rail.js` owns the left icon rail: one
entry per page, the tool slots and the settings slot. `ui_shell_nav.js` owns page
+325
View File
@@ -1,5 +1,330 @@
# 📈 Anomalous Model Browser Changelog
## v1.62.1 (Fix: Anomalous sometimes did not start) — 2026-11-05
- 修好一个偶发的启动故障:有时打开 ComfyUI,AMB 整个没加载,悬浮按钮和窗口都没有,刷新一下又好了。
- Fixed a startup failure that came now and then: ComfyUI opened with no Anomalous at all, no button and no window, until a reload.
## v1.62.0 (AI Workbench, AI Builds Workflows on the Canvas, Repaint an Area, Materials Side Panel) — 2026-10-29
- 音频库搬进了“角色语音”页:页面顶上的标签现在是 **角色 | 配音 | 音频库**,左边栏少了一项。加了 AI 工作台以后左边栏有 10 项,数字键只有 1–9,音频库就没键可按了;现在 9 打开角色语音,并回到你上次停的那个标签。
- The audio gallery moved into Voices: the tabs at the top read Characters | Voice-over | Audio, and the rail is one entry shorter. With the AI workbench the rail had ten entries for nine number keys, leaving the gallery without one; 9 now opens Voices on the view used last.
- AI 能拿刚出的图接着改了:搭“在这张图上重画”的工作流时,加载图像节点也认输出文件夹里的图(`xxx.png [output]`,ComfyUI 本来就支持);以前 input 文件夹里没有这张图就被拦下。左侧栏的“模型”“图库”不再显示成英文 models / gallery。
- The AI can redraw a picture it just made: a Load Image it builds now takes an output picture (`name.png [output]`, as ComfyUI itself does) instead of being refused when the input folder lacks it. The rail shows Models / Gallery again instead of the raw keys.
- 修复完成后的任务接续:AI 带着原任务 ID 查看结果、补面板,也会留在刚完成的任务里,不会另开空任务或把面板放进另一个任务。错误标记只在对应重试完成并成功后消除;等待审批、仍在运行、已取消,以及另一个节点或设置的成功都不算修好。
- Fixed follow-ups carrying an explicit task ID: reads and panels immediately after completion stay with that task instead of creating an empty task or placing a panel in another one. Error recovery requires a completed successful retry of the recorded target; pending approval, running or cancelled attempts and success on another node or setting do not clear it.
- 参数名自动纠正现在也用于审批和任务记录:你批准后,AI 改用正确参数名重试不再重复问,换模型的结果卡片也不会漏掉。展台和底部提问条同步同一道题的草稿,刷新时保留原来的光标、选区和编辑位置。
- Corrected argument names now carry through approval and task recording: retrying with the proper name reuses an unused approval, and model changes keep their result cards. The stage and bottom question strip share each answer draft; refresh preserves the caret, selection and editing surface.
- 修复任务接续与等待回答:按你最近一次未读回复接续任务,后台进度更新不再打乱顺序;选择题回答也能接回原任务,仍有工具正在执行的任务不会被新会话自动接走。读取答案不指定面板时也会正确处理超时;到时间自动选择或恰好收到你的回答,都会在当次调用里交给 AI。等选择题时发的留言和图片不再被读走却漏传。
- Fixed task resumption and answer waiting: resume by the latest unread user reply, including panel answers, rather than background updates; a task with a tool still running is excluded from automatic takeover. Reading answers without a panel now handles expired questions too. Timeout choices and user answers arriving at that boundary reach the AI in the same call; messages and pictures sent while it waits for a question are no longer consumed without delivery.
- 任务列表更准:AI 改正过来的错(比如参数写错、下一次就对了)不再让整个任务显示成“出错”,只有 AI 没跨过去的才算,红标改叫“有一步没做成”;AI 说做完以后马上再看一眼结果,这一眼记在原来的任务里,不再多出一个只有一步的空任务;接着给你看前后对比、问你满不满意,也回到原来那个任务里(任务重新打开),不再另起一个;你点“新任务”写好的任务一直等到 AI 来接(或者你再开一个新的),不会过了 12 小时悄悄失效、而页面还写着“等 AI 接手”。
- AI 少走弯路、少猜:
- AI 把必填参数写成了常见的另一个名字(比如打开工作流时把 `workflow` 写成 `name`、看图时把 `image` 写成 `file`)时,AMB 直接照它的意思做,并告诉 AI 正确的名字,省一次来回。只认考试里见过的这几种叫法;别的名字照旧报错、列出该写什么,免得猜错意思。
- 时间线里 AI“放大看图”那一步显示成代码名,改成了中文名。
- 去掉了“给 AI 的工具”里的 **日常**:所有工具都直接给。Antigravity、Claude Code、Cursor、VS Code 这些软件会自己按需读工具说明,AMB 再做一层“现查现用”只会让 AI 多绕路(一次实测 29 次调用里 18 次花在查工具和转一道上)。
- AMB 更新以后,AI 软件还拿着旧的工具列表来调用时,任务标题下面会出现一行黄字,告诉你怎么让它重新连接;AI 那边也收到同样的提醒。AI 软件重新取了列表,黄字自己消失。
- AI 加 LoRA 或换模型时,AMB 对比 LoRA 和主模型的底模:一个是 SD 1.5、一个是 SDXL 这种用不了的,会提醒 AI 并告诉它按底模去找;Pony 的 LoRA 用在 Illustrious 上这种能用但可能效果差的,也会轻轻提一句。只提醒,不拦着。
- AI 看模型详情时,能看到作者示例图用的采样器、步数、CFG、尺寸,LoRA 还有作者用的权重(来自扫描时存下的 Civitai 资料),还有版本名。
- 跑图被 ComfyUI 拒绝时,AI 收到的不再是原样的英文报错,而是“哪个节点、缺什么、该用哪个工具修”。
- 用 Gemini 3.8 Flash 实际考了 7 道常见任务(出图、换模型加 LoRA、问你再改、缺模型、找图看参数、搭放大分支、修断线),失败多半出在第一次用某个工具时参数写错,于是:参数写错时,AI 收到的报错会列出这个工具该写哪些参数,下一次就对;同一个东西在各工具里叫同一个名字(写提示词和跑图都叫 `prompt`,打开工作流和跑图都叫 `workflow`);等待时间写超了(比如 120 秒)按最长的 55 秒等,不再报错;按类型找模型时写单数(checkpoint、lora)也能找到。
- 很久以前(两个多小时前)留下、AI 一直没收到的留言或标注,不再把一个新对话拉进那个旧任务。
- 关卡窗口新增 **等不到回答时**:**停下等我**(默认)或 **按推荐的继续**。选后者时,到了等待时间你还没答,AI 标了推荐项的选择题就按推荐的选,展台和“决策”里都写着“到时间,按推荐选了”;没有推荐项的问题、你已经动手写了回答的问题、关卡里“先问我”的事,永远不会替你答。
- The task list says what is true: an error the AI got past (a wrong argument right on the next call) no longer marks the whole task as failed, only one it did not get past does, and the red mark reads "A step did not go through"; a look the AI takes right after saying it is done goes into that task instead of making an empty one-step task, and a before and after or a "happy with it?" it shows next opens that task again instead of starting another; a task you start with New task waits until an AI takes it (or you start another), instead of quietly lapsing after 12 hours while its page still says it waits.
- The AI takes fewer detours and guesses less:
- When a call writes its one needed argument under a known other name (`name` for open_workflow's `workflow`, `file` for `image`), Anomalous takes it as meant and tells the AI the right name, saving a round trip. Only the names seen in the exams count; any other is answered with what the tool takes, so no meaning is guessed.
- The timeline names the AI's "look closer at a picture" step instead of showing its code name.
- The tool set's **Daily** choice is gone: AI apps get every tool. Antigravity, Claude Code, Cursor and VS Code load tool descriptions as they need them, so looking tools up through Anomalous only added detours (in one run 18 of 29 calls).
- When an AI app still calls by the tool list of an older Anomalous, a yellow line under the task header says how to reconnect it, and the AI is told too; it goes once the app fetches the list again.
- Adding a LoRA or swapping a model, Anomalous compares the base models of the LoRA and the main model: one that cannot work (SD 1.5 on SDXL) is pointed out with how to find a fitting one; one that loads but may look off (a Pony LoRA on Illustrious) gets a softer note. A note only, never a block.
- Model details give the AI the sampler, steps, CFG and size the author's example pictures used, and for a LoRA the weight (from the Civitai record a scan keeps), and the version name.
- When ComfyUI refuses a run, the AI gets which node, what is missing and the tool that fixes it, instead of ComfyUI's raw report.
- Gemini 3.8 Flash took seven everyday tasks as an exam (a run, a model and LoRA swap, asking you first, a missing model, a picture's settings, an upscale branch, a broken link); most failed calls were a tool's first use with wrong arguments, so: an argument error now lists what the tool takes, so the next call is right; one thing has one argument name across tools (`prompt` for set_prompt as for run_workflow, `workflow` for open_workflow as for run_workflow); a wait longer than 55 s waits 55 s instead of failing; a model type in the singular (checkpoint, lora) finds them too.
- A message or marks the AI never got, left more than two hours ago, no longer draws a new chat into that old task.
- The Gates window has **When I don't answer**: **Wait for me** (the default) or **Go with the recommendation**, where a choice the AI marked a recommended option for takes it once your waiting time runs out, shown on the stage and under Decisions; a question without a recommendation or one you began to answer, and anything a gate asks first, is never answered for you.
- AI 问你的问题更好答:选项是一按就发出去的按钮,右边有箭头,问题下面写着“点一个就发给 AI”,点了马上变成选中的样子;**让 AI 定** 挪到问题右上角,不再跟选项挤在一起被当成“确认”;“都不是”那一行默认收起成 **都不是,我自己说…**,关卡卡片上的原因也收起成 **说说想怎么改…**,点了才出输入框。展台、留言框上方的提问条、关卡卡片都一样。关卡窗口里几条说明改对了:改画布也管加节点、连线、新建工作流;跑完以后那几句话在留言框上方;“先问我”和按钮上的字对上了。写了一半的回答不再被清掉:AI 又问了新问题、读走了另一个回答时,字和光标都还在,那个问题一直排在提问条最前面;窄窗口里你正在打字时,新问题只点亮展台标签,不把你切过去;用输入法按回车选字时,不再把半句回答发出去。
- Questions from the AI are easier to answer: options are buttons that send at a press, with an arrow and "One press sends it to the AI" above them, and the pressed one shows at once; **Let the AI decide** sits at the question's top right instead of under the options where it read as a confirm button; a line of your own stays folded as **None of these, I'll say it…**, and the Gates card's reason as **Say how to change it…**. The same on the stage, the question strip over the message box and the Gates card. Gates window texts fixed: changing the canvas includes adding nodes, wiring and new workflows; the lines after a run are over the message box; the intro names "Ask me" as the button does. A half-typed answer is no longer wiped when the AI asks something new or reads another answer: the text and the cursor stay, and that question stays first in the strip; in a narrow window a new question lights the stage tab instead of taking you there while you type; Enter that picks a word in an input method no longer sends half an answer.
- 局部重绘前先对一下范围:你圈得不一定准,AI 看了标好的图可以挪动、改形状(方框 / 椭圆)、往外扩、把边缘做软,然后在展台把 **你圈的** 和 **AI 调整后** 并排给你看,你点 **用 AI 调整的 / 用我圈的 / 我重新圈**,或者写一句想怎么改。选好以后 AMB 把带这个遮罩的图放进 input 文件夹,AI 收到文件名和一句常用重绘节点的提示;画布上怎么加节点、怎么连线仍然由 AI 自己搭,不套固定模板。
- Agree on the area before repainting: your marks are rough, so the AI can move, reshape (box or ellipse), grow and soften the area and show **Your marks** beside **The AI's area** on the stage; you pick **Use the AI's / Use mine / I'll mark it again** or write what to change. Anomalous then puts the picture with that mask into the input folder and the AI gets its name with a short note on the usual repaint nodes; how to build the repaint on the canvas stays the AI's, no fixed template.
- AI 问你问题后会等你:关卡窗口新增 **等我回答**(1 分钟 / 3 分钟 / 10 分钟,默认 3 分钟)。一次调用最多等 55 秒,你没答它就接着等,直到这个时间;你在回答框或留言框里打字时,停手一分钟内都算还在答。AI 写完说明或放出展台面板后会被提醒:收工前先问你下一步做什么、要不要开始(跑图、下载、扫描时除外),说明里写了问句的,提醒它改用提问。问题上的“这题跳过”改叫 **让 AI 定**,AI 收到后自己决定、记下决定,不再把同一个问题再问一遍。
- The AI waits for your answer: the Gates window has **Waiting for your answer** (1, 3 or 10 minutes, 3 by default). One call waits up to 55 s and the AI then waits on until that time, and while you type in an answer or the message box until a minute after your last key. After a note or a panel it is reminded to ask what next or whether to go ahead before it ends its turn (not while a run, download or scan goes on), and to put a question in its note into ask_user. "Skip this one" is now **Let the AI decide**: the AI decides, records it and does not ask again.
- 记录里不再出现“你在 AMB 里改了:保存图像 $$canvas-image-preview”:那是 ComfyUI 显示预览图时自己改的内部字段,不是你的改动,以前会被当成你的改动发给 AI。
- The record no longer says you changed a Save Image node's `$$canvas-image-preview`: ComfyUI sets it to show a node's picture, and it used to reach the AI as your change.
- AI 已经把任务做完了,你再留言或在图上标注:任务重新打开,留言框上方出现那句一键复制的话(“我在 Anomalous 里回复了,看看再接着做”),AI 回来接着在这个任务里做、回复也挂在你那条下面,不再另开一个新任务。给更早的任务留言也一样:AI 手上的任务做完或停下后,会回到你留言的那个任务。
- When the AI had already finished a task and you write to it or mark a picture, the task opens again: the line to copy for the AI shows over the message box, and the AI goes on in this task, answering under what you wrote, instead of starting a new one. The same holds when you write to an older task while the AI had moved on: once its own task is done or paused, it comes back to the one you wrote to.
- 同一个东西只用一个名字:界面上的“配方”统一叫 **工作流**(工作流页的卡片、保存和导出、操作记录、备份说明),“配方包”保留,它是发给别人的 .zip;AI 工作台里统一用“素材”“留言”“节点结构”“连接”“副本”“撤销”。
- One name for each thing: what the screen called a recipe is a **workflow** everywhere (the Workflows page, saving and exporting, the activity log, backups); a recipe package stays the .zip you give to others. The AI workbench says materials, message, node map and copy throughout.
- 标注窗口的“同时做成重绘遮罩”勾选框换成 **怎么改** 三选一:交给 AI 判断、只改标出的地方(其余保持原样,AMB 做好遮罩,AI 只重绘这些地方)、整张重新出图。你选的会作为你的要求写在 AI 收到的第一句,时间线里那条标注也写着“改法:…”;用“涂重绘区”笔刷时自动选中“只改标出的地方”。
- The marker's "make a repaint mask" checkbox is now **How to change it**, one of three: leave it to the AI, only the marked areas (the rest stays; Anomalous makes the mask and the AI repaints just those), or the whole picture anew. The choice is the first thing the AI reads, as yours, and the timeline row says it; painting with the repaint brush picks "only the marked areas".
- 任务列表更跟手:收起时列表也提前排好藏在左边,鼠标碰到左上角按钮(或从页面往左移到最左边缘)下一帧就滑出来;每几秒的自动刷新只重画变了的任务。点任务马上显示:看过的任务直接出来,鼠标停在任务上时先读好,再悄悄补上最新内容;展台里的图先用缩略图,显示得比缩略图大时才换原图。
- The task list keeps up: folded, it stays laid out out of sight and slides in from the next frame when the pointer reaches its button (or moves left onto the page's left edge); the regular refresh redraws only the tasks that changed. A picked task shows at once, from the copy read before or read ahead while the pointer rested on it, then fills in what changed; the stage's pictures load as thumbnails first and as the file itself only when shown larger.
- 任务标题栏的“专用标签页”改叫 **AI 改副本 ▾**:点开是带说明的二选一(改副本 / 改当前画布),不再点一下就悄悄切换;旁边的按钮叫 **切到副本**。快捷指令和关卡窗口用同样的说法。
- The header's "Own tab" is now **AI edits a copy ▾**, opening the two ways with what each means instead of switching at a click; beside it, **Show the copy**. The / commands and the Gates window say the same.
- 素材的“来源”列表:鼠标移到哪一行,哪一行就亮起来,当前选中的那项有底色和勾(之前被一条样式盖掉了);名字没被截断时不再弹白色小提示;搜索框里能用 ↑ ↓ 选、回车确定。
- The materials' Source list lights the row under the pointer and tints and ticks the one shown (a later style had hidden both); a name gets a tooltip only when it is cut short; ↑ ↓ and Enter work from the search.
- 展台入口显示面板数量;收起或查看其他栏时出现新增、更新,会亮主题色小圆点,打开查看后消失。
- The Stage button shows its panel count and a theme-colored dot for new or updated content while hidden; viewing the stage clears the dot.
- “回到最新一步”改为约 240 毫秒的短滚动,工作台刷新不会打断它;滚轮或触摸可随时接管,减少动态效果时直接定位,并补上按钮按下反馈。
- Jump to latest now uses a short 240 ms scroll that survives workbench refreshes, yields to user navigation, respects reduced motion, and gives immediate pressed feedback.
- AI 工作台新增右侧“成果”页:按步骤汇总图片,集中查看工作流、搭配、最新图片的提示词、模型下载请求和音频;复用图库的保存、参数对比与提示词复制入口。时间线上方改为简洁的成果统计,右侧按钮提示尚未查看的新成果。
- The AI workbench now collects pictures, workflows, combos, prompts, model download requests and audio in a Results side view, reusing gallery keep/comparison actions. A compact timeline summary and unseen-result badge replace the old results strip.
- AI 搭工作流支持一次添加并自动接线:`insert_after` 接回选中的下游,`wire_from` / `wire_to` 按唯一类型批量连接。正负条件歧义、多分支、已有输入冲突和循环在写入前报明;连接失败恢复原接线。成功回执包含实际连线、缺失输入与输出可达性,支持原生 Ctrl+Z 撤销。
- 现状简报新增有长度限制的流程链;节点搜索只返回少量摘要,具体参数按需读取,减少逐线调用和重复上下文。
- Canvas tools can add and wire a node in one call, safely splice a selected branch, and connect unique concrete types by node IDs. Ambiguity/conflicts/cycles are rejected before mutation; failed connections restore previous wiring. Compact receipts, bounded flow chains and brief node search reduce context.
- AI 加节点时能插在已有的线前面:`wire_to` 指向的输入原来接着线的,改接新节点,原来的来源接进新节点,比如在采样器前加 ControlNet(正向接正向、反向接反向),或在采样器和两个提示词前加 LoRA,一次调用接好。同类型的几个插口名字相同时自动配对;`insert_after` 会把 LoRA 的模型线和 CLIP 线一起接进去。
- add_node splices in front of a node: an input `wire_to` feeds that was already wired now goes through the new node, its old source feeding the new node (a ControlNet before a KSampler, positive to positive and negative to negative; a LoRA before the sampler and both prompts). Same-type sockets of the same name pair up, and `insert_after` splices every type a node passes through (a LoRA's MODEL and CLIP).
- 改参数不再靠固定的填空:`run_workflow` 和 `set_values` 都能按参数名改任何节点的参数,第三方节点也一样(比如 `{"steps": 30}`)。只有一个节点有这个参数就直接改;几个节点都有时(高清修复的两个采样器)什么都不改,列出是哪几个,让 AI 写上节点号。几个正向提示词框接进同一个采样器(分区提示词)时,`prompt` 也不再把同一句话填进所有框,而是停下来让 AI 分别写。
- Settings go by name instead of a growing list of blanks: run_workflow's and set_values' `values` set any node's setting, a third-party node's too ({"steps": 30}). The one node that has it is changed; a name several nodes have changes nothing and lists them, for the AI to name the node ({"12": {"denoise": 0.5}}). `prompt` no longer gives one text to several positive prompt boxes combined into one sampler (regional prompts).
- 一次调用加一组节点:`add_nodes` 按 ComfyUI 的 API 格式写一组节点,组里互相连,也能连画布上已有的节点(`"#12"`),AMB 先检查类型,排成一块放在空位上,接好线,整组算一步撤销。原来要十几次调用的局部重绘分支,现在一次。
- add_nodes adds a group of nodes in API form to the open canvas in one call: links inside the group, to canvas nodes as "#12" (outputs by slot or name) and wire_to into canvas nodes, checked first, laid out as one block and undone in one step.
- 图跑完后,第一个告诉 AI "跑完了" 的回复(不管是查进度还是别的工具)直接带着这次出的图,最多 4 张缩略图,每次运行只带一次。AI 不用再去取图,看完就能留言。
- The answer that first tells the AI its run ended, run_status or any other, carries up to four thumbnails of the run's pictures, once per run, so it need not fetch them with get_image_info.
- 跑完叫我:关卡窗口新增“等跑图”,默认“跑完叫我”。AI 提交后最多等 1 分钟(跑得快的当场就拿到结果),还没跑完就告诉你在跑,然后收工,不再一遍遍查进度(以前一张图要查 10 次左右)。跑完后,留言框上方有几句话,点一下复制,粘到 AI 的对话框里,它就接着看结果和图;AI 知道结果后这几句话自动消失。选“AI 自己等”就和以前一样。
- Tell AI when done: a new setting in the Gates window, on by default. The AI waits up to a minute after queueing (a quick run comes back with its pictures at once) and, if the run still goes, says so and ends its turn instead of polling run_status. When the run ends, lines over the message box are ready to copy into the AI's chat; they go once the AI knows. "AI waits" keeps the old way.
- 工具说明压短:常用工具的说明重写,去掉重复和绕的话,规则和“不要做”的都保留;开始任务时收到的规则从约 1900 字减到约 900 字。
- Shorter tool descriptions: the everyday tools' descriptions are rewritten without repeats, keeping their rules; begin_task's rules are about 900 characters instead of 1,900.
- 工具目录:AI 开始任务时,回复里附带一张按“想做的事”排好的工具目录(出图、画布、模型、提示词、图片、和你沟通……),AI 看一眼就直接去用对的工具,不用先到处试。目录从工具登记表自动生成,只列当前给 AI 的工具,新工具会自己出现。
- A directory of the tools by what the AI wants to do comes with begin_task's answer (and get_situation's while there is no task): each tool offered under its stage of the work with its title, built from the catalog so new tools show up by themselves.
- 给 AI 的话常驻在消息框上方:AI 停下来的时候(没在调用工具、没在跑图、没在等你回答),AI 工作台消息框上方总有一行话,点一下就复制,粘到 AI 的对话框里。话跟着情况变:图跑完了是“跑完了,看看结果”这几句;你在 AMB 里回复了、AI 还没收到,是“我在 Anomalous 里回复了,看看再接着做”;别的时候是“继续”。最后一句“用 Anomalous 接着做任务:…”给新开的 AI 对话用。原来挂在跑图那一步下面的几句话挪到了这里。“交给 AI”给你的那句话也不再叫 AI 用旧的 show_panel 问你。
- Lines for the AI's chat stay over the message box: whenever the AI is not in a call, a run or a question, the AI workbench offers a line to copy that fits what waits (about the run that ended; "I replied in Anomalous, take a look and go on" when it has not got your reply yet; else "Go on"), and one that picks the task up in a new chat. The lines under a finished run's step moved here. The Hand to AI sentence no longer tells the AI to ask with show_panel.
- AI 在 AMB 里问你时至少等 30 秒:就算它自己说不等(Gemini 常这样),你在 AMB 里点了选项,它在同一步里就收到,接着做,不用再回 AI 的对话框说“继续”。“加节点”工具的说明也提醒 AI:要加一组互相连着的节点(重绘、放大一段),用一次调用的“加一组节点”。
- ask_user waits at least 30 s even when the AI asks it not to, so an answer given in Anomalous right away reaches the same call and the AI goes on without a word in its chat. add_node's description points to add_nodes for several nodes wired to each other (an inpaint or upscale branch).
- 时间线更短:AI 连着做的事(查资料、加节点、连线、写提示词、改参数、记决策)三步以上、其中至少两步不是查资料时,叠成一行“做了 N 步”,下面一排小标签写做了什么(加了节点 ×5、连了线 ×10……),点开就是原来的每一步。行上有“撤销这一串”,一下撤掉这一串在画布上的改动。你的话、AI 留言、问你的、面板、出图、出错的、等你允许的、正在做的一直露在外面。AI 还在做时,这一行写着最后一步是什么。一个修手的任务从 43 行变成 18 行。
- A shorter timeline: three or more of the AI's working steps in a row (reads, canvas changes, prompts, settings, decisions; two of them not reads) fold into one row, "Did N steps", with chips of what they did and "Undo these" for their canvas changes; it opens to the steps as before. Your words, the AI's notes and questions, panels, runs, errors, steps waiting for you and running ones stay in sight. While the AI is at it, the row names its last step. A hand-fix task went from 43 rows to 18.
- 展台自动打开:AI 往展台放了新面板,右栏自己打开并跳到它,你关掉了或者在看节点结构、决策、素材也一样;AI 只是改已有的面板(比如进度)时不会。展台顶上有开关“有新东西就打开”,默认开着。窗口窄时不抢时间线,只在“展台”标签上亮个点;AI 问你问题时仍然切过去。
- The stage opens by itself: a new panel from the AI opens the side column at it, even when you closed it or were on another view; the AI changing a panel (a progress bar) does not. The switch "Open on new panels" in the stage's head turns it off. In a narrow window it lights the stage's tab instead of taking the timeline's place; a question still brings the stage forward.
- 打开右栏的按钮更醒目:时间线里每个面板那一行变成卡片,左边小图标表示里面是什么(问题、图、对比、节点结构),标题下一行写内容(“1 张图”“问你 · 2 个选项”),右边是实心按钮“在右侧打开”;右栏正在显示的那个面板,按钮换成绿点“正在右侧显示”。跑图那一步的“在右侧看大图”也换成同样的按钮。“在画布上找到”“撤销”“在图库里看”这些蓝色小字换成带图标的小按钮。
- Ways to the right are easier to see: each panel's line in the timeline is a card with an icon of what it holds, a line of what is in it ("1 picture", "Asks you · 2 options") and a solid "Show on the right" button, which turns into a green "On the right now" while that panel is in view there. A run's "See it large on the right" is the same button. "Find on canvas", "Undo" and "See it in the gallery" are small buttons with icons instead of blue text.
- 修复任务画布标签关闭或切换失败后误用当前工作流的问题:操作会明确报错,重新打开或明确指定新画布后才能继续。下载缺失模型现在也跟随任务画布;留言里的 `@` 搭配和工作流按完整名称查找,不再因模糊搜索前几项占满而误报缺失。
- Task canvas actions now fail when their recorded tab is closed or cannot be activated instead of using another workflow. Missing-model downloads follow the task tab, and message references resolve exact combo/workflow names before search limits.
- AI 工作台任务工具栏新增小巧的“运行状态”按钮与浮窗(移除遮挡时间线的常驻大横条):点开即可查看真实运行状态、排队、节点进度、用时和结果,运行中直接在按钮上显示进度百分比;没有收到运行信息时明确提示,不再空转显示“AI 正在工作”或把沉默当成完成。页面可见时自动补查状态。运行工具默认等待 15 秒并返回实际结果;外部提交可用 `track_run` 接入准确的运行 ID,不会重新出图。
- The AI workbench now places confirmed execution state, queue/node progress, elapsed time and results inside a compact toolbar button and popover (instead of a tall sticky banner), with visible-page refresh. Untracked work and silence are explicit. Run calls wait briefly for real results; `track_run` attaches external submission IDs without resubmitting.
### 右边一栏一次一样,按钮固定在右上角 / One view at a time on the right, switched from the top right
- 工作台右边那一栏一次只显示一样东西,用窗口右上角的三个按钮切换:**展台**、**节点结构**(AI 跑过的节点结构图,大图,可以选看哪一次运行)、**素材**(原来停在屏幕右边的素材小窗,现在就在这一栏里)。再点一下亮着的按钮收起这一栏;切走再切回来,内容都还在。时间线里每次运行都有 **在右侧看大图**。
- 任务标题和那一排按钮在时间线往下翻时一直停在顶上。
- 右边这一栏改成一张单独的卡片,不再用一条线硬隔开;默认窄一些,给中间的时间线留出地方。它左边的缝里有三个小点,鼠标移上去变成一条竖线,拖动调宽窄(记得住,双击恢复)。这条缝和时间线的滚动条分开,不会再一抓就抓错。
- 左边的任务列表用窗口左上角的按钮收起 / 展开;收起后鼠标移到按钮上,列表会临时浮出来,点任务或移开就收回。
- 素材在右边这一栏里时,鼠标移过卡片不再自动跳进详细说明(之前很难操作),点卡片上的 ⓘ 才看。
- 左边任务列表收起后,鼠标从按钮往下移去点任务时列表不会再中途消失(之前移得慢一点就会收回):鼠标在按钮和列表围成的范围里都算还在用,离开后才收回。
- The folded task list no longer vanishes on the way down from its button: it stays while the pointer is anywhere in the box around the button and the list, however slowly it moves.
- **模型补丁不再被当成 ControlNet**:Anima LLLite、Qwen-Image 分块 ControlNet、Z-Image Fun ControlNet 这类文件名字里常带 ControlNet,但只有「加载模型补丁」节点能读,它只看 `model_patches` 文件夹。现在拖进模型页会自动放进 `model_patches`;已经放在 `controlnet` 里的,模型页的“整理”会提示挪过去。
- Model patches (Anima LLLite, the Qwen-Image block-wise and Z-Image Fun ControlNets) are told from ControlNets by their header: imported into `model_patches`, and Tidy offers to move those already in `controlnet`.
- **AI 找模型时直接知道该用哪个节点**:`search_models` / `get_model` 给每个模型写上能加载它的节点;放错文件夹的会说该放哪、放在这里加载不了,同名文件已经在对的地方时会直接指给 AI。新工具 `move_model` 把放错的模型挪过去(新关卡“挪动模型文件”,默认先问你;从不覆盖文件)。AI 不必再去命令行翻 ComfyUI 的源码。AI 加节点时如果下拉列表是空的,会被告知“ComfyUI 没找到文件,可能放错了文件夹”。
- search_models and get_model say which node types load each model, and where a misfiled one belongs; `move_model` moves it there behind a new "files" gate (asks by default, never overwrites). An empty drop-down in add_node now says ComfyUI found no file for it.
- AI 第一次在任务的副本标签页上动手时,切换标签页不再被记成“你在画布上的改动”告诉 AI。
- The AI switching to its task tab is no longer reported to it as a canvas change of the user's.
- **工作台里直接用图库的图**:留言框左边新增图片按钮,打开图库(最新的在前,可按提示词、模型、seed 搜索),点选几张后 **加进留言**;打 **@** 时也能看到图库最新的图,打字会顺带在图库里搜。AI 收到的是图片本身和它的生成参数。
- Gallery pictures in the AI workbench: the message box's picture button picks them from the gallery (newest first, searchable), and "@" offers the gallery's newest pictures and searches it as you type; the AI gets each picture with what made it.
- **等 AI 接手时先备好素材**:“等 AI 接手”的任务是一页:最上面是 **把这句话发给你的 AI** 和 **复制这句话**,往下滚时它停在顶上,挑完图不用翻回去;下面的 **写一句要做什么**、**从图库挑图给 AI**(大图库可搜索,点一张就放到最下面的留言框,再点一下拿掉)都是可选的,点了才展开,行上写着已经放了几张;备好的都列在下面。AI 接手时一起收到。那句话改成让 AI 照任务里的交代做,没有交代才问你,不会一接手就停下来问。
- A task waiting for an AI is one page: **Send this to your AI** with **Copy** leads and stays at the top while the page scrolls; **Say what to do** and **Pictures from the gallery** (a big searchable grid feeding the message box) are optional folds that open when pressed and say how many pictures are in; what is ready is listed under them. The sentence now has the AI do what the task says, asking only when it says nothing.
- **不全屏时中间更宽敞**:AI 工作台的窗口不够高时(没有最大化),任务标题、状态、时间步数、“接着”和右边的工具并成一行,上下留白也收紧;输入框下面的提示只在出错时显示(平时鼠标停在输入框上能看到);等 AI 接手的页面图片变小、步骤靠紧。同样的窗口里,中间能多显示约两成内容;最大化时样子不变。
- In a low window (not maximised) the AI workbench keeps the task's head to one line, tightens the room around it, shows the message box's hint only for a failure (its words are the input's tooltip) and shrinks the waiting page's picture: about a fifth more room for the timeline.
- **换类型**:模型详情页多了 **换类型** 按钮,给“导入时类型选错了 / 代码认不出、放错了文件夹”的模型用。窗口里写明现在放在哪一类、文件头看起来像什么(看得出就替你选好并标“推荐”,不确定标“可能是”,看不出就不猜),每一类都写着用哪个节点加载;可以改挪到的根目录和子文件夹,下面显示完整路径。挪动和“整理”用的是同一套:封面、信息、备注一起走,从不覆盖文件,挪完 ComfyUI 的模型列表马上更新。导入单个模型后,提示里也有 **类型选错了?**,点一下直接改。
- "Change type" on a model's page, and "Wrong type?" in the toast after importing one model: the window says where it is and what its header suggests (picked and marked when it can tell, nothing guessed when it cannot), offers every type with its loader and where it goes, and moves it as Tidy does.
- 跨盘挪动、复制模型时,除了比大小,还会比对文件开头、中间、结尾各 1 MB 是否一模一样,对不上就不删原文件(只多花几毫秒,不需要事先有指纹)。
- A model copied to another drive is checked at its start, middle and end (a megabyte each) besides its size before the original goes; no fingerprint needed, no felt cost.
- 挪动、导入模型时,就算目标位置在最后一刻冒出一个同名文件,也绝不会被盖掉:这次挪动会失败,两边的文件都原样保留(之前在 Windows 上理论上有极小的机会覆盖)。
- Moving or importing a model never replaces a file that took the name at the last moment: the move fails and both files stay (on Windows a rename could replace it before).
- **AI 能放大看图、看预览,不用再自己写代码**:新工具 `zoom_image` 把一张图的某一块放大给 AI 看,还能把两张图的同一块并排比较(改之前和改之后);输出、输入、临时预览的图都能看,只看不改。出图结果里会列出这次运行的预览图(比如 OpenPose 骨骼、遮罩的预览),AI 用它们就能检查中间结果,不必自己去跑预处理器。AI 开任务、收到遮罩时都会被明确告知:要改图就跑 ComfyUI 的工作流,绝不用自己的代码去画、去改、去覆盖图片文件。
- zoom_image shows part of a picture larger (output, input or temp folder), with the same part of a second picture to compare, reading only; run_status lists a run's PreviewImage pictures; begin_task and the repaint note tell the AI to change pictures only by running workflows, never by editing picture files with its own code.
- **AI 能直接看输入文件夹里的图**:`get_image_info` 加了 `folder=input`,能看 ComfyUI 输入文件夹里的图(你标注时 AMB 存的重绘遮罩图就在那里):尺寸、要重画的区域在哪、占多大,也能附上图本身。交给 AI 遮罩图时也明说“它已经在输入文件夹里了,不用跑命令检查”,Gemini 不会再为这个弹窗问你。
- get_image_info with folder=input reads a picture in ComfyUI's input folder (the inpaint picture Anomalous writes for a repaint): its size and the area LoadImage gives as MASK, optionally attached; the marks message says the inpaint picture is in place, so the AI need not check it with a command.
- **AI 少去翻文件**:AMB 工具返回给 AI 的内容去掉了多余的空格、画布里 ComfyUI 自己的内部字段不再列出、操作记录默认只给最近 10 条,这样 Antigravity 不必把结果另存成文件再让 Gemini 去读。AI 开始任务时也会被告知:模型、图片、节点、画布和运行都有 AMB 的工具,不要自己去读 ComfyUI 的文件夹、源码或接口(那样每一步都要你点允许)。
- Tool results reach the AI as compact JSON, without the canvas's internal "$$" widgets, and recent_activity gives the last 10 by default, so apps such as Antigravity keep more of them in the chat instead of a file the AI must open; begin_task tells the AI that AMB's tools cover models, pictures, nodes, canvas and runs, so it need not read ComfyUI's folders, source or routes.
- 修复输入框上方的出图进度停在“节点 0/29 0%”不动的问题:现在它和时间线里那一步、标题栏的进度按钮一起更新。
- The run line over the message box now follows the run's progress with the timeline's bar and the header's button, instead of staying at the start.
- **最下面就能看到 AI 在做什么、要你做什么**:输入框上方多了一条“现在”栏:AI 出图时显示进度(节点、步数、百分比、进度条)和停止按钮;AI 某一步还在进行时说在做什么;AI 想做要你点头的事(比如挪模型、下载)时,直接在这里 **允许 / 不要**;AI 问你问题时,问题和选项就在这里,点一下就回答(有推荐的带 ☆,也能自己写一句或跳过),多选和挑图会带你去展台。
- A "now" strip over the message box: a run's progress with stop, the AI's step still going, a gate to allow or refuse right there, and the AI's open question with its options as buttons (one's own words or skip too; several picks and pictures to pick lead to the stage).
- **拖进来、粘贴进来的图片也能发给 AI**:把电脑里的图片拖到输入框上,或者直接 Ctrl+V 粘贴截图,它会存进 ComfyUI 输出文件夹里的 `ai_inbox`(图库里也看得到,从不覆盖文件,只收 PNG、JPEG、WebP、GIF,最大 25 MB),并加到这条消息上。从 ComfyUI 页面里拖来的输出图片直接加上,不另存。
- Pictures dropped on the message box or pasted are kept in the output folder's `ai_inbox` (seen in the gallery; never over a file; PNG, JPEG, WebP or GIF up to 25 MB) and attached; an output picture dragged from ComfyUI's page is attached as it is.
- **很长的 AI 留言先折叠**:超过八行左右的留言只显示开头,点 **展开全文** 看全部。
- Long AI notes are folded to their first lines with "Show all".
- **输入框固定在最下面,像聊天软件一样**:不管时间线多长、翻到哪里,留言框都停在任务这一栏的最下方。它上面是这条消息要带的东西:图片显示成小图,每张有 **✎ 标注**(画框、圈、写要改哪里,标好后单独发给 AI)和 **×**;@ 提到的模型、搭配显示成小标签。从图库挑的图不再往文字里塞“@文件名”。左边是 **图库** 和 **素材** 两个按钮,给了东西就亮起并显示数量。输入框会随内容变高:Enter 发送,Shift+Enter 换行;只发图片不写字也可以。往上翻了一屏以上,或者往上翻着看时 AI 有了新步骤,输入框右上方会出现圆形 **↓** 按钮(有新步骤时角上带数字),点一下回到最新;自己发出的消息会自动滚到眼前。时间线里你发的图片和 AI 出的图一样,可以查看、对比、收藏、标注。
- The message box is docked at the foot of the task column like a chat app's: over the input what the message carries (pictures with ✎ to mark up and ×, chips for what "@" names; gallery pictures are attached, not typed in), beside it the gallery and materials buttons with their counts, an input that grows (Enter sends, Shift+Enter breaks the line; pictures alone are fine), a round ↓ button back to the newest step (with the count of new ones) when you read more than a screen up or the AI adds steps meanwhile, and your own message scrolled into view. Pictures you sent carry the same tools as the AI's. The side column's grip sits in its own gap, three dots, beside the timeline's scrollbar instead of over it.
- **打开任务就停在最新一步**:像聊天软件一样,打开任务直接看到最新的进展。你停在最下面时,AI 的新步骤来了就跟着往下走,不管任务现在是什么状态(之前只在 AI 正在干活时才跟,打开任务时还会停在上一个任务翻到的位置);往上翻着看就不动你。
- A task opens at its newest step; read at the foot, the timeline follows new steps whatever the task's state (it used to follow only while the AI worked, and opened where the previous task was scrolled to); read further up, it stays put.
- **面板上的进度条不再冒充实时进度**:AI 画在面板上的进度条是它自己写的数,不会自己动。任务往前走了(那次出图结束了、AI 又画了新面板或留了言)而它没去更新,这条进度条会变成灰色,并写明“这是 AI 在几点写下的数,之后没有再更新”。出图的真实进度一直在时间线那一步、输入框上方和标题栏的运行状态里。AI 也会被告知:出图不用另画进度条,自己画的要自己更新。
- A panel's progress bar is the AI's own numbers: once the task goes past it without an update (a run ends, a later panel or note), the stage greys it and says when it was written. The AI is told a run's real progress shows already and to keep its own bars current.
- **还没出图就问进度不再算“出错”**:AI 在出图前调用 `run_status`,现在得到“这个任务还没有出图”的回答,任务不会因此标上“有一步出错”。
- run_status before any run answers state "none" instead of failing, so the task no longer shows a failed step for it.
- **宽窗口里时间线收在中间**:右边一栏关着、窗口又很宽时,时间线、标题和输入框不再横跨整个窗口,而是收成中间约 880 像素宽的一列(像聊天软件那样),滚动条仍在最右边。时间线里的图片按自己的大小从左排开,一张图不再被撑成横跨整行的大灰框。
- **AI 每一步都知道下一步做什么**:开始任务、出图和看进度、加节点连线、问你问题得到回答,这些回复里都多了一句 `next`:比如“还在跑,等 55 秒再看”“跑完了,先放大看图再告诉你结果”“还缺这几根线,照清单连”“缺模型,先查缺哪个”。小模型不擅长自己规划,但很会照着做。
- Answers of begin_task, run_workflow and run_status, the canvas changes and an answered ask_user carry next: the one thing to do now.
- **照着跑:日常出图一步完成**:AI 出图时可以直接说“用哪个存好的工作流、改什么”:`run_workflow` 带上工作流名,再带上要改的提示词、反向提示词、种子、尺寸或一张图(比如你标注过的重绘图),AMB 就在任务的副本标签页里打开它(第一次打开,之后接着用同一个标签页),填好这几处,其余照存好的样子不动,然后出图、等结果。尺寸只改起始的空白潜空间,高清修复里的 Latent 放大保持原来的放大尺寸。原来要 6、7 步(找工作流、打开、看画布、改提示词、改种子……),现在一步,小模型也不容易绕晕。填的内容和出图一起算一步,改画布那部分同样能 Ctrl+Z;如果你给“改画布”设的关卡比“用显卡”严,这一步按改画布的关卡来问你。画布报错是因为缺模型时,“现状一览”会让 AI 先查缺哪个模型,而不是乱换选项。
- run_workflow takes blanks for a daily run in one call: workflow (a saved one, opened in the task's tab once and reused), prompt, negative, seed, width, height and image (image_node for several LoadImage nodes); the rest stays as saved. The size goes to the latent a run starts from, never to a hires fix's latent upscale. With blanks a run asks as the stricter of the GPU's and the canvas's gate.
- **现状一览,AI 一次就知道到哪一步了**:新工具 `get_situation`,AI 调一次就拿到:任务(你最后说了什么、有几个问题在等你、定了哪些决策、出过哪些图、“AMB 之外”几次)、画布简报(每个节点一行:谁接到谁;**还缺哪几根线、各要什么类型、画布上哪些现有输出能接上**;ComfyUI 报错的节点)、最近一次出图,以及**下一步该做什么**。回复控制在约 3600 字以内,Antigravity 不会把它存成文件再让 AI 去读。新开对话或拿不准时先调它;连接说明里的那句话和规则也写上了。小模型(比如 Gemini Flash)不用再把整张图记在脑子里。
- `get_situation`: the task, the canvas in brief (a line per node, the inputs still unwired with the outputs that fit, ComfyUI's errors), the latest run and the next step in one answer under about 3,600 characters, for a new chat or an AI that lost track.
- **AI 每连一根线都知道还缺什么**:`add_node`、`connect_nodes`、`remove_nodes` 的回复里附上画布上还没接的输入,以及哪些现有输出能接上,像一张清单,照着补完就能跑。
- add_node, connect_nodes and remove_nodes answer with the inputs still unwired and what fits them.
- **AMB 之外做的事也看得见**:AMB 只看得到 AI 发给它的调用,AI 软件自己跑的命令和脚本它看不到。现在 AI 每调用一次 AMB,AMB 就看一眼输出和输入文件夹:上次调用以来有没有不是出图产生的新图片、有没有图片被原地改写(比如之前那张被覆盖了 6 次的图)。有的话,时间线上多一行橙色的 **AMB 之外**,列出是哪些文件,输出文件夹里的图直接显示;AI 同时收到提醒:可能是你自己做的,如果是它做的,改图只能跑工作流。只在 AI 调用时看,不在后台扫描;ComfyUI 正在出图时,新图等出图结束再判断,不会把正常出的图当成外面来的。AI 也多了一个工具 `log_outside`:用了别的工具(命令、脚本、读写文件)后说清做了什么、为什么、动了哪些文件,同样显示为一行 **AMB 之外**;任务的统计里会写“AMB 之外 N 次”。
- Pictures changed outside Anomalous show in the timeline: when the AI calls, Anomalous looks at the top of the output and input folders for pictures no run saved and pictures rewritten in place since its previous call, and tells the AI with its answer; new pictures wait while ComfyUI runs a prompt. `log_outside` is the AI's own word on a command or script it ran. Both are "Outside AMB" steps, counted on the task.
- **连接说明里多了一段给 AI 的规则**:第 ④ 步可以复制一段规则(先用 AMB 的工具、只通过跑工作流改图、在 AMB 里问和说明、用了别的工具就说一声),放进 AI 软件的“规则”里(Antigravity:Customizations → Rules → + Global),它每次对话都会读到。
- The connect guide has rules to paste into the AI app's own rules: use AMB's tools, change pictures only by running workflows, ask and explain in AMB, and say what was done outside it.
- In a wide window the task (its head, timeline and message box) keeps to a centred column of about 880 px, as chat apps do; the scrollbar stays at the edge. A step's pictures are cards of about their own size from the left, no longer one stretched across the row.
- **点得到、看得见**:选中的图片有粗框、发光和大对勾,图片罩上一层颜色,和鼠标停留一眼分得开;留言框左边的图片按钮在图库面板打开时亮起,留言里有图时显示张数。素材列表、AI“看了资料”那一行(带 › 可展开)、关卡选项、优先用 / 只能用、来源下拉、发送和复制按钮、模型页的标签、首页的“一键准备”、记录里可展开的一行,鼠标移上去都有明显的变化。点 **展台** 时即使还没有内容,也会打开并说明会放什么,不再没反应。“交给 AI · 0”在什么都没选时不能点。挪模型那一步的标题写文件名和挪到哪,不再是一串编号;只改了一个节点时不再出现两个“在画布上找到”;“接着哪个任务…”的字号和旁边一致。
- Clearer feedback: picked pictures get a thick glowing frame, a tint and a big tick; the picture button lights while its panel is open and counts the message's pictures; materials rows, the folded reads row (now with a ›), gate options, the mode switch, the source menu, send and copy buttons, the models page's tabs, the home page's set-up button and expandable activity rows all react clearly to the pointer. The Stage button opens an empty stage that says what will appear there; "Hand to AI · 0" is disabled with nothing picked; move_model's step names the file and where it goes; a one-node canvas change shows one "find on canvas"; the "Goes on from…" chip matches the size around it.
- 右边那一栏的宽度把手更好抓了:卡片左边整条空隙(从上到下)都能拖,把手平时就淡淡地显示,鼠标靠近时整条边亮起;点一下后也能用 ← → 调宽窄。
- The side column's grip catches the pointer across the whole gap left of the card, top to bottom, shows faintly at rest and lights the edge when pointed at; focused, ← → resize it.
- **节点结构的简单看法**:除了完整的方块图,现在默认是一张一眼能看懂的说明:每一段采样用哪个模型加哪些 LoRA(带权重)、正负提示词、从空白画布还是从上一段的结果开始(用哪个放大模型、缩放多少),最后保存在哪;和上一段一样的就写“和第 1 段一样”。点任意一项定位到画布上的节点;“简单 / 完整”随时切换,AMB 记得你的选择。
- The run map has a Simple view (now the default): per sampler, the model and LoRAs feeding it, its prompts, where it starts (an empty canvas, a picture, or the part before, upscaled) and where it is saved, "same as part N" when nothing changed; Simple / Full is kept on this browser.
- **删掉任务里的图以后**:时间线、本次成果、展台上的那张图显示成“图片已删除”的占位(鼠标停上去说明可在回收站找回),不再是破图,也点不开;@ 列表里不再出现它。AI 那边会收到“你删除了这张图,别再用它”;它要是还去读,会被告知“可能已被用户删除”。
- A task's pictures deleted since show as "Picture deleted" instead of a broken picture, and "@" no longer offers them; the AI hears plainly that the user deleted one, and a tool reading a gone picture says it may have been deleted.
- **关联任务**:从一个任务新开的任务会记住它“接着”哪个任务(AI 开的新任务、你看着某个任务时点的“新任务”都会)。标题下面显示“接着:…”“后续:…”,点一下跳过去;任务列表里后续任务带 ↳。可以解开,也可以自己选“接着哪个任务…”。AI 接手后续任务时会先收到上一个任务的决策。
- Tasks remember the task they go on from (an AI's new task while on another, or one the user starts from a task): links under the title lead both ways, the list marks follow-ups, the user can link or unlink, and an AI beginning a follow-up hears what was decided before.
- **决策记录**:聊天在哪里都行,结论留在 AMB。AI 用新工具 `log_decision` 把商量好的结论记成时间线上的 📌 卡片(定了什么、为什么、没选哪些、谁定的、跟哪几步有关);你在 AMB 里的选择(面板回答、允许或不允许、交出去的素材、AI 改画布的地方、停止)自动算作决策。右上角新增 **决策**,按时间列出全部决策,点一下回到那一步;AI 很久没记录时会轻轻提醒。AI 重新接手任务时会先收到这些决策。
- Key decisions stay in Anomalous: the AI records what was settled with `log_decision` (a card on the timeline: what, why, what was left out, who decided, the steps it is about); the user's own choices in Anomalous count as decisions too; a Decisions view in the side column lists them all with a link to each step, notes work done since the last one, and begin_task repeats them to an AI picking the task up.
- **留言框里的 @ 和 /**:打 **@** 弹出能搜索的列表(这个任务的图、搭配、模型、工作流、提示词、节点包、节点),选中的写成“@名字”并在下面显示成小标签,AI 收到的是确切的那个东西(图片连同生成参数一起)。开头打 **/** 是快捷指令:停止、撤销、展台、节点结构、素材、副本、当前画布、新任务马上生效;重跑、解释、继续帮你写好一句话。
- "@" in the AI workbench's message box names the task's pictures, combos, models, workflows, prompts, node packs and node types; the AI gets each in full with the message (pictures attached). "/" lists commands that act at once (stop, undo, side views, new task, canvas mode) or fill in words to send.
- **整个自定义节点包交给 AI**:素材里点“节点”,再在来源那一排点某个自定义节点包,上面会出现“整包交给 AI”。不知道该选哪个节点时,把整个包给它,AI 自己从包里挑;选“只能用”时,这个包里的节点都允许,别的第三方和付费节点照样拦下。AI 用 `get_toolbox` 能看到包里有哪些节点,`find_nodes` 搜索时也把它们排在前面。
- A whole custom node pack can be picked (from its source in the nodes' second column): the AI gets its node types in get_toolbox, finds them first, and with "only" every node of the pack is allowed.
- 小窗里的“来源”不再是一条要横着拖的长条,改成一个“来源:全部 ▾”按钮:点开是一张列表(来源多时上面有搜索框),点一个就选上并收起;Esc 或点别处也收起。
- In the side panel the sources (or types) are one "Source: All ▾" button opening a list to pick from, with a search when it is long, instead of a strip to scroll sideways.
- 素材在右边这一栏(小窗)里时改成一行一个的列表:左边是图标或小封面,中间是名字和来源,右边是勾选;搜索框和“全部 / 已选 / 常用”并成一行。同样的高度能看到的条目多了好几倍;展开成大窗口还是原来的卡片。
- In the side panel the materials picker lists one line per item (badge or small cover, name, where it is from, the tick) and puts the search and the filters on one line; the large window keeps the cards.
- AI 工作台和素材窗口里的滚动条变细,平时看不见,鼠标移到能滚动的那一块上才显出来。
- Scrollbars on the AI page and in the materials picker are thin and show only while the pointer is over what scrolls.
- The side column is a card of its own, narrower by default and resizable from its left edge (kept; double-click resets). The task list folds from a title-bar button and peeks over the page while pointed at. In the docked materials picker, hovering a card no longer opens its detail; its ⓘ does.
- The AI workbench's side column shows one view at a time, switched from the window's title bar: the stage, a run's node map at full size (pick the run), or the materials picker docked there. Pressing the shown one hides the column; each view keeps its state. Each run's step has "See it large". The task's header stays on top while the timeline scrolls.
### 回答发出去了没有,一眼看清 / See at a glance whether the AI has your answer
- 面板上每个回答、你的留言、批注、发给 AI 的图片下面都有醒目的状态标签:琥珀色 **已发送 · 等 AI 来取**,AI 拿到后变成绿色 **AI 已收到 · 时间**(取代原来那行灰色小字)。鼠标停在“已发送”上会提示 AI 停下时该怎么办。
- 面板上的问题可以点 **这题跳过**:可选的问题不再一直显示“等你回答”,AI 会收到“你跳过了”,自己拿主意。
- Each answer, message, set of marks and picture handed to the AI shows a pill: "Sent · waiting for the AI", then "The AI has it" with the time. Panel questions can be skipped; the AI reads `skipped: true`.
### 看得见 AI 跑了什么、AI 改副本、图片直接给 AI / What the AI ran, a copy for the AI, pictures straight to the AI
- 每次运行下面都有 **节点结构**:一行写明底模、几个 LoRA、尺寸、步数,点开是方块图(LoRA 带权重、正负提示词、采样器设置……,点方块定位到画布上的节点),还用黄色小块标出 **比上次改了什么**,比如“步数 20 → 30”“LoRA 权重 0.65 → 0.8”“新增「加载LoRA」”。图是 AMB 按 ComfyUI 实际收到的工作流画的,不靠 AI 描述;AI 讲解时用新工具 `show_run_map` 一次把它放上展台。
- **AI 改副本**(默认开):AI 第一次改画布前,AMB 把你正开着的工作流复制到以任务命名的新标签页(副本),之后它只在副本里改和运行;你切去别处,它下次动手前会切回来;你原来的工作流一笔不动。任务标题栏的“AI 改副本 ▾”能换成“改当前画布”,“切到副本”跳过去;关卡窗口里设定新任务的默认做法。
- **图片直接发给 AI**:图库右键“发给 AI”或看图窗口的 ✦,可以发给 AI 正在做的任务(也能开新任务),可附一句话。AI 收到图片本身和从图里读出的模型、LoRA、提示词、尺寸、采样设置。AI 搜图库时也能让结果直接带上前 6 张图。
- Each run shows the shape ComfyUI actually ran (boxes, links, settings; boxes locate their nodes) and what changed since the run before; `show_run_map` puts it on the stage. By default the AI works in a tab of the task's own: the open workflow is copied before its first change and brought back before later ones, the user's tab untouched (per task, and a default in the Gates window). Output pictures can be handed to the task an AI is on (gallery menu, inspector): the AI gets them attached with what made them; `search_images(with_pictures)` attaches the first six.
### 运行时看得更清楚,也能一键停 / See more of a run, and stop it in one click
- AI 运行工作流时,进度条旁边多了红色的 **■ 停止**(任务工具栏的运行状态按钮旁边也有):一点就停下这一次运行,排队中的移出队列、正在跑的立刻中断,只动这次运行。时间线上记着“你停止了运行”,AI 也会收到,不会自己再跑一遍;AI 自己也能用新工具 `stop_run` 停掉自己的运行。
- 进度条下面多了当前节点**约剩多久**、每步几秒、**显存还剩多少**,以及 ComfyUI **控制台**最新的一行(比如放大模型加载、`Model Initializing ...`),点开看最近几行。AI 收到的进度里也有这些,不用再去翻控制台。
- 出图一排上队,AMB 就读这次工作流,算出每个采样器要在多大的图上采样。放大模型放大后又整张重新采样、尺寸超出这张显卡的舒适范围、要解码特别大的图时,会出现黄色提醒,说清可能很慢或爆显存,以及怎么改(只想放大就单用放大模型,或先缩小、减步数、换分块解码)。AI 收到同样的提醒,并被要求先问你要不要停。
- 运行状态浮窗开着时按 Esc 只关浮窗,不再把整个 AMB 窗口一起关掉。
- A stop button on a running run (and beside the run-status button): only this run's prompts leave the queue or are interrupted; the step ends "cancelled", the AI hears that the user stopped it, and `stop_run` lets the AI stop its own. Runs show time left and per step, VRAM, and ComfyUI's console (the bar's note, models loading), for the user and the AI alike. Once queued, each sampler's working size is estimated from the prompt, with warnings for an upscaled picture sampled again, a size past the card or a huge decode. Esc closes the run-status card only.
### 素材面板的侧边栏小窗 / The materials picker as a side panel
- “素材”默认在屏幕右边打开成侧边栏小窗,不挡工作台和画布:种类是一排标签,卡片两列,点卡片在小窗里看说明。右上角 ⤢ 展开成原来的大窗口,大窗口里也能收回侧栏;AMB 记得你上次用的哪种。
- The materials picker opens as a side panel by default (the page stays usable), and expands to the large window and back; the last choice is remembered.
### AI 在画布上搭工作流 / The AI builds workflows on the canvas
- AI 可以在你眼前的画布上搭工作流了:查这台 ComfyUI 有哪些节点(包括自定义节点)、每个节点有哪些输入输出和可选项;新建一个工作流标签页(空白的,或者一次搭好整套并自动排好位置,不动你正在开的那个);加节点、连线、改参数、删节点。每一步都在画布上当场发生,是一次 Ctrl+Z,记在操作记录里,并在 AI 工作台那一步下面显示改了什么,连线也会写成“某节点的输入 model:(空)→ 加载模型 #4 · MODEL”。连不上的类型、不存在的选项、写错的参数会先被拦下并告诉 AI 正确的写法,画布不会留下半截。新建的标签页名字带时间,不会打开或覆盖你存过的工作流。连线会绕成死循环时也会先拦下。运行被 ComfyUI 拒绝时,AI 收到的是 ComfyUI 给的具体原因(哪个节点、什么问题),不再只有一句“运行失败”。
- The AI can build workflows on the canvas the user sees: look up this ComfyUI's node types (custom nodes too) with their inputs, outputs and choices; open a new tab, blank or built and laid out from a workflow in API form (the open one is left alone); add, wire, set and remove nodes. Each is one Ctrl+Z step, logged as the AI's and shown under its AI workbench step, rewiring included. Wrong types, choices and settings are refused with what fits, before anything changes. Wiring that would loop is refused too, and a run ComfyUI rejects reports ComfyUI's own reason.
### 局部重绘:涂一涂就交给 AI / Repaint an area: paint it, hand it to the AI
- 标注窗口多了“▧ 涂重绘区”笔刷(粗细可调):涂上要重画的地方(框、圈、画笔圈的地方也算),选“只改标出的地方”发给 AI,AMB 就在 ComfyUI 的 input 文件夹里做好一张带遮罩的原图(`anomalous_inpaint_….png`),AI 用 LoadImage 读进来就能局部重绘;AI 同时收到怎么接、重绘强度怎么选的说明。时间线上你的标注下面写着“已做成重绘遮罩”。AI 搭工作流时如果刚生成的文件还不在下拉列表里,AMB 会自动刷新一次列表。
- The marker's repaint brush and "only the marked areas": the picture with the painted area as its alpha goes to ComfyUI's input folder for LoadImage, and the AI is told how to inpaint with it. Canvas tools refresh ComfyUI's lists once when a choice is missing.
### AI 的问题在 AMB 里问 / The AI asks in Anomalous
- 你在图上标注或给 AI 留言后,AI 的分析和修改方案也回在 AMB 里:它的留言会写明“回复第几步”,你那条标注下面出现“AI 已回复 · 第几步”,两边点一下就互相跳过去。AI 的留言现在有排版:小标题、列表、加粗、`代码` 都会正常显示(只认这几种写法,不会运行网页代码)。AMB 告诉 AI:收到标注和留言就在 AMB 里回复,要你选修法就用选择题问。
- The AI replies in Anomalous: `report` takes `reply_to` (the user's message or marks), both steps link to each other; its notes are laid out (headings, lists, bold, code) with text only.
- AI 接了 AMB 的任务以后,要你做选择(比如“先测哪种题材?”)不再弹在 AI 软件的对话框里,而是出现在 AI 工作台(展台上有提示,你不在工作台时右下角也会提醒):几个选项按钮,AI 推荐的那个标着“推荐”,每个选项可以带一句说明;下面还有一行“都不是?写下你的想法”。你点了或写了,AI 在同一步里马上拿到答案,接着干活。AMB 告诉 AI 的说明里写明了:干 AMB 的活时,问题都这样问。
- New `ask_user` tool: a question with options (one may be marked recommended) and a line for an answer of the user's own, shown in the AI workbench; the AI waits for the answer in the same call. The server instructions tell AIs to ask this way, not in their own chat.
### AI 工作台换新外观 / A new look for the AI workbench
- 左边任务栏:“新任务”成了醒目的主按钮,“连接 AI”和“关卡”变成带图标的按钮;下面写着最近连接的是哪个 AI、什么时候;有任务在进行、等你回答或出错时,上面出现对应的筛选按钮和数量;每个任务前的状态换成清楚的图标,选中的任务左边有一条色条。
- 任务标题栏:状态标签带图标;“素材”“全部撤销”“显示展台”是统一样式的按钮,“改名”和“从列表移除”换成铅笔和垃圾桶图标(鼠标移上去有说明,删除悬停时变红)。
- 等 AI 接手的任务:左边一张插图,右边标题;下面是三个带编号、连成一条线的步骤(想让 AI 做什么 → 把这句话发给 AI,大号“复制这句话”按钮 → AI 的每一步会出现在这里,附淡淡的示意);最下面“第一次用?先把 AI 连上 AMB”是一张带插图的卡片,点“查看连接方法”就在原地展开。没有任何任务时的欢迎页也换成插图加同样的卡片。插图是用 ChatGPT 画的。
- The task list leads with a New task button, icon buttons for Connect an AI and the gates, which AI came last, quick filters by state and state icons; a task's header uses icon buttons; a waiting task shows a picture, three numbered steps and a connect card that opens in place; the empty page matches. Illustrations made with ChatGPT.
### 给 AI 的素材 / Materials for the AI
- AI 工作台每个任务上多了“素材”按钮:按来源分组列出这台 ComfyUI 的全部节点(自带的、你装的每个第三方节点包、只在网页里的小工具、会花积分的付费 API 节点单独放在最后),还有你的搭配、工作流、提示词和模型。勾选这次想让 AI 用的,点“交给 AI”;可以选“优先用这些”或“只能用这些节点”(自带节点照常可用,AI 想加别的第三方或付费节点会被拦下)。☆ 设为常用后,没挑素材的任务自动带上常用的。AI 下一步就会收到,时间线上也记着你给了什么、AI 收到没有。
- AI 能在新标签页里打开你存的工作流,在它的基础上改;工作流详情页也有“✦ 交给 AI”。AI 查节点时会标出你挑的节点和付费节点。
- 素材窗口重做:大窗口,顶部一个搜索框搜所有素材(按 / 直接搜),左边分类显示各自挑了几个、搜到几个;节点可以按来源筛选,每个第三方节点包有自己的彩色徽标;节点卡片画成一个迷你节点,输入输出的颜色和 ComfyUI 画布上一样,写着有几个参数;搭配、工作流、模型显示封面;提示词显示原文,并且能直接挑搭配和工作流里的提示词。鼠标移到卡片上,右边显示完整说明(节点的每个输入、参数、输出,提示词全文)。下面一排是挑好的素材,带缩略图,可以一个个去掉;“优先用 / 只能用”做成了开关。图标和节点包徽标是矢量画的,跟着深浅色主题变;空白时的工具箱插图,以及没有封面的搭配、工作流、模型用的占位封面,是用 ChatGPT 画的透明背景插图(每张几十 KB)。
- 打开不用等了:节点列表直接用网页里已有的节点信息,其他素材一次读完、提前读好、留一分钟,打开窗口时先显示留着的,再在后台刷新。
- 实测:你装的 10 个第三方节点包(Impact Pack、IPAdapter、Easy-Use、Comfyroll、rgthree、ControlNet 预处理器等)共 1642 种节点,AI 都能放到画布上;用 Comfyroll 节点搭的工作流真实跑出了图。
- A Materials button on each AI task: every node type of this ComfyUI grouped by source (built-in, each custom node pack, page-only helpers, paid API nodes last), plus combos, workflows, prompts and models; picks go to the AI as prefer or only-these-nodes (built-in nodes stay allowed), favourites stand in for tasks without picks. The AI can open a saved workflow in a new tab; a workflow's page can hand it to the AI.
### AI 后端修复 / AI backend fixes
- 出图结果按本次提交返回的准确运行 ID 归属,不再通过全局队列差集猜测;其他 AI、其他页面或后续手动排队不会混入。批次只提交成功一部分时,保留这些运行的进度和图片,并明确报告未全部提交;不会自动重跑整批。旧页面需刷新后才能接手出图,不支持请求 ID 的旧版 ComfyUI 前端会在提交前提示更新。同一页面正在提交时另一 AI 收到忙碌提示,待提交结束即可重试。
- Workflow runs now follow exact submission IDs in batch order. Partial submissions keep accepted runs/results and report the missing portion; no automatic resubmission. Stale pages cannot claim runs until refreshed, unsupported host frontends fail before queueing, and a second AI waits while the same page is submitting. Native widget/seed behavior is preserved and later manual batches do not delay the AI receipt.
- “允许一次”只用于一次调用,不会被相同参数的后续调用重复使用;无法建立确认记录时不会直接执行。修复确认超时后用户刚好回答、工具返回又覆盖答案的问题;重启后的待确认步骤仍能把后续答案交给 AI,回答也会唤醒正在等待的 AI。
- 指定任务或会话读取留言时,不再取走同一 AI 客户端其他任务的留言、标记和回答。
- 没找到排队记录、无法读取运行记录、跟踪超时均明确标为未完成,不再误报完成或永久显示运行中;批次内出错及时记录,批次进度保留总数。跟踪停止不会取消 ComfyUI 实际运行。
- 标图的校验和保存移到工作线程;跨盘图片路径返回校验错误,错误类型的 MCP 工具名返回协议错误。
- One-shot approvals cannot be reused, missing approval records fail closed, and timeout/restart races preserve user decisions and wake waiting readers. Task/session input stays isolated. Missing or unreadable run history and monitoring timeouts no longer imply success or leave receipts running forever. Marked-image validation/storage runs off the event loop; malformed image paths and tool names return controlled errors.
### AI 工作台与插件能力融合 / Integrated task results
- 工作台的出图结果和 AI 展台图片:鼠标移到图片上,右上角出现 ⓘ 图片与参数、⇄ 和当前画布对比、☆ 保存为(同一个保存菜单存成工作流、搭配和提示词),不再在每张图下面排三个大按钮。左上角可以勾选:勾两张(不同步骤、不同任务都行),页面底部出现“查看参数差异”,用插件现有的参数对比看真实差异;勾第三张会替换最早那张。“本次成果”直接显示缩略图,不用再点开。视频和临时文件没有这些参数操作,缩略图按需加载。
- AI 干活时你在 AMB 里做的改动(画布改动、改模型信息、存搭配和提示词等)会作为“你在 AMB 里改了”出现在同一条时间线上,画布改动附带修改前后;AI 下一次调用任何工具时会收到这条改动,并被告知保留它,不要改回去;时间线上写着 AI 是否已经知道。操作记录页里 AI 做的、以及 AI 干活期间你做的记录,旁边有“AI 任务 →”可以跳到对应任务。
- ComfyUI 在 AI 某一步进行到一半时重启,这一步不再永远显示“正在执行…”,而是标成没有结束。
- **AI 软件里直接显示 AMB 卡片**(MCP Apps,Claude、ChatGPT、VS Code 等支持的软件):AI 开始任务、出图、画面板、留言时,对话里会出现一张 AMB 卡片,显示它在做哪几步、最新的几张图,和它问你的问题。在卡片里点选项就算回答,AI 马上收到并接着做。为了安全,“关卡”只能在 AMB 里点允许,卡片里只提示你去 AMB。Gemini 命令行和 Claude Code 终端版不显示卡片,照常用 AMB 的工作台。
- **在图上标给 AI 看**:AI 工作台里的图片,鼠标移上去多了一个 ✎。点开后可以用框、圈、箭头、画笔(红黄青三种颜色)在图上标出要改的地方,每个标记写一句“这里要怎么改”,再写整体的想法,点“发给 AI”。AI 下一步就会收到这些说明和这张标好的图(能看图的 AI 能直接看到标在哪),时间线上也会显示你标的图和说明。标好的图存在 AMB 自己的记录里,不会混进图库。
- **运行进度 AI 也看得见**:AI 让工作流开始运行后,时间线上那一步下面有进度条:排队时写“前面还有几个”,运行时写正在跑哪个节点、第几步(比如 采样器 14/28)和节点做完了几个。AI 多了一个 `run_status` 工具,可以随时问进度,也可以等到跑完(每次最多等 55 秒),跑完直接拿到出的图;运行的回复里也会提醒它用这个,别凭时间猜有没有跑完。进度条右边显示百分比(和 ComfyUI 浏览器标签上的一样:正在跑的节点走到第几步)。AI 不用自己去问:运行期间它调用任何工具,回复里都附带一句当前进度(比如“采样器 14/28,50%,节点 3/9,已经 42 秒,还没跑完”),跑完后的下一次回复会告诉它跑完了、出了哪几张图,只说一次。
- **提示词改动一眼看清**:AI(或你自己)改了提示词时,记录里先列出“去掉 N 个”(红色划掉)和“加上 N 个”(绿色)的词,点“看词义”用 AMB 的翻译在每个词下面显示中文意思;“看完整的修改前 / 后”里,两段全文中变化的词也标了颜色,没变的变暗。其他文字改动会把不同的那一段标出来。操作记录里提示词改动最多保留 2000 字(以前 400 字),整段提示词都能对比,不会把被截断的半个词算成改动。
- **等 AI 接手的任务**重新排版:中间一个输入框写想让 AI 做什么(就是任务名),下面是要发给 AI 的那句话和“复制这句话”,再加三个小步骤;第一次连接 AI 的说明收进“AI 还没连上 AMB?”里,不再整屏铺开。等 AI 的时候不显示留言框。
- **交给 AI**:模型页、图片详情(右上角 ✦)和搭配卡片上多了“交给 AI”。写一句想让 AI 做什么(已经帮你填好,比如“用这个模型出一张图”),AMB 会建好一个带着这个模型、图片或搭配的任务,再给你一句话贴给 AI;AI 接手时会直接拿到它们,不用再自己去找。
- **斜杠命令**(MCP prompts):在 Claude 等软件里可以直接选“在工作台开一个任务”“出几张让我挑”“补齐缺的模型”“讲讲现在的画布”,AI 会按适合在工作台里看的方式去做。
- **流程图**:任务标题下面一排“模型 › 搭配 › 提示词 › 运行 › 挑图”,按 AI 做事的先后排列,每段标着做完了、正在做、等你点头或出了问题;点一下跳到时间线上那一步。
- **关卡**:任务列表上方的“关卡”里,改画布、用显卡出图配音、下载模型、联网扫描四类事,各自可以选“直接做 / 先问我 / 不许做”(默认只有下载要先问)。选了“先问我”,AI 的这一步会停在时间线上,列出它要做什么,等你点“允许”或“不要”(可以写上原因或想怎么改,AI 会收到);勾上“以后直接做”就不再问。你不在工作台时右下角会提示“AI 在问你”。AI 等不及先走了也没关系:你回答后它下一步会收到,允许的话再做一次,不会问第二遍。
- 画布操作直接显示记录中的“修改前 / 修改后”、新增或删除的节点;与操作记录页共用展示,保留定位和适用时的撤销。旧记录缺失时明确提示,不用当前画布猜历史。任务页更新隔离迟到请求,并整理了结果卡片、时间线和窄布局。
- 修复参数对比窗口遮挡撤销提示的问题,撤销按钮现在可直接点击。
- Task outputs and AI panel pictures now use the existing image inspector, Keep menu and image/canvas parameter comparison, as tools over each picture on hover; tick any two pictures (any step or task) and compare their settings from the bar at the foot of the page. The results strip shows thumbnails straight away. Changes you make in Anomalous while an AI works join its timeline, and the AI is told of them (to keep, not undo) with its next call; activity entries link to their AI task. Steps a restart interrupted are marked as not finished instead of running forever. In apps that support MCP Apps (Claude, ChatGPT, VS Code…) an Anomalous card in the conversation shows the task's steps, latest pictures and the AI's questions, answered with a click (gates are only allowed in Anomalous). Four ready-made prompts appear as slash commands. "Hand to AI" on a model's page, an image's inspector and a combo card starts a task carrying it, with a sentence to send the AI. A flow over each task shows its stages (models, combo, prompt, run, pictures) in order with their state. Gates: changing the canvas, using the GPU, downloading and scanning online can each go ahead, ask you first or never happen (downloads ask by default); a step that asks waits on the timeline with what it would do, for Allow or No (with your reason for the AI). Canvas changes share recorded before/after receipts with the activity page. Unavailable history is explicit; stale task responses cannot replace a newer selection. Refined result cards and timeline styling, and fixed Undo toasts being blocked by the comparison overlay.
### Fixes / 修复
- AI 工作台与 MCP 调用实时生命周期及消息送达:每个工具调用开始时立即写入“正在执行…”步骤并接手等待中的任务,完成后原位更新同一步骤(不再等到耗时操作结束后才显示,也不会重复生成步骤);用户在工作台留下的消息和展台面板回答,除了可通过 `get_answers` 读取外,还会在 AI 下一次成功调用任意工具时自动附带送达一次,并立即在界面上更新为“AI 已经看到”;多会话与显式 `task` ID 互不串任务。
- Live in-flight MCP step lifecycle and automatic delivery of user messages and panel answers: every tool call now starts a `"running"` step immediately and updates the same step in place on completion or failure; unread user messages and panel answers are piggybacked once onto the next successful MCP tool response (in addition to `get_answers`) and immediately notify the workbench UI; concurrent sessions and explicit `task` ids stay isolated.
- 展台内容按出现顺序保留:`update_panel` 新增的成图、说明等接在已有内容下面,即使 AI 把整张看板按“结果在前”重发,也不会倒置旧内容。省略旧内容不再将它删除;重复提交不重复追加。明确指定积木 ID 时仍可原位更新进度、问题或修正内容,已有回答保留。超出 30 块时提示新开面板,不截掉旧内容。旧记录保留其已保存的排版。
- Panel updates append new output after what the user already saw, preserve omitted blocks, and avoid duplicating unchanged resubmissions. Stable block ids allow deliberate in-place edits; invalid or over-limit updates leave the panel untouched. Historical records keep their saved layout.
- 展台右上角增加“关闭展台”,关闭后时间线展开;任务标题旁的“打开展台”或时间线里的面板入口可重新打开,保留面板和未提交的输入。当前任务更新不会强行重开。面板编号改为“来自第 N 步”,明确它的创建位置。The stage can be closed and reopened without losing panels or drafts; step labels now say “From step N”.
- AI 工作台:AI 先查询再调用 `begin_task`,现在会继续已接手的任务,保留用户标题和之前的步骤;重复调用不会拆任务。另一件事可用 `new_task=true` 明确新建,已结束或长时间闲置的任务仍分开记录。
- 展台打开历史任务时定位到底部;新增面板、更新最新面板及图片加载后继续跟随底部。往上翻看时保留位置,用“查看最新”回到底部,时间线的定位链接仍跳到对应面板开头。
- The workbench keeps preliminary reads and repeated `begin_task` calls in the active task; `new_task=true` explicitly starts separate work. The stage opens at the bottom, follows updates and loaded images while there, and preserves the reader's position when browsing earlier content.
### ✨ AI workbench: see everything the AI did, open it, undo it (AI 工作台)
- **A new page on the rail, *AI workbench***: every step an AI app (Claude Code, Cursor, Cherry Studio… connected through Anomalous's MCP) takes is kept, grouped into **tasks**. The AI names a task when it starts (a new `begin_task` tool); otherwise one app's calls that follow each other make one task.
- **A task is a timeline**: what it looked up (folded into one line, a click shows each), what it changed on the canvas, the workflow runs, downloads, scans and spoken lines, and the AI's own notes (a new `report` tool, also for pictures it made another way). Each step says when, how long it took, and why it failed when it did. Steps are **numbered in order** (1, 2, 3… down the line), each dot in its kind's colour; while the AI works, the next number waits at the foot.
- **What it made stays, and leads there**: the pictures a run made are kept with the task (watched until the run ends) and shown on top as *What it made*; a click opens the picture, a model opens its page, a combo opens the combo, a spoken line plays, a canvas change is found on the canvas or **undone** (also all of a task's canvas changes at once, stopping at anything you changed since).
- **Live**: a task the AI is still working on grows by itself and says *The AI is working…*; the page follows the newest task unless you picked another. While you are elsewhere, the rail's *AI workbench* gets a dot. Home gets *What the AI did lately*.
- **New task (新任务)**: start a task yourself (with what the AI should do, or empty): it waits for an AI (*Waiting for an AI*), and the next AI app that calls takes it over, so its work starts there instead of joining its last task.
- **Connect an AI (连接 AI)**: always beside the task list, and on an empty page or a waiting task: Anomalous's address, a sentence to copy to your AI so it adds the server itself (and starts on the waiting task), and the way for Claude Code, Gemini CLI, Cursor, VS Code and Cherry Studio.
- **Write to the AI from the workbench** (给 AI 留言): a box under a task's timeline. The message becomes a step of the task (*You*) and reaches the AI the next time it asks (`get_answers`, which now also brings the user's messages, and with no panel named waits for one); it says whether the AI has seen it. Anomalous cannot wake the AI: when it has stopped, the box says to ask in the AI app.
- The task list is grouped by day (today, yesterday, a date), each task with an icon for where it stands: working ●, waiting for you ?, a step failed !, done ✓.
- **The stage (展台)**: the AI's panels are shown in a column on the right, beside the timeline, so they stay in view however many steps come after them; the timeline keeps one line per panel (*… on the stage ▸*), which brings the stage to it. Like the timeline it reads from the top down: the task's panels in the order they came, each with its step number, **the newest at the bottom**; a new one scrolls the stage down to it while you are at the bottom, otherwise *↓ New panel* says it came. The strip of what a task made also runs oldest to newest, left to right. In a window too narrow for three columns, the timeline and the stage are two tabs (the stage tab lights up when a question waits, and comes forward when one arrives). The task list can be folded away (‹) for more room.
- While the AI works, only the steps that changed are drawn again: a diagram or a strip of pictures scrolled sideways no longer jumps back, and text half-typed into a panel or the message box stays.
- Diagrams read in order: a box that only feeds one further on (a prompt into the sampler) stands right before it instead of at the far left with an arrow across everything, and boxes in a column are sorted by what feeds them, so arrows cross less.
- Tasks can be renamed or removed from the list (only the record goes: pictures and the canvas stay). The newest 300 are kept, in ComfyUI's user folder. Without any AI connected, the page shows the address to add in the AI app.
- Only what goes through Anomalous's MCP is seen; the AI can report the rest. The rail now has ten entries: the number keys 1–9 open the first nine.
### 🧩 The AI draws panels, and asks you on them (AI 画面板,在面板上问你)
- **A new `show_panel` tool**: the AI lays out a panel in its task, in Anomalous's own style, from fixed blocks: headings and text (also as a tip or a warning), **pictures** (a row, a grid, or two **side by side to compare**), **your models as cards with their covers** (a click opens them), tables, lists, settings, a progress bar.
- **It can ask**: buttons (one pick or several) and a line to type into. Your answer is handed over and waits for the AI, which reads it with the new `get_answers` (it can wait up to a minute for you); each question says whether the AI has seen your answer yet. `update_panel` changes a panel (a progress bar going up) or closes its questions.
- **Safe by design**: nothing the AI sends is run or read as markup. Every block is checked on the server (only these kinds, plain text of bounded length, pictures only from ComfyUI's output folder, models only from your folders) and written into the page as text.
- When the AI shows a panel while the browser is elsewhere, a small card in the corner says so (*The AI asks you: …*, **Have a look**); a task with a question waiting says *Waiting for you* in the list.
### 🎞️ The AI's stage: marks on pictures, before / after, picks, diagrams (AI 展台)
- **Marks on a picture**: the AI circles or boxes places on a picture it made (*① the fingers are fixed*, *② softer light*); each mark has its number, the labels are listed under the picture, and a click on one **zooms the picture there** (again: back).
- **Before / after slider**: two pictures over each other, a line to drag across to compare the details.
- **Picks**: pictures side by side, each with the AI's reason and **Take this one** (the choice goes to the AI); the AI can mark the one it recommends. Each can also be **put on the canvas** (its saved workflow opens in a new tab, as dropping the picture on ComfyUI does) or dragged onto the canvas.
- **Diagrams**: boxes and arrows, laid out by Anomalous (in columns or rows), with a label on an arrow; a box naming a canvas node (◎) finds it on the canvas.
- The AI only sends data (places as fractions of the picture, what connects to what); every shape and line is drawn by Anomalous, so it looks the same whichever AI is connected, and no drawing code of the AI's is taken.
## v1.61.1 (Delete a Character) — 2026-10-28
- 角色卡上加了橘色的 **删除角色** 按钮(Anomalous_TTS 议题 #2):确认以后,角色的整个文件夹(模型、参考音频、设置)移到系统回收站,需要时能从回收站还原;U 盘、网络盘上没有回收站就不删。设置坏了、读不出来的角色也能删。需要 Anomalous_TTS v1.1.1;旧版 TTS 不显示这个按钮,其他照常。
- Character cards have an orange Delete character button (Anomalous_TTS issue #2): after a confirm, the character's whole folder (models, reference audio, settings) goes to the Recycle Bin, where it can be restored; on a drive without one nothing is deleted. Broken characters can be deleted too. Needs Anomalous_TTS v1.1.1; with an older one the button is not shown and nothing else changes.
## v1.61.0 (Quick Search, Canvas Menu, Combos From Pictures, Tidier Model Folders, Multi-Character Voice-over) — 2026-10-25
### 🔎 Quick search on the canvas: Alt+B (画布快速搜索)
+8 -8
View File
@@ -3,13 +3,13 @@
# 🚀 Anomalous Model Browser
**A model browser and creative workspace for ComfyUI**
*模型库 · 出图图库 · 工作流配方 · 搭配 · 提示词 · 角色配音 · 模型检查 · 操作记录*
*模型库 · 出图图库 · 工作流 · 搭配 · 提示词 · 角色配音 · 模型检查 · 操作记录*
<br/>
[![ComfyUI Manager](https://img.shields.io/badge/ComfyUI-Manager-green?style=for-the-badge&logo=comfyui)](https://github.com/ltdrdata/ComfyUI-Manager)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow?style=for-the-badge)](LICENSE)
[![Changelog](https://img.shields.io/badge/📖_Changelog-v1.61.0-blue?style=for-the-badge)](CHANGELOG.md)
[![Changelog](https://img.shields.io/badge/📖_Changelog-v1.62.1-blue?style=for-the-badge)](CHANGELOG.md)
[![Bilibili Video](https://img.shields.io/badge/Bilibili-视频演示-00A1D6?style=for-the-badge&logo=bilibili)](https://www.bilibili.com/video/BV1boHx61Enc/)
[![Bilibili Voices](https://img.shields.io/badge/Bilibili-配音演示-00A1D6?style=for-the-badge&logo=bilibili)](https://www.bilibili.com/video/BV1bkpZ6uEZQ/)
[![Reddit](https://img.shields.io/badge/Reddit-r%2Fcomfyui-FF4500?style=for-the-badge&logo=reddit)](https://www.reddit.com/r/comfyui/s/n38ihf3DLf)
@@ -86,10 +86,10 @@ Every page has an icon on the rail at the left of the window.
| Page | What you do there |
| :--- | :--- |
| **🏠 Home** | Pick what you want to do: a card per task, first steps (scan your model folders, a guided tour of the interface) and your latest activity. |
| **🕘 Activity** | Everything the plugin changed, by day: on the canvas (which node settings changed from what to what, nodes added or removed, workflows opened) and in files (models, covers, recipes, materials, notes, images, audio). **Find on canvas** jumps to the node; a canvas change can be undone with **Undo** until something changes it again, the rest is recorded only. Your own edits are not listed. |
| **🕘 Activity** | Everything the plugin changed, by day: on the canvas (which node settings changed from what to what, nodes added or removed, workflows opened) and in files (models, covers, workflows, materials, notes, images, audio). **Find on canvas** jumps to the node; a canvas change can be undone with **Undo** until something changes it again, the rest is recorded only. Your own edits are not listed. |
| **📦 Models** | Your models with covers, trigger words and base model. Chips at the top switch between types (Checkpoint, LoRA, VAE…) and list a whole type, subfolders included; the folder list narrows it to one folder. **+** adds a loader node to the canvas; edit a model's name, notes and cover, or scan just that model. |
| **🖼️ Gallery** | Your ComfyUI `output` folder. Search by prompt, model, LoRA, seed, file name or model hash; open an image to see the parameters it was made with, or drag it onto the canvas to get its workflow back. **☆** keeps a good one as a whole workflow, a combo (model, LoRAs, prompt) or just its prompts. |
| **🪡 Workflows** | Workflow recipes: save a whole workflow or a part of one together with its models, cover, notes and parameters. Each card shows whether the models are on this computer; drag it onto the canvas to load it. Versions can be compared and restored. A card's export button makes a **recipe package** (.zip) to give to someone; dropped into their ⇅, it becomes a recipe card. |
| **🪡 Workflows** | Saved workflows: save a whole workflow or a part of one together with its models, cover, notes and parameters. Each card shows whether the models are on this computer; drag it onto the canvas to load it. Versions can be compared and restored. A card's export button makes a **recipe package** (.zip) to give to someone; dropped into their ⇅, it becomes a workflow card. |
| **🧩 Combos** | A main model, a few LoRAs and a prompt you like, kept together. **Put on canvas** (or dragging the card onto the canvas) lays them out as a wired group of nodes; nothing already on the canvas changes. |
| **✍️ Prompts** | Prompt Studio, a page of the browser; **Beside the canvas** moves it next to the canvas to see both. Select a prompt node and its prompt boxes appear as tags, edited in place: click a tag to make it stronger or weaker, double-click to edit, drag to reorder, type to add (Chinese, Japanese and other languages are translated to English), and show each tag's meaning in Chinese, Japanese, Korean and other languages. Click a card (a few common ones, plus your saved prompts) to add it, or drag it straight onto a prompt box on the canvas (outlined green for positive, red for negative) or onto empty canvas for a new prompt node. With no prompt node selected you write a draft. |
| **🎙️ Voices / 🎧 Audio** | With [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS) installed: import GPT-SoVITS characters, pick a reference clip per emotion, fix pronunciations, write a script under Voice-over on the same page and generate it right there; generated audio is listed in the audio gallery. Without it, the page explains how to install it and nothing else depends on it. |
@@ -122,7 +122,7 @@ Open the browser with **Shift + B** (again to close), the floating button on the
**Network:** Scan looks models up on Civitai and prompt translation uses Google Translate, so both need to reach those sites (from mainland China usually through a proxy; when civitai.com cannot be reached the plugin tries Civitai's other address, civitai.red, which some networks reach, a cloud server in mainland China among them: names and trigger words come, covers usually not). Without them, pick **Offline** on the scan page (an online scan does the same by itself when it cannot connect): base models are read from the files, without covers or trigger words, and a later online scan fills in the rest. Downloading missing models needs the network too; for Hugging Face, **Settings → Model downloads** can use the hf-mirror.com mirror (Civitai has none). Browsing models, outputs, workflows and combos needs no network.
**Keep a copy of your own data:** **Settings → Backup → Export** packs Workflow Recipes, combos, saved prompts and node values, parameter presets, and the names, notes and covers you gave your models into one .zip; keep it on a cloud drive or a USB stick, and **Import** puts it back, also on another computer. (They live in `workflows/anomalous_recipes`, `workflows/anomalous_notebooks`, `workflows/anomalous_materials` and `workflows/anomalous_parameters` in your ComfyUI user folder.)
**Keep a copy of your own data:** **Settings → Backup → Export** packs workflows, combos, saved prompts and node values, parameter presets, and the names, notes and covers you gave your models into one .zip; keep it on a cloud drive or a USB stick, and **Import** puts it back, also on another computer. (They live in `workflows/anomalous_recipes`, `workflows/anomalous_notebooks`, `workflows/anomalous_materials` and `workflows/anomalous_parameters` in your ComfyUI user folder.)
</details>
@@ -198,10 +198,10 @@ Open the browser with **Shift + B** (again to close), the floating button on the
| 页面 | 在这里做什么 |
| :--- | :--- |
| **🏠 主页** | 想做什么就点哪张卡片。还有上手第一步(扫描模型文件夹、界面导览)和最近的操作记录。 |
| **🕘 记录** | 按天列出插件做过的每一处改动。<br>画布上:哪个节点的设置从什么改成了什么、加了或删了哪些节点、打开了哪个工作流。<br>文件上:模型、封面、配方、素材、笔记、图片、音频。<br>**在画布上找到** 可以直接跳到那个节点;画布上的改动在没被再改过之前可以点 **撤销**,其余只记录。你自己在画布上的修改不会记进来。 |
| **🕘 记录** | 按天列出插件做过的每一处改动。<br>画布上:哪个节点的设置从什么改成了什么、加了或删了哪些节点、打开了哪个工作流。<br>文件上:模型、封面、工作流、素材、笔记、图片、音频。<br>**在画布上找到** 可以直接跳到那个节点;画布上的改动在没被再改过之前可以点 **撤销**,其余只记录。你自己在画布上的修改不会记进来。 |
| **📦 模型** | 模型带封面、触发词和底模信息。<br>顶部标签切换类型(Checkpoint、LoRA、VAE…),连同子文件夹一起列出;左侧文件夹列表可以只看某个文件夹。<br>**+** 一键在画布上创建加载节点;也可以改名、写备注、换封面,或只扫描这一个模型。 |
| **🖼️ 图库** | 读取 ComfyUI 的 `output` 文件夹。<br>可以按提示词、模型、LoRA、seed、文件名或模型哈希搜索。<br>点开图片能看到生成参数;拖到画布上可以还原当时的工作流。<br>觉得好就点 **☆**:存成整个工作流、搭配(模型 + LoRA + 提示词),或者只存提示词。 |
| **🪡 工作流** | 工作流配方:把完整工作流或其中一段,连同模型、封面、备注和参数一起存下来。<br>卡片会显示这台电脑上模型是否齐全;拖到画布上即可载入。<br>可以比较、恢复历史版本。<br>卡片上的导出按钮能打成**配方包**(压缩包)发给别人,对方拖进 ⇅ 就多一张配方卡片。 |
| **🪡 工作流** | 存下的工作流:把完整工作流或其中一段,连同模型、封面、备注和参数一起存下来。<br>卡片会显示这台电脑上模型是否齐全;拖到画布上即可载入。<br>可以比较、恢复历史版本。<br>卡片上的导出按钮能打成**配方包**(压缩包)发给别人,对方拖进 ⇅ 就多一张工作流卡片。 |
| **🧩 搭配** | 把常用的主模型、几个 LoRA 和一段提示词存成一个搭配。<br>按 **放到画布**(或把卡片拖到画布上),就放出一组连好线的节点;画布上原有的东西不动。 |
| **✍️ 提示词** | 提示词工坊,浏览器里的一个页面;点“停到画布旁”可以移到画布旁边,边看画布边改。选中一个提示词节点,它的提示词框就以词块的样子出现在工坊里,直接改:点一下调权重,双击改字,拖动换位置,输入就加(中文、日文等会自动译成英文),还能在每个词下面显示它的中文、日文、韩文等语言的意思。<br>点词卡(几张常用的,加上你存的)加进去,或者直接拖到画布的提示词框里(正面绿框、负面红框),拖到空白处就新建一个提示词节点。没选中提示词节点时写的是草稿。 |
| **🎙️ 角色语音 / 🎧 音频库** | 装了 [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS) 后可用:<br>导入 GPT-SoVITS 角色,给每种情绪选参考音频,修正读音;切到“配音”写台词并直接生成。<br>生成的音频都在音频库里。没装时,页面会说明怎么安装,其他功能不受影响。 |
@@ -237,7 +237,7 @@ Open the browser with **Shift + B** (again to close), the floating button on the
**关于网络:** 扫描模型要连 Civitai(C 站)查封面和触发词,提示词翻译默认用谷歌翻译,这两样在国内一般要开网络代理才能用。不开代理也行:扫描页选 **离线**(联网扫描连不上时也会自动改成这样),只从模型文件推断底模,没有封面和触发词,以后能连上时再扫一次会自动补上。下载缺失的模型也要联网:Hugging Face 的模型可以在 **设置 → 模型下载** 里打开“Hugging Face 用国内镜像”,不用代理;C 站的没有镜像。(civitai.com 连不上时插件会自动试 Civitai 的另一个地址 civitai.red,有的网络能连上,比如国内的云服务器,这时能认出模型名字和触发词,封面多半还是下不来。)浏览模型、图库、工作流和搭配都不用联网。
**备份自己的数据:** **设置 → 备份 → 导出**,把工作流配方、搭配、存下的提示词和节点参数、参数预设,以及你给模型改的名字、备注和封面打成一个压缩包,放到云端硬盘或 U 盘里;换电脑或重装后用 **导入** 恢复。(它们存在 ComfyUI 用户目录下的 `workflows/anomalous_recipes`、`workflows/anomalous_notebooks`、`workflows/anomalous_materials` 与 `workflows/anomalous_parameters`。)
**备份自己的数据:** **设置 → 备份 → 导出**,把工作流、搭配、存下的提示词和节点参数、参数预设,以及你给模型改的名字、备注和封面打成一个压缩包,放到云端硬盘或 U 盘里;换电脑或重装后用 **导入** 恢复。(它们存在 ComfyUI 用户目录下的 `workflows/anomalous_recipes`、`workflows/anomalous_notebooks`、`workflows/anomalous_materials` 与 `workflows/anomalous_parameters`。)
</details>
+2 -1
View File
@@ -2,7 +2,7 @@ from . import (
activity_log, audio_catalog, folder_types, gallery_routes, materials, media_routes, model_catalog,
model_folder, model_media, model_metadata, model_resolution, model_type_listing, kept_images, quick_index, node_material, recipe_packages, recipes, scan_choice, scan_report, scan_summary,
local_guard, lora_info, mcp_server, model_download, model_import, model_placement, translation_routes, version_manager, tts_version, backup,
network_check, model_heads, model_usage, usage_ledger, sidecar_layout,
network_check, model_heads, model_usage, usage_ledger, sidecar_layout, ai_tasks_routes,
)
from .scanner import *
from .config import *
@@ -142,6 +142,7 @@ def setup_routes(app):
tts_version.register_routes(app)
activity_log.register_routes(app)
mcp_server.register_routes(app)
ai_tasks_routes.register_routes(app)
model_download.register_routes(app)
network_check.register_routes(app)
model_usage.register_routes(app)
+108
View File
@@ -0,0 +1,108 @@
"""What each thing an AI app can do through Anomalous's MCP endpoint is, in one place.
Every tool (mcp_tools.py reads, mcp_actions.py acts, mcp_task_tools.py the AI's own) has an
entry here: its kind (the AI workbench's colour, words and counts; None = not a step of its
own), the stage of the work it belongs to (the flow over a task: models, combo, prompt,
canvas, run, pictures, voice, library) and its gate (what it touches or costs, for the user's
rules on what needs their OK: canvas, gpu, download, network, files; None = nothing to guard).
tests/test_ai_tasks.py keeps this catalog and the tools in step.
The user's own changes in Anomalous (activity log actions) get stages too, so the ones made
while an AI works sit in the same flow as its steps (ai_tasks.note_activity).
"""
from collections import namedtuple
Entry = namedtuple("Entry", "kind stage gate")
STAGES = ("model", "combo", "prompt", "canvas", "run", "picture", "voice", "library")
GATES = ("canvas", "gpu", "download", "network", "files")
# Arguments with which a call also changes the canvas: it then asks as the stricter of its own gate and "canvas".
ALSO_CANVAS = {"run_workflow": ("workflow", "prompt", "negative", "seed", "width", "height", "image", "image_node", "values")}
TOOLS = {
# reading
"library_overview": Entry("look", "library", None),
"search_models": Entry("look", "model", None),
"get_model": Entry("look", "model", None),
"scan_report": Entry("look", "library", None),
"search_images": Entry("look", "picture", None),
"get_image_info": Entry("look", "picture", None),
"zoom_image": Entry("look", "picture", None),
"list_combos": Entry("look", "combo", None),
"list_workflows": Entry("look", "canvas", None),
"list_saved_prompts": Entry("look", "prompt", None),
"search_audio": Entry("look", "voice", None),
"recent_activity": Entry("look", "library", None),
"describe_canvas": Entry("look", "canvas", None),
"get_situation": Entry("look", None, None), # the task, the canvas in brief and the next step (mcp_situation.py)
"check_workflow_models": Entry("look", "model", None),
"download_status": Entry("look", "model", None),
"list_voices": Entry("look", "voice", None),
# acting
"set_prompt": Entry("canvas", "prompt", "canvas"),
"set_model": Entry("canvas", "model", "canvas"),
"add_lora": Entry("canvas", "model", "canvas"),
"place_combo": Entry("canvas", "combo", "canvas"),
"fix_workflow_models": Entry("canvas", "model", "canvas"),
"find_nodes": Entry("look", "canvas", None),
"add_node": Entry("canvas", "canvas", "canvas"),
"add_nodes": Entry("canvas", "canvas", "canvas"),
"connect_nodes": Entry("canvas", "canvas", "canvas"),
"set_values": Entry("canvas", "canvas", "canvas"),
"remove_nodes": Entry("canvas", "canvas", "canvas"),
"new_workflow": Entry("canvas", "canvas", "canvas"),
"open_workflow": Entry("canvas", "canvas", "canvas"),
"run_workflow": Entry("run", "run", "gpu"),
"download_missing_models": Entry("download", "model", "download"),
"download_model": Entry("download", "model", "download"),
"move_model": Entry("act", "model", "files"),
"scan_models": Entry("scan", "library", "network"),
"speak": Entry("speech", "voice", "gpu"),
"open_in_anomalous": Entry("show", "model", None),
# the AI's own: notes and panels are steps; the rest only steer the task
"report": Entry("note", None, None),
"log_decision": Entry("decision", None, None),
"log_outside": Entry("outside", None, None), # what it did with other tools (ai_outside.py, which also notes what changed)
"show_panel": Entry("panel", None, None),
"ask_user": Entry("panel", None, None),
"propose_repaint_area": Entry("panel", "picture", None),
"show_run_map": Entry("panel", None, None),
"begin_task": Entry(None, None, None),
"track_run": Entry(None, "run", None), # attaches an existing submission; creates its own run step
"update_panel": Entry(None, None, None),
"get_answers": Entry(None, None, None),
"run_status": Entry("look", "run", None),
"stop_run": Entry(None, "run", None), # ends a run step of its own; the step says who stopped it
"get_toolbox": Entry(None, None, None),
# Anomalous's card in the conversation (mcp_apps.py): its own calls, not the AI's steps
"app_task_view": Entry(None, None, None),
"app_answer": Entry(None, None, None),
}
# Activity log actions (activity_log.py ROUTE_ACTIONS, and "canvas") the user makes in Anomalous
# that belong in the task an AI is working on, with their stage. Downloads, imports, moves, scans
# and backups are left out: an AI's own calls can cause those too.
USER_ACTIONS = {
"canvas": "canvas",
"model_edit": "library", "model_cover": "library", "model_delete": "library",
"recipe_save": "canvas", "recipe_edit": "canvas", "recipe_delete": "canvas", "recipe_restore": "canvas",
"material_save": "prompt", "material_edit": "prompt", "material_delete": "prompt",
"preset_save": "run", "preset_rename": "run", "preset_delete": "run",
"image_delete": "picture", "audio_delete": "voice", "audio_save": "voice", "voice_settings": "voice",
"note_save": "library", "note_delete": "library",
}
def entry(tool, read_only=True):
"""The catalog entry of `tool`; one not listed is a read, or an act when it is not read-only."""
found = TOOLS.get(tool)
if found:
return found
return Entry("look" if read_only else "act", None, None)
def is_step(tool):
"""Whether a call of `tool` is a step of its own in a task."""
return entry(tool).kind is not None
+29 -6
View File
@@ -24,6 +24,7 @@ from .path_utils import atomic_write_json
MAX_ENTRIES = 500
MAX_LOG_BYTES = 2 * 1024 * 1024 # long canvas entries could otherwise outgrow the JSON writer's limit
MAX_VALUE_CHARS = 400
MAX_CHANGE_CHARS = 2000 # a before/after value: room for a whole prompt, so its receipt can compare tags
DIFF_CONTEXT_CHARS = 60 # kept before the first difference of a long before/after pair
MAX_CHANGES = 60
@@ -66,6 +67,7 @@ ACTIVITY_DETAIL = web.RequestKey("activity_detail", dict) if hasattr(web, "Reque
_lock = threading.RLock()
_entries = None
_listeners = [] # called with each new entry once it is saved (ai_tasks.note_activity)
def log_path():
@@ -104,19 +106,20 @@ def _text(value):
return value if isinstance(value, str) else json.dumps(value, ensure_ascii=False)
def _short(value):
def _short(value, limit=MAX_VALUE_CHARS):
text = _text(value)
return text if len(text) <= MAX_VALUE_CHARS else text[:MAX_VALUE_CHARS] + '…'
return text if len(text) <= limit else text[:limit] + '…'
def _short_values(item, keys):
"""`keys` of `item` shortened. A long before/after pair is cut around its first difference
instead of at the start, so a change late in a long prompt still shows."""
out = {key: _short(item[key]) for key in keys if key in item and item[key] is not None}
out = {key: _short(item[key], MAX_CHANGE_CHARS if key in ('before', 'after') else MAX_VALUE_CHARS)
for key in keys if key in item and item[key] is not None}
if 'before' not in out or 'after' not in out:
return out
old, new = _text(item['before']), _text(item['after'])
if len(old) <= MAX_VALUE_CHARS and len(new) <= MAX_VALUE_CHARS:
if len(old) <= MAX_CHANGE_CHARS and len(new) <= MAX_CHANGE_CHARS:
return out
same = 0
while same < min(len(old), len(new)) and old[same] == new[same]:
@@ -124,7 +127,7 @@ def _short_values(item, keys):
start = max(0, same - DIFF_CONTEXT_CHARS)
def cut(text):
return ('…' if start else '') + text[start:start + MAX_VALUE_CHARS] + ('…' if start + MAX_VALUE_CHARS < len(text) else '')
return ('…' if start else '') + text[start:start + MAX_CHANGE_CHARS] + ('…' if start + MAX_CHANGE_CHARS < len(text) else '')
out['before'], out['after'] = cut(old), cut(new)
return out
@@ -145,9 +148,29 @@ def add_entry(source, action, target='', detail=None):
entries.insert(0, entry)
_trim(entries)
atomic_write_json(log_path(), entries)
for listener in _listeners:
try:
listener(entry)
except Exception: # noqa: BLE001 - a listener must never fail the action recorded
logging.getLogger(__name__).warning('Anomalous activity log: a listener failed', exc_info=True)
return entry
def add_listener(listener):
if listener not in _listeners:
_listeners.append(listener)
def tag(entry_id, task_id):
"""Names the AI task an entry belongs to (made by the AI, or by the user while it worked)."""
with _lock:
entries = _load()
found = next((entry for entry in entries if entry.get('id') == entry_id), None)
if found and found.get('task') != task_id:
found['task'] = task_id
atomic_write_json(log_path(), entries)
def list_entries(limit=100, source=None):
with _lock:
entries = _load()
@@ -232,7 +255,7 @@ def _clean_changes(changes):
if not isinstance(change, dict):
continue
item = _short_values(change, ('kind', 'node', 'type', 'widget', 'before', 'after'))
if item.get('kind') in ('changed', 'added', 'removed', 'opened'):
if item.get('kind') in ('changed', 'wired', 'added', 'removed', 'opened'):
cleaned.append(item)
return cleaned
+109
View File
@@ -0,0 +1,109 @@
"""How long an AI waits for the user to answer its question (ask_user, a question on a panel).
One tool call waits at most ai_panels.MAX_WAIT (55 s, under the minute AI apps give a call), so
the answer to a question still open says whether to wait on with get_answers: until the time the
user set in the Gates window (ai_gates.answer_wait, from when it asked), and while the user is
typing an answer (the page posts POST /anomalous/ai_tasks/{id}/typing as they type) until a while
after their last key. Then the AI ends its turn and the user brings it back with a line to paste
(web/modules/ai_call_lines.js); or, when the user set so (ai_gates.no_answer "recommended"), each
open choice the AI marked a recommended option for takes it, marked as such for the user and the
AI (recommended_on_timeout), and the AI goes on; not once the user began writing (typed since the
question came): they stepped away mid-answer, and it waits for them. Typing is kept in memory only.
"""
import time
from . import ai_gates, ai_panels, ai_tasks
AFTER_TYPING = 60 # the user typed this recently: their answer is on its way
TOOK = ("The user did not answer in time and set that your recommended option goes then: Anomalous took it for "
"them (recommended_on_timeout). Go on with it; a question without a recommended option stays open for the user.")
STOP = "You waited as long as the user set: end your turn now. They answer in Anomalous and tell you in your chat to go on."
WROTE = ("The user began writing an answer, then stepped away: end your turn now. Nothing is chosen for them; they "
"finish it in Anomalous and tell you in your chat to go on.")
_typed = {} # task id -> time of the user's last key
def take_recommended(task_id, step):
"""Answers each open choice of panel step `step` that has a recommended option with it. Returns how many."""
taken = 0
for block in step["panel"].get("blocks") or []:
picked = [option["id"] for option in block.get("options") or [] if option.get("recommended")]
if block.get("type") != "choices" or not picked or block.get("id") in (step.get("answers") or {}):
continue
try:
ai_panels.answer(task_id, step["id"], block["id"], choice=picked if block.get("multiple") else picked[:1],
extra={"recommended_on_timeout": True})
taken += 1
except ai_panels.PanelError:
pass
return taken
def typing(task_id, now=None):
"""The user is typing in task `task_id` (an answer or a message). False for no such task."""
with ai_tasks._lock:
if not ai_tasks.get_task(task_id):
return False
_typed[task_id] = time.time() if now is None else now
return True
def wait_on(panel_id, now=None, allow_recommended=True):
"""For a question of panel `panel_id` still open: {keep_waiting, seconds_left?, typing?, note}, or {}."""
with ai_tasks._lock:
task, step = ai_panels._find(panel_id)
return _wait_on(task, step, now, allow_recommended)
def _wait_on(task, step, now, allow_recommended):
"""Check typing and commit a timeout choice under the same lock as user answers."""
if not step:
return {}
now = time.time() if now is None else now
typed = _typed.get(task["id"], 0)
left = max(step["at"] + ai_gates.answer_wait(), typed + AFTER_TYPING) - now
if left <= 0:
wrote = typed >= step["at"] # a half-written answer is theirs: no option is taken over it
if allow_recommended and not wrote and ai_gates.no_answer() == "recommended" and take_recommended(task["id"], step):
return {"keep_waiting": False, "took_recommended": True, "note": TOOK}
return {"keep_waiting": False, "note": WROTE if wrote else STOP}
typing_now = now - typed < AFTER_TYPING
return {"keep_waiting": True, "seconds_left": round(left), **({"typing": True} if typing_now else {}),
"note": ("The user is typing their answer" if typing_now else "The user has not answered yet")
+ ": call get_answers(panel, wait_seconds=55) now, and again while keep_waiting is true. Do not "
"end your turn or ask the same in your own chat."}
def for_answer(answer, panel_id=None, now=None):
"""get_answers' `answer`, with how to wait on when nothing came back but questions still open:
for panel `panel_id`, or without one for the newest open question while the user's time runs."""
waiting = answer.get("still_waiting") or []
if not waiting or answer.get("answers") or answer.get("messages"):
return answer
if panel_id is None:
found = [wait_on(panel, now) for panel in dict.fromkeys(item["panel"] for item in waiting)]
live = [item for item in found if item.get("keep_waiting")]
status = max(live, key=lambda item: item["seconds_left"]) if live else next((item for item in found if item), {})
took = any(item.get("took_recommended") for item in found)
return {**answer, **status, **({"took_recommended": True} if took else {})}
return {**answer, **wait_on(panel_id, now)}
def collect(client, panel_id=None, wait_seconds=0, task_id=None):
"""Wait unlocked, then deliver a timeout choice or a racing user answer in this same call."""
answer = ai_panels.get_answers(client, panel_id, wait_seconds, task_id=task_id)
if answer.get("answers") or answer.get("messages") or not answer.get("still_waiting"):
return answer
with ai_tasks._lock:
# Input may have arrived after the long wait released its lock. Read it before choosing.
answer = ai_panels.get_answers(client, panel_id, 0, task_id=task_id)
if answer.get("answers") or answer.get("messages") or not answer.get("still_waiting"):
return answer
status = for_answer(answer, panel_id)
fresh = ai_panels.get_answers(client, panel_id, 0, task_id=task_id)
if fresh.get("answers") or fresh.get("messages"):
if any(item.get("recommended_on_timeout") for item in fresh.get("answers", [])):
fresh["note"] = TOOK
return fresh
return {**status, **fresh} if fresh.get("still_waiting") else fresh
+41
View File
@@ -0,0 +1,41 @@
"""Who is calling Anomalous's MCP endpoint (mcp_server.py), for the AI workbench's record
(ai_tasks.py): a short name for the AI app, the session it runs when it keeps several, and a task
it names. Each comes from the request's headers or metadata and is cleaned to a short plain word."""
import re
_CLIENT_NAMES = (
("claude-code", "Claude Code"), ("claude", "Claude"), ("cursor", "Cursor"), ("cherry", "Cherry Studio"),
("windsurf", "Windsurf"), ("codex", "Codex"), ("gemini", "Gemini"), ("vscode", "VS Code"), ("cline", "Cline"),
)
def client_of(headers, params):
"""A short name for the AI app making a call: the client info it sends, else its User-Agent."""
meta = params.get("_meta") if isinstance(params.get("_meta"), dict) else {}
info = meta.get("io.modelcontextprotocol/clientInfo")
raw = info.get("name") if isinstance(info, dict) and isinstance(info.get("name"), str) else ""
if not raw:
raw = (headers.get("User-Agent") or "").split(" ")[0].split("/")[0]
lowered = raw.lower()
for key, name in _CLIENT_NAMES:
if key in lowered:
return name
clean = re.sub(r"[^\w .-]", "", raw).strip()[:40]
return clean or "AI"
def session_of(headers, params):
"""An optional session key when one app runs multiple MCP sessions at once."""
meta = params.get("_meta") if isinstance(params.get("_meta"), dict) else {}
raw = headers.get("Mcp-Session-Id") or headers.get("X-Anomalous-Session") or meta.get("sessionId") or meta.get("conversationId") or ""
return re.sub(r"[^\w .:-]", "", str(raw)).strip()[:64] if isinstance(raw, str) else ""
def task_of(headers, params, arguments=None):
"""An explicit task id from headers, request metadata or tool arguments."""
meta = params.get("_meta") if isinstance(params.get("_meta"), dict) else {}
arg_task = arguments.get("task") if isinstance(arguments, dict) else None
raw = arg_task or headers.get("X-Anomalous-Task") or meta.get("anomalous/task") or meta.get("task") or ""
return re.sub(r"[^\w-]", "", str(raw)).strip()[:64] if isinstance(raw, str) else ""
+101
View File
@@ -0,0 +1,101 @@
"""A task's key decisions, kept in Anomalous to look back on ("决策"). Talk may happen anywhere,
in the AI's own chat too; what was settled is kept here: the AI records it (log_decision: what,
why, what was left out, who decided, the steps it is about), and what the user decided in
Anomalous itself is read from the task's own steps (a panel answer, a gate's allow or no, the
materials handed over, the canvas mode, a stop). decisions() lists both, oldest first; gap() says
how much the AI did since its last recorded decision, for a quiet reminder."""
import time
import uuid
from . import ai_tasks
BY = ("user", "ai", "together")
MAX_TEXT = 600
MAX_REJECTED = 6
MAX_RELATED = 8
QUESTIONS = ("choices", "input", "picks")
def log(client, title, decision, reason="", rejected=None, by="user", related=None, now=None, session="", task_id=None):
"""A decision the AI records. `related`: ids of this task's steps it is about. Returns the step."""
title, decision = str(title or "").strip(), str(decision or "").strip()
if not title or not decision:
raise ValueError("title (what was decided about) and decision (what was decided) are needed")
if by not in BY:
raise ValueError(f"by is one of {', '.join(BY)}")
rejected = [ai_tasks._short(str(item).strip(), 200) for item in (rejected or []) if str(item).strip()][:MAX_REJECTED]
now = time.time() if now is None else now
record = {"title": ai_tasks._short(title, 200), "decision": ai_tasks._short(decision, MAX_TEXT),
"reason": ai_tasks._short(str(reason or "").strip(), MAX_TEXT), "rejected": rejected, "by": by, "related": []}
step = {"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": "log_decision", "kind": "decision", "ok": True,
"state": "done", "summary": "log_decision", "text": record["decision"], "decision": record, "results": []}
with ai_tasks._lock:
task = ai_tasks.current_task(client, now, session=session, task_id=task_id)
known = {item["id"] for item in task["steps"]}
record["related"] = [str(item) for item in (related or []) if str(item) in known][:MAX_RELATED]
ai_tasks._add_step(task, step)
ai_tasks._save(task)
ai_tasks._notify(task["id"])
return step
def decisions(task):
"""Every decision of `task`, oldest first: {type: logged | answer | gate | toolbox | canvas | stop, step, at, …}."""
out = []
for step in (task or {}).get("steps") or []:
tool = step.get("tool")
if step.get("kind") == "decision":
out.append({"type": "logged", "step": step["id"], "at": step["at"], **step["decision"]})
elif step.get("kind") == "panel":
blocks = {block.get("id"): block for block in step["panel"].get("blocks") or []}
for block_id, given in (step.get("answers") or {}).items():
block = blocks.get(block_id) or {}
out.append({"type": "answer", "step": step["id"], "at": given.get("at", step["at"]),
"question": block.get("question") or step["panel"].get("title") or "",
"answer": "、".join(given.get("labels") or []) if given.get("labels") and given.get("choice") != ["other"] else given.get("text", ""),
"skipped": bool(given.get("skipped")), "timed_out": bool(given.get("recommended_on_timeout"))})
elif (step.get("gate") or {}).get("at"):
asked = step["gate"]
out.append({"type": "gate", "step": step["id"], "at": asked["at"], "tool": tool,
"allowed": asked.get("decision") == "allow", "text": asked.get("text", "")})
elif tool == "user_toolbox":
box = step.get("toolbox") or {}
count = sum(len(value) for value in box.values() if isinstance(value, list))
out.append({"type": "toolbox", "step": step["id"], "at": step["at"], "mode": box.get("mode", "prefer"), "count": count})
elif tool == "user_canvas":
out.append({"type": "canvas", "step": step["id"], "at": step["at"], "mode": step.get("canvas")})
elif tool == "user_stop":
out.append({"type": "stop", "step": step["id"], "at": step["at"]})
return sorted(out, key=lambda item: item["at"])
def gap(task):
"""Runs and canvas changes of the AI since its last recorded decision ({runs, canvas}), or None when there are none."""
runs = canvas = 0
for step in reversed((task or {}).get("steps") or []):
if step.get("kind") == "decision":
break
runs += step.get("kind") == "run"
canvas += step.get("kind") == "canvas"
return {"runs": runs, "canvas": canvas} if runs or canvas else None
def lines(task, limit=12):
"""The latest decisions in a line each, for an AI picking the task up again."""
words = []
for item in decisions(task)[-limit:]:
if item["type"] == "logged":
words.append(f"{item['title']}: {item['decision']}" + (f" (because {item['reason']})" if item["reason"] else ""))
elif item["type"] == "answer":
words.append(f"Nobody answered \"{item['question']}\" in time; your recommended option was taken: {item['answer']}" if item["timed_out"] else
f"The user answered \"{item['question']}\": " + ("left it to you" if item["skipped"] else item["answer"]))
elif item["type"] == "gate":
words.append(f"The user {'allowed' if item['allowed'] else 'refused'} {item['tool']}" + (f": {item['text']}" if item["text"] else ""))
elif item["type"] == "toolbox":
words.append(f"The user handed over {item['count']} materials ({item['mode']})")
elif item["type"] == "canvas":
words.append("The user set the canvas mode to " + ("the task's own tab" if item["mode"] == "new" else "their open canvas"))
else:
words.append("The user stopped a run")
return words
+223
View File
@@ -0,0 +1,223 @@
"""Gates: what the user lets AI apps do through Anomalous's MCP endpoint without asking first.
Each acting tool has a gate in action_catalog.py (canvas, gpu, download, network, files) and each gate
a policy the user sets on the AI workbench: "auto" (go ahead), "ask" (wait for the user's OK)
or "never". mcp_server.py asks check() before running a tool. With "ask" the tool's step waits
in state "asking", showing what it would do, for up to WAIT_SECONDS: the user's Allow lets the
call go on, No answers the AI with the user's reason. When the user has not answered by then,
the AI is told to wait (get_answers) and call the same tool again; the user's later answer
reaches it as a message (ai_panels._collect), and an allowed call made again with the same
arguments goes ahead without asking twice. Policies are kept in the ComfyUI user folder
(anomalous/ai_gates.json); changing one is a write (local_guard.py keeps it to this computer).
The same window sets (anomalous/ai_tools.json) how an AI waits for a run (run_wait): "call", it ends its turn and the user tells it when the run is done (the AI
workbench gives them a line to paste), or "wait", it waits with run_status; how long in all an
AI waits for the user's answer to its question (answer_wait, ai_answer_wait.py), and what happens
when that time runs out (no_answer): "stop", the AI ends its turn and waits for the user, or
"recommended", a choice the AI marked a recommended option for takes it (never a gate's question).
"""
import json
import os
import threading
import time
from . import action_catalog, ai_tasks
from .path_utils import atomic_write_json
POLICIES = ("auto", "ask", "never")
RUN_WAITS = ("call", "wait")
ANSWER_WAITS = (60, 180, 600) # seconds; the first call waits 30-55 s, get_answers the rest
NO_ANSWERS = ("stop", "recommended")
DEFAULTS = {"canvas": "auto", "gpu": "auto", "download": "ask", "network": "auto", "files": "ask"}
WAIT_SECONDS = 50 # below the minute many AI apps give a tool call
ALLOWED_SECONDS = 30 * 60 # an Allow the AI did not wait for holds this long for the same call
MAX_ARGS = 8
_decided = threading.Condition(ai_tasks._lock)
def _path():
return os.path.join(os.path.dirname(ai_tasks.tasks_dir()), "ai_gates.json")
def policies():
try:
with open(_path(), encoding="utf-8") as source:
stored = json.load(source)
except (OSError, ValueError):
stored = {}
stored = stored if isinstance(stored, dict) else {}
return {gate: stored.get(gate) if stored.get(gate) in POLICIES else default for gate, default in DEFAULTS.items()}
def set_policy(gate, policy):
if gate not in DEFAULTS or policy not in POLICIES:
raise ValueError("Unknown gate or policy")
with _decided:
current = policies()
current[gate] = policy
atomic_write_json(_path(), current)
return current
def _tools_path():
return os.path.join(os.path.dirname(ai_tasks.tasks_dir()), "ai_tools.json")
def _tool_settings():
try:
with open(_tools_path(), encoding="utf-8") as source:
stored = json.load(source)
except (OSError, ValueError):
stored = {}
return stored if isinstance(stored, dict) else {}
def _set_tool_setting(key, value):
with _decided:
atomic_write_json(_tools_path(), {**_tool_settings(), key: value})
return value
def run_wait():
"""How an AI waits for its run: "call" (the user tells it when it is done) or "wait" (run_status)."""
chosen = _tool_settings().get("runs")
return chosen if chosen in RUN_WAITS else "call"
def set_run_wait(mode):
if mode not in RUN_WAITS:
raise ValueError("runs is call or wait")
return _set_tool_setting("runs", mode)
def answer_wait():
"""How long in all an AI waits for the user's answer, in seconds (one of ANSWER_WAITS)."""
chosen = _tool_settings().get("answer_wait")
return chosen if chosen in ANSWER_WAITS else 180
def set_answer_wait(seconds):
if seconds not in ANSWER_WAITS or isinstance(seconds, bool):
raise ValueError("answer_wait is 60, 180 or 600")
return _set_tool_setting("answer_wait", seconds)
def no_answer():
"""When the user's time runs out unanswered: "stop" (the AI waits for them) or "recommended"."""
chosen = _tool_settings().get("no_answer")
return chosen if chosen in NO_ANSWERS else "stop"
def set_no_answer(mode):
if mode not in NO_ANSWERS:
raise ValueError("no_answer is stop or recommended")
return _set_tool_setting("no_answer", mode)
def describe(tool, arguments):
"""What a call would do, as [{label, value}] for the card the user answers."""
out = []
for key, value in list((arguments or {}).items())[:MAX_ARGS]:
if key == "task":
continue
if key == "model_id": # its name, and the folder it is in
value = " · ".join(filter(None, (ai_tasks._model_result(value).get("name") or value, str(value).split(":")[0])))
out.append({"label": key, "value": ai_tasks._short(value, 400)})
return out
def _signature(tool, arguments):
return json.dumps([tool, {k: v for k, v in (arguments or {}).items() if k != "task"}], sort_keys=True, ensure_ascii=False)
def _blocked(text):
return {"content": [{"type": "text", "text": text}], "isError": True}
def _step(step_ref):
task = ai_tasks._load().get(step_ref[0]) if step_ref else None
step = next((item for item in (task or {}).get("steps", []) if item.get("id") == step_ref[1]), None) if task else None
return task, step
def check(tool, arguments, step_ref, now=None):
"""None when the call may go ahead; else the result to give the AI instead of running it."""
gate = action_catalog.entry(tool).gate
if gate and set(arguments or {}) & set(action_catalog.ALSO_CANVAS.get(tool, ())):
strict = {"auto": 0, "ask": 1, "never": 2}
gate = max((gate, "canvas"), key=lambda name: strict.get(policies().get(name, "auto"), 0))
policy = policies().get(gate, "auto") if gate else "auto"
if policy == "auto":
return None
if policy == "never":
return _blocked(f"The user does not let AI apps do this ({gate}) through Anomalous; tell them what you "
"wanted to do instead, or ask them to change it in the AI workbench (Gates).")
now = time.time() if now is None else now
signature = _signature(tool, arguments)
with _decided:
task, step = _step(step_ref)
if not step:
return _blocked("Approval could not be recorded; nothing was done. Retry after reopening the AI workbench.")
# An Allow given after the AI stopped waiting: the same call again goes ahead.
for earlier in task["steps"]:
asked = earlier.get("gate") or {}
if (asked.get("timed_out") and asked.get("signature") == signature
and asked.get("decision") == "allow" and not asked.get("used")
and now - asked.get("at", 0) < ALLOWED_SECONDS):
asked["used"] = True
earlier["state"] = "done"
step["gate"] = {"gate": gate, "signature": signature, "decision": "allow", "used": True, "earlier": earlier["id"]}
ai_tasks._save(task)
return None
step["state"] = "asking"
step["gate"] = {"gate": gate, "signature": signature, "lines": describe(tool, arguments), "asked": now}
ai_tasks._save(task)
task_id = task["id"]
ai_tasks._notify(task_id, {"gate": gate, "asks": True})
with _decided:
_decided.wait_for(lambda: "decision" in step["gate"], WAIT_SECONDS)
asked = step["gate"]
if asked.get("decision") == "allow":
asked["used"] = True
step["state"] = "running"
ai_tasks._save(task)
return None
if asked.get("decision") == "deny":
return _blocked("你没有允许 / The user said no to this in Anomalous." + (f" They said: {asked['text']}" if asked.get("text") else ""))
asked["timed_out"] = True
ai_tasks._save(task)
return {"content": [{"type": "text", "text": "Waiting for the user's OK in Anomalous's AI workbench: nothing was done yet. "
"Call get_answers (wait_seconds up to 55) to hear their answer; once they allow "
"it, call this tool again with the same arguments."}],
"structuredContent": {"waiting_for_user": True, "gate": gate}, "isError": False}
def decide(task_id, step_id, decision, text="", remember=False, now=None):
"""The user's answer to a step that asks: "allow" or "deny" (with words for the AI)."""
if decision not in ("allow", "deny"):
raise ValueError("decision is allow or deny")
with _decided:
task, step = _step((task_id, step_id))
asked = (step or {}).get("gate")
if not asked or step.get("state") != "asking":
raise ValueError("Nothing waits for an answer there")
asked.update(decision=decision, at=time.time() if now is None else now)
if text:
asked["text"] = ai_tasks._short(str(text).strip(), 1000)
if asked.get("timed_out"):
# The AI stopped waiting: it hears of the answer as a message, and an Allow waits for its call.
asked["delivered"] = False
step["state"] = "allowed" if decision == "allow" else "error"
else:
step["state"] = "running" if decision == "allow" else "error"
if decision == "deny":
step.update(ok=False, error="你没有允许 / You said no")
ai_tasks._save(task)
_decided.notify_all()
if remember and decision == "allow":
set_policy(asked["gate"], "auto")
ai_tasks._notify(task_id)
from . import ai_panels
ai_panels.wake()
return step
+210
View File
@@ -0,0 +1,210 @@
"""Marks the user draws on a picture to tell the AI where to change it (the AI workbench's marker,
web/modules/ui_ai_marker.js): boxes, circles, arrows and free lines, each with a note. The page
sends the picture with the marks drawn on it (a PNG) and the marks themselves; this keeps the
PNG in the ComfyUI user folder (anomalous/ai_marks/<task>/<step>.png, never the output folder, so
the gallery stays the user's pictures) and adds a "user_marks" step to the task. The AI gets it
like a message (ai_panels._collect, kind "marks": the notes with where each mark is, in
fractions of the picture) and, attached to the same answer, the marked picture as a thumbnail
(image_items), so an AI that can see pictures sees the marks. Only an output picture can be
marked; the PNG is checked (signature, size, dimensions) before it is kept.
The user also says how to change the picture (MODES), and the AI reads that choice first, as the
user's: leave it to the AI, change only the marked areas, or make the whole picture anew. For
"only the marked areas" the page sends a mask (the marker's "repaint" brush, and the boxes,
circles and lines; white where to repaint), and the picture is also written to ComfyUI's input
folder as anomalous_inpaint_<step>.png with that area as its alpha, so LoadImage gives the
picture and the area as its MASK for an inpaint workflow; the AI gets the file name with the marks.
"""
import base64
import io
import os
import re
import time
import uuid
import folder_paths
from . import ai_tasks
from .mcp_tools import _image_content
MAX_PNG_BYTES = 8 * 1024 * 1024
MAX_EDGE = 4096
MAX_MARKS = 20
MAX_NOTE = 300
SHAPES = ("box", "circle", "arrow", "pen")
MODES = {
"auto": "How to change it is up to you: say in report which way you took (repaint only the marked areas, "
"or make the picture anew) and why. To repaint only the marked areas, propose_repaint_area(marks=this "
"id) agrees on the area with the user and makes the picture to repaint.",
"local": "The user chose: change ONLY the marked areas and keep everything else exactly as it is. Repaint "
"them from inpaint_image (inpaint_note says how); do not make the whole picture anew. If the area "
"looks off, propose_repaint_area(marks=this id) agrees on a better one first.",
"whole": "The user chose: make the whole picture anew with these changes (its own model and prompt, "
"changed as the notes say; a new seed or img2img). Do not repaint only a part of it.",
}
FILE_NAME = re.compile(r"[0-9a-f]{10}\.png")
PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n"
class MarksError(ValueError):
pass
def path_of(task_id, name):
"""The kept PNG `name` of task `task_id`, or None (only names this module makes)."""
if not FILE_NAME.fullmatch(str(name)):
return None
path = os.path.join(ai_tasks.marks_dir(task_id), name)
return path if os.path.isfile(path) else None
def _png(data_url):
text = str(data_url or "")
if not text.startswith("data:image/png;base64,"):
raise MarksError("Expected the marked picture as a PNG data URL")
try:
data = base64.b64decode(text.split(",", 1)[1], validate=True)
except (ValueError, IndexError):
raise MarksError("The marked picture is not valid base64")
if len(data) > MAX_PNG_BYTES or not data.startswith(PNG_SIGNATURE):
raise MarksError("The marked picture is not a PNG of a usable size")
from PIL import Image
try:
with Image.open(io.BytesIO(data)) as image:
if image.format != "PNG" or max(image.size) > MAX_EDGE:
raise MarksError("The marked picture is too large")
image.verify()
except MarksError:
raise
except Exception: # noqa: BLE001 - whatever PIL says, it is not a picture to keep
raise MarksError("The marked picture cannot be read")
return data
def _fraction(value):
try:
return min(1.0, max(0.0, round(float(value), 4)))
except (TypeError, ValueError):
raise MarksError("Mark positions are fractions of the picture")
def _clean(marks, masked=False):
if not isinstance(marks, list) or (not marks and not masked) or len(marks) > MAX_MARKS:
raise MarksError(f"Between 1 and {MAX_MARKS} marks")
out = []
for mark in marks:
if not isinstance(mark, dict) or mark.get("shape") not in SHAPES:
raise MarksError(f"Each mark is one of {', '.join(SHAPES)}")
x, y, w, h = (_fraction(mark.get(key, 0)) for key in ("x", "y", "w", "h"))
out.append({"shape": mark["shape"], "x": x, "y": y, "w": min(w, 1 - x), "h": min(h, 1 - y),
"note": ai_tasks._short(str(mark.get("note") or "").strip(), MAX_NOTE)})
return out
def _words(marks, text, mode="auto"):
"""The message the AI reads: how to change the picture, the user's words, then each mark with where it is."""
lines = [MODES[mode]] + ([text] if text else [])
for number, mark in enumerate(marks, 1):
where = (f"x {round(mark['x'] * 100)}–{round((mark['x'] + mark['w']) * 100)}%, "
f"y {round(mark['y'] * 100)}–{round((mark['y'] + mark['h']) * 100)}% of the picture")
lines.append(f"Mark {number} ({mark['shape']}, {where}): {mark['note'] or '(no note)'}")
return "\n".join(lines)
def picture_path(picture):
"""Where an output picture {subfolder, filename} (as _output_media gives it) is on disk."""
return os.path.join(folder_paths.get_output_directory(), picture["subfolder"], picture["filename"])
def write_inpaint(picture, area_of, name_id):
"""The picture with its repaint area as alpha in ComfyUI's input folder (a new file, never one
that is there), so LoadImage gives the area as MASK: its name. `area_of(size)` makes the area
(an "L" image of that size, white where to repaint, soft edges kept)."""
from PIL import Image, ImageOps
try:
with Image.open(picture_path(picture)) as original:
area = area_of(original.size)
if not area.getbbox():
raise MarksError("The repaint area is empty")
out = original.convert("RGB")
out.putalpha(ImageOps.invert(area)) # ComfyUI's LoadImage: MASK = 1 - alpha
except MarksError:
raise
except Exception: # noqa: BLE001 - a picture PIL cannot read cannot be repainted either
raise MarksError("The picture cannot be read for repainting")
name = f"anomalous_inpaint_{name_id}.png"
with open(os.path.join(folder_paths.get_input_directory(), name), "xb") as target: # never replaces a file
out.save(target, "PNG")
return name
def _inpaint_file(picture, mask, step_id):
"""write_inpaint with the area the user drew (`mask`, PNG bytes, white where to repaint)."""
from PIL import Image
def drawn_area(size):
with Image.open(io.BytesIO(mask)) as drawn:
return drawn.convert("L").resize(size, Image.NEAREST).point(lambda value: 255 if value > 127 else 0)
return write_inpaint(picture, drawn_area, step_id)
def save(task_id, image, png, marks, text="", now=None, mask=None, mode=None):
"""Keeps the marked picture and adds the user's marks to the task, with how to change it (a
key of MODES; without one, "local" when there is a mask, as the page sent before there was a
choice). For "local", `mask` (a PNG data URL) also makes the picture ready to repaint
(_inpaint_file); the other ways keep no mask. Returns the step."""
mode = mode or ("local" if mask else "auto")
if mode not in MODES:
raise MarksError(f"How to change it is one of {', '.join(MODES)}")
if mode != "local":
mask = None
elif not mask:
raise MarksError("Changing only the marked areas needs a box, circle, line or painted area")
try:
picture = ai_tasks._output_media(image)
except (ValueError, OSError):
picture = None
if not picture:
raise MarksError("Only a picture in the output folder can be marked")
data = _png(png)
mask = _png(mask) if mask else None
marks = _clean(marks, masked=mask is not None)
text = ai_tasks._short(str(text or "").strip(), 2000)
step_id = uuid.uuid4().hex[:10]
now = time.time() if now is None else now
if not ai_tasks.get_task(task_id):
raise MarksError("No such task")
inpaint = _inpaint_file(picture, mask, step_id) if mask else None # outside the lock: a picture takes a moment
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
if not task:
raise MarksError("No such task")
os.makedirs(ai_tasks.marks_dir(task_id), exist_ok=True)
with open(os.path.join(ai_tasks.marks_dir(task_id), f"{step_id}.png"), "xb") as target: # a new name: never replaces
target.write(data)
step = {"id": step_id, "at": now, "ms": 0, "tool": "user_marks", "kind": "user", "ok": True, "state": "done",
"summary": "user_marks", "text": _words(marks, text, mode), "note": text, "image": picture, "marks": marks, "mode": mode,
"marked": f"{step_id}.png", "delivered": False, "results": [], **({"inpaint": inpaint} if inpaint else {})}
task["steps"].append(step)
del task["steps"][:-ai_tasks.MAX_STEPS]
task["updated"] = max(task.get("updated", 0), now)
task["ended"] = None # like a message (ai_panels.leave_message): the task goes on
ai_tasks._save(task)
ai_tasks._notify(task_id)
return step
def image_items(messages):
"""MCP image items of the marked pictures among `messages` and of the pictures the user handed
over (ai_pictures.py), their thumbnails, taking their private "_marked" / "_pictures" paths off
the messages."""
items = []
for message in messages:
for path in [message.pop("_marked", None), *message.pop("_pictures", [])]:
item = _image_content(path) if path else None
if item:
items.append(item)
return items
+88
View File
@@ -0,0 +1,88 @@
"""What the user points at in a message to the AI ("@" in the AI workbench's message box,
web/modules/ui_ai_mention.js): models, combos, workflows, prompts, node types, custom node packs
and output pictures, by the names the AI's tools use. Checked when the message is left (clean)
and given to the AI with it (message_of): each thing as get_toolbox gives a pick (lookup), the
pictures attached with what made them, as ai_pictures.py hands them over, so the AI needs no
guessing which "that LoRA" is."""
from . import ai_pictures
from .mcp_tools import ToolError, _resolve_image
KINDS = ("models", "combos", "workflows", "prompts", "nodes", "packs", "pictures")
MAX_REFS = 12
MAX_NAME = 300
class RefsError(ValueError):
pass
def clean(refs):
"""[{kind, id, label}] from the page: known kinds, text names, each thing once, at most MAX_REFS."""
if refs in (None, []):
return []
if not isinstance(refs, list) or len(refs) > MAX_REFS:
raise RefsError(f"refs: a list of up to {MAX_REFS} things")
out = []
for ref in refs:
if not isinstance(ref, dict) or ref.get("kind") not in KINDS:
raise RefsError(f"refs: each is {{kind, id}} with kind one of {', '.join(KINDS)}")
name = ref.get("id")
if not isinstance(name, str) or not name.strip() or len(name) > MAX_NAME:
raise RefsError(f"refs: an id is text of up to {MAX_NAME} characters")
label = str(ref.get("label") or name).strip()[:MAX_NAME]
item = {"kind": ref["kind"], "id": name.strip(), "label": label}
if not any(have["kind"] == item["kind"] and have["id"] == item["id"] for have in out):
out.append(item)
return out
def lookup(kind, name):
"""A combo, workflow, prompt or model as the AI's tools give it, or None when it is gone."""
from . import mcp_tools
if kind in ("combos", "workflows"):
lister = mcp_tools.list_combos if kind == "combos" else mcp_tools.list_workflows
return next(iter(lister(name, 1, exact_name=True)[kind]), None)
if kind == "prompts":
from .ai_toolbox_catalog import everything
return next((entry for entry in everything()["prompts"] if entry["id"] == name), None)
if kind == "models":
try:
found = mcp_tools.get_model(name)
except ToolError:
return None
return found[0] if isinstance(found, tuple) else found # without its cover picture
return None
def message_of(step):
"""The refs of a message step for the AI ("refs", each in full or marked missing) and the
pictures' files in "_pictures" for ai_marks.image_items to attach."""
items, files = [], []
for ref in step.get("refs") or []:
item = {"kind": ref["kind"], "id": ref["id"], "said_as": f"@{ref['label']}"}
if ref["kind"] == "pictures":
try:
rel, path = _resolve_image(ref["id"])[:2]
except ToolError:
item["missing"] = True
else:
item.update(image=rel, made_with=ai_pictures.brief(path))
files.append(path)
elif ref["kind"] == "nodes":
item["note"] = "a node type: find_nodes(types=[id]) gives it in full, add_node adds it"
elif ref["kind"] == "packs":
item["note"] = "a custom node pack (folder): its node types are those find_nodes lists with this source"
else:
found = lookup(ref["kind"], ref["id"])
if found is None:
item["missing"] = True
else:
item["details"] = found
items.append(item)
out = {"refs": items, "refs_note": "What the user's @ names in their text point at, and the pictures they attached, exactly."}
if files:
out["_pictures"] = files
return out
+197
View File
@@ -0,0 +1,197 @@
"""What happened outside Anomalous while an AI works on a task, as steps of kind "outside" in the
AI workbench. Anomalous sees only the calls made to it: an AI app may also run commands and
scripts of its own, and one that edits pictures that way loses their workflow and the user's
earlier result. Two ways to see it:
check() looks, when the AI calls Anomalous (never in the background), at what changed since its
previous call: pictures in ComfyUI's output folder no ComfyUI run saved, and pictures in the
output or input folder rewritten in place (changed a while after they were made). Only the top of
each folder, by the files' own times (made, changed), at most MAX_ENTRIES entries. While ComfyUI
runs a prompt, new pictures wait until it ends (its history names them then). What it finds is
a step and goes to the AI with its next answer (tell), which says it may well be the user's own
doing. Without a creation time (Linux), output pictures are "new or changed" and the input
folder is not looked at (an upload looks the same).
log() is the AI's own word: what it ran or changed outside Anomalous, why, and which files.
"""
import os
import threading
import time
import uuid
from collections import OrderedDict
import folder_paths
from . import ai_tasks
from .mcp_tools import tool_text
PICTURES = {".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp", ".tif", ".tiff", ".mp4", ".webm", ".mov"}
MAX_ENTRIES = 20000 # of a folder's top looked at in one check
MAX_FOUND = 12 # files named in one step
REWRITE_SECONDS = 60 # changed this long after it was made: rewritten, not just written
HISTORY_ITEMS = 300 # ComfyUI's latest runs whose saved pictures are no news
MAX_WATCHED = 16 # tasks whose last check is kept
MAX_TEXT, MAX_FILES, MAX_FILE = 300, 20, 200
_lock = threading.Lock()
_watched = OrderedDict() # task id -> {"since": time of the last full check, "told": {(folder, name, changed)}}
def _queue():
from server import PromptServer
return PromptServer.instance.prompt_queue
def _busy():
"""Whether ComfyUI runs a prompt now (its pictures are in no history yet)."""
try:
running, _pending = _queue().get_current_queue_volatile()
except Exception: # noqa: BLE001 - no server (tests) or an older queue: nothing runs that we know of
return False
return bool(running)
def _saved_by_runs():
"""Names of the pictures ComfyUI's latest runs saved at the top of the output folder."""
try:
history = _queue().get_history(max_items=HISTORY_ITEMS, map_function=lambda record: record.get("outputs") or {})
except Exception: # noqa: BLE001 - no server (tests)
return set()
names = set()
for outputs in history.values():
for output in outputs.values() if isinstance(outputs, dict) else ():
for items in output.values() if isinstance(output, dict) else ():
for item in items if isinstance(items, list) else ():
if isinstance(item, dict) and item.get("type") == "output" and not item.get("subfolder") and item.get("filename"):
names.add(item["filename"])
return names
def _times(entry):
"""(made, changed) of a folder entry; made is None where the system keeps no creation time."""
stat = entry.stat()
return getattr(stat, "st_birthtime", None), stat.st_mtime
def _changes(folder, root, since):
"""[(name, change, when)] of the pictures at the top of `root` changed after `since`:
"rewritten", "new" or (no creation time) "changed"."""
found = []
try:
with os.scandir(root) as entries:
for at, entry in enumerate(entries):
if at >= MAX_ENTRIES:
break
if os.path.splitext(entry.name)[1].lower() not in PICTURES or not entry.is_file(follow_symlinks=False):
continue
made, changed = _times(entry)
if changed <= since and (made is None or made <= since):
continue
if made is None:
found.append((entry.name, "changed", changed))
elif changed - made > REWRITE_SECONDS:
found.append((entry.name, "rewritten", changed))
elif made > since:
found.append((entry.name, "new", changed))
except OSError:
return []
return [item for item in found if folder == "output" or item[1] == "rewritten"]
def check(task_id, now=None):
"""What changed outside Anomalous since the last check of `task_id`, as a step of the task
(also returned), or None. The first check of a task only starts the watch."""
if not task_id:
return None
now = time.time() if now is None else now
with _lock:
watch = _watched.get(task_id)
if watch is None:
_watched[task_id] = {"since": now, "told": set()}
while len(_watched) > MAX_WATCHED:
_watched.popitem(last=False)
return None
_watched.move_to_end(task_id)
busy = _busy()
saved = set() if busy else _saved_by_runs()
files = []
for folder, root in (("output", folder_paths.get_output_directory()), ("input", folder_paths.get_input_directory())):
for name, change, when in _changes(folder, root, watch["since"]):
if busy and change != "rewritten":
continue # maybe the running prompt's: judged once it ends
if change != "rewritten" and name in saved:
continue
if (folder, name, when) not in watch["told"]:
watch["told"].add((folder, name, when))
files.append({"folder": folder, "name": name, "change": change})
if not busy:
# A file's change time can be a second or two after the clock read (Windows): kept as told
# until it is older than the next check's start.
watch.update(since=now, told={key for key in watch["told"] if key[2] > now})
if not files:
return None
step = {"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": "outside_files", "kind": "outside", "ok": True,
"state": "done", "summary": "outside_files", "files": files[:MAX_FOUND], "more": max(0, len(files) - MAX_FOUND),
"results": []}
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
if not task:
return None
ai_tasks._add_step(task, step)
ai_tasks._save(task)
ai_tasks._notify(task_id)
return step
def tell(result, step):
"""Puts what check() found into the AI's tool result."""
if not step or not isinstance(result, dict):
return result
files = [f"{item['folder']}/{item['name']} ({item['change']})" for item in step["files"]]
note = ("Anomalous saw pictures change outside its tools since your previous call; it shows them to the user. "
"It may be the user's own doing. If it was yours: never edit, crop, paint or overwrite pictures with code "
"of your own (that loses their workflow and the user's earlier result); change a picture by running a "
"ComfyUI workflow, look at one with zoom_image, and say what you ran outside Anomalous with log_outside.")
if result.get("isError"):
result["content"] = list(result.get("content") or []) + [{"type": "text", "text": f"{note} Files: {', '.join(files)}"}]
return result
data = result.get("structuredContent")
data = dict(data) if isinstance(data, dict) else {}
data["outside_changes"], data["outside_note"] = files, note
result["structuredContent"] = data
content = list(result.get("content") or [])
text = {"type": "text", "text": tool_text(data)}
if content and isinstance(content[0], dict) and content[0].get("type") == "text":
content[0] = text
else:
content.insert(0, text)
result["content"] = content
return result
def _line(value, limit, what):
value = str(value or "").strip()
if len(value) > limit:
raise ValueError(f"{what} is longer than {limit} characters.")
return value
def log(client, what, why="", files=None, now=None, session="", task_id=None):
"""The AI's word on what it did outside Anomalous, as a step of its task. Returns the step."""
what = _line(what, MAX_TEXT, "what")
if not what:
raise ValueError("what is empty: the command, script or change you made outside Anomalous.")
if files is not None and not isinstance(files, list):
raise ValueError("files is a list of the files you read, wrote or changed.")
named = [_line(item, MAX_FILE, "a file") for item in (files or [])[:MAX_FILES] if str(item or "").strip()]
now = time.time() if now is None else now
step = {"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": "log_outside", "kind": "outside", "ok": True,
"state": "done", "summary": f"log_outside: {what[:80]}", "what": what, "why": _line(why, MAX_TEXT, "why"),
"files": named, "results": []}
with ai_tasks._lock:
task = ai_tasks.current_task(client, now, session=session, task_id=task_id)
ai_tasks._add_step(task, step)
ai_tasks._save(task)
ai_tasks._notify(task["id"])
return step
+598
View File
@@ -0,0 +1,598 @@
"""Panels the AI draws in its task (show_panel), from Anomalous's own blocks, and the answers
the user gives on them.
A panel is a step of the AI's current task (ai_tasks.py) holding a title and a list of blocks:
heading, text, images (from ComfyUI's output folder, with marks: circles or boxes and a label
at places given as fractions of the picture), slider (two pictures over each other, a line to
drag between them), picks (pictures with a reason and "take this one"), models (the user's, by
model_id), diagram (boxes and arrows, laid out by the page; a box can name a canvas node),
table, list, params, choices (buttons; one or several picks) and input (a line of text),
progress, divider; and area (the user's repaint area beside the AI's, ai_repaint.py). Nothing the AI sends is run or rendered as markup, and no drawing of its own
(SVG) is taken: every block is checked here to be one of these shapes, with plain strings and
numbers of bounded size, pictures only from the output folder and models only from the user's
folders. The page draws them (ui_ai_panel.js).
The user's answers come back through POST /anomalous/ai_tasks/{task}/answer and wait in the
step until the AI asks for them (get_answers, which may wait a while for one) or makes its next
successful MCP tool call (attach_pending). The user can also write to the AI in a task
(POST /anomalous/ai_tasks/{task}/message): the message is a step of its own and reaches the AI
the same way.
"""
import os
import re
import threading
import time
import uuid
import folder_paths
from . import ai_marks, ai_tasks
from .mcp_tools import tool_text
MAX_BLOCKS = 30
MAX_ITEMS = 12
MAX_ROWS = 30
MAX_COLUMNS = 8
MAX_WAIT = 55
TONES = ("normal", "muted", "tip", "warning")
LAYOUTS = ("row", "grid", "compare")
INTERACTIVE = ("choices", "input", "picks")
INPAINT_NOTE = (
"inpaint_image is the picture with the area to repaint: LoadImage(image=inpaint_image) gives the picture and, as MASK, that area. "
"inpaint_image is already in ComfyUI's input folder, ready: do not check it with a command; "
"get_image_info(image=inpaint_image, folder='input', include_image=true) shows it and the area, zoom_image a part "
"of it larger. Repaint with a workflow; do not paint the picture with code of your own. "
"Use the picture's own model and prompt (get_image_info on image) and build it with add_node and its wiring "
"(in the picture's workflow, LoadImage -> VAEEncode(pixels, vae) -> SetLatentNoiseMask(samples, mask) in place "
"of the empty latent), e.g. new_workflow: CheckpointLoaderSimple, "
"LoadImage, CLIPTextEncode x2, VAEEncode(pixels, vae) -> SetLatentNoiseMask(samples, mask) -> KSampler "
"(denoise 0.5-0.8 keeps the look; 1.0 with VAEEncodeForInpaint(grow_mask_by 8) replaces it) -> VAEDecode -> "
"SaveImage; GrowMask softens the edge. Say in report what you set and why, ask_user the strength if unsure, "
"and show the old and new picture with a slider panel."
)
COLORS = ("red", "green", "blue", "yellow", "purple")
MAX_MARKS = 8
MAX_NODES = 24
MAX_EDGES = 40
_answered = threading.Condition(ai_tasks._lock)
class PanelError(ValueError):
"""What is wrong with a panel, for the AI to fix."""
def _text(value, limit, where, required=False):
if value is None and not required:
return ""
if not isinstance(value, (str, int, float)) or isinstance(value, bool):
raise PanelError(f"{where} must be text")
text = str(value).strip()
if required and not text:
raise PanelError(f"{where} is empty")
return text[:limit]
def _list(value, limit, where):
if value is None:
return []
if not isinstance(value, list):
raise PanelError(f"{where} must be a list")
if len(value) > limit:
raise PanelError(f"{where} has more than {limit} items")
return value
def _fraction(value, where):
if not isinstance(value, (int, float)) or isinstance(value, bool) or not 0 <= value <= 1:
raise PanelError(f"{where} must be a number from 0 to 1 (a fraction of the picture)")
return round(float(value), 4)
def _marks(value, where):
marks = []
for i, mark in enumerate(_list(value, MAX_MARKS, where)):
here = f"{where}[{i}]"
if not isinstance(mark, dict):
raise PanelError(f"{here} must be {{x, y, w, h | r, label, color}}")
out = {"x": _fraction(mark.get("x"), f"{here}.x"), "y": _fraction(mark.get("y"), f"{here}.y"),
"label": _text(mark.get("label"), 60, f"{here}.label"),
"color": mark.get("color") if mark.get("color") in COLORS else "red"}
if mark.get("w") is not None or mark.get("h") is not None:
out.update(w=_fraction(mark.get("w"), f"{here}.w"), h=_fraction(mark.get("h"), f"{here}.h"))
else:
out["r"] = _fraction(mark.get("r", 0.06), f"{here}.r")
marks.append(out)
return marks
def _image(item, where):
image = item.get("image") if isinstance(item, dict) else item
found = ai_tasks._output_media(str(image or ""))
if not found:
raise PanelError(f"{where}: {image!r} is not a picture in ComfyUI's output folder (subfolder/name.png, as search_images gives it)")
out = {**found, "caption": _text(item.get("caption") if isinstance(item, dict) else "", 120, f"{where}.caption")}
if isinstance(item, dict) and item.get("marks"):
out["marks"] = _marks(item["marks"], f"{where}.marks")
return out
def _diagram(block, where):
nodes, ids = [], set()
for i, node in enumerate(_list(block.get("nodes"), MAX_NODES, f"{where}.nodes")):
here = f"{where}.nodes[{i}]"
if not isinstance(node, dict):
raise PanelError(f"{here} must be {{id, label, detail, node_id}}")
node_id = _text(node.get("id") or f"n{i + 1}", 40, f"{here}.id")
if node_id in ids:
raise PanelError(f"{here}.id {node_id!r} twice")
ids.add(node_id)
out = {"id": node_id, "label": _text(node.get("label"), 40, f"{here}.label", True),
"detail": _text(node.get("detail"), 80, f"{here}.detail"),
"color": node.get("color") if node.get("color") in COLORS else ""}
if node.get("node_id") not in (None, ""):
out["node_id"] = _text(node.get("node_id"), 20, f"{here}.node_id")
nodes.append(out)
if not nodes:
raise PanelError(f"{where}.nodes is empty")
edges = []
for i, edge in enumerate(_list(block.get("edges"), MAX_EDGES, f"{where}.edges")):
here = f"{where}.edges[{i}]"
if not isinstance(edge, dict) or edge.get("from") not in ids or edge.get("to") not in ids:
raise PanelError(f"{here} must be {{from, to, label}} naming two node ids")
edges.append({"from": edge["from"], "to": edge["to"], "label": _text(edge.get("label"), 30, f"{here}.label")})
return {"nodes": nodes, "edges": edges, "direction": "down" if block.get("direction") == "down" else "right"}
def _model(item, where):
from .mcp_tools import ToolError, _resolve_model
from .model_media import _preview_url_for_model
model_id = item.get("model_id") if isinstance(item, dict) else item
try:
folder_type, path_idx, base_dir, file_path = _resolve_model(model_id)
except ToolError as error:
raise PanelError(f"{where}: {error}")
try:
cover = _preview_url_for_model(folder_type, path_idx, base_dir, file_path) or ""
except Exception: # noqa: BLE001 - a card without its cover is still a card
cover = ""
return {"model_id": str(model_id), "folder": folder_type, "path_idx": path_idx,
"rel": os.path.relpath(file_path, base_dir).replace(os.sep, "/"), "name": os.path.basename(file_path),
"cover": cover, "note": _text(item.get("note") if isinstance(item, dict) else "", 160, f"{where}.note")}
def clean_block(block, at, taken):
"""One block as the page may draw it, or PanelError."""
where = f"blocks[{at}]"
if not isinstance(block, dict) or not isinstance(block.get("type"), str):
raise PanelError(f"{where} must be an object with a type")
kind = block["type"]
out = {"type": kind}
if kind == "heading":
out["text"] = _text(block.get("text"), 120, f"{where}.text", True)
elif kind == "text":
out["text"] = _text(block.get("text"), 4000, f"{where}.text", True)
out["tone"] = block.get("tone") if block.get("tone") in TONES else "normal"
elif kind == "images":
items = _list(block.get("items"), MAX_ITEMS, f"{where}.items")
if not items:
raise PanelError(f"{where}.items is empty")
out["items"] = [_image(item, f"{where}.items[{i}]") for i, item in enumerate(items)]
out["layout"] = block.get("layout") if block.get("layout") in LAYOUTS else "row"
elif kind == "slider":
out.update(before=_image(block.get("before"), f"{where}.before"), after=_image(block.get("after"), f"{where}.after"),
before_label=_text(block.get("before_label"), 40, f"{where}.before_label"),
after_label=_text(block.get("after_label"), 40, f"{where}.after_label"))
elif kind == "picks":
items, ids = [], set()
raw = _list(block.get("items"), MAX_ITEMS, f"{where}.items")
if len(raw) < 2:
raise PanelError(f"{where}.items needs 2 to {MAX_ITEMS} pictures")
for i, item in enumerate(raw):
here = f"{where}.items[{i}]"
if not isinstance(item, dict):
raise PanelError(f"{here} must be {{id, image, label, reason, recommended}}")
item_id = _text(item.get("id") or f"p{i + 1}", 40, f"{here}.id")
if item_id in ids:
raise PanelError(f"{here}.id {item_id!r} twice")
ids.add(item_id)
picture = _image(item, here)
items.append({**picture, "id": item_id, "label": _text(item.get("label"), 60, f"{here}.label"),
"reason": _text(item.get("reason"), 200, f"{here}.reason"), "recommended": item.get("recommended") is True})
out.update(question=_text(block.get("question"), 200, f"{where}.question"), items=items,
multiple=block.get("multiple") is True)
elif kind == "run_map": # a run's real shape (ai_run_map.py), drawn by the page from the step; only show_run_map makes it
out["step"] = _text(block.get("step"), 20, f"{where}.step", True)
elif kind == "area": # the user's repaint area and the AI's, pictures in the task's marks folder; only ai_repaint makes it
if not all(ai_marks.FILE_NAME.fullmatch(str(block.get(side))) for side in ("yours", "ai")):
raise PanelError(f"{where}: an area block shows pictures Anomalous made")
out.update({side: block[side] for side in ("yours", "ai")}, grow=int(block.get("grow") or 0), feather=int(block.get("feather") or 0))
elif kind == "diagram":
out.update(_diagram(block, where))
elif kind == "models":
items = _list(block.get("items"), MAX_ITEMS, f"{where}.items")
if not items:
raise PanelError(f"{where}.items is empty")
out["items"] = [_model(item, f"{where}.items[{i}]") for i, item in enumerate(items)]
elif kind == "table":
columns = [_text(c, 60, f"{where}.columns") for c in _list(block.get("columns"), MAX_COLUMNS, f"{where}.columns")]
rows = []
for r, row in enumerate(_list(block.get("rows"), MAX_ROWS, f"{where}.rows")):
cells = _list(row, MAX_COLUMNS, f"{where}.rows[{r}]")
rows.append([_text(cell, 200, f"{where}.rows[{r}]") for cell in cells])
if not rows:
raise PanelError(f"{where}.rows is empty")
out.update(columns=columns, rows=rows)
elif kind == "list":
items = [_text(item, 300, f"{where}.items") for item in _list(block.get("items"), MAX_ROWS, f"{where}.items")]
if not items:
raise PanelError(f"{where}.items is empty")
out.update(items=items, ordered=block.get("ordered") is True)
elif kind == "params":
items = []
for i, item in enumerate(_list(block.get("items"), MAX_ROWS, f"{where}.items")):
if not isinstance(item, dict):
raise PanelError(f"{where}.items[{i}] must be {{label, value}}")
items.append({"label": _text(item.get("label"), 60, f"{where}.items[{i}].label", True),
"value": _text(item.get("value"), 300, f"{where}.items[{i}].value")})
if not items:
raise PanelError(f"{where}.items is empty")
out["items"] = items
elif kind == "choices":
options, ids = [], set()
raw = _list(block.get("options"), 8, f"{where}.options")
if len(raw) < 2:
raise PanelError(f"{where}.options needs 2 to 8 options")
for i, option in enumerate(raw):
if not isinstance(option, dict):
option = {"label": option}
label = _text(option.get("label"), 80, f"{where}.options[{i}].label", True)
option_id = _text(option.get("id") or f"o{i + 1}", 40, f"{where}.options[{i}].id")
if option_id in ids:
raise PanelError(f"{where}.options: id {option_id!r} twice")
ids.add(option_id)
options.append({"id": option_id, "label": label, "detail": _text(option.get("detail"), 200, f"{where}.options[{i}].detail"),
**({"recommended": True} if option.get("recommended") is True else {}),
**({"label_key": option["label_key"]} if re.fullmatch(r"ai[A-Z]\w{1,40}", str(option.get("label_key"))) else {})})
# other: a line for an answer none of the options is
out.update(question=_text(block.get("question"), 200, f"{where}.question"), options=options,
multiple=block.get("multiple") is True, other=block.get("other") is True)
elif kind == "input":
out.update(question=_text(block.get("question"), 200, f"{where}.question", True),
placeholder=_text(block.get("placeholder"), 80, f"{where}.placeholder"))
elif kind == "progress":
value, top = block.get("value"), block.get("max", 100)
if not all(isinstance(n, (int, float)) and not isinstance(n, bool) for n in (value, top)) or top <= 0:
raise PanelError(f"{where}: value and max must be numbers, max above 0")
out.update(label=_text(block.get("label"), 120, f"{where}.label"), value=max(0, min(value, top)), max=top)
elif kind == "divider":
pass
else:
raise PanelError(f"{where}.type {kind!r} is not one of heading, text, images, slider, picks, models, diagram, "
"table, list, params, choices, input, progress, divider")
if kind in INTERACTIVE or block.get("id") is not None:
block_id = _text(block.get("id") or f"q{at + 1}", 40, f"{where}.id")
if block_id in taken:
raise PanelError(f"{where}.id {block_id!r} twice")
taken.add(block_id)
out["id"] = block_id
return out
def clean_blocks(blocks):
blocks = _list(blocks, MAX_BLOCKS, "blocks")
if not blocks:
raise PanelError("blocks is empty")
taken = set()
return [clean_block(block, at, taken) for at, block in enumerate(blocks)]
def _block_ids(blocks):
"""Stable handles for deliberate edits; old records get them only when updated."""
taken = {block["id"] for block in blocks if block.get("id")}
out = []
for block in blocks:
block = dict(block)
if not block.get("id"):
block_id = uuid.uuid4().hex[:10]
while block_id in taken:
block_id = uuid.uuid4().hex[:10]
block["id"] = block_id
taken.add(block_id)
out.append(block)
return out
def _merge_blocks(existing, incoming):
"""Keep the order already seen, append new output, edit only identified blocks in place."""
merged = list(existing)
by_id = {block["id"]: at for at, block in enumerate(existing) if block.get("id")}
used = set()
for raw, block in zip(incoming, clean_blocks(incoming)):
if block["type"] in INTERACTIVE and not raw.get("id"):
# Positional q1/q2 defaults cannot identify a question in a partial append.
block.pop("id")
at = by_id.get(block.get("id"))
if at is None and not block.get("id"):
# A client may send its entire previous layout again. That cannot reorder it.
at = next((i for i, old in enumerate(existing) if i not in used
and {key: value for key, value in old.items() if key != "id"} == block), None)
if at is None and block["type"] == "progress":
matches = [i for i, old in enumerate(existing) if old["type"] == "progress"
and old.get("label") == block.get("label")]
if len(matches) == 1:
at = matches[0]
if at is None:
merged.append(block)
else:
if existing[at]["type"] != block["type"]:
raise PanelError("An existing block id must keep its type; use a new id for new output.")
merged[at] = {**block, **({"id": existing[at]["id"]} if existing[at].get("id") else {})}
used.add(at)
if len(merged) > MAX_BLOCKS:
raise PanelError(f"The panel would exceed {MAX_BLOCKS} blocks; use show_panel for the next stage of work.")
return _block_ids(merged)
def _find(panel_id):
"""(task, step) holding panel `panel_id`, or (None, None). Call under the lock."""
for task in ai_tasks._load().values():
for step in task.get("steps") or []:
if step.get("kind") == "panel" and step.get("panel", {}).get("id") == panel_id:
return task, step
return None, None
def show(client, title, blocks, now=None, session="", task_id=None, tool="show_panel", extra=None):
title = _text(title, 120, "title", True)
panel = {"id": uuid.uuid4().hex[:10], "title": title, "blocks": _block_ids(clean_blocks(blocks)), "closed": False}
now = time.time() if now is None else now
step = {"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": tool, "kind": "panel", "ok": True,
"summary": f"{tool}: {title}", "panel": panel, "answers": {}, "results": [], **(extra or {})}
with ai_tasks._lock:
task = ai_tasks.current_task(client, now, session=session, task_id=task_id, follow_up=True)
ai_tasks._add_step(task, step)
ai_tasks._save(task)
ai_tasks._notify(task["id"], {"panel": title, "asks": any(b["type"] in INTERACTIVE for b in panel["blocks"])})
return panel
def update(panel_id, title=None, blocks=None, closed=None):
with ai_tasks._lock:
task, step = _find(str(panel_id))
if not step:
raise PanelError(f"No panel {panel_id!r}; show_panel returns its id.")
# Validate the whole update before replacing the saved panel.
panel = dict(step["panel"])
if title is not None:
panel["title"] = _text(title, 120, "title", True)
if blocks is not None:
panel["blocks"] = _merge_blocks(panel["blocks"], blocks)
panel["changed"] = time.time() # when its numbers were written: the page greys a progress bar the task went past
if closed is not None:
panel["closed"] = bool(closed)
step["panel"] = panel
task["updated"] = max(task.get("updated", 0), time.time())
ai_tasks._save(task)
ai_tasks._notify(task["id"])
return panel
def answer(task_id, step_id, block_id, choice=None, text=None, now=None, skip=False, extra=None):
"""The user's answer to one block of a panel (from the page), or `skip` it, with `extra` for the AI
(ai_repaint.prepare). Returns the step's answers."""
with _answered:
task = ai_tasks._load().get(task_id)
step = next((item for item in (task or {}).get("steps") or [] if item.get("id") == step_id), None)
if not step or step.get("kind") != "panel":
raise PanelError("No such panel")
if step["panel"].get("closed"):
raise PanelError("The AI closed this panel")
block = next((b for b in step["panel"]["blocks"] if b.get("id") == block_id), None)
if not block or block["type"] not in INTERACTIVE:
raise PanelError("No such question")
if block_id in step.setdefault("answers", {}):
raise PanelError("Already answered")
entry = {"at": time.time() if now is None else now, "delivered": False, **(extra or {})}
if skip:
entry["skipped"] = True
elif block["type"] == "choices" and block.get("other") and not choice and isinstance(text, str) and text.strip():
entry.update(choice=["other"], labels=["other"], text=_text(text, 2000, "text", True))
elif block["type"] in ("choices", "picks"):
picked = choice if isinstance(choice, list) else [choice]
options = block["options"] if block["type"] == "choices" else block["items"]
known = {option["id"]: option.get("label") or option["id"] for option in options}
picked = [str(item) for item in picked if str(item) in known]
if not picked or (len(picked) > 1 and not block.get("multiple")):
raise PanelError("Pick one of the options")
entry.update(choice=picked, labels=[known[item] for item in picked])
else:
value = _text(text, 2000, "text", True)
entry["text"] = value
step["answers"][block_id] = entry
task["updated"] = max(task.get("updated", 0), entry["at"])
ai_tasks._save(task)
_answered.notify_all()
answers = dict(step["answers"])
ai_tasks._notify(task_id)
return answers
MAX_MESSAGE = 2000
def leave_message(task_id, text, refs=None, now=None):
"""A message from the user to the AI working on task `task_id`, with what its "@" point at and
the pictures attached (ai_message_refs.py); words may be left out when something is attached.
Returns the step."""
from . import ai_message_refs
try:
refs = ai_message_refs.clean(refs)
except ai_message_refs.RefsError as error:
raise PanelError(str(error))
text = _text(text, MAX_MESSAGE, "text", not refs)
now = time.time() if now is None else now
step = {"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": "user_message", "kind": "user", "ok": True,
"summary": "user_message", "text": text, "delivered": False, "results": [], **({"refs": refs} if refs else {})}
with _answered:
task = ai_tasks._load().get(task_id)
if not task:
raise PanelError("No such task")
task["steps"].append(step)
del task["steps"][:-ai_tasks.MAX_STEPS]
task["updated"] = max(task.get("updated", 0), now)
task["ended"] = None # writing to the AI goes on with the task: its next call comes back here
ai_tasks._save(task)
_answered.notify_all()
ai_tasks._notify(task_id)
return step
def _collect(client, panel_id, task_id=None):
"""Answers and messages not given to the AI yet (its own tasks, task `task_id`, or panel
`panel_id` and the messages of that panel's task), marked as given. Under the lock."""
out, waiting, messages, changed = [], [], [], set()
tasks = ai_tasks._load()
panel_task = _find(panel_id)[0] if panel_id is not None else None
named_task = tasks.get(task_id) if task_id else None
for task in tasks.values():
if panel_id is not None:
if task is not panel_task:
continue
elif task_id is not None:
if task is not named_task:
continue
elif task.get("client") != client:
continue
for step in task.get("steps") or []:
if step.get("kind") == "user" and not step.get("delivered") and (panel_id is None or task is panel_task):
step.update(delivered=True, delivered_at=time.time())
changed.add(task["id"])
message = {"id": step["id"], "task": task["id"], "text": step["text"], "at": step["at"]}
if step.get("tool") == "user_activity":
message["kind"] = "change" # something the user changed in Anomalous, not words to the AI
elif step.get("tool") == "user_pictures":
from . import ai_pictures
message.update(ai_pictures.message_of(step)) # the pictures themselves go with it
elif step.get("refs"):
from . import ai_message_refs
message.update(ai_message_refs.message_of(step)) # what its "@" point at, pictures attached
elif step.get("tool") == "user_stop":
message["kind"] = "stop" # the user stopped the AI's run (ai_run_control.py)
elif step.get("tool") == "user_toolbox":
message["kind"] = "toolbox" # what the user picked for the AI to work with (ai_toolbox.py)
elif step.get("tool") == "user_marks":
# Marks on a picture: the picture's path for the AI, and the marked one attached (ai_marks.image_items).
message.update(kind="marks", image="/".join(part for part in (step["image"].get("subfolder"), step["image"]["filename"]) if part),
mode=step.get("mode") or ("local" if step.get("inpaint") else "auto"),
_marked=ai_marks.path_of(task["id"], step.get("marked")))
if step.get("inpaint"):
message.update(inpaint_image=step["inpaint"], inpaint_note=INPAINT_NOTE)
messages.append(message)
continue
asked = step.get("gate") or {}
if asked.get("timed_out") and asked.get("decision") and not asked.get("delivered") and panel_id is None:
asked["delivered"] = True
changed.add(task["id"])
messages.append({"id": step["id"], "task": task["id"], "kind": "gate", "at": asked.get("at"),
"text": (f"The user allowed {step['tool']}: call it again with the same arguments now."
if asked["decision"] == "allow" else
f"The user said no to {step['tool']}." + (f" They said: {asked['text']}" if asked.get("text") else ""))})
continue
panel = step.get("panel")
if step.get("kind") != "panel" or (panel_id is not None and panel["id"] != panel_id):
continue
for block in panel["blocks"]:
if block["type"] not in INTERACTIVE:
continue
given = step.get("answers", {}).get(block["id"])
if not given:
if not panel.get("closed"):
waiting.append({"panel": panel["id"], "question": block["id"]})
continue
if given.get("delivered"):
continue
given.update(delivered=True, delivered_at=time.time())
changed.add(task["id"])
item = {"panel": panel["id"], "question": block["id"], "asked": block.get("question", "")}
item.update({key: given[key] for key in ("choice", "labels", "text", "skipped", "inpaint_image", "note", "recommended_on_timeout") if key in given})
out.append(item)
for tid in changed:
ai_tasks._save(tasks[tid])
return out, waiting, messages, changed
def wake():
"""Anything waiting in get_answers looks again (the user added something to a task)."""
with _answered:
_answered.notify_all()
def get_answers(client, panel_id=None, wait_seconds=0, task_id=None):
"""The user's new answers and messages; with `wait_seconds`, waits up to that long (MAX_WAIT)
for one: for a panel's question still open, or (no panel named) for anything the user sends."""
deadline = time.monotonic() + max(0, min(float(wait_seconds or 0), MAX_WAIT))
panel_id = str(panel_id) if panel_id else None
notified = set()
try:
with _answered:
if panel_id is not None and _find(panel_id)[1] is None:
raise PanelError(f"No panel {panel_id!r}; show_panel returns its id.")
while True:
answers, waiting, messages, changed = _collect(client, panel_id, task_id=task_id)
notified.update(changed)
left = deadline - time.monotonic()
if answers or messages or left <= 0 or (panel_id is not None and not waiting):
answer = {"answers": answers, "still_waiting": waiting}
if messages:
answer["messages"] = messages
return answer
_answered.wait(left)
finally:
for tid in notified:
ai_tasks._notify(tid)
def attach_pending(result, client, task_id=None):
"""Attaches unread user messages and panel answers to a successful tool result so an AI
that does not call get_answers still receives them once."""
if not isinstance(result, dict) or result.get("isError"):
return result
with _answered:
answers, waiting, messages, changed = _collect(client, None, task_id=task_id)
for tid in changed:
ai_tasks._notify(tid)
if not answers and not messages:
return result
data = result.get("structuredContent")
data = dict(data) if isinstance(data, dict) else {}
if messages:
data["user_messages"] = messages
if answers:
data["user_answers"] = answers
if waiting:
data["still_waiting"] = waiting
data["user_notice"] = (
"The user sent new input in the AI workbench. Read user_messages / user_answers and follow them; you may "
"reply in your chat or in Anomalous (report(reply_to=<its id>)), but once you and the user settle what to do, "
"record it with log_decision so the decision stays in Anomalous; "
"kind=change messages are changes the user made themselves while you worked: keep them, do not undo them; "
"kind=marks messages are places the user marked on a picture to change (the marked picture is attached): "
"say what you see and how you would fix it (show_panel can show the picture with your own marks), "
"ask_user which fix to do, and log_decision the fix chosen (related: the marks' id); "
"kind=toolbox messages are the nodes, combos, workflows, prompts and models the user picked for you to use; "
"kind=stop: the user stopped your run, so do not run it again until they say what to change; "
"kind=pictures: pictures the user handed you (attached), with made_with read from each picture's own workflow."
)
pictures = ai_marks.image_items(messages)
result["structuredContent"] = data
content = list(result.get("content") or [])
if content and isinstance(content[0], dict) and content[0].get("type") == "text":
content[0] = {"type": "text", "text": tool_text(data)}
else:
content.insert(0, {"type": "text", "text": tool_text(data)})
result["content"] = content + pictures
return result
+158
View File
@@ -0,0 +1,158 @@
"""Pictures the user hands the AI working on a task (the gallery card's menu, the image inspector's
✦): a user step with the pictures (shown on the timeline like the AI's own), which the AI gets as
a message (kind "pictures", ai_panels._collect) with each picture attached as a 512 px copy and
what made it in a few words (brief(): models, LoRAs, prompts, size, sampler; read from the
picture's own workflow by ai_run_map.py), so nothing needs describing. Pictures handed one after
another before the AI looked join one step. mark_gone() tells the page which of a task's
pictures were deleted since. save_upload() keeps a picture the user dropped or pasted into a
message in the output folder's ai_inbox, so the gallery has it and the message carries it like
any other: read as a picture first, size-capped, never over a file.
"""
import io
import json
import os
import re
import time
import uuid
from PIL import Image
from . import ai_run_map, ai_tasks
from .image_search import read_png_text
from .mcp_tools import ToolError, _resolve_image
MAX_PICTURES = 8
INBOX = "ai_inbox" # under the output folder
MAX_UPLOAD = 25 * 1024 * 1024
UPLOAD_TYPES = {"PNG": ".png", "JPEG": ".jpg", "WEBP": ".webp", "GIF": ".gif"}
class PicturesError(ValueError):
pass
def brief(path):
"""What made the picture at `path`, from its own workflow: {models, loras, positive, negative, size, sampler} or {}."""
try:
prompt = json.loads((read_png_text(path) if path.lower().endswith(".png") else {}).get("prompt") or "null")
shape = ai_run_map.map_of(prompt) if isinstance(prompt, dict) else None
except (OSError, ValueError):
shape = None
if not shape:
return {}
out = {}
for box in shape["boxes"]:
facts = box["facts"]
if box["role"] == "model" and facts.get("file"):
out.setdefault("models", []).append(facts["file"])
elif box["role"] == "lora" and facts.get("file"):
out.setdefault("loras", []).append(f"{facts['file']} ({facts.get('strength', '?')})")
elif box["role"] == "prompt" and facts.get("text"):
out.setdefault("negative" if facts.get("polarity") == "negative" else "positive", facts["text"])
elif box["role"] == "sampler":
out["sampler"] = {key: facts[key] for key in ("steps", "cfg", "sampler_name", "scheduler", "denoise") if key in facts}
if facts.get("width") and "size" not in out:
out["size"] = f"{facts['width']}x{facts['height']}"
return out
def mark_gone(task):
"""Marks each output picture `task` shows (results, panels, marks) that is no longer in the output
folder, deleted or moved: "gone": true, so the page says so instead of a broken picture."""
root = ai_tasks.folder_paths.get_output_directory()
def walk(value):
if isinstance(value, dict):
if value.get("folder") == "output" and value.get("filename") and \
not os.path.isfile(os.path.join(root, value.get("subfolder") or "", value["filename"])):
value["gone"] = True
for item in value.values():
walk(item)
elif isinstance(value, list):
for item in value:
walk(item)
walk((task or {}).get("steps"))
return task
def save_upload(data, name="", now=None):
"""Keeps picture bytes `data` (the file's own name `name`, for its stem) in output/ai_inbox.
Returns its "ai_inbox/name.ext"."""
if not data or len(data) > MAX_UPLOAD:
raise PicturesError(f"A picture up to {MAX_UPLOAD // (1024 * 1024)} MB")
try:
with Image.open(io.BytesIO(data)) as image:
kind = image.format
image.verify()
except (OSError, ValueError, SyntaxError, Image.DecompressionBombError):
raise PicturesError("Not a picture Anomalous can read (PNG, JPEG, WebP or GIF)")
if kind not in UPLOAD_TYPES:
raise PicturesError("Not a picture Anomalous can read (PNG, JPEG, WebP or GIF)")
stem = re.sub(r"[^\w\- ]+", "", os.path.splitext(os.path.basename(str(name or "")))[0]).strip()[:60] or "pasted"
folder = os.path.join(ai_tasks.folder_paths.get_output_directory(), INBOX)
os.makedirs(folder, exist_ok=True)
stamp = time.strftime("%Y%m%d-%H%M%S", time.localtime(time.time() if now is None else now))
for count in range(1, 100):
filename = f"{stem}_{stamp}{'' if count == 1 else f' ({count})'}{UPLOAD_TYPES[kind]}"
try:
with open(os.path.join(folder, filename), "xb") as out: # never over a file
out.write(data)
except FileExistsError:
continue
return f"{INBOX}/{filename}"
raise PicturesError("No free name for the picture")
def hand(task_id, images, text="", now=None):
"""Hands output pictures `images` ("subfolder/name.png") to task `task_id`'s AI. Returns the step."""
if not isinstance(images, list) or not 1 <= len(images) <= MAX_PICTURES:
raise PicturesError(f"Hand 1 to {MAX_PICTURES} pictures")
pictures = []
for image in images:
try:
rel = _resolve_image(image)[0]
except ToolError as error:
raise PicturesError(str(error))
subfolder, _, filename = rel.rpartition("/")
picture = {"type": "image", "folder": "output", "filename": filename, "subfolder": subfolder}
if picture not in pictures:
pictures.append(picture)
now = time.time() if now is None else now
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
if not task or task.get("ended"):
raise PicturesError("No such open task")
last = task["steps"][-1] if task["steps"] else None
if last and last.get("tool") == "user_pictures" and not last.get("delivered") and not text:
step = last # the AI has not looked yet: one step with all of them
step["results"] = (step["results"] + [item for item in pictures if item not in step["results"]])[:MAX_PICTURES]
step["at"] = now
else:
step = {"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": "user_pictures", "kind": "user", "ok": True,
"state": "done", "summary": "user_pictures", "delivered": False, "results": pictures,
"text": ai_tasks._short(str(text or "").strip(), 1000)}
task["steps"].append(step)
del task["steps"][:-ai_tasks.MAX_STEPS]
task["updated"] = max(task.get("updated", 0), now)
ai_tasks._save(task)
ai_tasks._notify(task_id)
return step
def message_of(step):
"""The AI's message for a pictures step: the pictures' paths and what made each; their files go
in "_pictures" for ai_marks.image_items to attach."""
paths, briefs, files = [], [], []
for item in step.get("results") or []:
rel = "/".join(part for part in (item.get("subfolder"), item["filename"]) if part)
try:
path = _resolve_image(rel)[1]
except ToolError:
continue # deleted since: the rest still go
paths.append(rel)
briefs.append({"image": rel, **brief(path)})
files.append(path)
words = step.get("text") or "The user handed you these pictures in Anomalous."
return {"kind": "pictures", "text": words, "images": paths, "made_with": briefs, "_pictures": files}
+184
View File
@@ -0,0 +1,184 @@
"""Where to repaint a picture the user marked, as the AI proposes it and the user agrees on the AI
workbench (the propose_repaint_area tool, mcp_task_tools.py).
The user's marks (ai_marks.py) are rough, and the AI sees the marked picture: it may move, reshape,
grow and soften the area: `areas` (boxes or ellipses as fractions of the picture, x/y the top left
as in the marks), `grow` (pixels added around each) and `feather` (pixels of soft edge). Anomalous
draws the user's area and the AI's over the picture into the task's marks folder and puts both side
by side on a panel (block "area") with three answers: the AI's, the user's own, or marking it
again; a line of their own says what to change. When the user picks an area, the picture goes
into ComfyUI's input folder with that area as its alpha (ai_marks.write_inpaint), so LoadImage
gives the picture and the area as its MASK; the answer the AI reads carries the file's name and a
short note of the usual repaint nodes. How to build the repaint on the canvas stays the AI's.
"""
import os
import uuid
import folder_paths
from . import ai_marks, ai_panels, ai_tasks
SHAPES = ("box", "ellipse")
MAX_AREAS = 8
MAX_GROW = 256
MAX_FEATHER = 128
PREVIEW_EDGE = 1024
TINT = (255, 64, 64)
CHOICES = ( # id, label for the AI, the page's words
("ai", "Use the AI's area", "aiRepaintUseAi"),
("mine", "Use the user's own marks", "aiRepaintUseMine"),
("again", "Mark it again", "aiRepaintAgain"),
)
AGAIN_NOTE = "The user marks the picture again: wait for their new marks with get_answers, then propose again."
WORDS_NOTE = "The user said what to change about the area: propose_repaint_area again with it."
class RepaintError(ValueError):
pass
def _number(value, low, high, what):
if isinstance(value, bool) or not isinstance(value, (int, float)) or not low <= value <= high:
raise RepaintError(f"{what} is a number from {low} to {high}")
return value
def _areas(areas):
if not isinstance(areas, list) or not 0 < len(areas) <= MAX_AREAS:
raise RepaintError(f"areas lists 1 to {MAX_AREAS} areas")
out = []
for at, area in enumerate(areas):
where = f"areas[{at}]"
if not isinstance(area, dict) or area.get("shape", "box") not in SHAPES:
raise RepaintError(f"{where} is {{shape: box | ellipse, x, y, w, h}}")
x, y = (_number(area.get(key), 0, 1, f"{where}.{key}") for key in ("x", "y"))
w, h = (_number(area.get(key), 0.005, 1, f"{where}.{key}") for key in ("w", "h"))
out.append({"shape": area.get("shape", "box"), "x": round(x, 4), "y": round(y, 4),
"w": round(min(w, 1 - x), 4), "h": round(min(h, 1 - y), 4)})
return out
def _from_marks(marks):
"""The user's marks as areas: a circle an ellipse in its box, a line or an arrow the box around it."""
return [{"shape": "ellipse" if mark["shape"] == "circle" else "box", **{key: mark[key] for key in ("x", "y", "w", "h")}}
for mark in marks if mark["w"] > 0 and mark["h"] > 0]
def area_image(size, areas, grow=0, feather=0):
"""An "L" image of `size`, white over `areas` grown by `grow` pixels, `feather` pixels soft at the edge."""
from PIL import Image, ImageDraw, ImageFilter
width, height = size
area = Image.new("L", size, 0)
draw = ImageDraw.Draw(area)
for shape in areas:
box = [shape["x"] * width - grow, shape["y"] * height - grow,
(shape["x"] + shape["w"]) * width + grow, (shape["y"] + shape["h"]) * height + grow]
(draw.ellipse if shape["shape"] == "ellipse" else draw.rectangle)(box, fill=255)
return area.filter(ImageFilter.GaussianBlur(feather / 2)) if feather else area
def _user_area(marks_step):
"""How to make the user's own area: what they painted when they asked for only the marked areas, else their marks."""
painted = marks_step.get("inpaint")
if painted:
def from_painted(size):
from PIL import Image, ImageOps
with Image.open(os.path.join(folder_paths.get_input_directory(), painted)) as kept:
return ImageOps.invert(kept.getchannel("A")).resize(size)
return from_painted
areas = _from_marks(marks_step.get("marks") or [])
if not areas:
raise RepaintError("These marks hold no area to repaint")
return lambda size: area_image(size, areas)
def _preview(task_id, picture, area_of):
"""The picture, made small, with the area tinted, kept in the task's marks folder: its name."""
from PIL import Image
try:
with Image.open(ai_marks.picture_path(picture)) as original:
base = original.convert("RGB")
except OSError:
raise RepaintError("The marked picture cannot be read")
base.thumbnail((PREVIEW_EDGE, PREVIEW_EDGE))
area = area_of(base.size).point(lambda value: value * 55 // 100)
shown = Image.composite(Image.new("RGB", base.size, TINT), base, area)
name = f"{uuid.uuid4().hex[:10]}.png"
os.makedirs(ai_tasks.marks_dir(task_id), exist_ok=True)
with open(os.path.join(ai_tasks.marks_dir(task_id), name), "xb") as target:
shown.save(target, "PNG")
return name
def propose(client, title, marks_id, areas=None, grow=12, feather=8, note="", session=""):
"""Shows the user their area and the AI's for marks `marks_id`; returns the panel to wait on."""
with ai_tasks._lock:
found = next(((task, step) for task in ai_tasks._load().values() for step in task.get("steps") or []
if step.get("id") == marks_id and step.get("tool") == "user_marks"), None)
if not found:
raise RepaintError("marks is the id of the user's marks (user_messages[].id, kind marks)")
task, marks_step = found
grow = int(_number(grow, 0, MAX_GROW, "grow"))
feather = int(_number(feather, 0, MAX_FEATHER, "feather"))
shapes = _areas(areas) if areas else _from_marks(marks_step.get("marks") or [])
if not shapes:
raise RepaintError("These marks hold no area: give areas")
picture = marks_step["image"]
with_edges = {"grow": grow, "feather": feather}
try:
from PIL import Image
with Image.open(ai_marks.picture_path(picture)) as original:
scale = min(1, PREVIEW_EDGE / max(original.size)) # grow and feather are pixels of the whole picture
except OSError:
raise RepaintError("The marked picture cannot be read")
yours = _preview(task["id"], picture, _user_area(marks_step))
ai = _preview(task["id"], picture, lambda size: area_image(size, shapes, grow * scale, feather * scale))
blocks = [{"type": "area", "yours": yours, "ai": ai, **with_edges}]
if note:
blocks.append({"type": "text", "text": note})
blocks.append({"type": "choices", "id": "answer", "other": True,
"options": [{"id": key, "label": label, "label_key": words, **({"recommended": True} if key == "ai" else {})}
for key, label, words in CHOICES]})
try:
return ai_panels.show(client, title, blocks, session=session, task_id=task["id"], tool="propose_repaint_area",
extra={"repaint": {"marks": marks_id, "areas": shapes, **with_edges}})
except ai_panels.PanelError as error:
raise RepaintError(str(error))
def prepare(task_id, step_id, choice, text, skip=False):
"""What the user's answer to a repaint panel brings the AI, made before it is recorded: the repaint
picture for an area picked (inpaint_image and its note; "your call" takes the AI's), else what to
do now; {} for other panels."""
step = next((item for item in (ai_tasks.get_task(task_id) or {}).get("steps") or [] if item.get("id") == step_id), None)
asked = (step or {}).get("repaint")
if not asked:
return {}
picked = "ai" if skip else (choice[0] if isinstance(choice, list) and choice else choice)
if picked not in ("ai", "mine"):
return {"note": WORDS_NOTE if str(text or "").strip() and not picked else AGAIN_NOTE}
marks_step = next((item for task in ai_tasks._load().values() for item in task.get("steps") or []
if item.get("id") == asked["marks"]), None)
if not marks_step:
raise RepaintError("The marks are gone")
area_of = (_user_area(marks_step) if picked == "mine"
else lambda size: area_image(size, asked["areas"], asked["grow"], asked["feather"]))
try:
name = ai_marks.write_inpaint(marks_step["image"], area_of, uuid.uuid4().hex[:10])
except ai_marks.MarksError as error:
raise RepaintError(str(error))
return {"inpaint_image": name, "note": ai_panels.INPAINT_NOTE}
def discard(prepared):
"""The repaint picture `prepare` made, when the answer was refused after all (the plugin's own file)."""
if prepared.get("inpaint_image"):
try:
os.remove(os.path.join(folder_paths.get_input_directory(), prepared["inpaint_image"]))
except OSError:
pass
+55
View File
@@ -0,0 +1,55 @@
"""Stopping a run an AI started (the workbench's stop button, the AI's stop_run): its prompts still
waiting leave ComfyUI's queue and the one running is interrupted, only while it is still that one
(ComfyUI's own cancel; never anyone else's prompt). The run's watcher (ai_task_runs.py) then ends
the step as "cancelled" with who stopped it; when the user did, the AI is told with its next answer."""
import time
import uuid
from . import ai_task_runs, ai_tasks
class RunControlError(ValueError):
pass
def _interrupt(queue, prompt_id):
"""Interrupts `prompt_id` if it is the prompt running now; True when it was."""
if hasattr(queue, "interrupt_if_running"):
return bool(queue.interrupt_if_running(prompt_id))
running, _ = queue.get_current_queue() # an older ComfyUI: check, then the global interrupt
if not any(item[1] == prompt_id for item in running):
return False
import nodes
nodes.interrupt_processing()
return True
def stop(step_id, by="user", task_id=None, now=None):
"""Stops run step `step_id` (in task `task_id` when given). Returns {"task", "removed", "interrupted"}."""
now = time.time() if now is None else now
with ai_tasks._lock:
tasks = ai_tasks._load()
task = next((item for item in tasks.values() if (not task_id or item["id"] == task_id)
and any(step["id"] == step_id for step in item.get("steps") or [])), None)
step = next((item for item in (task or {}).get("steps", []) if item["id"] == step_id), None)
if not step or step.get("tool") != "run_workflow":
raise RunControlError("No such run in this task")
if step.get("state") != "running" or not step.get("prompt_ids"):
raise RunControlError("This run is not going any more")
prompts = list(step["prompt_ids"])
step["stop"] = by
if by == "user":
task["steps"].append({
"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": "user_stop", "kind": "user", "ok": True,
"state": "done", "summary": "user_stop", "stopped": step_id, "delivered": False, "results": [],
"text": "The user stopped your run in Anomalous. Do not start it again unless they ask; ask_user what to change.",
})
del task["steps"][:-ai_tasks.MAX_STEPS]
task["updated"] = max(task.get("updated", 0), now)
ai_tasks._save(task)
queue = ai_task_runs._queue()
removed = sum(1 for prompt_id in prompts if queue.delete_queue_item(lambda item, pid=prompt_id: item[1] == pid))
interrupted = sum(1 for prompt_id in prompts if _interrupt(queue, prompt_id))
ai_tasks._notify(task["id"])
return {"task": task["id"], "removed": removed, "interrupted": interrupted}
+180
View File
@@ -0,0 +1,180 @@
"""The shape of a workflow run in a few boxes, so the user sees what the AI really ran: read from
the prompt ComfyUI queued (ai_task_runs.py keeps it on the run's step as "map"), never from
what the AI says. Plumbing (reroutes, primitives, notes) is left out and wired through; each
box has a role (model, lora, prompt, size, sampler, image, upscale, decode, save, other) and a
few facts the page words in the user's language (ui_ai_run_map.js); the links carry ComfyUI's
types. changes() says what differs from the run before it: a setting, a node added or gone.
"""
import os
from .model_constants import MODEL_EXTENSIONS
MAX_BOXES = 24 # what a stage diagram holds (ai_panels.MAX_NODES)
MAX_EDGES = 40
MAX_CHANGES = 10
HIDDEN = {"Reroute", "Note", "MarkdownNote", "PrimitiveNode", "PrimitiveInt", "PrimitiveFloat", "PrimitiveString",
"PrimitiveStringMultiline", "PrimitiveBoolean"}
SEEDS = {"seed", "noise_seed"}
SAMPLER_FACTS = ("steps", "cfg", "sampler_name", "scheduler", "denoise")
TEXT_CHARS = 80
VALUE_CHARS = 60
def _linked(value):
return isinstance(value, list) and len(value) == 2 and isinstance(value[0], (str, int))
def _literal(value):
return isinstance(value, (str, int, float, bool)) and not isinstance(value, list)
def _file_name(value):
return os.path.splitext(os.path.basename(str(value).replace("\\", "/")))[0]
def _output_type(class_type, index):
"""ComfyUI's type of output `index` of `class_type` ("MODEL", "LATENT"...), or ""."""
try:
import nodes
types = getattr(nodes.NODE_CLASS_MAPPINGS.get(class_type), "RETURN_TYPES", None) or ()
return str(types[int(index)]) if int(index) < len(types) else ""
except Exception: # noqa: BLE001 - no ComfyUI (tests) or an odd node: the link stays unnamed
return ""
def _role(class_type, inputs):
name = class_type.lower()
if "lora_name" in inputs:
return "lora"
if _linked(inputs.get("latent_image")) or "sampler" in name and not name.endswith("select"):
return "sampler"
if name == "loadimage" or name.startswith("loadimage"):
return "image"
if "upscale" in name:
return "upscale"
if "loader" in name:
return "model"
if isinstance(inputs.get("text"), str) or "textencode" in name:
return "prompt"
if name.startswith("empty") and "latent" in name:
return "size"
if name.startswith("vaedecode"):
return "decode"
if name.startswith(("save", "preview")):
return "save"
return "other"
def _facts(role, inputs):
literal = {key: value for key, value in inputs.items() if _literal(value)}
facts = {}
model = next((value for value in literal.values() if isinstance(value, str) and value.lower().endswith(MODEL_EXTENSIONS)), None)
if model:
facts["file"] = _file_name(model)[:VALUE_CHARS]
if role == "lora":
for key in ("strength_model", "strength"):
if isinstance(literal.get(key), (int, float)):
facts["strength"] = round(float(literal[key]), 3)
break
elif role == "prompt":
text = next((value for key, value in literal.items() if isinstance(value, str) and key.startswith("text")), "")
facts["text"] = " ".join(text.split())[:TEXT_CHARS]
elif role == "sampler":
facts.update({key: literal[key] for key in SAMPLER_FACTS if key in literal})
elif role == "image":
facts["file"] = str(literal.get("image") or "")[:VALUE_CHARS]
elif role == "save":
facts["prefix"] = str(literal.get("filename_prefix") or "")[:VALUE_CHARS]
if all(isinstance(literal.get(key), (int, float)) and literal[key] > 0 for key in ("width", "height")):
facts.update(width=int(literal["width"]), height=int(literal["height"]))
if isinstance(literal.get("batch_size"), int) and literal["batch_size"] > 1:
facts["batch"] = literal["batch_size"]
if role == "other" and not facts:
facts["values"] = [[key, str(value)[:VALUE_CHARS]] for key, value in literal.items()
if key not in SEEDS and str(value).strip()][:2]
return facts
def map_of(prompt):
"""{"boxes": [{id, type, title, role, facts}], "links": [{from, to, type}], "params": {...}} of API prompt `prompt`."""
nodes = {str(key): node for key, node in (prompt or {}).items() if isinstance(node, dict) and node.get("class_type")}
shown = [key for key in nodes if nodes[key]["class_type"] not in HIDDEN]
if len(shown) > MAX_BOXES: # keep what makes the picture; drop the helpers first
rank = {"other": 2, "save": 1}
shown = sorted(shown, key=lambda key: rank.get(_role(nodes[key]["class_type"], nodes[key].get("inputs") or {}), 0))[:MAX_BOXES]
keep = set(shown)
def source(link, seen=()):
"""The shown node (and the output's type) a link comes from, wired through hidden nodes."""
key, index = str(link[0]), link[1]
if key in keep:
return key, _output_type(nodes[key]["class_type"], index)
node = nodes.get(key)
if not node or key in seen:
return None, ""
upstream = next((value for value in (node.get("inputs") or {}).values() if _linked(value)), None)
return source(upstream, (*seen, key)) if upstream else (None, "")
boxes, links, params = [], [], {}
for key in sorted(shown, key=lambda item: (len(item), item)):
node = nodes[key]
inputs = node.get("inputs") or {}
role = _role(node["class_type"], inputs)
boxes.append({"id": key, "type": node["class_type"][:80], "role": role, "facts": _facts(role, inputs),
"title": str((node.get("_meta") or {}).get("title") or node["class_type"])[:80]})
for name, value in inputs.items():
if _linked(value):
found, kind = source(value)
if found and found != key and len(links) < MAX_EDGES:
links.append({"from": found, "to": key, "type": kind or name.upper(), "input": name})
elif _literal(value) and name not in SEEDS:
params[f"{key}.{name}"] = str(value)[:300]
_polarity(boxes, links)
return {"boxes": boxes, "links": links, "params": params}
def _polarity(boxes, links):
"""Marks prompt boxes positive or negative by the sampler input they reach."""
by_id = {box["id"]: box for box in boxes}
feeds = {}
for link in links:
feeds.setdefault(link["from"], []).append(link)
for box in boxes:
if box["role"] != "prompt":
continue
todo, seen = [box["id"]], set()
while todo and "polarity" not in box["facts"]:
here = todo.pop()
seen.add(here)
for link in feeds.get(here, []):
if link["input"] in ("positive", "negative") and by_id[link["to"]]["role"] == "sampler":
box["facts"]["polarity"] = link["input"]
break
if link["to"] not in seen:
todo.append(link["to"])
def changes(before, after):
"""What differs in map `after` from map `before`: [{node, title, key, from, to}] or a node added / gone."""
if not before or not after:
return []
titles = {box["id"]: box["title"] for box in before.get("boxes", [])}
titles.update({box["id"]: box["title"] for box in after.get("boxes", [])})
old_ids = {box["id"]: box["type"] for box in before.get("boxes", [])}
new_ids = {box["id"]: box["type"] for box in after.get("boxes", [])}
out = []
for key, kind in new_ids.items():
if old_ids.get(key) != kind:
out.append({"node": key, "title": titles[key], "change": "added"})
for key, kind in old_ids.items():
if new_ids.get(key) != kind:
out.append({"node": key, "title": titles[key], "change": "removed"})
gone = {item["node"] for item in out}
old, new = before.get("params", {}), after.get("params", {})
for name in sorted(set(old) | set(new), key=lambda item: (len(item.split(".")[0]), item)):
node, key = name.split(".", 1)
if node not in gone and old.get(name) != new.get(name):
out.append({"node": node, "title": titles.get(node, node), "key": key,
"from": (old.get(name) or "")[:VALUE_CHARS], "to": (new.get(name) or "")[:VALUE_CHARS]})
return out[:MAX_CHANGES]
+72
View File
@@ -0,0 +1,72 @@
"""Attach explicitly identified external ComfyUI submissions to an AI task; never guess ownership."""
import json
import math
import time
from . import ai_task_runs, ai_tasks
from .mcp_tools import ToolError
def checked_wait(value):
"""wait_seconds as seconds from 0 to 55: one call waits no longer, so a longer wait is cut to that."""
if isinstance(value, bool) or not isinstance(value, (int, float)) or not math.isfinite(value):
raise ToolError("wait_seconds is a number of seconds, up to 55.")
return min(max(value, 0), 55)
def track_run(ctx, prompt_ids, task=None, wait_seconds=15):
wait_seconds = checked_wait(wait_seconds)
if (not isinstance(prompt_ids, list) or not 1 <= len(prompt_ids) <= 8
or any(not isinstance(pid, str) or not pid or len(pid) > 128 for pid in prompt_ids)
or len(set(prompt_ids)) != len(prompt_ids)):
raise ToolError("Give 1–8 unique prompt_ids returned by your own ComfyUI /prompt submissions.")
client, session = ctx.get("client") or "AI", ctx.get("session") or ""
task_id = task or ctx.get("task") or ai_tasks.active_task_id(client, session)
with ai_tasks._lock:
target = ai_tasks._load().get(task_id)
if not target or target.get("ended"):
raise ToolError("Begin or resume an open task before attaching a run; pass its task ID.")
existing = None
for candidate in ai_tasks._load().values():
for step in candidate.get("steps", []):
ids = step.get("prompt_ids") or []
if set(ids).intersection(prompt_ids):
if candidate["id"] != task_id or ids != prompt_ids:
raise ToolError("A prompt ID is already attached to another run. Do not reassign it.")
existing = step
if existing is None:
try:
queue = ai_task_runs._queue()
running, pending = queue.get_current_queue()
known = {item[1] for item in [*running, *pending]}
missing = [pid for pid in prompt_ids if pid not in known and not queue.get_history(prompt_id=pid).get(pid)]
except Exception as error:
raise ToolError("Cannot read ComfyUI's queue/history; nothing was attached.") from error
if missing:
raise ToolError("Unknown prompt IDs: " + ", ".join(missing) + ". Use this server's submission receipts.")
receipt = {"prompt_ids": prompt_ids, "queued": len(prompt_ids)}
result = {"structuredContent": receipt, "content": []}
step = ai_tasks.record(client, "run_workflow", {}, result, time.time(), session=session, task_id=task_id)
if not step:
raise ToolError("Could not record the run; nothing was queued by this tool.")
step.update(prompt_ids=list(prompt_ids), source="external")
ai_tasks._save(target)
ai_task_runs.watch(step["id"], receipt)
existing = step
answer = ai_task_runs.status(client, session, task_id, wait_seconds, step_id=existing["id"])
return {"task": task_id, "attached": True, **answer}
def wait_for_receipt(result, client, session, task_id, wait_seconds=15, step_id=None):
"""Return authoritative execution state alongside submission receipts, with a bounded wait."""
answer = ai_task_runs.status(client, session, task_id, wait_seconds, step_id=step_id)
data = {**result.get("structuredContent", {}), **answer, "task": task_id}
result["structuredContent"] = data
content = list(result.get("content") or [])
text = {"type": "text", "text": json.dumps(data, ensure_ascii=False)}
if content and content[0].get("type") == "text":
content[0] = text
else:
content.insert(0, text)
result["content"] = content
return result
+221
View File
@@ -0,0 +1,221 @@
"""What a run an AI started shows beyond the node and its steps (ai_task_runs.py adds it to the
run's progress, for the AI and the workbench): ComfyUI's console since the run began (the
progress bar of the node running with its own note, "Model Initializing", the models it loads:
what the user sees in the console window), the time per step and the time left, the graphics
card's memory, and once per run estimate(): the size each sampler works at, read from the queued
prompt (never changed), with warnings for a run that will be very slow or will not fit, such as a
picture upscaled by a model and then sampled again at full size."""
import math
import re
import time
from datetime import datetime
ANSI = re.compile(r"\x1b\[[0-9;?]*[A-Za-z]")
# tqdm's bar: " 12%|## | 6/50 [00:03<00:22, 1.95it/s, a note]"
BAR = re.compile(r"(\d+)/(\d+)\s*\[([^<\]]*)<([^,\]]*),\s*([^,\]]*)(?:,\s*([^\]]*))?\]")
CONSOLE_LINES = 5
LINE_CHARS = 200
# Megapixels a sampler can work at comfortably on a 12 GB card; scaled by the card's memory.
COMFORT_MP_12GB = 4.2
LATENT_INPUTS = ("latent_image", "samples", "latent", "pixels", "image", "images")
NOT_PICTURES = {"model", "clip", "vae", "upscale_model", "positive", "negative", "conditioning", "control_net",
"mask", "sampler", "sigmas", "guider", "noise", "clip_vision", "style_model"}
def _logs():
try:
from app import logger
except ImportError:
return []
return list(logger.get_logs() or [])
def console_since(started, entries=None, limit=CONSOLE_LINES):
"""The console's last `limit` lines written since `started` (a time.time()), as shown: colours
gone, a progress bar redrawn in place kept at its latest."""
written = []
for entry in _logs() if entries is None else entries:
try:
at = datetime.fromisoformat(entry["t"]).timestamp()
except (KeyError, TypeError, ValueError):
continue
if at >= started - 1:
written.append(str(entry.get("m") or ""))
lines = []
for raw in "".join(written).split("\n"):
parts = [part for part in raw.split("\r") if part.strip()]
line = ANSI.sub("", parts[-1]).strip() if parts else ""
if line:
lines.append(line[:LINE_CHARS])
return lines[-limit:]
def _seconds(text):
"""tqdm's "01:23" / "1:02:03" in seconds, or None ("?")."""
try:
parts = [int(part) for part in text.strip().split(":")]
except ValueError:
return None
return sum(part * 60 ** index for index, part in enumerate(reversed(parts))) if 1 < len(parts) <= 3 else None
def bar_of(lines):
"""The progress bar on the console's last line: steps, seconds left and per step, its note; or None."""
match = BAR.search(lines[-1]) if lines else None
if not match:
return None
out = {"value": int(match[1]), "max": int(match[2])}
left = _seconds(match[4])
if left is not None and out["value"]:
out["left"] = left
rate = match[5].strip()
number = re.match(r"([\d.]+)\s*(s/it|it/s)", rate)
if number and float(number[1]) > 0:
out["rate"] = round(float(number[1]) if number[2] == "s/it" else 1 / float(number[1]), 2)
if match[6] and match[6].strip():
out["note"] = match[6].strip()[:80]
return out
class Pace:
"""Seconds per step and seconds left of the node running, from the steps seen over time
(for nodes whose bar is not on the console)."""
def __init__(self):
self.key, self.first = None, None
def update(self, key, value, most, now=None):
now = time.monotonic() if now is None else now
if key != self.key or self.first is None or value < self.first[1]:
self.key, self.first = key, (now, value)
return None
done = value - self.first[1]
if done <= 0:
return None
rate = (now - self.first[0]) / done
return {"rate": round(rate, 2), "left": max(0, round(rate * (most - value)))}
def vram():
"""The graphics card's free and total memory in MB, or None (no card, or no ComfyUI)."""
try:
import comfy.model_management as management
device = management.get_torch_device()
if getattr(device, "type", "") in ("cpu", "mps"):
return None
total, free = management.get_total_memory(device), management.get_free_memory(device)
except Exception: # noqa: BLE001 - memory is a hint; the run goes on without it
return None
return {"free_mb": int(free // 2 ** 26) * 64, "total_mb": int(total // 2 ** 20)}
def _image_size(name):
"""(width, height) of picture `name` in ComfyUI's input folder, or None."""
try:
import folder_paths
from PIL import Image
with Image.open(folder_paths.get_annotated_filepath(str(name))) as picture:
return picture.size
except Exception: # noqa: BLE001 - an unreadable picture only leaves the size unknown
return None
def _number(value):
return value if isinstance(value, (int, float)) and not isinstance(value, bool) and math.isfinite(value) else None
def _linked(value):
return isinstance(value, list) and len(value) == 2 and isinstance(value[0], (str, int))
def _model_factor(prompt, link):
"""How many times an upscale model enlarges, from its file name ("4x-UltraSharp", "x2plus"); 4 if unsaid."""
name = str(((prompt.get(str(link[0])) or {}).get("inputs") or {}).get("model_name") or "") if _linked(link) else ""
match = re.search(r"(?<![0-9])([1-8])x|x([1-8])(?![0-9])", name, re.I)
return int(match[1] or match[2]) if match else 4
def estimate(prompt, vram_total_mb=None, image_size=_image_size):
"""The size each sampler of API prompt `prompt` works at, and warnings: {"samplers", "warnings"}."""
sizes = {}
def size(node_id, seen):
node_id = str(node_id)
if node_id in sizes:
return sizes[node_id]
node = prompt.get(node_id)
if not isinstance(node, dict) or node_id in seen:
return None
seen.add(node_id)
kind, inputs = str(node.get("class_type") or ""), node.get("inputs") or {}
names = sorted((name for name, value in inputs.items() if _linked(value) and name not in NOT_PICTURES),
key=lambda name: LATENT_INPUTS.index(name) if name in LATENT_INPUTS else len(LATENT_INPUTS))
upstream = next((found for found in (size(inputs[name][0], seen) for name in names) if found), None)
width, height = _number(inputs.get("width")), _number(inputs.get("height"))
scale = _number(inputs.get("scale_by"))
out = upstream
if kind == "LoadImage":
dims = image_size(inputs.get("image"))
out = {"width": dims[0], "height": dims[1], "batch": 1, "upscaled": False} if dims else None
elif width and height and width > 0 and height > 0:
batch = _number(inputs.get("batch_size")) or (upstream or {}).get("batch", 1)
out = {"width": width, "height": height, "batch": int(batch), "upscaled": (upstream or {}).get("upscaled", False)}
elif upstream and (width or height): # one side given: the other keeps the picture's shape
ratio = (width or 0) / upstream["width"] if width else height / upstream["height"]
out = {**upstream, "width": upstream["width"] * ratio, "height": upstream["height"] * ratio}
elif upstream and scale:
out = {**upstream, "width": upstream["width"] * scale, "height": upstream["height"] * scale}
elif upstream and kind == "ImageScaleToTotalPixels" and _number(inputs.get("megapixels")):
ratio = math.sqrt(inputs["megapixels"] * 1024 * 1024 / (upstream["width"] * upstream["height"]))
out = {**upstream, "width": upstream["width"] * ratio, "height": upstream["height"] * ratio}
elif upstream and kind == "ImageUpscaleWithModel":
factor = _model_factor(prompt, inputs.get("upscale_model"))
out = {**upstream, "width": upstream["width"] * factor, "height": upstream["height"] * factor, "upscaled": True}
sizes[node_id] = out
return out
def steps_of(inputs):
if _number(inputs.get("steps")):
return int(inputs["steps"])
source = prompt.get(str(inputs["sigmas"][0])) if _linked(inputs.get("sigmas")) else None
return int(source["inputs"]["steps"]) if source and _number((source.get("inputs") or {}).get("steps")) else None
comfort = COMFORT_MP_12GB * max(0.5, min(3.0, (vram_total_mb or 12288) / 12288))
samplers, warnings = [], []
for node_id, node in sorted(prompt.items(), key=lambda item: str(item[0])):
if not isinstance(node, dict):
continue
inputs = node.get("inputs") or {}
title = str((node.get("_meta") or {}).get("title") or node.get("class_type") or node_id)[:80]
if _linked(inputs.get("latent_image")):
found = size(inputs["latent_image"][0], set())
if not found:
continue
entry = {"node": str(node_id), "title": title, "width": round(found["width"]), "height": round(found["height"]),
"batch": found["batch"], "steps": steps_of(inputs), "upscaled": found["upscaled"]}
samplers.append(entry)
megapixels = entry["width"] * entry["height"] * entry["batch"] / 1048576
if megapixels > comfort:
warnings.append({"kind": "upscale_resample" if found["upscaled"] else "big_sample",
**entry, "megapixels": round(megapixels, 1)})
elif node.get("class_type") == "VAEDecode" and _linked(inputs.get("samples")):
found = size(inputs["samples"][0], set())
if found and found["width"] * found["height"] * found["batch"] / 1048576 > comfort * 2:
warnings.append({"kind": "big_decode", "node": str(node_id), "title": title,
"width": round(found["width"]), "height": round(found["height"])})
return {"samplers": samplers[:12], "warnings": warnings[:6], "comfort_mp": round(comfort, 1)}
def warning_words(warning):
"""One sentence for the AI on a warning of estimate()."""
where = f"{warning['title']} (node {warning['node']}) works at {warning['width']}x{warning['height']}"
if warning["kind"] == "upscale_resample":
return (f"{where}{' for ' + str(warning['steps']) + ' steps' if warning.get('steps') else ''} after an upscale "
"model enlarged the picture: very slow and may not fit in memory. If the user only wants it bigger, the "
"upscale model alone (ImageUpscaleWithModel -> SaveImage, no sampler after it) is enough; else sample "
"smaller (scale down before VAEEncode), use fewer steps or a lower denoise, or a tiled sampler.")
if warning["kind"] == "big_sample":
return (f"{where}: {warning['megapixels']} megapixels, more than this card is comfortable with; expect it to be "
"very slow or run out of memory. Consider a smaller size and upscaling afterwards.")
return f"{where} when decoded: VAEDecodeTiled saves memory there."
+129
View File
@@ -0,0 +1,129 @@
"""Each task's own canvas tab, kept by Anomalous rather than left to the AI: with "new" (the
default, set in the Gates window; each task can change it) the page copies the workflow open
now into a new tab named after the task before the AI's first change, and brings that tab back
before each later change, read or run (web/modules/mcp_task_tab.js); new_workflow and
open_workflow make the tab they open the task's. The user's own tab is never changed. With
"current" the AI works on whatever tab is open, as before.
mcp_server.py marks each tool call with its task (begin_call); mcp_bridge.ask_page adds the
task's canvas to the page actions that touch the canvas (spec) and keeps the tab the page
reports (remember); tell() tells the AI once when a tab was made for it.
"""
import contextvars
import json
import os
import time
import uuid
from . import ai_tasks
from .path_utils import atomic_write_json
MODES = ("new", "current")
DEFAULT = "new"
# Page actions (web/modules/mcp_bridge.js) that work on the canvas, so on the task's tab.
TAB_ACTIONS = {"describe_canvas", "set_prompt", "set_model", "add_lora", "place_combo", "check_models", "fix_models",
"run_workflow", "download_missing", "add_node", "add_nodes", "connect_nodes", "set_values", "remove_nodes", "new_workflow", "open_workflow",
"fill_slots"}
_call = contextvars.ContextVar("anomalous_task_canvas", default=None)
def _path():
return os.path.join(os.path.dirname(ai_tasks.tasks_dir()), "ai_canvas.json")
def default_mode():
try:
with open(_path(), encoding="utf-8") as source:
mode = json.load(source).get("mode")
except (OSError, ValueError, AttributeError):
mode = None
return mode if mode in MODES else DEFAULT
def set_default(mode):
if mode not in MODES:
raise ValueError("canvas_tab is new or current")
atomic_write_json(_path(), {"mode": mode})
return mode
def mode_of(task):
mode = ((task or {}).get("canvas") or {}).get("mode")
return mode if mode in MODES else default_mode()
def set_mode(task_id, mode):
"""The user's choice for one task. Returns the task's canvas record."""
if mode not in MODES:
raise ValueError("canvas is new or current")
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
if not task:
raise ValueError("No such task")
if (task.get("canvas") or {}).get("mode", default_mode()) != mode:
# The user's choice: the AI hears of it, and it is one of the task's decisions (ai_decisions.py).
now = time.time()
ai_tasks._add_step(task, {"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": "user_canvas", "kind": "user", "ok": True,
"state": "done", "summary": "user_canvas", "canvas": mode, "delivered": False, "results": [],
"text": "The user chose that you work " + ("in a tab of the task's own." if mode == "new" else "on the canvas they have open.")})
task["canvas"] = {**(task.get("canvas") or {}), "mode": mode}
ai_tasks._save(task)
canvas = dict(task["canvas"])
ai_tasks._notify(task_id)
return canvas
def begin_call(task_id):
"""Marks the tool call running now as task `task_id`'s; the holder says later what the page opened."""
holder = {"task": task_id, "opened": None}
_call.set(holder)
return holder
def spec(action):
"""What the page needs for page action `action` of the call running now: {mode, tab, title}, or None."""
holder = _call.get()
if not holder or not holder.get("task") or action not in TAB_ACTIONS:
return None
task = ai_tasks.get_task(holder["task"])
if not task:
return None
return {"mode": mode_of(task), "tab": (task.get("canvas") or {}).get("tab"), "title": (task.get("title") or "")[:60]}
def remember(tab):
"""Keeps the tab the page made or opened for the call's task ({path, name, made?})."""
holder = _call.get()
if not holder or not holder.get("task") or not isinstance(tab, dict) or not isinstance(tab.get("path"), str):
return
kept = {"path": tab["path"][:300], "name": str(tab.get("name") or "")[:120], "at": time.time()}
with ai_tasks._lock:
task = ai_tasks._load().get(holder["task"])
if not task:
return
task["canvas"] = {**(task.get("canvas") or {}), "tab": kept}
ai_tasks._save(task)
ai_tasks._notify(holder["task"])
holder["opened"] = {**kept, "made": bool(tab.get("made"))}
def tell(result, holder):
"""Says in the tool's answer that Anomalous made the task a tab of its own (once, when it did)."""
opened = (holder or {}).get("opened")
if not opened or not opened["made"] or not isinstance(result, dict) or result.get("isError"):
return result
data = dict(result.get("structuredContent") or {})
data["canvas_tab"] = {"name": opened["name"], "note": (
f"Anomalous copied the user's open workflow to a new tab \"{opened['name']}\" for this task (the user's "
"setting): your canvas changes, reads and runs go there; the user's own tab stays as it was. Do not switch "
"tabs yourself.")}
result["structuredContent"] = data
content = list(result.get("content") or [])
text = {"type": "text", "text": json.dumps(data, ensure_ascii=False)}
if content and isinstance(content[0], dict) and content[0].get("type") == "text":
content[0] = text
else:
content.insert(0, text)
result["content"] = content
return result
+62
View File
@@ -0,0 +1,62 @@
"""The user's own changes in Anomalous while an AI works on a task: each activity log entry the
user makes (activity_log.py tells note_activity of every new entry) becomes a step of the task
the AI called within USER_STEP_SECONDS, and the entry names that task back. The AI is told of it
with its next call (ai_panels.attach_pending, a message of kind "change"), so it keeps the user's
change instead of working over it. The AI's own canvas changes (via mcp) are its steps already;
which actions count, and their stage, is action_catalog.USER_ACTIONS."""
import time
import uuid
from . import action_catalog, activity_log, ai_tasks
USER_STEP_SECONDS = 10 * 60
def _live_task(now):
"""The task an AI is working on now (it called within USER_STEP_SECONDS), or None."""
live = [task for task in ai_tasks._load().values() if not task.get("ended") and not task.get("waiting")
and now - task.get("updated", 0) < USER_STEP_SECONDS]
return max(live, key=lambda task: task.get("updated", 0)) if live else None
def _change_text(entry):
"""What the user changed, in words for the AI."""
if entry.get("action") == "image_delete" and entry.get("target"):
return (f"Deleted the picture {entry['target']} (to the Recycle Bin): it is gone from the output folder, "
"so do not use, show or hand it in again; another picture is needed for what it was for.")
if entry.get("source") != "canvas":
words = str(entry.get("action", "")).replace("_", " ")
return f"{words}: {entry['target']}" if entry.get("target") else words
lines = []
for change in (entry.get("detail") or {}).get("changes") or []:
if change.get("kind") == "changed":
lines.append(f"{change.get('node')} {change.get('widget')}: {change.get('before')} -> {change.get('after')}")
elif change.get("kind") == "opened":
lines.append(f"opened the workflow {change.get('node')}")
else:
lines.append(f"{change.get('kind')} {change.get('node')}")
return "Changed on the canvas: " + "; ".join(lines[:8])
def note_activity(entry):
"""A new activity log entry: a step of the AI's task when the user made it while the AI worked."""
stage = action_catalog.USER_ACTIONS.get(entry.get("action"))
if stage is None or (entry.get("detail") or {}).get("via") == "mcp":
return None
at = entry.get("time") or time.time()
with ai_tasks._lock:
task = _live_task(at)
if not task:
return None
step = {"id": uuid.uuid4().hex[:10], "at": at, "ms": 0, "tool": "user_activity", "kind": "user", "stage": stage,
"ok": True, "state": "done", "summary": entry.get("action", ""),
"text": ai_tasks._short(_change_text(entry), ai_tasks.MAX_NOTE), "activity": entry["id"],
"delivered": False, "results": []}
# Appended without moving "updated": the user's own changes do not keep the AI's task open.
task["steps"].append(step)
del task["steps"][:-ai_tasks.MAX_STEPS]
ai_tasks._save(task)
activity_log.tag(entry["id"], task["id"])
ai_tasks._notify(task["id"])
return step
+91
View File
@@ -0,0 +1,91 @@
"""Select a task to resume from unread user input, without changing task state or ownership;
whether a look belongs to the task just done; how many of a task's failures still stand.
Called under ai_tasks' lock when a caller has no recent open task. Explicit task selection
and the user's task waiting for the next AI take precedence in ai_tasks.
"""
import json
REPLIES = {"user_message", "user_marks", "user_pictures", "user_toolbox"}
REPLY_FRESH = 2 * 3600
AFTER_DONE = 5 * 60 # a look this soon after the AI said its task is done is part of that task, not a new one
def _reply_at(task, now):
"""Latest unread user input; a tool still running keeps its task from automatic takeover."""
if any(step.get("state") == "running" for step in task["steps"]):
return 0
replies = []
for step in task["steps"]:
if step.get("kind") == "user" and step.get("tool") in REPLIES:
replies.append(step)
elif step.get("kind") == "panel":
replies.extend((step.get("answers") or {}).values())
return max((r.get("at", 0) for r in replies if not r.get("delivered") and not r.get("recommended_on_timeout")
and 0 <= now - r.get("at", 0) < REPLY_FRESH), default=0)
def latest_replied(tasks, client, now):
"""The task of this app with the latest eligible reply, or None."""
candidates = ((at, task) for task in tasks.values() if task.get("client") == client and (at := _reply_at(task, now)))
_, task = max(candidates, key=lambda item: item[0], default=(0, None))
return task
def just_done(task, tasks, waiting, client, now, explicit=False):
"""`task` (the app's last one) when it said it is done (report done) moments ago: a look the AI
takes right after (checking its result) goes there, the task staying ended, and what it shows or
asks the user about it (a before and after, "happy with it?") opens it again; no new task is made
for either. Automatic selection yields to a waiting task or unread reply; an explicit
task id keeps the same precedence as selecting an open task."""
if not task or not task.get("ended") or not 0 <= now - task["ended"] < AFTER_DONE:
return None
if not any(step.get("tool") == "report" and step.get("at") == task["ended"] for step in task["steps"]):
return None # it ended by a pause or another task, not by the AI saying it is done
if not explicit and ((waiting and not waiting.get("ended")) or latest_replied(tasks, client, now)):
return None
return task
RECOVERY_TARGETS = {
"set_model": ("node_id",), "set_prompt": ("node_id", "role"), "set_values": ("node_id",),
"connect_nodes": ("from_node_id", "to_node_id", "input"), "remove_nodes": ("node_ids",),
"run_workflow": ("workflow",), "open_workflow": ("workflow",),
"get_model": ("model_id",), "move_model": ("model_id",), "download_model": ("url",),
"get_image_info": ("image",), "zoom_image": ("image",),
}
def recovery_target(tool, arguments):
"""Keep target identity, not prompt text or changed values, to distinguish independent retries."""
target = {key: arguments[key] for key in RECOVERY_TARGETS.get(tool, ()) if arguments.get(key) is not None}
if tool == "set_prompt":
target["role"] = arguments.get("role") or "positive"
for key, value in target.items():
if key.endswith("node_id"):
target[key] = str(value)
elif key == "node_ids" and isinstance(value, list):
target[key] = sorted(map(str, value))
if tool == "set_values" and isinstance(arguments.get("values"), dict):
values = arguments["values"]
if arguments.get("node_id") is None and values and all(isinstance(value, dict) for value in values.values()):
target["settings"] = {str(node): sorted(settings) for node, settings in values.items()}
else:
target["settings"] = sorted(values)
return json.dumps(target, sort_keys=True, ensure_ascii=False) if target else None
def unresolved(steps):
"""Only completed successful calls to the same recorded target resolve an earlier failure. A
failure that named no target (its arguments missing or misnamed, as a light model's first try
often is) is resolved by any later completed success of the same tool."""
later_ok, later_tools, count = set(), set(), 0
for step in reversed(steps):
key = (step.get("tool"), step.get("recovery_target"))
if step.get("ok") and step.get("state") in (None, "done"):
later_ok.add(key)
later_tools.add(key[0])
elif not step.get("ok") and key not in later_ok and not (key[1] is None and key[0] in later_tools):
count += 1
return count
+55
View File
@@ -0,0 +1,55 @@
"""Which task a task goes on from ("接着"), so work split over tasks is found together: a task an
AI begins as new while on another (ai_tasks.begin_task), or one the user starts while looking at
another (ai_tasks.start_task), keeps that one as its "from". The user can link a task to another
or take the link off (set_from); links() gives a task's own "from" and the tasks that go on from it,
and an AI beginning a task that goes on from another hears what was decided there."""
from . import ai_decisions, ai_tasks
class LinkError(ValueError):
pass
def set_from(task_id, parent_id):
"""Task `task_id` goes on from `parent_id` (None: from none). Refuses a loop. Returns the parent id."""
with ai_tasks._lock:
tasks = ai_tasks._load()
task = tasks.get(task_id)
if not task:
raise LinkError("No such task")
if parent_id is not None:
if parent_id not in tasks or parent_id == task_id:
raise LinkError("No such other task")
seen, at = set(), parent_id
while at and at not in seen: # going up from the parent must not come back here
if at == task_id:
raise LinkError("That task already goes on from this one")
seen.add(at)
at = (tasks.get(at) or {}).get("from")
task["from"] = parent_id
else:
task.pop("from", None)
ai_tasks._save(task)
ai_tasks._notify(task_id)
return parent_id
def links(task_id):
"""{from: {id, title} | None, next: [{id, title}]} of task `task_id`, the next oldest first."""
with ai_tasks._lock:
tasks = ai_tasks._load()
task = tasks.get(task_id) or {}
brief = lambda item: {"id": item["id"], "title": item.get("title") or ""} # noqa: E731
parent = tasks.get(task.get("from"))
later = sorted((item for item in tasks.values() if item.get("from") == task_id), key=lambda item: item.get("started", 0))
return {"from": brief(parent) if parent else None, "next": [brief(item) for item in later]}
def carried(task):
"""For an AI beginning `task`: the task it goes on from and what was decided there, or None."""
parent = ai_tasks.get_task(task.get("from")) if task.get("from") else None
if not parent:
return None
return {"task": parent["id"], "title": parent.get("title") or "", "decisions": ai_decisions.lines(parent),
"note": "This task goes on from that one; keep to what was decided there unless the user says otherwise."}
+442
View File
@@ -0,0 +1,442 @@
"""What a workflow run an AI started made: after run_workflow (mcp_server.py), the prompts
ComfyUI queued for it are watched in its history until they end, and the pictures and videos
they saved become the step's results (ai_tasks.py keeps the step). A thread per run, polling
every RUN_POLL_SECONDS, for at most ai_tasks.RUN_WATCH_SECONDS; nothing runs when no AI ran
anything. While it runs, where it stands (queued and how many are ahead, the node running and
its steps, nodes done; read from ComfyUI's queue and its progress registry, never changed) is
the step's "progress", sent to the page with each change, and status() answers the AI's
run_status with it, waiting for the end when asked. tell() puts the same news in the AI's
other answers: where the run stands while it goes, and once that it ended. The console's lines,
the time left and the card's memory join the progress, and the size each sampler works at with
its warnings is the step's "estimate" (ai_run_watch.py), the run's shape in boxes its "map" with
what changed since the task's run before it (ai_run_map.py); a run the user or the AI stopped
(ai_run_control.py) ends "cancelled"."""
import threading
import time
from . import ai_gates, ai_run_map, ai_run_watch, ai_tasks
from .mcp_tools import ToolError, _image_content, _resolve_image, retext, tool_text
RUN_POLL_SECONDS = 2.0
MAX_STATUS_WAIT = 55
MAX_PREVIEWS = 8 # the last PreviewImage pictures of a run the AI is told of
MAX_SHOWN = 4 # thumbnails of a finished run's pictures that come with the answer first telling of its end
def _queue():
from server import PromptServer
return PromptServer.instance.prompt_queue
def outputs_of(record_, folder="output"):
"""Saved pictures and videos in one history record; with `folder` "temp", what PreviewImage
nodes showed (a pose skeleton, a mask): the AI's to look at, not the user's results."""
out = []
for output in (record_.get("outputs") or {}).values():
for key in ("images", "gifs", "videos"):
for item in output.get(key) or []:
if isinstance(item, dict) and item.get("type") == folder and item.get("filename"):
out.append({"type": "image", "filename": item["filename"], "subfolder": item.get("subfolder") or "",
"folder": folder})
return out
def _registry():
"""ComfyUI's per-node progress of the prompt running now, or None (an older ComfyUI has none)."""
try:
from comfy_execution import progress
except ImportError:
return None
return progress.global_progress_registry
def progress_of(prompts):
"""Where the run of `prompts` stands: queued (how many are ahead), running (the node and its
steps, nodes done of all), or None when ComfyUI cannot say."""
try:
running, pending = _queue().get_current_queue()
except Exception: # noqa: BLE001 - no server (tests)
return None
running_ids = [item[1] for item in running]
order = [item[1] for item in sorted(pending, key=lambda item: item[0])]
current = next((pid for pid in prompts if pid in running_ids), None)
if not current:
waiting = [pid for pid in prompts if pid in order]
if not waiting:
return None
return {"state": "queued", "ahead": order.index(waiting[0]) + len(running_ids)}
out = {"state": "running", "prompt": prompts.index(current) + 1, "prompts": len(prompts)}
registry = _registry()
if registry is None or registry.prompt_id != current:
return out
nodes = dict(registry.nodes)
out["nodes_done"] = sum(1 for node in nodes.values() if node["state"].value == "finished")
out["nodes"] = len(registry.dynprompt.original_prompt)
active = next(((node_id, node) for node_id, node in nodes.items() if node["state"].value == "running"), None)
if active:
try:
info = registry.dynprompt.get_node(active[0]) or {}
except Exception: # noqa: BLE001 - a node of the run that is gone already
info = {}
out["node"] = str((info.get("_meta") or {}).get("title") or info.get("class_type") or active[0])[:80]
out["value"], out["max"] = int(active[1]["value"]), int(active[1]["max"])
# Like ComfyUI's own tab title: the running node's steps; without steps, the nodes done.
if out.get("max", 0) > 1:
out["percent"] = max(0, min(100, round(100 * out["value"] / out["max"])))
elif out["nodes"]:
out["percent"] = max(0, min(100, round(100 * out["nodes_done"] / out["nodes"])))
return out
def _show_progress(task_id, step_id, progress):
"""The run's progress on its step (kept in memory; the page hears of each change)."""
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
step = next((item for item in (task or {}).get("steps", []) if item["id"] == step_id), None)
if step and step.get("stop"):
progress = {**progress, "stopping": step["stop"]}
if not step or step.get("progress") == progress or step.get("state") != "running":
return
step["progress"] = progress
ai_tasks._notify(task_id, {"progress": {"step": step_id, **progress}})
def _finish_run(task_id, step_id, prompts, history, queued=None):
if not prompts:
_stop_run(task_id, step_id, "没有找到已排队的运行 / No queued run was found; completion cannot be confirmed")
return True
images, previews, failed, done = [], [], False, 0
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
stopped = next((item.get("stop") for item in (task or {}).get("steps", []) if item["id"] == step_id), None)
for prompt_id in prompts:
record_ = history.get(prompt_id)
if not record_:
if stopped and queued is not None and prompt_id not in queued:
done += 1 # taken out of the queue before it started: it never gets a history
continue
if stopped and (record_.get("status") or {}).get("status_str") == "error":
done += 1 # the interrupt ends a prompt "error"; it was stopped, not broken
continue
status = record_.get("status") or {}
if status.get("status_str") == "error":
failed = True
done += 1
elif status.get("completed"):
done += 1
for image in outputs_of(record_):
if image not in images: # a prompt ComfyUI answers from its cache repeats the same file
images.append(image)
previews += [image for image in outputs_of(record_, "temp") if image not in previews]
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
step = next((item for item in (task or {}).get("steps", []) if item["id"] == step_id), None)
if not step:
return True
failed = failed or bool(step.get("submission_error"))
stopped = step.get("stop")
results = [item for item in step["results"] if item["type"] != "image"] + images[:ai_tasks.MAX_RESULTS]
previews = previews[-MAX_PREVIEWS:]
if (done < len(prompts) and results == step["results"] and step.get("prompts") == len(prompts)
and previews == step.get("previews", []) and (not failed or step.get("ok") is False)):
return False # nothing new to keep: no write
step["results"] = results
if previews:
step["previews"] = previews
step["state"] = "running" if done < len(prompts) else ("cancelled" if stopped else "error" if failed else "done")
step["prompts"] = len(prompts)
if failed:
step["ok"] = False
step["error"] = step.get("submission_error") or "ComfyUI 运行出错 / The run failed"
if done >= len(prompts):
step.pop("progress", None)
step["ms"] = int((time.time() - step["at"]) * 1000)
if step["state"] in ("done", "error") and not step.get("told") and ai_gates.run_wait() == "call":
step["call_ai"] = True # the workbench offers the user a line to paste to the AI
task["updated"] = max(task["updated"], time.time())
ai_tasks._save(task)
ai_tasks._notify(task_id)
return done >= len(prompts)
def _stop_run(task_id, step_id, reason):
"""Monitoring ended without proof of completion; never pretend the run succeeded."""
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
step = next((item for item in (task or {}).get("steps", []) if item["id"] == step_id), None)
if not step or step.get("state") != "running":
return
step.update(state="stopped", ok=False, error=reason, ms=max(0, int((time.time() - step["at"]) * 1000)))
step.pop("progress", None)
task["updated"] = max(task["updated"], time.time())
ai_tasks._save(task)
ai_tasks._notify(task_id)
def watch(step_id, receipt):
"""Follow only IDs returned by this submission, in batch order; never infer ownership from the queue."""
with ai_tasks._lock:
task_id = next((task["id"] for task in ai_tasks._load().values()
if any(step["id"] == step_id for step in task.get("steps") or [])), None)
if not task_id:
return
prompts = receipt.get("prompt_ids") or []
if not prompts:
_stop_run(task_id, step_id, "缺少运行 ID / Missing submission IDs; refresh ComfyUI before retrying")
return
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
step = next((item for item in (task or {}).get("steps", []) if item["id"] == step_id), None)
if not step:
return # The user may remove the task while its submission is returning.
step["prompt_ids"] = list(prompts)
step_at = step["at"]
for other in task.get("steps", []): # a new run: no line to paste for an older one any more
if other is not step:
other.pop("call_ai", None)
if receipt.get("submission_error"):
step.update(submission_error=receipt["submission_error"], error=receipt["submission_error"], ok=False)
ai_tasks._save(task)
def follow():
start = time.monotonic()
pace, estimated = ai_run_watch.Pace(), False
while time.monotonic() - start < ai_tasks.RUN_WATCH_SECONDS:
try:
history = {pid: _queue().get_history(prompt_id=pid).get(pid) for pid in prompts}
except Exception: # noqa: BLE001 - stop the receipt, not the actual ComfyUI run
_stop_run(task_id, step_id, "无法读取运行记录 / Run history could not be read; check ComfyUI's queue")
return
try:
running, pending = _queue().get_current_queue()
queued = {item[1]: item for item in [*running, *pending]}
except Exception: # noqa: BLE001 - the queue only helps (estimate, a stopped run); history decides
queued = None
if not estimated: # from the queue, or from the history of a run that was quicker than this look
item = next((queued[pid] for pid in prompts if pid in (queued or {})), None) or next(((history.get(pid) or {}).get("prompt") for pid in prompts if (history.get(pid) or {}).get("prompt")), None)
estimated = _estimate(task_id, step_id, item)
if _finish_run(task_id, step_id, prompts, {k: v for k, v in history.items() if v},
None if queued is None else set(queued)):
return
try:
progress = progress_of(prompts)
if progress and progress["state"] == "running":
_more_progress(progress, step_at, pace)
except (AttributeError, KeyError, TypeError, ValueError):
progress = None # progress is optional; keep watching the authoritative history
if progress:
_show_progress(task_id, step_id, progress)
time.sleep(RUN_POLL_SECONDS)
_stop_run(task_id, step_id, "运行跟踪已超时,未停止出图 / Monitoring timed out; the workflow was not cancelled. Check ComfyUI's queue")
threading.Thread(target=follow, name="anomalous-ai-run", daemon=True).start()
def _estimate(task_id, step_id, item):
"""The run's estimate (ai_run_watch.estimate) and map (ai_run_map) from its queued prompt `item`,
kept on the step once, with what changed since the task's run before."""
if not item or len(item) < 3 or not isinstance(item[2], dict):
return False
memory = ai_run_watch.vram()
try:
found = ai_run_watch.estimate(item[2], (memory or {}).get("total_mb"))
shape = ai_run_map.map_of(item[2])
except Exception: # noqa: BLE001 - an odd prompt leaves the run without an estimate, nothing more
return True
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
steps = (task or {}).get("steps", [])
at = next((index for index, entry in enumerate(steps) if entry["id"] == step_id), None)
if at is None:
return True
before = next((entry["map"] for entry in reversed(steps[:at]) if entry.get("tool") == "run_workflow" and entry.get("map")), None)
steps[at].update(estimate=found, map=shape, changes=ai_run_map.changes(before, shape))
ai_tasks._save(task)
ai_tasks._notify(task_id)
return True
def _more_progress(progress, started, pace):
"""The console's last lines, its bar's time left (else the pace of the steps), the card's memory."""
lines = ai_run_watch.console_since(started)
if lines:
progress["console"] = lines
bar = ai_run_watch.bar_of(lines)
if bar and bar.get("note"):
progress["note"] = bar["note"]
timing = None
if bar and bar["max"] == progress.get("max"):
timing = {key: bar[key] for key in ("left", "rate") if key in bar}
if not timing and progress.get("max", 0) > 1:
timing = pace.update((progress.get("prompt"), progress.get("node")), progress["value"], progress["max"])
if timing:
progress.update(timing)
memory = ai_run_watch.vram()
if memory:
progress["vram"] = memory
def summary(run):
"""One sentence for the AI on where run step `run` stands."""
state, progress = run.get("state"), run.get("progress") or {}
warned = " ".join(ai_run_watch.warning_words(item) for item in (run.get("estimate") or {}).get("warnings", []))
warning = (f" Warning: {warned} Tell the user (report) and ask_user whether to stop it (stop_run)." if warned and state == "running" else "")
if state == "running" and progress.get("state") == "queued":
return f"Your run is queued, {progress.get('ahead', 0)} ahead of it; it has not started. Do not run it again.{warning}"
if state == "running":
parts = []
if progress.get("prompts", 0) > 1:
parts.append(f"run {progress['prompt']} of {progress['prompts']}")
if progress.get("node"):
parts.append(f"{progress['node']} step {progress['value']}/{progress['max']}" if progress.get("max", 0) > 1
else progress["node"])
if "percent" in progress:
parts.append(f"{progress['percent']}%")
if progress.get("nodes"):
parts.append(f"{progress['nodes_done']} of {progress['nodes']} nodes done")
if progress.get("left") is not None:
parts.append(f"about {progress['left']} s left in this node")
parts.append(f"{round(time.time() - run['at'])} s so far")
if progress.get("note"):
parts.append(f"console says: {progress['note']}")
return (f"Your run is still going ({', '.join(parts)}). It has not finished: {waiting_words()}; "
"do not look in the output folder or guess." + warning)
if state == "cancelled":
return ("The user stopped your run in Anomalous; do not start it again unless they ask." if run.get("stop") == "user"
else "Your run was stopped (stop_run).")
if state == "done":
pictures = [item["filename"] for item in run.get("results") or [] if item.get("type") == "image"]
return f"Your run finished in {round(run.get('ms', 0) / 1000)} s with {len(pictures)} picture(s): {', '.join(pictures[:8])}."
if state in ("asking", "allowed"):
return "Your run waits for the user's OK in Anomalous; it has not started."
return f"Your run did not finish: {run.get('error') or state}."
def news(task_id):
"""The run news for the AI working on task `task_id`: while its latest run goes, and once after it ends."""
with ai_tasks._lock:
task = ai_tasks._load().get(task_id) if task_id else None
run = next((step for step in reversed((task or {}).get("steps", [])) if step.get("tool") == "run_workflow"), None)
if not run or not run.get("prompt_ids") or run.get("told"):
return None
answer = {"state": run.get("state"), "summary": summary(run)}
if run.get("state") == "running" and run.get("progress"):
answer["progress"] = run["progress"]
if run.get("state") not in ("running", "asking", "allowed"):
run["told"], run["shown"] = True, False # show_pictures attaches them to this answer
run.pop("call_ai", None)
ai_tasks._save(task)
return answer
def waiting_words():
"""How the AI is to wait for a run still going, as the user chose (ai_gates.run_wait)."""
if ai_gates.run_wait() == "call":
return ("the user tells you when it is done (Anomalous gives them a line to paste to you): say it is running "
"and end your turn; do not wait with run_status")
return "call run_status with wait_seconds to wait for it"
def show_pictures(result, task_id):
"""Attaches thumbnails of the latest run's pictures to the answer that first tells the AI it ended
(status() or news() marked it), so it sees what came out without get_image_info; once per run."""
if not isinstance(result, dict) or result.get("isError") or not task_id:
return result
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
run = next((step for step in reversed((task or {}).get("steps", [])) if step.get("tool") == "run_workflow"), None)
if not run or run.get("shown") is not False:
return result
run["shown"] = True
ai_tasks._save(task)
shown, items = [], []
for item in run.get("results") or []:
rel = "/".join(part for part in (item.get("subfolder"), item.get("filename")) if part)
if item.get("type") != "image" or len(items) >= MAX_SHOWN:
continue
try:
picture = _image_content(_resolve_image(rel)[1])
except ToolError:
picture = None
if picture:
shown.append(rel)
items.append(picture)
if not items:
return result
data = dict(result.get("structuredContent") or {})
data["pictures_attached"] = shown
retext(result, data)
result["content"] = [*result["content"], *items]
return result
def tell(result, task_id):
"""Puts the run news (news()) into a tool result, so the AI need not go and look."""
run = news(task_id) if isinstance(result, dict) and not result.get("isError") else None
if not run:
return result
data = result.get("structuredContent")
data = dict(data) if isinstance(data, dict) else {}
data["your_run"] = run
retext(result, data)
return result
def status(client, session="", task_id=None, wait_seconds=0, step_id=None):
"""The AI's run_status: its task's latest run, where it stands, and what it made; with
`wait_seconds`, waits (up to MAX_STATUS_WAIT) until the run ends. No run yet is an answer, not
an error (an AI asks before it runs, and the task would show a failed step)."""
task_id = ai_tasks.active_task_id(client, session, task_id)
deadline = time.monotonic() + max(0.0, min(float(wait_seconds or 0), MAX_STATUS_WAIT))
while True:
task = ai_tasks.get_task(task_id) if task_id else None
run = next((step for step in reversed((task or {}).get("steps", []))
if step.get("tool") == "run_workflow" and (not step_id or step["id"] == step_id)), None)
if not run:
return {"state": "none", "summary": "No run in this task yet.",
"next": "run_workflow starts one (workflow: a saved one, with the prompt, seed or size to change)."}
if run.get("state") not in ("running", "asking", "allowed") or time.monotonic() >= deadline:
break
time.sleep(1)
answer = {"state": run.get("state"), "seconds": round(run.get("ms", 0) / 1000) if run.get("state") != "running"
else round(time.time() - run["at"]), "summary": summary(run)}
if run.get("state") not in ("running", "asking", "allowed") and not run.get("told"):
with ai_tasks._lock: # told here: the next answers need not repeat it
task = ai_tasks._load().get(task_id)
step = next((item for item in (task or {}).get("steps", []) if item["id"] == run["id"]), None)
if step is not None:
step["told"], step["shown"] = True, False # show_pictures attaches them to this answer
step.pop("call_ai", None)
ai_tasks._save(task)
if run.get("progress"):
answer["progress"] = run["progress"]
if (run.get("estimate") or {}).get("samplers"):
answer["estimate"] = run["estimate"]
if run.get("prompt_ids"):
answer["prompt_ids"] = run["prompt_ids"]
pictures = ["/".join(part for part in (item.get("subfolder"), item["filename"]) if part)
for item in run.get("results") or [] if item.get("type") == "image"]
if pictures:
answer["pictures"] = pictures
previews = ["/".join(part for part in (item.get("subfolder"), item["filename"]) if part) for item in run.get("previews") or []]
if previews:
answer["previews"] = previews
answer["previews_note"] = ("What the run's PreviewImage nodes showed (pose skeletons, masks…), in ComfyUI's temp "
"folder: zoom_image(image, folder='temp') or get_image_info(image, folder='temp', "
"include_image=true) shows them; there is no need to run a preprocessor yourself.")
if run.get("error"):
answer["error"] = run["error"]
# What to do now, in one line a small model can follow (get_situation and the canvas tools say theirs too).
answer["next"] = {
"running": (f"It is still running: {waiting_words()}. Do not guess it is done or run it again."
if ai_gates.run_wait() == "call" else
"run_status(wait_seconds=55): it is still running; do not guess it is done or run it again."),
"allowed": "run_status(wait_seconds=55): it is about to run.",
"done": ("The first answer telling you it ended has its pictures attached (pictures_attached): look at them, "
"then report what came out with images= its pictures; show_panel only to compare or ask. "
"zoom_image looks closer."),
"asking": "get_answers(wait_seconds=55): the run waits for the user's OK in Anomalous.",
"cancelled": "It was stopped and nothing more comes of it: ask_user what to change before running again.",
}.get(run.get("state"), "get_situation shows what went wrong (a missing model: check_workflow_models); fix it, then run again.")
return answer
+592
View File
@@ -0,0 +1,592 @@
"""AI tasks: what AI apps did through Anomalous's MCP endpoint, kept for the AI workbench page.
Every tool call an AI app makes (mcp_server.py) becomes a step of a task: a task is one piece
of work, named by the AI with begin_task, else the calls of one app that follow each other
without a long pause. A step starts in state "running" when the call begins and is updated in
place when it finishes: what was done (the tool, a short summary of its arguments), whether it
worked, and what came out of it, so the page can show it live and lead there: the models it
used, the combo it placed, the canvas change it made (by its activity log entry, for undo),
the pictures a run made (ai_task_runs.py watches ComfyUI's history until the run ends), the
audio it spoke. The AI can also leave a note (report), with pictures it made another way. What
each tool is (kind, stage of the work, gate) is action_catalog.py's; the user's own changes
made while an AI works join its task through ai_task_changes.py. A step a restart left
"running" is marked stopped when the record is read again.
The user can start a task first (New task, POST /anomalous/ai_tasks): it waits for an AI, and
the next app that calls takes it over, so the work it does next starts there instead of joining
the app's last task. Likewise an app whose task ended or paused goes on in the task the user wrote
to (a message, marks, pictures, materials) that it has not read yet, rather than starting one.
Tasks are files in the ComfyUI user folder (anomalous/ai_tasks/<id>.json), the newest
MAX_TASKS kept. Recording never fails a tool call: the record is a convenience. The page
learns of a change through the "anomalous.ai_task" event. Reading is open like the other
pages; renaming and removing are writes (local_guard.py keeps them to this computer). The
routes are in ai_tasks_routes.py.
"""
import json
import logging
import os
import re
import shutil
import threading
import time
import uuid
import folder_paths
from . import action_catalog, activity_log
from .ai_task_handoff import just_done, latest_replied, recovery_target, unresolved
from .path_utils import atomic_write_json
FORMAT = 1
MAX_TASKS = 300
MAX_STEPS = 400
MAX_TASK_BYTES = 2 * 1024 * 1024
IDLE_SECONDS = 20 * 60 # a longer pause between calls of one app starts a new task
MAX_TEXT = 300
MAX_NOTE = 2000
MAX_TITLE = 80
MAX_RESULTS = 40
RUN_WATCH_SECONDS = 30 * 60
MEDIA_EXTS = {".png", ".jpg", ".jpeg", ".webp", ".gif", ".mp4", ".webm"}
# What each tool is (its kind, stage and gate) is action_catalog.py's.
OWN_STEPS = {"report", "log_decision", "log_outside", "show_panel", "ask_user", "show_run_map", "propose_repaint_area"} # make their step themselves (with its text or panel)
_lock = threading.RLock()
_tasks = None # id -> task
_current = {} # client/session key -> id of the task its calls go to
_waiting = [None] # id of the task the user started that no AI has taken yet
log = logging.getLogger(__name__)
# ---------------------------------------------------------------- storage
def tasks_dir():
user_dir = folder_paths.get_user_directory() if hasattr(folder_paths, "get_user_directory") else None
if not user_dir:
user_dir = os.path.join(folder_paths.base_path, "user")
directory = os.path.join(user_dir, "anomalous", "ai_tasks")
os.makedirs(directory, exist_ok=True)
return directory
def _path(task_id):
return os.path.join(tasks_dir(), f"{task_id}.json")
def marks_dir(task_id):
"""Where the pictures the user marked for a task are kept (ai_marks.py); gone with the task."""
return os.path.join(os.path.dirname(tasks_dir()), "ai_marks", re.sub(r"[^\w-]", "", task_id))
def _forget(task_id):
"""The task's own files: its record and its marked pictures (the plugin's, not the user's pictures)."""
try:
os.remove(_path(task_id))
except OSError:
pass
shutil.rmtree(marks_dir(task_id), ignore_errors=True)
def _load():
global _tasks
if _tasks is None:
_tasks = {}
for name in os.listdir(tasks_dir()):
if not name.endswith(".json"):
continue
try:
with open(os.path.join(tasks_dir(), name), encoding="utf-8") as source:
task = json.load(source)
except (OSError, ValueError):
continue
if isinstance(task, dict) and task.get("format") == FORMAT and isinstance(task.get("id"), str):
_tasks[task["id"]] = task
if _stop_running(task):
_save(task)
# A task the user started before ComfyUI restarted still waits for the next AI.
waiting = [task for task in _tasks.values() if task.get("waiting") and not task.get("ended")]
if waiting and _waiting[0] is None:
_waiting[0] = max(waiting, key=lambda task: task.get("started", 0))["id"]
return _tasks
def _stop_running(task):
"""Steps a call or a run left "running" when ComfyUI stopped: nothing watches them any more."""
stopped = False
for step in task.get("steps") or []:
if step.get("state") == "asking" and step.get("gate") and not step["gate"].get("timed_out"):
step["gate"]["timed_out"] = True # no old process is still waiting for this decision
stopped = True
if step.get("state") == "running":
step.update(state="stopped", ok=False, error="ComfyUI 重启时这一步还没结束 / ComfyUI restarted before this step ended")
stopped = True
return stopped
def _save(task):
try:
atomic_write_json(_path(task["id"]), task, MAX_TASK_BYTES)
except (OSError, ValueError) as error:
log.warning("Anomalous AI tasks: %s not saved: %s", task["id"], error)
def _drop_oldest():
tasks = _load()
if len(tasks) <= MAX_TASKS:
return
for task in sorted(tasks.values(), key=lambda item: item.get("updated", 0))[: len(tasks) - MAX_TASKS]:
tasks.pop(task["id"], None)
_forget(task["id"]) # the pictures and files it made stay
def _notify(task_id, extra=None):
try:
from server import PromptServer
PromptServer.instance.send_sync("anomalous.ai_task", {"id": task_id, **(extra or {})})
except Exception: # noqa: BLE001 - tests and early start have no server; the page polls on open
pass
# ---------------------------------------------------------------- who is calling (named in ai_callers.py)
def _owner_key(client, session=""):
return f"{client}::{session}" if session else client
def _bind_owner(client, session, task_id):
_current[_owner_key(client, session)] = task_id
if not session or client not in _current:
_current[client] = task_id
# ---------------------------------------------------------------- tasks and steps
def _short(value, limit=MAX_TEXT):
text = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False)
return text if len(text) <= limit else text[:limit] + "…"
def _new_task(client, title="", now=None, session=""):
now = time.time() if now is None else now
task = {"format": FORMAT, "id": f"{int(now)}_{uuid.uuid4().hex[:8]}", "title": _short(title, MAX_TITLE),
"title_from": "ai" if title else "auto", "client": client, "started": now, "updated": now,
"ended": None, "steps": []}
if session:
task["session"] = session
_load()[task["id"]] = task
_bind_owner(client, session, task["id"])
_drop_oldest()
return task
def _claim_explicit(tasks, task_id, client, session=""):
if not task_id or task_id not in tasks or tasks[task_id].get("ended"):
return None
picked = tasks[task_id]
if picked.get("waiting"):
if _waiting[0] == task_id:
_waiting[0] = None
picked["waiting"] = False
if not picked.get("client"):
picked["client"] = client
_bind_owner(client, session, task_id)
return picked
def _take_waiting(client, now, session=""):
"""The task the user started, now this app's (its own open task ends), or None."""
task = _load().get(_waiting[0])
_waiting[0] = None
if not task or task.get("ended"): # it waits until an AI takes it, as its page says, or the user starts another
return None
owner = _owner_key(client, session)
previous = _load().get(_current.get(owner) or (None if session else _current.get(client)))
if previous and previous is not task and not previous.get("ended"):
previous["ended"] = previous.get("updated")
_save(previous)
task["client"] = client
if session:
task["session"] = session
task["waiting"] = False
_bind_owner(client, session, task["id"])
return task
ABOUT_TYPES = {"model": "model_id", "image": "image", "combo": "name", "workflow": "name"} # what a task the user starts can be about
def _about(items):
"""What the user handed to the AI with a task (a model, an output image, a combo), checked: plain strings only."""
out = []
for item in items if isinstance(items, list) else []:
key = ABOUT_TYPES.get(item.get("type")) if isinstance(item, dict) else None
if key and isinstance(item.get(key), str) and item[key].strip():
out.append({"type": item["type"], key: _short(item[key].strip(), 300),
**({"label": _short(item["label"], 120)} if isinstance(item.get("label"), str) else {})})
return out[:5]
def start_task(title="", now=None, about=None, from_task=None):
"""A task the user starts; it waits for the next AI app that calls. `about`: what the user
handed to the AI with it (from a model page, an image or a combo); `from_task`: the task it
goes on from (ai_task_links.py). Returns the task."""
now = time.time() if now is None else now
with _lock:
before = _load().get(_waiting[0])
if before and not before.get("steps"): # started before and never taken: this one replaces it
remove_task(before["id"])
task = {"format": FORMAT, "id": f"{int(now)}_{uuid.uuid4().hex[:8]}", "title": _short(str(title or "").strip(), MAX_TITLE),
"title_from": "user", "client": "", "started": now, "updated": now, "ended": None, "steps": [], "waiting": True}
if not task["title"]:
task["title_from"] = "auto"
if _about(about):
task["about"] = _about(about)
if from_task in _load():
task["from"] = from_task
_load()[task["id"]] = task
_waiting[0] = task["id"]
_drop_oldest()
_save(task)
_notify(task["id"])
return task
def active_task_id(client, session="", task_id=None):
"""The active task id for this caller without creating a new task."""
with _lock:
tasks = _load()
if task_id and task_id in tasks:
return task_id
found = _current.get(_owner_key(client, session)) or (None if session else _current.get(client))
return found if found in tasks else None
def current_task(client, now=None, session="", task_id=None, follow_up=False, read_only=False):
"""The task this app's calls go to now: an explicit task, one the user started for the next
AI, else its open one, unless it paused too long. `follow_up`: a panel, question or note, which
right after the AI said its task is done is about that task (ai_task_handoff.just_done).
`read_only` keeps such a task ended while a follow-up reopens it; both honor an explicit id."""
now = time.time() if now is None else now
with _lock:
tasks = _load()
task = tasks.get(task_id or _current.get(_owner_key(client, session)))
if (follow_up or read_only) and just_done(task, tasks, tasks.get(_waiting[0]), client, now, explicit=bool(task_id)):
_bind_owner(client, session, task["id"])
if follow_up:
task["ended"] = None
_save(task)
return task
picked = _claim_explicit(tasks, task_id, client, session)
if picked:
return picked
waiting = _take_waiting(client, now, session=session)
if waiting:
return waiting
task = tasks.get(_current.get(_owner_key(client, session)) or (None if session else _current.get(client)))
if task and not task.get("ended") and now - task.get("updated", 0) < IDLE_SECONDS:
_bind_owner(client, session, task["id"])
return task
if task and not task.get("ended"):
task["ended"] = task.get("updated")
_save(task)
return _take_replied(tasks, client, now, session) or _new_task(client, now=now, session=session)
def _take_replied(tasks, client, now, session="", previous=None):
"""The newest task of this app the user wrote to lately, since its AI last read (an ended one
opens again), now this app's instead of a new one (its own open task `previous` ends), or None."""
replied = latest_replied(tasks, client, now)
if replied is None:
return None
if previous and previous is not replied and not previous.get("ended"):
previous["ended"] = previous.get("updated")
_save(previous)
replied["ended"] = None
_bind_owner(client, session, replied["id"])
_save(replied)
return replied
def begin_task(client, title, now=None, new_task=False, session="", task_id=None):
"""Name or resume the active task; only an explicit new piece of work splits it."""
now = time.time() if now is None else now
with _lock:
tasks = _load()
previous = tasks.get(_current.get(_owner_key(client, session)) or (None if session else _current.get(client)))
task = None if new_task else _claim_explicit(tasks, task_id, client, session)
if task is None:
task = _take_waiting(client, now, session=session)
if task is None and not new_task and previous and not previous.get("ended") and now - previous.get("updated", 0) < IDLE_SECONDS:
task = previous
if task is None and not new_task:
task = _take_replied(tasks, client, now, session, previous)
if task is None:
if previous and not previous.get("ended"):
previous["ended"] = now
_save(previous)
task = _new_task(client, title, now, session=session)
if new_task and previous:
task["from"] = previous["id"] # begun from that one (ai_task_links.py)
elif not task.get("title") or task.get("title_from") == "auto":
task["title"] = _short(title, MAX_TITLE)
task["title_from"] = "ai"
_bind_owner(client, session, task["id"])
task["updated"] = now
_save(task)
_notify(task["id"])
return task
def _add_step(task, step):
task["steps"].append(step)
del task["steps"][:-MAX_STEPS]
task["updated"] = max(task.get("updated", 0), step["at"] + step.get("ms", 0) / 1000)
if task.get("title_from") == "auto" and not task.get("title") and step["kind"] not in ("look", "note") and step["ok"]:
task["title"] = step["summary"][:MAX_TITLE]
def summary_of(tool, arguments):
"""A few words on what a call asked for: its main argument."""
for key in ("title", "query", "text", "name", "workflow", "character", "model_id", "url", "what", "names", "type", "types"):
if arguments.get(key) not in (None, "", []):
value = arguments[key]
if key == "model_id": # "type:0:folder/file.safetensors": the file, and where it moves to
value = str(value).split(":", 2)[-1].rsplit("/", 1)[-1] + (f" → {arguments['to_type']}" if arguments.get("to_type") else "")
return _short(f"{tool}: {_short(value, 120)}", 160)
return tool
def _model_result(model_id):
from .mcp_tools import _resolve_model
try:
folder_type, path_idx, base_dir, file_path = _resolve_model(model_id)
except Exception: # noqa: BLE001 - a model that went away is still worth its id
return {"type": "model", "model_id": str(model_id)}
rel = os.path.relpath(file_path, base_dir).replace(os.sep, "/")
return {"type": "model", "model_id": str(model_id), "folder": folder_type, "path_idx": path_idx, "rel": rel,
"name": os.path.basename(file_path)}
def results_of(tool, arguments, data):
"""What came out of a call, for the page to show and lead to."""
data = data if isinstance(data, dict) else {}
out = []
if tool in ("set_model", "add_lora", "open_in_anomalous") and arguments.get("model_id"):
out.append(_model_result(arguments["model_id"]))
if tool == "place_combo" and arguments.get("name"):
out.append({"type": "combo", "name": _short(arguments["name"], 120)})
if isinstance(data.get("_activity"), str):
out.append({"type": "canvas", "activity": data["_activity"]})
if tool == "speak":
out.extend({"type": "audio", "file": _short(f, 200)} for f in data.get("files") or [] if isinstance(f, str))
if tool == "scan_models" and data.get("started"):
out.append({"type": "scan"})
if tool == "download_model" and data.get("started"):
out.append({**_download_result(data), "name": _short(str(data.get("model") or ""), 160)})
if tool == "download_missing_models":
out.extend(_download_result(item) for item in data.get("started") or [] if isinstance(item, dict))
return out[:MAX_RESULTS]
def _download_result(started):
"""A download the AI started (mcp_actions' answer): the file it makes, and its size when the site said."""
size = started.get("size")
return {"type": "download", "file": _short(str(started.get("to") or "").rsplit("/", 1)[-1], 200),
"size": size if isinstance(size, int) and size > 0 else None}
def _count_of(data):
"""How many things a read found: the length of its first list."""
if isinstance(data, dict):
for value in data.values():
if isinstance(value, list):
return len(value)
return None
def begin_step(client, tool, arguments, started=None, read_only=False, session="", task_id=None):
"""Starts an in-flight step in state 'running' before a tool executes. Returns (task_id, step_id) or None."""
if not action_catalog.is_step(tool) or tool in OWN_STEPS:
return None
try:
started = time.time() if started is None else started
args = arguments if isinstance(arguments, dict) else {}
what = action_catalog.entry(tool, read_only)
step = {"id": uuid.uuid4().hex[:10], "at": started, "ms": 0, "tool": tool, "kind": what.kind, "stage": what.stage,
"ok": True, "state": "running", "summary": summary_of(tool, args), "results": [],
"recovery_target": recovery_target(tool, args)}
with _lock:
task = current_task(client, started, session=session, task_id=task_id, read_only=read_only)
_add_step(task, step)
_save(task)
_notify(task["id"])
return task["id"], step["id"]
except Exception: # noqa: BLE001 - the record must never fail the AI's call
log.warning("Anomalous AI tasks: begin_step of %s failed", tool, exc_info=True)
return None
def finish_step(client, tool, arguments, result, started, read_only=False, step_ref=None, session="", task_id=None):
"""Completes an in-flight step (or creates one if step_ref is None) with its final outcome. Never raises."""
if not action_catalog.is_step(tool) or tool in OWN_STEPS:
return None
try:
args = arguments if isinstance(arguments, dict) else {}
data = result.get("structuredContent") if isinstance(result, dict) else None
failed = not isinstance(result, dict) or bool(result.get("isError"))
error = _short(" ".join(item.get("text", "") for item in result.get("content") or [] if isinstance(item, dict))) if failed and isinstance(result, dict) else ""
ms = max(0, int((time.time() - started) * 1000))
what = action_catalog.entry(tool, read_only)
kind = what.kind
waiting = isinstance(data, dict) and data.get("waiting_for_user") # a gate the user has not answered (ai_gates.py)
state = "error" if failed else "asking" if waiting else "running" if tool == "run_workflow" else "done"
summary = summary_of(tool, args)
results = [] if failed else results_of(tool, args, data)
with _lock:
task = _load().get(step_ref[0]) if isinstance(step_ref, tuple) and len(step_ref) == 2 else None
step = next((item for item in (task or {}).get("steps", []) if item.get("id") == step_ref[1]), None) if task else None
# A user can answer just after check() times out, before this call returns.
gate = (step or {}).get("gate") or {}
if waiting and gate.get("decision"):
failed = gate["decision"] == "deny"
state = "error" if failed else "done" if gate.get("used") else "allowed"
error = (step.get("error") or "你没有允许 / You said no") if failed else ""
if task is None:
task = current_task(client, started, session=session, task_id=task_id, read_only=read_only)
if step is None:
step = {"id": uuid.uuid4().hex[:10], "at": started, "ms": ms, "tool": tool, "kind": kind, "stage": what.stage,
"ok": not failed, "state": state, "summary": summary, "results": results}
_add_step(task, step)
else:
step.update(ms=ms, ok=not failed, state=state, summary=summary, results=results)
step["recovery_target"] = recovery_target(tool, args)
if error:
step["error"] = error
else:
step.pop("error", None)
if kind == "look" and not failed and tool.startswith(("search_", "list_")):
step["count"] = _count_of(data)
task["updated"] = max(task.get("updated", 0), step["at"] + ms / 1000)
if task.get("title_from") == "auto" and not task.get("title") and kind not in ("look", "note") and step["ok"]:
task["title"] = summary[:MAX_TITLE]
_save(task)
for result in results:
if result["type"] == "canvas":
activity_log.tag(result["activity"], task["id"]) # the activity page links back to the task
_notify(task["id"])
return step
except Exception: # noqa: BLE001 - the record must never fail the AI's call
log.warning("Anomalous AI tasks: step of %s not recorded", tool, exc_info=True)
return None
def record(client, tool, arguments, result, started, read_only=False, session="", task_id=None):
"""One tool call as a step of the app's current task. Never raises."""
return finish_step(client, tool, arguments, result, started, read_only, session=session, task_id=task_id)
# ---------------------------------------------------------------- notes from the AI
def _output_media(path):
"""An output-folder file the AI names ("subfolder/name.png"), as an image result, or None."""
root = os.path.realpath(folder_paths.get_output_directory())
full = os.path.realpath(os.path.join(root, str(path).replace("\\", "/").lstrip("/")))
if os.path.commonpath((root, full)) != root or not os.path.isfile(full):
return None
if os.path.splitext(full)[1].lower() not in MEDIA_EXTS:
return None
rel = os.path.relpath(full, root).replace(os.sep, "/")
subfolder, _, filename = rel.rpartition("/")
return {"type": "image", "filename": filename, "subfolder": subfolder, "folder": "output"}
def report(client, text, images=None, done=False, now=None, session="", task_id=None, reply_to=None):
"""A note from the AI, with pictures it made (in ComfyUI's output folder); `reply_to`: the user's
message or marks it answers (both steps then point at each other). Returns the step."""
text = str(text or "").strip()
if not text and not images:
raise ValueError("text or images is needed")
results, missing = [], []
for path in (images or [])[:MAX_RESULTS]:
item = _output_media(path)
(results.append(item) if item else missing.append(str(path)))
now = time.time() if now is None else now
step = {"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": "report", "kind": "note", "ok": True,
"state": "done", "summary": "report", "text": _short(text, MAX_NOTE), "results": results}
with _lock:
task = current_task(client, now, session=session, task_id=task_id, follow_up=True)
asked = next((item for item in task["steps"] if reply_to and item["id"] == reply_to and item.get("kind") == "user"), None)
if reply_to and not asked:
raise ValueError("reply_to is the id of the user's message or marks (user_messages[].id) in this task")
if asked:
step["reply_to"], asked["replied"] = asked["id"], step["id"]
_add_step(task, step)
if done:
task["ended"] = now
_save(task)
_notify(task["id"])
return step, missing
# ---------------------------------------------------------------- the page's view
def task_summary(task, now=None):
now = time.time() if now is None else now
steps = task.get("steps") or []
kinds = {}
for step in steps:
kinds[step["kind"]] = kinds.get(step["kind"], 0) + 1
images = [item for step in steps for item in step.get("results") or [] if item.get("type") == "image"]
asks = sum(1 for step in steps if step.get("kind") == "panel" and not step["panel"].get("closed")
for block in step["panel"]["blocks"] if block.get("type") in ("choices", "input", "picks")
and block.get("id") and block["id"] not in step.get("answers", {}))
asks += sum(1 for step in steps if step.get("state") == "asking")
return {
"id": task["id"], "title": task.get("title") or "", "client": task.get("client") or ("" if task.get("waiting") else "AI"),
"started": task.get("started"), "updated": task.get("updated"), "ended": task.get("ended"),
"waiting": bool(task.get("waiting")) and not task.get("ended"),
"live": any(step.get("state") == "running" and now - step.get("at", 0) < RUN_WATCH_SECONDS for step in steps),
"steps": len(steps), "kinds": kinds, "errors": unresolved(steps),
"images": len(images), "cover": images[-1] if images else None, "asks": asks, "from": task.get("from"),
}
def list_tasks(limit=100):
with _lock:
tasks = sorted(_load().values(), key=lambda task: task.get("updated", 0), reverse=True)
return [task_summary(task) for task in tasks[:limit]]
def get_task(task_id):
with _lock:
task = _load().get(task_id)
return json.loads(json.dumps(task)) if task else None
def rename_task(task_id, title):
with _lock:
task = _load().get(task_id)
if not task:
return None
task["title"] = _short(str(title or "").strip(), MAX_TITLE)
task["title_from"] = "user" if task["title"] else "auto"
_save(task)
return task
def remove_task(task_id):
with _lock:
task = _load().pop(task_id, None)
if not task:
return False
for client, current in list(_current.items()):
if current == task_id:
_current.pop(client, None)
if _waiting[0] == task_id:
_waiting[0] = None
_forget(task_id)
return True
+291
View File
@@ -0,0 +1,291 @@
"""The AI workbench's routes over ai_tasks.py (the record) and ai_panels.py (panels, answers,
messages): the list and one task (reads, open like the other pages), and the writes the page
makes - start a task, rename or remove one, answer a panel, write to the AI, answer a gate and
set the gates' policies (ai_gates.py), stop a run (ai_run_control.py), hand the AI pictures
(ai_pictures.py) - (local_guard.py keeps the writes to this computer)."""
import asyncio
from aiohttp import web
from . import activity_log, ai_decisions, ai_gates, ai_marks, ai_pictures, ai_run_control, ai_task_canvas, ai_task_changes, ai_task_links, ai_tasks, ai_toolbox
async def api_list(request):
try:
limit = max(1, min(300, int(request.query.get("limit", 100))))
except ValueError:
limit = 100
return web.json_response({"tasks": ai_tasks.list_tasks(limit)})
async def api_start(request):
"""POST /anomalous/ai_tasks: {title, about?, from?} - the user starts a task for the next AI app that
calls; `about` names what it is about ([{type: model | image | combo, model_id | image | name}]),
`from` the task it goes on from."""
try:
body = await request.json()
except (ValueError, UnicodeDecodeError):
body = {}
body = body if isinstance(body, dict) else {}
title = body.get("title") if isinstance(body.get("title"), str) else ""
parent = body.get("from") if isinstance(body.get("from"), str) else None
return web.json_response({"task": ai_tasks.task_summary(ai_tasks.start_task(title, about=body.get("about"), from_task=parent))})
async def api_get(request):
task = ai_tasks.get_task(request.match_info["task_id"])
if task:
task["canvas_mode"] = ai_task_canvas.mode_of(task) # the task's own choice, else the user's default
task["decisions"], task["decisions_gap"] = ai_decisions.decisions(task), ai_decisions.gap(task)
task["links"] = ai_task_links.links(task["id"]) # the task it goes on from, those going on from it
ai_pictures.mark_gone(task) # pictures deleted since: the page says so
return web.json_response({"task": task}) if task else web.json_response({"error": "not_found"}, status=404)
async def api_change(request):
"""POST /anomalous/ai_tasks/{id}: {"title": "…"} renames it, {"remove": true} takes it off the list,
{"canvas": "new" | "current"} sets whether the AI works in a tab of the task's own, {"from": id | null}
links it to the task it goes on from (or takes the link off)."""
task_id = request.match_info["task_id"]
try:
body = await request.json()
except (ValueError, UnicodeDecodeError):
body = None
if not isinstance(body, dict):
return web.json_response({"error": "Expected {title} or {remove: true}"}, status=400)
if body.get("remove") is True:
return web.json_response({"removed": ai_tasks.remove_task(task_id)})
if "canvas" in body:
try:
return web.json_response({"canvas": ai_task_canvas.set_mode(task_id, body["canvas"])})
except ValueError as error:
return web.json_response({"error": str(error)}, status=400)
if "from" in body:
try:
return web.json_response({"from": ai_task_links.set_from(task_id, body["from"] if isinstance(body["from"], str) else None)})
except ai_task_links.LinkError as error:
return web.json_response({"error": str(error)}, status=400)
if "title" in body:
task = ai_tasks.rename_task(task_id, body["title"])
return web.json_response({"task": ai_tasks.task_summary(task)}) if task else web.json_response({"error": "not_found"}, status=404)
return web.json_response({"error": "Expected {title} or {remove: true}"}, status=400)
async def api_answer(request):
"""POST /anomalous/ai_tasks/{id}/answer: {step, block, choice: [ids] | text | skip: true} - the user's answer on a panel."""
from . import ai_panels, ai_repaint
try:
body = await request.json()
except (ValueError, UnicodeDecodeError):
body = None
if not isinstance(body, dict):
return web.json_response({"error": "Expected {step, block, choice | text}"}, status=400)
task_id, step_id, skip = request.match_info["task_id"], str(body.get("step")), body.get("skip") is True
try:
# A repaint area picked: its picture is made first (outside the lock), so the AI gets it with the answer.
extra = await asyncio.to_thread(ai_repaint.prepare, task_id, step_id, body.get("choice"), body.get("text"), skip)
except ai_repaint.RepaintError as error:
return web.json_response({"error": str(error)}, status=400)
try:
answers = ai_panels.answer(task_id, step_id, str(body.get("block")), body.get("choice"), body.get("text"), skip=skip, extra=extra)
except ai_panels.PanelError as error:
ai_repaint.discard(extra)
return web.json_response({"error": str(error)}, status=400)
return web.json_response({"answers": answers})
async def api_typing(request):
"""POST /anomalous/ai_tasks/{id}/typing: the user is typing an answer or a message (ai_answer_wait.py)."""
from . import ai_answer_wait
if not ai_answer_wait.typing(request.match_info["task_id"]):
return web.json_response({"error": "not_found"}, status=404)
return web.json_response({"ok": True})
async def api_message(request):
"""POST /anomalous/ai_tasks/{id}/message: {text, refs?} - the user writes to the AI on that task (refs: its "@")."""
from . import ai_panels
try:
body = await request.json()
except (ValueError, UnicodeDecodeError):
body = None
if not isinstance(body, dict):
return web.json_response({"error": "Expected {text}"}, status=400)
try:
step = ai_panels.leave_message(request.match_info["task_id"], body.get("text"), body.get("refs"))
except ai_panels.PanelError as error:
return web.json_response({"error": str(error)}, status=400)
return web.json_response({"step": step})
async def _body(request):
try:
body = await request.json()
except (ValueError, UnicodeDecodeError):
return None
return body if isinstance(body, dict) else None
async def api_gate(request):
"""POST /anomalous/ai_tasks/{id}/gate: {step, decision: allow | deny, text?, remember?}."""
body = await _body(request)
if body is None:
return web.json_response({"error": "Expected {step, decision}"}, status=400)
try:
step = ai_gates.decide(request.match_info["task_id"], str(body.get("step")), body.get("decision"),
body.get("text") or "", bool(body.get("remember")))
except ValueError as error:
return web.json_response({"error": str(error)}, status=400)
return web.json_response({"step": step})
async def api_gates(request):
"""GET /anomalous/ai_gates: {policies: {gate: policy}, canvas_tab, tools}; POST {gate, policy} changes one,
POST {canvas_tab: new | current} whether new tasks work in a tab of their own (ai_task_canvas.py),
POST {runs: call | wait}
how the AI waits for a run (ai_gates.run_wait), POST {answer_wait: 60 | 180 | 600} for the user's answer,
POST {no_answer: stop | recommended} what happens when that time runs out."""
def settings():
return {"policies": ai_gates.policies(), "canvas_tab": ai_task_canvas.default_mode(),
"runs": ai_gates.run_wait(), "answer_wait": ai_gates.answer_wait(), "no_answer": ai_gates.no_answer()}
if request.method == "GET":
return web.json_response(settings())
body = await _body(request)
try:
if "canvas_tab" in (body or {}):
ai_task_canvas.set_default(body["canvas_tab"])
elif "runs" in (body or {}):
ai_gates.set_run_wait(body["runs"])
elif "answer_wait" in (body or {}):
ai_gates.set_answer_wait(body["answer_wait"])
elif "no_answer" in (body or {}):
ai_gates.set_no_answer(body["no_answer"])
else:
ai_gates.set_policy((body or {}).get("gate"), (body or {}).get("policy"))
except ValueError as error:
return web.json_response({"error": str(error)}, status=400)
return web.json_response(settings())
async def api_marks(request):
"""POST /anomalous/ai_tasks/{id}/marks: {image, png, marks, text, mode, mask} - the user's marks on a picture for the AI."""
from . import ai_panels
body = await _body(request)
if body is None:
return web.json_response({"error": "Expected {image, png, marks}"}, status=400)
try:
step = await asyncio.to_thread(ai_marks.save, request.match_info["task_id"], body.get("image"),
body.get("png"), body.get("marks"), body.get("text"), None, body.get("mask"), body.get("mode"))
except ai_marks.MarksError as error:
return web.json_response({"error": str(error)}, status=400)
ai_panels.wake()
return web.json_response({"step": step})
async def api_marked(request):
"""GET /anomalous/ai_tasks/{id}/marks/{name}: a marked picture the user sent."""
path = ai_marks.path_of(request.match_info["task_id"], request.match_info["name"])
if not path:
return web.json_response({"error": "Not found"}, status=404)
return web.FileResponse(path, headers={"Cache-Control": "private, max-age=86400"})
async def api_toolbox_pick(request):
"""POST /anomalous/ai_tasks/{id}/toolbox: {mode, nodes, combos, workflows, prompts, models} - what the user hands the AI."""
from . import ai_panels
body = await _body(request)
try:
step = ai_toolbox.pick(request.match_info["task_id"], body if body is not None else None, (body or {}).get("mode", "prefer"))
except ai_toolbox.ToolboxError as error:
return web.json_response({"error": str(error)}, status=400)
ai_panels.wake()
return web.json_response({"step": step})
async def api_pictures(request):
"""POST /anomalous/ai_tasks/{id}/pictures: {images: ["subfolder/name.png"], text?} - the user hands the AI pictures."""
from . import ai_panels, ai_pictures
body = await _body(request)
try:
step = await asyncio.to_thread(ai_pictures.hand, request.match_info["task_id"], (body or {}).get("images"), (body or {}).get("text") or "")
except ai_pictures.PicturesError as error:
return web.json_response({"error": str(error)}, status=400)
ai_panels.wake()
return web.json_response({"step": step})
async def api_upload(request):
"""POST /anomalous/ai_pictures/upload?name= with a picture's bytes - one the user dropped or pasted
into a message, kept in the output folder (ai_pictures.save_upload): {image, filename, subfolder}."""
from . import ai_pictures
data = await request.read()
try:
image = await asyncio.to_thread(ai_pictures.save_upload, data, request.query.get("name", ""))
except ai_pictures.PicturesError as error:
return web.json_response({"error": str(error)}, status=400)
except OSError as error:
return web.json_response({"error": f"Could not keep the picture: {error}"}, status=500)
subfolder, _, filename = image.rpartition("/")
return web.json_response({"image": image, "filename": filename, "subfolder": subfolder})
async def api_stop(request):
"""POST /anomalous/ai_tasks/{id}/runs/{step}/stop - the user stops a run the AI started."""
from . import ai_panels
try:
answer = await asyncio.to_thread(ai_run_control.stop, request.match_info["step_id"], "user", request.match_info["task_id"])
except ai_run_control.RunControlError as error:
return web.json_response({"error": str(error)}, status=400)
ai_panels.wake()
return web.json_response(answer)
async def api_toolbox(request):
"""GET /anomalous/ai_toolbox?kind=all: what the picker lists; GET/POST /anomalous/ai_toolbox/favorites."""
if request.path.endswith("/favorites"):
if request.method == "GET":
return web.json_response({"favorites": ai_toolbox.favorites()})
body = await _body(request)
try:
return web.json_response({"favorites": ai_toolbox.set_favorites(body if body is not None else None)})
except ai_toolbox.ToolboxError as error:
return web.json_response({"error": str(error)}, status=400)
try:
return web.json_response(await asyncio.to_thread(ai_toolbox.catalog, request.query.get("kind", "")))
except ai_toolbox.ToolboxError as error:
return web.json_response({"error": str(error)}, status=400)
except Exception as error: # noqa: BLE001 - ComfyUI's node list or a store unreadable: say so to the page
return web.json_response({"error": str(error)}, status=502)
def register_routes(app):
activity_log.add_listener(ai_task_changes.note_activity) # the user's changes while an AI works
app.router.add_get("/anomalous/ai_tasks", api_list)
app.router.add_post("/anomalous/ai_tasks", api_start)
app.router.add_get("/anomalous/ai_tasks/{task_id}", api_get)
app.router.add_post("/anomalous/ai_tasks/{task_id}", api_change)
app.router.add_post("/anomalous/ai_tasks/{task_id}/answer", api_answer)
app.router.add_post("/anomalous/ai_tasks/{task_id}/message", api_message)
app.router.add_post("/anomalous/ai_tasks/{task_id}/typing", api_typing)
app.router.add_post("/anomalous/ai_tasks/{task_id}/gate", api_gate)
app.router.add_post("/anomalous/ai_tasks/{task_id}/marks", api_marks)
app.router.add_get("/anomalous/ai_tasks/{task_id}/marks/{name}", api_marked)
app.router.add_post("/anomalous/ai_tasks/{task_id}/toolbox", api_toolbox_pick)
app.router.add_post("/anomalous/ai_tasks/{task_id}/runs/{step_id}/stop", api_stop)
app.router.add_post("/anomalous/ai_tasks/{task_id}/pictures", api_pictures)
app.router.add_post("/anomalous/ai_pictures/upload", api_upload)
app.router.add_get("/anomalous/ai_toolbox", api_toolbox)
app.router.add_get("/anomalous/ai_toolbox/favorites", api_toolbox)
app.router.add_post("/anomalous/ai_toolbox/favorites", api_toolbox)
app.router.add_get("/anomalous/ai_gates", api_gates)
app.router.add_post("/anomalous/ai_gates", api_gates)
+155
View File
@@ -0,0 +1,155 @@
"""What the user hands the AI to work with ("给 AI 的素材", web/modules/ui_ai_toolbox.js): node
types, whole custom node packs (by folder: every node type in it), combos, workflows, saved
prompts and models picked for a task, and how strictly: prefer
(the AI is told to use them first) or only (add_node and new_workflow refuse other node types,
except ComfyUI's own built-in ones and the helpers that live only in the page). Favourites
(user/anomalous/ai_toolbox.json) are what a task has until the user picks for it. A pick is a
"user_toolbox" step the AI receives like a message (kind toolbox, ai_panels.py); begin_task
names the task's picks and get_toolbox gives them in full. The catalogue the picker shows uses
the same names the AI's tools use: node types from ComfyUI's /object_info (mcp_canvas_edit.py),
the rest from ai_toolbox_catalog.py."""
import json
import os
import threading
import time
import uuid
from . import ai_tasks
from .path_utils import atomic_write_json
KINDS = ("nodes", "packs", "combos", "workflows", "prompts", "models")
MODES = ("prefer", "only")
MAX_ITEMS = 60
MAX_NAME = 300
_lock = threading.Lock()
class ToolboxError(ValueError):
pass
def _path():
return os.path.join(os.path.dirname(ai_tasks.tasks_dir()), "ai_toolbox.json")
def clean(picks):
"""{kind: [names]} for KINDS: strings only, each kind once per name, at most MAX_ITEMS."""
if not isinstance(picks, dict):
raise ToolboxError("Expected {nodes, packs, combos, workflows, prompts, models}")
out = {}
for kind in KINDS:
items = picks.get(kind) or []
if not isinstance(items, list) or len(items) > MAX_ITEMS:
raise ToolboxError(f"{kind}: a list of up to {MAX_ITEMS} names")
names = []
for item in items:
if not isinstance(item, str) or not item.strip() or len(item) > MAX_NAME:
raise ToolboxError(f"{kind}: names are text of up to {MAX_NAME} characters")
if item.strip() not in names:
names.append(item.strip())
out[kind] = names
return out
def count(box):
return sum(len((box or {}).get(kind) or []) for kind in KINDS)
def favorites():
try:
with open(_path(), encoding="utf-8") as source:
return clean(json.load(source))
except (OSError, ValueError):
return clean({})
def set_favorites(picks):
picks = clean(picks)
with _lock:
atomic_write_json(_path(), picks)
return picks
def effective(task):
"""The task's picks ({mode, kinds…, "from": task|favorites}), or None when it has none."""
box = (task or {}).get("toolbox")
if box:
return {**box, "from": "task"}
liked = favorites()
return {"mode": "prefer", **liked, "from": "favorites"} if count(liked) else None
def words(box):
"""The picks in a sentence or two for the AI."""
parts = []
labels = {"nodes": "node types", "packs": "custom node packs (every node type in these folders; get_toolbox lists them)",
"combos": "combos (place_combo)", "workflows": "workflows (open_workflow opens one in a new tab)",
"prompts": "prompts (combo:<name> / workflow:<name> are the prompts inside that combo or workflow)",
"models": "models (model_id for set_model / add_lora / get_model)"}
for kind in KINDS:
if box.get(kind):
parts.append(f"{labels[kind]}: {', '.join(box[kind])}")
how = ("Use ONLY these node types (and those of the picked packs) when you add nodes; ComfyUI's own built-in nodes stay allowed. "
if box.get("mode") == "only" else "Use these first. ")
return f"The user picked these for you to work with. {how}{'; '.join(parts)}. get_toolbox gives them in full."
def pick(task_id, picks, mode="prefer", now=None):
"""The user's picks for task `task_id`: kept on the task and given to the AI as a step. Returns the step."""
if mode not in MODES:
raise ToolboxError("mode is prefer or only")
picks = clean(picks)
now = time.time() if now is None else now
box = {"mode": mode, **picks}
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
if not task:
raise ToolboxError("No such task")
task["toolbox"] = box
step = {"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": "user_toolbox", "kind": "user", "ok": True,
"state": "done", "summary": "user_toolbox", "toolbox": box,
"text": words(box) if count(box) else "The user took back the things they picked for you; use what fits.",
"delivered": False, "results": []}
task["steps"].append(step)
del task["steps"][:-ai_tasks.MAX_STEPS]
task["updated"] = max(task.get("updated", 0), now)
ai_tasks._save(task)
ai_tasks._notify(task_id)
return step
def builtin(info):
"""Whether a node type (its /object_info entry) is ComfyUI's own and free: not a custom node, not a paid API node."""
module = str((info or {}).get("python_module") or "")
return module == "nodes" or module.startswith("comfy_extras")
def chosen(box, name, info):
"""Whether node type `name` (its /object_info entry) is among picks `box`: itself, or its pack."""
return bool(box) and (name in (box.get("nodes") or []) or source_of(info) in (box.get("packs") or []))
def refused(task_id, types, info):
"""The node types among `types` the task's "only" picks do not allow (page-only helpers, unknown to /object_info, pass)."""
box = effective(ai_tasks.get_task(task_id)) if task_id else None
if not box or box.get("mode") != "only":
return []
return sorted({name for name in types if name in info and not chosen(box, name, info[name]) and not builtin(info[name])})
def source_of(info):
"""Where a node type comes from: its custom node folder, "builtin", or "api" (paid API nodes)."""
module = str((info or {}).get("python_module") or "")
if module.startswith("custom_nodes."):
return module.split(".")[1]
return "api" if module.startswith("comfy_api_nodes") else "builtin"
def catalog(kind):
"""What the picker lists besides node types (ai_toolbox_catalog.py); the page has those itself."""
from .ai_toolbox_catalog import everything
if kind != "all":
raise ToolboxError("kind is all; node types come from the page")
return everything()
+86
View File
@@ -0,0 +1,86 @@
"""Everything the materials picker (web/modules/ui_ai_toolbox.js) lists besides node types, in
one answer (GET /anomalous/ai_toolbox?kind=all): combos with their cover and prompt, saved
workflows with their cover, tags and prompts, prompts (the saved ones, and the ones inside
combos and workflows, ids "combo:<name>" and "workflow:<name>"), and models with their cover,
by the names and ids the AI's tools use. Read in one go and kept a few seconds, so the picker
opens at once; node types come from the page's own node definitions. prompt_entries() also
gives get_toolbox the text of picked prompts."""
import threading
import time
from . import mcp_tools
from .model_catalog import _collect_all_scan_models
from .notebooks import _list_notebooks
from .quick_index import compact_models
from .recipe_store import _list_recipes, get_recipes_dir
CACHE_SECONDS = 15
MAX_TEXT = 1500
MAX_THUMBNAIL = 60_000 # an inline thumbnail larger than this is left out; the cover asset or the default stands in
_cache = {"at": 0.0, "data": None}
_lock = threading.Lock()
def _join(texts, limit=MAX_TEXT):
text = "\n".join(str(item).strip() for item in texts or [] if str(item or "").strip())
return text if len(text) <= limit else text[:limit] + "…"
def _combos():
out = []
for note in _list_notebooks():
data = note.get("data") or {}
combo, loras = mcp_tools.combo_view(note)
out.append({"name": combo["name"], "model": combo["main_model"], "loras": loras[:12], "base": combo.get("base_model") or "",
"prompt": combo["prompt"], "structure": data.get("kind") == "nodes",
"cover": (data.get("mainModel") or {}).get("preview_url") or "" if isinstance(data.get("mainModel"), dict) else "",
"image": data.get("source_image") if isinstance(data.get("source_image"), dict) else None})
return out
def _workflows():
out = []
for recipe in _list_recipes(get_recipes_dir()):
data = recipe.get("data") or {}
params = data.get("params") if isinstance(data.get("params"), dict) else {}
thumbnail = data.get("thumbnail") if isinstance(data.get("thumbnail"), str) and len(data["thumbnail"]) <= MAX_THUMBNAIL else None
out.append({"name": data.get("name") or recipe["filename"], "filename": recipe["filename"], "tags": (data.get("tags") or [])[:8],
"notes": str(data.get("notes") or "")[:300], "partial": data.get("workflow_scope") == "partial",
"nodes": len(params.get("nodes") or []), "base": params.get("baseModel") or "",
"prompt": _join(params.get("promptPositive")), "negative": _join(params.get("promptNegative"), 600),
"cover_asset": (data.get("presentation") or {}).get("cover_asset_id"), "thumbnail": thumbnail,
"image": data.get("source_image") if isinstance(data.get("source_image"), dict) else None,
"saved": data.get("timestamp") or 0})
return out
def _models():
out = []
for model in compact_models(_collect_all_scan_models(1, 0).get("models") or []):
place = f"{model['subfolder']}/{model['filename']}" if model["subfolder"] else model["filename"]
out.append({"id": f"{model['type']}:{model['path_idx']}:{place}", "name": model["name"] or model["civitai_name"] or model["filename"],
"file": model["filename"], "type": model["type"], "base": model["base"], "words": model["words"],
"cover": model["preview_url"], "folder": model["subfolder"]})
return out
def prompt_entries(combos, workflows):
"""The prompts the picker lists: saved ones (id = name), then those of combos and workflows."""
entries = [{"id": prompt["name"], "name": prompt["name"], "source": "saved", "tags": prompt["tags"],
"positive": prompt["positive"], "negative": prompt["negative"]} for prompt in mcp_tools.saved_prompts()]
entries += [{"id": f"combo:{combo['name']}", "name": combo["name"], "source": "combo", "tags": [],
"positive": combo["prompt"], "negative": ""} for combo in combos if combo["prompt"]]
entries += [{"id": f"workflow:{flow['name']}", "name": flow["name"], "source": "workflow", "tags": flow["tags"],
"positive": flow["prompt"], "negative": flow["negative"]} for flow in workflows if flow["prompt"] or flow["negative"]]
return entries
def everything(fresh=False):
"""{combos, workflows, prompts, models}, read at most every CACHE_SECONDS."""
with _lock:
if fresh or _cache["data"] is None or time.monotonic() - _cache["at"] > CACHE_SECONDS:
combos, workflows = _combos(), _workflows()
_cache.update(at=time.monotonic(), data={"combos": combos, "workflows": workflows,
"prompts": prompt_entries(combos, workflows), "models": _models()})
return _cache["data"]
+83 -29
View File
@@ -15,6 +15,7 @@ import time
import urllib.error
import urllib.request
from . import ai_gates, mcp_fill, model_advice, run_refusal
from .lora_info import trains_text_encoder
from .mcp_bridge import PageRefused, PageUnavailable, ask_page
from .mcp_tools import MAX_LIMIT, ToolError, _resolve_model, _schema
@@ -34,12 +35,6 @@ def _rel(base_dir, file_path):
return os.path.relpath(file_path, base_dir).replace(os.sep, "/")
def _model_ref(model_id):
"""{type, path} of a model, as a node's drop-down names it (relative to its folder)."""
folder_type, _idx, base_dir, file_path = _resolve_model(model_id)
return {"type": folder_type, "path": _rel(base_dir, file_path)}
def _local(ctx, method, path, body=None, timeout=20):
"""(status, JSON) of one request to this ComfyUI server's own routes."""
data = json.dumps(body).encode("utf-8") if body is not None else None
@@ -65,23 +60,42 @@ def describe_canvas(ctx):
return _page("describe_canvas")
def set_prompt(ctx, text, role="positive", node_id=None, mode="replace"):
def set_prompt(ctx, prompt, role="positive", node_id=None, mode="replace"):
if role not in ("positive", "negative") or mode not in ("replace", "append"):
raise ToolError("role is positive or negative; mode is replace or append.")
return _page("set_prompt", {"text": str(text), "role": role, "node_id": node_id, "mode": mode})
return _page("set_prompt", {"text": str(prompt), "role": role, "node_id": node_id, "mode": mode})
def _base_note(result, folder_type, file_path):
"""The page's answer without the model line it reports, and base_model_note when a LoRA on that
line does not suit its main model (model_advice.py)."""
if not isinstance(result, dict):
return result
line = result.pop("line", None) or {}
if folder_type == "loras":
note = model_advice.lora_note(file_path, line)
else:
notes = (model_advice.lora_note(path, {"model": line.get("model")}) for path in model_advice.lora_paths(line.get("loras")))
note = next((item for item in notes if item), None)
if note:
result["base_model_note"] = note
return result
def set_model(ctx, node_id, model_id):
return _page("set_model", {"node_id": node_id, "model": _model_ref(model_id)})
folder_type, _idx, base_dir, file_path = _resolve_model(model_id)
result = _page("set_model", {"node_id": node_id, "model": {"type": folder_type, "path": _rel(base_dir, file_path)}})
return _base_note(result, folder_type, file_path)
def add_lora(ctx, model_id, after_node_id=None, strength_model=1.0, strength_clip=1.0):
folder_type, _idx, base_dir, file_path = _resolve_model(model_id)
if folder_type != "loras":
raise ToolError("add_lora takes a model from the loras folder.")
return _page("add_lora", {"model": {"type": folder_type, "path": _rel(base_dir, file_path)},
"text_encoder": trains_text_encoder(file_path), "after_node_id": after_node_id,
"strength_model": float(strength_model), "strength_clip": float(strength_clip)})
result = _page("add_lora", {"model": {"type": folder_type, "path": _rel(base_dir, file_path)},
"text_encoder": trains_text_encoder(file_path), "after_node_id": after_node_id,
"strength_model": float(strength_model), "strength_clip": float(strength_clip)})
return _base_note(result, folder_type, file_path)
def place_combo(ctx, name):
@@ -121,9 +135,35 @@ def download_status(ctx):
"failed (error says why; starting it again continues it), cancelled."}
def run_workflow(ctx, batch=1):
def run_workflow(ctx, batch=1, wait_seconds=55, workflow=None, **slots):
from .ai_run_tracking import checked_wait
checked_wait(wait_seconds) # Reject a wait that is no number before anything can be submitted.
batch = max(1, min(8, int(batch)))
return _page("run_workflow", {"batch": batch})
filled = mcp_fill.fill(ctx, workflow, **slots) # a saved workflow and its blanks first (mcp_fill.py)
try:
answer = ask_page("run_workflow", {"batch": batch}, required_capability="prompt_ids")
except (PageUnavailable, PageRefused) as error:
raise ToolError(str(error))
ids = answer.get("prompt_ids") if isinstance(answer, dict) else None
page = answer if isinstance(answer, dict) else {}
if (not isinstance(ids, list) or not ids or len(ids) > batch
or any(not isinstance(pid, str) or not pid or len(pid) > 128 for pid in ids) or len(set(ids)) != len(ids)):
raise ToolError((run_refusal.say(answer.get("refusal")) or answer.get("submission_error") if isinstance(answer, dict) else None)
or "No valid submission receipts. Check ComfyUI's queue before retrying; do not repeat automatically.")
answer = {"prompt_ids": ids, "queued": len(ids), "requested": batch,
"note": ("If this answer does not say it is done, the user tells you when it is: say it is running and end "
"your turn; run_status then brings its pictures." if ai_gates.run_wait() == "call" else
"Follow these runs with run_status (wait_seconds up to 55).")
+ " Do not repeat the batch; accepted prompts are already queued."}
if filled:
answer["filled"] = filled
# Some prompts may have been accepted before validation or a transport failure stopped the batch.
if len(ids) < batch:
answer["submission_error"] = f"Only {len(ids)} of {batch} prompts were accepted."
told = run_refusal.say(page.get("refusal"))
if told:
answer["submission_error"] += f" {told}"
return answer
def open_in_anomalous(ctx, model_id):
@@ -215,7 +255,7 @@ def speak(ctx, character, text, seed=None, language="auto"):
# ---------------------------------------------------------------- registry
_NODE = {"type": "integer", "description": "A node id from describe_canvas."}
_NODE = {"type": "integer", "description": "A node id."}
_MODEL = {"type": "string", "description": "model_id from search_models."}
@@ -232,22 +272,23 @@ def _action(name, title, description, run, properties=None, required=(), destruc
ACTIONS = dict([
_action("describe_canvas", "Read the canvas",
"The workflow open in ComfyUI now: each node's id, type, title, settings, which prompt box is "
"positive or negative, and where its inputs come from. Call it before changing the canvas.",
"positive or negative, and where its inputs come from. Call it before changing settings; "
"get_situation gives the canvas in brief (a line a node, what is still unwired and what fits it).",
describe_canvas, read_only=True),
_action("set_prompt", "Write a prompt",
"Writes text into the workflow's positive or negative prompt box (or the one on node_id). "
"mode=append adds to what is there. One Ctrl+Z step.",
set_prompt, {"text": {"type": "string"}, "role": {"type": "string", "enum": ["positive", "negative"], "default": "positive"},
set_prompt, {"prompt": {"type": "string"}, "role": {"type": "string", "enum": ["positive", "negative"], "default": "positive"},
"node_id": _NODE, "mode": {"type": "string", "enum": ["replace", "append"], "default": "replace"}},
["text"], destructive=True),
["prompt"], destructive=True),
_action("set_model", "Swap a node's model",
"Puts a model into a loader node (checkpoint, LoRA, VAE…) on the canvas, replacing the one it had.",
"Puts a model into a loader node (checkpoint, LoRA, VAE…) on the canvas, replacing the one it had. "
"base_model_note in the answer: a LoRA and the main model are for different base models.",
set_model, {"node_id": _NODE, "model_id": _MODEL}, ["node_id", "model_id"], destructive=True),
_action("add_lora", "Add a LoRA",
"Inserts a LoRA loader after node after_node_id (default: the workflow's main model loader, past "
"the LoRAs already on it) and moves everything that node fed behind it. LoRAs that train the text "
"encoder also take the CLIP line (from the same node, or a UNet workflow's CLIP loader); others "
"patch the model only.",
"Inserts a LoRA loader after after_node_id (default: the main model loader, past its LoRAs) and moves "
"what that node fed behind it; the CLIP line too when the LoRA trains the text encoder. base_model_note in "
"the answer: the LoRA is for another base model than the main model.",
add_lora, {"model_id": _MODEL, "after_node_id": _NODE,
"strength_model": {"type": "number", "default": 1.0}, "strength_clip": {"type": "number", "default": 1.0}},
["model_id"]),
@@ -257,12 +298,11 @@ ACTIONS = dict([
"changes; ports to nodes outside the combo (sampler…) stay open.",
place_combo, {"name": {"type": "string", "description": "Combo name from list_combos."}}, ["name"]),
_action("check_workflow_models", "Check missing models",
"Which models the open workflow needs that this computer does not have under that name, and which "
"of them Anomalous can put back (same file found by hash, or a likely candidate).",
"Models the open workflow names that this computer lacks, and which Anomalous can put back (the same "
"file by hash, or a likely one).",
check_workflow_models, read_only=True),
_action("fix_workflow_models", "Fix missing models",
"Puts back every missing model that Anomalous found with certainty (same file by hash). "
"Likely candidates are left for the user. One Ctrl+Z step.",
"Puts back every missing model found by hash; likely ones are left to the user. One Ctrl+Z step.",
fix_workflow_models, destructive=True),
_action("download_missing_models", "Download missing models",
"Downloads the models the open workflow is missing whose source is certain: Civitai found by the "
@@ -282,8 +322,22 @@ ACTIONS = dict([
_action("download_status", "Downloads", "How the downloads of this ComfyUI run are going.", download_status,
read_only=True),
_action("run_workflow", "Run the workflow",
"Queues the open workflow in ComfyUI, like pressing Run. Uses the GPU; results appear in the output folder.",
run_workflow, {"batch": {"type": "integer", "minimum": 1, "maximum": 8, "default": 1}}),
"Queues the workflow (uses the GPU). In the same call: workflow (a saved one) opens in the task's tab "
"once and is reused; prompt, negative, seed (every sampler), width/height (the start latent), image "
"(its LoadImage) and values (any setting by name: {\"steps\": 30}, or {\"12\": {\"denoise\": 0.5}} when "
"several nodes have it) fill it in; the rest stays. Without workflow they fill the open canvas. Then "
"run_status, never the output folder.",
run_workflow, {"batch": {"type": "integer", "minimum": 1, "maximum": 8, "default": 1},
"wait_seconds": {"type": "number", "minimum": 0, "maximum": 55, "default": 55,
"description": "Seconds to wait for it to end."},
"workflow": {"type": "string", "description": "A name from list_workflows."},
"prompt": {"type": "string"}, "negative": {"type": "string"},
"seed": {"type": "integer", "minimum": 0},
"width": {"type": "integer", "minimum": 16, "maximum": 16384},
"height": {"type": "integer", "minimum": 16, "maximum": 16384},
"image": {"type": "string", "description": "subfolder/name.png, output or input folder."},
"image_node": {"type": "integer", "description": "Which LoadImage, when there are several."},
"values": {"type": "object", "description": "Other settings by name, any node's."}}),
_action("open_in_anomalous", "Show a model",
"Opens Anomalous Model Browser on this model's page, so the user sees its cover and details.",
open_in_anomalous, {"model_id": _MODEL}, ["model_id"]),
+213
View File
@@ -0,0 +1,213 @@
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Anomalous</title>
<!-- Anomalous's card in the AI app's conversation (api/mcp_apps.py). Everything the AI or the task
says is drawn with textContent; pictures are inline thumbnails; nothing is loaded from elsewhere. -->
<style>
:root { --bg: #ffffff; --soft: #f4f4f6; --line: #e2e2e8; --text: #1d1d22; --dim: #6b6b75; --accent: #7c5cdb; --ok: #2f9e6e; --warn: #c98a04; --bad: #d64545; }
:root.dark { --bg: #1d1d22; --soft: #26262d; --line: #36363f; --text: #ececf1; --dim: #9a9aa6; --accent: #b39eef; --ok: #52b98d; --warn: #fbbf24; --bad: #f87171; }
* { box-sizing: border-box; }
body { margin: 0; padding: 14px 16px; background: var(--bg); color: var(--text); font: 13.5px/1.55 system-ui, -apple-system, "Segoe UI", "Microsoft YaHei", sans-serif; }
.head { display: flex; align-items: center; gap: 8px; margin-bottom: 10px; }
.brand { color: var(--accent); font-weight: 600; font-size: 12px; }
.title { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; font-weight: 600; }
.state { padding: 1px 8px; border-radius: 999px; background: var(--soft); color: var(--dim); font-size: 11.5px; }
.state.live { color: var(--ok); }
.steps { display: flex; flex-direction: column; gap: 4px; margin: 0 0 10px; padding: 0; list-style: none; }
.steps li { display: flex; gap: 8px; color: var(--dim); font-size: 12.5px; }
.steps li::before { content: "•"; color: var(--ok); }
.steps li.running::before { color: var(--accent); }
.steps li.asking::before, .steps li.allowed::before { color: var(--warn); }
.steps li.bad::before { color: var(--bad); }
.steps li span { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.pictures { display: grid; grid-template-columns: repeat(auto-fill, minmax(96px, 1fr)); gap: 6px; margin-bottom: 10px; }
.pictures img { width: 100%; aspect-ratio: 1; object-fit: cover; border-radius: 8px; background: var(--soft); }
.box { margin-bottom: 10px; padding: 10px 12px; border: 1px solid var(--line); border-radius: 10px; background: var(--soft); }
.box.warn { border-color: var(--warn); }
.question { margin-bottom: 8px; font-weight: 600; }
.options { display: flex; flex-wrap: wrap; gap: 6px; }
button { padding: 6px 12px; border: 1px solid var(--line); border-radius: 8px; background: var(--bg); color: var(--text); font: inherit; font-size: 12.5px; cursor: pointer; }
button:hover:not(:disabled) { border-color: var(--accent); }
button:disabled { opacity: 0.5; cursor: default; }
button.primary { border-color: var(--accent); background: var(--accent); color: #fff; }
input { flex: 1; min-width: 0; padding: 6px 10px; border: 1px solid var(--line); border-radius: 8px; background: var(--bg); color: var(--text); font: inherit; }
.row { display: flex; gap: 6px; }
.foot { display: flex; align-items: center; justify-content: space-between; gap: 8px; color: var(--dim); font-size: 12px; }
.link { padding: 0; border: 0; background: none; color: var(--accent); }
.note { margin-bottom: 8px; color: var(--dim); font-size: 12px; }
</style>
</head>
<body>
<div id="card"><div class="note" id="loading"></div></div>
<script>
(() => {
let zh = /^zh/i.test(navigator.language || '');
const words = {
loading: ['Reading the task from Anomalous…', '正在从 AMB 读取任务…'],
live: ['Working', '进行中'], done: ['Done', '已完成'],
asking: ['The AI waits for your OK in Anomalous', 'AI 在等你在 AMB 里点头'],
askingHow: ['Allowing is only done in Anomalous\'s AI workbench, so the AI can never allow itself.', '为了安全,允许只能在 AMB 的 AI 工作台里点,AI 自己没法替你点。'],
openAmb: ['Open in Anomalous', '在 AMB 里看'], send: ['Send', '发送'], refresh: ['Refresh', '刷新'],
picked: ['I picked in the card: ', '我在卡片里选了:'], typed: ['My answer in the card: ', '我在卡片里回答:'],
youAnswered: ['You answered: ', '你回答了:'], none: ['No task to show yet.', '还没有任务可看。'],
};
const t = key => words[key][zh ? 1 : 0];
const card = document.getElementById('card');
document.getElementById('loading').textContent = t('loading');
// JSON-RPC with the AI app (MCP Apps): requests wait for their answer by id.
let nextId = 1;
const waiting = new Map();
const send = (message) => window.parent.postMessage({ jsonrpc: '2.0', ...message }, '*');
const request = (method, params) => new Promise((resolve, reject) => {
const id = nextId++;
waiting.set(id, { resolve, reject });
send({ id, method, params });
});
const notify = (method, params) => send({ method, params });
const resize = () => notify('ui/notifications/size-changed', { width: document.body.scrollWidth, height: document.body.scrollHeight });
let target = {}; // { panel } or { task }: what the card shows
let openUrl = '';
let polls = 0;
let timer = null;
function el(tag, className, text) {
const node = document.createElement(tag);
if (className) node.className = className;
if (text !== undefined) node.textContent = String(text);
return node;
}
async function callTool(name, args) {
const result = await request('tools/call', { name, arguments: args });
if (result?.isError) throw new Error(result.content?.[0]?.text || 'error');
return result?.structuredContent || {};
}
async function answer(question, choice, text, label) {
card.querySelectorAll('button, input').forEach((node) => { node.disabled = true; });
try {
await callTool('app_answer', { panel: question.panel, block: question.block, ...(choice ? { choice: [choice] } : { text }) });
// Also said in the conversation, so the AI goes on even when it is not waiting.
await request('ui/message', { role: 'user', content: [{ type: 'text', text: (choice ? t('picked') : t('typed')) + label }] }).catch(() => {});
} catch (error) {
card.append(el('div', 'note', error.message));
}
await load();
}
function questionBox(question) {
const box = el('div', 'box');
box.append(el('div', 'question', question.question));
if (question.type === 'input') {
const row = el('div', 'row');
const field = el('input');
const go = el('button', 'primary', t('send'));
go.onclick = () => field.value.trim() && answer(question, null, field.value.trim(), field.value.trim());
field.onkeydown = (event) => { if (event.key === 'Enter') go.click(); };
row.append(field, go);
box.append(row);
} else {
const options = el('div', 'options');
for (const option of question.options) {
const pick = el('button', '', option.label);
pick.onclick = () => answer(question, option.id, null, option.label);
options.append(pick);
}
box.append(options);
}
return box;
}
function draw(view) {
openUrl = view.open_url || openUrl;
const head = el('div', 'head');
head.append(el('span', 'brand', 'Anomalous'), el('span', 'title', view.title || ''),
el('span', `state${view.ended ? '' : ' live'}`, t(view.ended ? 'done' : 'live')));
const steps = el('ul', 'steps');
for (const step of view.steps || []) {
const line = el('li', `${step.state || ''}${step.ok === false ? ' bad' : ''}`);
line.append(el('span', '', step.text || step.tool));
steps.append(line);
}
const parts = [head, steps];
if (view.pictures?.length) {
const pictures = el('div', 'pictures');
for (const picture of view.pictures) {
const image = el('img');
image.alt = picture.name;
image.title = picture.name;
image.src = picture.src; // a data: thumbnail from Anomalous
image.onload = resize;
pictures.append(image);
}
parts.push(pictures);
}
for (const item of view.answered || []) parts.push(el('div', 'note', `${t('youAnswered')}${item.question ? `${item.question} ` : ''}${item.answer}`));
for (const question of view.questions || []) parts.push(questionBox(question));
if (view.asking?.length) {
const box = el('div', 'box warn');
box.append(el('div', 'question', t('asking')), el('div', 'note', t('askingHow')));
parts.push(box);
}
const foot = el('div', 'foot');
const open = el('button', 'link', t('openAmb'));
open.onclick = () => openUrl && request('ui/open-link', { url: openUrl }).catch(() => {});
const again = el('button', 'link', t('refresh'));
again.onclick = () => { polls = 0; load(); };
foot.append(open, again);
parts.push(foot);
card.replaceChildren(...parts);
resize();
// Follows the task for a few minutes while it goes on; Refresh starts again.
clearTimeout(timer);
if (!view.ended && polls++ < 36) timer = setTimeout(load, 5000);
}
async function load() {
try {
draw(await callTool('app_task_view', target));
} catch (_) {
card.replaceChildren(el('div', 'note', t('none')));
resize();
}
}
window.addEventListener('message', (event) => {
if (event.source !== window.parent) return;
const message = event.data;
if (!message || message.jsonrpc !== '2.0') return;
if (message.id !== undefined && waiting.has(message.id)) {
const { resolve, reject } = waiting.get(message.id);
waiting.delete(message.id);
if (message.error) reject(new Error(message.error.message));
else resolve(message.result);
return;
}
if (message.method === 'ui/notifications/tool-result') {
const data = message.params?.structuredContent || {};
target = data.panel ? { panel: String(data.panel) } : data.task ? { task: String(data.task) } : {};
polls = 0;
load();
} else if (message.method === 'ui/notifications/host-context-changed') {
document.documentElement.classList.toggle('dark', message.params?.theme === 'dark');
}
});
request('ui/initialize', { appInfo: { name: 'anomalous-task-card', version: '1.0.0' }, appCapabilities: {}, protocolVersion: '2026-01-26' })
.then((result) => {
const context = result?.hostContext || {};
if (context.locale) zh = /^zh/i.test(context.locale);
document.documentElement.classList.toggle('dark', context.theme === 'dark');
notify('ui/notifications/initialized', {});
resize();
})
.catch(() => {});
})();
</script>
</body>
</html>
+146
View File
@@ -0,0 +1,146 @@
"""Anomalous's card inside the AI app's conversation (the MCP Apps extension,
io.modelcontextprotocol/ui): AI apps that support it (Claude, ChatGPT, VS Code…) show the page
mcp_app/task_card.html under the results of the tools in CARD_TOOLS, in a sandboxed frame of
theirs. It is Anomalous's own page, never anything an AI wrote; the AI's data reaches it only as
text. The card reads the task through app_task_view and answers the AI's questions through
app_answer, two tools for the card only (visibility "app": apps that follow the extension keep
them from the AI; an AI that calls them anyway only reads its own task or answers its own
question). Their results go to the card, not the AI. The card never answers a gate:
an OK for what an AI may do is only given in Anomalous (ai_gates.py), so a tool the AI could
reach can never grant it. The pictures it shows are the gallery's thumbnails, inline (the frame
loads nothing from this server).
"""
import os
from . import ai_panels, ai_tasks
from .mcp_tools import ToolError, _image_content, _resolve_image, _schema
EXTENSION = "io.modelcontextprotocol/ui"
MIME = "text/html;profile=mcp-app"
CARD_URI = "ui://anomalous/task-card.html"
CARD_TOOLS = ("show_panel", "ask_user", "report", "run_workflow", "begin_task")
MAX_PICTURES = 6
MAX_STEPS = 8
HTML = os.path.join(os.path.dirname(__file__), "mcp_app", "task_card.html")
def resources():
return [{"uri": CARD_URI, "name": "task-card", "title": "Anomalous task card", "mimeType": MIME,
"description": "What the AI is doing in Anomalous: its steps, pictures and questions."}]
def read(uri):
"""resources/read for the card, or None for another address."""
if uri != CARD_URI:
return None
with open(HTML, encoding="utf-8") as source:
html = source.read()
return {"contents": [{"uri": CARD_URI, "mimeType": MIME, "text": html, "_meta": {"ui": {"prefersBorder": True}}}]}
def with_card(spec):
"""A tool's spec naming the card that shows its results (and the older flat key some apps read)."""
if spec["name"] not in CARD_TOOLS:
return spec
return {**spec, "_meta": {**spec.get("_meta", {}), "ui": {"resourceUri": CARD_URI}, "ui/resourceUri": CARD_URI}}
def _picture(result):
rel = "/".join(part for part in (result.get("subfolder"), result.get("filename")) if part)
try:
_, path = _resolve_image(rel)
except ToolError:
return None
item = _image_content(path)
return {"name": result["filename"], "src": f"data:{item['mimeType']};base64,{item['data']}"} if item else None
def _words(step):
"""A line for a step: its note or panel title, else what it was about (its summary without the tool)."""
summary = step.get("summary", "")
about = summary.split(": ", 1)[1] if summary.startswith(f"{step['tool']}: ") else summary
return ai_tasks._short(step.get("text") or (step.get("panel") or {}).get("title") or about or step["tool"], 120)
def _answered(step):
"""What the user already answered on a panel step: [{question, answer}]."""
blocks = {block["id"]: block for block in step["panel"]["blocks"]}
return [{"question": blocks.get(block, {}).get("question", ""), "answer": "(left to the AI)" if given.get("skipped") else "、".join(given.get("labels") or [given.get("text", "")])}
for block, given in step.get("answers", {}).items()]
def _questions(step):
"""The open questions of a panel step, as the card draws them (choices and picks as buttons, input as a line)."""
panel = step["panel"]
out = []
for block in panel["blocks"]:
if block["type"] not in ai_panels.INTERACTIVE or block["id"] in step.get("answers", {}) or panel.get("closed"):
continue
options = block.get("options") or [{"id": item["id"], "label": item.get("label") or item["id"]} for item in block.get("items", [])]
out.append({"panel": panel["id"], "block": block["id"], "type": block["type"], "question": block.get("question", ""),
"options": [{"id": option["id"], "label": option.get("label") or option["id"]} for option in options]})
return out
def task_view(ctx, panel=None, task=None):
"""The card's view of a task: the panel's task, task `task`, or the calling app's active one."""
with ai_tasks._lock:
found = ai_panels._find(str(panel))[0] if panel else None
task_id = found["id"] if found else task or ai_tasks.active_task_id(ctx.get("client") or "AI", ctx.get("session") or "")
record = ai_tasks.get_task(task_id) if task_id else None
if not record:
raise ToolError("No task to show.")
steps = [step for step in record["steps"] if step.get("kind") != "look"]
pictures = [result for step in record["steps"] for result in step.get("results") or [] if result.get("type") == "image"]
seen, shown = set(), []
for result in reversed(pictures):
key = (result.get("subfolder"), result.get("filename"))
if key in seen or len(shown) >= MAX_PICTURES:
continue
seen.add(key)
picture = _picture(result)
if picture:
shown.append(picture)
return {
"task": record["id"], "title": record.get("title") or "", "ended": bool(record.get("ended")),
"steps": [{"tool": step["tool"], "kind": step["kind"], "state": step.get("state", "done"), "ok": step.get("ok", True),
"stage": step.get("stage"), "text": _words(step)}
for step in steps[-MAX_STEPS:]],
"pictures": list(reversed(shown)),
"questions": [question for step in record["steps"] if step.get("kind") == "panel" for question in _questions(step)],
"answered": [item for step in record["steps"] if step.get("kind") == "panel" for item in _answered(step)][-3:],
"asking": [step.get("gate", {}).get("gate") for step in record["steps"] if step.get("state") == "asking"],
"open_url": f"{ctx.get('base_url', '')}/",
}
def answer(ctx, panel, block, choice=None, text=None):
"""The user's answer from the card to one of the AI's questions (never a gate)."""
task, step = None, None
with ai_tasks._lock:
task, step = ai_panels._find(str(panel))
if not step:
raise ToolError("No such panel.")
try:
ai_panels.answer(task["id"], step["id"], str(block), choice, text)
except ai_panels.PanelError as error:
raise ToolError(str(error))
return {"answered": True}
def _app_tool(name, title, description, run, properties, required=()):
return name, {"run": run, "context": True, "spec": {
"name": name, "title": title, "description": description, "inputSchema": _schema(properties, required),
"annotations": {"title": title, "readOnlyHint": name == "app_task_view"},
"_meta": {"ui": {"resourceUri": CARD_URI, "visibility": ["app"]}},
}}
APP_TOOLS = dict([
_app_tool("app_task_view", "Card: the task", "Only for Anomalous's own card in the conversation; never call it.",
task_view, {"panel": {"type": "string"}, "task": {"type": "string"}}),
_app_tool("app_answer", "Card: answer", "Only for Anomalous's own card, when the user answers there; never call it.",
answer, {"panel": {"type": "string"}, "block": {"type": "string"}, "choice": {"type": "array", "items": {"type": "string"}},
"text": {"type": "string"}}, ["panel", "block"]),
])
+37
View File
@@ -0,0 +1,37 @@
"""Normalize an unambiguous argument name before approval, execution and task recording.
Only a known other name counts (ALIASES, names light models were seen to write in the tool exams,
docs/architecture/tool-exam.md): a value of the right type under any other name may mean something
else (set_prompt {style: "anime"} is not a prompt), and that call is answered with what the tool takes.
"""
ALIASES = { # needed argument -> other names that mean it
"workflow": {"name", "workflow_name"},
"image": {"file", "filename", "image_name"},
"prompt": {"text", "positive"},
"model_id": {"model", "id"},
"node_id": {"node", "id"},
}
JSON_TYPES = {"string": (str,), "integer": (int,), "number": (int, float), "boolean": (bool,), "array": (list,), "object": (dict,)}
def prepare_arguments(spec, arguments):
"""Return (arguments, (wrong, right) or None), without changing the caller's dictionary."""
unknown = set(arguments) - set(spec["properties"])
missing = [key for key in spec.get("required", []) if key not in arguments]
if len(unknown) != 1 or len(missing) != 1:
return arguments, None
wrong, right = next(iter(unknown)), missing[0]
if wrong not in ALIASES.get(right, ()):
return arguments, None
kinds = JSON_TYPES.get(spec["properties"][right].get("type"))
value = arguments[wrong]
if not kinds or not isinstance(value, kinds) or (isinstance(value, bool) and bool not in kinds):
return arguments, None
corrected = {key: value for key, value in arguments.items() if key != wrong}
corrected[right] = value
return corrected, (wrong, right)
def renaming_note(name, renamed):
return f"Took {renamed[0]} as {renamed[1]}: {name} calls it {renamed[1]}."
+17 -5
View File
@@ -6,7 +6,8 @@ to every open ComfyUI page and waits. One page claims it (the first claim wins,
pages claim first: web/modules/mcp_bridge.js), does the action with Anomalous's own canvas
code and posts the outcome back; `claim` and `resolve` take those posts
(routes in mcp_server.py). With no page open, or none answering in time, the tool fails
with words the AI can pass on.
with words the AI can pass on. Actions on the canvas carry the task's own tab
(ai_task_canvas.py), and the tab the page made or opened for it is kept.
"""
import threading
@@ -32,13 +33,18 @@ def _server():
return PromptServer.instance
def ask_page(action, args=None, timeout=TIMEOUT_SECONDS):
def ask_page(action, args=None, timeout=TIMEOUT_SECONDS, required_capability=None):
"""Runs `action` in the open ComfyUI page; its result value. Call from a worker thread."""
server = _server()
if not getattr(server, "sockets", None):
raise PageUnavailable("ComfyUI is not open in a browser on this computer. Open it, then try again.")
from . import ai_task_canvas
task_canvas = ai_task_canvas.spec(action)
if task_canvas:
args = {**(args or {}), "_task_canvas": task_canvas}
request_id = uuid.uuid4().hex
entry = {"event": threading.Event(), "claimed": False, "outcome": None}
entry = {"event": threading.Event(), "claimed": False, "outcome": None, "required": required_capability}
with _lock:
if len(_waiting) >= MAX_WAITING:
raise PageUnavailable("Too many canvas actions are waiting; try again in a moment.")
@@ -52,16 +58,22 @@ def ask_page(action, args=None, timeout=TIMEOUT_SECONDS):
with _lock:
_waiting.pop(request_id, None)
outcome = entry["outcome"] or {}
if task_canvas and outcome.get("tab"):
ai_task_canvas.remember(outcome["tab"])
if not outcome.get("ok"):
raise PageRefused(str(outcome.get("error") or "The page could not do that."))
return outcome.get("value")
def claim(request_id):
def claim(request_id, capabilities=None):
"""True for the first page that asks to do request `request_id`."""
with _lock:
entry = _waiting.get(request_id)
if entry is None or entry["claimed"]:
if entry is None or entry["claimed"] or entry["outcome"] is not None:
return False
if entry.get("required") and (not isinstance(capabilities, list) or entry["required"] not in capabilities):
entry["outcome"] = {"ok": False, "error": "Refresh the ComfyUI browser page before running from AI. Nothing was queued."}
entry["event"].set()
return False
entry["claimed"] = True
return True
+314
View File
@@ -0,0 +1,314 @@
"""The MCP tools that build a workflow on the canvas the user sees: add_node, connect_nodes,
set_values, remove_nodes and new_workflow (a new tab, blank or built from a workflow in
ComfyUI's API form) run in the open page like mcp_actions.py's canvas tools
(web/modules/mcp_canvas_edit.js: each one Ctrl+Z step, in the activity log as the AI's, so the
AI workbench shows what it changed), and find_nodes reads the node types this ComfyUI has
from its own /object_info. Arguments are checked here for shape and size; the page checks
them against the nodes themselves before anything changes."""
import json
import threading
import time
from . import ai_tasks, ai_toolbox, mcp_fill
from .mcp_actions import _action, _local, _page
from .mcp_tools import ToolError
MAX_FOUND = 20
MAX_SEARCH = 5
MAX_CHOICES_NAMED = 60
MAX_REMOVE = 30
MAX_WORKFLOW_NODES = 200
MAX_WORKFLOW_BYTES = 1_000_000
INFO_SECONDS = 120
_info = {"at": 0.0, "data": None}
_info_lock = threading.Lock()
def object_info(ctx):
"""ComfyUI's node definitions, as its own page reads them (kept a little while)."""
with _info_lock:
if _info["data"] is None or time.monotonic() - _info["at"] > INFO_SECONDS:
status, data = _local(ctx, "GET", "/object_info", timeout=60)
if status != 200 or not isinstance(data, dict):
raise ToolError(f"ComfyUI's node list is unavailable ({status}).")
_info.update(at=time.monotonic(), data=data)
return _info["data"]
def forget_object_info():
"""After a model moved: the next look reads ComfyUI's lists anew."""
with _info_lock:
_info["data"] = None
def _input(name, conf, required, choices):
kind, options = (conf + [None, None])[:2] if isinstance(conf, list) else (conf, None)
options = options if isinstance(options, dict) else {}
out = {"name": name, "required": required}
listed = kind if isinstance(kind, list) else options.get("options") if kind == "COMBO" else None
if isinstance(listed, list):
out["type"] = "choice"
out["choices"] = [str(item) for item in listed[:choices]]
if len(listed) > choices:
out["choices_count"] = len(listed)
else:
out["type"] = str(kind)
for key in ("default", "min", "max", "multiline"):
if key in options and isinstance(options[key], (str, int, float, bool)):
out[key] = options[key] if not isinstance(options[key], str) else options[key][:200]
if options.get("forceInput"):
out["socket_only"] = True
return out
def _picked(ctx):
"""What the user picked for this task (ai_toolbox.py): its node types and packs come first in every search."""
return ai_toolbox.effective(ai_tasks.get_task(ctx["task"])) if ctx.get("task") else None
def _allowed(ctx, types):
"""Refuses node types the user's "only" picks leave out, before the page is asked."""
if not ctx.get("task"):
return
info = object_info(ctx)
refused = ai_toolbox.refused(ctx["task"], types, info)
if refused:
box = ai_toolbox.effective(ai_tasks.get_task(ctx["task"])) or {}
allowed = [*(box.get("nodes") or []), *(f"every node of the pack {pack}" for pack in box.get("packs") or [])]
raise ToolError(f"For this task the user allowed only these node types (and ComfyUI's built-in ones): "
f"{', '.join(allowed) or 'none'}. Not allowed: {', '.join(refused)}. "
"If you need one, ask the user in the AI workbench (show_panel) instead.")
def _node(type_name, info, choices):
inputs = []
for section, required in (("required", True), ("optional", False)):
for name, conf in ((info.get("input") or {}).get(section) or {}).items():
inputs.append(_input(name, conf, required, choices))
names = info.get("output_name") or info.get("output") or []
outputs = [{"slot": slot, "name": str(name), "type": str(kind)}
for slot, (name, kind) in enumerate(zip(names, info.get("output") or []))]
source = ai_toolbox.source_of(info)
return {"type": type_name, "display_name": info.get("display_name") or type_name, "category": info.get("category") or "",
"source": source, "description": str(info.get("description") or "")[:300], "inputs": inputs, "outputs": outputs,
**({"paid": True} if source == "api" else {}), **({"deprecated": True} if info.get("deprecated") else {})}
def find_nodes(ctx, query="", types=None):
info = object_info(ctx)
if types:
missing = [name for name in types if name not in info]
found = [_node(name, info[name], MAX_CHOICES_NAMED) for name in types if name in info][:MAX_FOUND]
return {"nodes": found, **({"not_found": missing} if missing else {})}
words = str(query or "").lower().split()
if not words:
raise ToolError("Give query (words in a node's name, category or description) or types.")
picked = _picked(ctx)
scored = []
for name, item in info.items():
haystack = " ".join(str(part or "") for part in (name, item.get("display_name"), item.get("category"), item.get("description"))).lower()
if all(word in haystack for word in words):
label = f"{name} {item.get('display_name') or ''}".lower()
scored.append((not ai_toolbox.chosen(picked, name, item), 0 if name.lower() == " ".join(words) else 1 if all(w in label for w in words) else 2,
bool(item.get("deprecated")), ai_toolbox.source_of(item) == "api", len(name), name))
scored.sort()
# Searching identifies candidates; don't repeat hundreds of model choices/settings for each.
nodes = []
for *_rest, name in scored[:MAX_SEARCH]:
detail = _node(name, info[name], 0)
nodes.append({"type": name, "display_name": str(detail["display_name"])[:80],
"source": str(detail["source"])[:80],
"description": detail["description"][:96],
"inputs": [f"{i['name'][:48]}:{i['type'][:48]}" for i in detail["inputs"][:8]],
"outputs": [f"{i['name'][:48]}:{i['type'][:48]}" for i in detail["outputs"][:8]],
**{k: detail[k] for k in ("paid", "deprecated") if k in detail},
**({"more_inputs": len(detail["inputs"]) - 8} if len(detail["inputs"]) > 8 else {})})
for node in nodes:
if ai_toolbox.chosen(picked, node["type"], info[node["type"]]):
node["picked_by_user"] = True
while len(nodes) > 1 and len(json.dumps(nodes, ensure_ascii=False)) > 3000:
nodes.pop()
return {"nodes": nodes, "total_matches": len(scored),
"note": "Brief candidates; types=[chosen type] gives settings/choices and output slots when needed. "
"Narrow query for other matches. picked_by_user: preferred; paid: spends credits, use only when asked."}
def _id(value, what):
if isinstance(value, bool) or not isinstance(value, (int, str)) or not str(value).strip() or len(str(value)) > 40:
raise ToolError(f"{what} is a node id from describe_canvas or a tool's answer.")
return value
def _type(value):
if not isinstance(value, str) or not value.strip() or len(value) > 200:
raise ToolError("type is a node type, as find_nodes gives it (e.g. KSampler).")
return value.strip()
def _next(answer):
"""The checklist's next step after a change (the page answers with what is still unwired)."""
if isinstance(answer, dict):
answer["next"] = ("Connect the new branch to an output; it does not reach one yet."
if answer.get("reaches_output") is False else
"Wire unwired inputs with connect_nodes; unique types need only the two node IDs."
if answer.get("unwired") else "No required input is unwired; run_workflow validates and runs the graph.")
return answer
def _ids(value, label):
if value is None:
return []
items = value if isinstance(value, list) else [value]
if not 1 <= len(items) <= 8:
raise ToolError(f"{label} names 1 to 8 node IDs.")
return [_id(item, label) for item in items]
def add_node(ctx, type, values=None, title="", near_node_id=None, insert_after=None, insert_type=None,
wire_from=None, wire_to=None, replace=False):
if not isinstance(replace, bool):
raise ToolError("replace must be true or false.")
if insert_type is not None and (insert_after is None or not isinstance(insert_type, str) or not 0 < len(insert_type) <= 60):
raise ToolError("insert_type names a through type (e.g. IMAGE) and needs insert_after.")
wiring = {"wire_from": _ids(wire_from, "wire_from"), "wire_to": _ids(wire_to, "wire_to"), "replace": replace}
if insert_after is not None:
wiring.update(insert_after=_id(insert_after, "insert_after"), insert_type=insert_type)
_allowed(ctx, [_type(type)])
return _next(_page("add_node", {"type": _type(type), "values": mcp_fill.flat_values(values), "title": str(title or "")[:120],
"near_node_id": None if near_node_id is None else _id(near_node_id, "near_node_id"), **wiring}))
def _api_nodes(nodes, canvas_refs=False):
"""A workflow (or group) in ComfyUI's API form, checked for shape and size; the page checks the rest.
With canvas_refs a link may name a canvas node as "#12", an output by slot or name, and a node wire_to."""
if not isinstance(nodes, dict) or not 0 < len(nodes) <= MAX_WORKFLOW_NODES:
raise ToolError(f"nodes is a workflow in ComfyUI's API form: {{id: {{class_type, inputs}}}}, up to {MAX_WORKFLOW_NODES} nodes.")
if len(json.dumps(nodes, ensure_ascii=False)) > MAX_WORKFLOW_BYTES:
raise ToolError("That workflow is too large.")
for key, spec in nodes.items():
if (len(key) > 40 or not isinstance(spec, dict) or not isinstance(spec.get("class_type"), str)
or not isinstance(spec.get("inputs", {}), dict)):
raise ToolError(f"Node {key[:40]}: each node is {{class_type, inputs: {{name: value or [node id, output slot]}}}}.")
_type(spec["class_type"])
mcp_fill.flat_values({name: value for name, value in spec.get("inputs", {}).items() if not isinstance(value, list)})
if not canvas_refs:
continue
if key.startswith("#"):
raise ToolError(f"Node {key[:40]}: a key of the group cannot start with #, which names canvas nodes.")
for name, value in spec.get("inputs", {}).items():
if isinstance(value, list) and (len(value) != 2 or not isinstance(value[0], (str, int)) or isinstance(value[1], bool)
or not isinstance(value[1], (str, int)) or len(str(value[1])) > 100):
raise ToolError(f"Node {key[:40]}, input {name[:100]}: a link is [a key of this group or \"#<canvas node id>\", output slot or name].")
if isinstance(value, list) and isinstance(value[1], str) and value[1].strip().isdigit():
value[1] = int(value[1])
if spec.get("wire_to") is not None:
spec["wire_to"] = _ids(spec["wire_to"], f"{key[:40]}.wire_to")
return nodes
def add_nodes(ctx, nodes, near_node_id=None, replace=False):
if not isinstance(replace, bool):
raise ToolError("replace must be true or false.")
_api_nodes(nodes, canvas_refs=True)
_allowed(ctx, [spec["class_type"] for spec in nodes.values()])
return _next(_page("add_nodes", {"nodes": nodes, "replace": replace,
"near_node_id": None if near_node_id is None else _id(near_node_id, "near_node_id")}, timeout=60))
def connect_nodes(ctx, from_node_id, to_node_id, input=None, output=None, replace=None):
if output is not None and (isinstance(output, bool) or not isinstance(output, (int, str)) or len(str(output)) > 100):
raise ToolError("output is the output's slot number or its name.")
output = int(output) if isinstance(output, str) and output.strip().isdigit() else output
if input is not None and (not isinstance(input, str) or not input or len(input) > 100):
raise ToolError("input is the input's name, as add_node, find_nodes or describe_canvas give it.")
if replace is not None and not isinstance(replace, bool):
raise ToolError("replace must be true or false.")
# Preserve the old explicit-input form's default output and replacement behaviour.
if input is not None and output is None:
output = 0
return _next(_page("connect_nodes", {"from_node_id": _id(from_node_id, "from_node_id"), "output": output,
"to_node_id": _id(to_node_id, "to_node_id"), "input": input, "replace": replace}))
def set_values(ctx, node_id=None, values=None):
if node_id is None:
return _page("set_values", {"node_id": None, "values": mcp_fill.node_values(values)})
if not values:
raise ToolError("values is an object {setting name: value}.")
return _page("set_values", {"node_id": _id(node_id, "node_id"), "values": mcp_fill.flat_values(values)})
def remove_nodes(ctx, node_ids):
if not isinstance(node_ids, list) or not 0 < len(node_ids) <= MAX_REMOVE:
raise ToolError(f"node_ids lists 1 to {MAX_REMOVE} node ids.")
return _next(_page("remove_nodes", {"node_ids": [_id(item, "node_ids") for item in node_ids]}))
def new_workflow(ctx, name="", nodes=None):
if nodes is not None:
_api_nodes(nodes)
_allowed(ctx, [spec["class_type"] for spec in nodes.values()])
return _page("new_workflow", {"name": str(name or "")[:60], "nodes": nodes}, timeout=60)
def open_workflow(ctx, workflow):
filename, name = mcp_fill.saved_workflow(workflow)
return _page("open_workflow", {"filename": filename, "name": name}, timeout=60)
_NODE_ID = {"type": "integer", "description": "A node id."}
_VALUES = {"type": "object", "description": "{setting: value}; a choice exactly as listed."}
_NEIGHBOURS = {"oneOf": [_NODE_ID, {"type": "array", "items": _NODE_ID, "minItems": 1, "maxItems": 8}]}
CANVAS_TOOLS = dict([
_action("find_nodes", "Look up node types",
"Node types, built in and custom. query: up to 5 brief candidates; types=[...]: their inputs, "
"choices, ranges and outputs.",
find_nodes, {"query": {"type": "string"}, "types": {"type": "array", "items": {"type": "string"}, "maxItems": MAX_FOUND}},
read_only=True),
_action("add_node", "Add a node",
"Adds a node, wired in the same call: wire_from feeds it, wire_to takes its output (an input already "
"fed is spliced, its old source now feeding the new node: a ControlNet or LoRA before a KSampler); "
"insert_after=ID splices it behind a node. Same type or same socket name wires itself; anything "
"ambiguous changes nothing and lists the choices; replace:true replaces what it cannot splice. One "
"Ctrl+Z step. Several nodes wired to each other (an inpaint or upscale branch): add_nodes, in one call.",
add_node, {"type": {"type": "string", "description": "A node type from find_nodes, e.g. KSampler."},
"values": _VALUES, "title": {"type": "string"}, "near_node_id": _NODE_ID,
"insert_after": _NODE_ID, "insert_type": {"type": "string", "description": "Through type, only if asked (e.g. IMAGE)."},
"wire_from": _NEIGHBOURS, "wire_to": _NEIGHBOURS, "replace": {"type": "boolean"}}, ["type"]),
_action("add_nodes", "Add a group of nodes",
"Adds a group of nodes in one call, as one block, in ComfyUI's API form: {\"a\": {\"class_type\": "
"\"VAEEncode\", \"inputs\": {\"pixels\": [\"#12\", 0], \"vae\": [\"#4\", \"VAE\"]}}, \"b\": {..., \"inputs\": "
"{\"samples\": [\"a\", 0]}, \"wire_to\": 15}}. A link names a key of the group or a canvas node \"#id\", and "
"an output slot or name; wire_to as in add_node. Checked first; one Ctrl+Z step; the answer maps keys "
"to node ids.",
add_nodes, {"nodes": {"type": "object"}, "near_node_id": _NODE_ID, "replace": {"type": "boolean"}}, ["nodes"]),
_action("connect_nodes", "Wire two nodes",
"Wires two nodes: every unique matching type, and same-type sockets of the same name (positive to "
"positive). Anything ambiguous (a lone CONDITIONING into positive or negative, wildcards) changes "
"nothing and lists the choices; then give output and input. replace:true lets it replace wires. One "
"Ctrl+Z step.",
connect_nodes, {"from_node_id": _NODE_ID, "output": {"type": "string", "description": "Optional output slot number or name."},
"to_node_id": _NODE_ID, "input": {"type": "string"}, "replace": {"type": "boolean"}},
["from_node_id", "to_node_id"]),
_action("set_values", "Change a node's settings",
"Changes settings of any node, a third party's too, checked against what each takes: {setting: value} "
"goes to the one node that has it; name the node when several do: {\"12\": {\"denoise\": 0.5}}; or "
"node_id with its settings. Prompts: set_prompt; models: set_model. One Ctrl+Z step.",
set_values, {"node_id": _NODE_ID, "values": _VALUES}, ["values"], destructive=True),
_action("remove_nodes", "Remove nodes",
"Removes nodes from the open workflow, with their wires. One Ctrl+Z step.",
remove_nodes, {"node_ids": {"type": "array", "items": _NODE_ID, "maxItems": MAX_REMOVE}}, ["node_ids"],
destructive=True),
_action("new_workflow", "Start a new workflow",
"Opens a new workflow tab, the user's open one untouched: blank, or built from nodes in ComfyUI's API "
"form ({\"1\": {\"class_type\": \"CLIPTextEncode\", \"inputs\": {\"text\": \"...\", \"clip\": [\"2\", 1]}}}: a value "
"or [node id, output slot]). Checked before it opens; the answer maps your ids to node ids.",
new_workflow, {"name": {"type": "string"}, "nodes": {"type": "object"}}),
_action("open_workflow", "Open a saved workflow",
"Opens one of the user's saved workflows (list_workflows names them) in a new tab, leaving the open one "
"as it was, to run it or build on it with add_node, connect_nodes and set_values.",
open_workflow, {"workflow": {"type": "string", "description": "A workflow name from list_workflows."}}, ["workflow"]),
])
+114
View File
@@ -0,0 +1,114 @@
"""A directory of the tools by what an AI wants to do, so it goes straight to the right one, as
ARCHITECTURE.md does for the code: each tool under its stage of the work (action_catalog.TOOLS),
with its title, in the catalog's order. It is built from the registry, so a new tool shows up by
itself (tests/test_mcp_directory.py checks that every stage has a heading). mcp_server.py puts it
into begin_task's answer, and into get_situation's while there is no task yet.
AI apps keep the tool list they fetched (Antigravity files it away and fetches it again only when
it reconnects), so after Anomalous is updated an app can go on with the old one: new tools missing,
gone ones still called. listed() keeps which list each app fetched last (a fingerprint, in
anomalous/ai_tool_lists.json); a call from an app whose list is not today's tells the AI, and notes
it on the task (task["stale_tools"]) so the workbench tells the user how to reconnect, until the app
fetches the list again.
"""
import hashlib
import json
import os
import time
from . import action_catalog, ai_tasks
from .mcp_tools import retext
from .path_utils import atomic_write_json
SECTIONS = ( # stage -> heading, in the order work usually goes
("run", "Run a workflow"), ("canvas", "The workflow on the canvas"), ("model", "Models"), ("prompt", "Prompts"),
("combo", "Combos"), ("picture", "Pictures"), ("voice", "Voices"), ("library", "The library"),
(None, "The task and the user"),
)
NOTE = "The tools by what you want to do: go straight to the one you need instead of looking around first."
STALE_NOTE = ("Your app holds an older list of Anomalous's tools (Anomalous was updated since it fetched it): some may "
"be missing or gone. The call went through, but tell the user in report to reconnect the app to Anomalous "
"(Antigravity: restart it, or switch the anomalous MCP server off and on) and to go on in a new chat.")
_lists = None # app -> fingerprint of the tool list it fetched last
def directory(specs):
"""{heading: "tool (title), ..."} for the tools of `specs` {name: spec}."""
sections = {}
for name, entry in action_catalog.TOOLS.items():
if name in specs:
sections.setdefault(entry.stage, []).append(f"{name} ({specs[name].get('title') or name})")
return {"note": NOTE, **{heading: ", ".join(sections[stage]) for stage, heading in SECTIONS if stage in sections}}
def attach(result, name, specs):
"""Puts the directory into begin_task's answer, and into get_situation's while there is no task."""
data = result.get("structuredContent") if isinstance(result, dict) and not result.get("isError") else None
if not isinstance(data, dict) or not (name == "begin_task" or (name == "get_situation" and not data.get("task"))):
return result
retext(result, {**data, "directory": directory(specs)})
return result
def fingerprint(tools):
return hashlib.sha256(json.dumps(tools, sort_keys=True, ensure_ascii=False).encode("utf-8")).hexdigest()[:16]
def _path():
return os.path.join(os.path.dirname(ai_tasks.tasks_dir()), "ai_tool_lists.json")
def _known():
global _lists
if _lists is None:
try:
with open(_path(), encoding="utf-8") as source:
stored = json.load(source)
except (OSError, ValueError):
stored = {}
_lists = stored if isinstance(stored, dict) else {}
return _lists
def listed(client, tools):
"""The app `client` fetched `tools`: kept, and the notes on its tasks go."""
with ai_tasks._lock:
known = _known()
if known.get(client) != fingerprint(tools):
known[client] = fingerprint(tools)
atomic_write_json(_path(), known)
changed = [task for task in ai_tasks._load().values() if task.get("stale_tools") and task.get("client") == client]
for task in changed:
task.pop("stale_tools")
ai_tasks._save(task)
for task in changed:
ai_tasks._notify(task["id"])
def stale(client, tools):
"""Whether the app `client` fetched another list than `tools` last (one never seen fetching: no)."""
with ai_tasks._lock:
known = _known().get(client)
return bool(known) and known != fingerprint(tools)
def tell_stale(result, task_id):
"""Says so in the call's answer, and notes it on the task for the workbench."""
if isinstance(result, dict):
data = result.get("structuredContent")
first = (result.get("content") or [None])[0]
if isinstance(data, dict) and not result.get("isError"):
retext(result, {**data, "tool_list": STALE_NOTE})
elif isinstance(first, dict) and first.get("type") == "text":
result["content"][0] = {**first, "text": f"{first.get('text', '')} {STALE_NOTE}"}
if not task_id:
return result
with ai_tasks._lock:
task = ai_tasks._load().get(task_id)
if task is None or task.get("stale_tools"):
return result
task["stale_tools"] = time.time()
ai_tasks._save(task)
ai_tasks._notify(task_id)
return result
+102
View File
@@ -0,0 +1,102 @@
"""run_workflow's blanks: a daily run in one call, so a small model need not read and edit the canvas.
With `workflow` (a saved workflow's name) the run opens it in the task's tab the first time and
reuses that tab after (the page marks a tab open_workflow opened with the workflow's file); what
the AI fills in goes into it first (web/modules/mcp_canvas_edit.js fill_slots): the positive and
negative prompt boxes, the seed of every sampler, the empty latent's width and height, the
picture its LoadImage reads (from the input or the output folder, found here, never copied;
image_node says which LoadImage when there are several), and `values`: any other setting by its
name, a third-party node's as well, for the one node that has it ({node id: {...}} when several
do; set_values takes the same). No fixed list of blanks grows here. The rest stays as saved. Without
`workflow` the blanks go into the canvas the run would run. All of it is checked before anything
changes, and it is one Ctrl+Z step logged as the AI's.
"""
from . import ai_task_canvas, ai_tasks
from .mcp_bridge import PageRefused, PageUnavailable, ask_page
from .mcp_tools import ToolError, _resolve_folder_image, list_workflows
SLOTS = ("prompt", "negative", "seed", "width", "height", "image", "image_node", "values")
MAX_VALUES = 60
MAX_TEXT = 20000
def flat_values(values):
"""{setting name: value}, checked for shape and size; the page checks what each setting takes."""
if values is None:
return {}
if not isinstance(values, dict) or len(values) > MAX_VALUES:
raise ToolError(f"values is an object {{setting name: value}} of up to {MAX_VALUES} settings.")
for name, value in values.items():
if len(name) > 100 or not (isinstance(value, (bool, int, float)) or (isinstance(value, str) and len(value) <= MAX_TEXT)):
raise ToolError(f"The value of {name[:100]} is a number, true/false or text of up to {MAX_TEXT} characters.")
return values
def node_values(values):
"""Settings by name ({setting name: value}) or for one node ({node id: {setting name: value}})."""
if not isinstance(values, dict) or not values:
raise ToolError('values is {setting name: value}, or {"<node id>": {setting name: value}} for one node.')
nested = {key: value for key, value in values.items() if isinstance(value, dict)}
for key, settings in nested.items():
if not str(key).isdigit():
raise ToolError(f"values.{str(key)[:40]}: only a node id takes an object of settings.")
flat_values(settings)
flat_values({key: value for key, value in values.items() if key not in nested})
if len(values) - len(nested) + sum(len(settings) for settings in nested.values()) > MAX_VALUES:
raise ToolError(f"values sets up to {MAX_VALUES} settings at once.")
return values
def saved_workflow(name):
"""(file name, name) of the user's saved workflow called `name`."""
from .recipe_store import _list_recipes, get_recipes_dir
name = str(name or "").strip()
if not name or not any(item["name"] == name for item in list_workflows(name, 50)["workflows"]):
raise ToolError(f"No saved workflow called {name}; list_workflows names them.")
filename = next((recipe["filename"] for recipe in _list_recipes(get_recipes_dir())
if (recipe["data"].get("name") or recipe["filename"]) == name), None)
return filename, name
def _picture(image):
"""The value a LoadImage takes for picture `image`: in the input folder as it is, from the
output folder with ComfyUI's "[output]" mark (it reads it from there)."""
for folder in ("input", "output"):
try:
rel, _path = _resolve_folder_image(image, folder)
except ToolError:
continue
return rel if folder == "input" else f"{rel} [output]"
raise ToolError(f"{image} is in neither ComfyUI's input nor its output folder; search_images lists the pictures in the output.")
def _page(action, args, timeout=30):
try:
return ask_page(action, args, timeout)
except (PageUnavailable, PageRefused) as error:
raise ToolError(str(error))
def fill(ctx, workflow=None, **slots):
"""Opens `workflow` for the task (once) and fills `slots` in; what was set, or None when nothing was asked."""
slots = {key: value for key, value in slots.items() if key in SLOTS and value not in (None, "")}
if not workflow and not slots:
return None
if "image" in slots:
slots["image"] = _picture(slots["image"])
if "values" in slots:
node_values(slots["values"])
filename = name = None
if workflow:
filename, name = saved_workflow(workflow)
task = ai_tasks.get_task(ctx.get("task")) if ctx.get("task") else None
no_tab_yet = bool(task) and ai_task_canvas.mode_of(task) == "new" and not (task.get("canvas") or {}).get("tab")
filled = None if workflow and no_tab_yet else _page("fill_slots", {"filename": filename, **slots})
if workflow and (filled is None or filled.get("needs_open")):
_page("open_workflow", {"filename": filename, "name": name}, timeout=60)
filled = _page("fill_slots", {"filename": filename, **slots})
if filled.get("needs_open"):
raise ToolError(f"The saved workflow {name} did not open; nothing was run.")
return {**({"workflow": name} if name else {}), **(filled.get("set") or {})}
+86
View File
@@ -0,0 +1,86 @@
"""zoom_image, the AI's closer look: part of a picture (in ComfyUI's output, input or temp folder)
cut out and shown larger, and the same part of a second picture beside it to compare a before
and an after, attached to the answer. It reads only: the pictures stay as they are. With it an AI
need not crop, enlarge or compare pictures with code of its own (each such command asks the user
to allow it). The region is in fractions of the picture, as the user's marks give theirs."""
import base64
import io
from PIL import Image
from .mcp_tools import ToolError, _resolve_folder_image, _tool
MIN_SIZE, MAX_SIZE, DEFAULT_SIZE = 256, 1024, 768
MAX_ENLARGE = 4 # a tiny part is shown at most this many times its size
MIN_PIXELS = 8
def _box(region, width, height):
"""Region {x, y, width, height} (fractions 0-1, default the whole picture) as a pixel box."""
region = region if isinstance(region, dict) else {}
try:
x, y = (min(1.0, max(0.0, float(region.get(key, 0)))) for key in ("x", "y"))
w, h = (min(1.0, max(0.0, float(region.get(key, 1)))) for key in ("width", "height"))
except (TypeError, ValueError):
raise ToolError("region is {x, y, width, height}, each a fraction 0-1 of the picture.")
left, top = round(x * width), round(y * height)
right, bottom = min(width, round((x + w) * width)), min(height, round((y + h) * height))
if right - left < MIN_PIXELS or bottom - top < MIN_PIXELS:
raise ToolError(f"The region is under {MIN_PIXELS} pixels across; give a larger one.")
return left, top, right, bottom
def _part(path, region, size):
"""(facts, MCP image item) of `region` of the picture at `path`, its longer side `size` at most."""
try:
with Image.open(path) as picture:
box = _box(region, picture.width, picture.height)
part = picture.crop(box).convert("RGB")
whole = f"{picture.width}x{picture.height}"
except (OSError, ValueError, Image.DecompressionBombError):
raise ToolError("The picture cannot be read.")
scale = min(size / max(part.size), MAX_ENLARGE)
shown = part.resize((max(1, round(part.width * scale)), max(1, round(part.height * scale))), Image.LANCZOS)
data = io.BytesIO()
shown.save(data, "JPEG", quality=88)
facts = {"picture": whole, "region": f"x {box[0]}-{box[2]}, y {box[1]}-{box[3]} px", "shown": f"{shown.width}x{shown.height}"}
return facts, {"type": "image", "data": base64.b64encode(data.getvalue()).decode("ascii"), "mimeType": "image/jpeg"}
def zoom_image(image, region=None, folder="output", compare_with="", compare_folder="", size=DEFAULT_SIZE):
try:
size = min(MAX_SIZE, max(MIN_SIZE, int(size)))
except (TypeError, ValueError):
raise ToolError(f"size is {MIN_SIZE}-{MAX_SIZE} pixels.")
rel, path = _resolve_folder_image(image, folder)
facts, item = _part(path, region, size)
answer = {"image": rel, "folder": folder, **facts}
extra = [item]
if compare_with:
other_folder = compare_folder or folder
other_rel, other_path = _resolve_folder_image(compare_with, other_folder)
other_facts, other_item = _part(other_path, region, size)
answer["compare_with"] = {"image": other_rel, "folder": other_folder, **other_facts}
answer["note"] = "Two pictures are attached: first image, then compare_with, the same region of each."
extra.append(other_item)
return answer, extra
_REGION = {"type": "object", "description": "Fractions 0-1, like the user's marks; default all.",
"properties": {name: {"type": "number", "minimum": 0, "maximum": 1} for name in ("x", "y", "width", "height")}}
_FOLDER = {"type": "string", "enum": ["output", "input", "temp"], "default": "output"}
LOOK_TOOLS = dict([
_tool("zoom_image", "Look closer at a picture",
"Part of a picture larger, at full detail (a hand, a face, a seam). folder: output, input or temp. "
"compare_with: a second picture, the same region beside it. Use it instead of cropping with code; "
"change pictures only by running a workflow, never by editing their files.",
zoom_image, {"image": {"type": "string", "description": "As the other tools name it."},
"region": _REGION, "folder": _FOLDER,
"compare_with": {"type": "string", "description": "A second picture."},
"compare_folder": {"type": "string", "enum": ["output", "input", "temp"],
"description": "Default: folder."},
"size": {"type": "integer", "minimum": MIN_SIZE, "maximum": MAX_SIZE, "default": DEFAULT_SIZE}},
["image"]),
])
+170
View File
@@ -0,0 +1,170 @@
"""Where a model loads, for the AI: search_models and get_model (mcp_tools.py finds the files)
say which node types list each file among their choices (ComfyUI's /object_info, kept a little
while by mcp_canvas_edit.py), and, when a file's header says it is another kind than the folder
it is in (model_placement.py, the models page's Tidy view), where it belongs and that nothing
loads it where it is. So the AI picks the right node at once, or has the model moved, instead of
searching ComfyUI itself. move_model moves one model with its covers and notes as Tidy does,
behind the "files" gate (the user says yes first unless they let it go ahead)."""
import os
import threading
import folder_paths
from . import mcp_canvas_edit, mcp_tools, model_advice, model_placement
from .mcp_actions import _action
from .mcp_tools import _LIMIT, _QUERY, ToolError
from .model_constants import MODEL_EXTENSIONS
from .model_import import Refused, import_types
MAX_LOADERS = 6
_lock = threading.Lock()
_index = {"of": None, "inputs": [], "by_choice": {}}
def _plain(name):
return str(name).replace("\\", "/")
def _choices(conf):
if not isinstance(conf, list) or not conf:
return None
if isinstance(conf[0], list):
return conf[0]
options = conf[1] if len(conf) > 1 and isinstance(conf[1], dict) else {}
return options.get("options") if conf[0] == "COMBO" and isinstance(options.get("options"), list) else None
def _built(info):
"""The drop-downs of model files in node definitions `info`: [(node type, choices)] and choice -> their places."""
with _lock:
if _index["of"] is info:
return _index
inputs, by_choice = [], {}
for node_type, item in info.items():
for section in ("required", "optional"):
for conf in ((item.get("input") or {}).get(section) or {}).values():
listed = [_plain(choice) for choice in _choices(conf) or [] if isinstance(choice, str)]
if not any(choice.lower().endswith(MODEL_EXTENSIONS) for choice in listed):
continue
for choice in listed:
by_choice.setdefault(choice, []).append(len(inputs))
inputs.append((node_type, frozenset(listed)))
_index.update(of=info, inputs=inputs, by_choice=by_choice)
return _index
def _folder(folder_type):
return frozenset(_plain(name) for name in folder_paths.get_filename_list(folder_type))
def _readers(ctx, folder_type, rel=None):
"""Node types with a drop-down of the whole folder `folder_type` (one that lists `rel`, when given)."""
index = _built(mcp_canvas_edit.object_info(ctx))
folder = _folder(folder_type)
if not folder:
return []
places = index["by_choice"].get(_plain(rel), []) if rel else range(len(index["inputs"]))
found = []
for at in places:
node_type, choices = index["inputs"][at]
if folder <= choices and node_type not in found:
found.append(node_type)
return found[:MAX_LOADERS]
def _about(ctx, model):
"""`model` (from mcp_tools) with what loads it: loaded_by, or belongs_in and placement when it is misfiled."""
folder_type, root, base_dir, path = mcp_tools._resolve_model(model["model_id"])
rel = os.path.relpath(path, base_dir).replace(os.sep, "/")
home = model_placement.belongs(model_placement.one(folder_type, root, rel, path))
if not home or home["works"]:
model["loaded_by"] = _readers(ctx, folder_type, rel)
return model
kind, to = home["kind"], home["to"]
readers = _readers(ctx, kind)
words = (f"Its header says it is a {kind} file, but it is in {folder_type}: the nodes listing it there cannot load it"
+ (f"; {', '.join(readers)} load {kind} files." if readers else "."))
there = os.path.join(folder_paths.get_folder_paths(kind)[to["root"]], *to["rel"].split("/"))
if os.path.isfile(there):
words += (f" A file of the same name is already in {kind} (model_id {kind}:{to['root']}:{to['rel']}): use that one. "
"If it is the same file, this copy is extra; the user can recycle it in Anomalous's models page (Tidy).")
else:
words += f" move_model with to_type {kind} moves it there (the user confirms first)."
model.update(belongs_in=kind, placement=words)
return model
def search_models(ctx, query="", type="", base_model="", source="", limit=20):
found = mcp_tools.search_models(query, type, base_model, source, limit)
for model in found["models"]:
_about(ctx, model)
return found
def get_model(ctx, model_id, include_cover=False):
detail, extra = mcp_tools.get_model(model_id, include_cover)
folder_type, _idx, _base, path = mcp_tools._resolve_model(model_id)
examples = model_advice.author_examples(path, folder_type, detail.get("sha256", ""))
if examples:
detail["author_examples"] = examples
return _about(ctx, detail), extra
def move_model(ctx, model_id, to_type):
folder_type, root, base_dir, path = mcp_tools._resolve_model(model_id)
rel = os.path.relpath(path, base_dir).replace(os.sep, "/")
item = model_placement.one(folder_type, root, rel, path)
home = model_placement.belongs(item)
to_type = str(to_type or "")
if to_type == folder_type:
raise ToolError(f"It is in {to_type} already.")
if to_type not in import_types():
raise ToolError(f"to_type is one of {', '.join(import_types())}.")
to = home["to"] if home and home["kind"] == to_type else model_placement.home_for(item, to_type)
if not to:
raise ToolError(f"This ComfyUI has no {to_type} folder.")
if os.path.exists(os.path.join(folder_paths.get_folder_paths(to_type)[to["root"]], *to["rel"].split("/"))):
raise ToolError(f"A file of the same name is already in {to_type} (model_id {to_type}:{to['root']}:{to['rel']}); "
"nothing was moved. Use that one; the user can recycle an extra copy in Anomalous's models page (Tidy).")
try:
moved = model_placement.move({"type": folder_type, "root": root, "rel": rel,
"to_type": to_type, "to_root": to["root"], "to_rel": to["rel"]})
except Refused as error:
raise ToolError(f"Not moved ({error.code}).")
mcp_canvas_edit.forget_object_info()
answer = {"moved": {"from": f"{folder_type}/{rel}", "to": f"{moved['type']}/{moved['rel']}"},
"model_id": f"{moved['type']}:{moved['root']}:{moved['rel']}",
"loaded_by": _readers(ctx, moved["type"], moved["rel"])}
if moved["left"]:
answer["left_behind"] = moved["left"] # covers or notes that could not go along
return answer
MODEL_TOOLS = dict([
_action("search_models", "Search models",
"Model files by name, trigger word or base model: model_id, and loaded_by (the node types that list "
"the file). A misfiled one comes with belongs_in: nothing loads it where it is.",
search_models, {
"query": _QUERY,
"type": {"type": "string", "description": "A models folder: checkpoints, loras, vae…"},
"base_model": {"type": "string", "description": "e.g. SDXL, Pony, Flux."},
"source": {"type": "string", "enum": ["civitai", "inferred_from_file", "not_scanned"],
"description": "Where the model's information came from."},
"limit": _LIMIT}, read_only=True),
_action("get_model", "Model details",
"Everything known about one model: Civitai name, version and link, trigger words, base model, description, "
"the user's own name and notes, author_examples (the sampler, steps, CFG, size and, for a LoRA, the weight "
"its author's example pictures used: a good start), and loaded_by (or belongs_in and placement) as in "
"search_models. include_cover attaches its cover picture.",
get_model, {"model_id": {"type": "string", "description": "From search_models."},
"include_cover": {"type": "boolean", "default": False}}, ["model_id"], read_only=True),
_action("move_model", "Move a model to its folder",
"Moves a model the user has in the wrong models folder (search_models says belongs_in) to where it loads, "
"with its covers and notes, as Anomalous's Tidy view does; never over a file. The answer gives its new "
"model_id and loaded_by; the canvas lists it from then on.",
move_model, {"model_id": {"type": "string", "description": "From search_models."},
"to_type": {"type": "string", "description": "The models folder to move it to: belongs_in from search_models."}},
["model_id", "to_type"], destructive=True),
])
+59
View File
@@ -0,0 +1,59 @@
"""Ready-made requests the user can start from their AI app (MCP prompts; Claude shows them as
slash commands such as /anomalous:pick_best): each is a few lines telling the AI how to do a
common job with Anomalous's tools so the user can watch, choose and step in on the AI workbench.
They only shape the AI's first message; they change nothing themselves."""
PROMPTS = {
"workbench_task": {
"title": "Start a task on the AI workbench",
"description": "Do something in ComfyUI the way the user can follow on Anomalous's AI workbench.",
"arguments": [{"name": "what", "description": "What to do", "required": True}],
"text": ("Use Anomalous for this: {what}\n"
"Call begin_task with a short title in my language first. Show your plan, then results, with "
"show_panel in the order they happen; when a choice is mine, ask it with ask_user (or a picks panel, "
"waiting with get_answers), never in this chat. Honour any user_messages that come with a tool result. End with report(done=true)."),
},
"pick_best": {
"title": "Make a few, let me pick",
"description": "Make several variations with the open workflow and let the user pick one.",
"arguments": [{"name": "what", "description": "What to vary (seed, LoRA weight, prompt…)", "required": True},
{"name": "count", "description": "How many (default 3)", "required": False}],
"text": ("Make {count} variations of the open ComfyUI workflow, changing: {what}.\n"
"Call begin_task first. Read the canvas (describe_canvas), change one thing per run, run the workflow "
"each time and find the pictures with search_images. Then show them with show_panel as a picks block, "
"each with what you changed as its reason, and wait for my pick with get_answers. Put the chosen "
"settings back on the canvas and report(done=true)."),
},
"fix_missing_models": {
"title": "Fix the missing models",
"description": "Find what the open workflow lacks and put it back, asking before downloads.",
"arguments": [],
"text": ("Check the open workflow's models with check_workflow_models (call begin_task first). Put back the ones "
"Anomalous found with certainty (fix_workflow_models). Show what is still missing with show_panel as a "
"table, and ask me with choices before downloading anything. report(done=true) when finished."),
},
"explain_canvas": {
"title": "Explain the open workflow",
"description": "A picture of the open workflow's main chain with a short explanation.",
"arguments": [],
"text": ("Explain the workflow open in ComfyUI. Call begin_task, read it with describe_canvas, then show_panel "
"with a diagram block of its main chain (loaders, prompts, sampler, decoding, saving; node_id on each "
"box) and a few lines of text on what each part does and which settings matter most. Change nothing."),
},
}
def list_prompts():
return {"prompts": [{"name": name, "title": item["title"], "description": item["description"], "arguments": item["arguments"]}
for name, item in PROMPTS.items()]}
def get_prompt(name, arguments=None):
"""prompts/get, or None for an unknown name or a missing required argument."""
item = PROMPTS.get(name)
arguments = arguments if isinstance(arguments, dict) else {}
if not item or any(arg["required"] and not str(arguments.get(arg["name"], "")).strip() for arg in item["arguments"]):
return None
values = {"what": "", "count": "3", **{key: str(value) for key, value in arguments.items()}}
return {"description": item["description"],
"messages": [{"role": "user", "content": {"type": "text", "text": item["text"].format(**values)}}]}
+133 -19
View File
@@ -8,22 +8,38 @@ POST /anomalous/mcp speaks the Model Context Protocol's Streamable HTTP transpor
Only this computer may connect: the peer must be loopback, no proxy header may be present,
and Host and Origin (when sent) must name a loopback host, which stops DNS rebinding.
Answers are always one JSON object; there are no server-initiated messages. The tools are
in mcp_tools.py (reading) and mcp_actions.py (acting); POST /anomalous/mcp/bridge is where
the open ComfyUI page claims and answers canvas actions (mcp_bridge.py).
in mcp_tools.py (reading), mcp_actions.py (acting) and mcp_task_tools.py (the AI workbench's
begin_task and report); every call is kept as a step of a task (ai_tasks.py) for the AI
workbench page, starting in state "running" and updated on completion, with unread user
messages and panel answers attached to the next successful tool response, and pictures changed
outside Anomalous since the AI's previous call shown and told (ai_outside.py). POST
/anomalous/mcp/bridge is where the open ComfyUI page claims and answers canvas actions
(mcp_bridge.py).
"""
import asyncio
import base64
import time
import urllib.parse
from aiohttp import web
from . import mcp_bridge
from . import ai_callers, ai_gates, ai_outside, ai_panels, ai_run_tracking, ai_task_canvas, ai_task_runs, ai_tasks, mcp_apps, mcp_bridge, mcp_directory, mcp_prompts
from .local_guard import is_local as _is_local, loopback_host as _loopback_host
from .mcp_actions import ACTIONS
from .mcp_tools import TOOLS as READ_TOOLS, call_tool
from .mcp_canvas_edit import CANVAS_TOOLS
from .mcp_model_use import MODEL_TOOLS
from .mcp_look import LOOK_TOOLS
from .mcp_situation import SITUATION_TOOLS
from .mcp_task_tools import TASK_TOOLS
from .mcp_tools import TOOLS as READ_TOOLS, call_tool, retext
from .mcp_arguments import prepare_arguments, renaming_note
TOOLS = {**READ_TOOLS, **ACTIONS}
TOOLS = {**READ_TOOLS, **MODEL_TOOLS, **LOOK_TOOLS, **ACTIONS, **CANVAS_TOOLS, **TASK_TOOLS, **SITUATION_TOOLS, **mcp_apps.APP_TOOLS}
CAPABILITIES = {
"tools": {"listChanged": False}, "prompts": {"listChanged": False}, "resources": {"listChanged": False},
"extensions": {mcp_apps.EXTENSION: {"mimeTypes": [mcp_apps.MIME]}}, # the card in the conversation (mcp_apps.py)
}
MODERN_VERSIONS = ("2026-07-28",)
LEGACY_VERSIONS = ("2025-11-25", "2025-06-18", "2025-03-26")
@@ -32,12 +48,37 @@ INSTRUCTIONS = (
"Tools over the user's ComfyUI library as Anomalous Model Browser keeps it: model files with "
"their Civitai information, output images with their generation settings, combos (a model, "
"LoRAs and a prompt), workflow recipes, saved prompts, generated audio, the scan status and "
"the activity log; and tools that act on the workflow open in ComfyUI (read it with "
"describe_canvas first), scan models, or have an Anomalous TTS voice speak. Canvas changes "
"the activity log; and tools that act on the workflow open in ComfyUI (get_situation gives flow chains; "
"describe_canvas only for details) or build one where the user watches: find_nodes searches brief types; "
"add_node wires in the same call (wire_from/wire_to; wire_to=ID splices it in front of that node, "
"insert_after=ID behind it); add_nodes adds a whole group in ComfyUI's API form in one call; "
"connect_nodes needs only two IDs unless sockets are ambiguous. Reuse receipts instead of rereading "
"the graph. new_workflow opens a tab only when needed. Tools also scan models or have an "
"Anomalous TTS voice speak. Canvas changes "
"are single Ctrl+Z steps and are logged as the AI's; nothing deletes files. Ids returned by "
"one tool (model_id, image, node ids) are what the others take. For what no tool does, tell "
"the user where in Anomalous to do it (the rail on the left: Models, Gallery, Workflows, "
"Combos, Prompts, Voices)."
"Combos, Prompts, Voices). begin_task's answer carries a directory of the tools by what you want to do: "
"look there and go straight to the tool. In a new chat on a task, or when unsure where things stand, call get_situation "
"first: the task, the canvas in brief (what is still unwired) and the next step in one answer. "
"The user watches your work in Anomalous's AI workbench: call begin_task "
"with a short title when you start what they asked for (it continues the waiting or active task; "
"new_task=true is only for a separate user request), and report to explain a choice, hand in "
"pictures made another way, or end the task (done=true). The user can write to you or answer "
"panel questions there: call get_answers to wait for or read them, and any unread user_messages "
"or user_answers attached to a tool result must also be honoured. While you work on an Anomalous task, ask "
"the user every question (a choice, a preference, yes or no) with ask_user, never in your own chat or its "
"question form: the user watches and answers in Anomalous. Until they say the task is done, end every round "
"with ask_user (what next, or whether to go ahead), not with a note or panel alone; only a run, download or "
"scan going on lets you end your turn without asking. Likewise write your explanations and replies "
"there with report (reply_to answers their message or marks), not only in your chat. Show plans, workflow explanations "
"and results in chronological order with show_panel; update_panel appends new content below what "
"the user already saw. Use run_workflow for execution: it returns real state/results after a bounded wait. "
"If you submit through /prompt or another tool, immediately call track_run with YOUR returned prompt_ids "
"and task ID, or the workbench cannot observe it. For running results use run_status(wait_seconds=55); "
"do not inspect output directories or resubmit to find out whether it finished. Change pictures only by "
"running workflows, never with code of your own; if you do use another tool (a command, a script, files), "
"say what you ran with log_outside."
)
LIST_TTL_MS = 300000
# Requests run tools in threads; a few at a time is plenty for one person's AI app.
@@ -79,8 +120,13 @@ def _result(id_, result, modern):
return web.json_response({"jsonrpc": "2.0", "id": id_, "result": result})
def _tools():
"""The tool list AI apps are offered, as tools/list gives it."""
return [mcp_apps.with_card(tool["spec"]) for tool in TOOLS.values()]
def _tool_list(modern):
result = {"tools": [tool["spec"] for tool in TOOLS.values()]}
result = {"tools": _tools()}
if modern:
result.update(ttlMs=LIST_TTL_MS, cacheScope="public")
return result
@@ -90,7 +136,7 @@ def _initialize(params):
requested = params.get("protocolVersion")
return {
"protocolVersion": requested if requested in LEGACY_VERSIONS else LEGACY_VERSIONS[0],
"capabilities": {"tools": {"listChanged": False}},
"capabilities": CAPABILITIES,
"serverInfo": SERVER_INFO,
"instructions": INSTRUCTIONS,
}
@@ -99,7 +145,7 @@ def _initialize(params):
def _discover():
return {
"supportedVersions": list(MODERN_VERSIONS + LEGACY_VERSIONS),
"capabilities": {"tools": {"listChanged": False}},
"capabilities": CAPABILITIES,
"_meta": {"io.modelcontextprotocol/serverInfo": SERVER_INFO},
"instructions": INSTRUCTIONS,
"ttlMs": LIST_TTL_MS,
@@ -122,13 +168,65 @@ def _modern_header_problem(request, method, params, version):
async def _call(params, request):
name = params.get("name")
arguments = params.get("arguments") or {}
if name not in TOOLS or not isinstance(arguments, dict):
raw_arguments = params.get("arguments") or {}
if not isinstance(name, str) or name not in TOOLS or not isinstance(raw_arguments, dict):
return None
ctx = {"base_url": f"{request.scheme}://{request.host}"} # loopback, checked in api_mcp
async with _slots:
result = await asyncio.to_thread(call_tool, TOOLS, name, arguments, ctx)
return result
arguments = dict(raw_arguments)
client = ai_callers.client_of(request.headers, params)
session = ai_callers.session_of(request.headers, params)
task_id = ai_callers.task_of(request.headers, params, arguments)
if "task" in arguments and "task" not in TOOLS[name]["spec"]["inputSchema"].get("properties", {}):
arguments.pop("task", None)
arguments, renamed = prepare_arguments(TOOLS[name]["spec"]["inputSchema"], arguments)
ctx = {"base_url": f"{request.scheme}://{request.host}", "client": client, "session": session, "task": task_id} # loopback, checked in api_mcp
started = time.time()
read_only = bool(TOOLS[name]["spec"].get("annotations", {}).get("readOnlyHint", True))
step_ref = await asyncio.to_thread(
ai_tasks.begin_step, client, name, arguments, started, read_only, session=session, task_id=task_id,
)
active_task = step_ref[0] if step_ref else task_id
if active_task:
ctx["task"] = active_task
outside = None
if name not in mcp_apps.APP_TOOLS: # the card's own calls do not reach the AI
outside = await asyncio.to_thread(ai_outside.check, active_task or ai_tasks.active_task_id(client, session))
canvas_call = ai_task_canvas.begin_call(active_task) # the page works on the task's own tab
try:
# The user's gate first (ai_gates.py): it may wait for their OK, or answer for them.
result = await asyncio.to_thread(ai_gates.check, name, arguments, step_ref)
if result is None:
async with _slots:
result = await asyncio.to_thread(call_tool, TOOLS, name, arguments, ctx)
except BaseException as error:
failed_result = {"content": [{"type": "text", "text": f"{name} failed: {error}"}], "isError": True}
await asyncio.to_thread(
ai_tasks.finish_step, client, name, arguments, failed_result, started, read_only,
step_ref=step_ref, session=session, task_id=active_task,
)
raise
if renamed and not result.get("isError") and isinstance(result.get("structuredContent"), dict):
retext(result, {**result["structuredContent"], "arguments_note": renaming_note(name, renamed)})
step = await asyncio.to_thread(
ai_tasks.finish_step, client, name, arguments, result, started, read_only,
step_ref=step_ref, session=session, task_id=active_task,
)
ai_task_canvas.tell(result, canvas_call)
if name == "run_workflow" and step and step.get("state") == "running":
ai_task_runs.watch(step["id"], result.get("structuredContent") or {})
await asyncio.to_thread(ai_run_tracking.wait_for_receipt, result, client, session, active_task,
ai_run_tracking.checked_wait(arguments.get("wait_seconds", 55)), step["id"])
# The user's news and the run's go to the AI, never to the card's own calls (they do not reach the AI).
if name not in mcp_apps.APP_TOOLS and isinstance(result, dict) and not result.get("isError"):
resolved_task = active_task or await asyncio.to_thread(ai_tasks.active_task_id, client, session=session)
if name not in ("run_workflow", "run_status", "stop_run") and resolved_task:
await asyncio.to_thread(ai_task_runs.tell, result, resolved_task)
await asyncio.to_thread(ai_task_runs.show_pictures, result, resolved_task)
if name != "get_answers" and name not in mcp_apps.APP_TOOLS and isinstance(result, dict) and not result.get("isError"):
await asyncio.to_thread(ai_panels.attach_pending, result, client, task_id=resolved_task or ("" if session else None))
mcp_directory.attach(result, name, {key: tool["spec"] for key, tool in TOOLS.items() if key not in mcp_apps.APP_TOOLS})
if name not in mcp_apps.APP_TOOLS and mcp_directory.stale(client, _tools()):
await asyncio.to_thread(mcp_directory.tell_stale, result, active_task)
return ai_outside.tell(result, outside)
async def api_mcp(request):
@@ -171,7 +269,23 @@ async def api_mcp(request):
if method == "ping":
return _result(id_, {}, modern)
if method == "tools/list":
return _result(id_, _tool_list(modern), modern)
listing = _tool_list(modern)
await asyncio.to_thread(mcp_directory.listed, ai_callers.client_of(request.headers, params), listing["tools"])
return _result(id_, listing, modern)
if method == "prompts/list":
return _result(id_, mcp_prompts.list_prompts(), modern)
if method == "prompts/get":
prompt = mcp_prompts.get_prompt(params.get("name"), params.get("arguments"))
if prompt is None:
return _error(id_, INVALID_PARAMS, f"Unknown prompt or missing arguments: {params.get('name')}")
return _result(id_, prompt, modern)
if method == "resources/list":
return _result(id_, {"resources": mcp_apps.resources()}, modern)
if method == "resources/read":
resource = await asyncio.to_thread(mcp_apps.read, params.get("uri"))
if resource is None:
return _error(id_, INVALID_PARAMS, f"Unknown resource: {params.get('uri')}")
return _result(id_, resource, modern)
if method == "tools/call":
result = await _call(params, request)
if result is None:
@@ -192,7 +306,7 @@ async def api_mcp_bridge(request):
if not isinstance(body, dict) or not isinstance(body.get("id"), str):
return web.json_response({"error": "Expected {id, claim} or {id, outcome}"}, status=400)
if body.get("claim") is True:
return web.json_response({"granted": mcp_bridge.claim(body["id"])})
return web.json_response({"granted": mcp_bridge.claim(body["id"], body.get("capabilities"))})
return web.json_response({"accepted": mcp_bridge.resolve(body["id"], body.get("outcome"))})
+98
View File
@@ -0,0 +1,98 @@
"""get_situation: where an AI's task stands, in one answer small enough to stay in its chat.
The task (what the user asked last, questions and gates still waiting for them, decisions so
far, pictures made, steps done outside Anomalous), the canvas in brief (web/modules/
mcp_canvas_brief.js: flow chains, brief node lines, missing inputs and outputs that fit,
ComfyUI's errors), the latest run, and the one thing to do next. For an AI starting a new chat
on a task or unsure where it is: a small model reads this instead of describe_canvas, find_nodes
and the task's history. Unread messages of the user's come with it as with every answer
(ai_panels.attach_pending).
"""
from . import ai_decisions, ai_task_runs, ai_tasks
from .mcp_bridge import PageRefused, PageUnavailable, ask_page
from .mcp_tools import _schema, tool_text
BUDGET = 3600 # characters: Antigravity files away an answer past about 4,000 and has the AI open it
MAX_TEXT = 200
def _last_asked(task):
"""The user's latest words or marks in the task, in short."""
for step in reversed(task.get("steps") or []):
if step.get("kind") == "user" and step.get("tool") in ("user_message", "user_marks", "user_pictures"):
return {"what": step["tool"].removeprefix("user_"), "text": str(step.get("text") or "")[:MAX_TEXT], "id": step["id"]}
return None
def _canvas():
try:
return ask_page("describe_canvas", {"brief": True}, timeout=15)
except PageUnavailable as error:
return {"unavailable": str(error)}
except PageRefused as error: # no workflow open, or the task's tab cannot be opened
return {"unavailable": str(error)}
def _next(summary, run, canvas):
if summary["asks"]:
return f"{summary['asks']} question(s) or gate(s) wait for the user: get_answers waits for their answer."
if run.get("state") in ("queued", "running", "asking", "allowed"):
return "A run is going: run_status(wait_seconds=55) waits for it; do not run again."
if any("not in list" in str(item.get("error")) for item in canvas.get("errors") or ()):
return ("A model or file the workflow names is not on this computer: check_workflow_models says which and what "
"can stand in (fix_workflow_models, download_missing_models); do not guess other choices.")
if canvas.get("errors"):
return "ComfyUI refused the last run: fix the nodes in canvas.errors (set_values, connect_nodes), then run_workflow."
if canvas.get("unwired"):
return "Wire canvas.unwired with connect_nodes (fits lists outputs that fit each), then run_workflow."
if run.get("state") == "done" and run.get("pictures"):
return "Look at the run's pictures (zoom_image) and tell the user with report what came out."
return "Go on with what the user asked (task.asked); run_workflow runs the canvas, report tells the user."
def get_situation(ctx, task=None):
client, session = ctx.get("client") or "AI", ctx.get("session") or ""
task_id = ai_tasks.active_task_id(client, session, task or ctx.get("task"))
record = ai_tasks.get_task(task_id) if task_id else None
run = {"state": "none"}
answer = {}
if record:
summary = ai_tasks.task_summary(record)
pictures = [item["filename"] for step in record.get("steps") or [] for item in step.get("results") or [] if item.get("type") == "image"]
answer["task"] = {"id": record["id"], "title": record.get("title") or "", "steps": summary["steps"],
"asked": _last_asked(record), "waiting_for_user": summary["asks"],
"decisions": ai_decisions.lines(record, limit=4), "pictures": pictures[-3:],
"outside_anomalous": summary["kinds"].get("outside", 0)}
status = ai_task_runs.status(client, session, task_id)
run = {key: status[key] for key in ("state", "summary", "pictures") if status.get(key)}
else:
summary = {"asks": 0}
answer["task"] = None
answer["task_note"] = "No task yet: begin_task names what the user asked for."
canvas = _canvas()
answer.update(run=run, canvas=canvas)
answer["next"] = _next(summary, run, canvas)
# Trim detail before chains and decisions; preserve actionable missing inputs and errors.
while len(tool_text(answer)) > BUDGET and canvas.get("lines"):
canvas["lines"].pop()
canvas["more_nodes"] = canvas.get("more_nodes", 0) + 1
while len(tool_text(answer)) > BUDGET and canvas.get("chains"):
canvas["chains"].pop()
canvas["more_chains"] = canvas.get("more_chains", 0) + 1
while len(tool_text(answer)) > BUDGET and record and answer["task"]["decisions"]:
answer["task"]["decisions"].pop(0)
return answer
SITUATION_TOOLS = {
"get_situation": {"run": get_situation, "context": True, "spec": {
"name": "get_situation", "title": "Where things stand",
"description": "The whole picture in one call: the task (what the user asked last, what waits for them, decisions, "
"pictures), the canvas in brief (flow chains, inputs nothing feeds and what fits them, errors), the "
"latest run, and next: the one thing to do now. Call it first in a new chat, and whenever unsure.",
"inputSchema": _schema({"task": {"type": "string", "description": "begin_task's id (optional)."}}),
"annotations": {"title": "Where things stand", "readOnlyHint": True, "destructiveHint": False,
"idempotentHint": True, "openWorldHint": False},
}},
}
+423
View File
@@ -0,0 +1,423 @@
"""MCP tools that shape the AI workbench's record (ai_tasks.py): begin_task names a piece of
work, report leaves a note for the user (with pictures made outside Anomalous's tools),
show_panel / update_panel draw a panel from Anomalous's blocks (ai_panels.py), get_answers
reads what the user picked or typed on it, log_outside keeps what the AI did with other tools
(ai_outside.py)."""
from . import ai_answer_wait, ai_decisions, ai_marks, ai_message_refs, ai_outside, ai_panels, ai_repaint, ai_run_control, ai_run_tracking, ai_task_links, ai_task_runs, ai_tasks, ai_toolbox
from .mcp_tools import MAX_LIMIT, ToolError, _schema
PACK_TYPES = 150 # node types listed per picked pack
ASK_WAIT_MIN = 30 # ask_user waits this long even when told not to: the user, reading the work, often answers at once
ASK_NEXT = ("Before you end your turn, ask_user what to do next or whether to go ahead: a note or a panel alone leaves "
"the user waiting. Only a run, a download or a scan going on lets you end it without asking.")
def begin_task(ctx, title, new_task=False):
title = str(title or "").strip()
if not title:
raise ToolError("title is empty: a few words on what the user asked for.")
task = ai_tasks.begin_task(
ctx.get("client") or "AI", title, new_task=bool(new_task),
session=ctx.get("session") or "", task_id=ctx.get("task"),
)
answer = {"task": task["id"], "title": task["title"], "note": (
"Continue this task; do not begin another for its steps. Every AMB call is a step the user sees, opens and "
"undoes in Anomalous's AI workbench. Ask them with ask_user (they answer there, not in your chat); report "
"explains a choice or hands in pictures, done=true at the end. Run with run_workflow; a run queued another "
"way needs track_run(prompt_ids) with your own IDs. In the task's own tab, do not switch tabs yourself. "
"Everything is in AMB's tools (models, pictures, node types, the canvas, runs and their previews): do not "
"read ComfyUI's folders or source, run preprocessors or call its routes yourself. Change pictures only by "
"running workflows, never by editing files with code. If you use another tool anyway, log_outside right "
"after. Unsure where things stand: get_situation. The directory below lists the tools by what you want to do.")}
answer["next"] = ("get_situation shows where things stand; a daily run is one call: run_workflow(workflow: a saved "
"one, with the prompt, seed or size to change).")
box = ai_toolbox.effective(ai_tasks.get_task(task["id"]))
if box:
answer["toolbox"] = ai_toolbox.words(box)
goes_on = ai_task_links.carried(task)
if goes_on:
answer["continues"] = goes_on # the task this one goes on from, and what was decided there
decided = ai_decisions.lines(ai_tasks.get_task(task["id"]))
if decided:
answer["decisions"] = decided # what was settled so far, the user's own choices in Anomalous too
if task.get("about"):
# Handed over from a model page, an image or a combo in Anomalous: the ids the tools take.
answer["about"] = task["about"]
answer["about_note"] = ("The user started this task from these in Anomalous; work with them: model_id goes to "
"set_model / add_lora / get_model, image to get_image_info, a combo's name to place_combo, "
"a workflow's name to open_workflow.")
return answer
def report(ctx, text="", images=None, done=False, reply_to=None, task=None):
try:
step, missing = ai_tasks.report(
ctx.get("client") or "AI", text, images or [], bool(done),
session=ctx.get("session") or "", task_id=task or ctx.get("task"), reply_to=reply_to,
)
except ValueError as error:
raise ToolError(str(error))
answer = {"noted": True, "pictures": len(step["results"])}
if not done:
answer["next"] = ASK_NEXT
if any(mark in str(text or "") for mark in ("?", "?")):
answer["next"] = "Your note asks the user something: ask it with ask_user (options to press) instead. " + ASK_NEXT
if missing:
answer["not_found"] = missing
answer["note"] = "Pictures are paths inside ComfyUI's output folder (subfolder/name.png), as search_images gives them."
return answer
def log_outside(ctx, what, why="", files=None, task=None):
try:
step = ai_outside.log(ctx.get("client") or "AI", what, why, files, session=ctx.get("session") or "",
task_id=task or ctx.get("task"))
except ValueError as error:
raise ToolError(str(error))
return {"logged": True, "step": step["id"]}
def log_decision(ctx, title, decision, reason="", rejected=None, by="user", related=None, task=None):
try:
step = ai_decisions.log(ctx.get("client") or "AI", title, decision, reason, rejected, by, related,
session=ctx.get("session") or "", task_id=task or ctx.get("task"))
except ValueError as error:
raise ToolError(str(error))
return {"recorded": True, "step": step["id"], "related": step["decision"]["related"]}
def show_panel(ctx, title, blocks, task=None):
try:
panel = ai_panels.show(
ctx.get("client") or "AI", title, blocks,
session=ctx.get("session") or "", task_id=task or ctx.get("task"),
)
except ai_panels.PanelError as error:
raise ToolError(str(error))
asks = [block["id"] for block in panel["blocks"] if block["type"] in ai_panels.INTERACTIVE]
answer = {"panel": panel["id"], "shown": True,
"blocks": [{"id": block["id"], "type": block["type"]} for block in panel["blocks"]]}
if asks:
answer["questions"] = asks
answer["note"] = ("The user answers in Anomalous when they get to it; call get_answers with this panel id "
"(wait_seconds up to 55 waits for it) before going on with what depends on it.")
else:
answer["next"] = ASK_NEXT
return answer
def ask_user(ctx, question, options=None, multiple=False, other=True, wait_seconds=ai_panels.MAX_WAIT, task=None):
"""A question in the AI workbench, where the user watches the work: options to press (one may be
recommended), a line of their own; waits for the answer."""
block = ({"type": "choices", "id": "answer", "options": options, "multiple": bool(multiple), "other": bool(other)}
if options else {"type": "input", "id": "answer", "question": str(question or "")[:200]})
try:
panel = ai_panels.show(ctx.get("client") or "AI", question, [block], session=ctx.get("session") or "",
task_id=task or ctx.get("task"), tool="ask_user")
except ai_panels.PanelError as error:
raise ToolError(str(error))
return _answer_of(ctx, panel["id"], wait_seconds)
def propose_repaint_area(ctx, title, marks, areas=None, grow=12, feather=8, note="", wait_seconds=ai_panels.MAX_WAIT):
"""The area to repaint for the user's marks, theirs beside the AI's for them to pick (ai_repaint.py); waits like ask_user."""
try:
panel = ai_repaint.propose(ctx.get("client") or "AI", title, str(marks or ""), areas, grow, feather,
str(note or "")[:400], session=ctx.get("session") or "")
except ai_repaint.RepaintError as error:
raise ToolError(str(error))
return _answer_of(ctx, panel["id"], wait_seconds)
def _answer_of(ctx, panel_id, wait_seconds):
"""The answer to the question of panel `panel_id` once the user gives it, waiting at least ASK_WAIT_MIN."""
client = ctx.get("client") or "AI"
try:
answer = ai_answer_wait.collect(client, panel_id, max(ASK_WAIT_MIN, float(wait_seconds or 0)))
given = next((item for item in answer["answers"] if item["question"] == "answer"), None)
if not given:
if answer.get("messages") and answer.get("still_waiting"):
# Read the user's new instruction before considering an automatic choice.
answer.update(ai_answer_wait.wait_on(panel_id, allow_recommended=False))
return {"answered": False, "panel": panel_id, **answer}
except ai_panels.PanelError as error:
raise ToolError(str(error))
out = {"answered": True, "panel": panel_id, "next": "Go on with the user's answer; log_decision it when it settles a choice."}
if answer.get("messages"):
out["messages"] = answer["messages"]
if given.get("inpaint_image"):
out.update(inpaint_image=given["inpaint_image"], inpaint_note=given["note"])
elif given.get("note"):
out["next"] = given["note"]
if given.get("skipped"):
out.update(skipped=True, note="The user left this to you: decide it yourself, log_decision what you chose and "
"go on. Do not ask the same again.")
elif given.get("text"):
out["text"] = given["text"]
if given.get("choice") == ["other"]:
out["note"] = "None of your options: the user wrote their own answer."
else:
out.update(choice=given.get("choice"), labels=given.get("labels"))
if given.get("recommended_on_timeout"):
out.update(recommended_on_timeout=True, next=ai_answer_wait.TOOK)
return out
def update_panel(ctx, panel, title=None, blocks=None, closed=None, task=None):
del ctx, task
try:
updated = ai_panels.update(panel, title, blocks, closed)
except ai_panels.PanelError as error:
raise ToolError(str(error))
return {"updated": True, "blocks": [{"id": block.get("id"), "type": block["type"]}
for block in updated["blocks"]]}
def run_status(ctx, wait_seconds=0, task=None):
try:
return ai_task_runs.status(ctx.get("client") or "AI", ctx.get("session") or "", task or ctx.get("task"), wait_seconds)
except ValueError as error:
raise ToolError(str(error))
def show_run_map(ctx, title, run=None, task=None):
client, session = ctx.get("client") or "AI", ctx.get("session") or ""
task_id = ai_tasks.active_task_id(client, session, task or ctx.get("task"))
runs = [step for step in (ai_tasks.get_task(task_id) or {}).get("steps", []) if step.get("tool") == "run_workflow"] if task_id else []
found = next((step for step in reversed(runs) if step.get("map") and (not run or step["id"] == run)), None)
if not found:
raise ToolError("No run of this task has its shape yet: it is read once ComfyUI has queued the run (run_workflow).")
try:
panel = ai_panels.show(client, title, [{"type": "run_map", "step": found["id"]}], session=session, task_id=task_id,
tool="show_run_map")
except ai_panels.PanelError as error:
raise ToolError(str(error))
boxes = found["map"]["boxes"]
return {"panel": panel["id"], "shown": True, "run": found["id"], "changes": found.get("changes") or [],
"boxes": [{"node_id": box["id"], "title": box["title"], "role": box["role"]} for box in boxes]}
def stop_run(ctx, task=None):
client, session = ctx.get("client") or "AI", ctx.get("session") or ""
task_id = ai_tasks.active_task_id(client, session, task or ctx.get("task"))
run = next((step for step in reversed((ai_tasks.get_task(task_id) or {}).get("steps", []))
if step.get("tool") == "run_workflow"), None) if task_id else None
if not run:
raise ToolError("No run in this task yet.")
try:
done = ai_run_control.stop(run["id"], by="ai", task_id=task_id)
except ai_run_control.RunControlError as error:
raise ToolError(str(error))
return {**done, "stopped": True, "note": "Stopping: run_status(wait_seconds=10) confirms it ended."}
def get_toolbox(ctx, task=None):
from .mcp_canvas_edit import find_nodes
task_id = task or ctx.get("task") or ai_tasks.active_task_id(ctx.get("client") or "AI", ctx.get("session") or "")
box = ai_toolbox.effective(ai_tasks.get_task(task_id)) if task_id else None
if not box:
return {"picked": False, "note": "The user picked nothing for this task; use what fits."}
answer = {"picked": True, "mode": box["mode"], "note": ai_toolbox.words(box)}
if box.get("nodes"):
found = find_nodes(ctx, types=box["nodes"])
answer["nodes"] = found["nodes"]
if found.get("not_found"):
answer["page_only_nodes"] = found["not_found"] # helpers that live only in the page: add_node takes them
if box.get("packs"):
from .mcp_canvas_edit import object_info
info = object_info(ctx)
answer["packs"] = []
for pack in box["packs"]:
types = sorted(name for name, item in info.items() if ai_toolbox.source_of(item) == pack)
answer["packs"].append({"pack": pack, "count": len(types), "node_types": [
{"type": name, "display_name": info[name].get("display_name") or name, "category": info[name].get("category") or ""}
for name in types[:PACK_TYPES]]})
answer["packs_note"] = "Every node type of a pack is the user's pick; find_nodes(types=[...]) gives one in full."
for kind in ("combos", "workflows", "prompts"):
if box.get(kind):
answer[kind] = [found for name in box[kind] if (found := ai_message_refs.lookup(kind, name)) is not None]
if box.get("models"):
answer["models"] = [ai_message_refs.lookup("models", model_id) or {"model_id": model_id, "missing": True} for model_id in box["models"]]
return answer
def get_answers(ctx, panel=None, wait_seconds=0, task=None):
try:
client = ctx.get("client") or "AI"
task_id = task or ctx.get("task") or None
if not task_id and ctx.get("session"):
task_id = ai_tasks.active_task_id(client, ctx["session"]) or ""
answer = ai_answer_wait.collect(client, panel, wait_seconds, task_id=task_id)
return answer, ai_marks.image_items(answer.get("messages") or []) # marked pictures, as pictures
except ai_panels.PanelError as error:
raise ToolError(str(error))
BLOCKS_HELP = (
"blocks, top to bottom, each {type, ...}: heading {text}; text {text, tone: normal|muted|tip|warning}; "
"images {items: [{image: 'subfolder/name.png' from search_images, caption, marks: [{x, y, r | w, h, label, "
"color: red|green|blue|yellow|purple}]}], layout: row|grid|compare} (marks: circles or boxes with a label, x/y/w/h/r "
"as fractions 0-1 of the picture, x/y the centre; a click on the label zooms there); "
"slider {before: {image}, after: {image}, before_label, after_label} (one picture over the other, a line to drag); "
"picks {id, question, items: [{id, image, label, reason, recommended}] (2-12), multiple} (pictures to choose from, "
"each with 'take this one'; the user can also put a picture's workflow back on the canvas); "
"diagram {nodes: [{id, label, detail, node_id, color}], edges: [{from, to, label}], direction: right|down} (boxes and "
"arrows Anomalous lays out; node_id from describe_canvas makes a box find that node); "
"models {items: [{model_id from search_models, note}]} (cards with their covers, a click opens them); "
"table {columns: [...], rows: [[...]]}; list {items: [...], ordered}; params {items: [{label, value}]}; "
"choices {id, question, options: [{id, label, detail}] (2-8), multiple} (buttons the user presses); "
"input {id, question, placeholder} (a line the user types); progress {label, value, max} (your own "
"numbers: they never move by themselves, so keep them current with update_panel; a run's real progress shows "
"in the workbench already, draw none for it); divider {}. "
"Each block may have a stable id; the tool returns block ids in display order. "
"Plain text only (no markup, no links); up to 30 blocks, 12 pictures or models a block."
)
_TASK_PROP = {"type": "string", "description": "begin_task's id (optional)."}
def _task_tool(name, title, description, run, properties, required=()):
return name, {"run": run, "context": True, "spec": {
"name": name, "title": title, "description": description,
"inputSchema": _schema(properties, required),
"annotations": {"title": title, "readOnlyHint": name in ("run_status", "get_toolbox"), "destructiveHint": False,
"idempotentHint": False, "openWorldHint": False},
}}
def _question_with_images(run):
"""The question receipt also carries pictures the user sent while it was waiting."""
def call(ctx, **arguments):
result = run(ctx, **arguments)
return result, ai_marks.image_items(result.get("messages") or [])
return call
TASK_TOOLS = dict([
_task_tool("track_run", "Track an externally submitted run",
"Immediately after YOU submit to this ComfyUI server via /prompt or another tool, attach the exact "
"prompt_ids returned by your submissions to your task. This does NOT submit or rerun anything. "
"The workbench then shows real queue/node progress and results. Never use another user's IDs or "
"guess from queue differences. Repeating the same IDs is safe. Waits up to wait_seconds and returns "
"state, summary, pictures and errors; if still running, keep waiting with run_status(wait_seconds=55).",
ai_run_tracking.track_run, {"prompt_ids": {"type": "array", "items": {"type": "string"}, "minItems": 1, "maxItems": 8},
"task": _TASK_PROP, "wait_seconds": {"type": "number", "minimum": 0, "maximum": 55, "default": 15}}, ["prompt_ids"]),
_task_tool("begin_task", "Begin a task",
"Call first: names what the user asked for in Anomalous's AI workbench, where they see, open and undo "
"each of your steps. Continues the waiting or active task; new_task=true only for a separate request.",
begin_task, {"title": {"type": "string", "description": "A few words, in the user's language."},
"new_task": {"type": "boolean", "default": False}},
["title"]),
_task_tool("report", "Tell the user",
"A note in the AI workbench, where the user reads your work: your choices, what to look at, what is "
"left, your answers. Markdown headings, lists, bold and code are laid out. reply_to: the id of the "
"user's message you answer. images: output pictures (subfolder/name.png) to keep with the task. "
"done=true ends it.",
report, {"text": {"type": "string"},
"images": {"type": "array", "items": {"type": "string"}, "maxItems": MAX_LIMIT},
"done": {"type": "boolean", "default": False},
"reply_to": {"type": "string", "description": "The message id you answer."},
"task": _TASK_PROP}),
_task_tool("log_outside", "Say what you did outside Anomalous",
"Right after using a tool that is not Anomalous's (a command, a script, files, another server): what "
"you ran, why, and the files it touched, for the task's timeline. Prefer Anomalous's tools.",
log_outside, {"what": {"type": "string"}, "why": {"type": "string"},
"files": {"type": "array", "items": {"type": "string"}, "maxItems": ai_outside.MAX_FILES},
"task": _TASK_PROP},
["what"]),
_task_tool("log_decision", "Record a decision",
"Records a settled decision of the task (a style, a model, a fix, what to drop) where the user looks "
"back on it: title, decision, reason, rejected options, by (user, ai or together), related step ids. "
"The user's own choices in Anomalous are kept already.",
log_decision, {"title": {"type": "string"}, "decision": {"type": "string"}, "reason": {"type": "string"},
"rejected": {"type": "array", "items": {"type": "string"}, "maxItems": ai_decisions.MAX_REJECTED},
"by": {"type": "string", "enum": list(ai_decisions.BY), "default": "user"},
"related": {"type": "array", "items": {"type": "string"}, "maxItems": ai_decisions.MAX_RELATED},
"task": _TASK_PROP},
["title", "decision"]),
_task_tool("show_panel", "Show the user a panel",
"Draws a panel in the AI workbench, in Anomalous's own style, from blocks: to compare pictures side by "
"side, present models with their covers, lay out settings, a table, and to ask (buttons or a line of "
"text). Use a new panel for each new phase (plan, workflow, then results), in the order they happen. "
"Use it when a choice is the user's, or to show results better than words do. " + BLOCKS_HELP,
show_panel, {"title": {"type": "string"},
"blocks": {"type": "array", "items": {"type": "object"}, "maxItems": ai_panels.MAX_BLOCKS},
"task": _TASK_PROP},
["title", "blocks"]),
_task_tool("ask_user", "Ask the user",
"Asks the user in the AI workbench and waits: a question with 2-8 options ({label, detail, "
"recommended}; mark your suggestion recommended=true), and a line for their own answer unless "
"other=false; no options: a free answer. Use it for every choice that is the user's, not your chat's "
f"question form. Waits at least {ASK_WAIT_MIN} s, up to wait_seconds; unanswered and keep_waiting: wait on "
"with get_answers. End every round of work with it.",
_question_with_images(ask_user), {"question": {"type": "string"},
"options": {"type": "array", "maxItems": 8, "items": {"type": "object", "properties": {
"label": {"type": "string"}, "detail": {"type": "string"}, "recommended": {"type": "boolean"}}}},
"multiple": {"type": "boolean", "default": False}, "other": {"type": "boolean", "default": True},
"wait_seconds": {"type": "number", "minimum": 0, "maximum": ai_panels.MAX_WAIT, "default": ai_panels.MAX_WAIT},
"task": _TASK_PROP},
["question"]),
_task_tool("propose_repaint_area", "Agree on the area to repaint",
"Before repainting only the marked areas: the user's marks are rough, so show them the area you would "
"repaint. marks: the marks' message id. areas: boxes or ellipses {shape, x, y, w, h} as fractions of the "
"picture (x, y its top left), or none to keep the marks; grow: pixels added around; feather: pixels of "
"soft edge. The user sees theirs and yours side by side and picks one, or marks again; with an area "
"picked the answer has inpaint_image (the picture with that area as its alpha, in ComfyUI's input "
"folder) and inpaint_note. Build the repaint yourself. Waits like ask_user.",
_question_with_images(propose_repaint_area), {"title": {"type": "string", "description": "A few words, in the user's language."},
"marks": {"type": "string"},
"areas": {"type": "array", "maxItems": ai_repaint.MAX_AREAS, "items": {"type": "object", "properties": {
"shape": {"type": "string", "enum": list(ai_repaint.SHAPES)}, "x": {"type": "number"},
"y": {"type": "number"}, "w": {"type": "number"}, "h": {"type": "number"}}}},
"grow": {"type": "integer", "minimum": 0, "maximum": ai_repaint.MAX_GROW, "default": 12},
"feather": {"type": "integer", "minimum": 0, "maximum": ai_repaint.MAX_FEATHER, "default": 8},
"note": {"type": "string", "description": "Why you moved it, in a line."},
"wait_seconds": {"type": "number", "minimum": 0, "maximum": ai_panels.MAX_WAIT, "default": ai_panels.MAX_WAIT}},
["title", "marks"]),
_task_tool("update_panel", "Change a panel",
"Adds new output at the BOTTOM of an existing panel, preserving earlier content and its order. "
"Send only the new blocks; repeated unchanged blocks are kept once. To deliberately correct an "
"existing block or update a progress bar/question in place, include its returned block id and "
"the same type. Omitted blocks are retained, not deleted. Never reuse an old block id for a new "
"result: append it or use show_panel for the next phase. Up to 30 total blocks per panel. "
"Can also change title, or set closed=true to stop answers; existing answers stay.",
update_panel, {"panel": {"type": "string"}, "title": {"type": "string"},
"blocks": {"type": "array", "items": {"type": "object"}, "maxItems": ai_panels.MAX_BLOCKS},
"closed": {"type": "boolean"}, "task": _TASK_PROP}, ["panel"]),
_task_tool("run_status", "How the run is going",
"Your task's latest run: none, queued (runs ahead), running (node, steps, percent), done (its "
"pictures; the first answer telling you it ended attaches up to 4) or failed; summary says it in a "
"sentence. wait_seconds (up to 55) waits for the end. Call it instead of guessing or looking in "
"folders.",
run_status, {"wait_seconds": {"type": "number", "minimum": 0, "maximum": ai_task_runs.MAX_STATUS_WAIT, "default": 0},
"task": _TASK_PROP}),
_task_tool("show_run_map", "Show what the run really was",
"Puts the shape of your task's latest run (or `run`, a run step id) on the user's stage: the nodes it "
"really ran as boxes (models, LoRAs with weights, prompts, size, sampler settings, saving), linked, "
"each one locates its node on the canvas, with what changed since the run before. Anomalous reads it "
"from the prompt ComfyUI queued, so use this instead of drawing the workflow yourself; add your "
"explanation with report. title: a few words in the user's language.",
show_run_map, {"title": {"type": "string"}, "run": {"type": "string"}, "task": _TASK_PROP}, ["title"]),
_task_tool("stop_run", "Stop the run",
"Stops your task's run: its queued prompts leave the queue, the running one is interrupted (never "
"anyone else's). When run_status warns it is far too big and the user agrees, or when the user asks.",
stop_run, {"task": _TASK_PROP}),
_task_tool("get_toolbox", "What the user picked for you",
"The node types, whole custom node packs, combos, saved workflows, prompts and models the user picked in "
"the AI workbench for this task, in full (inputs and outputs of the nodes, each pack's node types, the "
"combos' models and prompts…), and how strictly: prefer (use them first) or only (add_node and "
"new_workflow accept only those node types and the packs' besides ComfyUI's built-in ones). "
"begin_task says when there are any.",
get_toolbox, {"task": _TASK_PROP}),
_task_tool("get_answers", "Read the user's answers and messages",
"What the user picked or typed on your panels and wrote to you in the AI workbench since you last "
"asked, each once (kind=change: something they changed themselves; keep it). panel: one panel. "
"wait_seconds (up to 55) waits when nothing is there. still_waiting: questions not answered, and "
"keep_waiting=true: call it again; skipped=true: they left it to you, decide yourself.",
get_answers, {"panel": {"type": "string"},
"wait_seconds": {"type": "number", "minimum": 0, "maximum": ai_panels.MAX_WAIT, "default": 0},
"task": _TASK_PROP}),
])
+146 -61
View File
@@ -1,6 +1,7 @@
"""The MCP endpoint's (mcp_server.py) reading tools: views of the library (mcp_actions.py acts).
`call_tool` runs any tool of a registry, these and mcp_actions.py's.
`call_tool` runs any tool of a registry, these and mcp_actions.py's. search_models and get_model
are offered through mcp_model_use.py, which adds what loads each model.
Each tool is a plain function over the same stores the browser's pages use (model catalog
and sidecars, output gallery, combos, workflow recipes, saved prompts, generated audio, scan
@@ -19,6 +20,7 @@ import os
import re
import folder_paths
from PIL import Image
from .activity_log import list_entries
from .audio_catalog import _gallery_page as _audio_page
@@ -28,6 +30,7 @@ from .kept_images import _kept_images
from .material_store import _list_materials, _read_material, get_materials_dir
from .media_routes import _build_card_thumbnail
from .metadata import get_metadata
from .mcp_arguments import prepare_arguments, renaming_note
from .model_catalog import _iter_search_models
from .model_constants import MEDIA_EXTENSIONS, MODEL_EXTENSIONS, PREVIEW_SUFFIXES
from .notebooks import _list_notebooks
@@ -37,6 +40,7 @@ from .scan_report import last_scan_path
from .scan_summary import _summarize
MAX_LIMIT = 50
MAX_SEARCH_PICTURES = 6 # search_images(with_pictures): the first few attached
MAX_TEXT = 1500
MAX_IMAGE_BYTES = 1_500_000
IMAGE_TYPES = {".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".webp": "image/webp"}
@@ -110,10 +114,50 @@ def _resolve_image(image):
except ValueError:
raise ToolError("image is a path under ComfyUI's output folder, as search_images returns it.")
if os.path.splitext(path)[1].lower() not in IMAGE_TYPES or not os.path.isfile(path):
raise ToolError(f"No output image at {image}.")
raise ToolError(f"No output image at {image}: it is not (or no longer) in ComfyUI's output folder; "
"the user may have deleted it. search_images lists the pictures there.")
return rel, path
def _resolve_folder_image(image, folder):
"""(rel, path) of picture `image` ('subfolder/file.png') in ComfyUI's output, input (where
LoadImage reads) or temp folder (where PreviewImage writes), checked to stay inside it."""
if folder == "output":
return _resolve_image(image)
if folder not in ("input", "temp"):
raise ToolError("folder is output, input or temp.")
rel = str(image or "").replace("\\", "/").strip("/")
try:
path = resolve_within(getattr(folder_paths, f"get_{folder}_directory")(), rel)
except ValueError:
raise ToolError(f"image is a path under ComfyUI's {folder} folder.")
if os.path.splitext(path)[1].lower() not in IMAGE_TYPES or not os.path.isfile(path):
raise ToolError(f"No picture at {image} in ComfyUI's {folder} folder.")
return rel, path
def _input_facts(path):
"""Size of an input or preview picture and, when it has an alpha, the area LoadImage gives as MASK (1 - alpha)."""
try:
with Image.open(path) as picture:
facts = {"width": picture.width, "height": picture.height}
if "A" not in picture.getbands():
facts["mask"] = "none: no alpha, so LoadImage's MASK is empty"
return facts
area = picture.getchannel("A").point(lambda value: 255 if value < 128 else 0)
except (OSError, ValueError, Image.DecompressionBombError):
return {"note": "The picture cannot be read."}
box = area.getbbox()
if not box:
facts["mask"] = "none: fully opaque, so LoadImage's MASK is empty"
return facts
share = area.histogram()[255] / (facts["width"] * facts["height"])
left, top, right, bottom = (round(100 * value / size) for value, size in zip(box, (facts["width"], facts["height"]) * 2))
facts["mask"] = {"share": f"{round(100 * share, 1)}%", "area": f"x {left}-{right}%, y {top}-{bottom}% of the picture",
"meaning": "LoadImage gives this area as MASK: what to repaint"}
return facts
def _image_content(path):
"""An MCP image item: the 512 px thumbnail the gallery uses, or None when too big or unreadable."""
try:
@@ -172,8 +216,9 @@ def _each_model():
def search_models(query="", type="", base_model="", source="", limit=20):
terms, wanted_base, found = _terms(query), str(base_model or "").lower(), []
total = 0
type = str(type or "").lower()
for folder_type, path_idx, base_dir, file_path in _each_model():
if type and folder_type != type:
if type and type not in (folder_type, folder_type.rstrip("s")): # "checkpoint" finds the checkpoints too
continue
meta = get_metadata(file_path)
if wanted_base and wanted_base not in str(meta.get("baseModel") or "").lower():
@@ -196,6 +241,7 @@ def get_model(model_id, include_cover=False):
detail.update({
"trigger_words": list(meta.get("trainedWords") or []),
"civitai_name": meta.get("name") or "",
"version": meta.get("version_name") or "",
"civitai_url": meta.get("civitai_url") or "",
"download_link": meta.get("source_url") or "",
"user_notes": meta.get("custom_notes") or "",
@@ -256,7 +302,7 @@ def scan_report():
# ---------------------------------------------------------------- outputs
def search_images(query="", limit=20, page=1):
def search_images(query="", limit=20, page=1, with_pictures=False):
images = _gallery_listing(folder_paths.get_output_directory(), False, str(query or "").strip())
limit = _limit(limit)
try:
@@ -265,7 +311,7 @@ def search_images(query="", limit=20, page=1):
page = 1
kept = _kept_lookup()
chosen = images[(page - 1) * limit:page * limit]
return {
found = {
"total_matches": len(images), "page": page,
"images": [{"image": "/".join(part for part in (item["subfolder"], item["filename"]) if part),
"created": _when(item.get("mtime")),
@@ -273,9 +319,25 @@ def search_images(query="", limit=20, page=1):
if (item["subfolder"], item["filename"]) in kept else {})}
for item in chosen],
}
if not with_pictures:
return found
pictures = [] # the first few themselves, in the order listed, so an AI that sees pictures need not ask for each
for entry in found["images"][:MAX_SEARCH_PICTURES]:
item = _image_content(_resolve_image(entry["image"])[1])
entry["attached"] = bool(item)
if item:
pictures.append(item)
return found, pictures
def get_image_info(image, include_image=False):
def get_image_info(image, include_image=False, folder="output"):
if folder != "output":
rel, path = _resolve_folder_image(image, folder)
info = {"image": rel, "folder": folder, "created": _when(os.path.getmtime(path)), **_input_facts(path)}
extra = [item for item in [_image_content(path)] if include_image and item]
if include_image:
info["image_attached"] = bool(extra)
return info, extra
rel, path = _resolve_image(image)
info = {"image": rel, "created": _when(os.path.getmtime(path))}
chunks = read_png_text(path) if path.lower().endswith(".png") else {}
@@ -332,40 +394,47 @@ def _model_name(model):
return str(model or "")
def list_combos(query="", limit=20):
def combo_view(note):
"""A combo (one of _list_notebooks) as the AI and the materials picker see it, with its LoRAs and texts."""
data = note.get("data") or {}
if data.get("kind") == "nodes": # its own node structure: model and text slots by their names
structure = data.get("structure") or {}
values = data.get("values") or {}
slots = structure.get("slots") or []
models = [{"slot": s.get("label") or s.get("widget"), "folder": s.get("folder") or "",
"file": os.path.basename(str(values.get(s["id"]) or "").replace("\\", "/"))}
for s in slots if s.get("kind") == "model"]
texts = [{"slot": s.get("label") or s.get("widget"), "text": _text(values.get(s["id"]), 400)}
for s in slots if s.get("kind") == "text"]
combo = {"name": note.get("name"), "kind": "node structure", "models": models, "texts": texts,
"nodes": [n.get("name") or n.get("type") for n in structure.get("nodes") or []],
"main_model": models[0]["file"] if models else "", "prompt": texts[0]["text"] if texts else ""}
return combo, [model["file"] for model in models[1:]] + [text["text"] for text in texts[1:]]
loras = [_model_name(lora) for lora in data.get("loras") or []]
return {"name": note.get("name"), "base_model": data.get("baseModel") or "", "main_model": _model_name(data.get("mainModel")),
"loras": loras, "prompt": _text(data.get("promptEn"), 800)}, loras
def list_combos(query="", limit=20, *, exact_name=False):
terms, found = _terms(query), []
for note in _list_notebooks():
data = note.get("data") or {}
if data.get("kind") == "nodes": # its own node structure: model and text slots by their names
structure = data.get("structure") or {}
values = data.get("values") or {}
slots = structure.get("slots") or []
models = [{"slot": s.get("label") or s.get("widget"), "folder": s.get("folder") or "",
"file": os.path.basename(str(values.get(s["id"]) or "").replace("\\", "/"))}
for s in slots if s.get("kind") == "model"]
texts = [{"slot": s.get("label") or s.get("widget"), "text": _text(values.get(s["id"]), 400)}
for s in slots if s.get("kind") == "text"]
combo = {"name": note.get("name"), "kind": "node structure", "models": models, "texts": texts,
"nodes": [n.get("name") or n.get("type") for n in structure.get("nodes") or []],
"main_model": models[0]["file"] if models else "", "prompt": texts[0]["text"] if texts else ""}
loras = [model["file"] for model in models[1:]] + [text["text"] for text in texts[1:]]
else:
loras = [_model_name(lora) for lora in data.get("loras") or []]
combo = {"name": note.get("name"), "base_model": data.get("baseModel") or "",
"main_model": _model_name(data.get("mainModel")), "loras": loras,
"prompt": _text(data.get("promptEn"), 800)}
if not _matches(terms, combo["name"], combo["main_model"], combo["prompt"], " ".join(loras)):
combo, loras = combo_view(note)
if exact_name and combo["name"] != query:
continue
if not exact_name and not _matches(terms, combo["name"], combo["main_model"], combo["prompt"], " ".join(loras)):
continue
found.append(combo)
return {"total_matches": len(found), "combos": found[:_limit(limit)]}
def list_workflows(query="", limit=20):
def list_workflows(query="", limit=20, *, exact_name=False):
terms, found = _terms(query), []
for recipe in _list_recipes(get_recipes_dir()):
data = recipe.get("data") or {}
params = json.dumps(data.get("params") or {}, ensure_ascii=False)
if not _matches(terms, data.get("name"), " ".join(data.get("tags") or []), data.get("notes"), params):
if exact_name and (data.get("name") or recipe["filename"]) != query:
continue
if not exact_name and not _matches(terms, data.get("name"), " ".join(data.get("tags") or []), data.get("notes"), params):
continue
found.append({"name": data.get("name") or recipe["filename"], "tags": data.get("tags") or [],
"notes": _text(data.get("notes"), 500), "saved": _when((data.get("timestamp") or 0) / 1000),
@@ -373,8 +442,8 @@ def list_workflows(query="", limit=20):
return {"total_matches": len(found), "workflows": found[:_limit(limit)]}
def list_saved_prompts(query="", limit=20):
terms, found = _terms(query), []
def saved_prompts():
"""Every saved prompt plan: {name, tags, positive, negative}."""
directory = get_materials_dir()
for summary in _list_materials(directory):
if summary.get("kind") != "prompt_plan" or summary.get("moved_to_recipe"):
@@ -383,14 +452,18 @@ def list_saved_prompts(query="", limit=20):
plan = _read_material(os.path.join(directory, summary["filename"])).get("plan") or {}
except (OSError, ValueError, KeyError):
continue
prompt = {"name": summary.get("name"), "tags": summary.get("tags") or [],
"positive": _text(plan.get("positive"), 1200), "negative": _text(plan.get("negative"), 600)}
if _matches(terms, prompt["name"], " ".join(prompt["tags"]), prompt["positive"], prompt["negative"]):
found.append(prompt)
yield {"name": summary.get("name"), "tags": summary.get("tags") or [],
"positive": _text(plan.get("positive"), 1200), "negative": _text(plan.get("negative"), 600)}
def list_saved_prompts(query="", limit=20):
terms = _terms(query)
found = [prompt for prompt in saved_prompts()
if _matches(terms, prompt["name"], " ".join(prompt["tags"]), prompt["positive"], prompt["negative"])]
return {"total_matches": len(found), "prompts": found[:_limit(limit)]}
def recent_activity(limit=20, source=""):
def recent_activity(limit=10, source=""):
entries = list_entries(_limit(limit), source if source in ("canvas", "file") else None)
return {"entries": [{"time": _when(entry.get("time")), "source": entry.get("source"), "action": entry.get("action"),
"target": entry.get("target") or "", "detail": entry.get("detail") or {}} for entry in entries]}
@@ -405,7 +478,7 @@ def _schema(properties=None, required=()):
return schema
_QUERY = {"type": "string", "description": "Words that must all appear (case-insensitive); empty lists everything."}
_QUERY = {"type": "string", "description": "Words that must all appear; empty lists all."}
_LIMIT = {"type": "integer", "minimum": 1, "maximum": MAX_LIMIT, "default": 20}
@@ -422,32 +495,21 @@ TOOLS = dict([
_tool("library_overview", "Library overview",
"How many models of each type and base model the user has, how many are scanned, and how many "
"combos, workflow recipes and saved prompts they kept. A good first call.", library_overview),
_tool("search_models", "Search models",
"Find model files (checkpoints, LoRAs, VAEs…) by name, trigger word or base model. Returns model_id for get_model.",
search_models, {
"query": _QUERY,
"type": {"type": "string", "description": "ComfyUI models folder, e.g. checkpoints, loras, vae, diffusion_models."},
"base_model": {"type": "string", "description": "Part of the base model name, e.g. SDXL, Pony, Flux, SD 1.5."},
"source": {"type": "string", "enum": ["civitai", "inferred_from_file", "not_scanned"],
"description": "Where the model's information came from."},
"limit": _LIMIT}),
_tool("get_model", "Model details",
"Everything known about one model: Civitai name and link, trigger words, base model, description, "
"the user's own name and notes. include_cover attaches its cover picture.",
get_model, {"model_id": {"type": "string", "description": "From search_models."},
"include_cover": {"type": "boolean", "default": False}}, ["model_id"]),
_tool("scan_report", "Scan status",
"Which models are matched on Civitai, which are not and why, which were never scanned, and what the last scan did.",
scan_report),
_tool("search_images", "Search output images",
"Find images ComfyUI generated, newest first, by words in their prompt, model or settings (e.g. a LoRA "
"name or 'seed:123'). Returns image paths for get_image_info.",
search_images, {"query": _QUERY, "limit": _LIMIT, "page": {"type": "integer", "minimum": 1, "default": 1}}),
"Output images, newest first, by words in their prompt, model or settings (a LoRA name, 'seed:123'). "
"with_pictures attaches the first 6 (512 px).",
search_images, {"query": _QUERY, "limit": _LIMIT, "page": {"type": "integer", "minimum": 1, "default": 1},
"with_pictures": {"type": "boolean", "default": False}}),
_tool("get_image_info", "Image generation settings",
"The nodes and settings that produced an output image (prompts, models, seed, sampler…) and whether the "
"user kept it as a recipe, combo or prompt. include_image attaches a 512 px copy.",
get_image_info, {"image": {"type": "string", "description": "From search_images, e.g. 'ComfyUI_00012_.png'."},
"include_image": {"type": "boolean", "default": False}}, ["image"]),
"How an output image was made (prompts, models, seed, sampler…) and whether the user kept it. "
"folder=input: a picture LoadImage reads; folder=temp: a run's preview; both with their size and MASK "
"area. include_image attaches a 512 px copy; zoom_image shows a part larger.",
get_image_info, {"image": {"type": "string", "description": "From search_images; for folder=input, the name LoadImage takes."},
"include_image": {"type": "boolean", "default": False},
"folder": {"type": "string", "enum": ["output", "input", "temp"], "default": "output"}}, ["image"]),
_tool("list_combos", "Combos",
"The user's combos (搭配): a main model, LoRAs and a prompt kept together, or (kind \"node "
"structure\") model nodes and text nodes saved from the canvas with their links, listed by slot.", list_combos,
@@ -463,19 +525,28 @@ TOOLS = dict([
{"query": _QUERY, "limit": _LIMIT}),
_tool("recent_activity", "Activity log",
"What Anomalous changed recently, newest first: canvas edits it made and files it saved, scanned or deleted.",
recent_activity, {"limit": _LIMIT, "source": {"type": "string", "enum": ["canvas", "file"]}}),
recent_activity, {"limit": {**_LIMIT, "default": 10}, "source": {"type": "string", "enum": ["canvas", "file"]}}),
])
def tool_text(data):
"""A result as the AI reads it: JSON without spaces. Apps keep a long one out of the chat and
have the AI open it as a file (Antigravity past about 4,000 characters), so every character counts."""
return json.dumps(data, ensure_ascii=False, separators=(",", ":"))
def call_tool(tools, name, arguments, ctx=None):
"""An MCP tools/call result for tool `name` of `tools` (run in a worker thread). Tools marked
`context` get `ctx` ({base_url} of this server) first."""
tool = tools[name]
spec = tool["spec"]["inputSchema"]
arguments, renamed = prepare_arguments(spec, arguments)
unknown = set(arguments) - set(spec["properties"])
missing = [key for key in spec.get("required", []) if key not in arguments]
if unknown or missing:
return _failure(f"Unknown arguments {sorted(unknown)}" if unknown else f"Missing arguments {missing}")
wrong = "; ".join(part for part in (unknown and f"Unknown arguments {sorted(unknown)}", missing and f"missing {missing}") if part)
takes = ", ".join(key + (" (needed)" if key in spec.get("required", []) else "") for key in spec["properties"] if key != "task")
return _failure(f"{wrong}. {name} takes: {takes or 'nothing'}.")
try:
value = tool["run"](ctx, **arguments) if tool.get("context") else tool["run"](**arguments)
except ToolError as error:
@@ -484,9 +555,23 @@ def call_tool(tools, name, arguments, ctx=None):
logging.getLogger(__name__).warning("Anomalous MCP tool %s failed", name, exc_info=True)
return _failure(f"{name} failed: {error}")
data, extra = value if isinstance(value, tuple) else (value, [])
return {"content": [{"type": "text", "text": json.dumps(data, ensure_ascii=False)}, *extra],
if renamed and isinstance(data, dict):
data = {**data, "arguments_note": renaming_note(name, renamed)}
return {"content": [{"type": "text", "text": tool_text(data)}, *extra],
"structuredContent": data, "isError": False}
def retext(result, data):
"""A tool result with `data` as its structured content and as its text (the rest of its content kept)."""
result["structuredContent"] = data
content = list(result.get("content") or [])
text = {"type": "text", "text": tool_text(data)}
if content and isinstance(content[0], dict) and content[0].get("type") == "text":
content[0] = text
else:
content.insert(0, text)
result["content"] = content
def _failure(message):
return {"content": [{"type": "text", "text": message}], "isError": True}
+182
View File
@@ -0,0 +1,182 @@
"""What Anomalous knows about using a model well, told to the AI so it need not guess:
- the settings its author's example pictures were made with (the Civitai record a scan keeps
beside the file: images[].meta), summed up for get_model: how many pictures, the sampler and
scheduler most used, the range of steps and CFG, the most used size and, for a LoRA, the
weight it was used at: by its file hash, name or Civitai version, else the LoRA most of the
pictures name (their names are the author's own files, and often an older version's);
- whether a LoRA suits the main model it is put on: add_lora and set_model compare their base
models (the scan's, else what the file's header tells) and say so when the families differ
(an SD 1.5 LoRA on an SDXL checkpoint does nothing useful) or, more softly, when two bases of one
family differ (a Pony LoRA on an Illustrious checkpoint). A note only: the call is made.
"""
import json
import os
from collections import Counter
import folder_paths
from . import metadata
# Base model names (Civitai's, or what model_kind.py infers) by the architecture their files share.
FAMILIES = (
("SDXL", ("sdxl", "pony", "illustrious", "noob")), ("SD 1.x", ("sd 1",)), ("SD 2.x", ("sd 2",)),
("SD 3", ("sd 3", "sd3")), ("Flux.1", ("flux.1", "flux 1", "chroma")), ("Flux.2", ("flux.2", "flux 2")),
)
# Bases trained far from their family's: a LoRA of one on another of the family loads but may look off.
OWN_WAYS = ("pony", "illustrious", "noob", "chroma")
MAIN_TYPES = ("checkpoints", "diffusion_models", "unet")
MAX_EXAMPLES = 20
def family(base):
"""The architecture family of a base model name ("" when none is known)."""
lowered = str(base or "").strip().lower()
if not lowered or lowered in ("other", "unknown"):
return ""
for name, marks in FAMILIES:
if lowered.startswith(marks):
return name
return lowered
def base_of(file_path):
"""A model file's base model: the scan's, else what its header tells."""
meta = metadata.get_metadata(file_path)
return meta.get("baseModel") or meta.get("base_guess") or ""
def _main_base(value):
"""The base model of a main loader's choice (its drop-down value), or ""."""
for folder_type in MAIN_TYPES:
try:
path = folder_paths.get_full_path(folder_type, value)
except (KeyError, TypeError):
path = None
if path:
return base_of(path)
return ""
def lora_paths(values):
"""The files of LoRA drop-down values that ComfyUI can find."""
paths = []
for value in values or []:
try:
path = folder_paths.get_full_path("loras", value)
except (KeyError, TypeError):
path = None
if path:
paths.append(path)
return paths
def lora_note(lora_path, line):
"""A note when the LoRA at `lora_path` does not suit the main model of `line` (the page's
{"model": the main loader's choice}), else None."""
main = _main_base((line or {}).get("model") or "")
lora = base_of(lora_path)
if not main or not lora or main.lower() == lora.lower():
return None
if family(main) != family(lora):
return (f"Base model clash: this LoRA is for {lora}, the main model ({line['model']}) is {main}; it will not "
f"work as meant. Pick one with search_models(type='loras', base_model='{main}'), or remove it.")
if lora.lower().startswith(OWN_WAYS) or main.lower().startswith(OWN_WAYS):
return (f"This LoRA was made for {lora}, the main model is {main}: same family, so it loads, but it may "
f"look weaker or off. search_models(type='loras', base_model='{main}') finds ones made for it.")
return None
def _number(value):
try:
number = float(str(value).strip())
except (TypeError, ValueError):
return None
return number if number == number and abs(number) < 1e6 else None
def _span(numbers):
"""'25' or '20–30' for a list of numbers, '' for none."""
if not numbers:
return ""
low, high = min(numbers), max(numbers)
say = lambda number: f"{number:g}"
return say(low) if low == high else f"{say(low)}–{say(high)}"
def _common(values, total):
"""'euler (6 of 10)': the most used value and in how many of the `total` pictures."""
counted = Counter(value for value in values if value)
if not counted:
return ""
value, times = counted.most_common(1)[0]
return str(value) if times == total else f"{value} ({times} of {total})"
def _loras(meta):
"""[(who, weight)] of the LoRAs one picture's settings name: who is a hash, a file name or "v<version>"."""
found = []
for item in meta.get("resources") or []:
if isinstance(item, dict) and str(item.get("type", "")).lower() == "lora":
found.append((str(item.get("hash") or item.get("name") or "").lower(), _number(item.get("weight"))))
for item in meta.get("additionalResources") or []:
if isinstance(item, dict) and str(item.get("type", "lora")).lower() == "lora":
found.append((os.path.splitext(os.path.basename(str(item.get("name", ""))))[0].lower(), _number(item.get("strength"))))
for item in meta.get("civitaiResources") or []:
if isinstance(item, dict) and str(item.get("type", "")).lower() == "lora" and item.get("modelVersionId") is not None:
found.append((f"v{item['modelVersionId']}", _number(item.get("weight"))))
return [(who, weight) for who, weight in found if who and weight is not None]
def _lora_weights(metas, own):
"""The weights this LoRA was used at in its pictures: those naming it (`own`: its hash, file
name, version), else the one LoRA most of them name; none when two are named as often."""
per = [_loras(meta) for meta in metas]
named = Counter(who for found in per for who in {who for who, _weight in found})
mine = {who for who in named if who in own}
if not mine:
common = [(who, times) for who, times in named.most_common(2) if times * 5 >= len(metas) * 3]
if common and (len(common) == 1 or common[0][1] > common[1][1]):
mine = {common[0][0]}
return [weight for found in per for who, weight in found if who in mine]
def _record(file_path):
"""The Civitai record kept beside the model (its .info), or None."""
base_path = os.path.splitext(file_path)[0]
info_base = metadata._info_base(file_path, base_path)
for path in (metadata.find(info_base, ".info"), f"{info_base}.civitai.info"):
if path and os.path.isfile(path):
try:
with open(path, encoding="utf-8") as handle:
data = json.load(handle)
except (OSError, ValueError):
continue
if isinstance(data, dict) and isinstance(data.get("images"), list):
return data
return None
def author_examples(file_path, folder_type, sha256=""):
"""What the author's example pictures were made with, summed up; {} when nothing is kept."""
record = _record(file_path)
metas = [image.get("meta") for image in (record or {}).get("images") or []
if isinstance(image, dict) and isinstance(image.get("meta"), dict)][:MAX_EXAMPLES]
if not metas:
return {}
sizes = [meta.get("Size") or (f"{meta['width']}x{meta['height']}" if meta.get("width") and meta.get("height") else "")
for meta in metas]
found = {
"pictures": len(metas),
"sampler": _common([str(meta.get("sampler") or "") for meta in metas if str(meta.get("sampler")).lower() != "undefined"], len(metas)),
"scheduler": _common([str(meta.get("scheduler") or meta.get("Schedule type") or "") for meta in metas], len(metas)),
"steps": _span([n for n in (_number(meta.get("steps")) for meta in metas) if n is not None]),
"cfg": _span([n for n in (_number(meta.get("cfgScale")) for meta in metas) if n is not None]),
"size": _common([str(size) for size in sizes], len(metas)),
"clip_skip": _span([n for n in (_number(meta.get("clipSkip") or meta.get("Clip skip")) for meta in metas) if n is not None]),
}
if folder_type == "loras":
own = {(sha256 or "")[:10].lower(), os.path.splitext(os.path.basename(file_path))[0].lower(), f"v{record.get('id')}"} - {""}
found["lora_weight"] = _span(_lora_weights(metas, own))
return {key: value for key, value in found.items() if value}
+41 -5
View File
@@ -44,7 +44,7 @@ from .model_download import (
from .path_utils import resolve_within
KINDS = ("checkpoints", "loras", "vae", "text_encoders", "diffusion_models", "controlnet", "embeddings",
"clip_vision", "upscale_models", "ipadapter")
"clip_vision", "upscale_models", "ipadapter", "model_patches")
SEARCH_DEPTH = 2
SEARCH_LIMIT = 20000
TIME_SLACK_MS = 2000
@@ -266,17 +266,53 @@ def _same_drive(first, second):
return False
def _rename_new(source, dest):
"""Renames `source` to `dest` and never over a file there, even one that appeared since
free_path looked (FileExistsError). Windows' rename refuses a taken name by itself; elsewhere
a hard link does (of a link itself, not what it points to), with a last look for the file
systems that have none."""
if os.name == "nt":
os.rename(source, dest)
return
try:
os.link(source, dest, follow_symlinks=False)
except FileExistsError:
raise
except OSError: # no hard links here (FAT, some network shares)
if os.path.lexists(dest):
raise FileExistsError(dest)
os.rename(source, dest)
return
os.remove(source)
def _looks_whole(source, copy):
"""Whether `copy` has `source`'s size and its very bytes at the start, the middle and the end
(a megabyte each): a cut or garbled copy shows, at no felt cost, with no fingerprint needed."""
size = os.path.getsize(source)
if os.path.getsize(copy) != size:
return False
with open(source, "rb") as first, open(copy, "rb") as second:
for at in sorted({0, max(0, size // 2 - CHUNK // 2), max(0, size - CHUNK)}):
first.seek(at)
second.seek(at)
if first.read(CHUNK) != second.read(CHUNK):
return False
return True
def move_into(source, dest, keep):
"""Moves (or with `keep` copies) `source` to `dest`; a copy goes through "<dest>.part"."""
"""Moves (or with `keep` copies) `source` to `dest`, never over a file; a copy goes through
"<dest>.part" and is checked (_looks_whole) before the original goes."""
if not keep and _same_drive(source, dest):
os.replace(source, dest)
_rename_new(source, dest)
return
part = dest + ".part"
try:
shutil.copyfile(source, part)
if os.path.getsize(part) != os.path.getsize(source):
if not _looks_whole(source, part):
raise Refused("incomplete")
os.replace(part, dest)
_rename_new(part, dest)
except BaseException:
if os.path.exists(part):
os.remove(part) # our own unfinished copy
+47 -7
View File
@@ -8,11 +8,14 @@ GET /anomalous/placement/check[?deep=1] -> {misplaced, duplicates, unchecked}
duplicates: files with the same SHA-256 — known from scans, else read now for files of
the same size (small ones always, large ones with deep). unchecked: the same-size large
files a deep check would still read.
GET /anomalous/placement/look?type&root&rel -> {kind, sure, base}: what one model's header says
it is, even when unsure (the models page's "Change type", ui_model_retype.js).
POST /anomalous/placement/move {type, root, rel, to_type, to_root, to_rel} -> moves a model
and its sidecars (covers, info, the user's notes); never over a file (" (2)"). Only for
the computer ComfyUI runs on, unless remote changes are allowed (local_guard.may_change).
Removing a copy goes through /anomalous/delete_model (Recycle Bin). Headers and computed
belongs() is the same check for one file (the AI's search_models and move_model,
api/mcp_model_use.py). Removing a copy goes through /anomalous/delete_model (Recycle Bin). Headers and computed
SHA-256s are kept in memory per file (path, size, time).
"""
@@ -116,7 +119,7 @@ def _card(item):
return {"name": name or stem, "version": version, "preview_url": url}
def _home_for(item, kind):
def home_for(item, kind):
"""Where a file of `kind` goes: its own subfolder and name, in a folder of that type,
preferably on its drive (a move is then a rename)."""
roots = []
@@ -136,7 +139,7 @@ def misplaced(files):
kind = result.get("kind")
if not kind or not result.get("sure") or kind == item["type"] or kind not in import_types():
continue
home = _home_for(item, kind)
home = home_for(item, kind)
if home:
found.append({"type": item["type"], "root": item["root"], "rel": item["rel"], "size": item["size"],
"kind": kind, "base": result.get("base", ""), "works": (item["type"], kind) in WORKS_ANYWAY,
@@ -144,6 +147,18 @@ def misplaced(files):
return found
def one(folder_type, root, rel, path):
"""One model file as library() lists it."""
stat = os.stat(path)
return {"type": folder_type, "root": root, "rel": rel, "path": path, "size": stat.st_size, "dev": stat.st_dev, "stat": stat}
def belongs(item):
"""misplaced() of one file (from one()): where it belongs, or None when it is where it goes or its kind is unsure."""
found = misplaced([item])
return found[0] if found else None
def _known_hash(item):
try:
value = str(get_metadata(item["path"]).get("hash") or "").lower()
@@ -206,20 +221,36 @@ def _sidecars(path):
return existing(os.path.join(folder, stem), SIDECAR_SUFFIXES)
def move(data):
folder_type = data.get("type")
def _source(folder_type, root, rel):
"""(root index, clean rel, file) of the model at type `folder_type`, root index `root`, `rel`."""
if folder_type not in import_types():
raise Refused("bad_type")
paths = folder_paths.get_folder_paths(folder_type) or []
try:
index = int(data.get("root", 0))
index = int(root)
if index < 0:
raise IndexError(index)
base = os.path.realpath(paths[index])
rel = clean_folder(data.get("rel"))
rel = clean_folder(rel)
source = model_reference_within(base, rel.replace("/", os.sep))
except (ValueError, TypeError, IndexError):
raise Refused("bad_path")
if not os.path.isfile(source) or not rel.lower().endswith(MODEL_EXTENSIONS):
raise Refused("gone")
return index, rel, source
def look(folder_type, root, rel):
"""What the header of one model says it is: {type, root, rel, kind, sure, base}."""
index, rel, source = _source(folder_type, root, rel)
found = _cached(_heads, source, os.stat(source), inspect_file)
return {"type": folder_type, "root": index, "rel": rel, "kind": found.get("kind") or "",
"sure": bool(found.get("sure")), "base": found.get("base") or ""}
def move(data):
folder_type = data.get("type")
index, rel, source = _source(folder_type, data.get("root", 0), data.get("rel"))
to_type, to_root, root_path, dest = destination(
{"type": data.get("to_type"), "root": data.get("to_root", 0), "rel": data.get("to_rel")}, os.path.getsize(source))
sidecars = _sidecars(source)
@@ -251,6 +282,14 @@ async def api_check(request):
return web.json_response(await asyncio.to_thread(check, deep))
async def api_look(request):
query = request.query
try:
return web.json_response(await asyncio.to_thread(look, query.get("type"), query.get("root", "0"), query.get("rel")))
except Refused as error:
return web.json_response({"error": error.code}, status=404 if error.code == "gone" else 400)
async def api_move(request):
if not may_change(request):
return web.json_response({"error": "not_local"}, status=403)
@@ -270,4 +309,5 @@ async def api_move(request):
def register_routes(app):
app.router.add_get("/anomalous/placement/check", api_check)
app.router.add_get("/anomalous/placement/look", api_look)
app.router.add_post("/anomalous/placement/move", api_move)
+56
View File
@@ -0,0 +1,56 @@
"""ComfyUI's refusal of a run, said as what to do next: run_workflow's answer when ComfyUI does not
take the prompt. The page passes the refusal on as data (web/modules/mcp_run.js `refusal`: its
error and each node's, from ComfyUI's validation); each problem becomes one line naming the node,
what is wrong and the tool that fixes it, so the AI does not have to read ComfyUI's raw report and
guess. Kinds not listed here keep ComfyUI's own message."""
MAX_LINES = 8
NODE_HINTS = {
"required_input_missing": "node {node} ({class_type}) has nothing wired into {input}: connect_nodes into it, or add_node with wire_to={node}",
"bad_linked_input": "node {node} ({class_type}): {input} is wired from an output of the wrong kind; connect_nodes the right one ({details})",
"return_type_mismatch": "node {node} ({class_type}): {input} is wired from an output of the wrong kind; connect_nodes the right one ({details})",
"invalid_input_type": "node {node} ({class_type}): {input} has a value of the wrong kind ({details}); set_values with the right one",
"value_smaller_than_min": "node {node} ({class_type}): {input} = {value} is below its minimum ({details}); set_values within the range",
"value_bigger_than_max": "node {node} ({class_type}): {input} = {value} is above its maximum ({details}); set_values within the range",
"value_not_in_list": ("node {node} ({class_type}): {input} = {value} is not one of its choices. A model this computer "
"lacks: check_workflow_models / fix_workflow_models, or search_models then set_model; otherwise "
"set_values with one of its choices (find_nodes lists them)"),
}
TOP_HINTS = {
"prompt_no_outputs": ("the workflow has no output node: add SaveImage (or PreviewImage) behind the VAE Decode with "
"add_node(type='SaveImage', wire_from=<its id>)"),
"invalid_prompt": "{message} {details}: a node type this ComfyUI does not have; find_nodes for one it has, or ask the user to install its node pack",
}
class _Blank(dict):
def __missing__(self, key):
return "?"
def _say(template, item):
return template.format_map(_Blank({key: str(value) for key, value in item.items() if value not in (None, "")}))
def say(refusal):
"""What to do about ComfyUI's refusal (the page's `refusal` data), or "" when it says nothing."""
if not isinstance(refusal, dict):
return ""
lines = []
for item in refusal.get("nodes") or []:
if isinstance(item, dict):
template = NODE_HINTS.get(item.get("type"))
lines.append(_say(template, item) if template else
_say("node {node} ({class_type}): {message} {details}", item).replace(" ?", ""))
if not lines:
template = TOP_HINTS.get(refusal.get("type"))
if template:
lines.append(_say(template, refusal))
elif refusal.get("message"):
lines.append(_say("{message} {details}", refusal).replace(" ?", ""))
if not lines:
return ""
more = len(lines) - MAX_LINES
return ("ComfyUI did not take the run: " + "; ".join(lines[:MAX_LINES]) + (f"; and {more} more" if more > 0 else "")
+ ". Fix these, then run_workflow again.")
+5
View File
@@ -3,6 +3,11 @@
Start with the repository-root [`ARCHITECTURE.md`](../../ARCHITECTURE.md), then
open only the topic needed for the current task.
- [`ui-flows.md`](ui-flows.md): the UI map's hand-written part, in Chinese: what each part of
a page is for, what a click does and why, and one name per thing (a glossary the structure
check enforces). Where a word on screen is drawn and what it calls comes from the code:
`node tools/ui_map.mjs find <words>` (`tools/ui_map.mjs`, its index `tools/ui_map_scan.mjs`;
`build` writes the whole map to the untracked `docs/ui-map.md`).
- [`backend.md`](backend.md): Python routes, filesystem safety, persistence,
scan state, model sidecars, and backend performance.
- [`frontend.md`](frontend.md): extension bootstrap, UI ownership, localization,
+115 -1
View File
@@ -249,6 +249,92 @@ descriptions. Its picks go to `/anomalous/scan_all` as `targets`, like a list pa
## MCP endpoint
`track_run(prompt_ids, task, wait_seconds)` attaches existing submissions from this server to
an open task without running anything. IDs must exist in its queue/history; each is owned by
one recorded run. Retrying an identical batch returns that run, not the task's newest run.
External callers must supply their own returned IDs; no queue-difference attribution is used.
`run_workflow` waits up to `wait_seconds` (default 15, max 55) after starting its watcher,
outside the submission semaphore. Text and structured responses include actual state,
results/error and next action. Long runs still require `run_status(wait_seconds=55)`;
the stateless MCP transport cannot wake an idle AI on its own. Task `live` reflects an
in-flight recorded step, not recency of contact.
AI gate decisions are single-use: `ai_gates.check` consumes an immediate approval before
dispatch; only an unused timed-out approval may authorize an identical retry. Missing
step records fail closed for `ask`. `finish_step` preserves decisions received after a
gate timeout; loading an unanswered gate after restart makes it eligible for late-answer
delivery. Gate decisions wake both gate waiters and `get_answers` readers. Explicit task
selection in `ai_panels._collect` is exclusive, including when that task no longer exists;
session-scoped `get_answers` resolves its own active task before collecting input.
`api/ai_task_handoff.py` selects automatic resumption candidates without mutating ownership:
it ranks unread user messages, marks, materials and panel answers
by the input's own timestamp, not the task's last background update. Only inputs less than
two hours old qualify; future timestamps and automatic timeout choices do not. A task with
an in-flight step is excluded from automatic takeover; explicit task selection is unchanged.
For five minutes after `report(done=True)`, reads stay in the finished task and panels/notes
reopen it. `ai_tasks.current_task` applies this to both implicit and explicit task ids, including
step recording without a start receipt. Automatic follow-ups yield to waiting tasks and unread
replies; an explicit id keeps precedence and does not consume another task waiting for an AI.
Task error counts only treat completed successful steps as recovery, never running, asking,
allowed or cancelled steps. New receipts keep a compact `recovery_target` (explicit node,
connection, workflow or resource, and setting names for `set_values`), so another target's
success does not hide the failure. Legacy receipts lacking this identity retain tool-level
matching with each other; no targets are guessed or migrated into historical records.
`ai_answer_wait.collect` is shared by `get_answers` and question tools: after the bounded wait,
typing/deadline checks, automatic choices and a final nonblocking collection share the task
lock. A user answer that won the race is returned in that call, without a stale stop note.
Without a panel id, deadlines are checked once per waiting panel, including expired ones.
Typing counts from real keys only (the page posts at most every 5 s) and holds the deadline for
`AFTER_TYPING` after the last one; a question the user typed on since it was asked is never
answered on timeout, the AI is told they began an answer and ends its turn.
Question receipts preserve messages and attach their pictures even when no option was chosen.
Run monitoring failures end the receipt as `stopped` with a reason without cancelling the
host workflow. No detected queue entry is not success. Batch progress keeps the complete
prompt list, and partial errors are saved even before all prompts finish. Marked-image
validation and persistence run in a worker thread; invalid cross-drive paths are validation
errors rather than uncaught route failures.
Run ownership is explicit: the page claims `run_workflow` with the `prompt_ids` capability,
then returns `{prompt_ids, queued, requested, submission_error?}` from native submissions.
An old page's claim fails with a refresh message before dispatch. The server validates the
bounded, unique ID list and retains its order; `ai_task_runs.watch` queries only those IDs,
never a global queue difference. A partial batch continues tracking accepted IDs but finishes
as an error with its pictures retained. `run_status` includes these IDs. Missing receipts
cannot trigger guessing or an automatic retry.
While a run goes, its watcher adds `api/ai_run_watch.py`'s readings to the progress: the
console lines written since the step began (ComfyUI's `app.logger` buffer, `\r` redraws kept
at their latest, colours stripped), the tqdm bar's note, seconds left and per step (else a pace
from the steps seen), and the card's free / total memory. From the queued prompt it estimates,
once, the size each sampler's `latent_image` works at (following LoadImage's file, empty
latents, scale and upscale-model nodes) and keeps `estimate` on the step: warnings for an
upscale model's output sampled again, a sampler past the card's comfort (4.2 MP on 12 GB,
scaled by its memory) and a huge `VAEDecode`. `run_status` and `your_run` carry them in words.
From the same prompt (or the history of a run quicker than the first look) the watcher keeps
`map` (`api/ai_run_map.py`: boxes by role with their facts, links typed from
`NODE_CLASS_MAPPINGS[...].RETURN_TYPES`, reroutes and primitives wired through, seeds left out
of `params`) and `changes` against the task's previous run with a map.
Canvas actions carry the task's own tab: `mcp_server._call` marks the call with its task
(`ai_task_canvas.begin_call`, a context variable that `asyncio.to_thread` copies), `ask_page`
adds `_task_canvas: {mode, tab, title}` to the page actions in `TAB_ACTIONS` and keeps the tab
an outcome reports (`remember`); `tell` adds `canvas_tab` to the answer once when a tab was made.
`download_missing` carries this context too. A recorded tab that is missing or cannot be
activated fails on the page before executing the action, rather than using another workflow.
Message `@` references resolve combo/workflow names with the internal `exact_name` filter
in `mcp_tools` before limiting results; public list-tool searches remain fuzzy.
The default mode is in `anomalous/ai_canvas.json` (GET/POST `/anomalous/ai_gates` `canvas_tab`),
a task's own in its `canvas` record (POST `/anomalous/ai_tasks/{id}` `{canvas}`).
`POST /anomalous/ai_tasks/{id}/pictures` (`api/ai_pictures.py`) hands output pictures to an open
task; pictures handed before the AI looked join one step.
`api/ai_run_control.py` stops a run by its own prompt IDs only (dequeue, then
`interrupt_if_running`); with `stop` set, an interrupted prompt's "error" history and a
dequeued prompt's missing one both count as ended and the step becomes `cancelled`.
`POST /anomalous/mcp` (`api/mcp_server.py`) answers one JSON-RPC message per POST with
one JSON object; GET and DELETE get 405, and nothing is sent unasked. A request whose
`params._meta` carries `io.modelcontextprotocol/protocolVersion` is served as 2026-07-28:
@@ -258,6 +344,13 @@ supported list. `initialize` selects the legacy shape (2025-03-26 to 2025-11-25)
session id is minted, so both eras are stateless. Unknown methods are -32601 (404 for
modern requests).
`api/mcp_arguments.py` owns unambiguous required-argument renaming: one needed argument missing,
one unknown argument whose name is a known other name for it (ALIASES, from the tool exams) and
whose value has its type. A same-typed value under any other name is not taken. The MCP server normalizes
before recording a step or checking a gate, so the approval signature, displayed arguments,
execution and result references all use the same names. Direct `mcp_tools.call_tool` callers
use the same normalization; the successful receipt tells the AI which name was corrected.
The peer must be loopback with no proxy header (`Forwarded`, `X-Forwarded-*`,
`X-Real-IP`), and the Host and any Origin must name a loopback host; anything else is
403. ComfyUI's own origin middleware also applies.
@@ -269,7 +362,10 @@ A tool's own failure (bad id, missing file, unexpected error) is a result with
`isError: true`, not a protocol error. Pictures attached on request are the gallery's
512 px thumbnails (at most 1.5 MB).
Acting tools (`api/mcp_actions.py`) never delete or rename files. Canvas actions go
Acting tools (`api/mcp_actions.py`, and `api/mcp_canvas_edit.py` for building a workflow)
never delete or rename files; `remove_nodes` removes canvas nodes only, one Ctrl+Z step, and
`new_workflow` opens a new unsaved tab named after the AI's words and the time, so it never
opens or replaces a saved workflow. Canvas actions go
through `mcp_bridge.ask_page`: it sends `{id, action, args}` as the `anomalous.mcp`
websocket event to every open page and waits (30 s, 90 s for model checks; at most 20
waiting). The first page to POST `{id, claim: true}` to `/anomalous/mcp/bridge` gets it
@@ -280,6 +376,24 @@ server's own `/anomalous/scan_all`, `/anomalous/download/status`, `/anomalous_tt
minutes for the file). Downloads go through the page (`model_download.js` downloadMissing /
downloadFrom), so the placement rules and the put-in after a download are the UI's own. Canvas changes are posted to the activity log with `via: "mcp"`.
`find_nodes(query=...)` returns up to five
brief candidates within a small text budget; `types=[...]` explicitly requests full settings.
`add_node` accepts `insert_after`, optional `insert_type`, and `wire_from`/`wire_to` (one ID or up
to eight IDs). `connect_nodes` accepts just two node IDs. Both return actual connections,
remaining required inputs and whether the branch reaches an output; reachability is not run
validation. Same-type sockets of the same name pair up (a ControlNet's positive/negative into a
sampler's). An occupied input that `wire_to` feeds is spliced when the new node takes that type: the
old source feeds the new node's matching input. Otherwise automatic wiring never replaces occupied
inputs without `replace:true`, except the selected downstream edge of an insertion; `insert_after`
splices every type the node passes through (a LoRA's MODEL and CLIP). The legacy explicit `input` form still defaults
to output slot zero and permits replacement. Ambiguity is an error with bounded candidates.
`add_nodes` takes a group in API form for the open canvas: a link names a node of the group or a
canvas node as `"#12"` and an output by slot or name; a node's `wire_to` feeds canvas nodes as
`add_node` does, splicing only into inputs the group itself leaves free. The page lays the group out
as one block in a free spot and applies it in one transaction with the same rollback.
`get_situation` includes at most six path chains and eight node lines; its overall 3600-character
budget trims these and reports omitted counts. Full canvas details remain explicitly available.
## Backups
`api/backup.py` packs the user's data into one .zip (`POST /anomalous/backup/export`, then
+43
View File
@@ -5,6 +5,33 @@ localization, media, or explicit canvas mutations.
## Bootstrap and module ownership
`web/modules/ai_task_status.js` derives confirmed task execution state;
`web/modules/ui_ai_status.js` renders the task toolbar's compact execution-status button and popover from run steps,
with a stop button beside it while a run goes. `web/modules/ui_ai_run_live.js` draws a running run for both the
popover and the timeline: bar and percent, time left and per step, VRAM, the console's note and last lines in a
`<details>`, the stop button (`POST /anomalous/ai_tasks/{id}/runs/{step}/stop`), and the step's size `estimate`
with its warnings (also kept after the run). `web/modules/ui_ai_run_map.js` draws a run's `map` under its step
(one line, the changes as chips, the boxes in a `<details>`, scaled to the column) and for a `run_map` stage block
(the panel ctx's `stepOf`). `web/modules/mcp_task_tab.js` wraps each page action in `mcp_bridge.js`: with the task's
mode "new", it clicks the task tab's button in `.workflow-tabs` (falling back to `Workspace.NextOpenedWorkflow`)
and waits for `activeWorkflow` to be it. A missing or unactivatable recorded tab fails the action;
it never substitutes the current workflow. Only a task without a recorded tab copies
`app.graph.serialize()` into a new tab before its first change. Missing-model downloads follow
the same task tab as reads and runs. Explicit `new_workflow` / `open_workflow` can replace the tab; the outcome
reports `tab`. `ui_ai_task_canvas.js` is the header's mode switch and tab chip.
The header shares progress with the timeline. Websocket updates also update the in-memory
step; a five-second visible-page refresh recovers missed events. The loop stops when the
workbench/page is hidden or removed, and resumes on open or browser visibility return.
Unended tasks with no active step await an update; silence never means completion.
`mcp_run.js` follows one native `app.queuePrompt` request by its host request ID and
`promptQueueing`/`promptQueued` events. A temporary `api.queuePrompt` wrapper preserves the
original receiver/arguments and collects only that batch's submission responses, stopping
on its first failure or completion. Cleanup restores the API and removes both listeners on
all exits. The batch receipt can return before the native queue finishes later manual work;
the native promise remains error-handled. Already-busy or unsupported host queues are refused
before calling the native method. Seed changes, widgets and serialization remain host-owned.
Share and import is the Workflows page's ⇅ (`ui_share.js`). Its share code
(`share_code.js`, no DOM) is AMB2: the workflow's lean form — node id, type, widget values,
position to 10 px, and only what differs from a fresh node of the type (`LiteGraph.createNode`
@@ -288,6 +315,22 @@ skips volatile values, and calls the host's widget hooks with the current
four-argument contract. Serialized workflow records are never treated as live
widget objects.
MCP canvas construction uses `canvas_wiring.js` to plan all connections against live sockets
before mutation. Automatic pairs require a unique concrete type on both sides, or same-type
sockets of the same name; wildcard, union and other ambiguous sockets require explicit selection.
An occupied input the new node feeds is spliced when the node takes that type (its old source
feeds the node). Insertion splices each type the node passes through, one output per type, and
needs an explicit downstream selection when an output branches. Final topology is checked for
cycles after planned replacements. `mcp_canvas_edit.js` brackets application in one native
beforeChange/afterChange transaction. On failure it removes the added node and restores prior
connections; callback failures during restoration are reported as incomplete rollback.
Asynchronous node preparation verifies the active graph, workflow key and serialized content
before writing. Successful receipts report links and output reachability, not execution success.
The activity log's limited undo is separate from ComfyUI's native Ctrl+Z for topology changes.
`mcp_canvas_brief.js` collapses unbranched paths within each flow, grouping IMAGE/LATENT into
the picture spine so auxiliary VAE/MASK inputs do not split it, keeps positive/negative roles,
walks each edge once (including malformed cycles), and bounds the resulting text.
## Optional capabilities
`web/hash_resolver.js` keeps the local hash cache and may hook a compatible graph
+10
View File
@@ -123,3 +123,13 @@ body:has(#anomalous-toolbox-modal[style*="display: flex"]) #anomalous-sidebar-to
**以后怎么做**
- `node tools/check_structure.mjs` 现在会拒绝 `:has()` 里出现 `[style…]`,报出文件和行号。
- 同一个坑第二次出现,就把它做成检查,不再只补一段经验。
## 2026-11-05 · 偶尔打开 ComfyUI,AMB 整个不见了
**现象**:没有悬浮按钮,没有窗口,控制台里是几十条 `Cannot access 'shared' before initialization`;刷新一下又好了。大约十几次加载碰到一次。
**原因**:ComfyUI 把 `web/` 下每个 .js 都当成扩展,同时各自导入一遍,谁先跑完全看网络。AMB 的模块有一个很大的循环导入,`ui_ai_workbench.js` 加载时就要写 `ui_ai_results.js` 里的 `const shared`。从 `main.js` 进来顺序是对的;碰巧先从 `ui_ai_results.js` 或 `ui_ai_step_body.js` 进来,`shared` 还没初始化,这个模块就报错,而且错误会被缓存,整个 AMB 跟着起不来。
**做法**:`shared` 搬进不导入任何东西的 `ui_ai_shared.js`。`tests/module_cycles.mjs` 查“加载时就读同一个循环里的 const”,这种写法以后过不了测试。
**以后**:会在加载时用到的值,放在循环外面的模块里。别只从 `main.js` 想加载顺序。要逐个模块验证,在浏览器里给每个模块开一个 iframe 单独 import。
+75
View File
@@ -0,0 +1,75 @@
# AI 工具考试(MCP 工具改动后用)
AMB 给 AI 软件的工具,以轻量模型(Gemini Flash)能顺畅用为准。改工具(加一个、改说明、改参数)以后,不靠一次次手试,而是让 Flash 把下面的固定考题做一遍,再用打分脚本看数字。
## 预算(`tests/test_tool_budget.py` 自动检查)
2026-10-26 量的:53 个工具,约 4.3 万字符(约 1.1 万 token),说明书约 3200 字符。Gemini 3.8 Flash 在 Antigravity 里用它们调用了约 570 次,几乎没有调用错误。所以现在这样就是上限:
- 工具总数不再增加:加一个,就要合并或删掉一个。
- 每个工具的说明不超过 700 字符(`show_panel` 例外:它的各种块没有自己的格式定义,说明就是格式)。
- 一次调用最多 2 个必填参数,其余都有默认值。
- 报错要写下一步怎么做(哪个节点、缺什么、调哪个工具),不要只转发原始报错。
- 所有工具都直接给,不做"转一道"的调用:Antigravity 自己会在第一次用某个工具前读它的说明文件,服务器这边再做一层按需只会让 AI 多绕路(10-26 那次 29 次调用里有 18 次花在查工具和转一道上,所以"日常"工具组删了)。
要放宽这些数字,必须同时跑一遍考试,证明 Flash 还是顺的。
## 怎么考
1. 在隔离的测试 ComfyUI(8199 端口,自己的 user 和 output 文件夹,模型只读)上开 AMB,页面开着(画布上的事由页面做)。关卡里把"下载模型""联网扫描""挪动模型文件"设成"不许做"。
2. 在测试 ComfyUI 的工作流页存好三个工作流:`exam_txt2img`(一个能出图的文生图)、`exam_missing`(同一个,主模型的文件名改成这台电脑没有的)、`exam_broken`(同一个,KSampler 的 model 线断开)。
3. AI 软件连到这个测试 ComfyUI(Antigravity:MCP 设置里 anomalous 的地址改成 8199,考完改回来)。连之前、每次开考之前都先问用户。
4. 每道题先在 AMB 里点"新任务",再开一个新对话,把"开头那句话"(后面加一句"只用 Anomalous 的工具,不要运行命令或读写文件")发给 AI;需要你回答的地方照"你怎么答"去答。
Antigravity 也能用它自带的命令行开对话:`resources/bin/language_server.exe agentapi new-conversation --model=flash "<开头那句话>"`(`send-message <对话> "<话>"` 接着说)。要先设好三个环境变量:`ANTIGRAVITY_LS_ADDRESS`(它日志里 "for HTTP" 那个端口)、`ANTIGRAVITY_CSRF_TOKEN`(它的 language_server 启动参数 `--csrf_token`,只在本机有效,不要写进任何文件)、`ANTIGRAVITY_PROJECT_ID`(平时用的那个项目)。注意:这样开的对话里 Gemini 不去读工具说明文件,参数名靠猜,比在界面里开的对话更难,算"最差情况"。
5. 全部做完后打分:
```bash
node tools/tool_exam.mjs <测试 ComfyUI 的 user 文件夹>/anomalous/ai_tasks --since=2026-10-26T20:00
```
每道题一行:AI 调用了几次(calls)、失败几次(failed)、绕路几次(detours:以前版本的查工具,或者拿着旧的工具列表来调用)、问了你几次、你答了几次、跑了几次图、失败几次、有没有收尾、用了几分钟。最后一行是总数,下面一行是失败率和绕路率。
6. 和上一次的分数比:失败率、绕路率不能变高,每题的调用次数不能明显变多。
## 考题
| # | 开头那句话 | 你怎么答 | 合格 |
| --- | --- | --- | --- |
| 1 | 用 Anomalous 出一张图:用工作流 exam_txt2img,提示词写"一只在窗台上晒太阳的橘猫",跑一张。 | 不用答 | 出图;不超过 6 次调用 |
| 2 | 用 Anomalous 打开工作流 exam_txt2img,把主模型换成另一个 Illustrious 底模,再加一个适合它的 LoRA,权重照作者示例图的来,跑一张。 | 它问用哪个时选推荐的 | 用了 `set_model`、`add_lora`;LoRA 权重来自 `get_model` 的 author_examples;没有底模不配的提醒,或有提醒后换掉了 |
| 3 | 用 Anomalous 给最新那张 exam 开头的图想两个改法,问我选哪个,再照我选的做。 | 选不是推荐的那个 | 用 `ask_user` 问、等到你回答;照你选的做 |
| 4 | 用 Anomalous 修图库里最新一张图的手:我在 AMB 里圈出来。 | 在图上圈手,确认它调整后的范围 | `propose_repaint_area` → 你确认 → AI 自己加重绘节点 → 跑出来 |
| 5 | 用 Anomalous 打开工作流 exam_missing,它缺一个模型。检查缺了什么,处理好以后跑一张。 | 不用答 | 用 `check_workflow_models`,下载被关卡拦下后换上一个有的模型,跑出来 |
| 6 | 用 Anomalous 找出最近一张用了 LoRA 的图,告诉我当时用的是哪个 LoRA、权重多少、种子多少。 | 不用答 | `search_images` + `get_image_info`,答对 |
| 7 | 用 Anomalous 给工作流 exam_txt2img 加一个放大分支(放大到 1.5 倍),跑一张。 | 不用答 | 搭好并接线,跑出来 |
| 8 | 用 Anomalous 跑一下工作流 exam_broken。 | 不用答 | 收到"哪个节点缺什么"的提示后用 `connect_nodes` 接上,再跑成功;失败不超过 3 次 |
## 2026-10-26 第一次考试(Antigravity 命令行开的对话,Gemini 3.8 Flash)
7 道题(第 4 题要在图上手动圈,没考),107 次调用,失败 19 次(18%),绕路 0。题都做成了:出图、换模型加 LoRA、问了再照答案做、缺模型时没下载、答对图的参数、搭放大分支、断线时照提示接好。
失败几乎都出在"第一次用某个工具":空着参数调一次(7 次)、故意传个假参数看报错里列出的参数(6 次)、凭感觉写参数名(4 次,比如给 `set_prompt` 写 `prompt`、给 `open_workflow` 写 `name`)。另有 1 次等待时间写了 120 秒,1 次是关卡按设置拦下下载。
据此改了:参数写错时报错列出该写哪些参数(下一次就对);同一个东西在各工具里用同一个参数名(提示词都叫 `prompt`,工作流都叫 `workflow`);等待时间超了按 55 秒;按类型找模型时单数也认。也试过在每个工具说明开头写上参数,重考时没有变化(这种对话里 Gemini 只看工具名),又撤掉了。
## 2026-10-26 第二次考试(同样的条件,8 道题都考)
| 题 | 调用 | 失败 | 结果 |
|---|---|---|---|
| 1 出图 | 9 | 2 | 出图;改完以后重考:3 次调用、0 失败(`run_workflow` 一次带上工作流和提示词) |
| 2 换模型加 LoRA | 18 | 3 | 出图 |
| 3 问你再改 | 14 | 2 | 先没改就跑了一张,又发了一个"测试问答"(选项"改法 A / 改法 B",没写内容);之后正经问了两个改法,照答案做了 |
| 4 圈手重绘 | 29 | 3 | 请你圈手 → `propose_repaint_area` → 你确认 → 自己搭重绘 → 出图 |
| 5 缺模型 | 21 | 6 | 下载、联网扫描、挪文件各被关卡拦了一次,最后换上有的模型跑出来 |
| 6 找图看参数 | 5 | 0 | 答对,还跳过了更新但没用 LoRA 的那张 |
| 7 搭放大分支 | 31 | 7 | 搭好跑出来;中途连着空参数试了三个工具,又绕着查了几遍工作流和记录 |
| 8 修断线 | 12 | 6 | 照提示接好跑出来(2 次是故意的拒绝) |
139 次调用,失败 29 次(21%),每题都做成了。失败的原因和第一次一样,大多是第一次用某个工具时空着参数先调一次;同一道题的调用次数两次考差得很多(第 1 题 9 次和 3 次,第 7 题 17 次和 31 次),一次考试只能看大方向。
注意:`ask_user`、`get_answers` 参数写错时不进时间线,分数里也就没有(第 3 题的那次是从 Antigravity 的记录里数的)。
据此改了:只差一个必填参数、又正好多了一个不认识的参数、这个名字是考试里见过的那个参数的另一种叫法(`api/mcp_arguments.py` 的 ALIASES)、值的类型也对得上时,AMB 把它当成那个必填参数(`open_workflow {name: …}` 就是 `{workflow: …}`);类型对得上但名字没见过的不算,意思可能不一样。回复里写一句"把 name 当成了 workflow",下次就写对。时间线里"放大看图"那一步原来显示成代码名,补了名字,并加了测试:每个工具都要有中英文名字。
看到但没改:任务做完(`report` 写了做完)以后 AI 又看了一下,会多出一个只有一步的空任务;Gemini 有时没先问就动手、用假问题试工具。
+138
View File
@@ -0,0 +1,138 @@
# 界面地图:点哪里、去哪里、为什么
这份是界面地图的**手写部分**,只写代码里看不出来的东西:每一块是干什么的、点了以后发生什么、为什么这样设计、同一个东西统一叫什么。
具体在哪个文件、哪一行、调哪个接口,不要在这里找,直接问代码(永远是最新的):
```text
node tools/ui_map.mjs find 屏幕上的字 比如 find AI 改副本:哪个文件画的、样式在哪、调哪个接口、后端哪个函数
node tools/ui_map.mjs find /anomalous/ai_tasks 这个接口由谁处理、前端谁在调
node tools/ui_map.mjs find ui_ai_marker.js 这个文件显示哪些字、调哪些接口、被谁用
node tools/ui_map.mjs build 写出 docs/ui-map.md:按页面列出全部文件、文字和接口(不进版本库)
```
写这份文档的规矩(`node tools/check_structure.mjs` 会检查):
- 「」 里只放**屏幕上的原字**(中文界面),一个字都不能差;太长的句子可以只写开头,结尾加 …。界面改了字,检查就会报错,提醒你回来改这里。
- 文件名和接口用反引号,比如 `ui_ai_task_canvas.js`、`/anomalous/ai_tasks/{task_id}`;不存在了检查也会报错。
- 最后的“统一叫法”表:一个东西只用一个名字。表里“不再用”的词,只要还出现在界面文字里,检查就报错。
## AI 工作台
三栏:左边任务列表,中间一个任务的时间线,右边侧栏。窗口窄时左边和中间轮流显示,右边变成标签页。
### 左栏:任务列表(`ui_ai_task_list.js`)
| 看到的 | 点了以后 | 后端 |
| --- | --- | --- |
| 「新任务」 | 写一句要 AI 做什么(可以空着),任务显示“等 AI 接手”;下一个来调用的 AI 直接接手它,不会接在旧任务后面 | `POST /anomalous/ai_tasks` |
| 「连接 AI」 | 弹出连接说明:地址、复制给 AI 的那句话、各个软件怎么加(`ui_ai_connect.js`) | 无 |
| 「关卡」 | AI 做哪些事要先问你(改画布、用显卡、下载……),还有 AI 怎么等你回答、怎么等跑图(`ui_ai_gate.js`) | `/anomalous/ai_gates` |
| 「全部」「进行中」「等你回答」「出错」 | 只筛这一栏,不改任何东西 | 无 |
| 一个任务 | 中间显示它。看过的、鼠标停过的马上出来,再悄悄补上最新内容(`ai_task_cache.js`) | `GET /anomalous/ai_tasks/{task_id}` |
收起:标题栏最左边的按钮(「收起 / 展开任务列表…」)。收起后列表仍在左边排好、藏起来,鼠标碰到这个按钮、或者从页面往左移到最左边缘,下一帧就滑出来;离开后 0.35 秒收回,点了任务马上收回。
为什么:列表每 5 秒刷新一次,只重画变了的行(`sync_rows.js`),所以滚动位置不会跳、鼠标下面的行不会闪。
### 中间:任务的标题栏
| 看到的 | 点了以后 | 后端 |
| --- | --- | --- |
| 「AI 改副本」 / 「AI 改当前画布」 | 打开二选一:「改副本」(先把当前工作流复制到新标签页,AI 只改副本)或「改当前画布」。点开才换,不会点一下就悄悄切换 | `POST /anomalous/ai_tasks/{task_id}`(canvas) |
| 「切到副本」 | 切到这个任务的副本标签页,并显示画布(`mcp_task_tab.js`) | 无 |
| 「撤销画布上的改动」 | 按操作记录把 AI 在画布上的改动一步步撤销(`canvas_undo.js`) | 无 |
| 「改名」 / 「从列表移除」 | 移除只删这条记录,图、画布、工作流都不动 | `POST /anomalous/ai_tasks/{task_id}` |
| 「接着哪个任务…」 | 把这个任务接在另一个后面,两边都显示链接(`ui_ai_task_links.js`) | `POST /anomalous/ai_tasks/{task_id}`(from) |
AMB 更新以后,AI 软件还拿着旧的工具列表来调用时,标题栏下面出现一行黄字「AI 软件拿的还是旧的工具列表,会多走弯路。请重新连接:Antigravity 重启一下,或在它的 MCP 设置里把 anomalous 关掉再打开;然后开个新对话。」,AI 那边也会收到同样的提醒(`mcp_directory.stale`,按每个 AI 软件上次取的列表比对)。AI 软件重新取工具列表(`tools/list`)后这行自动消失。
标题栏右上角的一排是侧栏的开关:「展台」「节点结构」「决策」「素材」「成果」(`ui_ai_side_views.js`),同一时间只开一个。
### 中间:等 AI 接手的任务(`ui_ai_ready.js`)
「新任务」之后、AI 还没来时,中间是「等 AI 接手这个任务」一页:
- 最上面是「把这句话发给你的 AI」和「复制这句话」,页面往下滚时它停在顶上,挑完图不用再翻回去找。
- 下面都是可选的,收成两行,点了才展开:「写一句要做什么」(会成为任务名,那句话也跟着带上它)、「从图库挑图给 AI」(点一张就放到最下面的留言框,再点拿掉)。已经备好的东西列在下面。
- 最下面是还没连上 AI 时的「查看连接方法」。
为什么:以前分“开始”“备料”两页,去备料以后还要切回来复制那句话;第一步还是一个输入框,看着像必须填。现在主要的一步放最前面,可选的都收起来。那句话也不再叫 AI 一接手就停下来问你,而是照任务里的交代做,没交代才问。
### AI 问你的问题(`ui_ai_panel.js`、`ui_ai_now.js`、`ui_ai_gate.js`)
展台里的选择题、留言框上方的提问条、关卡卡片都按同一个做法:
- 选项是一按就发出去的按钮,右边有箭头,上面写着「点一个就发给 AI」;点了马上变成选中的样子,再显示“已发送 / AI 已收到”。
- 「让 AI 定」放在问题的右上角,是一个虚线小按钮,不跟选项挤在一起,免得被当成“确认”。
- 自己写的那一行默认收起:「都不是,我自己说…」,关卡卡片上是「说说想怎么改…」,点了才出输入框。
- 写了一半的回答一直留着(只留在这个页面里):AI 又问了新问题、读走了另一个回答,问题重画时字、光标位置和选区都还在,那一行也不会收起;展台和底部提问条同步同一道题的草稿,刷新只把焦点还给原来正在编辑的位置;这个问题一直排在提问条最前面。窄窗口里你正在打字时,新来的问题只点亮「展台」标签,不把你切过去。
- 只有真按键才算“在打字”(每几秒告诉一次服务器):停手一分钟以后照常算时间,光标停在框里不算。动手写过回答的问题,到时间也不会按推荐的替你选,AI 停下等你。
为什么:看到一个框,人就以为要填;选项和下面的按钮排在一起,人就以为要先选再确认。写到一半的话被清掉、被切走,或者人走开了被替你选掉,都比多等一会儿糟。
### 中间:时间线
- 从上往下读,编号 1、2、3……;AI 连着查资料、连着改画布的几步折成一行(`ui_ai_chain.js`),点开看每一步。
- 每一步下面写它做了什么(`ui_ai_step_body.js`),出的东西能直接去:图片进看图器、模型进模型页、画布改动能定位和撤销(`ui_ai_results.js`)。
- 图片上鼠标移上去有工具(`ui_ai_images.js`):「图片与参数」、「保存为…」、勾两张对比,还有 ✎ 标注。
- 时间线和展台里的图先用缩略图(`/anomalous/output_thumbnail`),显示得比缩略图大时才换原图,打开任务不用等大图。
### 标注:「标给 AI 看:哪里要改」(`ui_ai_marker.js`)
框、圈、箭头、画笔,每个标记写一句要怎么改;右边选「怎么改」:
- 「交给 AI 判断」:AI 看说明自己决定。
- 「只改标出的地方」:其余保持原样,框、圈、画笔和涂抹的地方做成遮罩放进 input 文件夹,AI 只重绘这些地方。用「涂重绘区」时自动选上。
- 「整张重新出图」:按说明整张重画,不做局部重绘。
「发给 AI」以后(`POST /anomalous/ai_tasks/{task_id}/marks`),AI 下一步收到的第一句就是你选的改法,写明是你的要求(`ai_marks.py`)。时间线里这条标注写着「改法:{mode}」,方便核对 AI 有没有照做。
### 中间最下面:留言框(`ui_ai_message.js`)
- 「给 AI 留言,比如:背景再亮一点。…」:Enter 发送(`POST /anomalous/ai_tasks/{task_id}/message`)。
- 打 @ 提到素材和图(`ui_ai_mention.js`),开头打 / 是快捷指令(`ui_ai_commands.js`):/停止、/撤销、/展台、/节点结构、/素材、/副本、/当前画布 马上生效;/重跑、/解释、/继续 帮你写好一句话。
- 「从图库选图给 AI」(`ui_ai_gallery_pick.js`),拖进来或粘贴截图(`ui_ai_drop.js`,存进输出文件夹的 ai_inbox)。
- 留言框上面的一条(`ui_ai_now.js`):正在出图的进度、要你允许的事、AI 问你的问题;AI 停下时给出可以复制给它的句子,比如「继续」(`ui_ai_call.js`)。
为什么:AMB 不能主动叫醒外部的 AI。发出去的留言会等着,AI 下一次调用任何工具时一起收到(`ai_panels.py` 的 attach_pending),或者它来读回答时收到。AI 已经停了,就要你回 AI 软件里说一句,所以那里总有一句可以复制的话。
### 右栏:侧栏
| 开关 | 里面是什么 | 主要文件 |
| --- | --- | --- |
| 「展台」 | AI 画的面板,从上往下排,旧的在上、新的在下;要你回答的在这里回答(`POST /anomalous/ai_tasks/{task_id}/answer`)。来了新面板而你在往上看时,出现「↓ 查看最新」 | `ui_ai_side.js`、`ui_ai_panel.js`、`ui_ai_stage.js` |
| 「节点结构」 | 这次运行实际用的节点、和上次比改了什么 | `ui_ai_run_map.js` |
| 「决策」 | 你在面板上的选择、允许和拒绝、AI 记下的关键决定 | `ui_ai_decisions.js` |
| 「素材」 | 「给 AI 的素材」:挑模型、节点、搭配、工作流、提示词交给 AI;「来源」列表里鼠标所在的行会亮,选中的打勾 | `ui_ai_toolbox.js`、`ui_ai_facet_menu.js` |
| 「成果」 | 这个任务出的图、工作流、搭配、提示词、音频,按步骤排 | `ui_ai_outputs.js` |
## 设计上的固定做法
- **从上往下读**:时间线、展台都是旧的在上、新的在下;看最新的时候自动跟着往下,往上翻着看时不打断,只出现“去最新”的按钮。
- **重画只重做变了的部分**:工作台随时在刷新,滚动位置、打了一半的字、展开的东西都不能丢。
- **会改东西的地方说清楚后果**:二选一要写每一种是什么意思;删除写明删的是什么、不删什么。
- **鼠标移上去要有反馈**:列表行、菜单项悬停变亮,当前选中的有底色或打勾。
- **AI 改画布默认改副本**:你正开着的工作流不被 AI 动。
## 统一叫法
| 说法 | English | 不再用 | 不再用(English) |
| --- | --- | --- | --- |
| 副本(AI 改的那份工作流) | copy | 专用标签页、自己的标签页、新开标签页 | own tab |
| 素材(交给 AI 的模型、节点……) | materials | 材料 | — |
| 留言(你写给 AI 的话) | message | 消息 | — |
| 节点结构 | node map | 结构图、运行结构 | node structure |
| 连接 AI | connect an AI | 最近接入、接入过 | — |
| 撤销 | undo | 撤回 | — |
| 展台 | stage | 画板、舞台 | — |
| 成果 | results | 产出 | — |
| 关卡 | gates | 闸门 | — |
| 工作流(存下的整套工作流,“工作流”页的卡片) | workflow | 配方(配方包、搭配方案除外) | recipe(recipe package 除外)、recipes |
| 配方包(把一个工作流打包发给别人的 .zip) | recipe package | — | — |
| 搭配(主模型 + LoRA + 提示词) | combo | — | — |
“不再用”里可以写例外,比如“配方(配方包、搭配方案除外)”:这两个词里的“配方”不算。`{recipe}` 这样的占位符也不算。
## 还没统一的
- `node tools/ui_map.mjs build` 的最后一节列出没有文件用到的文字(2026-10-25 清理过一次,当时为 0),清理前先确认不是拼出来的 key、也不是后端错误信息当 key 用的(`t(error.message)`)。
+12
View File
@@ -192,3 +192,15 @@ sampler's model and prompts as a node structure, values untouched (a workflow wi
sampler falls back to the older main model, LoRAs and positive prompt). Placing a combo
only adds a new group; nodes already on the canvas are never changed (their values are
changed in Current node). This is one of the feature principles in AGENTS.md.
## AD-022 — What an AI does outside Anomalous is seen, not prevented
An AI app may run commands and scripts of its own next to Anomalous's tools; MCP shows a
server only the calls made to it, and Anomalous does not read another app's logs or
history. So it makes its own tools the easy way (the AI is told what each one covers),
asks the AI to say what it did elsewhere (`log_outside`, and rules the user pastes into
the AI app), and sees for itself what matters most: when the AI calls, it looks at the
top of ComfyUI's output and input folders for pictures no run saved and pictures
rewritten in place since its previous call, shows them in the task and tells the AI.
It never looks in the background, never blocks or undoes anything, and words it as
something the user may have done. Anomalous itself never overwrites a file.
+153 -7
View File
@@ -10,6 +10,10 @@
AI 软件在调用会改东西的工具前,一般会先问你同不同意。
任务使用专用画布时,读取、运行和下载缺失模型都会跟随它的标签页。标签页关闭或切换失败后,
工具会明确报错;请重新打开原工作流,或让 AI 明确调用 `open_workflow` / `new_workflow` 为任务指定新画布,
再继续操作。不会自动用当前另一张画布替代。
English summary at the end.
---
@@ -30,6 +34,8 @@ http://127.0.0.1:8188/anomalous/mcp
**Cherry Studio**:设置 → MCP → 添加 → 快速创建,类型选 **可流式传输的 HTTP(streamableHttp)**,URL 填上面的地址。需要较新的版本。
**最省事的办法**:打开 AMB 的 AI 工作台,点任务列表上方的 **连接 AI**,复制里面那句话发给你的 AI(Claude Code、Gemini、Cursor 这类会改自己配置的都行),它会自己把 AMB 加进去。
**Claude Code**:
```bash
@@ -79,11 +85,12 @@ claude mcp add --transport http anomalous http://127.0.0.1:8188/anomalous/mcp
| 工具 | 查什么 |
|---|---|
| library_overview | 各类模型的数量、底模分布、扫描情况,存了多少搭配、工作流和提示词 |
| search_models / get_model | 按名字、触发词、底模、来源找模型;看某个模型的 C 站信息、描述、你写的备注,可以附上封面 |
| search_models / get_model | 按名字、触发词、底模、来源找模型;看某个模型的 C 站信息、版本、描述、你写的备注,以及作者示例图用的采样器、步数、CFG、尺寸(LoRA 还有作者用的权重),可以附上封面。每个模型都写明哪些节点能加载它;放错文件夹的(比如名字叫 ControlNet、其实是模型补丁的 Anima LLLite)会说明该放哪、放在这里加载不了 |
| zoom_image | 把一张图的某一块放大给 AI 看(手、脸、接缝、标注的区域),也可以把两张图的同一块并排比较(改之前和改之后)。输出、输入、预览图都能看,只看不改。AI 不必再自己写代码裁图、比图 |
| scan_report | 哪些模型匹配上了 C 站,哪些没有、原因是什么,哪些还没扫,上次扫描做了什么 |
| search_images / get_image_info | 按提示词、模型、参数找出图;看某张图的全部生成参数,可以附上缩略图 |
| search_images / get_image_info | 按提示词、模型、参数找出图;看某张图的全部生成参数,可以附上缩略图。`folder=input` 时看 ComfyUI 输入文件夹里的图(比如你标注时 AMB 存的重绘遮罩图):尺寸、要重画的区域在哪、占多大,也能附上图,不用 AI 自己跑命令去看 |
| list_combos | 你的搭配(主模型 + LoRA + 提示词) |
| list_workflows | 你的工作流配方,带标签、备注和主要设置 |
| list_workflows | 你存下的工作流,带标签、备注和主要设置 |
| list_saved_prompts | 提示词工坊里存的提示词 |
| search_audio | 用 Anomalous TTS 生成过的语音,带台词和角色 |
| recent_activity | 插件最近改过什么(操作记录) |
@@ -93,20 +100,144 @@ claude mcp add --transport http anomalous http://127.0.0.1:8188/anomalous/mcp
| 工具 | 做什么 |
|---|---|
| describe_canvas | 读当前工作流:每个节点的编号、类型、设置,哪个是正向/反向提示词框 |
| find_nodes | 搜索最多五个节点摘要;需要具体参数和选项时,用 `types=[节点类型]` 读取 |
| add_node / connect_nodes | 添加节点并按唯一类型自动连线,或把节点插进已有路径;见下方例子 |
| add_nodes | 一次加一组节点:按 ComfyUI 的 API 格式写,组里的节点互相连,也能连画布上已有的节点(写 `"#12"`,输出写编号或名字),排成一块放在空位上,整组算一步撤销 |
| set_prompt | 改写或追加正向/反向提示词 |
| set_model | 换某个加载器节点里的模型 |
| add_lora | 插入一个 LoRA,默认接在主模型加载器和已有的 LoRA 后面。LoRA 文件里练过文本编码器的(SD1.5、SDXL、Illustrious 的大多是),CLIP 线也接上:从 checkpoint 接,或者从 UNet 工作流里单独的 CLIP 加载器接,正反向提示词一起改接到它后面;没练过的(多数 Flux、Anima 的 LoRA)只接模型线 |
| add_lora | 插入一个 LoRA,默认接在主模型加载器和已有的 LoRA 后面。LoRA 文件里练过文本编码器的(SD1.5、SDXL、Illustrious 的大多是),CLIP 线也接上:从 checkpoint 接,或者从 UNet 工作流里单独的 CLIP 加载器接,正反向提示词一起改接到它后面;没练过的(多数 Flux、Anima 的 LoRA)只接模型线。LoRA 和主模型的底模对不上时(SD 1.5 的 LoRA 用在 SDXL 上),回复里会提醒 AI 并告诉它按底模去找;换模型(set_model)也一样 |
| place_combo | 把一个搭配作为一组新节点摆到画布上,原有节点不动 |
| check_workflow_models / fix_workflow_models | 检查缺失的模型;把能确定找回的(同一个文件)放回去,“可能是”的留给你在模型检查里确认 |
| run_workflow | 像按“运行”一样把工作流加入队列 |
| run_workflow | 像按“运行”一样把工作流加入队列。ComfyUI 不收(缺线、模型不在、数值超范围、没有输出节点)时,AI 收到的是“哪个节点、缺什么、该用哪个工具修”,不是原样的报错 |
| open_in_anomalous | 在插件里打开某个模型的详情页给你看 |
| scan_models | 扫描新模型、重新查找没匹配上的,或扫描指定的模型(会联网访问 Civitai) |
| download_missing_models | 下载当前工作流缺的模型:只下能确定来源的(C 站按指纹找到、或工作流带着文件链接),放在设置里“模型下载”说的位置,核对指纹后自动换上 |
| download_model | 按链接(C 站模型版本、Hugging Face / GitHub 文件)或 C 站指纹下载一个模型到指定类型的文件夹,不覆盖已有文件 |
| download_status | 看下载进度和结果 |
| move_model | 把放错文件夹的模型挪到它该在的文件夹(封面、备注跟着走),和模型页“整理”里的“挪过去”一样;不会盖掉同名文件。默认先问你 |
| list_voices / speak | 列出 Anomalous TTS 的角色;让某个角色读一段台词并保存(最多等两分钟) |
| begin_task / report | 在 AI 工作台里起一个任务的标题;给你留言、交上它用别的方式做出来的图、结束任务(见第 5 节) |
| run_workflow 的填空 | 日常出图一步完成:带上 `workflow`(存好的工作流名),第一次在任务的副本标签页里打开它,之后接着用这个标签页;`prompt` / `negative` 填正、反向提示词,`seed` 填所有采样器的种子,`width` / `height` 填空白潜空间的尺寸,`image` 填“加载图片”读的图(输出或输入文件夹里的,比如标注后的重绘图;有几个“加载图片”时用 `image_node` 指定);其他任何参数用 `values` 按名字改,比如 `{"steps": 30}`,第三方节点的参数也一样:只有一个节点有这个参数就直接改,几个节点都有(比如高清修复的两个采样器都有 `denoise`)就要写节点号 `{"12": {"denoise": 0.5}}`,否则什么都不改并列出是哪几个。几个正向提示词框接进同一个采样器(分区提示词)时,`prompt` 也会停下来,让 AI 用 `values` 分别写。其余照存好的样子不动。不带 `workflow` 时填到当前画布上。填空和出图一起算一步,改画布那部分也能 Ctrl+Z;画布关卡比显卡关卡严时,按画布关卡问你 |
| 工具目录 | 不是一个工具:AI 开始任务时(还没有任务时调现状一览也一样),回复里附带一张按“想做的事”排好的工具目录(出图、画布、模型、提示词、图片……每个工具一行标题),AI 看一眼就直接去用对的工具,不用到处试。目录从 AMB 的工具登记表自动生成,只列当前给 AI 的工具,加了新工具会自己出现 |
| get_situation | 一次拿到任务、流程链、少量节点摘要、缺失输入和可匹配输出、错误及最近运行。每个回复的 `next` 提示下一步。新开对话或拿不准时先调它;简报预算约 3600 字,减少 Antigravity 将大回复转存为文件的情况 |
| log_outside | AI 用了 AMB 以外的工具(命令、脚本、读写文件)后,说清做了什么、为什么、动了哪些文件;显示在时间线上(见第 5 节) |
| ask_user | 在 AI 工作台里问你一个问题:几个选项(AI 推荐的那个标着“推荐”),加一行“都不是?写下你的想法”;AI 在同一步里等你回答,至少等 30 秒(就算它自己说不等),你在 AMB 里点了,它直接接着做 |
| show_panel / update_panel / get_answers | 在 AI 工作台里画一个面板(对比图片、模型卡片、表格、按钮和输入框);改它或关掉它的问题;读你的回答(见第 5 节) |
## 5. 注意
## 5. AI 工作台:AI 做了什么,一目了然(1.62 起)
任务工具栏里的“运行状态”按钮点开后显示真实排队、当前节点进度、用时、结果或错误(运行中按钮上直接显示百分比,不占时间线版面);可见时每 5 秒补查一次。任务没有接入运行时显示“尚未收到运行信息”,不会假称 AI 正在出图,也不会因为一段时间没消息就标成完成。
让 AI 优先使用 `run_workflow`:默认等待 55 秒并返回实际状态和已生成的图片。仍在运行时使用 `run_status(wait_seconds=55)` 继续等待,不必找输出文件夹。若 AI 通过脚本、`/prompt` 或其他工具提交,须立即调用 `track_run(prompt_ids=[提交响应中的ID], task=begin_task返回的任务ID)`,只接入它自己提交的 ID;这个工具不会重复出图。升级后重启 ComfyUI、刷新页面,并在 AI 软件中重新连接 MCP 以发现新工具。AMB 不能主动唤醒已经停止调用工具的 AI。
“允许”只放行这一次操作;AI 等待超时后再调用,同一份允许也只消费一次。要持续放行请使用“以后直接做”。ComfyUI 重启前尚未回答的确认仍可回答,AI 需重新调用原工具。指定任务或会话读取的消息不会被另一个任务提前取走。
如果运行显示跟踪超时或记录不可用,表示 AMB 无法确认结果,并不表示它取消了出图;请查看 ComfyUI 的队列。没有发现排队记录也不会算作出图成功。
运行结果只关联这次提交返回的运行 ID。批次部分提交成功时,已经排队的部分继续跟踪并保留图片,AI 会收到未全部提交的说明,不应重跑整批。升级后需要重启 ComfyUI 并刷新页面;旧页面不能接手出图。页面正在提交另一批时会提示稍后重试,这只限制提交过程,不需要等显卡把整批跑完。
出图后不必回图库寻找功能:鼠标移到结果图片上,右上角的 **ⓘ** 打开完整的图片与参数,**⇄** 和当前画布对比,**☆** 用同一个菜单保存成工作流、搭配或提示词。图片左上角可以 **勾选**:勾两张(不同步骤、不同任务都行),页面底部出现 **查看参数差异**;它读取图片实际保存的工作流,不依赖 AI 口述参数。没有可用工作流的图片会显示原因,视频没有这些参数操作。AI 展台的图片、挑选卡片也一样;滑动对比下面有 **查看参数差异**。
**AI 软件里的 AMB 卡片**(Claude、ChatGPT、VS Code 等支持 MCP Apps 的软件):AI 开始任务、出图、画面板或留言时,对话里会出现一张 AMB 的卡片:它做了哪几步、最新几张图、它问你的问题。直接在卡片里点选项就是回答,AI 会马上收到并继续。卡片是 AMB 自己的页面,AI 只能给它数据。要你点头的“关卡”只能在 AMB 里允许,卡片只提示你去 AMB;这样 AI 没有办法替你点。Gemini 命令行、Claude Code 终端版这类只显示文字的软件看不到卡片,照常在 AMB 的 AI 工作台里看和回答。
**交给 AI**:模型页的按钮行、图片详情右上角的 **✦**、搭配卡片上都有 **交给 AI**。写一句想让 AI 做什么(已经预先填好),AMB 会建一个带着这个模型、图片或搭配的任务,并给你一句话复制给 AI;AI 调用 `begin_task` 接手时就拿到了它们的编号,不用再自己搜。
**斜杠命令**:支持 MCP prompts 的软件(比如 Claude)里可以直接选 AMB 提供的四个现成请求:在工作台开一个任务、出几张让我挑、补齐缺的模型、讲讲现在的画布。
**在图上标给 AI 看**:工作台里图片右上角的 **✎** 打开标注:框、圈、箭头、画笔,三种颜色,每个标记写一句要怎么改,再写整体的想法,点 **发给 AI**。AI 下一次调用工具(或 `get_answers`)时收到这些说明和标好的图;能看图的 AI 直接看得到标在哪里。标好的图存在 AMB 自己的记录里,不进图库,任务删掉时一起删。
**让 AI 局部重绘**:标注里选 **▧ 涂重绘区**,用笔刷把要重画的地方涂上(笔刷粗细可调;框、圈、画笔圈出的地方也算),右边 **怎么改** 选 **只改标出的地方**(用笔刷时会自动选上)再发给 AI。另外两种是 **交给 AI 判断**(AI 看说明自己定)和 **整张重新出图**(不做局部重绘);你选的会作为你的要求写在 AI 收到的第一句,时间线里那条标注也写着“改法:…”。AMB 会在 ComfyUI 的 input 文件夹里放一张 `anomalous_inpaint_….png`:原图加上你涂的区域做遮罩。AI 收到时会被告诉怎么用它(LoadImage 读进来就带遮罩,用原图的模型和提示词,重绘强度 0.5~0.8 保留原样子,1.0 完全重画),不确定时用选择题问你强度,画完用前后对比给你看。这张文件删任务时不会删,因为你的工作流可能还在用它。
**先对一下重画的范围**:圈得不准也没关系。AI 准备只重画标出的地方时,会先在展台把 **你圈的** 和 **AI 调整后** 的范围并排给你看(红色是要重画的地方;AI 可以挪位置、改成方框或椭圆、往外扩、把边缘做软,图下写着外扩和羽化了多少像素)。点 **用 AI 调整的**、**用我圈的** 或 **我重新圈**,也可以在下面写一句(比如“再往手腕多圈一点”),AI 会照着再调一次;点 **让 AI 定** 就用 AI 调整的。选好以后 AMB 才把带这个范围的 `anomalous_inpaint_….png` 放进 input 文件夹,节点怎么搭由 AI 自己来。
**AI 改副本**(默认):AI 第一次改画布前,AMB 先把你正开着的工作流复制到一个以任务命名的新标签页(副本),之后 AI 的修改、读画布和运行都只在那一页;你切去别的标签页也没关系,它下次动手前 AMB 会先切回去。你原来的工作流一笔不动。AI 用 `new_workflow` / `open_workflow` 打开的标签页就直接算这个任务的。任务标题栏有 **AI 改副本 ▾**(点开可以换成 **改当前画布**,每种都写着是什么意思)和 **切到副本**(点一下切过去);**关卡** 窗口里的“AI 改画布时”设定新任务的默认做法。
**这次跑的节点结构**:每次运行那一步下面写着一行“底模 · 几个 LoRA · 尺寸 · 步数 · 几个节点”,点开有 **简单** 和 **完整** 两种看法(AMB 记得你上次选的)。简单:按每一段采样列出“用哪个模型加哪些 LoRA、提示词是什么、从空白画布还是从上一段的结果(放大多少)开始”,最后保存在哪;后一段和前一段一样的就写“和第 1 段一样”。完整:方块图,哪个底模、LoRA 和权重、正负提示词开头、尺寸、采样器设置、保存,方块之间的线标着类型。两种看法里点任意一项都能定位到画布上的节点。这张图是 AMB 按 ComfyUI 实际收到的工作流画的,不靠 AI 描述。上面的黄色小块是 **比上次改了什么**(比如“K采样器·步数:20 → 30”“新增「加载LoRA」”)。AI 讲解时可以用 `show_run_map` 一次把同一张图放上展台,不用自己拼。
**把图片直接发给 AI**:图库里右键一张图选 **发给 AI**,或者看图窗口的 **✦**:AI 正在做任务时会问你“发给当前任务”还是“开新任务”。发给当前任务时可以写一句想让它看什么;AI 收到的是图片本身,加上 AMB 从图里读出的模型、LoRA、提示词、尺寸和采样设置,不用你再描述。还没被 AI 读到之前连着发的几张会并成一步。AI 自己搜图库时也可以让结果直接带上前 6 张图(`search_images` 的 `with_pictures`)。
**让 AI 在画布上搭工作流**:AI 可以先用 `find_nodes` 查这台 ComfyUI 有哪些节点、每个节点要接什么,然后用 `new_workflow` 新开一个标签页(空白,或者一次把整套工作流搭好并排好位置;你正开着的工作流不受影响),再用 `add_node`、`connect_nodes`、`set_values`、`remove_nodes` 加节点、连线、改参数、删节点。每一步你都能在画布上看到,按 Ctrl+Z 能撤回;AI 工作台那一步下面写着改了什么(包括哪根线从哪里连到哪里)。类型接不上、选项不存在时,画布不会动,AI 会收到能用的写法。
**给 AI 的素材**:窗口右上角的 **✦ 素材** 默认在工作台右边那一栏打开(在别的页面打开时停靠在屏幕右边):不挡时间线和画布,上面搜索,种类是一排标签,卡片两列,点卡片在小窗里看详细说明(“返回列表”回去),下面挑好点交给 AI。右上角 **⤢** 展开成大窗口,大窗口标题栏的侧栏按钮再收回去;AMB 记得你上次用的是哪种。大窗口里:上面搜索所有素材,左边选种类,节点可以按来源筛(ComfyUI 自带、你装的每个第三方节点包、付费 API 节点),卡片上看得到节点的输入输出、搭配和工作流的封面、提示词原文;鼠标移到卡片上右边显示完整说明。提示词里也能挑搭配和工作流里的提示词。不知道该选哪个节点时,在来源里点某个自定义节点包(小窗里点“来源:全部 ▾”打开列表,可以搜索),点上面出现的 **整包交给 AI**:AI 会从这个包里挑合适的节点用,“只能用”时这个包的节点都允许。勾选后点 **交给 AI**,选“优先用这些”或“只能用这些节点”(自带节点照常可用,别的第三方和付费节点会被拦下)。☆ 设为常用:没挑素材的任务自动带上常用的。AI 用 `get_toolbox` 看到你挑的东西的详细说明,用 `open_workflow` 在新标签页打开你存的工作流。工作流详情页的 **✦ 交给 AI** 会新开一个带着这个工作流的任务。
**运行进度**:AI 运行工作流后,时间线上那一步下面有进度条(排队时前面还有几个;运行时哪个节点、第几步、节点做完几个)。进度条右边是百分比,旁边是红色的 **■ 停止**:一点就停这一次运行(还在排队的移出队列,正在跑的立刻中断,只动这次运行),AI 也会收到“你停止了运行”,不会自己再跑一遍。任务工具栏的运行状态按钮旁边也有一个小 ■。进度条下面写着当前节点 **约剩多久**、每步几秒、**显存剩多少**,再下面一行是 ComfyUI **控制台**最新的那一行(比如 `Model Initializing ...`、`Requested to load …`),点开能看最近几行;这些 AI 也同样看得到。出图刚排上队,AMB 会读这次的工作流算出每个采样器要在多大的图上采样:如果是放大模型放大后又整张重新采样、尺寸超过这张显卡的舒适范围、或者要解码特别大的图,会出现黄色提醒,告诉你可能很慢或爆显存,以及怎么改(只想放大就用纯模型超分,或者先缩小、减步数、分块解码);AI 收到同样的提醒,会被要求先问你要不要停(它也能用 `stop_run` 停自己的运行)。AI 用 `run_status` 问同样的进度,或者等到跑完再继续,不会凭时间猜;运行期间 AI 调用别的工具时,回复里的 `your_run` 也带着一句当前进度,跑完后告诉它一次结果,所以它不用自己去翻输出文件夹或者看屏幕。跑完后,第一个告诉 AI "跑完了" 的回复里直接带着这次出的图(最多 4 张缩略图),AI 不用再一张张去取,看完就能直接给你留言。
**流程图和关卡**:任务标题下面的一排流程按 AI 做事的先后列出它经过的阶段(模型、搭配、提示词、运行、挑图……),点一下跳到那一步。任务列表上方的 **关卡** 里,改画布、用显卡出图配音、下载模型、联网扫描、挪动模型文件五类事各自可以选 **直接做 / 先问我 / 不许做**(默认下载和挪文件先问)。选了“先问我”,AI 的那一步会停在时间线上列出它要做什么,你点 **允许** 才做;点 **不要** 时可以写原因或想怎么改,AI 会收到。AI 最多等 50 秒,等不及的话会先去问你的回答(`get_answers`),你允许后它再做一次,不会问第二遍。AMB 把所有工具都给 AI 软件,软件自己按需读工具说明(Antigravity 在第一次用某个工具前读它的说明文件)。**等我回答** 是 AI 问你问题以后最多等多久(1 分钟 / 3 分钟 / 10 分钟):你在回答框或留言框里打字时,停手一分钟内都不算超时(只算真按键,光标停在框里不算);时间到了 AI 收工,你答完以后点一下消息框上方那句话复制给它。AI 每轮做完会先问你下一步,跑图、下载、扫描时除外;问题上的 **让 AI 定** 表示交给 AI 自己决定。AI 软件会记住它拿到的工具列表,AMB 更新以后要让它重新连接(Antigravity:重启,或在 MCP 设置里把 anomalous 关掉再打开),再开个新对话;它还拿着旧列表时,任务标题下面会有一行黄字提醒你。**等不到回答时** 默认是 **停下等我**;选 **按推荐的继续**,到了时间你还没答,AI 标了推荐项的选择题就按推荐的选,展台和“决策”里会写明“到时间,按推荐选了”;没有推荐项的问题、你已经动手写了回答的问题、关卡问你的事永远不会替你答。**等跑图** 默认是 **跑完叫我**:AI 提交后最多等 1 分钟,跑得快的当场就拿到结果,还没跑完就告诉你在跑,然后收工,不再每 55 秒查一次进度;跑完后,AI 工作台消息框上方有几句话(“跑完了,看看结果”“再来一次,换个种子”,出错时是“出错了,看看是哪里的问题”),点一下复制,粘到 AI 的对话框里,AI 的下一次调用就拿到结果和图。AI 一知道结果,这几句话就换回平常的。平常 AI 停下来的时候(没在调用工具、没在跑图、没在等你回答),那里也总有一行话可以复制:你在 AMB 里回复了而 AI 还没收到,是“我在 Anomalous 里回复了,看看再接着做”,别的时候是“继续”;最后一句“用 Anomalous 接着做任务:…”给新开的 AI 对话用。想让 AI 自己等到跑完,选 **AI 自己等**。这一项马上生效,不用开新对话。
**少调用几次就接好线**:以下编号只是例子,以当前画布为准。
```text
add_node(type="ImageScale", insert_after=8)
```
原来 `#8 → #9 保存` 会变成 `#8 → 新放大节点 → #9 保存`。如果 #8 同时连到保存和预览,工具会要求用 `wire_to=9` 指定要插入的分支。新节点让几种类型穿过时(LoRA 的模型线和 CLIP 线)每一种都接进去;只想插一种时用 `insert_type="MODEL"` 等指定。没有下游时用 `wire_from=8` 延长末端。
```text
add_node(type="VAEEncodeForInpaint", wire_from=[12, 4], wire_to=3, replace=true)
connect_nodes(from_node_id=12, to_node_id=15)
```
前一个调用按唯一类型连接图片、遮罩、VAE 和采样器的 LATENT;`replace=true` 明确允许替换采样器原有的输入。后一个只报两个节点号,把它们之间唯一匹配的类型一次接好。同类型的几个插口名字对得上时自动配对(ControlNet 的 positive/negative 对采样器的 positive/negative);对不上的(一个提示词该接正向还是反向)和通配类型不会猜,工具会要求指定插口;多来源争抢同一输入、循环和未获替换许可的已有连线都会在改动前报错。显式指定插口的旧调用仍保留原有替换行为。
```text
add_node(type="ControlNetApplyAdvanced", wire_from=[20, 21], wire_to=5)
add_node(type="LoraLoader", wire_to=[5, 2, 3])
```
```text
add_nodes(nodes={
"enc": {"class_type": "VAEEncode", "inputs": {"pixels": ["#12", "IMAGE"], "vae": ["#4", "VAE"]}},
"mask": {"class_type": "SetLatentNoiseMask", "inputs": {"samples": ["enc", 0], "mask": ["#12", "MASK"]}, "wire_to": 3}
}, replace=true)
```
一组节点一次加好:`#12`(加载图片)和 `#4`(模型加载器)是画布上已有的节点,`enc`、`mask` 是这组里新加的。`wire_to=3` 把这组的结果接进采样器 #3;它原来的空白潜空间被替换,所以要 `replace=true`。原来要十几次调用的局部重绘分支,现在一次。
`wire_to` 指向的输入原来已经接着线时,会把新节点插在前面:采样器 #5 的正向、反向原来接着提示词 #2、#3,现在改接 ControlNet,#2、#3 接进它同名的正向、反向输入;#20(ControlNet 加载器)和 #21(加载图片)按类型接上。第二个调用把采样器的模型线和两个提示词的 CLIP 线都改从 LoRA 走,LoRA 自己接上原来的模型和 CLIP。新节点没有这种类型的输入(比如用加载图片顶替原来的图)才算替换,要 `replace=true`。
成功后直接返回实际接线、还缺的输入和是否通向输出,通常不用再读整张画布。输出可达只说明存在路径,运行前仍由 ComfyUI 校验。一次成功操作可用原生 Ctrl+Z 撤销;执行中连接失败会恢复原接线,第三方节点回调若妨碍恢复会明确报出,需要检查画布。升级后重启 ComfyUI、刷新页面,并重新连接 MCP;若 AI 软件缓存工具定义,再开一个对话。
**AI 干活时你自己改的东西**:AI 在做一个任务时(10 分钟内调用过),你在 AMB 里做的改动,比如画布上的改动、改模型信息、存搭配或提示词,会以“你在 AMB 里改了”出现在这个任务的时间线上,画布改动带修改前后。AI 下一次调用任何工具时会收到它(并被告知保留你的改动,不要改回去),时间线上写着它是否已经知道。操作记录页里,AI 做的和 AI 干活时你做的记录旁边有 **AI 任务 →**。
画布操作下面直接显示操作记录中的 **修改前 / 修改后**,可以定位对应节点,并在允许时撤销。长提示词可能只保留变化附近的片段;早期操作记录被清理后会提示不可用,不会用现在的画布替代历史值。操作记录页使用相同的改动展示。
左边栏的 **AI 工作台** 记下 AI 通过这个接口做的每一步,按**任务**分好:
- AI 开始干活时会调用 `begin_task` 起个标题;没起的话,同一个 AI 软件连续做的事自动算一个任务,停了 20 分钟以上就算新的。
- 每个任务是一条时间线:查了什么(自动折叠成一行,点开看每一次)、画布上改了什么(AI 连着查资料、改画布、记决策,三步以上、其中至少两步不是查资料时,叠成一行“做了 N 步”,下面一排小标签写做了什么,点开看每一步;“撤销这一串”一下撤掉这一串在画布上的改动)、运行、下载、扫描、配音,还有 AI 自己的留言(`report` 工具,也可以把它用别的方式做出来的图交上来)。
- **做出来的东西都留着,点一下就到**:运行出的图会一直等到跑完再收进任务里,最上面的“这次做出来的”可以直接点开看大图;用到的模型点一下打开模型页,搭配打开搭配,配音可以直接听;画布上的改动可以“在画布上找到”,也可以**撤销**(一步,或者整个任务的画布改动一起,从最后一步往前,遇到你自己后来又改过的地方就停下)。
- AI 还在干活时,时间线会自己往下长;你在别的页面时,左边栏的 AI 工作台会亮一个小点。首页也有“AI 最近做了什么”。
- **给 AI 留言**:任务时间线下面有个留言框,比如“背景再亮一点”“先别跑了”。留言会成为任务里的一步,AI 下一次调用任何工具(或 `get_answers`)时都会自动收到,下面写着它有没有看到。AMB 没法叫醒已经停下的 AI:它如果已经结束回合,请回到 AI 软件里让它继续。
- 左边的任务按日期分组(今天、昨天、某天),前面的小图标表示状态:● 进行中、? 等你回答、! 有一步出错、✓ 完成。
- **新任务**:点任务列表上方的 **+ 新任务**,写上要 AI 做什么(也可以空着)。这个任务会显示“等 AI 接手”,下一个来调用的 AI 就会接手它。页面最上面是要发给 AI 的那句话和 **复制这句话**,往下滚也一直在顶上。等的时候可以展开 **从图库挑图给 AI**,在大图库里点图片,它们会放到最下面的留言框上;在留言框里写上要交代的(@ 提到模型和搭配),用留言框左边的 **素材** 按钮挑模型和节点,点发送。这些都是可选的,备好的都列在那页,AI 接手时一起收到。之后的步骤都记在这里,不会接在它上一个任务后面。AI 即使先查资料再调用 `begin_task`,也会继续这个任务,保留你定的标题;只有另一件事才用 `new_task=true` 明确另开,或先用 `report(done=true)` 结束当前任务。
- 任务可以改名、从列表移除(只删记录,图和画布不受影响)。记录存在 ComfyUI 的 user 文件夹里,保留最近 300 个。
- 只看得到经过这个接口的事。AI 绕开它做的,可以让它用 `report` 报告。
**删掉任务里的图**:在图库里删掉的图(进回收站)在 AI 工作台里显示成“图片已删除”的占位,点不开也不能再 @;AI 会被告知你删了它,不会再拿它来用。从回收站还原后,它会自己重新显示出来。
**关联任务**:AI 开新任务时,或者你看着某个任务点“新任务”时,新任务会记住它“接着”哪个任务。任务标题下面显示“接着:某任务”和“后续:某任务”,点一下就跳过去,任务列表里后续任务前面有 ↳。关联错了点 × 解开;没关联上的点“接着哪个任务…”自己选。AI 接手一个后续任务时,会先收到上一个任务里定下的决策。
**AMB 之外**:AMB 只看得到 AI 发给它的调用,AI 软件自己跑的命令和脚本它看不到。所以 AI 每调用一次 AMB,AMB 就看一眼输出和输入文件夹最上面一层:上次调用以来,有没有不是出图产生的新图片,有没有图片被原地改写(做出来一分钟以后又被改了)。有的话,时间线上多一行橙色的 **AMB 之外**,写明是哪些文件(输出文件夹里的图直接显示出来),AI 也会在这次的回复里收到提醒:可能是你自己做的,如果是它做的,改图只能跑工作流,并用 `log_outside` 说明。只在 AI 调用时看,不在后台扫描;只看文件名和时间,最多看两万个文件;ComfyUI 正在出图时,新图要等这次出图结束再判断(那时 ComfyUI 的记录里有它保存的图)。输入文件夹只看被改写的图(你上传的图和新建的看起来一样);Linux 上系统不记文件的创建时间,只看输出文件夹,写作“新建或改写”。AI 自己用 `log_outside` 说的也是一行 **AMB 之外**,写着做了什么、为什么、动到的文件。
**给 AI 的规则**:连接说明里的第 ④ 步是一段现成的规则(先用 AMB 的工具、只通过跑工作流改图、在 AMB 里问和说明、用了别的工具就 `log_outside`)。放进 AI 软件的“规则”里,它每次对话都会读到,比只靠 MCP 自带的说明更管用(Antigravity:Customizations → Rules → + Global)。AI 不一定每次都照做,所以上面那一层由 AMB 自己看。
**决策记录**:讨论在哪里都行,在 AI 自己的对话框里聊也可以,但商量出的结论会留在 AMB。AI 用 `log_decision` 把“定了什么、为什么、没选哪些、谁定的、跟哪几步有关”记成时间线上一张带 📌 的卡片;你在 AMB 里自己做的选择(面板上的回答、允许或不允许、交出去的素材、AI 改画布的地方、停止运行)会自动算进去。窗口右上角的 **决策** 按时间列出这个任务的所有决策,每条都能跳回时间线上的那一步;AI 跑了图或改了画布却很久没记录决策时,这里会轻轻提一句。AI 下次接手这个任务时,也会先收到这些决策。
**成果**:时间线最上面一行写着这个任务做出了什么(“本次成果:8 张图 · 1 个工作流 · 1 个搭配”),点 **在右侧看**,或者窗口右上角的 **成果**,右栏就按新的在上列出来:图片按每次跑图分组(标着第几步、几点,点一下跳回时间线上那一步;每张图照样能 ☆ 保存、勾两张对比参数),AI 搭的工作流(任务自己的标签页或你开着的画布,**在画布打开** 切过去),用过的搭配,最新那张图的正向和负向提示词(各有 **复制**),用到的模型和发起过下载的模型(不代表下载完成),配音(▶ 直接播放)。工作流、搭配、提示词三处都能一键存下:存的是出最新那张图的那一套(**存成配方**、**把最新那张图存成搭配**、**存进提示词库**),和那张图的 ☆ 保存菜单是同一个做法;想存别的图,就点那张图的 ☆。有新成果时,**成果** 按钮上会亮一个数字,它不会自己弹开。
**留言框**一直停在任务这一栏的最下面,上方的“现在”栏显示 AI 出图的进度(可以停止)、要你允许的事(直接点 **允许 / 不要**)和 AI 问你的问题(点选项就回答)。图片可以拖到留言框上,或者 Ctrl+V 粘贴截图:存进输出文件夹的 `ai_inbox` 后加到消息上。它上面显示这条消息要带的图(每张可以 **✎ 标注** 或 **×** 拿掉)和 @ 提到的东西;Enter 发送,Shift+Enter 换行,只发图片也行。往上翻着看时 AI 有了新步骤,会出现 **下面有新的步骤**。**留言框里的 @ 和 /**:在留言框里打 **@**,会弹出一张能搜索的列表:这个任务出过的图、图库里最新的图(打字时也按提示词、模型、seed 在图库里搜)、搭配、模型、工作流、提示词、节点包、节点。留言框左边的 **图库** 按钮打开图库:点选几张(一次最多 8 张),点 **加进留言**,它们显示在输入框上方。用 @ 选中的会写成“@名字”,并在输入框上方显示成小标签;发出去时 AI 收到的是你指的那个东西本身(模型的准确 ID、搭配的参数、图片和它的生成参数),不用它去猜。在开头打 **/** 是快捷指令:/停止、/撤销、/展台、/节点结构、/素材、/副本、/当前画布、/新任务 马上生效,不用等 AI;/重跑、/解释、/继续 帮你写好一句话,改一改再发。↑↓ 选,回车或 Tab 确认,Esc 关掉。
**AI 画的面板**(`show_panel`)显示在工作台右边的**展台**上,后面再多步骤也不会被顶走。AI 放了新面板,右栏自己打开并跳到它,你关掉了或者在看别的也一样(AI 只是改了已有的面板时不会);不想被打扰,把展台顶上的 **有新东西就打开** 关掉。窗口窄时它不抢时间线,只在“展台”标签上亮个点。右边这一栏一次只显示一样东西,用窗口右上角的按钮切换:**展台**(有问题等你回答时带个小黄点)、**节点结构**(AI 跑过的节点结构图,大图;上面可以选看哪一次运行,时间线里运行那一步的“在右侧看大图”也会打开它)、**决策**、**素材**、**成果**。再点一下正亮着的那个就收起这一栏;切走再切回来,展台的位置和挑了一半的素材都还在。任务标题和那一排按钮在时间线往下翻时一直停在顶上。时间线上每个面板留一张小卡片:小图标、标题、里面有什么(“1 张图”“问你 · 2 个选项”),和紫色的 **在右侧打开** 按钮;右栏正在显示的那一张写着绿色的“正在右侧显示”。展台默认跟着最新的面板,你翻回旧面板就停在你看的那张;窗口窄的时候,时间线和展台变成两个标签页切换。右边这一栏是一张单独的卡片,它左边的整条空隙都能拖着调宽窄(鼠标靠近时整条边会亮起;点一下后也能按 ← → 调;AMB 记得你调的宽度,双击恢复默认),时间线总会留够地方。窗口左上角的按钮收起 / 展开左边的任务列表;收起以后,鼠标移到这个按钮上,任务列表会临时浮出来,点一个任务或者鼠标移开就自动收回。素材在右边这一栏里时,鼠标移过卡片不会再跳进详细说明,点右边的 ⓘ 才看。小窗里素材是一行一个的列表(图标、名字、来源、勾选),搜索框旁边三个小图标是“全部 / 已选 / 常用”;展开成大窗口就是带预览的卡片。这些地方的滚动条平时是隐藏的,鼠标移到能滚动的那一块上才显出来。AI 可以用插件提供的“积木”拼面板:标题和文字(也可以是提示、警告)、图片(一排、网格,或者两张并排对比)、你的模型(带封面的卡片,点了打开)、表格、列表、参数、进度条。它还能**问你**:几个按钮(单选或多选)、一行输入框。你的回答会交出去,等 AI 用 `get_answers` 来取(它可以等你最多一分钟);每个问题下面有一个状态标签:琥珀色的 **已发送 · 等 AI 来取**,AI 拿到后变成绿色的 **AI 已收到 · 时间**。你的留言、批注、发给它的图片也是一样。想让 AI 自己决定的题点 **让 AI 定**,AI 会自己拿主意,不再问一遍。AMB 没法主动叫醒 AI,所以回答之后要等它下一次来问。
**展台积木**(1.62):AI 还可以在图上**画圈标注**(圈上是编号,标签列在图下面,点标签放大那一块)、摆**拉帘对比**(两张图叠在一起,拖中间的线)、摆**选片盘**(每张图带 AI 的理由和“就选这张”;“放回画布”会在新标签页里打开这张图里存的工作流,也可以直接拖到画布上)、画**示意图**(框和箭头由插件排版;带 ◎ 的框点一下定位到画布上的节点)。AI 只给位置和文字,圈、线、框都是插件画的,所以接哪家 AI 都一样好看,也不会收 AI 写的 SVG 代码。
面板里的东西都由插件检查过才画出来:只认这几种积木,只是纯文字,图片只能来自 ComfyUI 的 output 文件夹,模型只能是你自己的,AI 写的代码不会运行。AMB 没开着的时候 AI 画了面板,右下角会弹一个小卡片“AI 在问你”,点“去看看”就到。
展台右上角的 **× 关闭展台**只收起这一栏,不删除面板、不结束任务;AI 改已有的面板时不会重开,放新面板时会重开(把 **有新东西就打开** 关掉就不会)。窗口右上角的“展台”,或时间线里的“在右侧打开”可以恢复,未提交的输入会保留。“来自第 N 步”指创建这个面板的那一步;后续修改同一个面板不会改变它的来源编号。
**按过程往下看**:建议 AI 把“搭配方案 → 节点说明 → 成图结果”作为不同阶段依次展示。更新同一个面板时,新内容也只接在底部,已有内容不会因为 AI 重排或省略而消失。`show_panel` / `update_panel` 会返回积木 ID;明确带上已有 ID 才是在原位置修正那块内容,进度条也可以按唯一的标签更新。单面板最多 30 块,满了就用 `show_panel` 展示下一阶段。面板里的进度条是 AI 自己写的数,不会自己动:任务往前走了(那次出图结束、AI 又画了面板或留了言、任务结束)而它没有更新,展台就把它画成灰色,并写明是几点写下的;出图的真实进度在时间线那一步和标题栏的运行状态里。旧记录只保留了最终排版,没有每次更新前的顺序,因此不会自动重排旧记录。
## 6. 注意
- **隐私**:如果 AI 软件用的是云端模型,查询结果(模型名、提示词、图片的生成参数)会发给那家 AI 服务商。附上封面或缩略图时,图片也会一起发过去。介意的话可以用本地模型,或者不让 AI 附图。
- 附图时会用到插件自己的缩略图缓存,和浏览图库时一样,不动你的原图。
@@ -116,6 +247,21 @@ claude mcp add --transport http anomalous http://127.0.0.1:8188/anomalous/mcp
## English
Output pictures in a task and on AI panels carry the existing image inspector, image/canvas parameter comparison and Keep menu as tools shown on hover; tick any two (any step or task) and the bar at the foot of the page compares their actual embedded workflows. Changes you make in Anomalous while an AI works on a task (it called within 10 minutes) join its timeline and reach the AI once with its next call, marked as yours to keep; activity entries link to their task. Canvas steps show recorded before/after values with node navigation and undo where available, shared with the activity page. Missing historical receipts and images without reusable workflows are reported explicitly.
Apps that support MCP Apps show Anomalous's own card (`ui://anomalous/task-card.html`) under begin_task, show_panel, report and run_workflow results: the task's steps, latest thumbnails and the AI's open questions, answered from the card (gates are never answerable there, only in Anomalous). Four prompts (workbench_task, pick_best, fix_missing_models, explain_canvas) appear as slash commands.
The stage reads chronologically: use `show_panel` for successive phases. `update_panel` appends
new blocks below existing content, keeps omitted blocks and deduplicates unchanged resubmissions.
Tools return stable block ids for deliberate in-place corrections (including progress/questions);
an id keeps its type and position. A uniquely labelled progress bar also updates without an id.
A progress bar holds the AI's own numbers: once the task goes past it without an update (a run
ends, a later panel or note, the task ends), the stage greys it and says when it was written.
The total remains bounded to 30 blocks; use a new panel for the next phase. Old records retain
their saved layout because earlier revisions were not recorded.
While ComfyUI runs, Anomalous serves an MCP endpoint at `http://127.0.0.1:8188/anomalous/mcp` (use your ComfyUI port). Add it as a Streamable HTTP server in your AI app (Claude Code: `claude mcp add --transport http anomalous <url>`; Cursor and VS Code: a `url` / `"type": "http"` entry; Claude Desktop: through `npx mcp-remote <url> --transport http-only`). Both the 2026-07-28 protocol and the earlier `initialize`-based versions are served.
Reading tools: library overview, model search and details (optionally with the cover), scan status, output-image search and generation settings (optionally with a thumbnail), combos, workflow recipes, saved prompts, generated audio and the activity log. Acting tools: read the open workflow, write prompts, swap a node's model, add a LoRA, place a combo, check and fix missing models, run the workflow, open a model in Anomalous, start scans, list Anomalous TTS voices and have one speak. Canvas actions need ComfyUI open in a browser on this computer; each is one Ctrl+Z step and is logged as the AI's. Nothing deletes or renames files. Only this computer may connect; requests through a proxy, from other hosts or from web pages are refused. When your AI app uses a cloud model, what the tools return (and any attached picture) is sent to that provider.
Since 1.62 the **AI workbench** page keeps every step an AI app takes through this endpoint, grouped into tasks (named with `begin_task`, else one app's calls without a 20-minute pause): what it is currently running (`running` in real time) and what it looked up, changed, ran, downloaded, scanned or spoke, its notes (`report`, which also hands in pictures made another way), and what each step made, with links to the picture, the model, the combo, the canvas change (find or undo) and the audio. It updates live; the rail gets a dot while you are elsewhere. With `show_panel` the AI draws a panel there from fixed blocks (text, pictures to compare, your models as cards, tables, settings, progress, buttons and a text line); the user's answers and task messages are returned on `get_answers` (which can wait up to 55 s) and are also automatically attached once to the AI's next successful tool call. Blocks are checked on the server and drawn as text; nothing the AI sends runs.
Reading tools: library overview, model search and details (optionally with the cover; each model says which node types load it, and a model in the wrong folder for what it is says where it belongs), scan status, output-image search and generation settings (optionally with a thumbnail), combos, workflow recipes, saved prompts, generated audio and the activity log. Acting tools: read the open workflow, write prompts, swap a node's model, add a LoRA, place a combo, check and fix missing models, run the workflow, open a model in Anomalous, start scans, list Anomalous TTS voices and have one speak. Canvas actions need ComfyUI open in a browser on this computer; each is one Ctrl+Z step and is logged as the AI's. Nothing deletes or renames files; `move_model` moves a misfiled model (with its covers and notes) to its folder, after the user says yes unless the "files" gate lets it go ahead. Only this computer may connect; requests through a proxy, from other hosts or from web pages are refused. When your AI app uses a cloud model, what the tools return (and any attached picture) is sent to that provider.
+9
View File
@@ -48,6 +48,15 @@ node tests/third_party_loras.mjs
## 4. 写新测试
结果集成的浏览器回归测试是 `tests/ai_results.html`:验证勾选两张图对比(视频和临时文件不能勾、第三张替换最早的、清除)、真实参数对比的读取/应用/撤销、撤销按钮的遮罩层级、保存菜单、视频没有参数工具、图片按需展开,以及安全的画布前后值展示。只替换网络与宿主图边界,加载真实插件模块。通过时页面末尾显示 `ALL PASSED`;它需要单独用浏览器运行,不包含在下面的命令行测试里。
展台的浏览器回归测试是 `tests/ai_side.html`:在插件目录启动仅本机可访问的静态服务器
(`python -m http.server 8202 --bind 127.0.0.1`),用浏览器打开 `/tests/ai_side.html`。
页面应显示 `ALL PASSED`,覆盖旧记录、长面板、面板更新、图片加载、翻阅时保留位置和隐藏标签页。
它加载实际展台模块及样式,只替换不用的 ComfyUI 画布接口;不包含在上面的一键命令中。
同一静态服务器上的 `tests/ai_drafts.html` 验证展台和底部提问条的回答草稿同步、重画保留文字和选区、同时刷新不抢错焦点。它加载实际组件,仅替换网络和宿主边界;页面应显示 `ALL PASSED`,不包含在一键命令中。
- 前端规则测试放 `tests/<名字>.mjs`,用 `node:assert`,直接导入 `web/modules/` 里的真实模块,只替换浏览器和 ComfyUI 这些外部部分。
- 后端测试放 `tests/test_<名字>.py`,用 `unittest`。文件一律建在 `tempfile` 的临时文件夹里,网络请求用 `unittest.mock` 换掉。
- 测试里不要写自己电脑上的真实路径、真实模型名或个人信息。
Binary file not shown.

Before

Width:  |  Height:  |  Size: 384 KiB

After

Width:  |  Height:  |  Size: 406 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 449 KiB

After

Width:  |  Height:  |  Size: 480 KiB

+8 -1
View File
@@ -30,6 +30,11 @@ LORA_MARKS = ("lora_up.", "lora_down.", "lora_a.", "lora_b.", "lora.up.", "lora.
"lokr_w1", ".lora_linear_layer.", "dora_scale")
CONTROL_MARKS = ("control_model_", "input_hint_block", "controlnet_cond_embedding", "controlnet_blocks",
"controlnet_x_embedder", "zero_convs", "controlnet_mid_block")
# Tensors only a model patch has (ComfyUI's "Load Model Patch" node, models/model_patches): Anima
# LLLite, the Qwen-Image block-wise and Z-Image Fun ControlNets, SigLIP features, Wan's Uni3C. Some
# are called ControlNets, but the ControlNet loader cannot read them.
PATCH_KEYS = {"lllite_conditioning1.conv1.weight", "controlnet_blocks.0.y_rms.weight", "feature_embedder.mid_layer_norm.bias",
"control_all_x_embedder.2-1.weight", "controlnet_patch_embedding.weight"}
DIFFUSION_STARTS = ("double_blocks.", "single_blocks.", "joint_blocks.", "transformer_blocks.",
"single_transformer_blocks.", "input_blocks.", "blocks.0.self_attn", "net.blocks.", "net.x_embedder.")
@@ -210,6 +215,8 @@ def _kind(keys, raw, text):
names = set(keys)
if names and names <= EMBEDDING_KEYS and names & {"emb_params", "clip_l", "clip_g", "string_to_param"}:
return "embeddings", True
if names & PATCH_KEYS:
return "model_patches", True
if any(mark in text for mark in CONTROL_MARKS):
return "controlnet", True
diffusion = any(key.startswith(("model.diffusion_model.", "diffusion_model.") + DIFFUSION_STARTS) for key in keys)
@@ -252,7 +259,7 @@ def classify(fmt, header):
raw = "\n".join(keys).lower()
text = raw.replace(".", "_")
kind, sure = _kind(keys, raw, text)
if kind in ("vae", "text_encoders", "clip_vision", "upscale_models", "ipadapter"):
if kind in ("vae", "text_encoders", "clip_vision", "upscale_models", "ipadapter", "model_patches"):
base, base_sure = "", True # shared by several base models: the type's own folder
elif kind == "embeddings":
base, base_sure = _embedding_base(header), True
+20 -2
View File
@@ -24,8 +24,9 @@ assert.deepEqual(diffSnapshots(before, after).map(c => [c.kind, c.node, c.widget
]);
assert.deepEqual(diffSnapshots(before, before), []);
// Buttons and nameless widgets hold no value.
const withButton = snapshotGraph(graph([{ id: 1, type: 'X', title: 'X', widgets: [{ name: 'run', type: 'button', value: 1 }, { value: 5 }] }]));
// Buttons, nameless widgets and ComfyUI's own preview hold no value.
const withButton = snapshotGraph(graph([{ id: 1, type: 'X', title: 'X', widgets: [{ name: 'run', type: 'button', value: 1 }, { value: 5 },
{ name: '$$canvas-image-preview', value: 'a.png' }] }]));
assert.deepEqual(withButton.get('1').widgets, {});
// Object values compare by content; a widget that appears counts as a change.
@@ -33,6 +34,23 @@ const a = snapshotGraph(graph([node(1, 'Img', { image: { name: 'a.png' } })]));
const b = snapshotGraph(graph([node(1, 'Img', { image: { name: 'a.png' }, extra: 2 })]));
assert.deepEqual(diffSnapshots(a, b).map(c => [c.widget, c.before, c.after]), [['extra', '', '2']]);
// Rewiring: what feeds an input, in words; not the inputs a removed node fed.
const wired = (links, extra = []) => {
const nodes = [
{ id: 1, type: 'CheckpointLoaderSimple', title: 'Load', outputs: [{ name: 'MODEL' }] },
{ id: 2, type: 'LoraLoader', title: 'LoRA', outputs: [{ name: 'MODEL' }] },
{ id: 3, type: 'KSampler', title: 'KSampler', inputs: [{ name: 'model', link: links.model ?? null }] },
...extra,
];
const table = new Map([[10, { origin_id: 1, origin_slot: 0 }], [11, { origin_id: 2, origin_slot: 0 }]]);
return snapshotGraph({ _nodes: nodes, links: table, getNodeById: id => nodes.find(n => n.id === id) });
};
assert.deepEqual(diffSnapshots(wired({}), wired({ model: 10 })).map(c => [c.kind, c.node, c.widget, c.before, c.after]),
[['wired', 'KSampler #3', 'model', '', 'Load #1 · MODEL']]);
assert.deepEqual(diffSnapshots(wired({ model: 10 }), wired({ model: 11 }))[0].after, 'LoRA #2 · MODEL');
const gone = snapshotGraph({ _nodes: [{ id: 3, type: 'KSampler', title: 'KSampler', inputs: [{ name: 'model', link: null }] }] });
assert.deepEqual(diffSnapshots(wired({ model: 10 }), gone).map(c => c.kind), ['removed', 'removed'], 'the removal says it');
assert.equal(changeNodeId({ node: 'KSampler #12' }), '12');
assert.equal(changeNodeId({ node: 'Title with # sign #7' }), '7');
assert.equal(changeNodeId({ node: 'no id' }), null);
+40
View File
@@ -0,0 +1,40 @@
import assert from 'node:assert/strict';
import { callLines, NEW_CHAT } from '../web/modules/ai_call_lines.js';
const ai = (tool, more = {}) => ({ id: tool, tool, kind: 'look', state: 'done', at: 1, ...more });
const first = task => callLines(task)?.lines[0];
assert.equal(callLines(null), null);
assert.equal(callLines({ ended: 5, steps: [ai('report')] }), null, 'an ended task waits for nothing');
assert.equal(callLines({ ended: 5, steps: [ai('report'), { tool: 'user_marks', kind: 'user', delivered: false }] })?.lines[0], 'aiCallReplied',
'unless the user wrote to it afterwards');
const waiting = { waiting: true, steps: [{ tool: 'user_message', kind: 'user', delivered: false }] };
assert.equal(callLines(waiting), null, 'a task waiting for its first AI has the line that starts it on its own card');
const task = { title: 'Sleepy girl', steps: [ai('begin_task'), ai('report', { kind: 'note' })] };
assert.deepEqual(callLines(task), { lead: 'aiCallLead', lines: ['aiCallGoOn', NEW_CHAT] });
task.steps.push({ tool: 'user_message', kind: 'user', delivered: false });
assert.equal(first(task), 'aiCallReplied', 'words the AI has not got');
task.steps.at(-1).delivered = true;
assert.equal(first(task), 'aiCallGoOn');
task.steps.push({ tool: 'user_activity', kind: 'user', delivered: false });
assert.equal(first(task), 'aiCallGoOn', 'something the user changed is not a reply');
const panel = ai('ask_user', { kind: 'panel', answers: { answer: { choice: ['fix'], delivered: false } } });
task.steps.push(panel);
assert.equal(first(task), 'aiCallReplied', 'an answer on a panel the AI stopped waiting for');
panel.answers.answer.delivered = true;
const gate = ai('run_workflow', { kind: 'run', state: 'allowed', gate: { timed_out: true, decision: 'allow', delivered: false } });
task.steps.push(gate);
assert.equal(first(task), 'aiCallReplied', 'a gate answered after the AI stopped waiting');
gate.gate.delivered = true;
const run = ai('run_workflow', { kind: 'run', state: 'done', call_ai: true });
task.steps.push(run, { tool: 'user_message', kind: 'user', delivered: false });
assert.deepEqual(callLines(task), { lead: 'aiCallDoneLead', lines: ['aiCallDone', 'aiCallAgain', NEW_CHAT] }, 'a run that ended comes first');
run.state = 'error';
assert.deepEqual(callLines(task).lines, ['aiCallFailed', NEW_CHAT]);
delete run.call_ai;
assert.equal(first(task), 'aiCallReplied', 'once told of the run, the reply is left');
console.log('ai_call_lines: lines by what waits for the AI passed');
+46
View File
@@ -0,0 +1,46 @@
<!doctype html>
<meta charset="utf-8">
<title>AI answer draft regression</title>
<script type="importmap">{"imports":{"/scripts/app.js":"data:text/javascript,export const app = {};","/scripts/api.js":"data:text/javascript,export const api = new EventTarget();"}}</script>
<pre id="results">Running…</pre>
<div id="stage"></div><div id="now"></div>
<script type="module">
import { renderPanel } from '../web/modules/ui_ai_panel.js';
import { nowStrip } from '../web/modules/ui_ai_now.js';
// No real task writes: only this fixture's fake typing/answer boundary is replaced.
window.fetch = async () => ({ ok: true, json: async () => ({}) });
const results = document.querySelector('#results');
const stage = document.querySelector('#stage');
const strip = nowStrip();
document.querySelector('#now').append(strip);
const task = { id: 'draft-test', steps: [{ id: 'q1', kind: 'panel', panel: {
id: 'panel1', title: 'Question', blocks: [{ type: 'input', id: 'answer', question: 'Your answer?' }]
}, answers: {} }] };
const ctx = { taskId: task.id, changed() {}, refresh() {}, openStage() {}, showStep() {} };
const redraw = () => stage.replaceChildren(renderPanel(task.steps[0], ctx));
const settle = () => new Promise(resolve => requestAnimationFrame(() => requestAnimationFrame(resolve)));
let failures = 0;
const check = (condition, text) => { results.textContent += `${condition ? 'PASS' : 'FAIL'} ${text}\n`; if (!condition) failures++; };
try {
results.textContent = '';
redraw(); strip.update(task, ctx);
let field = stage.querySelector('input');
field.value = 'abcdef';
field.dispatchEvent(new Event('input'));
check(strip.querySelector('input').value === 'abcdef', 'same question shares its draft between stage and bottom strip');
field.focus(); field.setSelectionRange(1, 4, 'backward');
redraw(); await settle();
field = stage.querySelector('input');
check(field.value === 'abcdef', 'redraw preserves answer text');
check(document.activeElement === field, 'redraw restores focus');
check(field.selectionStart === 1 && field.selectionEnd === 4 && field.selectionDirection === 'backward', 'redraw preserves selection instead of jumping to the end');
const bottom = strip.querySelector('input');
bottom.focus();
redraw();
task.steps.push({ id: 'q2', kind: 'panel', panel: { id: 'panel2', blocks: [{type: 'input', id: 'answer'}] }, answers: {} });
strip.update(task, ctx);
await settle();
check(document.activeElement === strip.querySelector('input'), 'simultaneous refresh keeps focus on the surface the user was editing');
results.textContent += failures ? `${failures} FAILED` : 'ALL PASSED';
} catch (error) { results.textContent += `FAIL ${error.stack}`; }
</script>
+124
View File
@@ -0,0 +1,124 @@
<!doctype html>
<meta charset="utf-8">
<title>AI result integration regression</title>
<link rel="stylesheet" href="../web/styles/07-prompt-studio.css">
<link rel="stylesheet" href="../web/styles/27-param-compare.css">
<link rel="stylesheet" href="../web/styles/37-ai-workbench.css">
<link rel="stylesheet" href="../web/styles/41-ai-results.css">
<style>
:root { --amb-bg-panel:#202027; --amb-text-main:#eee; --amb-text-soft:#bbb; --amb-text-dim:#999; --amb-border:#454550; --amb-link:#b39eef; --amb-ok:#52b98d; --amb-ink-rgb:240,240,250; --amb-scrim:#0008; }
body { background:#17171b; color:#eee; font:14px sans-serif; margin:20px; }
#fixture { max-width:760px; } #results { white-space:pre-wrap; }
</style>
<!-- Host graph and network are the only substituted boundaries. All AMB rendering/actions are real imports. -->
<script type="importmap">{"imports":{"/scripts/app.js":"data:text/javascript,export const app = globalThis.ambTestApp;","/scripts/api.js":"data:text/javascript,export const api = new EventTarget();"}}</script>
<script>
const sampler = cfg => ({id:3,type:'TestSampler',title:'Sampler',mode:0,inputs:[],widgets:[{name:'cfg',value:cfg},{name:'seed',value:123}]});
const graph = {_nodes:[sampler(7)],links:{},getNodeById(id){return this._nodes.find(node=>String(node.id)===String(id));}};
globalThis.ambTestApp = {graph,rootGraph:graph,canvas:{graph}};
globalThis.LiteGraph = {LGraph:class {configure(workflow){this._nodes=workflow.nodes;this.links={};}}};
const requests = [];
globalThis.fetch = async (url,options={}) => {
requests.push(String(url));
if(String(url).includes('inspect_image_material')) {
const source=JSON.parse(options.body).source_image;
if(source.filename==='missing.png') return new Response('{}',{status:400});
return new Response(JSON.stringify({workflow:{nodes:[sampler(source.filename==='b.png'?8:7)]}}),{headers:{'Content-Type':'application/json'}});
}
if(String(url).includes('kept_images')) return new Response(JSON.stringify({images:[]}));
throw Error(`Unexpected request: ${url}`);
};
</script>
<pre id="results">Running…</pre>
<div id="fixture"></div>
<script type="module">
import { compareBar, imageResults } from '../web/modules/ui_ai_images.js';
import { canvasChanges } from '../web/modules/ui_canvas_changes.js';
import { closeKeepMenu } from '../web/modules/ui_keep_menu.js';
import { slider } from '../web/modules/ui_ai_stage.js';
const fixture=document.querySelector('#fixture'), results=document.querySelector('#results');
const a={filename:'a.png'}, b={filename:'b.png'}, c={filename:'c.png'}, video={filename:'clip.mp4'};
const check=(value,message)=>{if(!value)throw Error(message);results.textContent+=`PASS ${message}
`;};
const settle=()=>new Promise(resolve=>requestAnimationFrame(()=>requestAnimationFrame(resolve)));
const waitFor=async predicate=>{for(let i=0;i<120;i++){if(predicate())return;await settle();}throw Error('Timed out');};
const closeCompare=()=>document.querySelector('.anomalous-compare-done').click();
const tickOf=(box,name)=>[...box.querySelectorAll('figure')].find(f=>f.querySelector('img')?.alt===name)?.querySelector('.anomalous-ai-tick');
try {
results.textContent='';
const bar=compareBar();
const cards=imageResults([a,b,a,video,{filename:'a.png',folder:'temp'}],{});
fixture.append(cards,bar);
check(cards.querySelectorAll('.anomalous-ai-tick').length===2,'only output pictures can be ticked (no videos or temp files, no duplicates)');
check(requests.length===0,'rendering result tools does not preload metadata');
check(bar.hidden,'no compare bar until a picture is ticked');
tickOf(cards,'a.png').click();
check(!bar.hidden && bar.querySelector('.anomalous-ai-compare-go').hidden,'one tick asks for a second');
tickOf(cards,'b.png').click();
check(!bar.querySelector('.anomalous-ai-compare-go').hidden,'two ticks can be compared');
const later=imageResults([c],{});fixture.append(later);
tickOf(later,'c.png').click();
check(!tickOf(cards,'a.png').classList.contains('is-on') && tickOf(cards,'b.png').classList.contains('is-on'),'a third tick replaces the oldest, also on pictures drawn before');
bar.querySelector('.anomalous-ai-compare-go').click();
await waitFor(()=>document.querySelector('.anomalous-compare-row:not(.is-head)'));
check([...document.querySelectorAll('.anomalous-compare-value')].map(n=>n.textContent).join(',')==='8,7','actual parameter comparer reads both ticked workflows');
closeCompare();
const tools=[...cards.querySelectorAll('figure')].find(f=>f.querySelector('img')?.alt==='b.png').querySelector('.anomalous-ai-pic-tools');
tools.children[1].click();
await waitFor(()=>document.querySelector('.anomalous-compare-apply'));
document.querySelector('.anomalous-compare-apply').click();
check(ambTestApp.graph._nodes[0].widgets[0].value===8,'real comparison apply writes the canvas parameter');
await waitFor(()=>document.querySelector('.anomalous-mixer-toast.is-show'));
const undo=document.querySelector('.anomalous-mixer-toast-action');
const r=undo.getBoundingClientRect();
check(undo.contains(document.elementFromPoint(r.x+r.width/2,r.y+r.height/2)),'Undo remains clickable above the comparison overlay');
undo.click();
check(ambTestApp.graph._nodes[0].widgets[0].value===7,'real comparison undo restores the prior value');
check(ambTestApp.graph._nodes[0].widgets[1].value===123,'seed is never changed by comparison apply');
closeCompare();
tools.children[2].click();
await waitFor(()=>document.querySelector('.anomalous-keep-row'));
check(document.querySelectorAll('.anomalous-keep-row').length===3,'result Keep uses the existing workflow/combo/prompt menu');
closeKeepMenu();
check(![...cards.querySelectorAll('figure')].find(f=>f.querySelector('video'))?.querySelector('.anomalous-ai-pic-tools'),'videos offer no unsupported parameter or save-workflow tools');
bar.querySelector('.anomalous-ai-compare-clear').click();
check(bar.hidden && !fixture.querySelector('.anomalous-ai-tick.is-on'),'clearing unticks every picture');
const many=imageResults(Array.from({length:15},(_,i)=>({filename:`${i}.png`})),{});
fixture.append(many);
check(many.querySelectorAll('figure').length===12,'initial image DOM is bounded');
many.querySelector(':scope > button').click();
check(many.querySelectorAll('figure').length===15,'more results are reachable on demand');
const moved=[];
const receipt=canvasChanges({detail:{total:5,changes:[
{kind:'changed',node:'Sampler #3',widget:'cfg',before:'7',after:'8'},
{kind:'changed',node:'Prompt #6',widget:'text',before:'<img src=x onerror=alert(1)>',after:'soft light'},
{kind:'removed',node:'Old #9'},
{kind:'added',node:'New #10'},
]}},{locate:change=>moved.push(change.node)});
fixture.replaceChildren(receipt);
check(receipt.querySelectorAll('.anomalous-canvas-change').length===3,'receipt initially shows three changes');
check(!receipt.querySelector('img'),'recorded text is not interpreted as markup');
check(!receipt.querySelector('.is-removed button'),'removed nodes do not offer locate');
receipt.querySelector('button').click();check(moved[0]==='Sampler #3','locate passes the recorded node to its owner');
receipt.querySelector('details').open=true;
await settle();
check(receipt.querySelectorAll('.anomalous-canvas-change').length===4,'remaining changes expand without losing prior rows');
check(canvasChanges(null).textContent.includes('不可用'),'old records have an explicit unavailable state');
const tagReceipt=canvasChanges({detail:{total:1,changes:[{kind:'changed',node:'Prompt #2',widget:'text',before:'masterpiece, best quality, slime girl, purple hair, smile',after:'masterpiece, best quality, vampire girl, silver hair, smile'}]}});
check(tagReceipt.querySelectorAll('.anomalous-canvas-tag.is-out').length===2&&tagReceipt.querySelectorAll('.anomalous-canvas-tag.is-in').length===2,'a prompt receipt lists the tags taken out and put in');
tagReceipt.querySelector('.anomalous-canvas-change-full').open=true;await settle();
check(tagReceipt.querySelectorAll('.anomalous-canvas-mark.is-in').length===2,'the whole texts mark the tags put in');
const cut=canvasChanges({detail:{total:1,changes:[{kind:'changed',node:'P',widget:'text',before:'…iece, best quality, a, b, slime girl, c, d, smi…',after:'…iece, best quality, a, b, vampire girl, c, d, sm…'}]}});
check([...cut.querySelectorAll('.anomalous-canvas-tag')].map(n=>n.textContent).join('|')==='slime girl|vampire girl','half tags at cut ends are never called changed');
const comparisonSlider=slider({before:a,after:b});
check(Boolean(comparisonSlider.querySelector('.anomalous-ai-slider-compare')),'AI slider connects to real parameter comparison');
const broken=imageResults([a,{filename:'missing.png'}],{});fixture.append(broken);
tickOf(broken,'a.png').click();tickOf(broken,'missing.png').click();
bar.querySelector('.anomalous-ai-compare-go').click();
await waitFor(()=>document.querySelector('.anomalous-compare-summary.is-error'));
check(!document.querySelector('.anomalous-compare-apply'),'missing workflow fails without an apply action');
closeCompare();
results.textContent+='ALL PASSED\n';
document.title='PASS AI result integration';
} catch(error) { results.textContent+=`FAIL ${error.stack}\n`;document.title='FAIL AI result integration'; }
</script>
+115
View File
@@ -0,0 +1,115 @@
<!doctype html>
<meta charset="utf-8">
<title>AI stage browser regression</title>
<link rel="stylesheet" href="../web/styles/40-ai-side.css">
<link rel="stylesheet" href="../web/styles/38-ai-panels.css">
<link rel="stylesheet" href="../web/styles/45-ai-side-views.css">
<style>
body { background: #202027; color: #eee; font: 14px sans-serif; --amb-link: #a58af9; --amb-text-soft: #ccc; --amb-text-dim: #aaa; --amb-ink-rgb: 255,255,255; }
#stage { width: 620px; height: 420px; border: 1px solid #777; }
#results { white-space: pre-wrap; }
</style>
<!-- Only the unused ComfyUI canvas boundary is stubbed; panel DOM and CSS are real. -->
<script type="importmap">{"imports":{"/scripts/app.js":"data:text/javascript,export const app = {};","/scripts/api.js":"data:text/javascript,export const api = new EventTarget();"}}</script>
<div id="results">Running…</div>
<div id="switch"></div>
<div id="stage" class="anomalous-ai-stage"></div>
<script type="module">
import { renderStage, stageNews, stageBadge, focusPanel } from '../web/modules/ui_ai_side.js';
import { sideSwitch } from '../web/modules/ui_ai_side_views.js';
const box = document.querySelector('#stage');
const results = document.querySelector('#results');
const stage = {};
const ctx = { numberOf: id => id, tab() {}, changed() {}, close() {
stage.hidden = true;
box.style.display = 'none';
} };
const panel = id => ({ id, kind: 'panel', panel: { id, title: `Panel ${id}`, blocks:
Array.from({ length: 25 }, (_, n) => ({ type: 'text', text: `${id}: line ${n + 1}` })) }, answers: {} });
let task = { id: 'historical', steps: [panel('1')] };
const settle = () => new Promise(resolve => requestAnimationFrame(() => requestAnimationFrame(resolve)));
const body = () => box.querySelector('.anomalous-ai-stage-body');
const bottom = () => body().scrollHeight - body().clientHeight - body().scrollTop < 2;
const check = (value, label) => { if (!value) throw Error(label); results.textContent += `PASS ${label}\n`; };
async function draw() { const news = stageNews(task, stage); renderStage(box, task, stage, ctx, news.arrived); await settle(); }
try {
results.textContent = '';
const badgeState = {};
const badgeTask = { id: 'badge-test', steps: [panel('a')] };
stageNews(badgeTask, badgeState);
check(stageBadge(badgeTask, badgeState, false).count === 1 && !badgeState.panelUnread, 'historical content has a count without a false unread dot');
badgeTask.steps.push(panel('b'));
check(stageBadge(badgeTask, badgeState, false).unread, 'new panel while hidden lights unread dot');
check(stageBadge(badgeTask, badgeState, false).unread, 'refresh keeps unread dot');
check(!stageBadge(badgeTask, badgeState, true).unread, 'opening the stage clears unread dot');
badgeTask.steps[0].panel.title = 'Older panel edited';
check(stageBadge(badgeTask, badgeState, false).unread, 'editing an older panel lights unread dot too');
const badgeBar = sideSwitch({ view: 'map' }, true, { stage: stageBadge(badgeTask, badgeState, false), asking: 0 }, () => {});
document.querySelector('#switch').append(badgeBar);
check(badgeBar.firstChild.classList.contains('is-unread') && badgeBar.firstChild.querySelector('.is-total').textContent === '2',
'stage button renders muted total and unread indicator');
check(badgeBar.firstChild.getAttribute('aria-label').includes('有未查看'), 'unread is also announced to assistive technology');
check(!stageBadge(badgeTask, badgeState, true).unread, 'viewing the update clears the dot');
badgeTask.steps[0].panel.title = 'Live update';
check(!stageBadge(badgeTask, badgeState, true).unread, 'visible updates do not create unread reminders');
stageNews({ id: 'other-badge-task', steps: [] }, badgeState);
const emptyBadge = stageBadge({ steps: [] }, badgeState, false);
check(emptyBadge.count === 0 && !emptyBadge.unread, 'another empty task clears count and unread state');
await draw();
check(bottom() && body().scrollTop > 100, 'historical single tall panel opens at bottom');
task.steps.push(panel('2'));
await draw();
check(bottom(), 'new tall panel follows to its bottom');
check([...body().children].map(el => el.dataset.panelId).join(',') === '1,2', 'oldest first');
task.steps[1].panel.blocks.push({ type: 'text', text: 'new content '.repeat(80) });
await draw();
check(bottom(), 'update_panel follows with unchanged panel count');
const img = document.createElement('img');
body().lastElementChild.append(img);
img.style.height = '250px';
img.dispatchEvent(new Event('load'));
await settle();
check(bottom(), 'late image layout follows to bottom');
body().scrollTop = 100;
body().dispatchEvent(new Event('scroll'));
task.steps[1].panel.blocks.push({ type: 'text', text: 'another update' });
await draw();
check(body().scrollTop === 100 && stage.unseen, 'reading earlier content stays put on update');
box.querySelector('.anomalous-ai-stage-new').click();
await settle();
check(bottom() && !stage.unseen && !box.querySelector('.anomalous-ai-stage-new'), 'see latest resumes bottom follow');
focusPanel(stage, '1');
await draw();
const deadline = performance.now() + 3000;
while (body().scrollTop >= 10 && performance.now() < deadline) await settle();
check(body().scrollTop < 10, 'timeline link goes to panel start');
const unchanged = body().firstElementChild;
await draw();
check(body().firstElementChild === unchanged, 'unchanged panels keep their DOM');
task = { id: 'hidden', steps: [panel('3')] };
box.style.display = 'none';
await draw();
check(!stage.placed, 'hidden stage waits for visible layout');
box.style.display = 'flex';
await draw();
check(bottom() && stage.placed, 'showing stage tab positions latest');
task.steps[0].panel.blocks.push({ type: 'input', id: 'draft', question: 'Draft' });
await draw();
const draft = box.querySelector('input');
draft.value = 'keep this draft';
const keptPanel = body().firstElementChild;
box.querySelector('[aria-label="关闭展台"]').click();
check(stage.hidden && !box.clientHeight, 'close button hides stage');
await draw();
check(stage.hidden, 'task refresh does not reopen stage');
focusPanel(stage, '3');
box.style.display = 'flex';
await draw();
check(!stage.hidden && body().firstElementChild === keptPanel && draft.value === 'keep this draft',
'timeline entry reopens without discarding drafts');
stage.hidden = true;
stageNews({ id: 'another-task', steps: [] }, stage);
check(!stage.hidden, 'another task resets closed state');
results.textContent += 'ALL PASSED';
} catch (error) { results.textContent += `FAIL ${error.stack}`; }
</script>
+51
View File
@@ -0,0 +1,51 @@
// Where the AI's stage blocks go (web/modules/ai_stage_layout.js): diagram layers and boxes, marks.
import assert from 'node:assert/strict';
import { arrowEnds, diagramBoxes, diagramLayout, markBox } from '../web/modules/ai_stage_layout.js';
const nodes = ['model', 'lora', 'prompt', 'sampler', 'save'].map(id => ({ id }));
const edges = [{ from: 'model', to: 'lora' }, { from: 'lora', to: 'sampler' }, { from: 'prompt', to: 'sampler' }, { from: 'sampler', to: 'save' }];
const { place, layers, slots } = diagramLayout(nodes, edges);
assert.deepEqual(Object.fromEntries([...place].map(([id, at]) => [id, at.layer])), { model: 0, lora: 1, prompt: 1, sampler: 2, save: 3 },
'a prompt that only feeds the sampler stands right before it');
assert.equal(layers, 4);
assert.equal(slots, 2);
assert.deepEqual(place.get('prompt'), { layer: 1, slot: 1 }, 'after the box fed from the left');
// The user's chain: model → LoRA → LoRA → sampler → save, the prompt into the sampler.
const chain = diagramLayout(['ckpt', 'lora1', 'lora2', 'pos', 'ks', 'save'].map(id => ({ id })), [
{ from: 'ckpt', to: 'lora1' }, { from: 'lora1', to: 'lora2' }, { from: 'lora2', to: 'ks' }, { from: 'pos', to: 'ks' }, { from: 'ks', to: 'save' }]);
assert.deepEqual(Object.fromEntries([...chain.place].map(([id, at]) => [id, at.layer])), { ckpt: 0, lora1: 1, lora2: 2, pos: 2, ks: 3, save: 4 });
// Within a layer, by where their feeders are: crossing arrows untangle.
const cross = diagramLayout(['a', 'b', 'x', 'y'].map(id => ({ id })), [{ from: 'a', to: 'y' }, { from: 'b', to: 'x' }]);
assert.equal(cross.place.get('y').slot, 0);
assert.equal(cross.place.get('x').slot, 1);
// A loop does not grow forever; a missing end is ignored.
const loop = diagramLayout([{ id: 'a' }, { id: 'b' }], [{ from: 'a', to: 'b' }, { from: 'b', to: 'a' }, { from: 'a', to: 'zz' }]);
assert.ok(loop.layers <= 2);
// Boxes: layers in columns to the right, a shorter layer centred.
const size = { w: 100, h: 40, gapX: 50, gapY: 10 };
const right = diagramBoxes(nodes, edges, 'right', size);
assert.deepEqual(right.boxes.get('model'), { x: 0, y: 25, w: 100, h: 40 }, 'a layer of one is centred');
assert.deepEqual(right.boxes.get('lora'), { x: 150, y: 0, w: 100, h: 40 });
assert.equal(right.width, 3 * 150 + 100);
assert.equal(right.height, 90);
const down = diagramBoxes(nodes, edges, 'down', size);
assert.equal(down.boxes.get('sampler').y, 2 * 60);
// Arrows meet the facing sides; backwards they go centre to centre.
assert.deepEqual(arrowEnds({ x: 0, y: 0, w: 100, h: 40 }, { x: 150, y: 0, w: 100, h: 40 }), { x1: 100, y1: 20, x2: 150, y2: 20 });
assert.deepEqual(arrowEnds({ x: 150, y: 0, w: 100, h: 40 }, { x: 0, y: 0, w: 100, h: 40 }), { x1: 200, y1: 20, x2: 50, y2: 20 });
assert.deepEqual(arrowEnds({ x: 0, y: 0, w: 100, h: 40 }, { x: 0, y: 60, w: 100, h: 40 }, 'down'), { x1: 50, y1: 40, x2: 50, y2: 60 });
// Marks: a box by its centre and size, a circle by its centre and radius (of the width).
assert.deepEqual(markBox({ x: 0.5, y: 0.5, w: 0.2, h: 0.1 }), { left: 40, top: 45, width: 20, height: 10, round: false });
const circle = markBox({ x: 0.6, y: 0.3, r: 0.05 });
assert.equal(circle.round, true);
assert.equal(circle.height, null);
assert.ok(Math.abs(circle.left - 55) < 1e-9 && Math.abs(circle.width - 10) < 1e-9 && Math.abs(circle.top - 30) < 1e-9);
assert.equal(markBox({ x: 0.01, y: 0.5, r: 0.1 }).left, 0, 'kept on the picture');
console.log('ai_stage_layout: ok');
+18
View File
@@ -0,0 +1,18 @@
import assert from 'node:assert/strict';
import { taskStatus } from '../web/modules/ai_task_status.js';
const task = { started: 10, updated: 99, steps: [] };
assert.equal(taskStatus(task, 100).state, 'untracked');
assert.equal(taskStatus(task, 100).age, 90, 'user activity does not pretend the AI made contact');
const run = { id: 'r', tool: 'run_workflow', state: 'running', at: 20, progress: { state: 'queued', ahead: 2 } };
task.steps.push(run);
assert.equal(taskStatus(task, 100).state, 'queued');
run.progress = { state: 'running', max: 20, value: 7 };
assert.equal(taskStatus(task, 100).seconds, 80);
run.state = 'done'; run.ms = 8000;
assert.equal(taskStatus(task, 100).state, 'done');
assert.equal(taskStatus(task, 100).seconds, 8);
task.steps.push({ tool: 'describe_canvas', state: 'running', at: 80 });
assert.equal(taskStatus(task, 100).state, 'tool', 'a later tool does not replay an old run as current');
task.steps.at(-1).state = 'done'; run.state = 'error';
assert.equal(taskStatus(task, 100).state, 'error');
console.log('ai_task_status: confirmed run state and untracked work passed');
+126
View File
@@ -0,0 +1,126 @@
// The AI workbench's rules (web/modules/ai_task_view.js): folding reads, words, counts, addresses.
import assert from 'node:assert/strict';
import {
activityIds, audioResult, chainTools, duration, flowStages, foldLooks, groupSteps, isVideo, panelGist, lookTools, newestPicture, outputCount, outputUrl, outputPicture,
pictureKey, picturePreview, uniquePictures, stepDetail, taskCounts, taskOutputs, taskResults, taskState, toolKey,
} from '../web/modules/ai_task_view.js';
const look = (tool, ok = true, state = 'done') => ({ kind: 'look', tool, ok, state, summary: tool });
const act = (tool, kind = 'canvas') => ({ kind, tool, ok: true, state: 'done', summary: `${tool}: K` });
// Reads that follow each other fold into one row; a running or failed read and what the user looks for stay rows.
const items = groupSteps([look('search_models'), look('describe_canvas'), act('report', 'note'), look('list_combos', false), look('search_images')]);
assert.deepEqual(items.map(item => item.type), ['looks', 'step', 'step', 'looks']);
assert.equal(items[0].steps.length, 2);
const inFlight = groupSteps([look('search_models'), look('describe_canvas', true, 'running')]);
assert.deepEqual(inFlight.map(item => item.type), ['looks', 'step'], 'an in-flight read stays visible until it finishes');
assert.deepEqual(groupSteps(undefined), []);
assert.deepEqual(lookTools([look('a'), look('b'), look('a')]), [{ tool: 'a', count: 2 }, { tool: 'b', count: 1 }]);
// The AI's work in a row (reads, canvas changes, decisions) folds into a chain of three or more.
const work = [look('find_nodes'), act('add_node'), act('add_node'), act('connect_nodes'), act('log_decision', 'decision'), act('run_workflow', 'run'), look('run_status')];
const chained = groupSteps(work);
assert.deepEqual(chained.map(item => item.type), ['chain', 'step', 'looks']);
assert.equal(chained[0].steps.length, 5);
assert.deepEqual(chainTools(chained[0].steps), [{ tool: 'look', kind: 'look', count: 1 }, { tool: 'add_node', kind: 'canvas', count: 2 },
{ tool: 'connect_nodes', kind: 'canvas', count: 1 }, { tool: 'log_decision', kind: 'decision', count: 1 }]);
assert.deepEqual(foldLooks(chained[0].steps).map(item => item.type), ['looks', 'step', 'step', 'step', 'step'], 'a chain opens to its steps, reads still folded');
assert.deepEqual(groupSteps([look('a'), act('set_prompt'), look('b')]).map(item => item.type), ['looks', 'step', 'looks'], 'one change among reads stays in sight');
assert.deepEqual(groupSteps([act('set_prompt'), act('set_values')]).map(item => item.type), ['step', 'step'], 'two steps stay as they are');
assert.deepEqual(groupSteps([look('a'), look('b'), look('c')]).map(item => item.type), ['looks'], 'only reads stay a row of reads');
const broken = groupSteps([act('add_node'), act('add_node'), { ...act('connect_nodes'), ok: false }, act('add_node'), act('add_node'), act('add_node')]);
assert.deepEqual(broken.map(item => item.type), ['step', 'step', 'step', 'chain'], 'a failed step stays in sight and splits the work');
assert.deepEqual(groupSteps([act('add_node'), act('add_node'), { ...act('set_values'), state: 'asking' }]).map(item => item.type), ['step', 'step', 'step'],
'a step waiting at a gate is not folded away');
// Words.
assert.equal(toolKey('place_combo'), 'aiTool_place_combo');
assert.equal(toolKey('x<y>'), 'aiTool_xy');
assert.equal(stepDetail(act('place_combo')), 'K');
assert.equal(stepDetail({ tool: 'describe_canvas', summary: 'describe_canvas' }), '');
assert.equal(stepDetail({ tool: 'report', summary: 'something else' }), 'something else');
// A task's state and counts.
assert.equal(taskState({ live: true, errors: 1 }), 'live');
assert.equal(taskState({ live: true, asks: 1 }), 'asks', 'a question waiting for the user comes first');
assert.equal(taskState({ waiting: true }), 'waiting', 'a task the user started, before an AI takes it');
assert.deepEqual(taskCounts({ asks: 2 }), [{ key: 'aiCountAsks', values: { count: 2 } }]);
assert.equal(taskState({ errors: 2 }), 'error');
assert.equal(taskState({}), 'idle', 'silence without an explicit end is not completion');
assert.equal(taskState({ ended: 100 }), 'done');
assert.deepEqual(taskCounts({ kinds: { canvas: 2, look: 5, speech: 1 }, images: 3, errors: 1 }), [
{ key: 'aiCountCanvas', values: { count: 2 } }, { key: 'aiCountImages', values: { count: 3 } },
{ key: 'aiCountSpeech', values: { count: 1 } }, { key: 'aiCountErrors', values: { count: 1 } },
]);
assert.deepEqual(taskCounts({}), []);
// What a task made, and where it is served.
const made = taskResults({ steps: [
{ results: [{ type: 'image', filename: 'a.png', subfolder: 'x' }, { type: 'canvas', activity: 'e1' }] },
{ results: [{ type: 'model', model_id: 'm' }, { type: 'weird' }] },
] });
assert.equal(made.image.length, 1);
assert.equal(made.canvas[0].activity, 'e1');
assert.equal(made.model.length, 1);
assert.equal(outputUrl({ filename: 'a b.png', subfolder: 'x/y' }), '/view?filename=a+b.png&subfolder=x%2Fy&type=output');
assert.deepEqual(audioResult({ file: 'audio/Aris/a.flac' }), { filename: 'a.flac', subfolder: 'audio/Aris', folder: 'output' });
assert.deepEqual(audioResult({ file: 'a.flac' }), { filename: 'a.flac', subfolder: '', folder: 'output' });
assert.equal(isVideo({ filename: 'clip.MP4' }), true);
assert.equal(isVideo({ filename: 'a.png' }), false);
// Output-only reuse: never compare an input/temp file with a same-named output by accident.
assert.equal(outputPicture({ filename: 'a.png', folder: 'temp' }), null);
assert.equal(outputPicture({ filename: 'a.mp4' }), null);
assert.equal(outputPicture({ filename: 'a.flac' }), null);
assert.deepEqual(outputPicture({ filename: 'a.png', subfolder: 'x' }), { type: 'output', filename: 'a.png', subfolder: 'x' });
const repeated = uniquePictures([{ filename: 'a.png' }, { filename: 'a.png', folder: 'output' }, { filename: 'a.png', subfolder: 'b' }]);
assert.equal(repeated.length, 2);
assert.notEqual(pictureKey(repeated[0]), pictureKey(repeated[1]));
assert.match(picturePreview({ filename: 'a b.png', subfolder: 'x/y' }), /^\/anomalous\/output_thumbnail\?filename=a\+b.png&subfolder=x%2Fy$/);
assert.equal(picturePreview({ filename: 'a.mp4' }), outputUrl({ filename: 'a.mp4' }));
// How long a step took.
assert.equal(duration(400), null);
assert.deepEqual(duration(48_200), { key: 'aiSeconds', values: { count: 48 } });
assert.deepEqual(duration(150_000), { key: 'aiMinutes', values: { count: 3 } });
// The flow: stages in the order they came, each with its last step's state.
const staged = (id, stage, extra = {}) => ({ id, stage, ok: true, state: 'done', results: [], ...extra });
const flow = flowStages({ steps: [
staged('a', 'model'), staged('b', 'prompt'), staged('c', 'model', { state: 'running' }),
{ id: 'd', kind: 'note', ok: true, results: [{ type: 'image', filename: 'x.png' }] }, { id: 'e', kind: 'user', ok: true },
staged('f', 'run', { state: 'asking' }), staged('g', 'canvas', { ok: false, state: 'stopped' }),
] });
assert.deepEqual(flow.map(item => [item.stage, item.state, item.first, item.count]), [
['model', 'running', 'a', 2], ['prompt', 'done', 'b', 1], ['picture', 'done', 'd', 1], ['run', 'asking', 'f', 1], ['canvas', 'error', 'g', 1],
]);
assert.deepEqual(flowStages(null), []);
// The activity entries a task shows: the AI's canvas changes and the user's own.
assert.deepEqual(activityIds({ steps: [{ results: [{ type: 'canvas', activity: 'a1' }, { type: 'model' }] }, { tool: 'user_activity', activity: 'u1', results: [] }] }), ['a1', 'u1']);
console.log('ai_task_view: ok');
// What a panel holds, for its card in the timeline.
assert.deepEqual(panelGist({ blocks: [{ type: 'heading' }, { type: 'slider' }, { type: 'images', items: [{}, {}] }] }),
{ icon: 'image', parts: [{ key: 'aiGistCompare', values: {} }, { key: 'aiGistPictures', values: { count: 2 } }] });
assert.equal(panelGist({ blocks: [{ type: 'choices', options: [{}, {}] }, { type: 'images', items: [{}] }] }).icon, 'question', 'a question comes first');
assert.deepEqual(panelGist({ blocks: [{ type: 'run_map' }] }), { icon: 'nodes', parts: [{ key: 'aiGistMap', values: {} }] });
assert.deepEqual(panelGist({ blocks: [{ type: 'text' }] }), { icon: 'panel', parts: [] }, 'words only: no line under the title');
console.log('ai_task_view: panel gist ok');
// Results: each thing once, the steps that brought pictures newest first, the newest picture still there to keep from.
const outputs = taskOutputs({ steps: [
{ id: 'r1', results: [{ type: 'image', filename: 'a.png' }, { type: 'image', filename: 'b.png' }, { type: 'canvas', activity: 'e1' }] },
{ id: 'c1', results: [{ type: 'combo', name: 'K' }, { type: 'model', model_id: 'm1' }, { type: 'canvas', activity: 'e2' }] },
{ id: 'r2', results: [{ type: 'image', filename: 'c.png', gone: true }, { type: 'image', filename: 'a.png' }, { type: 'image', filename: 'v.mp4' }] },
{ id: 'n1', results: [{ type: 'combo', name: 'K' }, { type: 'model', model_id: 'm1' }, { type: 'download', file: 'hand.safetensors', size: 9 },
{ type: 'audio', file: 'audio/A/1.flac' }, { type: 'scan' }] },
] });
assert.deepEqual(outputs.runs.map(run => [run.step.id, run.pictures.map(image => image.filename)]), [['r2', ['c.png', 'v.mp4']], ['r1', ['a.png', 'b.png']]]);
assert.deepEqual([outputs.combos.length, outputs.models.length, outputs.downloads.map(item => item.file), outputs.audio.length, outputs.canvas], [1, 1, ['hand.safetensors'], 1, 2]);
assert.equal(outputCount(outputs), 4 + 1 + 1 + 1 + 1);
assert.deepEqual(newestPicture(outputs), { type: 'output', filename: 'a.png', subfolder: '' }, 'past a deleted picture and a video');
assert.equal(newestPicture(taskOutputs({ steps: [] })), null);
assert.equal(outputCount(taskOutputs(null)), 0);
console.log('ai_task_view: results ok');
+27
View File
@@ -0,0 +1,27 @@
// node tests/ai_text_format.mjs — the little layout of the AI's notes: blocks and inline pieces, never markup.
import assert from 'node:assert/strict';
import { inline, parseNote } from '../web/modules/ai_text_format.js';
assert.deepEqual(inline('删掉 **多出来的手**,负向词加 `extra arms`。'), [
{ text: '删掉 ' }, { text: '多出来的手', bold: true }, { text: ',负向词加 ' }, { text: 'extra arms', code: true }, { text: '。' },
]);
assert.deepEqual(inline('<b>不是网页</b>'), [{ text: '<b>不是网页</b>' }], 'tags stay text');
const blocks = parseNote([
'### 发现的问题',
'1. **多长了一只手**:腰部多出一截手腕',
'2. 元素串色',
'',
'怎么修:',
'第二行接着同一段',
'- 方案 1:局部重绘',
'* 方案 2:换 Seed',
'---',
'结论',
].join('\n'));
assert.deepEqual(blocks.map(block => [block.type, block.lines.length]), [
['heading', 1], ['numbers', 2], ['paragraph', 2], ['bullets', 2], ['rule', 0], ['paragraph', 1],
]);
assert.deepEqual(blocks[1].lines[0][0], { text: '多长了一只手', bold: true });
assert.deepEqual(parseNote(''), []);
console.log('ai_text_format: ok');
+60
View File
@@ -0,0 +1,60 @@
// node tests/ai_toolbox_data.mjs — the materials picker's data: node types from the page's own definitions, the catalogue, search, keeping.
import assert from 'node:assert/strict';
import { catalogItems, keptCatalog, loadCatalog, nodeItems, nodeSources, packLabel, searchItems } from '../web/modules/ai_toolbox_data.js';
assert.equal(packLabel('comfyui_controlnet_aux'), 'controlnet aux');
assert.equal(packLabel('ComfyUI-Impact-Pack'), 'Impact Pack');
assert.equal(packLabel('rgthree-comfy'), 'rgthree');
assert.equal(packLabel('Comfyroll_CustomNodes'), 'Comfyroll CustomNodes');
assert.equal(packLabel('ComfyUI'), 'ComfyUI');
const def = (module, extra = {}) => ({ nodeData: { python_module: module, ...extra } });
const registered = {
KSampler: def('nodes', {
display_name: 'KSampler', category: 'sampling',
input: { required: { model: ['MODEL'], seed: ['INT', { default: 0 }], sampler_name: [['euler', 'dpmpp_2m']], positive: ['CONDITIONING'] },
optional: { note: ['STRING', { forceInput: true }] } },
output: ['LATENT'], output_name: ['LATENT'],
}),
FaceDetailer: def('custom_nodes.ComfyUI-Impact-Pack', { display_name: 'FaceDetailer', nodeSource: { displayText: 'Impact Pack' }, input: {}, output: ['IMAGE'] }),
'Context (rgthree)': def('custom_nodes.rgthree-comfy', { display_name: 'Context (rgthree)', input: {}, output: [] }),
KlingTextToVideoNode: def('comfy_api_nodes.nodes_kling', { display_name: 'Kling', api_node: true, input: {}, output: ['VIDEO'] }),
'Fast Groups Muter (rgthree)': { title: 'Fast Groups Muter (rgthree)' },
Reroute: { title: 'Reroute' },
};
const nodes = nodeItems(registered);
const by = Object.fromEntries(nodes.map(item => [item.id, item]));
assert.deepEqual(by.KSampler.inputs.map(input => input.name), ['model', 'positive', 'note'], 'sockets, a forced input among them');
assert.deepEqual(by.KSampler.settings.map(setting => [setting.name, setting.type, setting.choices]), [['seed', 'INT', 0], ['sampler_name', 'choice', 2]]);
assert.deepEqual(by.KSampler.outputs, [{ slot: 0, name: 'LATENT', type: 'LATENT' }]);
assert.deepEqual([by.FaceDetailer.source.kind, by.FaceDetailer.source.label], ['pack', 'Impact Pack']);
assert.equal(by['Context (rgthree)'].source.label, 'rgthree');
assert.ok(by.KlingTextToVideoNode.paid);
assert.deepEqual([by['Fast Groups Muter (rgthree)'].source.key, by['Fast Groups Muter (rgthree)'].page], ['rgthree-comfy', true], 'a page-only helper goes with its pack');
assert.equal(by.Reroute.source.kind, 'builtin');
assert.deepEqual(nodeSources(nodes).map(source => [source.key, source.count]),
[['builtin', 2], ['ComfyUI-Impact-Pack', 1], ['rgthree-comfy', 2], ['api', 1]], 'own first, packs by name, paid last');
const items = catalogItems({
combos: [{ name: '赛博少女', model: 'a.safetensors', loras: ['neon.safetensors'], prompt: '1girl, neon city' }],
workflows: [{ name: 'SDXL 高清修复', tags: ['SDXL'], prompt: 'castle' }],
prompts: [{ id: 'combo:赛博少女', name: '赛博少女', source: 'combo', positive: '1girl, neon city', negative: '', tags: [] }],
models: [{ id: 'loras:0:neon.safetensors', name: 'Neon', file: 'neon.safetensors', type: 'loras', base: 'SDXL', words: ['neonstyle'] }],
});
assert.deepEqual(items.map(item => [item.kind, item.id]), [['combos', '赛博少女'], ['workflows', 'SDXL 高清修复'], ['prompts', 'combo:赛博少女'], ['models', 'loras:0:neon.safetensors']]);
assert.deepEqual(searchItems(items, 'neon').map(item => item.kind), ['models', 'combos', 'prompts'], 'a title that starts with the words first');
assert.deepEqual(searchItems(items, 'sdxl castle').map(item => item.id), ['SDXL 高清修复']);
assert.deepEqual(searchItems(nodes, 'impact').map(item => item.id), ['FaceDetailer'], 'a pack name finds its nodes');
assert.equal(searchItems(items, '').length, items.length);
// Kept a minute; one request however many ask at once; a failure says why and keeps nothing.
let calls = 0;
const fetcher = async () => { calls++; return { ok: true, json: async () => ({ combos: [{ name: 'x', loras: [] }] }) }; };
const [a, b] = await Promise.all([loadCatalog({ fetcher, fresh: true }), loadCatalog({ fetcher })]);
assert.equal(calls, 1);
assert.equal(a, b);
await loadCatalog({ fetcher });
assert.equal(calls, 1, 'kept');
assert.equal(keptCatalog()[0].id, 'x');
await assert.rejects(loadCatalog({ fresh: true, fetcher: async () => ({ ok: false, status: 502, json: async () => ({ error: 'no' }) }) }), /no/);
console.log('ai_toolbox_data: ok');
+78
View File
@@ -0,0 +1,78 @@
// node tests/canvas_build.mjs — the pure part of building a workflow on the canvas.
import assert from 'node:assert/strict';
import { apiLinks, checkedValue, freeSpot, layered, placedBlock, typesFit, valuePlan } from '../web/modules/canvas_build.js';
assert.ok(typesFit('MODEL', 'MODEL'));
assert.ok(typesFit('model', 'MODEL'), 'case does not matter');
assert.ok(typesFit('*', 'LATENT') && typesFit('IMAGE', '*'));
assert.ok(typesFit('INT', 'INT,FLOAT'), 'one of a list');
assert.ok(!typesFit('CLIP', 'MODEL'));
const sampler = {
id: 3, type: 'KSampler',
widgets: [
{ name: 'steps', value: 20, options: { min: 1, max: 10000 } },
{ name: 'sampler_name', value: 'euler', options: { values: ['euler', 'dpmpp_2m', 'dpmpp_2m_sde'] } },
{ name: 'denoise', value: 1, options: { min: 0, max: 1 } },
{ name: 'run', type: 'button', value: null },
],
};
const ckpt = { name: 'ckpt_name', value: 'a.safetensors', options: { values: () => ['SDXL\\b.safetensors', 'a.safetensors'] } };
const note = { name: 'text', value: '' };
const flag = { name: 'tiled', value: false };
assert.equal(checkedValue(3, sampler.widgets[0], 30), 30);
assert.equal(checkedValue(3, sampler.widgets[0], '30'), 30, 'a number in text');
assert.throws(() => checkedValue(3, sampler.widgets[0], 0), /between 1 and 10000/);
assert.throws(() => checkedValue(3, sampler.widgets[0], 'many'), /takes a number/);
assert.equal(checkedValue(4, ckpt, 'sdxl/B.safetensors'), 'SDXL\\b.safetensors', 'a choice by its path, either slash, any case');
assert.throws(() => checkedValue(3, sampler.widgets[1], 'dpmpp_2m_karras'), /close: dpmpp_2m$/);
assert.throws(() => checkedValue(3, sampler.widgets[1], 'ddim'), /choices: euler \| dpmpp_2m/);
const picture = { name: 'image', value: '', options: { values: [] } };
assert.equal(checkedValue(7, picture, 'ComfyUI_00001_.png [output]'), 'ComfyUI_00001_.png [output]', 'an output picture, with an empty input folder');
assert.throws(() => checkedValue(7, picture, 'ComfyUI_00001_.png'), /no choices at all/);
assert.equal(checkedValue(5, note, 'hello'), 'hello');
assert.throws(() => checkedValue(5, note, 3), /takes text/);
assert.throws(() => checkedValue(6, flag, 'yes'), /true or false/);
const plan = valuePlan(sampler, { steps: 28, sampler_name: 'dpmpp_2m' });
assert.deepEqual(plan.map(([widget, value]) => [widget.name, value]), [['steps', 28], ['sampler_name', 'dpmpp_2m']]);
assert.throws(() => valuePlan(sampler, { cfg_scale: 7 }), /no setting cfg_scale; it has: steps, sampler_name, denoise\./);
assert.deepEqual(valuePlan(sampler, undefined), []);
// A new node goes right of everything, below what it would cover.
const box = (x, y, w = 300, h = 200) => ({ pos: [x, y], size: [w, h] });
const fresh = box(0, 0, 300, 120);
assert.deepEqual(freeSpot(fresh, []), [100, 100]);
assert.deepEqual(freeSpot(fresh, [box(100, 100), box(500, 50)]), [880, 50]);
const near = box(100, 100);
const under = freeSpot(fresh, [near, box(480, 100)], near);
assert.equal(under[0], 480);
assert.ok(under[1] >= 100 + 200 + 30, 'moved below the node it would cover');
// A workflow in API form: its links, and columns by distance from the sources.
const workflow = {
1: { class_type: 'CheckpointLoaderSimple', inputs: { ckpt_name: 'a.safetensors' } },
2: { class_type: 'CLIPTextEncode', inputs: { text: 'girl', clip: ['1', 1] } },
3: { class_type: 'KSampler', inputs: { model: ['1', 0], positive: ['2', 0], steps: 20 } },
4: { class_type: 'EmptyLatentImage', inputs: { width: 1024 } },
5: { class_type: 'Note', inputs: { text: ['9', 0] } },
};
const links = apiLinks(workflow);
assert.deepEqual(links, [
{ from: '1', slot: 1, to: '2', input: 'clip' },
{ from: '1', slot: 0, to: '3', input: 'model' },
{ from: '2', slot: 0, to: '3', input: 'positive' },
], 'a link to a node not in the workflow is not a link');
const sized = Object.fromEntries(Object.keys(workflow).map(key => [key, box(0, 0, 300, 100)]));
const spots = layered(sized, links);
assert.equal(spots[1][0], spots[4][0], 'sources share the first column');
assert.ok(spots[2][0] > spots[1][0] && spots[3][0] > spots[2][0], 'each step right of what feeds it');
assert.notEqual(spots[1][1], spots[4][1], 'one column, one above the other');
assert.deepEqual(Object.keys(layered(sized, [{ from: '1', to: '2' }, { from: '2', to: '1' }])).sort(), ['1', '2', '3', '4', '5'], 'a loop still lays out');
// A group added to a canvas keeps its columns and moves as one block right of the node it is near, clear of others.
const block = placedBlock(sized, links, [near, box(480, 100)], near);
assert.equal(block[1][0], 480, 'the first column starts where a single node would');
assert.equal(block[2][0] - block[1][0], spots[2][0] - spots[1][0], 'columns kept');
assert.ok(block[1][1] >= 100 + 200 + 30, 'the whole block below what it would cover');
console.log('canvas_build: ok');
+210
View File
@@ -0,0 +1,210 @@
import assert from 'node:assert/strict';
import { addWires, applyWires, groupWires, pairWires, reachesOutput, validateWires } from '../web/modules/canvas_wiring.js';
class Graph {
constructor() { this._nodes = []; this.links = {}; this.serial = 0; this.nodeSerial = 0; }
getNodeById(id) { return this._nodes.find(n => String(n.id) === String(id)); }
getLink(id) { return this.links[id]; }
add(n) { n.id = ++this.nodeSerial; n.graph = this; this._nodes.push(n); }
remove(n) {
for (const link of Object.values(this.links)) {
if (link.origin_id === n.id || link.target_id === n.id) this.getNodeById(link.target_id).disconnectInput(link.target_slot);
}
this._nodes = this._nodes.filter(item => item !== n);
}
}
function node(g, inputs = [], outputs = [], terminal = false) {
const n = {
inputs: inputs.map(([name, type]) => ({ name, type, link: null })),
outputs: outputs.map(([name, type]) => ({ name, type, links: [] })),
constructor: { nodeData: { output_node: terminal } },
disconnectInput(slot) {
const id = this.inputs[slot].link, link = this.graph.links[id];
if (link) {
const out = this.graph.getNodeById(link.origin_id).outputs[link.origin_slot];
out.links = out.links.filter(i => i !== id);
delete this.graph.links[id];
}
this.inputs[slot].link = null;
},
connect(slot, to, target) {
if (this.fail) { this.fail = false; return null; }
to.disconnectInput(target);
const id = ++this.graph.serial;
const link = { id, origin_id: this.id, origin_slot: slot, target_id: to.id, target_slot: target };
this.graph.links[id] = link;
this.outputs[slot].links.push(id); to.inputs[target].link = id;
return link;
},
};
if (g) g.add(n);
return n;
}
const snapshot = g => Object.values(g.links).map(l => [l.origin_id, l.origin_slot, l.target_id, l.target_slot].join(':')).sort();
const wire = (g, plan, added = null, replace = false) => applyWires(g, validateWires(g, plan, replace), added);
// Insertion keeps the entire selected branch connected and reaches the real output.
{
const g = new Graph(), decode = node(g, [], [['IMAGE','IMAGE']]);
const save = node(g, [['images','IMAGE']], [], true);
decode.connect(0, save, 0);
const scale = node(null, [['image','IMAGE']], [['IMAGE','IMAGE']]);
const result = wire(g, addWires(g, scale, { insert_after: decode.id }), scale);
assert.equal(result.length, 2);
assert.equal(result[1].replaced_from, decode.id);
assert.equal(g.getLink(save.inputs[0].link).origin_id, scale.id);
assert.equal(reachesOutput(g, scale), true);
save.mode = 2;
assert.equal(reachesOutput(g, scale), false, 'muted outputs do not count');
}
// Fan-out requires an explicit branch; the other preview is not rewired.
{
const g = new Graph(), decode = node(g, [], [['IMAGE','IMAGE']]);
const save = node(g, [['image','IMAGE']], [], true), preview = node(g, [['image','IMAGE']], [], true);
decode.connect(0, save, 0); decode.connect(0, preview, 0);
const scale = node(null, [['image','IMAGE']], [['IMAGE','IMAGE']]);
const before = snapshot(g);
assert.throws(() => addWires(g, scale, { insert_after: decode.id }), /branches/);
assert.deepEqual(snapshot(g), before);
wire(g, addWires(g, scale, { insert_after: decode.id, wire_to: [save.id] }), scale);
assert.equal(g.getLink(preview.inputs[0].link).origin_id, decode.id);
}
// Four inpainting wires in one transaction; replacing an occupied latent needs opt-in.
{
const g = new Graph(), load = node(g, [], [['IMAGE','IMAGE'], ['MASK','MASK']]);
const ckpt = node(g, [], [['MODEL','MODEL'], ['VAE','VAE']]);
const empty = node(g, [], [['LATENT','LATENT']]), sampler = node(g, [['latent_image','LATENT']], [['LATENT','LATENT']]);
empty.connect(0, sampler, 0);
const encode = node(null, [['pixels','IMAGE'], ['mask','MASK'], ['vae','VAE']], [['LATENT','LATENT']]);
const plan = addWires(g, encode, { wire_from: [load.id, ckpt.id], wire_to: [sampler.id] });
assert.equal(plan.length, 4);
assert.throws(() => validateWires(g, plan), /replace:true/);
assert.equal(wire(g, plan, encode, true).length, 4);
assert.equal(reachesOutput(g, encode), false);
}
// Ambiguity, wildcard and conflicting sources fail before mutation.
{
const g = new Graph(), text = node(g, [], [['CONDITIONING','CONDITIONING']]);
const sampler = node(g, [['positive','CONDITIONING'], ['negative','CONDITIONING']]);
assert.throws(() => pairWires(text, sampler), /positive.*negative/);
assert.equal(pairWires(text, sampler, { output: 0, input: 'negative' }).length, 1);
const any = node(g, [], [['any','*']]);
assert.throws(() => pairWires(any, sampler), /wildcards/);
assert.equal(pairWires(any, sampler, { output: 0, input: 'negative' }).length, 1);
const other = node(g, [], [['CONDITIONING','CONDITIONING']]);
assert.throws(() => validateWires(g, [...pairWires(text, sampler, { output: 0, input: 'positive' }),
...pairWires(other, sampler, { output: 0, input: 'positive' })]), /Several sources/);
assert.equal(Object.keys(g.links).length, 0);
}
// Cycles in the final planned topology are refused, including a detached new node.
{
const g = new Graph(), a = node(g, [['in','IMAGE']], [['out','IMAGE']]), b = node(g, [['in','IMAGE']], [['out','IMAGE']]);
a.connect(0,b,0);
const added = node(null, [['in','IMAGE']], [['out','IMAGE']]);
assert.throws(() => validateWires(g, addWires(g, added, { wire_from:[b.id], wire_to:[a.id] })), /loop/);
}
// Failure on the second link removes the newly added node and restores the original branch.
{
const g = new Graph(), decode = node(g, [], [['IMAGE','IMAGE']]), save = node(g, [['images','IMAGE']], [], true);
decode.connect(0,save,0);
const before = snapshot(g), added = node(null, [['image','IMAGE']], [['IMAGE','IMAGE']]);
added.fail = true;
assert.throws(() => wire(g, addWires(g, added, { insert_after: decode.id }), added), /Original connections restored/);
assert.deepEqual(snapshot(g), before);
assert.equal(g._nodes.length, 2);
}
// An existing-node batch must restore BOTH replaced inputs if the second connection fails.
{
const g = new Graph(), old = node(g, [], [['IMAGE','IMAGE'], ['MASK','MASK']]);
const next = node(g, [], [['IMAGE','IMAGE'], ['MASK','MASK']]);
const target = node(g, [['pixels','IMAGE'], ['mask','MASK']]);
old.connect(0, target, 0); old.connect(1, target, 1);
const before = snapshot(g), connect = next.connect;
next.connect = function(slot, ...args) { return slot === 1 ? null : connect.call(this, slot, ...args); };
assert.throws(() => wire(g, pairWires(next, target), null, true), /Original connections restored/);
assert.deepEqual(snapshot(g), before);
}
// Widget callbacks that fail before wiring must leave no new node and preserve the old path.
{
const g = new Graph(), a = node(g, [], [['IMAGE','IMAGE']]), b = node(g, [['images','IMAGE']]);
a.connect(0,b,0);
const added = node(null, [['image','IMAGE']], [['IMAGE','IMAGE']]), before = snapshot(g);
const plan = validateWires(g, addWires(g, added, { insert_after: a.id }));
assert.throws(() => applyWires(g, plan, added, () => { throw new Error('widget failed'); }), /Original connections restored/);
assert.equal(g._nodes.length, 2);
assert.deepEqual(snapshot(g), before);
}
// A text-to-image graph for the splices below: checkpoint, two prompts, sampler.
function textToImage() {
const g = new Graph(), ckpt = node(g, [], [['MODEL','MODEL'], ['CLIP','CLIP'], ['VAE','VAE']]);
const pos = node(g, [['clip','CLIP']], [['CONDITIONING','CONDITIONING']]), neg = node(g, [['clip','CLIP']], [['CONDITIONING','CONDITIONING']]);
const sampler = node(g, [['model','MODEL'], ['positive','CONDITIONING'], ['negative','CONDITIONING']], [['LATENT','LATENT']]);
ckpt.connect(1, pos, 0); ckpt.connect(1, neg, 0);
ckpt.connect(0, sampler, 0); pos.connect(0, sampler, 1); neg.connect(0, sampler, 2);
return { g, ckpt, pos, neg, sampler };
}
const controlNet = () => node(null, [['positive','CONDITIONING'], ['negative','CONDITIONING'], ['control_net','CONTROL_NET'], ['image','IMAGE']],
[['positive','CONDITIONING'], ['negative','CONDITIONING']]);
const source = (g, n, slot) => g.getLink(n.inputs[slot].link).origin_id;
// Same-name sockets pair up; a ControlNet in front of the sampler takes each prompt at its own input.
{
const { g, pos, neg, sampler } = textToImage();
const loader = node(g, [], [['CONTROL_NET','CONTROL_NET']]), picture = node(g, [], [['IMAGE','IMAGE'], ['MASK','MASK']]);
const cn = controlNet();
assert.equal(pairWires(cn, sampler).length, 2);
const result = wire(g, addWires(g, cn, { wire_from: [loader.id, picture.id], wire_to: [sampler.id] }), cn);
assert.equal(result.length, 6);
assert.deepEqual([source(g, cn, 0), source(g, cn, 1), source(g, cn, 2), source(g, cn, 3)], [pos.id, neg.id, loader.id, picture.id]);
assert.deepEqual([g.getLink(sampler.inputs[1].link).origin_slot, g.getLink(sampler.inputs[2].link).origin_slot], [0, 1]);
assert.equal(source(g, sampler, 1), cn.id);
assert.throws(() => addWires(g, controlNet(), { insert_after: pos.id }), /#new\.positive.*connect_nodes/);
}
// A LoRA behind a checkpoint splices MODEL and CLIP; the two prompts are two CLIP branches to choose.
{
const { g, ckpt, pos, neg, sampler } = textToImage();
const lora = () => node(null, [['model','MODEL'], ['clip','CLIP']], [['MODEL','MODEL'], ['CLIP','CLIP']]);
assert.throws(() => addWires(g, lora(), { insert_after: ckpt.id }), /choose wire_to.*: 4, 2, 3/);
const first = lora();
assert.equal(wire(g, addWires(g, first, { insert_after: ckpt.id, wire_to: [sampler.id, pos.id, neg.id] }), first).length, 5);
assert.deepEqual([source(g, sampler, 0), source(g, pos, 0), source(g, neg, 0)], [first.id, first.id, first.id]);
const second = lora();
assert.equal(wire(g, addWires(g, second, { wire_to: [sampler.id, pos.id, neg.id] }), second).length, 5, 'the same splice in front');
assert.deepEqual([source(g, second, 0), source(g, second, 1), source(g, pos, 0)], [first.id, first.id, second.id]);
}
// A splice with two inputs of that type and neither named alike is left to the AI; a plain replacement still asks.
{
const g = new Graph(), decode = node(g, [], [['IMAGE','IMAGE']]), save = node(g, [['images','IMAGE']], [], true);
decode.connect(0, save, 0);
assert.throws(() => addWires(g, node(null, [['image1','IMAGE'], ['image2','IMAGE']], [['IMAGE','IMAGE']]), { wire_to: [save.id] }), /image1 or image2/);
const scale = node(null, [['image','IMAGE']], [['IMAGE','IMAGE']]);
wire(g, addWires(g, scale, { wire_to: [save.id] }), scale);
assert.deepEqual([source(g, scale, 0), source(g, save, 0)], [decode.id, scale.id]);
const loader = node(null, [], [['IMAGE','IMAGE']]);
assert.throws(() => validateWires(g, addWires(g, loader, { wire_to: [save.id] })), /replace:true/);
}
// A group in API form: links inside it, to canvas nodes as "#id" and outputs by name, then wire_to spliced; one rollback for all.
{
const { g, ckpt, sampler } = textToImage();
const picture = node(g, [], [['IMAGE','IMAGE'], ['MASK','MASK']]);
const made = { a: node(null, [['pixels','IMAGE'], ['vae','VAE']], [['LATENT','LATENT']]), b: node(null, [['samples','LATENT'], ['mask','MASK']], [['LATENT','LATENT']]) };
const empty = node(g, [], [['LATENT','LATENT']]);
sampler.inputs.push({ name: 'latent_image', type: 'LATENT', link: null });
empty.connect(0, sampler, 3);
const nodes = { a: { inputs: { pixels: [`#${picture.id}`, 0], vae: [`#${ckpt.id}`, 'VAE'] } }, b: { inputs: { samples: ['a', 'LATENT'], mask: [`#${picture.id}`, 'MASK'] }, wire_to: sampler.id } };
assert.throws(() => validateWires(g, groupWires(g, made, nodes)), /latent_image is already wired.*replace:true/,
'the samples of b come from a, so it replaces the empty latent instead of splicing it');
const plan = validateWires(g, groupWires(g, made, nodes), true);
assert.equal(plan.length, 5, 'four links in, one into the sampler');
const before = snapshot(g);
made.b.fail = true;
assert.throws(() => applyWires(g, plan, Object.values(made)), /Original connections restored/);
assert.deepEqual(snapshot(g), before);
assert.equal(g._nodes.includes(made.a), false, 'every node of the group is taken out again');
made.b.fail = false;
assert.equal(applyWires(g, plan, Object.values(made)).length, 5);
assert.equal(source(g, sampler, 3), made.b.id);
assert.throws(() => groupWires(g, { c: node(null, [['image','IMAGE']], []) }, { c: { inputs: { image: ['#999', 0] } } }), /No node #999/);
assert.throws(() => groupWires(g, { c: node(null, [['image','IMAGE']], []) }, { c: { inputs: { image: ['zz', 0] } } }), /neither a node of this group/);
assert.throws(() => groupWires(g, { c: node(null, [['image','IMAGE']], []) }, { c: { inputs: { pixels: [`#${picture.id}`, 0] } } }), /Node c .* has no input pixels; it has: image/);
}
console.log('Typed wiring: insertion, splices, same-name pairs, groups, fan-out, inpainting, ambiguity, cycles and rollback passed.');
+6 -1
View File
@@ -1,6 +1,6 @@
// Where a LoRA goes and how it is wired (web/modules/graph_splice.js), on a small stand-in graph.
import assert from 'node:assert/strict';
import { chainEnd, loraInsertionCapabilities, mainModelLoaders, planLoraInsertion, spliceLora } from '../web/modules/graph_splice.js';
import { chainEnd, loraChain, loraInsertionCapabilities, mainModelLoaders, modelLineStart, planLoraInsertion, spliceLora } from '../web/modules/graph_splice.js';
const SLOTS = {
CheckpointLoaderSimple: [[], ['MODEL', 'CLIP', 'VAE']],
@@ -85,6 +85,11 @@ function checkpointWorkflow() {
const lora2 = spliceLora({ graph: g, plan: plan2, anchorNode: lora, insertedNode: node(null, 'LoraLoaderModelOnly') });
assert.equal(source(g, sampler, 0), `LoraLoaderModelOnly#${lora2.id}`);
assert.equal(source(g, pos, 0), `LoraLoader#${lora.id}`);
// The model line, for the server's base-model check: the loader it starts at and its LoRAs in order.
assert.deepEqual(loraChain(g, ckpt, 'MODEL'), [lora, lora2]);
assert.equal(modelLineStart(g, lora2), ckpt);
assert.equal(modelLineStart(g, sampler), ckpt);
assert.equal(modelLineStart(g, ckpt), ckpt, 'a loader starts its own line');
}
{
+79
View File
@@ -0,0 +1,79 @@
// node tests/mcp_canvas_brief.mjs — the canvas in brief for an AI: a line per node, what a run still
// needs wired and what on the canvas fits it, ComfyUI's errors.
import assert from 'node:assert/strict';
import { canvasBrief, flowChains, unwired } from '../web/modules/mcp_canvas_brief.js';
/** A node as LiteGraph keeps it, with ComfyUI's data on which inputs a run needs. */
function node(id, type, inputs = [], outputs = [], { required = inputs.map(input => input.name), mode = 0, title } = {}) {
const made = { id, type, title: title ?? type, mode, inputs, outputs };
Object.defineProperty(made, 'constructor', { value: { nodeData: { input: { required: Object.fromEntries(required.map(name => [name, ['X']])) } } } });
return made;
}
const socket = (name, type, link = null, more = {}) => ({ name, type, link, ...more });
const out = (name, type = name) => ({ name, type });
function graph(nodes, links) {
return { _nodes: nodes, links, getNodeById: id => nodes.find(item => item.id === id) };
}
const nodes = [
node(1, 'CheckpointLoaderSimple', [], [out('MODEL'), out('CLIP'), out('VAE')]),
node(2, 'CLIPTextEncode', [socket('clip', 'CLIP', 1), socket('text', 'STRING', null, { widget: { name: 'text' } })], [out('CONDITIONING')], { title: '正面' }),
node(3, 'KSampler', [socket('model', 'MODEL', 2), socket('positive', 'CONDITIONING', 3), socket('negative', 'CONDITIONING'), socket('latent_image', 'LATENT')], [out('LATENT')]),
node(4, 'EmptyLatentImage', [], [out('LATENT')]),
node(5, 'LatentUpscale', [socket('samples', 'LATENT', 4)], [out('LATENT')]), // fed by #3: wiring it back would loop
node(6, 'KSampler', [socket('model', 'MODEL')], [out('LATENT')], { mode: 4 }), // bypassed: not run as it is
node(7, 'ImageScale', [socket('image', 'IMAGE', 5), socket('mask', 'MASK')], [out('IMAGE')], { required: ['image'] }), // mask is optional
];
const links = { 1: { origin_id: 1, origin_slot: 1 }, 2: { origin_id: 1, origin_slot: 0 }, 3: { origin_id: 2, origin_slot: 0 }, 4: { origin_id: 3, origin_slot: 0 }, 5: { origin_id: 99, origin_slot: 0 } };
const g = graph(nodes, links);
const brief = canvasBrief(g, { 3: { errors: [{ message: 'Required input is missing', details: 'negative' }] } });
assert.equal(brief.node_count, 7);
assert.equal(brief.lines[1], '#2 CLIPTextEncode "正面" | clip←#1 | gives CONDITIONING', 'a widget input is no socket');
assert.equal(brief.lines[2], '#3 KSampler | model←#1 positive←#2 negative←?CONDITIONING latent_image←?LATENT | gives LATENT');
assert.equal(brief.lines[5], '#6 KSampler [bypassed] | model←?MODEL | gives LATENT');
assert.equal(brief.lines[6], '#7 ImageScale | image←#99 | gives IMAGE', 'an optional input left free is no gap');
assert.deepEqual(brief.unwired, [
{ node: 3, input: 'negative', type: 'CONDITIONING', fits: ['#2 CONDITIONING'] },
{ node: 3, input: 'latent_image', type: 'LATENT', fits: ['#4 LATENT'] }, // not #5 (it is fed by #3), not #6 (bypassed)
], 'a bypassed node asks for nothing');
assert.deepEqual(brief.errors, [{ node: 3, error: 'Required input is missing: negative' }]);
assert.equal(canvasBrief(g).errors, undefined);
const many = graph(Array.from({ length: 35 }, (_, at) => node(at + 1, 'SaveImage', [socket('images', 'IMAGE')], [])), {});
const long = canvasBrief(many);
assert.equal(long.lines.length, 8);
assert.equal(long.more_nodes, 27);
assert.equal(long.unwired.length, 8);
assert.equal(long.more_unwired, 27);
assert.equal(long.unwired[0].fits, undefined, 'nothing on the canvas gives an IMAGE');
assert.deepEqual(unwired(graph([], {})), { unwired: [] });
assert(brief.chains.some(chain => chain.includes('CONDITIONING:positive')));
assert(!brief.chains.some(chain => chain.includes('#6 ')), 'bypassed nodes are not active paths');
const serial = graph([
node(1, 'EmptyLatentImage'), node(2, 'KSampler', [socket('latent_image', 'LATENT', 1)]),
node(3, 'VAEDecode', [socket('samples', 'LATENT', 2)]), node(4, 'SaveImage', [socket('images', 'IMAGE', 3)]),
], { 1: { origin_id: 1 }, 2: { origin_id: 2 }, 3: { origin_id: 3 } });
assert.deepEqual(flowChains(serial).chains, [
'#1 EmptyLatentImage --LATENT:latent_image--> #2 KSampler --LATENT:samples--> #3 VAEDecode --IMAGE:images--> #4 SaveImage',
]);
serial._nodes.push(node(5, 'VAELoader'));
serial._nodes[2].inputs.push(socket('vae', 'VAE', 5));
serial.links[5] = { origin_id: 5 };
assert(flowChains(serial).chains[0].includes('#2 KSampler --LATENT:samples--> #3 VAEDecode --IMAGE:images--> #4 SaveImage'),
'auxiliary VAE input must not split the picture spine');
serial._nodes[2].inputs.pop();
delete serial.links[5];
serial._nodes[0].inputs = [socket('loop', 'IMAGE', 4)];
serial.links[4] = { origin_id: 4 };
assert.equal(flowChains(serial).chains.length, 1, 'a malformed cyclic graph still terminates');
assert(flowChains(serial).chains[0].length <= 240);
const branches = graph([node(1, 'Source'), ...Array.from({ length: 20 }, (_, i) =>
node(i + 2, 'Target', [socket(i % 2 ? 'positive' : 'negative', 'CONDITIONING', i + 1)]))],
Object.fromEntries(Array.from({ length: 20 }, (_, i) => [i + 1, { origin_id: 1 }])));
assert.equal(flowChains(branches).chains.length, 6);
assert.equal(flowChains(branches).more_chains, 14);
console.log('mcp_canvas_brief: ok');
+71
View File
@@ -0,0 +1,71 @@
// node tests/mcp_fill.mjs — run_workflow's blanks planned on the canvas: prompts, seeds, size, the
// picture a LoadImage reads; nothing planned when a blank has nowhere to go.
import assert from 'node:assert/strict';
import { fillPlan, valuesPlan } from '../web/modules/mcp_fill.js';
const text = (name, value = '') => ({ name, value });
const number = (name, value, min = 0, max = 0xffffffffffffffff) => ({ name, value, options: { min, max } });
function node(id, type, widgets, { outputs = [], inputs = [], mode = 0, title, required = {} } = {}) {
const made = { id, type, title: title ?? type, mode, widgets, outputs, inputs };
Object.defineProperty(made, 'constructor', { value: { nodeData: { input: { required } } } });
return made;
}
const multiline = ['STRING', { multiline: true }];
const prompts = node(1, 'PromptPair', [text('positive', 'old'), text('negative', 'bad')], { required: { positive: multiline, negative: multiline } });
const sampler = node(2, 'KSampler', [number('seed', 1), number('steps', 20, 1, 10000)]);
const custom = node(3, 'SamplerCustom', [number('noise_seed', 5)]);
const latent = node(4, 'EmptyLatentImage', [number('width', 512, 16, 16384), number('height', 512, 16, 16384), number('batch_size', 1, 1, 64)], { outputs: [{ name: 'LATENT', type: 'LATENT' }] });
const scale = node(5, 'ImageScale', [number('width', 512, 16, 16384), number('height', 512, 16, 16384)], { outputs: [{ name: 'IMAGE', type: 'IMAGE' }] });
const base = node(6, 'LoadImage', [text('image', 'a.png')], { title: '原图' });
const marks = node(7, 'LoadImage', [text('image', 'b.png')], { title: '标注' });
const resting = node(8, 'KSampler', [number('seed', 9)], { mode: 4 }); // bypassed: left alone
const upscale = node(9, 'LatentUpscale', [number('width', 1152, 16, 16384), number('height', 1728, 16, 16384)],
{ outputs: [{ name: 'LATENT', type: 'LATENT' }], inputs: [{ name: 'samples', type: 'LATENT' }] }); // a hires fix keeps its size
const g = { _nodes: [prompts, sampler, custom, latent, scale, base, marks, resting, upscale] };
const { plans, set } = fillPlan(g, { prompt: '海边少女', negative: 'blurry', seed: 42, width: 1024, height: 1536, image: 'x.png [output]', image_node: 7 });
assert.deepEqual(set, { prompt: [1], negative: [1], seed: [2, 3], size: [4], image: [7] }, 'not the IMAGE scaler, the latent upscale or the bypassed sampler');
const values = plans.flatMap(({ node: item, plan }) => plan.map(([widget, value]) => [item.id, widget.name, value]));
assert.deepEqual(values, [[1, 'positive', '海边少女'], [1, 'negative', 'blurry'], [2, 'seed', 42], [3, 'noise_seed', 42],
[4, 'width', 1024], [4, 'height', 1536], [7, 'image', 'x.png [output]']]);
assert.equal(prompts.widgets[0].value, 'old', 'only planned: the page applies it');
assert.deepEqual(fillPlan(g, { width: 768 }).set, { size: [4] }, 'one side alone');
assert.deepEqual(fillPlan(g, {}).plans, []);
assert.throws(() => fillPlan(g, { image: 'x.png' }), /Several LoadImage nodes \(#6 "原图", #7 "标注"\): give image_node/);
assert.throws(() => fillPlan(g, { image: 'x.png', image_node: 2 }), /Node 2 is no LoadImage/);
assert.throws(() => fillPlan(g, { width: 8 }), /width of node 4 is between 16 and 16384/);
assert.throws(() => fillPlan({ _nodes: [sampler] }, { prompt: 'a' }), /No positive prompt box/);
assert.throws(() => fillPlan({ _nodes: [prompts] }, { seed: 1 }), /No sampler with a seed/);
assert.throws(() => fillPlan({ _nodes: [scale] }, { height: 768 }), /No empty latent/);
// Any other setting by name, a third-party node's too; a name several nodes have asks for the node.
const sharpen = node(10, 'ImageSharpen_ThirdParty', [number('sharpen_radius', 1, 1, 31)]);
const second = node(11, 'KSampler', [number('seed', 3), number('steps', 20, 1, 10000)]);
assert.deepEqual(fillPlan({ _nodes: [sampler, sharpen] }, { values: { steps: 30, sharpen_radius: 3 } }).set, { values: [2, 10] });
assert.throws(() => fillPlan({ _nodes: [sampler, second] }, { values: { steps: 30 } }), /steps is a setting of #2, #11: give the one to change as \{"<node id>"/);
const both = valuesPlan({ _nodes: [sampler, second] }, { 11: { steps: 8 } });
assert.deepEqual(both.map(({ node: item, plan }) => [item.id, plan.map(([widget, value]) => [widget.name, value])]), [[11, [['steps', 8]]]]);
assert.throws(() => valuesPlan({ _nodes: [sampler] }, { cfg: 5 }), /No node on the canvas has a setting cfg/);
assert.throws(() => valuesPlan({ _nodes: [sampler] }, { 99: { steps: 5 } }), /No node 99/);
assert.throws(() => fillPlan({ _nodes: [sampler] }, { seed: 1, values: { 2: { seed: 2 } } }), /seed is set by a blank and by values/);
// Two prompts combined into one sampler (regional prompts) are not given one text; a refiner's own prompt is.
{
const links = { 1: { target_id: 33, target_slot: 0 }, 2: { target_id: 33, target_slot: 1 }, 3: { target_id: 34, target_slot: 0 }, 4: { target_id: 35, target_slot: 0 } };
const g2 = { links, getNodeById(id) { return this._nodes.find(item => item.id === id); } };
const encoder = (id, link) => node(id, 'CLIPTextEncode', [text('text', '')], { outputs: [{ name: 'CONDITIONING', type: 'CONDITIONING', links: [link] }], required: { text: multiline } });
const combine = node(33, 'ConditioningCombine', [], { outputs: [{ name: 'CONDITIONING', type: 'CONDITIONING', links: [3] }],
inputs: [{ name: 'conditioning_1', type: 'CONDITIONING' }, { name: 'conditioning_2', type: 'CONDITIONING' }] });
const into = id => node(id, 'KSampler', [number('seed', 1)], { inputs: [{ name: 'positive', type: 'CONDITIONING' }] });
g2._nodes = [encoder(31, 1), encoder(32, 2), combine, into(34)];
g2._nodes.forEach(item => { item.graph = g2; });
assert.throws(() => fillPlan(g2, { prompt: 'a castle' }), /Several positive prompt boxes feed one sampler \(#31, #32\)/);
g2._nodes = [encoder(31, 1), encoder(36, 4), combine, into(34), into(35)];
links[1] = { target_id: 34, target_slot: 0 };
g2._nodes.forEach(item => { item.graph = g2; });
assert.deepEqual(fillPlan(g2, { prompt: 'a castle' }).set, { prompt: [31, 36] }, 'base and refiner prompts each feed their own sampler');
}
console.log('mcp_fill: ok');
+103
View File
@@ -0,0 +1,103 @@
import assert from 'node:assert/strict';
import { queueTracked } from '../web/modules/mcp_run.js';
// Only the host is substituted: its queue drains batches, emits request IDs, and calls widgets around submissions.
function host({ failAt = 0, manual = false, refuse = null } = {}) {
const api = new EventTarget();
const events = (name, detail) => api.dispatchEvent(Object.assign(new Event(name), { detail }));
let count = 0;
const hooks = [];
api.queuePrompt = async () => {
const n = ++count;
await Promise.resolve();
if (n === failAt) throw Object.assign(new Error('invalid graph'), refuse ? { response: refuse, toString: () => 'Prompt outputs failed validation' } : {});
return { prompt_id: `p${n}` };
};
const app = {
nextQueueRequestId: 0, processingQueue: false, queueItems: [],
async queuePrompt(number, batch) {
const requestId = this.nextQueueRequestId++;
this.queueItems.push({ number, batch, requestId });
events('promptQueueing', { requestId });
if (this.processingQueue) return false;
this.processingQueue = true;
try {
while (this.queueItems.length) {
const item = this.queueItems.pop();
let accepted = 0;
for (let i = 0; i < item.batch; i++) {
hooks.push('before');
try {
const result = await api.queuePrompt(item.number, {});
assert.ok(result.prompt_id);
if (manual && count === 1) void this.queuePrompt(0, 1);
} catch { break; }
accepted++;
hooks.push('after');
}
if (accepted) events('promptQueued', { requestId: item.requestId });
}
} finally { this.processingQueue = false; }
return true;
},
};
return { app, api, hooks };
}
for (const manual of [false, true]) {
const { app, api, hooks } = host({ manual });
const original = api.queuePrompt;
const result = await queueTracked(app, api, 2);
assert.deepEqual(result.prompt_ids, ['p1', 'p2']);
assert.equal(result.queued, 2);
assert.equal(api.queuePrompt, original);
await new Promise(setImmediate); // a later manual batch may still be draining
assert.deepEqual(hooks, Array(manual ? 3 : 2).fill(['before', 'after']).flat());
}
{
const { app, api } = host({ manual: true });
const submit = api.queuePrompt;
let release;
const held = new Promise(resolve => { release = resolve; });
let count = 0;
api.queuePrompt = async function (...args) {
if (++count === 3) await held;
return submit.apply(this, args);
};
const result = await queueTracked(app, api, 2);
assert.deepEqual(result.prompt_ids, ['p1', 'p2']);
assert.equal(app.processingQueue, true, 'AI receipt does not wait for the manual batch behind it');
release();
await new Promise(setImmediate);
}
{
const { app, api } = host({ failAt: 2, manual: true });
const result = await queueTracked(app, api, 3);
assert.deepEqual(result.prompt_ids, ['p1'], 'manual submission after a partial failure is not captured');
assert.match(result.submission_error, /invalid graph/);
}
{
const { app, api } = host({ failAt: 1 });
assert.equal((await queueTracked(app, api)).queued, 0);
const result = await queueTracked(app, api);
assert.deepEqual(result.prompt_ids, ['p2'], 'failed capture leaves no listeners or wrapper behind');
}
{
const { app, api } = host();
const first = queueTracked(app, api, 2);
await assert.rejects(queueTracked(app, api), /another batch/);
assert.deepEqual((await first).prompt_ids, ['p1', 'p2']);
delete app.nextQueueRequestId;
await assert.rejects(queueTracked(app, api), /Update the frontend/);
}
{
// ComfyUI's refusal goes on as data too, for the server to say what to do next.
const refuse = { error: { type: 'prompt_outputs_failed_validation', message: 'Prompt outputs failed validation', details: '' },
node_errors: { 3: { class_type: 'KSampler', errors: [{ type: 'required_input_missing', message: 'Required input is missing', details: 'model', extra_info: { input_name: 'model' } }] } } };
const result = await queueTracked(...Object.values(host({ failAt: 1, refuse })).slice(0, 2));
assert.equal(result.queued, 0);
assert.equal(result.refusal.type, 'prompt_outputs_failed_validation');
assert.deepEqual(result.refusal.nodes, [{ node: '3', class_type: 'KSampler', type: 'required_input_missing', message: 'Required input is missing',
details: 'model', input: 'model', value: undefined }]);
}
console.log('mcp_run: exact receipts, manual interleaving, partial failure, busy guard and cleanup passed');
+68
View File
@@ -0,0 +1,68 @@
import assert from 'node:assert/strict';
import fs from 'node:fs/promises';
// Load the actual router, substituting only host/UI imports. No real graph, queue or downloads.
let source = await fs.readFile(new URL('../web/modules/mcp_task_tab.js', import.meta.url), 'utf8');
source = source.replace(/^import .*;\r?\n/gm, '');
source = 'const { app, t, showWorkbenchToast, tabName } = globalThis.taskTabHost;\n' + source;
const workflow = { activeWorkflow: null, openWorkflows: [] };
let copies = 0;
let switchesWork = true;
const app = {
extensionManager: { workflow, command: { execute: async () => {} } },
graph: { serialize: () => ({ nodes: [] }) },
async loadGraphData(_data, _clean, _restore, name) {
copies++;
workflow.activeWorkflow = { path: name, filename: name };
workflow.openWorkflows.push(workflow.activeWorkflow);
},
};
globalThis.taskTabHost = { app, t: value => value, tabName: value => value, showWorkbenchToast() {} };
globalThis.document = { querySelectorAll: () => workflow.openWorkflows.map(item => ({
click() { if (switchesWork) workflow.activeWorkflow = item; },
})) };
globalThis.requestAnimationFrame = callback => callback();
// Exercise timeout/fallback branches without waiting several real seconds.
globalThis.setTimeout = callback => callback();
const { onTaskTab } = await import('data:text/javascript;base64,' + Buffer.from(source).toString('base64'));
const spec = { mode: 'new', tab: { path: 'task-A' }, title: 'Task A' };
let calls = 0;
const run = () => { calls++; return workflow.activeWorkflow.path; };
const opened = [];
const remember = value => opened.push(value);
function reset(hasTask = true) {
calls = copies = 0;
opened.length = 0;
switchesWork = true;
workflow.activeWorkflow = { path: 'user-B' };
workflow.openWorkflows = [...(hasTask ? [{ path: 'task-A' }] : []), workflow.activeWorkflow];
}
for (const action of ['run_workflow', 'describe_canvas', 'check_models', 'download_missing', 'set_values']) {
reset();
assert.equal(await onTaskTab(action, spec, run, remember), 'task-A', `${action} follows its task`);
assert.equal(calls, 1);
for (const closed of [true, false]) {
reset(!closed);
switchesWork = false;
await assert.rejects(onTaskTab(action, spec, run, remember), /closed or could not be activated/);
assert.equal(calls, 0, `${action} must not fall through on a missing/stuck tab`);
assert.equal(copies, 0, 'an unrelated workflow must not be cloned as a replacement');
assert.equal(workflow.activeWorkflow.path, 'user-B');
}
}
reset();
assert.equal(await onTaskTab('run_workflow', { mode: 'current', tab: spec.tab }, run, remember), 'user-B');
reset(false);
assert.equal(await onTaskTab('describe_canvas', { mode: 'new' }, run, remember), 'user-B');
assert.equal(copies, 0);
await onTaskTab('set_values', { mode: 'new', title: 'new-task' }, run, remember);
assert.equal(copies, 1);
assert.equal(opened[0].path, 'new-task');
assert.equal(opened[0].made, true);
reset(false);
await onTaskTab('new_workflow', spec, async () => {
await app.loadGraphData({}, true, true, 'replacement');
return 'created';
}, remember);
assert.equal(opened[0].path, 'replacement', 'explicit replacement remains available after closing a tab');
console.log('Task canvas routing: switch, closed/loading failure, download, current mode and explicit replacement passed.');
+69
View File
@@ -0,0 +1,69 @@
// ComfyUI imports every .js under web/ on its own, all at once, so any module of an import cycle can happen
// to run first. A module that uses, as it loads, a `const` of a module in its own cycle then fails now and
// then, and AMB with it: no button, no window (docs/architecture/lessons.md). Such a value goes in a module
// outside the cycle.
import assert from 'node:assert/strict';
import { readFileSync, readdirSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
const folder = join(dirname(fileURLToPath(import.meta.url)), '..', 'web', 'modules');
const IMPORT = /^import\s+\{([^}]*)\}\s+from\s+'\.\/([\w.]+\.js)'/gm;
const modules = new Map(readdirSync(folder).filter(name => name.endsWith('.js')).map((name) => {
const text = readFileSync(join(folder, name), 'utf8').replace(/\r\n/g, '\n');
const imports = [...text.matchAll(IMPORT)].map(([, names, from]) => ({ from, names: names.split(',').map(item => item.trim().split(/\s+as\s+/)).filter(([first]) => first) }));
return [name, { text, imports, to: new Set(imports.map(item => item.from)) }];
}));
/** The modules in cycles, each with the set of those in the same cycle (Tarjan, on the imports). */
function cycles() {
const index = new Map();
const low = new Map();
const stack = [];
const groups = new Map();
const visit = (name) => {
index.set(name, index.size);
low.set(name, index.get(name));
stack.push(name);
for (const next of modules.get(name)?.to || []) {
if (!modules.has(next)) continue;
if (!index.has(next)) {
visit(next);
low.set(name, Math.min(low.get(name), low.get(next)));
} else if (stack.includes(next)) low.set(name, Math.min(low.get(name), index.get(next)));
}
if (low.get(name) !== index.get(name)) return;
const group = new Set();
let member;
do {
member = stack.pop();
group.add(member);
} while (member !== name);
for (const item of group) groups.set(item, group);
};
for (const name of modules.keys()) if (!index.has(name)) visit(name);
return groups;
}
/** Is `local` used by a line at the top of `text`, that is one run as the module loads? */
function usedAsItLoads(text, local) {
return text.split('\n').some(line => /^[\w(]/.test(line)
&& !/^(import|function|async function|export (async )?function|export default function|class)\b/.test(line)
&& new RegExp(`\\b${local}\\b`).test(line.split('=>')[0]));
}
const groups = cycles();
const found = [];
for (const [name, { text, imports }] of modules) {
for (const { from, names } of imports) {
if (!groups.get(name)?.has(from) || from === name) continue;
const source = modules.get(from).text;
for (const [exported, local = exported] of names) {
const isValue = new RegExp(`^export (const|let) ${exported}\\b`, 'm').test(source);
if (isValue && usedAsItLoads(text, local)) found.push(`${name} uses ${exported} of ${from} as it loads`);
}
}
}
assert.deepEqual(found, []);
console.log('Module cycles: nothing reads a value of its own cycle as it loads.');
+22
View File
@@ -0,0 +1,22 @@
// The rail's labels and the page titles name their screen words by key (labelKey, TITLE_KEYS),
// so a search for t('…') does not find them: each must still be in both languages.
// (ce35145 removed 'models' and 'gallery' as unused; the rail then showed the raw keys.)
import assert from 'node:assert/strict';
import fs from 'node:fs';
import { i18n } from '../web/modules/locales.js';
const read = name => fs.readFileSync(new URL(`../web/modules/${name}`, import.meta.url), 'utf8');
const railKeys = [...read('ui_shell_rail.js').matchAll(/labelKey: '([A-Za-z]+)'/g)].map(match => match[1]);
const titles = read('ui_shell_nav.js').match(/const TITLE_KEYS = \{([\s\S]*?)\};/)[1];
const titleKeys = [...titles.matchAll(/: '([A-Za-z]+)'/g)].map(match => match[1]);
assert.ok(railKeys.includes('models') && railKeys.includes('gallery'));
assert.ok(titleKeys.includes('gallery'));
for (const key of new Set([...railKeys, ...titleKeys])) {
for (const lang of ['zh', 'en']) assert.ok(i18n[lang][key], `${lang} has no '${key}'`);
}
// Every rail page has a number key (1-9, ui_shell_nav.js): the AI workbench pushed the audio
// gallery off the keys until it became a view of Voices.
const railPages = read('ui_shell_rail.js').match(/export const RAIL_PAGES = Object\.freeze\(\[([\s\S]*?)\]\);/)[1];
assert.ok([...railPages.matchAll(/\{ page: '/g)].length <= 9, 'more rail pages than number keys');
+4 -4
View File
@@ -16,14 +16,14 @@ assert.equal(pageForNode({ model: true }, kinds), 'assistant'); // a loader
assert.equal(pageForNode({ prompt: true, model: true }, kinds), 'assistant'); // a loader with prompt boxes: its models first
// Number keys: 1-9 from the top of the rail.
const pages = ['home', 'activity', 'models', 'gallery', 'recipes', 'combos', 'prompts', 'voices', 'audio-gallery'].map(page => ({ page }));
const pages = ['home', 'activity', 'ai', 'models', 'gallery', 'recipes', 'combos', 'prompts', 'voices'].map(page => ({ page }));
const canvas = { closest: () => null };
const field = { closest: selector => (selector.includes('textarea') ? {} : null) };
const key = (code, extra = {}) => ({ code, key: code.slice(-1), target: canvas, ...extra });
assert.equal(railPageForKey(key('Digit1'), pages), 'home');
assert.equal(railPageForKey(key('Digit7'), pages), 'prompts');
assert.equal(railPageForKey(key('Numpad9'), pages), 'audio-gallery');
assert.equal(railPageForKey(key('Digit8'), pages), 'prompts');
assert.equal(railPageForKey(key('Numpad9'), pages), 'voices');
assert.equal(railPageForKey(key('Digit0'), pages), null);
assert.equal(railPageForKey(key('KeyW'), pages), null); // W stays ComfyUI's (the Workflows sidebar)
assert.equal(railPageForKey(key('Digit3', { target: field }), pages), null); // typing a number into a box
@@ -31,7 +31,7 @@ assert.equal(railPageForKey(key('Digit3', { ctrlKey: true }), pages), null);
assert.equal(railPageForKey(key('Digit3', { shiftKey: true }), pages), null);
assert.equal(railPageForKey(key('Digit3', { isComposing: true }), pages), null); // a Chinese input method choosing a word
assert.equal(railPageForKey(key('Digit3', { repeat: true }), pages), null);
assert.equal(railPageForKey({ code: '', key: '4', target: canvas }, pages), 'gallery'); // no code: by the key
assert.equal(railPageForKey({ code: '', key: '5', target: canvas }, pages), 'gallery'); // no code: by the key
assert.equal(railPageForKey(key('Digit9'), pages.slice(0, 3)), null); // fewer pages than keys
console.log('shell_open_rules: ok');
+5 -3
View File
@@ -50,14 +50,16 @@ class ActivityLogTests(unittest.TestCase):
self.assertLessEqual(len(entry['target']), activity_log.MAX_VALUE_CHARS + 1)
def test_a_change_late_in_a_long_text_still_shows(self):
before = ', '.join(f'tag{i}' for i in range(200)) # far longer than MAX_VALUE_CHARS
after = before.replace('tag180', 'tag180, extra')
before = ', '.join(f'tag{i}' for i in range(600)) # far longer than MAX_CHANGE_CHARS
after = before.replace('tag580', 'tag580, extra')
item = activity_log._clean_changes([{'kind': 'changed', 'node': 'n', 'widget': 'text',
'before': before, 'after': after}])[0]
self.assertNotEqual(item['before'], item['after'])
self.assertIn('extra', item['after'])
self.assertTrue(item['before'].startswith('…'))
self.assertLessEqual(len(item['after']), activity_log.MAX_VALUE_CHARS + 2)
self.assertLessEqual(len(item['after']), activity_log.MAX_CHANGE_CHARS + 2)
short = activity_log._clean_changes([{'kind': 'changed', 'node': 'n', 'widget': 'text', 'before': before[:1500], 'after': 'x'}])[0]
self.assertEqual(short['before'], before[:1500], 'a whole prompt is kept for its receipt')
def test_failure_responses_are_not_recorded(self):
ok = web.json_response({'success': True})
+115
View File
@@ -0,0 +1,115 @@
"""Completion follow-ups and error recovery use real task records in temporary storage."""
import unittest
from test_ai_tasks import isolate, ok, failed
from api import ai_panels, ai_tasks
from api.ai_task_handoff import AFTER_DONE
class CompletionEdges(unittest.TestCase):
def setUp(self):
isolate()
def done(self, session="one"):
task = ai_tasks.begin_task("A", "Finished", now=100, session=session)
ai_tasks.report("A", "Done", done=True, now=110, session=session)
return task
def test_explicit_read_after_done_stays_in_finished_task(self):
task = self.done()
ref = ai_tasks.begin_step("A", "get_situation", {}, 111, True, session="one", task_id=task["id"])
ai_tasks.finish_step("A", "get_situation", {}, ok({}), 111, True, step_ref=ref)
self.assertEqual(ref[0], task["id"])
self.assertEqual(task["ended"], 110)
self.assertEqual(len(ai_tasks.list_tasks()), 1)
def test_explicit_panel_uses_named_finished_task_not_another_active_task(self):
finished = self.done()
active = ai_tasks.begin_task("A", "Other", now=112, session="one", new_task=True)
panel = ai_panels.show("A", "Result", [{"type": "text", "text": "Before / after"}],
now=113, session="one", task_id=finished["id"])
self.assertEqual(ai_panels._find(panel["id"])[0]["id"], finished["id"])
self.assertIsNone(finished["ended"])
self.assertFalse(active["steps"])
def test_read_without_step_reference_keeps_explicit_finished_task(self):
task = self.done()
ai_tasks.record("A", "describe_canvas", {}, ok({}), 111, True, session="one", task_id=task["id"])
self.assertEqual(len(ai_tasks.list_tasks()), 1)
self.assertEqual(task["steps"][-1]["tool"], "describe_canvas")
self.assertEqual(task["ended"], 110)
def test_running_retry_does_not_hide_previous_failure(self):
ai_tasks.record("A", "set_model", {"node_id": 1, "model_id": "missing"}, failed("Missing model"), 100)
ref = ai_tasks.begin_step("A", "set_model", {"node_id": 1, "model_id": "found"}, 101)
task = ai_tasks.get_task(ref[0])
self.assertEqual(ai_tasks.task_summary(task)["errors"], 1)
ai_tasks.finish_step("A", "set_model", {"node_id": 1, "model_id": "found"}, ok({}), 101, step_ref=ref)
self.assertEqual(ai_tasks.task_summary(ai_tasks.get_task(ref[0]))["errors"], 0)
def test_waiting_for_approval_is_not_a_successful_retry(self):
ai_tasks.record("A", "set_prompt", {"prompt": "x"}, failed("Failed"), 100)
ai_tasks.record("A", "set_prompt", {"prompt": "x"}, ok({"waiting_for_user": True}), 101)
self.assertEqual(ai_tasks.list_tasks()[0]["errors"], 1)
def test_a_retry_with_the_arguments_put_right_resolves_a_failure_that_named_none(self):
ai_tasks.record("A", "set_model", {"node": 1, "model": "x"}, failed("Unknown arguments"), 100)
ai_tasks.record("A", "get_model", {}, failed("model_id is required"), 101)
ai_tasks.record("A", "set_model", {"node_id": 1, "model_id": "x"}, ok({}), 102)
ai_tasks.record("A", "get_model", {"model_id": "loras:0:a.safetensors"}, ok({}), 103)
self.assertEqual(ai_tasks.list_tasks()[0]["errors"], 0)
def test_success_on_a_different_node_does_not_resolve_a_failure(self):
ai_tasks.record("A", "set_model", {"node_id": 1, "model_id": "missing"}, failed("Missing model"), 100)
ai_tasks.record("A", "set_model", {"node_id": 2, "model_id": "found"}, ok({}), 101)
self.assertEqual(ai_tasks.list_tasks()[0]["errors"], 1)
def test_cancelled_run_is_not_a_successful_retry(self):
task = {"id": "cancelled", "steps": [
{"tool": "run_workflow", "kind": "run", "ok": False, "state": "error", "results": []},
{"tool": "run_workflow", "kind": "run", "ok": True, "state": "cancelled", "results": []},
]}
self.assertEqual(ai_tasks.task_summary(task)["errors"], 1)
def test_automatic_followup_yields_to_a_waiting_task(self):
finished = self.done()
waiting = ai_tasks.start_task("Next", now=111)
ref = ai_tasks.begin_step("A", "get_situation", {}, 112, True, session="one")
self.assertEqual(ref[0], waiting["id"])
self.assertEqual(finished["ended"], 110)
def test_explicit_followup_does_not_consume_an_unrelated_waiting_task(self):
finished = self.done()
waiting = ai_tasks.start_task("Next", now=111)
ref = ai_tasks.begin_step("A", "get_situation", {}, 112, True, session="one", task_id=finished["id"])
self.assertEqual(ref[0], finished["id"])
self.assertTrue(waiting["waiting"])
self.assertEqual(ai_tasks._waiting[0], waiting["id"])
def test_automatic_followup_does_not_cross_sessions_or_completion_window(self):
for session, when in (("two", 111), ("one", 110 + AFTER_DONE)):
with self.subTest(session=session, when=when):
isolate()
finished = self.done()
ref = ai_tasks.begin_step("A", "get_situation", {}, when, True, session=session)
self.assertNotEqual(ref[0], finished["id"])
def test_a_different_setting_does_not_resolve_the_failed_setting(self):
ai_tasks.record("A", "set_values", {"node_id": 1, "values": {"steps": 0}}, failed("Invalid steps"), 100)
ai_tasks.record("A", "set_values", {"node_id": 1, "values": {"cfg": 5}}, ok({}), 101)
self.assertEqual(ai_tasks.list_tasks()[0]["errors"], 1)
ai_tasks.record("A", "set_values", {"node_id": "1", "values": {"steps": 20}}, ok({}), 102)
self.assertEqual(ai_tasks.list_tasks()[0]["errors"], 0)
def test_recovery_target_survives_reloading_task_records(self):
ai_tasks.record("A", "set_model", {"node_id": 1, "model_id": "missing"}, failed("Missing"), 100)
ai_tasks.record("A", "set_model", {"node_id": 2, "model_id": "present"}, ok({}), 101)
ai_tasks._tasks = None
self.assertEqual(ai_tasks.list_tasks()[0]["errors"], 1)
def test_batch_settings_keep_node_and_setting_identity(self):
ai_tasks.record("A", "set_values", {"values": {"1": {"steps": 0}}}, failed("Invalid steps"), 100)
ai_tasks.record("A", "set_values", {"values": {"1": {"cfg": 5}}}, ok({}), 101)
self.assertEqual(ai_tasks.list_tasks()[0]["errors"], 1)
ai_tasks.record("A", "set_values", {"values": {"1": {"steps": 20}}}, ok({}), 102)
self.assertEqual(ai_tasks.list_tasks()[0]["errors"], 0)
+68
View File
@@ -0,0 +1,68 @@
"""A task's key decisions (ai_decisions.py): the AI records them with log_decision, the user's own
choices in Anomalous are read from the task's steps, and an AI picking the task up hears them."""
import time
import unittest
from test_ai_tasks import isolate, ok
from api import ai_decisions, ai_gates, ai_panels, ai_task_canvas, ai_tasks, ai_toolbox, mcp_task_tools
from api.mcp_tools import ToolError
class DecisionTests(unittest.TestCase):
def setUp(self):
isolate()
self.task = ai_tasks.begin_task("A", "水彩少女")["id"]
self.ctx = {"client": "A", "task": self.task}
def test_the_ai_records_what_was_settled_and_what_it_is_about(self):
run = ai_tasks.record("A", "run_workflow", {}, ok({}), time.time())
answer = mcp_task_tools.log_decision(self.ctx, "背景", "水晶湖畔,光线偏冷", reason="和发色更搭",
rejected=["晨光森林"], by="together", related=[run["id"], "nope"])
self.assertEqual(answer["related"], [run["id"]], "only this task's steps")
step = ai_tasks.get_task(self.task)["steps"][-1]
self.assertEqual((step["kind"], step["decision"]["by"], step["decision"]["rejected"]), ("decision", "together", ["晨光森林"]))
with self.assertRaises(ToolError):
mcp_task_tools.log_decision(self.ctx, "背景", "")
with self.assertRaises(ToolError):
mcp_task_tools.log_decision(self.ctx, "背景", "x", by="boss")
def test_the_users_own_choices_are_decisions_too_and_the_ai_hears_them_again(self):
shown = ai_panels.show("A", "风格", [{"type": "choices", "id": "style", "question": "选风格?", "options": ["水彩", "厚涂"]},
{"type": "input", "id": "more", "question": "补充?"}], task_id=self.task)
step = next(item for item in ai_tasks.get_task(self.task)["steps"] if item.get("kind") == "panel")
ai_panels.answer(self.task, step["id"], "style", ["o1"])
ai_panels.answer(self.task, step["id"], "more", skip=True)
ai_toolbox.pick(self.task, {"nodes": ["KSampler"]}, "only")
ai_task_canvas.set_mode(self.task, "current")
ai_task_canvas.set_mode(self.task, "current") # no change: no second decision
mcp_task_tools.log_decision(self.ctx, "尺寸", "竖图 832×1216", by="user")
found = ai_decisions.decisions(ai_tasks.get_task(self.task))
self.assertEqual([item["type"] for item in found], ["answer", "answer", "toolbox", "canvas", "logged"])
self.assertEqual((found[0]["question"], found[0]["answer"]), ("选风格?", "水彩"))
self.assertTrue(found[1]["skipped"])
self.assertEqual((found[2]["mode"], found[2]["count"]), ("only", 1))
self.assertEqual(found[3]["mode"], "current")
begun = mcp_task_tools.begin_task(self.ctx, "继续")
self.assertIn("尺寸: 竖图 832×1216", begun["decisions"])
self.assertIn('The user answered "选风格?": 水彩', begun["decisions"])
self.assertTrue(shown["id"])
def test_a_gate_answer_is_a_decision_and_the_gap_counts_work_since_the_last_one(self):
self.assertIsNone(ai_decisions.gap(ai_tasks.get_task(self.task)))
ai_tasks.record("A", "run_workflow", {}, ok({}), time.time())
ai_tasks.record("A", "set_values", {}, ok({}), time.time())
self.assertEqual(ai_decisions.gap(ai_tasks.get_task(self.task)), {"runs": 1, "canvas": 1})
mcp_task_tools.log_decision(self.ctx, "步数", "30 步")
self.assertIsNone(ai_decisions.gap(ai_tasks.get_task(self.task)))
with ai_tasks._lock:
task = ai_tasks._load()[self.task]
task["steps"].append({"id": "g1", "at": time.time(), "tool": "download_model", "kind": "download", "ok": True,
"state": "asking", "gate": {"gate": "download", "signature": "x"}, "results": []})
ai_tasks._save(task)
ai_gates.decide(self.task, "g1", "deny", "先别下")
gate = next(item for item in ai_decisions.decisions(ai_tasks.get_task(self.task)) if item["type"] == "gate")
self.assertEqual((gate["tool"], gate["allowed"], gate["text"]), ("download_model", False, "先别下"))
if __name__ == "__main__":
unittest.main()
+201
View File
@@ -0,0 +1,201 @@
"""Gates (api/ai_gates.py): what AI apps may do without asking, the step that waits for the
user's OK, an answer that comes after the AI stopped waiting. Everything in temporary folders."""
import json
import os
import sys
import tempfile
import threading
import time
import unittest
from unittest import mock
from pathlib import Path
PLUGIN = Path(__file__).resolve().parents[1]
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1]), str(PLUGIN / "tests")]
from api import ai_gates, ai_panels, ai_tasks # noqa: E402
from test_ai_tasks import isolate, ok # noqa: E402
GATES = tempfile.mkdtemp(prefix="amb_ai_gates_")
class GateTests(unittest.TestCase):
def setUp(self):
isolate()
ai_gates._path = lambda: os.path.join(GATES, "ai_gates.json")
if os.path.exists(ai_gates._path()):
os.remove(ai_gates._path())
def call(self, tool, arguments):
"""What mcp_server does: a running step, the gate, then the outcome recorded."""
ref = ai_tasks.begin_step("Claude", tool, arguments, time.time(), False)
verdict = ai_gates.check(tool, arguments, ref)
result = verdict if verdict is not None else ok({"done": True})
ai_tasks.finish_step("Claude", tool, arguments, result, time.time(), False, step_ref=ref)
return ref, verdict
def answer_soon(self, ref, decision, text=""):
def later():
for _ in range(100):
task = ai_tasks.get_task(ref[0])
if any(step.get("state") == "asking" for step in task["steps"]):
ai_gates.decide(ref[0], ref[1], decision, text)
return
time.sleep(0.02)
threading.Thread(target=later).start()
def test_policies(self):
self.assertEqual(ai_gates.policies(), ai_gates.DEFAULTS)
self.assertEqual(ai_gates.set_policy("gpu", "ask")["gpu"], "ask")
self.assertEqual(ai_gates.policies()["gpu"], "ask")
with self.assertRaises(ValueError):
ai_gates.set_policy("gpu", "maybe")
with open(ai_gates._path(), "w", encoding="utf-8") as target:
json.dump({"canvas": "never", "gpu": "nonsense"}, target)
self.assertEqual((ai_gates.policies()["canvas"], ai_gates.policies()["gpu"]), ("never", "auto"))
def test_how_long_the_ai_waits_for_an_answer(self):
ai_gates._tools_path = lambda: os.path.join(GATES, "ai_tools.json")
if os.path.exists(ai_gates._tools_path()):
os.remove(ai_gates._tools_path())
self.assertEqual(ai_gates.answer_wait(), 180)
self.assertEqual(ai_gates.set_answer_wait(600), 600)
self.assertEqual(ai_gates.answer_wait(), 600)
for wrong in (30, True, "600"):
with self.assertRaises(ValueError):
ai_gates.set_answer_wait(wrong)
def test_how_the_ai_waits_for_a_run(self):
ai_gates._tools_path = lambda: os.path.join(GATES, "ai_tools.json")
if os.path.exists(ai_gates._tools_path()):
os.remove(ai_gates._tools_path())
self.assertEqual(ai_gates.run_wait(), "call", "the user tells the AI when it is done")
ai_gates.set_answer_wait(600)
self.assertEqual(ai_gates.set_run_wait("wait"), "wait")
self.assertEqual((ai_gates.run_wait(), ai_gates.answer_wait()), ("wait", 600), "one setting keeps the other")
with open(ai_gates._tools_path(), "w", encoding="utf-8") as target:
target.write("[1]")
self.assertEqual(ai_gates.run_wait(), "call", "a broken file: the defaults")
with self.assertRaises(ValueError):
ai_gates.set_run_wait("soon")
def test_auto_and_never(self):
self.assertIsNone(self.call("set_prompt", {"prompt": "x"})[1])
self.assertIsNone(self.call("search_models", {})[1], "reads have no gate")
ai_gates.set_policy("canvas", "never")
ref, verdict = self.call("set_prompt", {"prompt": "x"})
self.assertTrue(verdict["isError"])
self.assertEqual(ai_tasks.get_task(ref[0])["steps"][-1]["state"], "error")
def test_ask_and_allow(self):
ai_gates.set_policy("gpu", "ask")
ref = ai_tasks.begin_step("Claude", "run_workflow", {"batch": 2}, time.time(), False)
self.answer_soon(ref, "allow")
self.assertIsNone(ai_gates.check("run_workflow", {"batch": 2}, ref))
step = ai_tasks.get_task(ref[0])["steps"][0]
self.assertEqual((step["state"], step["gate"]["decision"]), ("running", "allow"))
self.assertEqual(step["gate"]["lines"], [{"label": "batch", "value": "2"}])
def test_ask_and_no_with_words(self):
ai_gates.set_policy("download", "ask")
ref = ai_tasks.begin_step("Claude", "download_model", {"type": "loras", "url": "u"}, time.time(), False)
self.answer_soon(ref, "deny", "use the one I have")
verdict = ai_gates.check("download_model", {"type": "loras", "url": "u"}, ref)
self.assertTrue(verdict["isError"])
self.assertIn("use the one I have", verdict["content"][0]["text"])
def test_one_allow_cannot_authorize_another_call(self):
ai_gates.set_policy("gpu", "ask")
ref = ai_tasks.begin_step("Claude", "run_workflow", {}, time.time())
self.answer_soon(ref, "allow")
self.assertIsNone(ai_gates.check("run_workflow", {}, ref))
with mock.patch.object(ai_gates, "WAIT_SECONDS", 0):
_, verdict = self.call("run_workflow", {})
self.assertTrue(verdict["structuredContent"]["waiting_for_user"])
def test_late_allow_is_consumed_only_once_even_after_retry(self):
ai_gates.set_policy("gpu", "ask")
with mock.patch.object(ai_gates, "WAIT_SECONDS", 0):
ref, _ = self.call("run_workflow", {})
ai_gates.decide(*ref, "allow")
self.assertIsNone(self.call("run_workflow", {})[1])
self.assertTrue(self.call("run_workflow", {})[1]["structuredContent"]["waiting_for_user"])
def test_ask_fails_closed_without_its_record(self):
ai_gates.set_policy("gpu", "ask")
for ref in (None, ("missing", "missing")):
self.assertTrue(ai_gates.check("run_workflow", {}, ref)["isError"])
def test_late_decision_survives_tool_completion_race(self):
ai_gates.set_policy("gpu", "ask")
for decision, expected in (("allow", "allowed"), ("deny", "error")):
arguments = {"batch": 1 if decision == "allow" else 2}
ref = ai_tasks.begin_step("Claude", "run_workflow", arguments, time.time())
with mock.patch.object(ai_gates, "WAIT_SECONDS", 0):
verdict = ai_gates.check("run_workflow", arguments, ref)
ai_gates.decide(*ref, decision)
ai_tasks.finish_step("Claude", "run_workflow", {}, verdict, time.time(), step_ref=ref)
step = ai_tasks.get_task(ref[0])["steps"][-1]
self.assertEqual(step["state"], expected)
self.assertEqual(step["ok"], decision == "allow")
def test_restart_makes_pending_gate_answer_deliverable(self):
ai_gates.set_policy("gpu", "ask")
ref = ai_tasks.begin_step("Claude", "run_workflow", {}, time.time())
task = ai_tasks._load()[ref[0]]
task["steps"][-1].update(state="asking", gate={"gate": "gpu", "signature": ai_gates._signature("run_workflow", {})})
ai_tasks._save(task)
ai_tasks._tasks = None # the call waiting on the old process is gone
ai_gates.decide(*ref, "allow")
self.assertEqual(ai_tasks.get_task(ref[0])["steps"][-1]["state"], "allowed")
self.assertEqual(ai_panels.get_answers("Claude", task_id=ref[0])["messages"][0]["kind"], "gate")
def test_late_decision_wakes_a_waiting_answer_reader(self):
ai_gates.set_policy("gpu", "ask")
with mock.patch.object(ai_gates, "WAIT_SECONDS", 0):
ref, _ = self.call("run_workflow", {})
waiting = threading.Event()
original = ai_panels._answered.wait
def wait(timeout=None):
waiting.set()
return original(timeout)
answers = []
with mock.patch.object(ai_panels._answered, "wait", side_effect=wait):
reader = threading.Thread(target=lambda: answers.append(ai_panels.get_answers("Claude", task_id=ref[0], wait_seconds=3)))
reader.start()
self.assertTrue(waiting.wait(1))
ai_gates.decide(*ref, "allow")
reader.join(1)
woke_promptly = not reader.is_alive()
reader.join(3)
self.assertTrue(woke_promptly, "a decision should wake get_answers, not wait out its timeout")
self.assertEqual(answers[0]["messages"][0]["kind"], "gate")
def test_an_answer_after_the_ai_stopped_waiting(self):
ai_gates.set_policy("gpu", "ask")
ai_gates.WAIT_SECONDS, wait = 0.05, ai_gates.WAIT_SECONDS
try:
ref, verdict = self.call("run_workflow", {"batch": 1})
finally:
ai_gates.WAIT_SECONDS = wait
self.assertTrue(verdict["structuredContent"]["waiting_for_user"])
self.assertFalse(verdict["isError"])
self.assertEqual(ai_tasks.get_task(ref[0])["steps"][0]["state"], "asking")
self.assertEqual(ai_tasks.list_tasks()[0]["asks"], 1, "the task asks the user")
ai_gates.decide(ref[0], ref[1], "allow", remember=True)
self.assertEqual(ai_tasks.get_task(ref[0])["steps"][0]["state"], "allowed")
self.assertEqual(ai_gates.policies()["gpu"], "auto", "remembered: no more asking")
message = ai_panels.attach_pending(ok({}), "Claude", task_id=ref[0])["structuredContent"]["user_messages"][0]
self.assertEqual(message["kind"], "gate")
self.assertIn("call it again", message["text"])
ai_gates.set_policy("gpu", "ask")
again, verdict = self.call("run_workflow", {"batch": 1})
self.assertIsNone(verdict, "the same call goes ahead without asking twice")
steps = ai_tasks.get_task(ref[0])["steps"]
self.assertEqual(steps[0]["state"], "done")
with self.assertRaises(ValueError):
ai_gates.decide(ref[0], ref[1], "allow")
if __name__ == "__main__":
unittest.main()
+205
View File
@@ -0,0 +1,205 @@
"""Task handoff and answer deadlines, using only the existing temporary test store."""
import threading
import time
import unittest
from concurrent.futures import ThreadPoolExecutor
from unittest import mock
from test_ai_tasks import isolate
from api import ai_answer_wait, ai_gates, ai_panels, ai_tasks, mcp_task_tools
class HandoffEdges(unittest.TestCase):
def setUp(self):
isolate()
ai_answer_wait._typed.clear()
self.ctx = {"client": "A", "session": "one"}
def ask(self, recommended=True):
options = [{"id": "a", "label": "A", "recommended": recommended}, {"id": "b", "label": "B"}]
with mock.patch.object(mcp_task_tools, "ASK_WAIT_MIN", 0):
return mcp_task_tools.ask_user(self.ctx, "Which?", options, wait_seconds=0)
def test_unscoped_timeout_delivers_auto_answer_in_same_call(self):
with mock.patch.object(ai_gates, "answer_wait", return_value=0):
self.ask()
with mock.patch.object(ai_gates, "no_answer", return_value="recommended"):
answer, _ = mcp_task_tools.get_answers(self.ctx)
self.assertEqual(answer["answers"][0]["choice"], ["a"])
self.assertTrue(answer["answers"][0]["recommended_on_timeout"])
self.assertEqual(answer["still_waiting"], [])
def test_unscoped_expired_question_tells_ai_to_stop(self):
with mock.patch.object(ai_gates, "answer_wait", return_value=0):
self.ask(False)
answer, _ = mcp_task_tools.get_answers(self.ctx)
self.assertIs(answer.get("keep_waiting"), False)
self.assertIn("end your turn", answer["note"])
def test_user_answer_wins_race_with_auto_pick_and_is_returned_now(self):
with mock.patch.object(ai_gates, "answer_wait", return_value=0):
asked = self.ask()
original = ai_panels.answer
def user_first(task_id, step_id, block_id, **kwargs):
original(task_id, step_id, block_id, choice=["b"])
return original(task_id, step_id, block_id, **kwargs)
with mock.patch.object(ai_gates, "no_answer", return_value="recommended"), \
mock.patch.object(ai_panels, "answer", side_effect=user_first):
answer, _ = mcp_task_tools.get_answers(self.ctx, panel=asked["panel"])
self.assertEqual(answer["answers"][0]["choice"], ["b"])
self.assertNotIn("recommended_on_timeout", answer["answers"][0])
self.assertNotIn("end your turn", answer.get("note", ""))
def test_resume_uses_user_reply_time_not_background_task_update(self):
older = ai_tasks.begin_task("A", "Older", now=100)
ai_panels.leave_message(older["id"], "first", now=110)
newer = ai_tasks.begin_task("A", "Newer", now=120, new_task=True)
ai_panels.leave_message(newer["id"], "second", now=130)
older["updated"] = 140 # a late run monitor / metadata update is not a new user reply
ai_tasks._current.clear()
self.assertEqual(ai_tasks.current_task("A", now=150)["id"], newer["id"])
def test_new_session_does_not_take_another_sessions_inflight_task(self):
busy = ai_tasks.begin_task("A", "Busy", now=100, session="one")
ai_panels.leave_message(busy["id"], "keep working", now=101)
running = ai_tasks.begin_step("A", "describe_canvas", {}, 102, True, session="one")
self.assertIsNotNone(running)
other = ai_tasks.current_task("A", now=103, session="two")
self.assertNotEqual(other["id"], busy["id"])
self.assertEqual(ai_tasks.active_task_id("A", "one"), busy["id"])
def test_recent_panel_answer_can_resume_its_task(self):
asked = self.ask()
task, step = ai_panels._find(asked["panel"])
task["ended"] = time.time()
ai_panels.answer(task["id"], step["id"], "answer", choice=["b"])
ai_tasks._current.clear()
self.assertEqual(ai_tasks.current_task("A", session="next")["id"], task["id"])
def test_typing_defers_automatic_choice(self):
asked = self.ask()
task, step = ai_panels._find(asked["panel"])
late = step["at"] + ai_gates.answer_wait() + 5
ai_answer_wait.typing(task["id"], now=late - 1)
with mock.patch.object(ai_gates, "no_answer", return_value="recommended"):
answer = ai_answer_wait.wait_on(asked["panel"], now=late)
self.assertTrue(answer["keep_waiting"])
self.assertTrue(answer["typing"])
self.assertFalse(step.get("answers"))
def test_a_half_written_answer_is_never_overruled(self):
asked = self.ask()
task, step = ai_panels._find(asked["panel"])
ai_answer_wait.typing(task["id"], now=step["at"] + 30) # then they walked away
late = step["at"] + 30 + ai_answer_wait.AFTER_TYPING + ai_gates.answer_wait()
with mock.patch.object(ai_gates, "no_answer", return_value="recommended"):
answer = ai_answer_wait.wait_on(asked["panel"], now=late)
self.assertIs(answer["keep_waiting"], False, "the AI does not wait forever")
self.assertIn("began writing", answer["note"])
self.assertFalse(step.get("answers"), "the recommended option is not taken over their words")
def test_typing_before_the_question_does_not_count_as_an_answer(self):
asked = self.ask()
task, step = ai_panels._find(asked["panel"])
ai_answer_wait.typing(task["id"], now=step["at"] - 100) # a message to the AI, before it asked
with mock.patch.object(ai_gates, "no_answer", return_value="recommended"):
answer = ai_answer_wait.wait_on(asked["panel"], now=step["at"] + ai_gates.answer_wait() + 1)
self.assertTrue(answer.get("took_recommended"))
def test_ineligible_replies_never_resume_a_task(self):
for entry in ({"at": 100}, {"at": 7301}, {"at": 7299, "delivered": True},
{"at": 7299, "recommended_on_timeout": True}):
with self.subTest(entry=entry):
isolate()
task = ai_tasks.begin_task("A", "Old", now=100)
task["steps"].append({"kind": "panel", "answers": {"answer": entry}})
ai_tasks._current.clear()
self.assertNotEqual(ai_tasks.current_task("A", now=7300)["id"], task["id"])
def test_timeout_with_multiple_panels_delivers_once_and_keeps_other_question(self):
asked = self.ask()
other = self.ask(False)
task, step = ai_panels._find(asked["panel"])
step["at"] -= ai_gates.answer_wait() + 5
with mock.patch.object(ai_gates, "no_answer", return_value="recommended"):
answer, _ = mcp_task_tools.get_answers(self.ctx)
again, _ = mcp_task_tools.get_answers(self.ctx)
self.assertEqual(len(answer["answers"]), 1)
self.assertEqual(answer["still_waiting"], [{"panel": other["panel"], "question": "answer"}])
self.assertEqual(again["answers"], [])
self.assertTrue(again["keep_waiting"])
def test_ask_user_returns_a_racing_user_answer(self):
original = ai_panels.answer
def user_first(task_id, step_id, block_id, **kwargs):
original(task_id, step_id, block_id, choice=["b"])
return original(task_id, step_id, block_id, **kwargs)
with mock.patch.object(ai_gates, "answer_wait", return_value=0), \
mock.patch.object(ai_gates, "no_answer", return_value="recommended"), \
mock.patch.object(ai_panels, "answer", side_effect=user_first):
got = self.ask()
self.assertTrue(got["answered"])
self.assertEqual(got["choice"], ["b"])
def test_ask_user_delivers_messages_and_images_received_while_waiting(self):
from api import ai_marks
original = ai_panels.get_answers
sent = []
def message_first(client, panel_id=None, *args, **kwargs):
if not sent:
task, _ = ai_panels._find(panel_id)
sent.append(ai_panels.leave_message(task["id"], "Stop and read this first"))
return original(client, panel_id, *args, **kwargs)
with mock.patch.object(mcp_task_tools, "ASK_WAIT_MIN", 0), \
mock.patch.object(ai_panels, "get_answers", side_effect=message_first), \
mock.patch.object(ai_marks, "image_items", return_value=[{"type": "image"}]) as images:
got, media = mcp_task_tools.TASK_TOOLS["ask_user"]["run"](self.ctx, question="Which?", wait_seconds=0)
self.assertFalse(got["answered"])
self.assertEqual(got["messages"][0]["id"], sent[0]["id"])
self.assertEqual(media, [{"type": "image"}])
images.assert_called_once_with(got["messages"])
self.assertNotIn("messages", original("A"))
def test_simultaneous_user_answer_and_timeout_commit_only_one_choice(self):
asked = self.ask()
task, step = ai_panels._find(asked["panel"])
late = step["at"] + ai_gates.answer_wait() + 5
start = threading.Barrier(2)
def user():
start.wait(timeout=5)
try:
ai_panels.answer(task["id"], step["id"], "answer", choice=["b"])
return True
except ai_panels.PanelError as error:
self.assertEqual(str(error), "Already answered")
return False
def timeout():
start.wait(timeout=5)
return ai_answer_wait.wait_on(asked["panel"], now=late)
with mock.patch.object(ai_gates, "no_answer", return_value="recommended"), ThreadPoolExecutor(2) as pool:
user_result, timeout_result = pool.submit(user), pool.submit(timeout)
won = user_result.result(timeout=10)
timeout_result.result(timeout=10)
received = ai_panels.get_answers("A", asked["panel"])
self.assertEqual(len(received["answers"]), 1)
self.assertEqual(received["answers"][0]["choice"], ["b"] if won else ["a"])
self.assertEqual(bool(received["answers"][0].get("recommended_on_timeout")), not won)
self.assertEqual(ai_panels.get_answers("A", asked["panel"])["answers"], [])
def test_message_arriving_at_deadline_prevents_an_automatic_choice(self):
asked = self.ask()
task, step = ai_panels._find(asked["panel"])
step["at"] -= ai_gates.answer_wait() + 5
original = ai_panels.get_answers
sent = []
def message_after_snapshot(*args, **kwargs):
snapshot = original(*args, **kwargs)
if not sent:
sent.append(ai_panels.leave_message(task["id"], "Stop, I changed my mind"))
return snapshot
with mock.patch.object(ai_panels, "get_answers", side_effect=message_after_snapshot), \
mock.patch.object(ai_gates, "no_answer", return_value="recommended"):
result, _ = mcp_task_tools.get_answers(self.ctx)
self.assertEqual(result["messages"][0]["id"], sent[0]["id"])
self.assertEqual(result["answers"], [])
self.assertFalse(step.get("answers"))
+156
View File
@@ -0,0 +1,156 @@
"""Marks the user draws on a picture for the AI (api/ai_marks.py): checked, kept with the task,
given to the AI as a message with the marked picture attached. Temporary folders only."""
import base64
import io
import os
import sys
import tempfile
import time
import unittest
from pathlib import Path
from unittest import mock
PLUGIN = Path(__file__).resolve().parents[1]
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1]), str(PLUGIN / "tests")]
from PIL import Image # noqa: E402
from api import ai_marks, ai_panels, ai_tasks, mcp_task_tools # noqa: E402
from api.mcp_tools import ToolError # noqa: E402
from test_ai_tasks import isolate, ok # noqa: E402
OUTPUT = tempfile.mkdtemp(prefix="amb_ai_marks_output_")
def png(size=(64, 48)):
buffer = io.BytesIO()
Image.new("RGB", size, (200, 30, 30)).save(buffer, "PNG")
return "data:image/png;base64," + base64.b64encode(buffer.getvalue()).decode()
MARKS = [{"shape": "box", "x": 0.2, "y": 0.1, "w": 0.2, "h": 0.2, "note": "头发再亮一点"}, {"shape": "pen", "x": 0.5, "y": 0.5, "w": 0.9, "h": 0.9}]
class MarkTests(unittest.TestCase):
def setUp(self):
isolate()
Image.new("RGB", (8, 8)).save(os.path.join(OUTPUT, "a.png"))
self.patch = mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT)
self.patch.start()
ai_tasks.record("Claude", "describe_canvas", {}, ok({}), time.time(), True)
self.task = ai_tasks.list_tasks()[0]["id"]
def tearDown(self):
self.patch.stop()
def test_marks_are_kept_with_the_task_and_reach_the_ai_with_the_picture(self):
step = ai_marks.save(self.task, "a.png", png(), MARKS, "整体暖一点")
self.assertEqual(step["marks"][1]["w"], 0.5, "a mark stays inside the picture")
kept = ai_marks.path_of(self.task, step["marked"])
self.assertTrue(kept and kept.startswith(ai_tasks.marks_dir(self.task)))
self.assertNotIn(OUTPUT, kept, "never in the output folder")
self.assertIn("Mark 1 (box, x 20–40%, y 10–30% of the picture): 头发再亮一点", step["text"])
with mock.patch.object(ai_marks, "_image_content", return_value={"type": "image", "data": "x", "mimeType": "image/webp"}):
result = ai_panels.attach_pending(ok({}), "Claude", task_id=self.task)
message = result["structuredContent"]["user_messages"][0]
self.assertEqual((message["kind"], message["image"]), ("marks", "a.png"))
self.assertNotIn("_marked", message, "the path on disk stays here")
self.assertEqual(result["content"][-1]["type"], "image", "the marked picture is attached")
self.assertTrue(ai_tasks.get_task(self.task)["steps"][-1]["delivered"])
ai_tasks.remove_task(self.task)
self.assertFalse(os.path.exists(ai_tasks.marks_dir(self.task)), "gone with its task")
def test_get_answers_gives_the_marked_picture_too(self):
ai_marks.save(self.task, "a.png", png(), MARKS)
with mock.patch.object(ai_marks, "_image_content", return_value={"type": "image", "data": "x", "mimeType": "image/webp"}):
answer, pictures = mcp_task_tools.get_answers({"client": "Claude", "task": self.task})
self.assertEqual(answer["messages"][0]["kind"], "marks")
self.assertEqual(len(pictures), 1)
def test_the_ai_replies_under_the_marks_in_anomalous(self):
marks = ai_marks.save(self.task, "a.png", png(), MARKS)
with mock.patch.object(ai_marks, "_image_content", return_value={"type": "image", "data": "x", "mimeType": "image/webp"}):
result = ai_panels.attach_pending(ok({}), "Claude", task_id=self.task)
self.assertIn("report(reply_to=", result["structuredContent"]["user_notice"])
reply = mcp_task_tools.report({"client": "Claude", "task": self.task}, "### 发现" + chr(10) + "- 多了一只手", reply_to=marks["id"])
self.assertTrue(reply["noted"])
steps = {step["id"]: step for step in ai_tasks.get_task(self.task)["steps"]}
note = next(step for step in steps.values() if step["tool"] == "report")
self.assertEqual((note["reply_to"], steps[marks["id"]]["replied"]), (marks["id"], note["id"]))
with self.assertRaises(ToolError):
mcp_task_tools.report({"client": "Claude", "task": self.task}, "x", reply_to=note["id"]) # not the user's
def test_marks_on_an_ended_task_open_it_again_for_the_ais_next_call(self):
mcp_task_tools.report({"client": "Claude"}, "做完了", done=True)
self.assertTrue(ai_tasks.get_task(self.task)["ended"])
marks = ai_marks.save(self.task, "a.png", png(), MARKS)
self.assertIsNone(ai_tasks.get_task(self.task)["ended"])
with mock.patch.object(ai_marks, "_image_content", return_value={"type": "image", "data": "x", "mimeType": "image/webp"}):
reply = mcp_task_tools.report({"client": "Claude"}, "改好了", reply_to=marks["id"])
self.assertTrue(reply["noted"], "the reply lands under the marks, not in a new task")
self.assertEqual(len(ai_tasks.list_tasks()), 1)
def test_a_repaint_mask_becomes_the_picture_with_that_area_as_its_mask(self):
inputs = tempfile.mkdtemp(prefix="amb_ai_marks_input_")
buffer = io.BytesIO()
mask = Image.new("L", (64, 48), 0)
mask.paste(255, (0, 0, 32, 48)) # the left half
mask.save(buffer, "PNG")
data = "data:image/png;base64," + base64.b64encode(buffer.getvalue()).decode()
with mock.patch.object(ai_marks.folder_paths, "get_input_directory", return_value=inputs):
step = ai_marks.save(self.task, "a.png", png(), [], "去掉多余的手", mask=data)
with self.assertRaises(ai_marks.MarksError):
blank = io.BytesIO()
Image.new("L", (8, 8), 0).save(blank, "PNG")
ai_marks.save(self.task, "a.png", png(), [], "", mask="data:image/png;base64," + base64.b64encode(blank.getvalue()).decode())
self.assertRegex(step["inpaint"], r"^anomalous_inpaint_[0-9a-f]{10}\.png$")
with Image.open(os.path.join(inputs, step["inpaint"])) as made:
self.assertEqual((made.mode, made.size), ("RGBA", (8, 8)))
self.assertEqual((made.getpixel((1, 4))[3], made.getpixel((6, 4))[3]), (0, 255), "marked: transparent, so LoadImage's MASK is 1 there")
with mock.patch.object(ai_marks, "_image_content", return_value=None):
message = ai_panels.attach_pending(ok({}), "Claude", task_id=self.task)["structuredContent"]["user_messages"][0]
self.assertEqual(message["inpaint_image"], step["inpaint"])
self.assertIn("SetLatentNoiseMask", message["inpaint_note"])
self.assertEqual(os.listdir(inputs), [step["inpaint"]], "an empty mask writes nothing")
def test_how_to_change_it_reaches_the_ai_first_as_the_users_choice(self):
inputs = tempfile.mkdtemp(prefix="amb_ai_marks_input_")
buffer = io.BytesIO()
Image.new("L", (8, 8), 255).save(buffer, "PNG")
mask = "data:image/png;base64," + base64.b64encode(buffer.getvalue()).decode()
with mock.patch.object(ai_marks.folder_paths, "get_input_directory", return_value=inputs):
whole = ai_marks.save(self.task, "a.png", png(), MARKS, "亮一点", mask=mask, mode="whole")
local = ai_marks.save(self.task, "a.png", png(), MARKS, mask=mask, mode="local")
left = ai_marks.save(self.task, "a.png", png(), MARKS)
self.assertEqual((whole["mode"], "inpaint" in whole), ("whole", False), "anew: no mask is kept even when one is sent")
self.assertEqual(os.listdir(inputs), [local["inpaint"]])
self.assertEqual(left["mode"], "auto")
for step, words in ((whole, "make the whole picture anew"), (local, "change ONLY the marked areas"), (left, "up to you")):
self.assertIn(words, step["text"].split(chr(10))[0], "the choice is the first thing the AI reads")
with mock.patch.object(ai_marks, "_image_content", return_value=None):
messages = ai_panels.attach_pending(ok({}), "Claude", task_id=self.task)["structuredContent"]["user_messages"]
self.assertEqual([message["mode"] for message in messages], ["whole", "local", "auto"])
self.assertEqual(["inpaint_image" in message for message in messages], [False, True, False])
def test_what_is_refused(self):
for image, picture, marks in [("../a.png", png(), MARKS), ("nope.png", png(), MARKS), ("a.png", "data:image/jpeg;base64,AAAA", MARKS),
("a.png", "data:image/png;base64," + base64.b64encode(b"not a png").decode(), MARKS),
("a.png", png((5000, 10)), MARKS), ("a.png", png(), []), ("a.png", png(), [{"shape": "star"}]),
("a.png", png(), MARKS * 11)]:
with self.assertRaises(ai_marks.MarksError, msg=(image, marks[:1] if marks else marks)):
ai_marks.save(self.task, image, picture, marks)
with self.assertRaises(ai_marks.MarksError):
ai_marks.save("no-task", "a.png", png(), MARKS)
for mode in ("sometimes", "local"): # not a way; only the marked areas without an area to repaint
with self.assertRaises(ai_marks.MarksError, msg=mode):
ai_marks.save(self.task, "a.png", png(), MARKS, mode=mode)
self.assertIsNone(ai_marks.path_of(self.task, "../../x.png"))
def test_path_on_another_drive_is_a_validation_error(self):
with mock.patch.object(ai_tasks, "_output_media", side_effect=ValueError("Paths don't have the same drive")):
with self.assertRaises(ai_marks.MarksError):
ai_marks.save(self.task, "Z:/outside.png", png(), MARKS)
if __name__ == "__main__":
unittest.main()
+82
View File
@@ -0,0 +1,82 @@
"""What a message's "@" point at (ai_message_refs.py): checked when it is left, given to the AI
with the message, each thing in full or marked missing, pictures attached with what made them."""
import unittest
from unittest import mock
from test_ai_tasks import OUTPUT, isolate, ok
from test_ai_run_map import lora_workflow, picture
from api import ai_message_refs, ai_panels, ai_tasks
from api.mcp_tools import ToolError
class RefsTests(unittest.TestCase):
def setUp(self):
isolate()
self.task = ai_tasks.begin_task("A", "Refs")["id"]
patcher = mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT)
patcher.start()
self.addCleanup(patcher.stop)
def test_what_the_at_points_at_reaches_the_ai_in_full(self):
combo = {"name": "水彩风景", "model": "a.safetensors", "prompt": "watercolor"}
refs = [{"kind": "combos", "id": "水彩风景", "label": "水彩风景"}, {"kind": "combos", "id": "水彩风景"},
{"kind": "models", "id": "loras:0:gone.safetensors", "label": "gone"},
{"kind": "pictures", "id": picture("ref.png", lora_workflow()), "label": "ref.png"},
{"kind": "nodes", "id": "KSampler", "label": "K采样器"}]
step = ai_panels.leave_message(self.task, "用 @水彩风景 的感觉重画 @ref.png", refs)
self.assertEqual(len(step["refs"]), 4, "each thing once")
with mock.patch("api.mcp_tools.list_combos", return_value={"combos": [combo]}), \
mock.patch("api.mcp_tools.get_model", side_effect=ToolError("gone")):
result = ai_panels.attach_pending(ok({}), "A", task_id=self.task)
message = result["structuredContent"]["user_messages"][0]
by_kind = {item["kind"]: item for item in message["refs"]}
self.assertEqual(by_kind["combos"]["details"], combo)
self.assertEqual(by_kind["combos"]["said_as"], "@水彩风景")
self.assertTrue(by_kind["models"]["missing"])
self.assertEqual(by_kind["pictures"]["made_with"]["models"], ["Kirazuri_v4"])
self.assertIn("find_nodes", by_kind["nodes"]["note"])
self.assertEqual(sum(1 for item in result["content"] if item.get("type") == "image"), 1, "the picture itself goes along")
self.assertNotIn("_pictures", message)
def test_a_deleted_picture_is_marked_gone_and_the_ai_told_plainly(self):
import os
from api import ai_pictures, ai_task_changes
name = picture("bye.png")
ai_pictures.hand(self.task, [name, picture("stay.png")])
os.remove(os.path.join(OUTPUT, name))
marked = ai_pictures.mark_gone(ai_tasks.get_task(self.task))
results = marked["steps"][-1]["results"]
self.assertEqual([item.get("gone", False) for item in results], [True, False])
self.assertNotIn("gone", ai_tasks.get_task(self.task)["steps"][-1]["results"][0], "only the page's copy is marked")
words = ai_task_changes._change_text({"action": "image_delete", "target": "bye.png", "source": "gallery"})
self.assertIn("do not use, show or hand it in again", words)
def test_a_plain_message_has_no_refs_and_bad_refs_are_refused(self):
self.assertNotIn("refs", ai_panels.leave_message(self.task, "继续"))
# Pictures alone, no words: still a message.
alone = ai_panels.leave_message(self.task, "", [{"kind": "pictures", "id": "ref.png", "label": "ref.png"}])
self.assertEqual((alone["text"], [ref["id"] for ref in alone["refs"]]), ("", ["ref.png"]))
for refs in ("x", [{"kind": "files", "id": "a"}], [{"kind": "models", "id": ""}], [{"kind": "nodes", "id": str(n)} for n in range(13)]):
with self.assertRaises(ai_panels.PanelError, msg=refs):
ai_panels.leave_message(self.task, "看这个", refs)
self.assertEqual(ai_message_refs.clean(None), [])
def test_exact_names_are_resolved_before_search_limits(self):
from api import mcp_tools
names = [f'cat {i}' for i in range(60)] + ['cat']
combos = [{'name': name, 'data': {'promptEn': 'watercolor'}} for name in names]
recipes = [{'filename': f'{i}.json', 'data': {'name': name}} for i, name in enumerate(names)]
recipes.append({'filename': 'unnamed.json', 'data': {}})
with mock.patch.object(mcp_tools, '_list_notebooks', return_value=combos), \
mock.patch.object(mcp_tools, '_list_recipes', return_value=recipes):
for kind, lister in [('combos', mcp_tools.list_combos), ('workflows', mcp_tools.list_workflows)]:
self.assertNotIn('cat', [item['name'] for item in lister('cat', 5)[kind]])
result = ai_message_refs.message_of({'refs': [{'kind': kind, 'id': 'cat', 'label': 'cat'}]})
self.assertEqual(result['refs'][0]['details']['name'], 'cat')
self.assertIsNone(ai_message_refs.lookup(kind, 'missing'))
self.assertEqual(ai_message_refs.lookup('workflows', 'unnamed.json')['name'], 'unnamed.json')
if __name__ == "__main__":
unittest.main()
+112
View File
@@ -0,0 +1,112 @@
"""What changed outside Anomalous while an AI works (api/ai_outside.py): pictures no run saved and
pictures rewritten in place, found when the AI calls, told once; and the AI's own word on what it
did with other tools. Pictures go to temporary folders; their times are set by the test."""
import json
import os
import sys
import tempfile
import time
import unittest
from pathlib import Path
from unittest import mock
PLUGIN = Path(__file__).resolve().parents[1]
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1]), str(PLUGIN / "tests")]
from api import ai_outside, ai_tasks, mcp_task_tools # noqa: E402
from test_ai_tasks import isolate, ok # noqa: E402
class OutsideTests(unittest.TestCase):
def setUp(self):
isolate()
ai_outside._watched.clear()
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.dirs = {name: Path(os.path.realpath(temp.name)) / name for name in ("output", "input")}
for path in self.dirs.values():
path.mkdir()
self.times, self.saved, self.busy = {}, set(), False
for patcher in (
mock.patch.object(ai_outside.folder_paths, "get_output_directory", return_value=str(self.dirs["output"])),
mock.patch.object(ai_outside.folder_paths, "get_input_directory", return_value=str(self.dirs["input"])),
mock.patch.object(ai_outside, "_times", side_effect=lambda entry: self.times[entry.name]),
mock.patch.object(ai_outside, "_saved_by_runs", side_effect=lambda: set(self.saved)),
mock.patch.object(ai_outside, "_busy", side_effect=lambda: self.busy),
):
patcher.start()
self.addCleanup(patcher.stop)
ai_tasks.record("Gemini", "describe_canvas", {}, ok({}), time.time(), True)
self.task = ai_tasks.list_tasks()[0]["id"]
def picture(self, folder, name, made, changed):
(self.dirs[folder] / name).write_bytes(b"x")
self.times[name] = (made, changed)
def found(self, now):
step = ai_outside.check(self.task, now=now)
return step and {(item["folder"], item["name"], item["change"]) for item in step["files"]}
def test_new_and_rewritten_pictures_not_a_runs_nor_an_upload(self):
self.assertIsNone(ai_outside.check(self.task, now=100)) # the task's first call only starts the watch
self.picture("output", "script.png", 150, 150) # no run saved it
self.picture("output", "run_00001_.png", 150, 151)
self.saved = {"run_00001_.png"}
self.picture("output", "old.png", 10, 160) # rewritten in place
self.picture("output", "notes.txt", 150, 150) # not a picture
self.picture("output", "before.png", 10, 20) # unchanged
self.picture("input", "upload.png", 150, 150) # new in the input folder: an upload looks the same
self.picture("input", "mask.png", 10, 170)
self.assertEqual(self.found(200), {("output", "script.png", "new"), ("output", "old.png", "rewritten"),
("input", "mask.png", "rewritten")})
step = ai_tasks.get_task(self.task)["steps"][-1]
self.assertEqual((step["tool"], step["kind"], step["results"]), ("outside_files", "outside", [])) # no results: not the task's pictures
self.assertIsNone(ai_outside.check(self.task, now=300)) # told once
self.picture("output", "late.png", 10, 352) # its change time ahead of the clock read at 350
self.assertEqual(self.found(350), {("output", "late.png", "rewritten")})
self.assertIsNone(ai_outside.check(self.task, now=351)) # still after this check's start: not told again
self.times["old.png"] = (10, 380)
self.assertEqual(self.found(400), {("output", "old.png", "rewritten")}) # rewritten again
def test_new_pictures_wait_while_a_prompt_runs(self):
ai_outside.check(self.task, now=100)
self.busy = True
self.picture("output", "maybe_the_runs.png", 150, 150)
self.picture("output", "old.png", 10, 160)
self.assertEqual(self.found(200), {("output", "old.png", "rewritten")})
self.busy = False
self.assertEqual(self.found(300), {("output", "maybe_the_runs.png", "new")}) # no run saved it after all; old.png told already
def test_without_a_creation_time_output_only(self):
ai_outside.check(self.task, now=100)
self.picture("output", "x.png", None, 150)
self.picture("input", "y.png", None, 150)
self.assertEqual(self.found(200), {("output", "x.png", "changed")})
self.assertIsNone(ai_outside.check(None))
def test_the_ai_is_told(self):
step = {"files": [{"folder": "output", "name": "old.png", "change": "rewritten"}]}
told = ai_outside.tell(ok({"done": True}), step)
self.assertEqual(told["structuredContent"]["outside_changes"], ["output/old.png (rewritten)"])
self.assertIn("log_outside", json.loads(told["content"][0]["text"])["outside_note"])
failed = ai_outside.tell({"content": [{"type": "text", "text": "boom"}], "isError": True}, step)
self.assertEqual(failed["content"][0]["text"], "boom")
self.assertIn("output/old.png", failed["content"][-1]["text"])
self.assertNotIn("outside_note", ai_outside.tell(ok({}), None)["structuredContent"])
def test_the_ais_own_word(self):
answer = mcp_task_tools.log_outside({"client": "Gemini"}, "python crop.py", "to cut out the hand", ["output/a.png", " "])
step = ai_tasks.get_task(self.task)["steps"][-1]
self.assertEqual(answer["step"], step["id"])
self.assertEqual((step["tool"], step["kind"], step["what"], step["why"], step["files"]),
("log_outside", "outside", "python crop.py", "to cut out the hand", ["output/a.png"]))
self.assertEqual(ai_tasks.list_tasks()[0]["kinds"]["outside"], 1)
for what in (" ", "x" * 301):
with self.assertRaises(mcp_task_tools.ToolError):
mcp_task_tools.log_outside({"client": "Gemini"}, what)
with self.assertRaises(mcp_task_tools.ToolError):
mcp_task_tools.log_outside({"client": "Gemini"}, "x", files="a.png")
if __name__ == "__main__":
unittest.main()
+487
View File
@@ -0,0 +1,487 @@
"""Panels the AI draws from Anomalous's blocks (api/ai_panels.py): what is let through, the
user's answers, and the AI waiting for them. Kept in a temporary folder."""
import copy
import json
import os
import sys
import tempfile
import threading
import time
import unittest
from pathlib import Path
from unittest import mock
PLUGIN = Path(__file__).resolve().parents[1]
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
from api import ai_answer_wait, ai_gates, ai_panels, ai_tasks, mcp_task_tools # noqa: E402
TMP = tempfile.mkdtemp(prefix="amb_ai_panels_")
OUTPUT = tempfile.mkdtemp(prefix="amb_ai_panels_out_")
Path(OUTPUT, "a.png").write_bytes(b"png")
Path(OUTPUT, "b.png").write_bytes(b"png")
def isolate():
ai_tasks.tasks_dir = lambda: TMP
ai_tasks._tasks = None
ai_tasks._current.clear()
ai_tasks._waiting[0] = None
for name in os.listdir(TMP):
os.remove(os.path.join(TMP, name))
CHOICES = {"type": "choices", "id": "pick", "question": "选哪张?", "options": [{"id": "a", "label": "左边"}, {"id": "b", "label": "右边"}]}
class BlockTests(unittest.TestCase):
def test_every_kind_of_block_and_what_is_left_out(self):
with mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT):
blocks = ai_panels.clean_blocks([
{"type": "heading", "text": " 两张对比 "},
{"type": "text", "text": "<b>不是网页</b>", "tone": "loud"},
{"type": "images", "items": [{"image": "a.png", "caption": "左"}, "b.png"], "layout": "compare"},
{"type": "table", "columns": ["项", "值"], "rows": [["步数", 40], ["CFG", 7.5]]},
{"type": "list", "items": ["一", "二"], "ordered": True},
{"type": "params", "items": [{"label": "种子", "value": 42}]},
CHOICES,
{"type": "input", "question": "还想改什么?"},
{"type": "progress", "label": "放大", "value": 150, "max": 100},
{"type": "divider", "extra": "ignored"},
])
self.assertEqual(blocks[0]["text"], "两张对比")
self.assertEqual(blocks[1], {"type": "text", "text": "<b>不是网页</b>", "tone": "normal"}, "text stays text")
self.assertEqual(blocks[2]["items"][1], {"type": "image", "filename": "b.png", "subfolder": "", "folder": "output", "caption": ""})
self.assertEqual(blocks[3]["rows"], [["步数", "40"], ["CFG", "7.5"]])
self.assertEqual(blocks[6]["options"][1], {"id": "b", "label": "右边", "detail": ""})
self.assertEqual(blocks[7]["id"], "q8", "a question without an id gets one")
self.assertEqual(blocks[8]["value"], 100)
self.assertEqual(blocks[9], {"type": "divider"})
def test_refused(self):
bad = [
[],
[{"type": "html", "html": "<script>"}],
[{"type": "text", "text": " "}],
[{"type": "images", "items": ["../secret.png"]}],
[{"type": "images", "items": [{"image": "a.png"}] * 13}],
[{"type": "choices", "options": ["only one"]}],
[{"type": "choices", "options": [{"id": "x", "label": "1"}, {"id": "x", "label": "2"}]}],
[CHOICES, CHOICES],
[{"type": "models", "items": ["loras:0:../../x.safetensors"]}],
[{"type": "progress", "value": "half"}],
[{"type": "heading", "text": {"a": 1}}],
[{"type": "text", "text": "x"}] * 31,
]
with mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT):
for blocks in bad:
with self.assertRaises(ai_panels.PanelError, msg=str(blocks)[:80]):
ai_panels.clean_blocks(blocks)
class StageTests(unittest.TestCase):
"""Marks on pictures, the slider, picks and the diagram: data only, laid out by the page."""
def setUp(self):
isolate()
def test_marks_slider_picks_and_diagram(self):
with mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT):
blocks = ai_panels.clean_blocks([
{"type": "images", "items": [{"image": "a.png", "marks": [
{"x": 0.6, "y": 0.5, "label": "手指修好了", "color": "green"},
{"x": 0.2, "y": 0.2, "w": 0.3, "h": 0.2, "label": "光影", "color": "pink"}]}]},
{"type": "slider", "before": {"image": "a.png"}, "after": "b.png", "before_label": "改之前", "after_label": "改之后"},
{"type": "picks", "id": "lora", "question": "哪个权重?", "items": [
{"id": "w06", "image": "a.png", "label": "0.6", "reason": "偏淡"},
{"image": "b.png", "label": "0.8", "reason": "推荐", "recommended": True}]},
{"type": "diagram", "direction": "sideways", "nodes": [
{"id": "m", "label": "模型", "node_id": 4}, {"id": "s", "label": "采样", "color": "blue"}],
"edges": [{"from": "m", "to": "s", "label": "MODEL"}]},
])
marks = blocks[0]["items"][0]["marks"]
self.assertEqual(marks[0], {"x": 0.6, "y": 0.5, "label": "手指修好了", "color": "green", "r": 0.06})
self.assertEqual(marks[1], {"x": 0.2, "y": 0.2, "label": "光影", "color": "red", "w": 0.3, "h": 0.2})
self.assertEqual((blocks[1]["before"]["filename"], blocks[1]["after"]["filename"]), ("a.png", "b.png"))
self.assertEqual([item["id"] for item in blocks[2]["items"]], ["w06", "p2"])
self.assertTrue(blocks[2]["items"][1]["recommended"])
self.assertEqual(blocks[2]["id"], "lora")
self.assertEqual(blocks[3]["direction"], "right")
self.assertEqual(blocks[3]["nodes"][0]["node_id"], "4")
self.assertEqual(blocks[3]["edges"], [{"from": "m", "to": "s", "label": "MODEL"}])
def test_stage_blocks_refused(self):
bad = [
[{"type": "images", "items": [{"image": "a.png", "marks": [{"x": 1.5, "y": 0}]}]}],
[{"type": "images", "items": [{"image": "a.png", "marks": [{"x": 0.1, "y": 0.1}] * 9}]}],
[{"type": "slider", "before": "a.png"}],
[{"type": "picks", "items": [{"image": "a.png"}]}],
[{"type": "picks", "items": [{"id": "x", "image": "a.png"}, {"id": "x", "image": "b.png"}]}],
[{"type": "diagram", "nodes": []}],
[{"type": "diagram", "nodes": [{"id": "a", "label": "A"}], "edges": [{"from": "a", "to": "zz"}]}],
[{"type": "svg", "svg": "<svg onload=alert(1)>"}],
]
with mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT):
for blocks in bad:
with self.assertRaises(ai_panels.PanelError, msg=str(blocks)[:80]):
ai_panels.clean_blocks(blocks)
def test_a_pick_is_an_answer(self):
with mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT):
shown = ai_panels.show("A", "选一张", [{"type": "picks", "id": "pick", "question": "哪张?",
"items": [{"id": "l", "image": "a.png", "label": "左"}, {"id": "r", "image": "b.png"}]}])
task_id = ai_tasks.list_tasks()[0]["id"]
step = ai_tasks.get_task(task_id)["steps"][0]
self.assertEqual(ai_tasks.list_tasks()[0]["asks"], 1)
with self.assertRaises(ai_panels.PanelError):
ai_panels.answer(task_id, step["id"], "pick", ["x"])
ai_panels.answer(task_id, step["id"], "pick", ["r"])
got = ai_panels.get_answers("A", shown["id"])["answers"]
self.assertEqual(got, [{"panel": shown["id"], "question": "pick", "asked": "哪张?", "choice": ["r"], "labels": ["r"]}])
class AnswerTests(unittest.TestCase):
def setUp(self):
isolate()
def test_show_answer_and_read_once(self):
shown = mcp_task_tools.show_panel({"client": "Claude"}, "挑一张", [{"type": "text", "text": "看看"}, CHOICES,
{"type": "input", "id": "more", "question": "还要改吗"}])
self.assertEqual(shown["questions"], ["pick", "more"])
task = ai_tasks.list_tasks()[0]
self.assertEqual((task["asks"], task["kinds"]), (2, {"panel": 1}))
step = ai_tasks.get_task(task["id"])["steps"][0]
self.assertEqual(step["panel"]["title"], "挑一张")
with self.assertRaises(ai_panels.PanelError):
ai_panels.answer(task["id"], step["id"], shown["blocks"][0]["id"], text="not a question")
self.assertEqual(ai_panels.get_answers("Claude", shown["panel"])["answers"], [])
with self.assertRaises(ai_panels.PanelError):
ai_panels.answer(task["id"], step["id"], "pick", ["c"])
with self.assertRaises(ai_panels.PanelError):
ai_panels.answer(task["id"], step["id"], "pick", ["a", "b"]) # one pick only
ai_panels.answer(task["id"], step["id"], "pick", ["b"])
with self.assertRaises(ai_panels.PanelError):
ai_panels.answer(task["id"], step["id"], "pick", ["a"]) # already answered
self.assertEqual(ai_tasks.list_tasks()[0]["asks"], 1)
got, pictures = mcp_task_tools.get_answers({"client": "Claude"}, shown["panel"])
self.assertEqual(pictures, [], "no marked pictures here")
self.assertEqual(got["answers"], [{"panel": shown["panel"], "question": "pick", "asked": "选哪张?", "choice": ["b"], "labels": ["右边"]}])
self.assertEqual(got["still_waiting"], [{"panel": shown["panel"], "question": "more"}])
self.assertEqual(ai_panels.get_answers("Claude")["answers"], [], "each answer is given once")
ai_panels.answer(task["id"], step["id"], "more", text=" 背景再亮一点 ")
self.assertEqual(ai_panels.get_answers("Claude")["answers"][0]["text"], "背景再亮一点")
self.assertEqual(ai_panels.get_answers("Cursor")["answers"], [], "another app's panels are not its")
def test_a_skipped_question_and_when_the_ai_got_each_answer(self):
shown = ai_panels.show("A", "细节", [CHOICES, {"type": "input", "id": "more", "question": "补充细节(可选)"}])
task = ai_tasks.list_tasks()[0]["id"]
step = ai_tasks.get_task(task)["steps"][0]["id"]
ai_panels.answer(task, step, "pick", ["a"])
ai_panels.answer(task, step, "more", skip=True)
self.assertEqual(ai_tasks.list_tasks()[0]["asks"], 0, "a skipped question waits no more")
got = ai_panels.get_answers("A", shown["id"])
self.assertEqual(got["answers"][1], {"panel": shown["id"], "question": "more", "asked": "补充细节(可选)", "skipped": True})
answers = ai_tasks.get_task(task)["steps"][0]["answers"]
self.assertTrue(all(given["delivered"] and given["delivered_at"] >= given["at"] for given in answers.values()),
"the page shows when the AI got each answer")
self.assertTrue(ai_panels.leave_message(task, "继续")["id"])
ai_panels.get_answers("A")
self.assertTrue(ai_tasks.get_task(task)["steps"][-1]["delivered_at"])
def test_waiting_for_an_answer(self):
shown = ai_panels.show("A", "等你", [CHOICES])
task = ai_tasks.list_tasks()[0]
step = ai_tasks.get_task(task["id"])["steps"][0]
threading.Timer(0.3, lambda: ai_panels.answer(task["id"], step["id"], "pick", "a")).start()
started = time.monotonic()
got = ai_panels.get_answers("A", shown["id"], wait_seconds=5)
self.assertLess(time.monotonic() - started, 3)
self.assertEqual(got["answers"][0]["choice"], ["a"])
started = time.monotonic()
self.assertEqual(ai_panels.get_answers("A", shown["id"], wait_seconds=5), {"answers": [], "still_waiting": []},
"nothing left to wait for: no wait")
self.assertLess(time.monotonic() - started, 1)
def test_messages_from_the_user(self):
ai_tasks.record("Claude", "describe_canvas", {}, {"content": [], "structuredContent": {}, "isError": False}, 1.0, True)
task_id = ai_tasks.list_tasks()[0]["id"]
with self.assertRaises(ai_panels.PanelError):
ai_panels.leave_message(task_id, " ")
with self.assertRaises(ai_panels.PanelError):
ai_panels.leave_message("nope", "hi")
threading.Timer(0.3, lambda: ai_panels.leave_message(task_id, " 背景再亮一点 ")).start()
started = time.monotonic()
got, _ = mcp_task_tools.get_answers({"client": "Claude"}, None, 5)
self.assertLess(time.monotonic() - started, 3, "a message wakes an AI waiting with no panel")
self.assertEqual([m["text"] for m in got["messages"]], ["背景再亮一点"])
self.assertEqual(ai_panels.get_answers("Claude"), {"answers": [], "still_waiting": []}, "each message once")
step = ai_tasks.get_task(task_id)["steps"][-1]
self.assertEqual((step["kind"], step["delivered"]), ("user", True))
ai_panels.leave_message(task_id, "先别跑了")
self.assertEqual(ai_panels.get_answers("Cursor"), {"answers": [], "still_waiting": []}, "another app does not get it")
# Asking about one panel brings that panel's task's messages too.
shown = ai_panels.show("Claude", "问", [CHOICES])
got = ai_panels.get_answers("Claude", shown["id"])
self.assertEqual([m["text"] for m in got["messages"]], ["先别跑了"])
def test_update_and_close(self):
shown = ai_panels.show("A", "进度", [{"type": "progress", "value": 1, "max": 4}, CHOICES])
ai_panels.update(shown["id"], blocks=[{"type": "progress", "value": 3, "max": 4}, CHOICES])
task_id = ai_tasks.list_tasks()[0]["id"]
step = ai_tasks.get_task(task_id)["steps"][0]
self.assertEqual(step["panel"]["blocks"][0]["value"], 3)
self.assertGreaterEqual(step["panel"]["changed"], step["at"]) # when its numbers were written
ai_panels.update(shown["id"], closed=True)
with self.assertRaises(ai_panels.PanelError):
ai_panels.answer(task_id, step["id"], "pick", "a")
self.assertEqual(ai_tasks.list_tasks()[0]["asks"], 0)
with self.assertRaises(mcp_task_tools.ToolError):
mcp_task_tools.update_panel({}, "nope", title="x")
with self.assertRaises(mcp_task_tools.ToolError):
mcp_task_tools.get_answers({}, "nope")
with self.assertRaises(mcp_task_tools.ToolError):
mcp_task_tools.show_panel({}, "坏的", [{"type": "html"}])
def test_later_results_append_even_when_client_prepends_them(self):
plan = [{"type": "heading", "text": "搭配方案"},
{"type": "text", "text": "先选模型"},
{"type": "diagram", "nodes": [{"id": "sampler", "label": "采样器"}], "edges": []}]
result = [{"type": "heading", "text": "成图结果"}, {"type": "images", "items": ["a.png"]}]
shown = mcp_task_tools.show_panel({"client": "A"}, "过程", plan)
with mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT):
changed = mcp_task_tools.update_panel({}, shown["panel"], blocks=result + plan)
again = mcp_task_tools.update_panel({}, shown["panel"], blocks=result + plan)
task_id = ai_tasks.list_tasks()[0]["id"]
task = ai_tasks.get_task(task_id)
blocks = task["steps"][0]["panel"]["blocks"]
self.assertEqual([b["type"] for b in blocks], ["heading", "text", "diagram", "heading", "images"])
self.assertEqual([blocks[0]["text"], blocks[3]["text"]], ["搭配方案", "成图结果"])
self.assertEqual(changed["blocks"][:3], shown["blocks"], "existing handles and order stay")
self.assertEqual(again, changed, "retrying does not duplicate content")
ai_tasks._tasks = None
self.assertEqual(ai_tasks.get_task(task_id)["steps"][0]["panel"]["blocks"], blocks,
"chronological order survives reload")
def test_partial_update_keeps_earlier_content_answers_and_progress_position(self):
shown = ai_panels.show("A", "过程", [CHOICES, {"type": "progress", "value": 1, "max": 4}])
task_id = ai_tasks.list_tasks()[0]["id"]
step = ai_tasks.get_task(task_id)["steps"][0]
ai_panels.answer(task_id, step["id"], "pick", "a")
progress_id = shown["blocks"][1]["id"]
changed = ai_panels.update(shown["id"], blocks=[{"type": "text", "text": "结果好了"},
{"type": "progress", "id": progress_id, "value": 4, "max": 4}])
self.assertEqual([b["type"] for b in changed["blocks"]], ["choices", "progress", "text"])
self.assertEqual(changed["blocks"][1]["value"], 4)
self.assertEqual(ai_panels.get_answers("A")["answers"][0]["choice"], ["a"])
self.assertEqual(ai_panels.get_answers("A")["answers"], [])
def test_invalid_update_keeps_panel_and_disk_unchanged(self):
shown = ai_panels.show("A", "保留标题", [{"type": "text", "text": "保留内容"}])
task_id = ai_tasks.list_tasks()[0]["id"]
before = copy.deepcopy(ai_tasks.get_task(task_id))
bad = [
[{"type": "html"}],
[{"type": "progress", "id": shown["blocks"][0]["id"], "value": 1}],
[{"type": "text", "text": str(i)} for i in range(ai_panels.MAX_BLOCKS)],
[{"type": "text", "id": "same", "text": "a"}, {"type": "divider", "id": "same"}],
]
for blocks in bad:
with self.subTest(blocks=blocks), self.assertRaises(ai_panels.PanelError):
ai_panels.update(shown["id"], title="不能留下的新标题", blocks=blocks)
self.assertEqual(ai_tasks.get_task(task_id), before)
with open(ai_tasks._path(task_id), encoding="utf-8") as source:
self.assertEqual(json.load(source), before)
def test_anonymous_followup_question_does_not_replace_answered_question(self):
original = {"type": "input", "question": "先画什么?"}
followup = {"type": "input", "question": "结果要改哪里?"}
shown = ai_panels.show("A", "问答", [original])
task_id = ai_tasks.list_tasks()[0]["id"]
step = ai_tasks.get_task(task_id)["steps"][0]
first_id = shown["blocks"][0]["id"]
ai_panels.answer(task_id, step["id"], first_id, text="海边")
updated = ai_panels.update(shown["id"], blocks=[followup])
self.assertEqual([b["question"] for b in updated["blocks"]], [original["question"], followup["question"]])
self.assertNotEqual(updated["blocks"][1]["id"], first_id)
again = ai_panels.update(shown["id"], blocks=[followup])
self.assertEqual(again["blocks"], updated["blocks"], "an unchanged retry keeps the question id")
self.assertEqual(ai_tasks.list_tasks()[0]["asks"], 1)
self.assertEqual(ai_panels.get_answers("A")["answers"][0]["text"], "海边")
def test_legacy_blocks_keep_order_when_updated(self):
plan = [{"type": "text", "text": "之前的方案"}, {"type": "progress", "value": 1}]
shown = ai_panels.show("A", "旧面板", plan)
for block in shown["blocks"]:
block.pop("id") # old records did not give static blocks handles
changed = ai_panels.update(shown["id"], blocks=[{"type": "text", "text": "后来的结果"},
plan[0], {"type": "progress", "value": 100}])
self.assertEqual([b["type"] for b in changed["blocks"]], ["text", "progress", "text"])
self.assertEqual(changed["blocks"][1]["value"], 100)
self.assertEqual(len({b["id"] for b in changed["blocks"]}), 3)
def test_attach_pending_delivers_once_and_skips_errors(self):
shown = ai_panels.show("Claude", "挑一张", [CHOICES])
task_id = ai_tasks.list_tasks()[0]["id"]
step = ai_tasks.get_task(task_id)["steps"][0]
ai_panels.answer(task_id, step["id"], "pick", "a")
ai_panels.leave_message(task_id, "再来一张特写")
notified = []
with mock.patch.object(ai_tasks, "_notify", side_effect=notified.append):
failed = {"content": [{"type": "text", "text": "err"}], "structuredContent": {}, "isError": True}
self.assertEqual(ai_panels.attach_pending(copy.deepcopy(failed), "Claude", task_id=task_id), failed)
self.assertEqual(notified, [], "failed tool calls must not consume pending messages or answers")
ok_res = {"content": [{"type": "text", "text": json.dumps({"total": 1})}], "structuredContent": {"total": 1}, "isError": False}
delivered = ai_panels.attach_pending(ok_res, "Claude", task_id=task_id)
self.assertIn(task_id, notified)
self.assertEqual([m["text"] for m in delivered["structuredContent"]["user_messages"]], ["再来一张特写"])
self.assertEqual(delivered["structuredContent"]["user_answers"][0]["choice"], ["a"])
again = {"content": [{"type": "text", "text": "{}"}], "structuredContent": {}, "isError": False}
self.assertNotIn("user_messages", ai_panels.attach_pending(again, "Claude", task_id=task_id)["structuredContent"])
def test_explicit_task_never_consumes_another_sessions_input(self):
first = ai_tasks.begin_task("Claude", "first", session="one")["id"]
second = ai_tasks.begin_task("Claude", "second", session="two")["id"]
ai_panels.leave_message(first, "first only")
ai_panels.leave_message(second, "second only")
result = {"content": [], "structuredContent": {}}
ai_panels.attach_pending(result, "Claude", task_id=first)
self.assertEqual([m["text"] for m in result["structuredContent"]["user_messages"]], ["first only"])
self.assertFalse(ai_tasks.get_task(second)["steps"][-1]["delivered"])
self.assertEqual(ai_panels.get_answers("Claude", task_id="missing").get("messages", []), [])
self.assertEqual(ai_panels.get_answers("Claude", task_id=second)["messages"][0]["text"], "second only")
def test_get_answers_tool_resolves_its_session_before_collecting(self):
first = ai_tasks.begin_task("Claude", "first", session="one")["id"]
second = ai_tasks.begin_task("Claude", "second", session="two")["id"]
ai_panels.leave_message(first, "one")
ai_panels.leave_message(second, "two")
self.assertNotIn("messages", mcp_task_tools.get_answers({"client": "Claude", "session": "unknown"})[0])
answer, _ = mcp_task_tools.get_answers({"client": "Claude", "session": "two", "task": ""})
self.assertEqual([m["text"] for m in answer["messages"]], ["two"])
self.assertFalse(ai_tasks.get_task(first)["steps"][-1]["delivered"])
answer, _ = mcp_task_tools.get_answers({"client": "Claude", "session": "", "task": ""})
self.assertEqual([m["text"] for m in answer["messages"]], ["one"], "legacy callers without routing still read their own input")
class AskUserTests(unittest.TestCase):
"""ask_user: the AI's question shows in the workbench and the answer comes back to the same call."""
def setUp(self):
isolate()
self.ctx = {"client": "Gemini", "session": "s1"}
def answer_later(self, delay=0, **given):
def go():
time.sleep(delay)
for _ in range(100):
tasks = ai_tasks.list_tasks()
task = ai_tasks.get_task(tasks[0]["id"]) if tasks else None
step = next((item for item in (task or {}).get("steps", []) if item.get("kind") == "panel"), None)
if step:
ai_panels.answer(task["id"], step["id"], "answer", **given)
return
time.sleep(0.02)
threading.Thread(target=go, daemon=True).start()
def test_a_pick_comes_back_and_the_recommended_one_is_marked(self):
options = [{"label": "光与暗双生少女", "detail": "高难度", "recommended": True}, {"label": "猫耳娘咖啡馆"}]
self.answer_later(choice=["o2"])
got = mcp_task_tools.ask_user(self.ctx, "先测哪种双人题材?", options, wait_seconds=5)
self.assertEqual((got["answered"], got["labels"]), (True, ["猫耳娘咖啡馆"]))
step = ai_tasks.get_task(ai_tasks.list_tasks()[0]["id"])["steps"][0]
block = step["panel"]["blocks"][0]
self.assertEqual((step["panel"]["title"], block["options"][0].get("recommended"), block["other"]), ("先测哪种双人题材?", True, True))
self.assertEqual(step["tool"], "ask_user", "the timeline says the AI asked")
def test_a_skipped_question_says_so(self):
self.answer_later(skip=True)
got = mcp_task_tools.ask_user(self.ctx, "要不要加光效?", ["要", "不要"], wait_seconds=5)
self.assertEqual((got["answered"], got["skipped"]), (True, True))
self.assertIn("left this to you", got["note"])
def test_their_own_answer_when_no_option_fits(self):
self.answer_later(text="冰与火,但要室内")
got = mcp_task_tools.ask_user(self.ctx, "选哪个?", ["甲", "乙"], wait_seconds=5)
self.assertEqual((got["text"], got["answered"]), ("冰与火,但要室内", True))
self.assertIn("own answer", got["note"])
def test_it_waits_a_while_even_when_told_not_to(self):
self.answer_later(delay=0.3, choice=["o1"])
with mock.patch.object(mcp_task_tools, "ASK_WAIT_MIN", 5):
got = mcp_task_tools.ask_user(self.ctx, "修不修手?", ["修", "不修"], wait_seconds=0)
self.assertEqual((got["answered"], got["labels"]), (True, ["修"]), "the user answering in Anomalous needs no word in the chat")
def test_unanswered_it_waits_on_for_the_users_time_and_while_they_type(self):
with mock.patch.object(mcp_task_tools, "ASK_WAIT_MIN", 0):
got = mcp_task_tools.ask_user(self.ctx, "手指怎么修?", ["只重画手指", "整张重画"], wait_seconds=0)
self.assertEqual((got["answered"], got["keep_waiting"]), (False, True))
self.assertGreater(got["seconds_left"], 170, "three minutes unless the user set otherwise")
task_id = ai_tasks.list_tasks()[0]["id"]
asked = ai_tasks.get_task(task_id)["steps"][0]["at"]
late = asked + ai_gates.answer_wait() + 5
self.assertFalse(ai_answer_wait.wait_on(got["panel"], now=late)["keep_waiting"])
self.assertIn("end your turn", ai_answer_wait.wait_on(got["panel"], now=late)["note"])
self.assertTrue(ai_answer_wait.typing(task_id, now=late - 10))
typing = ai_answer_wait.wait_on(got["panel"], now=late)
self.assertEqual((typing["keep_waiting"], typing["typing"]), (True, True), "typing keeps the AI waiting")
answer, _ = mcp_task_tools.get_answers(self.ctx, panel=got["panel"])
self.assertTrue(answer["keep_waiting"], "get_answers says it too")
self.assertFalse(ai_answer_wait.typing("no_such_task"))
def test_on_the_users_word_time_running_out_takes_the_recommended_option(self):
from api import ai_decisions
options = [{"label": "只重画手指", "recommended": True}, {"label": "整张重画"}]
with mock.patch.object(mcp_task_tools, "ASK_WAIT_MIN", 0):
got = mcp_task_tools.ask_user(self.ctx, "手指怎么修?", options, wait_seconds=0)
plain = mcp_task_tools.ask_user(self.ctx, "几张?", ["一", "二"], wait_seconds=0)
task_id = ai_tasks.list_tasks()[0]["id"]
late = time.time() + ai_gates.answer_wait() + 5
self.assertIn("end your turn", ai_answer_wait.wait_on(got["panel"], now=late)["note"], "by default nobody answers for the user")
with mock.patch.object(ai_gates, "no_answer", return_value="recommended"):
told = ai_answer_wait.wait_on(got["panel"], now=late)
self.assertIn("end your turn", ai_answer_wait.wait_on(plain["panel"], now=late)["note"], "no recommended option: left open")
self.assertEqual((told["keep_waiting"], told["took_recommended"]), (False, True))
answer, _ = mcp_task_tools.get_answers(self.ctx, panel=got["panel"])
item = answer["answers"][0]
self.assertEqual((item["labels"], item["recommended_on_timeout"]), (["只重画手指"], True))
self.assertIn("in time", ai_decisions.lines(ai_tasks.get_task(task_id))[-1], "the decisions say how it was answered")
def test_the_recommended_option_taken_on_time_comes_in_the_same_call(self):
options = [{"label": "只重画手指", "recommended": True}, {"label": "整张重画"}]
with mock.patch.object(mcp_task_tools, "ASK_WAIT_MIN", 0), mock.patch.object(ai_gates, "answer_wait", return_value=0):
asked = mcp_task_tools.ask_user(self.ctx, "手指怎么修?", options, wait_seconds=0)
self.assertFalse(asked["answered"], "the default: the AI waits for the user")
with mock.patch.object(ai_gates, "no_answer", return_value="recommended"):
answer, _ = mcp_task_tools.get_answers(self.ctx, panel=asked["panel"])
again = mcp_task_tools.ask_user(self.ctx, "还要放大吗?", options, wait_seconds=0)
self.assertEqual(answer["answers"][0]["labels"], ["只重画手指"], "get_answers hands it over at once")
self.assertIn("took it for them", answer["note"])
self.assertEqual((again["answered"], again["labels"], again["recommended_on_timeout"]), (True, ["只重画手指"], True))
def test_a_note_or_panel_alone_says_to_ask_before_stopping(self):
self.assertIn("ask_user", mcp_task_tools.report(self.ctx, "换好了 LoRA")["next"])
self.assertIn("ask it with ask_user", mcp_task_tools.report(self.ctx, "要不要再亮一点?")["next"])
self.assertNotIn("next", mcp_task_tools.report(self.ctx, "做完了", done=True))
shown = mcp_task_tools.show_panel(self.ctx, "方案", [{"type": "text", "text": "先修手"}])
self.assertIn("ask_user", shown["next"])
def test_no_options_is_a_line_and_no_answer_yet_says_how_to_keep_waiting(self):
with mock.patch.object(mcp_task_tools, "ASK_WAIT_MIN", 0):
got = mcp_task_tools.ask_user(self.ctx, "想要几张?", wait_seconds=0)
self.assertFalse(got["answered"])
self.assertIn("get_answers", got["note"])
step = ai_tasks.get_task(ai_tasks.list_tasks()[0]["id"])["steps"][0]
self.assertEqual(step["panel"]["blocks"][0]["type"], "input")
def test_other_text_only_where_the_question_allows_it(self):
panel = ai_panels.show("Gemini", "q", [{"type": "choices", "id": "answer", "options": ["a", "b"]}])
task = ai_tasks.list_tasks()[0]["id"]
step = ai_tasks.get_task(task)["steps"][0]["id"]
with self.assertRaises(ai_panels.PanelError):
ai_panels.answer(task, step, "answer", text="随便")
self.assertTrue(panel["id"])
if __name__ == "__main__":
unittest.main()
+120
View File
@@ -0,0 +1,120 @@
"""The area to repaint, agreed between the AI and the user (api/ai_repaint.py): the AI's area beside
the user's marks on a panel, and once the user picks one, the picture in the input folder with that
area as its alpha. Temporary folders only."""
import os
import sys
import tempfile
import time
import unittest
from pathlib import Path
from unittest import mock
PLUGIN = Path(__file__).resolve().parents[1]
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1]), str(PLUGIN / "tests")]
from PIL import Image # noqa: E402
from api import ai_marks, ai_panels, ai_repaint, ai_tasks, mcp_task_tools # noqa: E402
from api.mcp_tools import ToolError # noqa: E402
from test_ai_marks import MARKS, png # noqa: E402
from test_ai_tasks import isolate, ok # noqa: E402
from test_mcp import rpc # noqa: E402 (its module keeps tasks and gates in temporary folders; setUp's isolate comes after)
OUTPUT = tempfile.mkdtemp(prefix="amb_repaint_output_")
INPUT = tempfile.mkdtemp(prefix="amb_repaint_input_")
CTX = {"client": "Claude"}
AREA = [{"shape": "box", "x": 0.25, "y": 0.25, "w": 0.25, "h": 0.25}]
class RepaintTests(unittest.TestCase):
def setUp(self):
isolate()
Image.new("RGB", (200, 100), (20, 120, 200)).save(os.path.join(OUTPUT, "girl.png"))
for name in os.listdir(INPUT):
os.remove(os.path.join(INPUT, name))
for patch in (mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT),
mock.patch.object(ai_tasks.folder_paths, "get_input_directory", return_value=INPUT),
mock.patch.object(mcp_task_tools, "ASK_WAIT_MIN", 0)):
patch.start()
self.addCleanup(patch.stop)
ai_tasks.record("Claude", "describe_canvas", {}, ok({}), time.time(), True)
self.task = ai_tasks.list_tasks()[0]["id"]
self.marks = ai_marks.save(self.task, "girl.png", png((200, 100)), MARKS, "手指怪怪的")
def propose(self, **more):
return mcp_task_tools.propose_repaint_area(CTX, "手指重画的范围", self.marks["id"], wait_seconds=0, **more)
def panel_step(self, panel_id):
return ai_panels._find(panel_id)[1]
def pick(self, panel_id, choice=None, text=None, skip=False):
step = self.panel_step(panel_id)
extra = ai_repaint.prepare(self.task, step["id"], choice, text, skip)
ai_panels.answer(self.task, step["id"], "answer", choice, text, skip=skip, extra=extra)
return extra
def test_both_areas_side_by_side_with_three_answers(self):
got = self.propose(areas=AREA, grow=6, feather=4, note="往手腕多圈一点")
self.assertEqual((got["answered"], got["keep_waiting"]), (False, True))
step = self.panel_step(got["panel"])
self.assertEqual(step["tool"], "propose_repaint_area")
area, note, choices = step["panel"]["blocks"]
self.assertEqual((area["type"], area["grow"], area["feather"], note["text"]), ("area", 6, 4, "往手腕多圈一点"))
for side in ("yours", "ai"):
self.assertTrue(ai_marks.path_of(self.task, area[side]), "drawn into the task's marks folder")
self.assertEqual([option["id"] for option in choices["options"]], ["ai", "mine", "again"])
self.assertEqual(choices["options"][0]["label_key"], "aiRepaintUseAi")
self.assertEqual(os.listdir(INPUT), [], "nothing in the input folder before the user picks")
def test_the_ais_area_becomes_the_picture_to_repaint(self):
got = self.propose(areas=AREA, grow=6, feather=0)
extra = self.pick(got["panel"], ["ai"])
with Image.open(os.path.join(INPUT, extra["inpaint_image"])) as made:
alpha = made.getchannel("A")
self.assertEqual(made.size, (200, 100))
self.assertEqual(alpha.getpixel((62, 31)), 0, "inside: repaint (MASK = 1 - alpha)")
self.assertEqual(alpha.getpixel((47, 22)), 0, "grown by 6 px around the box")
self.assertEqual(alpha.getpixel((150, 80)), 255, "outside stays")
answer, _ = mcp_task_tools.get_answers(CTX, panel=got["panel"])
item = answer["answers"][0]
self.assertEqual((item["inpaint_image"], item["note"]), (extra["inpaint_image"], ai_panels.INPAINT_NOTE))
def test_their_own_marks_your_call_and_marking_again(self):
mine = self.pick(self.propose(areas=AREA)["panel"], ["mine"])
with Image.open(os.path.join(INPUT, mine["inpaint_image"])) as made:
self.assertEqual(made.getchannel("A").getpixel((60, 20)), 0, "their box (20-40%, 10-30%) as they drew it")
self.assertIn("inpaint_image", self.pick(self.propose(areas=AREA)["panel"], skip=True), "your call takes the AI's")
self.assertEqual(self.pick(self.propose()["panel"], ["again"]), {"note": ai_repaint.AGAIN_NOTE})
self.assertEqual(self.pick(self.propose()["panel"], text="再往左一点"), {"note": ai_repaint.WORDS_NOTE})
def test_a_refused_answer_leaves_no_picture(self):
panel = self.propose(areas=AREA)["panel"]
self.pick(panel, ["ai"])
extra = ai_repaint.prepare(self.task, self.panel_step(panel)["id"], ["mine"], None)
with self.assertRaises(ai_panels.PanelError):
ai_panels.answer(self.task, self.panel_step(panel)["id"], "answer", ["mine"], extra=extra)
ai_repaint.discard(extra)
self.assertEqual(len(os.listdir(INPUT)), 1, "only the first answer's picture stays")
def test_through_the_server_it_is_one_step_of_its_own(self):
result = rpc("tools/call", {"name": "propose_repaint_area", "arguments": {
"title": "范围", "marks": self.marks["id"], "wait_seconds": 0}}, "propose_repaint_area")["result"]
self.assertFalse(result["isError"], result)
steps = [step for step in ai_tasks.get_task(self.task)["steps"] if step["tool"] == "propose_repaint_area"]
self.assertEqual([step["kind"] for step in steps], ["panel"], "no second step without its panel")
self.assertTrue(ai_tasks.list_tasks())
def test_what_is_refused(self):
with self.assertRaises(ToolError):
mcp_task_tools.propose_repaint_area(CTX, "x", "nope", wait_seconds=0)
with self.assertRaises(ToolError):
self.propose(areas=[{"shape": "star", "x": 0, "y": 0, "w": 0.1, "h": 0.1}])
with self.assertRaises(ToolError):
self.propose(grow=999)
with self.assertRaises(ai_panels.PanelError):
ai_panels.clean_blocks([{"type": "area", "yours": "../x.png", "ai": "0123456789.png"}])
self.assertEqual(ai_repaint.prepare(self.task, self.marks["id"], ["ai"], None), {}, "not a repaint panel")
if __name__ == "__main__":
unittest.main()
+179
View File
@@ -0,0 +1,179 @@
"""A run's shape and what changed (ai_run_map.py), the task's own canvas tab (ai_task_canvas.py),
pictures handed to the AI (ai_pictures.py) and show_run_map."""
import io
import json
import os
import time
import unittest
from unittest import mock
from PIL import Image
from PIL.PngImagePlugin import PngInfo
from test_ai_tasks import OUTPUT, isolate, ok
from api import ai_panels, ai_pictures, ai_run_map, ai_task_canvas, ai_tasks, mcp_task_tools
def lora_workflow(steps=20, strength=0.65):
return {
"4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "sdxl/Kirazuri_v4.safetensors"}, "_meta": {"title": "Kirazuri"}},
"10": {"class_type": "LoraLoader", "inputs": {"model": ["4", 0], "clip": ["4", 1], "lora_name": "agefx.safetensors",
"strength_model": strength, "strength_clip": 1}},
"11": {"class_type": "Reroute", "inputs": {"": ["10", 0]}},
"6": {"class_type": "CLIPTextEncode", "inputs": {"clip": ["10", 1], "text": "irena, grey hair, witch hat"}},
"7": {"class_type": "CLIPTextEncode", "inputs": {"clip": ["10", 1], "text": "lowres"}},
"5": {"class_type": "EmptyLatentImage", "inputs": {"width": 1024, "height": 1536, "batch_size": 2}},
"3": {"class_type": "KSampler", "inputs": {"model": ["11", 0], "positive": ["6", 0], "negative": ["7", 0], "latent_image": ["5", 0],
"seed": 123, "steps": steps, "cfg": 5, "sampler_name": "euler", "scheduler": "normal", "denoise": 1}},
"8": {"class_type": "VAEDecode", "inputs": {"samples": ["3", 0], "vae": ["4", 2]}},
"9": {"class_type": "SaveImage", "inputs": {"images": ["8", 0], "filename_prefix": "irena"}},
}
class MapTests(unittest.TestCase):
def test_boxes_roles_facts_and_links_through_plumbing(self):
shape = ai_run_map.map_of(lora_workflow())
boxes = {box["id"]: box for box in shape["boxes"]}
self.assertNotIn("11", boxes, "a reroute is wired through, not shown")
self.assertEqual(boxes["4"]["role"], "model")
self.assertEqual(boxes["4"]["facts"]["file"], "Kirazuri_v4")
self.assertEqual((boxes["10"]["role"], boxes["10"]["facts"]["strength"]), ("lora", 0.65))
self.assertEqual(boxes["6"]["facts"], {"text": "irena, grey hair, witch hat", "polarity": "positive"})
self.assertEqual(boxes["7"]["facts"]["polarity"], "negative")
self.assertEqual(boxes["5"]["facts"], {"width": 1024, "height": 1536, "batch": 2})
self.assertEqual(boxes["3"]["facts"], {"steps": 20, "cfg": 5, "sampler_name": "euler", "scheduler": "normal", "denoise": 1})
self.assertIn({"from": "10", "to": "3", "type": "MODEL", "input": "model"}, [
{**link, "type": "MODEL"} for link in shape["links"] if link["from"] == "10" and link["to"] == "3"])
self.assertNotIn("3.seed", shape["params"], "the seed changes every run; it is no change to report")
def test_changes_since_the_run_before(self):
before = ai_run_map.map_of(lora_workflow())
after_workflow = lora_workflow(steps=30, strength=0.8)
after_workflow["12"] = {"class_type": "LoraLoader", "inputs": {"model": ["10", 0], "lora_name": "light.safetensors"}}
del after_workflow["9"]
changes = ai_run_map.changes(before, ai_run_map.map_of(after_workflow))
self.assertIn({"node": "12", "title": "LoraLoader", "change": "added"}, changes)
self.assertIn({"node": "9", "title": "SaveImage", "change": "removed"}, changes)
self.assertIn({"node": "3", "title": "KSampler", "key": "steps", "from": "20", "to": "30"}, changes)
self.assertIn({"node": "10", "title": "LoraLoader", "key": "strength_model", "from": "0.65", "to": "0.8"}, changes)
self.assertEqual(ai_run_map.changes(None, before), [])
def test_a_huge_workflow_keeps_what_makes_the_picture(self):
workflow = lora_workflow()
for index in range(40):
workflow[f"x{index}"] = {"class_type": "SomeHelper", "inputs": {"value": index}}
shape = ai_run_map.map_of(workflow)
self.assertEqual(len(shape["boxes"]), ai_run_map.MAX_BOXES)
self.assertTrue({"3", "4", "10", "6", "5"} <= {box["id"] for box in shape["boxes"]})
class TaskCanvasTests(unittest.TestCase):
def setUp(self):
isolate()
self.task = ai_tasks.begin_task("A", "Irena portrait")["id"]
def test_the_task_gets_its_tab_and_the_ai_is_told_once(self):
self.assertEqual(ai_task_canvas.default_mode(), "new")
self.assertIsNone(ai_task_canvas.spec("set_values"), "outside a tool call there is no task")
holder = ai_task_canvas.begin_call(self.task)
self.assertEqual(ai_task_canvas.spec("set_values"), {"mode": "new", "tab": None, "title": "Irena portrait"})
self.assertIsNone(ai_task_canvas.spec("open_model"), "not a canvas action")
ai_task_canvas.remember({"path": "workflows/Irena portrait 101500.json", "name": "Irena portrait 101500", "made": True})
self.assertEqual(ai_tasks.get_task(self.task)["canvas"]["tab"]["path"], "workflows/Irena portrait 101500.json")
told = ai_task_canvas.tell(ok({"done": True}), holder)
self.assertIn("Irena portrait 101500", told["structuredContent"]["canvas_tab"]["note"])
again = ai_task_canvas.begin_call(self.task)
self.assertEqual(ai_task_canvas.spec("add_node")["tab"]["name"], "Irena portrait 101500")
self.assertNotIn("canvas_tab", ai_task_canvas.tell(ok({}), again)["structuredContent"])
def test_the_user_chooses_per_task_and_by_default(self):
ai_task_canvas.set_mode(self.task, "current")
ai_task_canvas.begin_call(self.task)
self.assertEqual(ai_task_canvas.spec("set_values")["mode"], "current")
other = ai_tasks.begin_task("B", "Other", new_task=True)["id"]
ai_task_canvas.set_default("current")
self.assertEqual(ai_task_canvas.mode_of(ai_tasks.get_task(other)), "current")
with self.assertRaises(ValueError):
ai_task_canvas.set_mode(self.task, "sideways")
with self.assertRaises(ValueError):
ai_task_canvas.set_default("sideways")
ai_task_canvas.set_default("new")
def picture(name, workflow=None):
info = PngInfo()
if workflow:
info.add_text("prompt", json.dumps(workflow))
Image.new("RGB", (64, 96), (40, 80, 120)).save(os.path.join(OUTPUT, name), pnginfo=info)
return name
class PictureTests(unittest.TestCase):
def setUp(self):
isolate()
self.task = ai_tasks.begin_task("A", "Look at these")["id"]
self.patch = mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT)
self.patch.start()
self.addCleanup(self.patch.stop)
def test_pictures_reach_the_ai_themselves_with_what_made_them(self):
ai_pictures.hand(self.task, [picture("one.png", lora_workflow())])
ai_pictures.hand(self.task, [picture("two.png")])
steps = ai_tasks.get_task(self.task)["steps"]
self.assertEqual(len(steps), 1, "pictures handed before the AI looked join one step")
self.assertEqual([item["filename"] for item in steps[0]["results"]], ["one.png", "two.png"])
result = ai_panels.attach_pending(ok({}), "A", task_id=self.task)
message = result["structuredContent"]["user_messages"][0]
self.assertEqual((message["kind"], message["images"]), ("pictures", ["one.png", "two.png"]))
made = message["made_with"][0]
self.assertEqual((made["models"], made["loras"], made["positive"], made["size"]),
(["Kirazuri_v4"], ["agefx (0.65)"], "irena, grey hair, witch hat", "1024x1536"))
self.assertEqual(message["made_with"][1], {"image": "two.png"})
self.assertEqual(sum(1 for item in result["content"] if item.get("type") == "image"), 2)
self.assertNotIn("_pictures", message)
def test_only_output_pictures_and_open_tasks(self):
for images in ([], ["../secret.png"], ["missing.png"], [picture("x.png")] * 9):
with self.assertRaises(ai_pictures.PicturesError):
ai_pictures.hand(self.task, images)
with self.assertRaises(ai_pictures.PicturesError):
ai_pictures.hand("nope", [picture("y.png")])
def test_a_dropped_picture_is_kept_in_the_inbox_never_over_a_file(self):
data = io.BytesIO()
Image.new("RGB", (8, 8), "red").save(data, "JPEG")
first = ai_pictures.save_upload(data.getvalue(), "../../草稿 参考.png", now=1000)
second = ai_pictures.save_upload(data.getvalue(), "../../草稿 参考.png", now=1000)
self.addCleanup(lambda: [os.remove(os.path.join(OUTPUT, rel)) for rel in (first, second)])
self.assertTrue(first.startswith("ai_inbox/草稿 参考_") and first.endswith(".jpg"), first) # its stem, its real type
self.assertNotEqual(first, second)
ai_pictures.hand(self.task, [first]) # an output picture like any other
for bad in (b"", b"not a picture", b"x" * (ai_pictures.MAX_UPLOAD + 1)):
with self.assertRaises(ai_pictures.PicturesError):
ai_pictures.save_upload(bad, "x.png")
class ShowRunMapTests(unittest.TestCase):
def setUp(self):
isolate()
self.task = ai_tasks.begin_task("A", "Run")["id"]
self.ctx = {"client": "A", "task": self.task}
def test_the_stage_shows_the_latest_run_that_has_its_shape(self):
from api.mcp_tools import ToolError
with self.assertRaises(ToolError):
mcp_task_tools.show_run_map(self.ctx, "结构")
step = ai_tasks.record("A", "run_workflow", {}, ok({}), time.time())
with ai_tasks._lock:
task = ai_tasks._load()[self.task]
task["steps"][-1].update(map=ai_run_map.map_of(lora_workflow()), changes=[])
ai_tasks._save(task)
answer = mcp_task_tools.show_run_map(self.ctx, "结构")
self.assertEqual(answer["run"], step["id"])
panel = ai_tasks.get_task(self.task)["steps"][-1]["panel"]
self.assertEqual(panel["blocks"][0]["type"], "run_map")
self.assertEqual(panel["blocks"][0]["step"], step["id"])
if __name__ == "__main__":
unittest.main()
+70
View File
@@ -0,0 +1,70 @@
"""External submissions: exact ownership, duplicate registration, and authoritative results."""
import time
import unittest
from types import SimpleNamespace
from unittest import mock
from test_ai_tasks import isolate, ok
from api import ai_run_tracking, ai_task_runs, ai_tasks
from api.mcp_tools import ToolError
class TrackingTests(unittest.TestCase):
def setUp(self):
isolate()
self.task = ai_tasks.begin_task("A", "Test")["id"]
self.ctx = {"client": "A", "task": self.task}
self.queue = mock.Mock()
self.queue.get_current_queue.return_value = ([], [(1, "one"), (2, "two")])
self.queue.get_history.side_effect = lambda prompt_id: {prompt_id: {
"status": {"completed": True}, "outputs": {"1": {"images": [{"filename": prompt_id + ".png", "type": "output"}]}}
}} if prompt_id in ("one", "two") else {}
def test_external_completion_and_repeated_attachment(self):
with mock.patch.object(ai_task_runs, "_queue", return_value=self.queue), \
mock.patch.object(ai_task_runs.threading, "Thread", side_effect=lambda target, **kw: SimpleNamespace(start=target)):
answer = ai_run_tracking.track_run(self.ctx, ["one"], wait_seconds=0)
self.assertEqual(answer["state"], "done")
self.assertEqual(answer["pictures"], ["one.png"])
ai_run_tracking.track_run(self.ctx, ["two"], wait_seconds=0)
again = ai_run_tracking.track_run(self.ctx, ["one"], wait_seconds=0)
self.assertEqual(again["prompt_ids"], ["one"], "retry answers this receipt, not the latest run")
self.assertEqual(len(ai_tasks.get_task(self.task)["steps"]), 2)
def test_unknown_or_already_owned_ids_are_not_attached(self):
with mock.patch.object(ai_task_runs, "_queue", return_value=self.queue), mock.patch.object(ai_task_runs, "watch"):
with self.assertRaises(ToolError):
ai_run_tracking.track_run(self.ctx, ["unknown"], wait_seconds=0)
self.assertEqual(ai_tasks.get_task(self.task)["steps"], [])
ai_run_tracking.track_run(self.ctx, ["one"], wait_seconds=0)
other = ai_tasks.begin_task("B", "Other")["id"]
with self.assertRaises(ToolError):
ai_run_tracking.track_run({"client": "B", "task": other}, ["one"], wait_seconds=0)
self.assertEqual(ai_tasks.get_task(other)["steps"], [])
def test_bad_wait_is_rejected_before_registration(self):
for value in ("later", float("nan"), True):
with self.assertRaises(ToolError):
ai_run_tracking.track_run(self.ctx, ["one"], wait_seconds=value)
self.assertEqual(ai_tasks.get_task(self.task)["steps"], [])
self.assertEqual([ai_run_tracking.checked_wait(value) for value in (-1, 30, 120)], [0, 30, 55], "too long: one call's longest")
def test_recent_contact_and_silence_do_not_claim_execution(self):
self.assertFalse(ai_tasks.task_summary(ai_tasks.get_task(self.task))["live"])
step = ai_tasks.record("A", "run_workflow", {}, ok({}), time.time())
self.assertTrue(ai_tasks.task_summary(ai_tasks.get_task(self.task))["live"])
ai_task_runs._finish_run(self.task, step["id"], ["one"], {"one": {"status": {"completed": True}}})
self.assertFalse(ai_tasks.task_summary(ai_tasks.get_task(self.task))["live"])
def test_receipt_wait_includes_actual_failure(self):
step = ai_tasks.record("A", "run_workflow", {}, ok({}), time.time())
ai_task_runs._finish_run(self.task, step["id"], ["one"], {"one": {"status": {"status_str": "error"}}})
result = ok({"prompt_ids": ["one"], "queued": 1})
ai_run_tracking.wait_for_receipt(result, "A", "", self.task, 0, step["id"])
self.assertEqual(result["structuredContent"]["state"], "error")
self.assertEqual(result["structuredContent"]["prompt_ids"], ["one"])
self.assertIn('"state": "error"', result["content"][0]["text"])
if __name__ == "__main__":
unittest.main()
+140
View File
@@ -0,0 +1,140 @@
"""A run's console, time left and size warnings (ai_run_watch.py), and stopping it (ai_run_control.py)."""
import time
import unittest
from datetime import datetime
from unittest import mock
from test_ai_tasks import isolate, ok
from api import ai_panels, ai_run_control, ai_run_watch, ai_task_runs, ai_tasks
def logs(started, *chunks):
at = datetime.fromtimestamp(started + 1).isoformat()
return [{"t": at, "m": chunk} for chunk in chunks]
class ConsoleTests(unittest.TestCase):
def test_the_bar_redrawn_in_place_is_its_latest_and_colours_go(self):
started = time.time()
entries = [{"t": datetime.fromtimestamp(started - 60).isoformat(), "m": "old line\n"}] + logs(
started, "\x1b[32m[INFO]\x1b[0m Requested to load RRDBNet\n",
"\r 0%| | 0/50 [00:00<?, ?it/s, Model Initializing ...]",
"\r 12%|#2 | 6/50 [00:03<00:22, 1.95it/s, Model Initializing ...]")
lines = ai_run_watch.console_since(started, entries)
self.assertEqual(lines[0], "[INFO] Requested to load RRDBNet")
self.assertTrue(lines[-1].endswith("Model Initializing ...]"))
self.assertNotIn("old line", lines)
bar = ai_run_watch.bar_of(lines)
self.assertEqual((bar["value"], bar["max"], bar["left"], bar["note"]), (6, 50, 22, "Model Initializing ..."))
self.assertAlmostEqual(bar["rate"], 0.51, places=2)
def test_no_bar_when_the_last_line_is_not_one(self):
self.assertIsNone(ai_run_watch.bar_of(["[INFO] got prompt"]))
self.assertIsNone(ai_run_watch.bar_of([]))
self.assertNotIn("left", ai_run_watch.bar_of(["0/50 [00:00<?, ?it/s]"]))
def test_pace_from_steps_seen(self):
pace = ai_run_watch.Pace()
self.assertIsNone(pace.update("k", 2, 20, now=100))
self.assertEqual(pace.update("k", 6, 20, now=108), {"rate": 2.0, "left": 28})
self.assertIsNone(pace.update("other", 1, 20, now=110), "another node starts over")
def prompt_upscaled_then_sampled():
return {
"1": {"class_type": "LoadImage", "inputs": {"image": "in.png"}},
"2": {"class_type": "UpscaleModelLoader", "inputs": {"model_name": "4x-UltraSharp.pth"}},
"3": {"class_type": "ImageUpscaleWithModel", "inputs": {"upscale_model": ["2", 0], "image": ["1", 0]}},
"4": {"class_type": "VAEEncode", "inputs": {"pixels": ["3", 0], "vae": ["9", 2]}},
"5": {"class_type": "KSampler", "inputs": {"model": ["9", 0], "latent_image": ["4", 0], "steps": 50, "denoise": 0.4},
"_meta": {"title": "Detail pass"}},
"6": {"class_type": "VAEDecode", "inputs": {"samples": ["5", 0], "vae": ["9", 2]}},
"9": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "a.safetensors"}},
}
class EstimateTests(unittest.TestCase):
def test_an_upscaled_picture_sampled_again_is_warned_with_the_plain_upscale(self):
found = ai_run_watch.estimate(prompt_upscaled_then_sampled(), 12288, image_size=lambda name: (1024, 1024))
self.assertEqual(found["samplers"][0], {"node": "5", "title": "Detail pass", "width": 4096, "height": 4096,
"batch": 1, "steps": 50, "upscaled": True})
kinds = [warning["kind"] for warning in found["warnings"]]
self.assertEqual(kinds, ["upscale_resample", "big_decode"])
self.assertIn("upscale model alone", ai_run_watch.warning_words(found["warnings"][0]))
def test_a_usual_picture_has_no_warning_and_sizes_follow_scale_nodes(self):
prompt = {
"1": {"class_type": "EmptyLatentImage", "inputs": {"width": 832, "height": 1216, "batch_size": 2}},
"2": {"class_type": "KSampler", "inputs": {"latent_image": ["1", 0], "steps": 28}},
"3": {"class_type": "LatentUpscaleBy", "inputs": {"samples": ["2", 0], "scale_by": 1.5}},
"4": {"class_type": "BasicScheduler", "inputs": {"steps": 12}},
"5": {"class_type": "SamplerCustom", "inputs": {"latent_image": ["3", 0], "sigmas": ["4", 0]}},
}
found = ai_run_watch.estimate(prompt, 24576)
self.assertEqual([(item["width"], item["height"], item["batch"], item["steps"]) for item in found["samplers"]],
[(832, 1216, 2, 28), (1248, 1824, 2, 12)])
self.assertEqual(found["warnings"], [])
def test_a_small_card_is_warned_sooner(self):
prompt = {"1": {"class_type": "EmptyLatentImage", "inputs": {"width": 2048, "height": 1536, "batch_size": 1}},
"2": {"class_type": "KSampler", "inputs": {"latent_image": ["1", 0], "steps": 20}}}
self.assertEqual(ai_run_watch.estimate(prompt, 24576)["warnings"], [])
self.assertEqual(ai_run_watch.estimate(prompt, 6144)["warnings"][0]["kind"], "big_sample")
def test_odd_prompts_do_not_break_it(self):
loop = {"1": {"class_type": "X", "inputs": {"image": ["2", 0]}}, "2": {"class_type": "Y", "inputs": {"image": ["1", 0]}},
"3": {"class_type": "KSampler", "inputs": {"latent_image": ["1", 0]}}, "4": "junk"}
self.assertEqual(ai_run_watch.estimate(loop), {"samplers": [], "warnings": [], "comfort_mp": 4.2})
class StopTests(unittest.TestCase):
def setUp(self):
isolate()
self.task = ai_tasks.begin_task("A", "Test")["id"]
self.step = ai_tasks.record("A", "run_workflow", {}, ok({}), time.time())
with ai_tasks._lock:
task = ai_tasks._load()[self.task]
task["steps"][-1]["prompt_ids"] = ["running", "waiting"]
ai_tasks._save(task)
self.queue = mock.Mock(spec=["delete_queue_item", "interrupt_if_running", "get_current_queue"])
self.queue.delete_queue_item.side_effect = lambda match: match((2, "waiting"))
self.queue.interrupt_if_running.side_effect = lambda pid: pid == "running"
def test_the_user_stops_it_and_the_ai_is_told(self):
with mock.patch.object(ai_task_runs, "_queue", return_value=self.queue):
answer = ai_run_control.stop(self.step["id"], "user", self.task)
self.assertEqual((answer["removed"], answer["interrupted"]), (1, 1))
# The interrupted prompt ends "error" in its history; the removed one never gets one.
ai_task_runs._finish_run(self.task, self.step["id"], ["running", "waiting"],
{"running": {"status": {"status_str": "error"}}}, queued=set())
step = ai_tasks.get_task(self.task)["steps"][0]
self.assertEqual((step["state"], step["ok"], step["stop"]), ("cancelled", True, "user"))
self.assertIn("user stopped", ai_task_runs.summary(step))
result = ai_panels.attach_pending(ok({}), "A", task_id=self.task)
self.assertEqual(result["structuredContent"]["user_messages"][0]["kind"], "stop")
def test_only_a_run_still_going_can_be_stopped(self):
ai_task_runs._finish_run(self.task, self.step["id"], ["running", "waiting"],
{"running": {"status": {"completed": True}}, "waiting": {"status": {"completed": True}}})
with mock.patch.object(ai_task_runs, "_queue", return_value=self.queue), self.assertRaises(ai_run_control.RunControlError):
ai_run_control.stop(self.step["id"], "user", self.task)
with self.assertRaises(ai_run_control.RunControlError):
ai_run_control.stop("nope", "user", self.task)
self.queue.delete_queue_item.assert_not_called()
def test_the_ai_hears_the_warnings_while_its_run_waits_or_goes(self):
warning = {"kind": "big_sample", "node": "2", "title": "KSampler", "width": 4096, "height": 4096, "megapixels": 16.0}
run = {"state": "running", "at": time.time(), "progress": {"state": "queued", "ahead": 1}, "estimate": {"warnings": [warning]}}
self.assertIn("16.0 megapixels", ai_task_runs.summary(run))
self.assertIn("stop_run", ai_task_runs.summary({**run, "progress": {"state": "running"}}))
self.assertNotIn("Warning", ai_task_runs.summary({**run, "state": "done", "ms": 1000}))
def test_a_stopped_prompt_still_queued_is_not_counted_done(self):
with mock.patch.object(ai_task_runs, "_queue", return_value=self.queue):
ai_run_control.stop(self.step["id"], "ai", self.task)
self.assertFalse(ai_task_runs._finish_run(self.task, self.step["id"], ["running", "waiting"], {}, queued={"waiting"}))
self.assertEqual(ai_tasks.get_task(self.task)["steps"][0]["state"], "running")
if __name__ == "__main__":
unittest.main()
+27
View File
@@ -0,0 +1,27 @@
"""Task canvas metadata reaches missing-model downloads as well as canvas edits."""
import unittest
from test_ai_tasks import isolate
from api import ai_task_canvas, ai_tasks
class CanvasTests(unittest.TestCase):
def setUp(self):
isolate()
self.addCleanup(ai_task_canvas.begin_call, None)
def test_missing_model_download_carries_its_task_tab(self):
task = ai_tasks.begin_task('A', 'Canvas')
ai_task_canvas.set_mode(task['id'], 'new')
ai_task_canvas.begin_call(task['id'])
ai_task_canvas.remember({'path': 'task-A.json', 'name': 'A'})
spec = ai_task_canvas.spec('download_missing')
self.assertEqual(spec['mode'], 'new')
self.assertEqual(spec['tab']['path'], 'task-A.json')
ai_task_canvas.set_mode(task['id'], 'current')
self.assertEqual(ai_task_canvas.spec('download_missing')['mode'], 'current')
self.assertIsNone(ai_task_canvas.spec('open_model'))
if __name__ == '__main__':
unittest.main()
+44
View File
@@ -0,0 +1,44 @@
"""Tasks that go on from one another (ai_task_links.py): kept when an AI begins a new task while on
another and when the user starts one from a task, changed or taken off by the user, never a loop,
and an AI beginning the next task hears what was decided in the one before."""
import unittest
from test_ai_tasks import isolate
from api import ai_task_links, ai_tasks, mcp_task_tools
class LinkTests(unittest.TestCase):
def setUp(self):
isolate()
self.ctx = {"client": "A", "session": "s1"}
def test_a_new_task_goes_on_from_the_one_the_ai_was_on_and_hears_its_decisions(self):
first = mcp_task_tools.begin_task(self.ctx, "海边少女")["task"]
mcp_task_tools.log_decision({**self.ctx, "task": first}, "构图", "竖图")
second = mcp_task_tools.begin_task(self.ctx, "换成夜景", new_task=True)
self.assertEqual(ai_tasks.get_task(second["task"])["from"], first)
self.assertEqual(second["continues"]["task"], first)
self.assertIn("构图: 竖图", second["continues"]["decisions"])
self.assertEqual(ai_task_links.links(first)["next"], [{"id": second["task"], "title": "换成夜景"}])
self.assertEqual(ai_task_links.links(second["task"])["from"], {"id": first, "title": "海边少女"})
self.assertEqual(ai_tasks.list_tasks()[0]["from"], first)
def test_the_user_starts_links_and_unlinks_but_never_in_a_loop(self):
one = ai_tasks.begin_task("A", "一")["id"]
two = ai_tasks.start_task("二", from_task=one)["id"]
self.assertEqual(ai_tasks.begin_task("B", "二")["id"], two, "an AI takes the task the user started")
self.assertEqual(ai_tasks.get_task(two)["from"], one)
three = ai_tasks.start_task("三", from_task="gone")["id"]
self.assertNotIn("from", ai_tasks.get_task(three))
ai_task_links.set_from(three, two)
with self.assertRaises(ai_task_links.LinkError):
ai_task_links.set_from(one, three) # one → three → two → one would loop
with self.assertRaises(ai_task_links.LinkError):
ai_task_links.set_from(one, one)
ai_task_links.set_from(three, None)
self.assertIsNone(ai_task_links.links(three)["from"])
self.assertIsNone(ai_task_links.carried(ai_tasks.get_task(one)))
if __name__ == "__main__":
unittest.main()
+246
View File
@@ -0,0 +1,246 @@
"""A run an AI started (api/ai_task_runs.py): where it stands, read from a stand-in ComfyUI queue
and progress registry, on its step and for the AI's run_status. Temporary folders only."""
import sys
import threading
import time
import unittest
from enum import Enum
from pathlib import Path
from types import SimpleNamespace
from unittest import mock
PLUGIN = Path(__file__).resolve().parents[1]
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1]), str(PLUGIN / "tests")]
from api import ai_gates, ai_task_runs, ai_tasks # noqa: E402
from test_ai_tasks import isolate, ok # noqa: E402
class State(Enum):
Pending = "pending"
Running = "running"
Finished = "finished"
class Queue:
def __init__(self, running=(), pending=()):
self.running, self.pending = list(running), list(pending)
def get_current_queue(self):
return self.running, self.pending
def registry(prompt_id, nodes):
prompt = {"3": {"class_type": "KSampler", "_meta": {"title": "采样器"}}, "4": {"class_type": "CheckpointLoaderSimple"},
"8": {"class_type": "VAEDecode"}, "9": {"class_type": "SaveImage"}}
return SimpleNamespace(prompt_id=prompt_id, nodes=nodes,
dynprompt=SimpleNamespace(original_prompt=prompt, get_node=lambda node_id: prompt[node_id]))
class ProgressTests(unittest.TestCase):
def setUp(self):
isolate()
def test_queued_then_running_with_the_node_and_its_steps(self):
queue = Queue(running=[(5, "other")], pending=[(7, "mine"), (6, "first")])
with mock.patch.object(ai_task_runs, "_queue", return_value=queue):
self.assertEqual(ai_task_runs.progress_of(["mine"]), {"state": "queued", "ahead": 2})
queue.running, queue.pending = [(7, "mine")], []
nodes = {"4": {"state": State.Finished, "value": 1, "max": 1}, "3": {"state": State.Running, "value": 14, "max": 28}}
with mock.patch.object(ai_task_runs, "_registry", return_value=registry("mine", nodes)):
got = ai_task_runs.progress_of(["mine"])
self.assertEqual(got, {"state": "running", "prompt": 1, "prompts": 1, "nodes_done": 1, "nodes": 4,
"node": "采样器", "value": 14, "max": 28, "percent": 50})
with mock.patch.object(ai_task_runs, "_queue", side_effect=RuntimeError("no server")):
self.assertIsNone(ai_task_runs.progress_of(["mine"]))
def test_progress_reaches_the_step_and_run_status_waits_for_the_end(self):
ai_tasks.record("Claude", "run_workflow", {"batch": 1}, ok({"queued": 1}), time.time())
task_id = ai_tasks.list_tasks()[0]["id"]
step_id = ai_tasks.get_task(task_id)["steps"][0]["id"]
with mock.patch.object(ai_tasks, "_notify") as told:
ai_task_runs._show_progress(task_id, step_id, {"state": "running", "value": 3, "max": 20})
ai_task_runs._show_progress(task_id, step_id, {"state": "running", "value": 3, "max": 20})
self.assertEqual(told.call_count, 1, "the page hears only of changes")
self.assertEqual(told.call_args[0][1]["progress"]["value"], 3)
status = ai_task_runs.status("Claude")
self.assertEqual((status["state"], status["progress"]["max"]), ("running", 20))
self.assertIn("end your turn", status["next"], "by default the user tells the AI when it is done")
with mock.patch.object(ai_gates, "run_wait", return_value="wait"):
self.assertIn("run_status(wait_seconds=55)", ai_task_runs.status("Claude")["next"])
history = {"p1": {"status": {"completed": True}, "outputs": {
"9": {"images": [{"filename": "a.png", "subfolder": "x", "type": "output"}]},
"12": {"images": [{"filename": "pose_00001_.png", "type": "temp"}]}}}} # a PreviewImage: the AI's to look at
threading.Timer(0.3, ai_task_runs._finish_run, (task_id, step_id, ["p1"], history)).start()
status = ai_task_runs.status("Claude", wait_seconds=5)
self.assertEqual((status["state"], status["pictures"], status["previews"]), ("done", ["x/a.png"], ["pose_00001_.png"]))
self.assertIn("folder='temp'", status["previews_note"])
self.assertEqual([item["filename"] for item in ai_tasks.get_task(task_id)["steps"][0]["results"]], ["a.png"], "previews are no results")
self.assertNotIn("progress", ai_tasks.get_task(task_id)["steps"][0], "a finished run has no progress left")
def test_a_run_saves_only_when_something_changes(self):
ai_tasks.record("A", "run_workflow", {}, ok({}), time.time())
task_id = ai_tasks.list_tasks()[0]["id"]
step_id = ai_tasks.get_task(task_id)["steps"][0]["id"]
with mock.patch.object(ai_tasks, "_save") as saved:
ai_task_runs._finish_run(task_id, step_id, ["p1"], {})
ai_task_runs._finish_run(task_id, step_id, ["p1"], {})
self.assertEqual(saved.call_count, 1)
def test_no_run_yet(self):
ai_tasks.record("A", "describe_canvas", {}, ok({}), time.time(), True)
self.assertEqual(ai_task_runs.status("A")["state"], "none") # an answer: the task shows no failed step
def run_step(self):
step = ai_tasks.record("A", "run_workflow", {}, ok({"queued": 1}), time.time())
return ai_tasks.active_task_id("A"), step["id"]
def test_no_queued_prompt_is_not_success(self):
task, step = self.run_step()
ai_task_runs._finish_run(task, step, [], {})
self.assertEqual(ai_task_runs.status("A")["state"], "stopped")
self.assertFalse(ai_tasks.get_task(task)["steps"][-1]["ok"])
def test_failure_is_recorded_even_when_other_prompts_are_pending(self):
task, step = self.run_step()
ai_task_runs._finish_run(task, step, ["one", "two"], {})
ai_task_runs._finish_run(task, step, ["one", "two"], {"one": {"status": {"status_str": "error"}}})
self.assertFalse(ai_tasks.get_task(task)["steps"][-1]["ok"])
self.assertIn("error", ai_task_runs.status("A"))
def test_watcher_timeout_and_history_failure_stop_the_receipt(self):
# Run the actual watcher synchronously; replace only time and host queue boundaries.
def inline_thread(target, **kwargs):
return SimpleNamespace(start=target)
for fail_history in (False, True):
isolate()
task, step = self.run_step()
queue = mock.Mock()
queue.get_history.side_effect = OSError("unavailable")
with mock.patch.object(ai_task_runs.threading, "Thread", side_effect=inline_thread), \
mock.patch.object(ai_task_runs, "_queue", return_value=queue), \
mock.patch.object(ai_tasks, "RUN_WATCH_SECONDS", 60 if fail_history else 0):
ai_task_runs.watch(step, {"prompt_ids": ["mine"]})
status = ai_task_runs.status("A")
self.assertEqual(status["state"], "stopped")
self.assertIn("history" if fail_history else "timed out", status["error"])
def test_batch_progress_keeps_total_and_current_index(self):
task, step = self.run_step()
queue = Queue(running=[(2, "two")])
with mock.patch.object(ai_task_runs, "_queue", return_value=queue), mock.patch.object(ai_task_runs, "_registry", return_value=None):
self.assertEqual(ai_task_runs.progress_of(["one", "two"]), {"state": "running", "prompt": 2, "prompts": 2})
def test_two_tasks_collect_only_their_receipts_and_keep_batch_order(self):
def record(filename):
return {"status": {"completed": True}, "outputs": {"1": {"images": [{"filename": filename, "type": "output"}]}}}
history = {pid: record(pid + ".png") for pid in ("z-first", "a-second", "other-ai", "manual")}
queue = mock.Mock()
queue.get_history.side_effect = lambda prompt_id: {prompt_id: history[prompt_id]}
with mock.patch.object(ai_task_runs.threading, "Thread", side_effect=lambda target, **kw: SimpleNamespace(start=target)), \
mock.patch.object(ai_task_runs, "_queue", return_value=queue):
task, step = self.run_step()
ai_task_runs.watch(step, {"prompt_ids": ["z-first", "a-second"]})
second = ai_tasks.begin_task("B", "other")["id"]
other = ai_tasks.record("B", "run_workflow", {}, ok({}), time.time())
ai_task_runs.watch(other["id"], {"prompt_ids": ["other-ai"], "submission_error": "Only 1 of 2 prompts accepted"})
self.assertEqual(ai_task_runs.status("A")["pictures"], ["z-first.png", "a-second.png"])
self.assertEqual(ai_task_runs.status("B")["pictures"], ["other-ai.png"])
self.assertEqual(ai_task_runs.status("B")["state"], "error")
self.assertNotIn(mock.call(prompt_id="manual"), queue.get_history.call_args_list)
class NewsTests(unittest.TestCase):
"""The AI hears where its run stands in its other answers, and once that it ended."""
def setUp(self):
isolate()
self.step = ai_tasks.record("A", "run_workflow", {}, ok({"queued": 1}), time.time())
self.task = ai_tasks.active_task_id("A")
with ai_tasks._lock:
task = ai_tasks._load()[self.task]
task["steps"][-1]["prompt_ids"] = ["p1"]
ai_tasks._save(task)
def told(self):
return ai_task_runs.tell({"content": [{"type": "text", "text": "{}"}], "structuredContent": {"n": 1}}, self.task)
def test_while_it_goes_every_answer_says_where_it_stands(self):
with mock.patch.object(ai_tasks, "_notify"):
ai_task_runs._show_progress(self.task, self.step["id"], {"state": "running", "prompt": 1, "prompts": 1, "node": "KSampler",
"value": 14, "max": 28, "percent": 50, "nodes_done": 3, "nodes": 9})
for _ in range(2):
result = self.told()
run = result["structuredContent"]["your_run"]
self.assertEqual((run["state"], run["progress"]["percent"]), ("running", 50))
self.assertIn("KSampler step 14/28, 50%, 3 of 9 nodes done", run["summary"])
self.assertIn("your_run", result["content"][0]["text"], "the text says it too")
self.assertEqual(result["structuredContent"]["n"], 1, "the tool's own answer stays")
def test_the_end_is_told_once(self):
history = {"p1": {"status": {"completed": True}, "outputs": {"9": {"images": [{"filename": "a.png", "type": "output"}]}}}}
with mock.patch.object(ai_tasks, "_notify"):
ai_task_runs._finish_run(self.task, self.step["id"], ["p1"], history)
self.assertIn("finished", self.told()["structuredContent"]["your_run"]["summary"])
self.assertNotIn("your_run", self.told()["structuredContent"])
def test_run_status_telling_the_end_is_enough(self):
with mock.patch.object(ai_tasks, "_notify"):
ai_task_runs._finish_run(self.task, self.step["id"], ["p1"], {"p1": {"status": {"completed": True}}})
self.assertIn("finished", ai_task_runs.status("A")["summary"])
self.assertNotIn("your_run", self.told()["structuredContent"])
def test_the_pictures_come_once_with_the_answer_that_tells_the_end(self):
history = {"p1": {"status": {"completed": True}, "outputs": {"9": {"images": [
{"filename": f"{n}.png", "subfolder": "x", "type": "output"} for n in range(6)]}}}}
with mock.patch.object(ai_tasks, "_notify"):
ai_task_runs._finish_run(self.task, self.step["id"], ["p1"], history)
picture = {"type": "image", "data": "AAAA", "mimeType": "image/webp"}
with mock.patch.object(ai_task_runs, "_resolve_image", side_effect=lambda rel: (rel, "/out/" + rel)), mock.patch.object(ai_task_runs, "_image_content", return_value=picture):
before = ai_task_runs.show_pictures({"content": [{"type": "text", "text": "{}"}], "structuredContent": {}}, self.task)
self.assertEqual(len(before["content"]), 1, "nothing before the AI is told it ended")
result = ai_task_runs.show_pictures(self.told(), self.task)
self.assertEqual(result["structuredContent"]["pictures_attached"], ["x/0.png", "x/1.png", "x/2.png", "x/3.png"])
self.assertEqual([item["type"] for item in result["content"]], ["text"] + ["image"] * ai_task_runs.MAX_SHOWN)
self.assertIn("pictures_attached", result["content"][0]["text"])
again = ai_task_runs.show_pictures({"content": [{"type": "text", "text": "{}"}], "structuredContent": {}}, self.task)
self.assertEqual(len(again["content"]), 1, "once per run")
def test_a_line_to_paste_until_the_ai_knows(self):
with mock.patch.object(ai_tasks, "_notify"), mock.patch.object(ai_gates, "run_wait", return_value="call"):
ai_task_runs._finish_run(self.task, self.step["id"], ["p1"], {"p1": {"status": {"completed": True}}})
self.assertTrue(ai_tasks.get_task(self.task)["steps"][-1]["call_ai"], "the workbench offers the line")
self.told()
self.assertNotIn("call_ai", ai_tasks.get_task(self.task)["steps"][-1], "gone once the AI is told")
def test_no_line_when_the_ai_waits_or_a_new_run_starts(self):
with mock.patch.object(ai_tasks, "_notify"), mock.patch.object(ai_gates, "run_wait", return_value="wait"):
ai_task_runs._finish_run(self.task, self.step["id"], ["p1"], {"p1": {"status": {"completed": True}}})
self.assertNotIn("call_ai", ai_tasks.get_task(self.task)["steps"][-1])
with ai_tasks._lock:
task = ai_tasks._load()[self.task]
task["steps"][-1]["call_ai"] = True
ai_tasks._save(task)
newer = ai_tasks.record("A", "run_workflow", {}, ok({"queued": 1}), time.time())
with mock.patch("threading.Thread"):
ai_task_runs.watch(newer["id"], {"prompt_ids": ["p2"]})
self.assertNotIn("call_ai", ai_tasks.get_task(self.task)["steps"][0], "an older run offers no line once another starts")
def test_an_old_run_told_before_brings_no_pictures(self):
with mock.patch.object(ai_tasks, "_notify"):
ai_task_runs._finish_run(self.task, self.step["id"], ["p1"], {"p1": {"status": {"completed": True}, "outputs": {
"9": {"images": [{"filename": "a.png", "type": "output"}]}}}})
with ai_tasks._lock:
task = ai_tasks._load()[self.task]
task["steps"][-1]["told"] = True # told before pictures came with it: no "shown" mark
ai_tasks._save(task)
result = ai_task_runs.show_pictures({"content": [{"type": "text", "text": "{}"}], "structuredContent": {}}, self.task)
self.assertEqual(len(result["content"]), 1)
def test_no_run_no_news(self):
ai_tasks.begin_task("A", "other", new_task=True)
self.assertIsNone(ai_task_runs.news(ai_tasks.active_task_id("A")))
if __name__ == "__main__":
unittest.main()
+414
View File
@@ -0,0 +1,414 @@
"""The AI workbench's record (api/ai_tasks.py): tasks, steps, what they made, notes, the routes.
Everything is kept in a temporary folder: no test writes into the real ComfyUI user folder."""
import asyncio
import json
import os
import sys
import tempfile
import unittest
from pathlib import Path
from unittest import mock
PLUGIN = Path(__file__).resolve().parents[1]
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
from api import action_catalog, activity_log, ai_callers, ai_panels, ai_task_changes, ai_task_runs, ai_tasks, ai_tasks_routes, mcp_task_tools # noqa: E402
from api.mcp_server import TOOLS # noqa: E402
from api.ai_task_handoff import AFTER_DONE, REPLY_FRESH
TMP = tempfile.mkdtemp(prefix="amb_ai_tasks_")
LOG = tempfile.mkdtemp(prefix="amb_ai_activity_")
OUTPUT = tempfile.mkdtemp(prefix="amb_ai_output_")
def isolate():
"""Point the record at an empty temporary folder (called by every test module that makes calls)."""
ai_tasks.tasks_dir = lambda: TMP
ai_tasks._tasks = None
ai_tasks._current.clear()
ai_tasks._waiting[0] = None
for name in os.listdir(TMP):
os.remove(os.path.join(TMP, name))
activity_log.log_path = lambda: os.path.join(LOG, "activity_log.json")
activity_log._entries = None
activity_log.add_listener(ai_task_changes.note_activity)
if os.path.exists(activity_log.log_path()):
os.remove(activity_log.log_path())
def ok(data):
return {"content": [{"type": "text", "text": json.dumps(data)}], "structuredContent": data, "isError": False}
def failed(message):
return {"content": [{"type": "text", "text": message}], "isError": True}
class TaskTests(unittest.TestCase):
def setUp(self):
isolate()
def test_calls_of_one_app_make_one_task_until_a_long_pause(self):
ai_tasks.record("Claude Code", "search_models", {"query": "arona"}, ok({"models": [1, 2, 3]}), 1000.0, True)
ai_tasks.record("Claude Code", "place_combo", {"name": "K"}, ok({"_activity": "e1"}), 1001.0)
ai_tasks.record("Cursor", "list_combos", {}, ok({"combos": []}), 1002.0, True)
tasks = ai_tasks.list_tasks()
self.assertEqual(len(tasks), 2)
claude = next(task for task in tasks if task["client"] == "Claude Code")
self.assertEqual((claude["steps"], claude["kinds"]), (2, {"look": 1, "canvas": 1}))
self.assertEqual(claude["title"], "place_combo: K", "named after its first change")
detail = ai_tasks.get_task(claude["id"])
look, place = detail["steps"]
self.assertEqual(look["count"], 3)
self.assertEqual(place["results"], [{"type": "combo", "name": "K"}, {"type": "canvas", "activity": "e1"}])
# Kept on disk, read back after a restart.
ai_tasks._tasks = None
self.assertEqual(ai_tasks.get_task(claude["id"])["steps"][1]["tool"], "place_combo")
# A long pause starts a new task; the old one is ended.
with mock.patch.object(ai_tasks.time, "time", return_value=time_after(claude, ai_tasks.IDLE_SECONDS + 5)):
ai_tasks.record("Claude Code", "describe_canvas", {}, ok({}), ai_tasks.time.time(), True)
self.assertEqual(len([task for task in ai_tasks.list_tasks() if task["client"] == "Claude Code"]), 2)
self.assertTrue(ai_tasks.get_task(claude["id"])["ended"])
def test_begin_task_keeps_preliminary_reads_and_names_it(self):
ai_tasks.record("Claude", "describe_canvas", {}, ok({}), 10.0, True)
first = ai_tasks.list_tasks()[0]["id"]
answer = mcp_task_tools.begin_task({"client": "Claude"}, "画一张海边的阿罗娜")
self.assertEqual(answer["task"], first)
self.assertIsNone(ai_tasks.get_task(first)["ended"])
ai_tasks.record("Claude", "run_workflow", {"batch": 2}, failed("No workflow is open"), 20.0)
task = ai_tasks.get_task(answer["task"])
self.assertEqual(task["title"], "画一张海边的阿罗娜")
step = task["steps"][-1]
self.assertFalse(step["ok"])
self.assertEqual(step["error"], "No workflow is open")
self.assertEqual(step["results"], [])
self.assertEqual(ai_tasks.list_tasks()[0]["errors"], 1)
with self.assertRaises(mcp_task_tools.ToolError):
mcp_task_tools.begin_task({"client": "Claude"}, " ")
def test_in_flight_step_updates_in_place_without_duplicates(self):
waiting = ai_tasks.start_task("生成测试", now=50.0)
ref = ai_tasks.begin_step("Gemini", "speak", {"character": "Arona", "text": "hello"}, started=60.0)
self.assertIsNotNone(ref)
task_id, step_id = ref
self.assertEqual(task_id, waiting["id"], "starting a tool claims the waiting task immediately")
running_task = ai_tasks.get_task(task_id)
self.assertEqual(len(running_task["steps"]), 1)
self.assertEqual(running_task["steps"][0]["state"], "running")
self.assertTrue(ai_tasks.task_summary(running_task, now=61.0)["live"])
finished = ai_tasks.finish_step(
"Gemini", "speak", {"character": "Arona", "text": "hello"},
ok({"done": True, "files": ["audio/Arona/1.flac"]}), 60.0, step_ref=ref,
)
self.assertEqual(finished["id"], step_id)
done_task = ai_tasks.get_task(task_id)
self.assertEqual(len(done_task["steps"]), 1, "completed step updates in place, no duplicate")
self.assertEqual((done_task["steps"][0]["state"], done_task["steps"][0]["ok"]), ("done", True))
self.assertEqual(done_task["steps"][0]["results"], [{"type": "audio", "file": "audio/Arona/1.flac"}])
# A failing tool updates its in-flight step to error.
err_ref = ai_tasks.begin_step("Gemini", "set_model", {"model_id": "bad"}, started=70.0)
ai_tasks.finish_step("Gemini", "set_model", {"model_id": "bad"}, failed("No model file"), 70.0, step_ref=err_ref)
err_step = ai_tasks.get_task(task_id)["steps"][1]
self.assertEqual((err_step["id"], err_step["state"], err_step["ok"], err_step["error"]),
(err_ref[1], "error", False, "No model file"))
def test_sessions_and_explicit_task_ids_do_not_collide(self):
t1 = ai_tasks.begin_task("Claude", "任务一", now=100.0, session="s1")
t2 = ai_tasks.begin_task("Claude", "任务二", now=101.0, session="s2")
self.assertNotEqual(t1["id"], t2["id"])
ai_tasks.record("Claude", "search_models", {"query": "one"}, ok({"models": [1]}), 102.0, True, session="s1")
ai_tasks.record("Claude", "search_models", {"query": "two"}, ok({"models": [1, 2]}), 103.0, True, session="s2")
self.assertEqual(ai_tasks.get_task(t1["id"])["steps"][0]["summary"], "search_models: one")
self.assertEqual(ai_tasks.get_task(t2["id"])["steps"][0]["summary"], "search_models: two")
# An explicit task id routes even when called from a fallback client name.
ai_tasks.record("AI", "describe_canvas", {}, ok({}), 104.0, True, task_id=t1["id"])
self.assertEqual(len(ai_tasks.get_task(t1["id"])["steps"]), 2)
def test_what_steps_made(self):
with mock.patch.object(ai_tasks, "_model_result", side_effect=lambda model_id: {"type": "model", "model_id": model_id}):
ai_tasks.record("A", "add_lora", {"model_id": "m1"}, ok({"_activity": "e2"}), 1.0)
ai_tasks.record("A", "speak", {"character": "Aris", "text": "hi"}, ok({"done": True, "files": ["audio/Aris/a.flac"]}), 2.0)
ai_tasks.record("A", "scan_models", {"what": "new"}, ok({"started": True}), 3.0)
ai_tasks.record("A", "download_model", {"type": "loras", "url": "https://civitai.com/models/1"},
ok({"started": True, "to": "SDXL/hand.safetensors", "size": 2048, "model": "Hands — v2"}), 4.0)
ai_tasks.record("A", "download_missing_models", {}, ok({"started": [{"model": "vae.safetensors", "to": "vae.safetensors", "size": None}],
"not_started": [{"model": "x.safetensors", "reason": "no_source"}]}), 5.0)
steps = ai_tasks.get_task(ai_tasks.list_tasks()[0]["id"])["steps"]
self.assertEqual([step["kind"] for step in steps], ["canvas", "speech", "scan", "download", "download"])
self.assertEqual(steps[0]["results"], [{"type": "model", "model_id": "m1"}, {"type": "canvas", "activity": "e2"}])
self.assertEqual(steps[1]["results"], [{"type": "audio", "file": "audio/Aris/a.flac"}])
self.assertEqual(steps[2]["results"], [{"type": "scan"}])
self.assertEqual(steps[3]["results"], [{"type": "download", "file": "hand.safetensors", "size": 2048, "name": "Hands — v2"}])
self.assertEqual(steps[4]["results"], [{"type": "download", "file": "vae.safetensors", "size": None}], "only what started")
self.assertIsNone(ai_tasks.record("A", "begin_task", {"title": "x"}, ok({}), 6.0), "begin_task is not a step")
def test_a_run_keeps_the_pictures_it_made(self):
ai_tasks.record("A", "run_workflow", {"batch": 1}, ok({"queued": 1}), 1.0) # no server: nothing to watch
task_id = ai_tasks.list_tasks()[0]["id"]
step = ai_tasks.get_task(task_id)["steps"][0]
history = {"p1": {"status": {"completed": True, "status_str": "success"},
"outputs": {"9": {"images": [{"filename": "a.png", "subfolder": "x", "type": "output"},
{"filename": "t.png", "subfolder": "", "type": "temp"}]}}}}
self.assertFalse(ai_task_runs._finish_run(task_id, step["id"], ["p1", "p2"], history), "p2 still runs")
self.assertEqual(ai_tasks.get_task(task_id)["steps"][0]["state"], "running")
history["p2"] = {"status": {"status_str": "error"}, "outputs": history["p1"]["outputs"]} # cached: same file
self.assertTrue(ai_task_runs._finish_run(task_id, step["id"], ["p1", "p2"], history))
step = ai_tasks.get_task(task_id)["steps"][0]
self.assertEqual(step["results"], [{"type": "image", "filename": "a.png", "subfolder": "x", "folder": "output"}])
self.assertEqual((step["state"], step["ok"]), ("error", False))
self.assertEqual(ai_tasks.list_tasks()[0]["cover"]["filename"], "a.png")
def test_report_keeps_only_pictures_in_the_output_folder(self):
os.makedirs(os.path.join(OUTPUT, "sub"), exist_ok=True)
Path(OUTPUT, "sub", "made.png").write_bytes(b"png")
Path(OUTPUT, "notes.txt").write_text("x")
with mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT):
answer = mcp_task_tools.report({"client": "A"}, "图 2 的手不对,换了种子", ["sub/made.png", "../secret.png", "notes.txt", "nope.png"], True)
self.assertEqual(answer["pictures"], 1)
self.assertEqual(answer["not_found"], ["../secret.png", "notes.txt", "nope.png"])
task = ai_tasks.get_task(ai_tasks.list_tasks()[0]["id"])
self.assertTrue(task["ended"])
self.assertEqual(task["steps"][0]["text"], "图 2 的手不对,换了种子")
self.assertEqual(task["steps"][0]["results"], [{"type": "image", "filename": "made.png", "subfolder": "sub", "folder": "output"}])
with self.assertRaises(mcp_task_tools.ToolError):
mcp_task_tools.report({"client": "A"}, " ")
def test_rename_remove_and_the_oldest_go(self):
ai_tasks.record("A", "describe_canvas", {}, ok({}), 1.0, True)
task_id = ai_tasks.list_tasks()[0]["id"]
self.assertEqual(ai_tasks.rename_task(task_id, " 我的任务 ")["title"], "我的任务")
self.assertTrue(ai_tasks.remove_task(task_id))
self.assertFalse(os.path.exists(os.path.join(TMP, f"{task_id}.json")))
self.assertFalse(ai_tasks.remove_task(task_id))
with mock.patch.object(ai_tasks, "MAX_TASKS", 3):
for index in range(5):
ai_tasks.begin_task("A", f"t{index}", now=100.0 + index, new_task=True)
self.assertEqual([task["title"] for task in ai_tasks.list_tasks()], ["t4", "t3", "t2"])
self.assertEqual(len(os.listdir(TMP)), 3)
def test_a_failure_the_ai_got_past_does_not_mark_the_task(self):
ai_tasks.record("Gemini", "check_workflow_models", {"workflow": "x"}, failed("Unknown arguments ['workflow']"), 100.0, True)
ai_tasks.record("Gemini", "check_workflow_models", {}, ok({"missing": []}), 101.0, True)
self.assertEqual(ai_tasks.list_tasks()[0]["errors"], 0, "corrected on the next call")
ai_tasks.record("Gemini", "run_workflow", {}, failed("ComfyUI did not take the run"), 102.0)
self.assertEqual(ai_tasks.list_tasks()[0]["errors"], 1, "still not run")
def test_a_look_right_after_done_stays_in_that_task(self):
ai_tasks.record("Gemini", "run_workflow", {}, ok({}), 100.0)
done, _ = ai_tasks.report("Gemini", "做好了", done=True, now=110.0)
task_id = ai_tasks.list_tasks()[0]["id"]
ai_tasks.record("Gemini", "get_situation", {}, ok({}), 114.0, True)
self.assertEqual(len(ai_tasks.list_tasks()), 1, "no empty task for one look")
task = ai_tasks.get_task(task_id)
self.assertEqual((len(task["steps"]), task["ended"]), (3, 110.0), "still done")
ai_panels.show("Gemini", "前后对比", [{"type": "text", "text": "好了"}], now=116.0)
self.assertEqual((len(ai_tasks.list_tasks()), ai_tasks.get_task(task_id)["ended"]), (1, None),
"showing or asking about what it just did opens that task again")
ai_tasks.report("Gemini", "完成", done=True, now=118.0)
ai_tasks.record("Gemini", "set_prompt", {"prompt": "x"}, ok({}), 120.0)
self.assertEqual(len(ai_tasks.list_tasks()), 2, "new work is a new task")
ai_tasks.record("Gemini", "get_situation", {}, ok({}), 120.0 + AFTER_DONE, True)
self.assertEqual(ai_tasks.get_task(task_id)["steps"][-1]["tool"], "report", "only right after done")
def test_a_task_the_user_started_waits_however_long(self):
started = ai_tasks.start_task("画一张魔女", now=100.0)
later = 100.0 + 18 * 3600
with mock.patch.object(ai_tasks.time, "time", return_value=later):
ai_tasks.record("Gemini", "get_situation", {}, ok({}), later, True)
self.assertEqual(ai_tasks.get_task(started["id"])["client"], "Gemini", "taken the next day, as its page says")
def test_a_task_the_user_starts_waits_for_the_next_app(self):
ai_tasks.record("Claude", "describe_canvas", {}, ok({}), 100.0, True)
old = ai_tasks.list_tasks()[0]["id"]
first = ai_tasks.start_task("", now=200.0)
started = ai_tasks.start_task(" 画一张海边的女孩 ", now=210.0)
self.assertIsNone(ai_tasks.get_task(first["id"]), "an untaken empty one is replaced")
self.assertTrue(ai_tasks.task_summary(started)["waiting"])
self.assertFalse(ai_tasks.task_summary(started)["live"])
with mock.patch.object(ai_tasks.time, "time", return_value=220.0):
ai_tasks.record("Claude", "search_models", {"query": "x"}, ok({"models": []}), 220.0, True)
task = ai_tasks.get_task(started["id"])
self.assertEqual((task["client"], task["waiting"], task["title"], len(task["steps"])), ("Claude", False, "画一张海边的女孩", 1))
self.assertTrue(ai_tasks.get_task(old)["ended"], "the app's last task ends")
with mock.patch.object(ai_tasks.time, "time", return_value=230.0):
ai_tasks.record("Claude", "describe_canvas", {}, ok({}), 230.0, True)
self.assertEqual(len(ai_tasks.get_task(started["id"])["steps"]), 2, "the app goes on in it")
# begin_task takes a waiting task too, naming it when the user did not.
unnamed = ai_tasks.start_task("", now=300.0)
with mock.patch.object(ai_tasks.time, "time", return_value=310.0):
got = mcp_task_tools.begin_task({"client": "Cursor"}, "整理 LoRA")
self.assertEqual((got["task"], ai_tasks.get_task(unnamed["id"])["title"]), (unnamed["id"], "整理 LoRA"))
def test_a_task_handed_over_from_anomalous_tells_the_ai_what_it_is_about(self):
about = [{"type": "model", "model_id": "loras:0:a/b.safetensors", "label": "B"}, {"type": "image", "image": "x/1.png"},
{"type": "combo", "name": "K"}, {"type": "file", "path": "C:/secret"}, {"type": "model", "model_id": 3}, "junk"]
started = ai_tasks.start_task("用这个模型出图:B", about=about)
self.assertEqual([item["type"] for item in ai_tasks.get_task(started["id"])["about"]], ["model", "image", "combo"],
"only the known kinds, as plain strings")
got = mcp_task_tools.begin_task({"client": "Claude"}, "出图")
self.assertEqual(got["task"], started["id"])
self.assertEqual(got["about"][0]["model_id"], "loras:0:a/b.safetensors")
self.assertEqual(got["title"], "用这个模型出图:B", "the user's words stay the title")
self.assertNotIn("about", ai_tasks.start_task("plain"), "nothing handed over: no about")
def test_begin_after_claim_keeps_user_task_and_repeated_begin_is_safe(self):
for title in ("", "用户定的标题"):
with self.subTest(title=title):
isolate()
started = ai_tasks.start_task(title, now=100)
with mock.patch.object(ai_tasks.time, "time", return_value=110):
ai_tasks.record("Gemini", "describe_canvas", {}, ok({}), 110, True)
first = mcp_task_tools.begin_task({"client": "Gemini"}, "AI 标题")
second = mcp_task_tools.begin_task({"client": "Gemini"}, "重复调用")
self.assertEqual((first["task"], second["task"]), (started["id"], started["id"]))
task = ai_tasks.get_task(started["id"])
self.assertEqual(task["title"], title or "AI 标题")
self.assertEqual(len(task["steps"]), 1)
self.assertIsNone(task["ended"])
self.assertEqual(len(ai_tasks.list_tasks()), 1)
def test_begin_can_explicitly_split_and_completed_or_idle_tasks_stay_separate(self):
first = ai_tasks.begin_task("A", "first", now=100)
second = ai_tasks.begin_task("A", "second", now=110, new_task=True)
self.assertNotEqual(first["id"], second["id"])
self.assertEqual(first["ended"], 110)
ai_tasks.report("A", "done", done=True, now=120)
third = ai_tasks.begin_task("A", "third", now=130)
self.assertNotEqual(second["id"], third["id"])
fourth = ai_tasks.begin_task("A", "fourth", now=130 + ai_tasks.IDLE_SECONDS)
self.assertNotEqual(third["id"], fourth["id"])
self.assertIsNotNone(third["ended"])
other = ai_tasks.begin_task("B", "other", now=140)
self.assertNotEqual(fourth["id"], other["id"])
def test_every_tool_is_in_the_catalog(self):
self.assertEqual(set(TOOLS) - set(action_catalog.TOOLS), set())
self.assertEqual(set(action_catalog.TOOLS) - set(TOOLS), set(), "no entries for tools that are gone")
for tool, what in action_catalog.TOOLS.items():
self.assertIn(what.stage, (None, *action_catalog.STAGES), tool)
self.assertIn(what.gate, (None, *action_catalog.GATES), tool)
read_only = TOOLS[tool]["spec"].get("annotations", {}).get("readOnlyHint", True)
if what.gate:
self.assertFalse(read_only, f"{tool} is guarded but says it only reads")
self.assertEqual(ai_tasks.list_tasks(), [])
def test_steps_carry_their_stage(self):
ai_tasks.record("A", "search_models", {"query": "x"}, ok({"models": []}), 1.0, True)
ai_tasks.record("A", "set_prompt", {"prompt": "x"}, ok({}), 2.0)
steps = ai_tasks.get_task(ai_tasks.list_tasks()[0]["id"])["steps"]
self.assertEqual([(s["kind"], s["stage"]) for s in steps], [("look", "model"), ("canvas", "prompt")])
def test_steps_left_running_by_a_restart_are_stopped(self):
ref = ai_tasks.begin_step("A", "describe_canvas", {}, 1.0, True)
ai_tasks._tasks = None # ComfyUI started again
step = ai_tasks.get_task(ref[0])["steps"][0]
self.assertEqual((step["state"], step["ok"]), ("stopped", False))
self.assertIn("restarted", step["error"])
ai_tasks._tasks = None
self.assertEqual(ai_tasks.get_task(ref[0])["steps"][0]["state"], "stopped", "saved, said once")
def test_the_users_own_changes_join_the_task_the_ai_works_on(self):
now = ai_tasks.time.time()
ai_tasks.record("A", "place_combo", {"name": "K"}, ok({"_activity": "none"}), now)
task_id = ai_tasks.list_tasks()[0]["id"]
mine = activity_log.add_entry("canvas", "canvas", "wf", {"changes": [
{"kind": "changed", "node": "KSampler", "widget": "cfg", "before": "7", "after": "5"}], "total": 1})
ais = activity_log.add_entry("canvas", "canvas", "wf", {"changes": [{"kind": "added", "node": "Lora"}], "via": "mcp"})
activity_log.add_entry("file", "model_download", "x.safetensors") # an AI's download could make this
steps = ai_tasks.get_task(task_id)["steps"]
self.assertEqual([s["tool"] for s in steps], ["place_combo", "user_activity"])
self.assertEqual((steps[1]["activity"], steps[1]["stage"]), (mine["id"], "canvas"))
self.assertEqual(activity_log.list_entries()[2]["task"], task_id, "the entry names its task")
self.assertNotIn("task", ais)
# The AI hears of it with its next call, once.
result = ai_panels.attach_pending(ok({}), "A", task_id=task_id)
change = result["structuredContent"]["user_messages"][0]
self.assertEqual(change["kind"], "change")
self.assertIn("KSampler cfg: 7 -> 5", change["text"])
self.assertNotIn("user_messages", ai_panels.attach_pending(ok({}), "A", task_id=task_id)["structuredContent"])
self.assertTrue(ai_tasks.get_task(task_id)["steps"][1]["delivered"])
def test_no_task_hears_of_changes_once_the_ai_is_done_or_gone(self):
ai_tasks.record("A", "describe_canvas", {}, ok({}), ai_tasks.time.time(), True)
gone = ai_tasks.list_tasks()[0]["id"]
ai_tasks._load()[gone]["updated"] -= ai_task_changes.USER_STEP_SECONDS + 1 # the AI went quiet
activity_log.add_entry("file", "model_edit", "a.safetensors")
ai_tasks.report("B", "done", done=True) # this one ended
activity_log.add_entry("file", "model_edit", "a.safetensors")
for summary in ai_tasks.list_tasks():
self.assertNotIn("user_activity", [s["tool"] for s in ai_tasks.get_task(summary["id"])["steps"]])
def test_the_ais_canvas_step_tags_its_activity_entry(self):
entry = activity_log.add_entry("canvas", "canvas", "wf", {"changes": [{"kind": "added", "node": "L"}], "via": "mcp"})
ai_tasks.record("A", "add_lora", {}, ok({"_activity": entry["id"]}), ai_tasks.time.time())
self.assertEqual(activity_log.list_entries()[0]["task"], ai_tasks.list_tasks()[0]["id"])
def test_the_ai_goes_on_in_the_task_the_user_wrote_to(self):
ai_tasks.begin_task("A", "睡衣少女")
ai_tasks.report("A", "做完了", done=True)
old_id = ai_tasks.list_tasks()[0]["id"]
ai_tasks.begin_task("A", "另一件事", new_task=True)
ai_tasks.report("A", "也做完了", done=True)
self.assertNotEqual(ai_tasks.list_tasks()[0]["id"], old_id)
asked = ai_panels.leave_message(old_id, "这根手指怪怪的")
ai_tasks.record("A", "describe_canvas", {}, ok({}), ai_tasks.time.time(), True)
self.assertEqual(ai_tasks.get_task(old_id)["steps"][-1]["tool"], "describe_canvas", "its next call lands in the task written to")
self.assertIsNone(ai_tasks.get_task(old_id)["ended"])
ai_panels.attach_pending(ok({}), "A")
self.assertTrue(mcp_task_tools.report({"client": "A"}, "改好了", reply_to=asked["id"])["noted"])
def test_a_reply_left_unread_long_ago_does_not_draw_a_new_chat(self):
ai_tasks.begin_task("A", "前天的")
ai_tasks.report("A", "做完了", done=True)
old_id = ai_tasks.list_tasks()[0]["id"]
ai_panels.leave_message(old_id, "这根手指怪怪的", now=ai_tasks.time.time() - REPLY_FRESH - 60)
ai_tasks._current.clear() # ComfyUI started again since: no task is this app's open one
ai_tasks.record("A", "describe_canvas", {}, ok({}), ai_tasks.time.time(), True)
self.assertNotEqual(ai_tasks.get_task(old_id)["steps"][-1]["tool"], "describe_canvas", "a new task, not the one written to days ago")
def test_a_reply_elsewhere_does_not_pull_the_ai_off_its_open_task(self):
ai_tasks.begin_task("A", "旧的")
ai_tasks.report("A", "做完了", done=True)
old_id = ai_tasks.list_tasks()[0]["id"]
ai_tasks.begin_task("A", "正在做", new_task=True)
busy = ai_tasks.list_tasks()[0]["id"]
ai_panels.leave_message(old_id, "顺便看看这个")
ai_tasks.record("A", "describe_canvas", {}, ok({}), ai_tasks.time.time(), True)
self.assertEqual(ai_tasks.get_task(busy)["steps"][-1]["tool"], "describe_canvas")
def test_client_names(self):
self.assertEqual(ai_callers.client_of({}, {"_meta": {"io.modelcontextprotocol/clientInfo": {"name": "claude-code"}}}), "Claude Code")
self.assertEqual(ai_callers.client_of({"User-Agent": "Cursor/1.2 (win)"}, {}), "Cursor")
self.assertEqual(ai_callers.client_of({"User-Agent": "my<tool>/1"}, {}), "mytool")
self.assertEqual(ai_callers.client_of({}, {}), "AI")
def test_routes(self):
ai_tasks.record("A", "describe_canvas", {}, ok({}), 1.0, True)
task_id = ai_tasks.list_tasks()[0]["id"]
def call(handler, method="GET", body=None, task=None):
request = mock.Mock()
request.query = {}
request.match_info = {"task_id": task} if task else {}
request.json = lambda: asyncio.sleep(0, body)
response = asyncio.run(handler(request))
return response.status, json.loads(response.text)
self.assertEqual(call(ai_tasks_routes.api_list)[1]["tasks"][0]["id"], task_id)
self.assertEqual(call(ai_tasks_routes.api_get, task=task_id)[1]["task"]["id"], task_id)
self.assertEqual(call(ai_tasks_routes.api_get, task="nope")[0], 404)
self.assertEqual(call(ai_tasks_routes.api_change, body={"title": "新名字"}, task=task_id)[1]["task"]["title"], "新名字")
self.assertEqual(call(ai_tasks_routes.api_change, body=[], task=task_id)[0], 400)
self.assertTrue(call(ai_tasks_routes.api_change, body={"remove": True}, task=task_id)[1]["removed"])
started = call(ai_tasks_routes.api_start, body={"title": "新的"})[1]["task"]
self.assertEqual((started["title"], started["waiting"]), ("新的", True))
def time_after(task, seconds):
return task["updated"] + seconds
if __name__ == "__main__":
unittest.main()
+142
View File
@@ -0,0 +1,142 @@
"""What the user hands the AI to work with (api/ai_toolbox.py): picks checked and kept with the
task, given to the AI once as a message and in begin_task, favourites standing in, and "only"
refusing other custom or paid node types. A stand-in /object_info; temporary folders only."""
import os
import sys
import tempfile
import time
import unittest
from pathlib import Path
from unittest import mock
PLUGIN = Path(__file__).resolve().parents[1]
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1]), str(PLUGIN / "tests")]
from api import ai_panels, ai_tasks, ai_toolbox, mcp_canvas_edit, mcp_task_tools # noqa: E402
from api.mcp_tools import ToolError # noqa: E402
from test_ai_tasks import isolate, ok # noqa: E402
FAVOURITES = os.path.join(tempfile.mkdtemp(prefix="amb_ai_toolbox_"), "ai_toolbox.json")
INFO = {
"KSampler": {"python_module": "nodes", "display_name": "KSampler", "input": {"required": {"model": ["MODEL"]}}, "output": ["LATENT"]},
"ImageBlur": {"python_module": "comfy_extras.nodes_post_processing", "input": {}, "output": ["IMAGE"]},
"FaceDetailer": {"python_module": "custom_nodes.ComfyUI-Impact-Pack", "display_name": "FaceDetailer", "input": {}, "output": ["IMAGE"]},
"SAMLoader": {"python_module": "custom_nodes.ComfyUI-Impact-Pack", "input": {}, "output": ["SAM_MODEL"]},
"KlingTextToVideo": {"python_module": "comfy_api_nodes.nodes_kling", "input": {}, "output": ["VIDEO"]},
}
class ToolboxTests(unittest.TestCase):
def setUp(self):
isolate()
if os.path.exists(FAVOURITES):
os.remove(FAVOURITES)
for target, value in ((ai_toolbox, "_path"), (mcp_canvas_edit, "object_info")):
patcher = mock.patch.object(target, value, (lambda: FAVOURITES) if value == "_path" else (lambda ctx: INFO))
patcher.start()
self.addCleanup(patcher.stop)
ai_tasks.record("Gemini", "describe_canvas", {}, ok({}), time.time(), True)
self.task = ai_tasks.list_tasks()[0]["id"]
self.ctx = {"client": "Gemini", "task": self.task, "base_url": "http://127.0.0.1:1"}
def test_a_pick_is_kept_and_reaches_the_ai_once(self):
step = ai_toolbox.pick(self.task, {"nodes": ["FaceDetailer", "FaceDetailer"], "combos": ["赛博少女"]}, "prefer")
self.assertEqual(step["toolbox"]["nodes"], ["FaceDetailer"], "each once")
self.assertEqual(ai_tasks.get_task(self.task)["toolbox"]["combos"], ["赛博少女"])
result = ai_panels.attach_pending(ok({}), "Gemini", task_id=self.task)
message = result["structuredContent"]["user_messages"][0]
self.assertEqual(message["kind"], "toolbox")
self.assertIn("FaceDetailer", message["text"])
self.assertNotIn("user_messages", ai_panels.attach_pending(ok({}), "Gemini", task_id=self.task)["structuredContent"])
begun = mcp_task_tools.begin_task(self.ctx, "继续")
self.assertIn("Use these first", begun["toolbox"])
def test_favourites_stand_in_until_the_task_has_its_own(self):
self.assertIsNone(ai_toolbox.effective(ai_tasks.get_task(self.task)))
ai_toolbox.set_favorites({"nodes": ["SAMLoader"]})
self.assertEqual(ai_toolbox.effective(ai_tasks.get_task(self.task))["from"], "favorites")
ai_toolbox.pick(self.task, {"nodes": []}, "prefer")
box = ai_toolbox.effective(ai_tasks.get_task(self.task))
self.assertEqual((box["from"], box["nodes"]), ("task", []), "an empty pick is the user's choice too")
def test_only_refuses_other_custom_and_paid_nodes_but_not_built_in_or_page_helpers(self):
ai_toolbox.pick(self.task, {"nodes": ["FaceDetailer"]}, "only")
self.assertEqual(ai_toolbox.refused(self.task, ["KSampler", "ImageBlur", "FaceDetailer", "Reroute", "SAMLoader", "KlingTextToVideo"], INFO),
["KlingTextToVideo", "SAMLoader"])
with mock.patch.object(mcp_canvas_edit, "_page", return_value={}) as page:
with self.assertRaisesRegex(ToolError, "allowed only these node types.*FaceDetailer.*Not allowed: SAMLoader"):
mcp_canvas_edit.add_node(self.ctx, "SAMLoader")
with self.assertRaises(ToolError):
mcp_canvas_edit.new_workflow(self.ctx, "x", {"1": {"class_type": "KlingTextToVideo", "inputs": {}}})
page.assert_not_called()
mcp_canvas_edit.add_node(self.ctx, "KSampler")
page.assert_called_once()
def test_a_whole_pack_allows_its_nodes_lists_them_first_and_in_full(self):
ai_toolbox.pick(self.task, {"packs": ["ComfyUI-Impact-Pack"]}, "only")
self.assertEqual(ai_toolbox.refused(self.task, ["FaceDetailer", "SAMLoader", "KSampler", "KlingTextToVideo"], INFO),
["KlingTextToVideo"], "every node of the pack, and the built-in ones")
with mock.patch.object(mcp_canvas_edit, "_page", return_value={}):
with self.assertRaisesRegex(ToolError, "built-in ones\): every node of the pack ComfyUI-Impact-Pack"):
mcp_canvas_edit.add_node(self.ctx, "KlingTextToVideo")
found = mcp_canvas_edit.find_nodes(self.ctx, query="e")
self.assertEqual({node["type"] for node in found["nodes"][:2]}, {"FaceDetailer", "SAMLoader"})
self.assertTrue(all(node["picked_by_user"] for node in found["nodes"][:2]))
box = mcp_task_tools.get_toolbox(self.ctx)
self.assertEqual(box["packs"], [{"pack": "ComfyUI-Impact-Pack", "count": 2, "node_types": [
{"type": "FaceDetailer", "display_name": "FaceDetailer", "category": ""},
{"type": "SAMLoader", "display_name": "SAMLoader", "category": ""}]}])
self.assertIn("custom node packs", box["note"])
def test_prefer_refuses_nothing_and_find_nodes_lists_picks_first(self):
ai_toolbox.pick(self.task, {"nodes": ["SAMLoader"]}, "prefer")
self.assertEqual(ai_toolbox.refused(self.task, ["KlingTextToVideo"], INFO), [])
found = mcp_canvas_edit.find_nodes(self.ctx, query="e")
self.assertEqual(found["nodes"][0]["type"], "SAMLoader")
self.assertTrue(found["nodes"][0]["picked_by_user"])
paid = next(node for node in found["nodes"] if node["type"] == "KlingTextToVideo")
self.assertEqual((paid["source"], paid.get("paid")), ("api", True))
self.assertEqual(found["nodes"][-1]["type"], "KlingTextToVideo", "paid nodes last")
def test_get_toolbox_gives_the_picks_in_full(self):
self.assertFalse(mcp_task_tools.get_toolbox(self.ctx)["picked"])
ai_toolbox.pick(self.task, {"nodes": ["FaceDetailer", "Fast Groups Muter (rgthree)"], "models": ["loras:0:gone.safetensors"]}, "only")
with mock.patch("api.mcp_tools.get_model", side_effect=ToolError("gone")):
box = mcp_task_tools.get_toolbox(self.ctx)
self.assertEqual((box["mode"], box["nodes"][0]["type"], box["page_only_nodes"]), ("only", "FaceDetailer", ["Fast Groups Muter (rgthree)"]))
self.assertTrue(box["models"][0]["missing"])
def test_the_picker_gets_everything_in_one_answer_and_prompts_inside_combos_and_workflows(self):
from api import ai_toolbox_catalog as catalog
combos = [{"name": "赛博少女", "model": "a.safetensors", "loras": [], "base": "", "prompt": "1girl, neon", "structure": False,
"cover": "/anomalous/image?x", "image": None}]
flows = [{"name": "高清修复", "filename": "f.json", "tags": ["SDXL"], "prompt": "city", "negative": "blurry"},
{"name": "空的", "filename": "e.json", "tags": [], "prompt": "", "negative": ""}]
with mock.patch.object(catalog, "_combos", return_value=combos), mock.patch.object(catalog, "_workflows", return_value=flows), \
mock.patch.object(catalog, "_models", return_value=[{"id": "loras:0:x.safetensors"}]), \
mock.patch("api.mcp_tools.saved_prompts", return_value=iter([{"name": "夜景", "tags": [], "positive": "night", "negative": ""}])):
data = ai_toolbox.catalog("all")
ids = [entry["id"] for entry in data["prompts"]]
self.assertEqual(ids, ["夜景", "combo:赛博少女", "workflow:高清修复"], "a workflow without prompts has none to pick")
ai_toolbox.pick(self.task, {"prompts": ["workflow:高清修复"]}, "prefer")
box = mcp_task_tools.get_toolbox(self.ctx)
self.assertEqual((box["prompts"][0]["positive"], box["prompts"][0]["negative"]), ("city", "blurry"))
self.assertIn("workflow:<name>", ai_toolbox.words(ai_toolbox.effective(ai_tasks.get_task(self.task))))
self.assertIs(ai_toolbox.catalog("all"), data, "kept a few seconds")
catalog._cache.update(at=0.0, data=None)
def test_what_is_refused(self):
for picks, mode in [(None, "prefer"), ({"nodes": "FaceDetailer"}, "prefer"), ({"nodes": [3]}, "prefer"),
({"nodes": ["x" * 400]}, "prefer"), ({"nodes": ["a"] * 0 + [str(i) for i in range(61)]}, "prefer"),
({"nodes": []}, "strict")]:
with self.assertRaises(ai_toolbox.ToolboxError, msg=(picks, mode)):
ai_toolbox.pick(self.task, picks, mode)
with self.assertRaises(ai_toolbox.ToolboxError):
ai_toolbox.pick("no-task", {}, "prefer")
with self.assertRaises(ai_toolbox.ToolboxError):
ai_toolbox.catalog("nodes")
if __name__ == "__main__":
unittest.main()
+171 -12
View File
@@ -13,9 +13,19 @@ from aiohttp.test_utils import make_mocked_request
PLUGIN = Path(__file__).resolve().parents[1]
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
from api import lora_info, mcp_actions, mcp_bridge, mcp_server, mcp_tools # noqa: E402
from api import ( # noqa: E402
ai_gates, ai_panels, ai_tasks, lora_info, mcp_actions, mcp_apps, mcp_bridge, mcp_canvas_edit, mcp_model_use, mcp_server,
mcp_tools,
)
MODERN = "2026-07-28"
CTX = {"base_url": "http://127.0.0.1:1"}
# Every call is kept as an AI workbench step: keep them in a temporary folder, never the user's.
_AI_TASKS = tempfile.mkdtemp(prefix="amb_mcp_ai_tasks_")
ai_tasks.tasks_dir = lambda: _AI_TASKS
ai_tasks._tasks = None
_GATES = tempfile.mkdtemp(prefix="amb_mcp_gates_")
ai_gates._path = lambda: os.path.join(_GATES, "ai_gates.json")
def meta(version=MODERN):
@@ -44,6 +54,12 @@ def modern_headers(method, name=None):
class ProtocolTests(unittest.TestCase):
def test_non_string_tool_names_return_protocol_error(self):
for name in ([], {}, 123):
status, body = post({"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": name}})
self.assertEqual(status, 200)
self.assertEqual(body["error"]["code"], -32602)
def test_legacy_clients_initialize_then_list_tools(self):
status, body = post({"jsonrpc": "2.0", "id": 1, "method": "initialize",
"params": {"protocolVersion": "2025-06-18", "capabilities": {}, "clientInfo": {"name": "c", "version": "1"}}})
@@ -61,6 +77,10 @@ class ProtocolTests(unittest.TestCase):
status, body = post({"jsonrpc": "2.0", "id": 3, "method": "initialize", "params": {"protocolVersion": "2030-01-01"}})
self.assertEqual(body["result"]["protocolVersion"], "2025-11-25")
def test_every_tool_is_listed(self):
status, body = post({"jsonrpc": "2.0", "id": 1, "method": "tools/list"}, {"MCP-Protocol-Version": "2025-06-18"})
self.assertEqual({tool["name"] for tool in body["result"]["tools"]}, set(mcp_server.TOOLS), "AI apps pick what they need themselves")
def test_modern_requests_are_stateless_and_headers_must_match(self):
status, body = post({"jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {"_meta": meta()}},
modern_headers("tools/list"))
@@ -90,8 +110,8 @@ class ProtocolTests(unittest.TestCase):
{"MCP-Protocol-Version": "2030-01-01", "Mcp-Method": "tools/list"})
self.assertEqual((status, body["error"]["code"]), (400, -32022))
self.assertIn(MODERN, body["error"]["data"]["supported"])
status, body = post({"jsonrpc": "2.0", "id": 2, "method": "resources/list", "params": {"_meta": meta()}},
modern_headers("resources/list"))
status, body = post({"jsonrpc": "2.0", "id": 2, "method": "sampling/createMessage", "params": {"_meta": meta()}},
modern_headers("sampling/createMessage"))
self.assertEqual((status, body["error"]["code"]), (404, -32601))
self.assertEqual(post({}, method="GET")[0], 405)
self.assertEqual(post([{"jsonrpc": "2.0", "id": 1, "method": "ping"}])[0], 400) # no batches
@@ -123,18 +143,26 @@ class ToolTests(unittest.TestCase):
metas = {str(lora): {"name": "Neon City", "baseModel": "SDXL 1.0", "trainedWords": ["neon"], "info_source": "civitai",
"description": "<p>Bright <b>lights</b></p>", "custom_notes": "mine"},
str(self.root / "plain.ckpt"): {"info_source": ""}}
# ComfyUI's LoRA loader lists the whole loras folder (with its own separators).
listed = ["plain.ckpt", "sub\\neon_city.safetensors"]
info = {"LoraLoader": {"input": {"required": {"lora_name": [listed]}}}}
tools = mcp_model_use.MODEL_TOOLS
with mock.patch.object(mcp_tools, "_iter_search_models", return_value=files), \
mock.patch.object(mcp_tools, "get_metadata", side_effect=lambda path: metas[path]), \
mock.patch.object(mcp_tools, "resolve_folder_subdir", side_effect=lambda t, i, s: (str(self.root), str(self.root / s.strip("/")))):
found = mcp_tools.call_tool(mcp_tools.TOOLS, "search_models", {"query": "neon sdxl"})["structuredContent"]
mock.patch.object(mcp_tools, "resolve_folder_subdir", side_effect=lambda t, i, s: (str(self.root), str(self.root / s.strip("/")))), \
mock.patch.object(mcp_canvas_edit, "object_info", return_value=info), \
mock.patch.object(mcp_model_use.folder_paths, "get_filename_list", return_value=listed):
found = mcp_tools.call_tool(tools, "search_models", {"query": "neon sdxl"}, CTX)["structuredContent"]
self.assertEqual(found["total_matches"], 1)
model = found["models"][0]
self.assertEqual((model["model_id"], model["source"]), ("loras:0:sub/neon_city.safetensors", "civitai"))
self.assertEqual(mcp_tools.call_tool(mcp_tools.TOOLS, "search_models", {"source": "not_scanned"})["structuredContent"]["total_matches"], 1)
detail = mcp_tools.call_tool(mcp_tools.TOOLS, "get_model", {"model_id": model["model_id"]})["structuredContent"]
self.assertEqual((detail["description"], detail["user_notes"]), ("Bright lights", "mine"))
self.assertEqual((model["model_id"], model["source"], model["loaded_by"]), ("loras:0:sub/neon_city.safetensors", "civitai", ["LoraLoader"]))
self.assertEqual(mcp_tools.call_tool(tools, "search_models", {"source": "not_scanned"}, CTX)["structuredContent"]["total_matches"], 1)
for kind, count in (("lora", 2), ("LoRAs", 2), ("checkpoints", 0)):
self.assertEqual(mcp_tools.call_tool(tools, "search_models", {"type": kind}, CTX)["structuredContent"]["total_matches"], count, kind)
detail = mcp_tools.call_tool(tools, "get_model", {"model_id": model["model_id"]}, CTX)["structuredContent"]
self.assertEqual((detail["description"], detail["user_notes"], detail["loaded_by"]), ("Bright lights", "mine", ["LoraLoader"]))
for bad in ("loras:0:../../secret.safetensors", "loras:0:sub/missing.safetensors", "nonsense"):
result = mcp_tools.call_tool(mcp_tools.TOOLS, "get_model", {"model_id": bad})
result = mcp_tools.call_tool(tools, "get_model", {"model_id": bad}, CTX)
self.assertTrue(result["isError"], bad)
def test_image_settings_come_from_the_png_and_stay_inside_output(self):
@@ -154,9 +182,26 @@ class ToolTests(unittest.TestCase):
self.assertEqual(data["kept_as"], {"combo": "C"})
self.assertTrue(mcp_tools.call_tool(mcp_tools.TOOLS, "get_image_info", {"image": "../out.png"})["isError"])
def test_a_picture_in_the_input_folder_and_its_mask(self):
from PIL import Image
picture = Image.new("RGBA", (100, 50), (200, 100, 50, 255))
picture.paste((200, 100, 50, 0), (10, 5, 30, 25)) # transparent: LoadImage's MASK, what to repaint
picture.save(self.root / "anomalous_inpaint_x.png")
Image.new("RGB", (8, 8)).save(self.root / "plain.png")
with mock.patch.object(mcp_tools.folder_paths, "get_input_directory", return_value=str(self.root)):
data = mcp_tools.call_tool(mcp_tools.TOOLS, "get_image_info", {"image": "anomalous_inpaint_x.png", "folder": "input", "include_image": True})
info = data["structuredContent"]
self.assertEqual((info["folder"], info["width"], info["height"]), ("input", 100, 50))
self.assertEqual((info["mask"]["share"], info["mask"]["area"]), ("8.0%", "x 10-30%, y 10-50% of the picture"))
self.assertTrue(info["image_attached"])
plain = mcp_tools.call_tool(mcp_tools.TOOLS, "get_image_info", {"image": "plain.png", "folder": "input"})["structuredContent"]
self.assertTrue(plain["mask"].startswith("none"))
for bad in ("../plain.png", "missing.png"):
self.assertTrue(mcp_tools.call_tool(mcp_tools.TOOLS, "get_image_info", {"image": bad, "folder": "input"})["isError"], bad)
def test_bad_arguments_are_tool_errors(self):
self.assertTrue(mcp_tools.call_tool(mcp_tools.TOOLS, "get_model", {})["isError"])
self.assertTrue(mcp_tools.call_tool(mcp_tools.TOOLS, "search_models", {"delete": True})["isError"])
self.assertTrue(mcp_tools.call_tool(mcp_model_use.MODEL_TOOLS, "get_model", {}, CTX)["isError"])
self.assertTrue(mcp_tools.call_tool(mcp_model_use.MODEL_TOOLS, "search_models", {"delete": True}, CTX)["isError"])
class BridgeTests(unittest.TestCase):
@@ -219,6 +264,24 @@ class ServerActionTests(unittest.TestCase):
"""Actions that go through this server's own routes."""
CTX = {"base_url": "http://127.0.0.1:8188"}
def test_run_requires_exact_receipts_and_retains_partial_batches(self):
with mock.patch.object(mcp_actions, "ask_page", return_value={"prompt_ids": ["p1"], "queued": 99}) as ask:
got = mcp_actions.run_workflow(self.CTX, 2)
self.assertEqual(got["queued"], 1)
self.assertIn("submission_error", got)
self.assertEqual(ask.call_args.kwargs["required_capability"], "prompt_ids")
for reply in ({"queued": 1}, {"prompt_ids": []}, {"prompt_ids": ["a", "a"]}, {"prompt_ids": [{}]}):
with mock.patch.object(mcp_actions, "ask_page", return_value=reply), self.assertRaises(mcp_tools.ToolError):
mcp_actions.run_workflow(self.CTX, 2)
def test_old_page_cannot_claim_a_tracked_run(self):
server = mock.Mock(sockets={"test": object()})
server.send_sync.side_effect = lambda event, data: self.assertFalse(mcp_bridge.claim(data["id"]))
with mock.patch.object(mcp_bridge, "_server", return_value=server):
with self.assertRaisesRegex(mcp_bridge.PageRefused, "Refresh"):
mcp_bridge.ask_page("run_workflow", required_capability="prompt_ids", timeout=1)
self.assertEqual(mcp_bridge._waiting, {})
def test_scanning_picked_models_groups_them_by_folder(self):
calls = []
models = {"loras:0:a/x.safetensors": ("loras", 0, "/m", "/m/a/x.safetensors"),
@@ -319,5 +382,101 @@ class LoraHeaderTests(unittest.TestCase):
self.assertTrue(lora_info.trains_text_encoder(str(Path(self.temp.name) / "old.ckpt"))) # unknown: assume it does
def test_tool_call_records_in_flight_step_and_piggybacks_pending_user_input(self):
task = ai_tasks.start_task("画一张白毛魔女")
ai_panels.leave_message(task["id"], "背景加点星光")
seen_in_flight = []
def fake_summarize():
current = ai_tasks.get_task(task["id"])
seen_in_flight.append([s["state"] for s in current["steps"] if s["tool"] == "scan_report"])
return dict.fromkeys(("total", "matched", "unmatched", "new", "pending"), 0) | {"unmatched_models": [], "new_models": []}
call = {"jsonrpc": "2.0", "id": 9, "method": "tools/call",
"params": {"name": "scan_report", "arguments": {"task": task["id"]}, "_meta": meta()}}
with mock.patch.object(mcp_tools, "_summarize", side_effect=fake_summarize), \
mock.patch.object(mcp_tools, "last_scan_path", return_value="no such file"):
status, body = post(call, modern_headers("tools/call", "scan_report"))
self.assertEqual(status, 200)
self.assertEqual(seen_in_flight, [["running"]], "step is visible as running while tool executes")
steps = [s for s in ai_tasks.get_task(task["id"])["steps"] if s["tool"] == "scan_report"]
self.assertEqual(len(steps), 1, "same step updated in place without duplicates")
self.assertEqual(steps[0]["state"], "done")
msgs = body["result"]["structuredContent"].get("user_messages", [])
self.assertEqual([m["text"] for m in msgs], ["背景加点星光"])
def rpc(method, params=None, name=None):
status, body = post({"jsonrpc": "2.0", "id": 1, "method": method, "params": {**(params or {}), "_meta": meta()}},
modern_headers(method, name))
return body
class CardAndPromptTests(unittest.TestCase):
"""MCP Apps (the card in the conversation, mcp_apps.py) and prompts (mcp_prompts.py)."""
def test_the_card_is_a_resource_named_by_its_tools(self):
tools = {tool["name"]: tool for tool in rpc("tools/list")["result"]["tools"]}
self.assertEqual(tools["show_panel"]["_meta"]["ui"]["resourceUri"], mcp_apps.CARD_URI)
self.assertNotIn("_meta", tools["search_models"])
self.assertEqual(tools["app_answer"]["_meta"]["ui"]["visibility"], ["app"], "kept from the AI by apps that show cards")
listed = rpc("resources/list")["result"]["resources"]
self.assertEqual([item["uri"] for item in listed], [mcp_apps.CARD_URI])
content = rpc("resources/read", {"uri": mcp_apps.CARD_URI}, mcp_apps.CARD_URI)["result"]["contents"][0]
self.assertEqual(content["mimeType"], "text/html;profile=mcp-app")
self.assertIn("ui/initialize", content["text"])
self.assertNotIn("innerHTML", content["text"], "the card draws text as text")
self.assertEqual(rpc("resources/read", {"uri": "ui://other"}, "ui://other")["error"]["code"], -32602)
status, body = post({"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {"protocolVersion": "2025-06-18"}})
self.assertIn(mcp_apps.EXTENSION, body["result"]["capabilities"]["extensions"])
def test_the_card_reads_the_task_and_answers_questions_but_never_gates(self):
shown = rpc("tools/call", {"name": "show_panel", "arguments": {"title": "Pick", "blocks": [
{"type": "choices", "id": "w", "question": "Which?", "options": [{"id": "a", "label": "A"}, {"id": "b", "label": "B"}]}]}},
"show_panel")["result"]["structuredContent"]
before = len(ai_tasks.get_task(ai_tasks.list_tasks()[0]["id"])["steps"])
view = rpc("tools/call", {"name": "app_task_view", "arguments": {"panel": shown["panel"]}}, "app_task_view")["result"]["structuredContent"]
self.assertEqual(view["questions"][0]["options"][1], {"id": "b", "label": "B"})
self.assertTrue(view["open_url"].startswith("http://127.0.0.1"))
done = rpc("tools/call", {"name": "app_answer", "arguments": {"panel": shown["panel"], "block": "w", "choice": ["b"]}}, "app_answer")
self.assertFalse(done["result"]["isError"])
self.assertEqual(len(ai_tasks.get_task(view["task"])["steps"]), before, "the card's calls are not steps")
got = ai_panels.get_answers("test", shown["panel"])
self.assertEqual(got["answers"][0]["choice"], ["b"])
self.assertEqual(set(mcp_apps.APP_TOOLS), {"app_task_view", "app_answer"}, "no tool the AI could reach answers a gate")
def test_prompts(self):
names = [item["name"] for item in rpc("prompts/list")["result"]["prompts"]]
self.assertIn("pick_best", names)
got = rpc("prompts/get", {"name": "pick_best", "arguments": {"what": "seed"}}, "pick_best")["result"]
text = got["messages"][0]["content"]["text"]
self.assertIn("3 variations", text)
self.assertIn("seed", text)
self.assertEqual(rpc("prompts/get", {"name": "pick_best", "arguments": {}}, "pick_best")["error"]["code"], -32602)
self.assertEqual(rpc("prompts/get", {"name": "nope"}, "nope")["error"]["code"], -32602)
class ArgumentTests(unittest.TestCase):
def test_a_wrong_argument_is_answered_with_what_the_tool_takes(self):
result = rpc("tools/call", {"name": "set_prompt", "arguments": {"text": "一只猫", "colour": "red"}}, "set_prompt")["result"]
self.assertTrue(result["isError"])
text = result["content"][0]["text"]
self.assertIn("Unknown arguments ['colour', 'text']; missing ['prompt']", text)
self.assertIn("set_prompt takes: prompt (needed), role, node_id, mode", text, "the AI corrects itself in one go")
def test_the_one_needed_argument_under_another_name_is_taken(self):
tools = {"open": {"spec": {"inputSchema": {"properties": {"workflow": {"type": "string"}, "seed": {"type": "integer"}},
"required": ["workflow"]}},
"run": lambda workflow, seed=0: {"opened": workflow}}}
result = mcp_tools.call_tool(tools, "open", {"name": "exam_txt2img"})
self.assertFalse(result["isError"])
self.assertEqual(result["structuredContent"]["opened"], "exam_txt2img")
self.assertIn("Took name as workflow", result["structuredContent"]["arguments_note"], "so the next call is right")
for wrong in ({"name": 3}, {"name": "a", "other": "b"}, {"name": True}, {"style": "anime"}):
with self.subTest(arguments=wrong):
self.assertTrue(mcp_tools.call_tool(tools, "open", wrong)["isError"], "another type, more than one guess, or a name not known to mean it: asked again")
if __name__ == "__main__":
unittest.main()

Some files were not shown because too many files have changed in this diff Show More