AI workbench: record in-flight MCP steps and piggyback pending user answers/messages
This commit is contained in:
+2
-2
@@ -147,7 +147,7 @@ covered by the `styles.css` manifest.
|
||||
instead of joining its last task (one waiting at a time, found again after a restart).
|
||||
`begin_task` resumes a waiting or active task, including preliminary reads, and keeps a user's title;
|
||||
`new_task=true` explicitly separates a new request. Ended or idle tasks are not resumed.
|
||||
`mcp_server.py` records each call after it ran (the app's name from its client
|
||||
`mcp_server.py` starts an in-flight `"running"` step before each call and updates the same step in place when it finishes or fails (the app's name from its client
|
||||
info or User-Agent); `api/mcp_task_tools.py` holds the AI's own tools for it, `begin_task`
|
||||
and `report` (a note, with pictures from the output folder it made another way).
|
||||
`web/modules/ui_ai_workbench.js` is the page (rail entry `ai`): the tasks, one task's
|
||||
@@ -188,7 +188,7 @@ covered by the `styles.css` manifest.
|
||||
in through `POST /anomalous/ai_tasks/{id}/answer` and wait there until the AI reads them
|
||||
(`get_answers`, which can wait up to 55 s for one; Anomalous cannot call the AI). The user
|
||||
can also write to the AI on a task (`POST /anomalous/ai_tasks/{id}/message`, a `user` step);
|
||||
`get_answers` brings those messages too, and with no panel named waits for one.
|
||||
`get_answers` brings those messages too, and with no panel named waits for one; unread answers and user messages are also piggybacked once onto the next successful MCP tool response (`ai_panels.attach_pending`) so AI apps that do not poll `get_answers` still receive them immediately.
|
||||
`web/modules/ui_ai_panel.js` draws a panel in the timeline (textContent only) and the toast
|
||||
when the AI shows one while the user is elsewhere. The stage blocks are data the page lays
|
||||
out, never drawings of the AI's own (no SVG is taken): marks on pictures (circles or boxes at
|
||||
|
||||
+3
-1
@@ -2,7 +2,9 @@
|
||||
|
||||
## v1.62.0 — unreleased
|
||||
|
||||
### Fixes / 修复
|
||||
### Fixes / 修复
|
||||
- AI 工作台与 MCP 调用实时生命周期及消息送达:每个工具调用开始时立即写入“正在执行…”步骤并接手等待中的任务,完成后原位更新同一步骤(不再等到耗时操作结束后才显示,也不会重复生成步骤);用户在工作台留下的消息和展台面板回答,除了可通过 `get_answers` 读取外,还会在 AI 下一次成功调用任意工具时自动附带送达一次,并立即在界面上更新为“AI 已经看到”;多会话与显式 `task` ID 互不串任务。
|
||||
- Live in-flight MCP step lifecycle and automatic delivery of user messages and panel answers: every tool call now starts a `"running"` step immediately and updates the same step in place on completion or failure; unread user messages and panel answers are piggybacked once onto the next successful MCP tool response (in addition to `get_answers`) and immediately notify the workbench UI; concurrent sessions and explicit `task` ids stay isolated.
|
||||
- 展台内容按出现顺序保留:`update_panel` 新增的成图、说明等接在已有内容下面,即使 AI 把整张看板按“结果在前”重发,也不会倒置旧内容。省略旧内容不再将它删除;重复提交不重复追加。明确指定积木 ID 时仍可原位更新进度、问题或修正内容,已有回答保留。超出 30 块时提示新开面板,不截掉旧内容。旧记录保留其已保存的排版。
|
||||
- Panel updates append new output after what the user already saw, preserve omitted blocks, and avoid duplicating unchanged resubmissions. Stable block ids allow deliberate in-place edits; invalid or over-limit updates leave the panel untouched. Historical records keep their saved layout.
|
||||
- 展台右上角增加“关闭展台”,关闭后时间线展开;任务标题旁的“打开展台”或时间线里的面板入口可重新打开,保留面板和未提交的输入。当前任务更新不会强行重开。面板编号改为“来自第 N 步”,明确它的创建位置。The stage can be closed and reopened without losing panels or drafts; step labels now say “From step N”.
|
||||
|
||||
+74
-27
@@ -13,12 +13,13 @@ numbers of bounded size, pictures only from the output folder and models only fr
|
||||
folders. The page draws them (ui_ai_panel.js).
|
||||
|
||||
The user's answers come back through POST /anomalous/ai_tasks/{task}/answer and wait in the
|
||||
step until the AI asks for them (get_answers, which may wait a while for one). The user can
|
||||
also write to the AI in a task (POST /anomalous/ai_tasks/{task}/message): the message is a
|
||||
step of its own and reaches the AI the same way. Anomalous cannot call the AI itself; the page
|
||||
says so where it asks.
|
||||
step until the AI asks for them (get_answers, which may wait a while for one) or makes its next
|
||||
successful MCP tool call (attach_pending). The user can also write to the AI in a task
|
||||
(POST /anomalous/ai_tasks/{task}/message): the message is a step of its own and reaches the AI
|
||||
the same way.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
@@ -321,14 +322,14 @@ def _find(panel_id):
|
||||
return None, None
|
||||
|
||||
|
||||
def show(client, title, blocks, now=None):
|
||||
def show(client, title, blocks, now=None, session="", task_id=None):
|
||||
title = _text(title, 120, "title", True)
|
||||
panel = {"id": uuid.uuid4().hex[:10], "title": title, "blocks": _block_ids(clean_blocks(blocks)), "closed": False}
|
||||
now = time.time() if now is None else now
|
||||
step = {"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": "show_panel", "kind": "panel", "ok": True,
|
||||
"summary": f"show_panel: {title}", "panel": panel, "answers": {}, "results": []}
|
||||
with ai_tasks._lock:
|
||||
task = ai_tasks.current_task(client, now)
|
||||
task = ai_tasks.current_task(client, now, session=session, task_id=task_id)
|
||||
ai_tasks._add_step(task, step)
|
||||
ai_tasks._save(task)
|
||||
ai_tasks._notify(task["id"], {"panel": title, "asks": any(b["type"] in INTERACTIVE for b in panel["blocks"])})
|
||||
@@ -412,19 +413,27 @@ def leave_message(task_id, text, now=None):
|
||||
return step
|
||||
|
||||
|
||||
def _collect(client, panel_id):
|
||||
"""Answers and messages not given to the AI yet (its own tasks, or panel `panel_id` and the
|
||||
messages of that panel's task), marked as given. Under the lock."""
|
||||
def _collect(client, panel_id, task_id=None):
|
||||
"""Answers and messages not given to the AI yet (its own tasks, task `task_id`, or panel
|
||||
`panel_id` and the messages of that panel's task), marked as given. Under the lock."""
|
||||
out, waiting, messages, changed = [], [], [], set()
|
||||
tasks = ai_tasks._load()
|
||||
panel_task = _find(panel_id)[0] if panel_id is not None else None
|
||||
for task in ai_tasks._load().values():
|
||||
if panel_id is None and task.get("client") != client:
|
||||
named_task = tasks.get(task_id) if task_id else None
|
||||
for task in tasks.values():
|
||||
if panel_id is not None:
|
||||
if task is not panel_task:
|
||||
continue
|
||||
elif named_task is not None:
|
||||
if task is not named_task and task.get("client") != client:
|
||||
continue
|
||||
elif task.get("client") != client:
|
||||
continue
|
||||
for step in task.get("steps") or []:
|
||||
if step.get("kind") == "user" and not step.get("delivered") and (panel_id is None or task is panel_task):
|
||||
step["delivered"] = True
|
||||
changed.add(task["id"])
|
||||
messages.append({"task": task["id"], "text": step["text"], "at": step["at"]})
|
||||
messages.append({"id": step["id"], "task": task["id"], "text": step["text"], "at": step["at"]})
|
||||
continue
|
||||
panel = step.get("panel")
|
||||
if step.get("kind") != "panel" or (panel_id is not None and panel["id"] != panel_id):
|
||||
@@ -444,25 +453,63 @@ def _collect(client, panel_id):
|
||||
item = {"panel": panel["id"], "question": block["id"], "asked": block.get("question", "")}
|
||||
item.update({key: given[key] for key in ("choice", "labels", "text") if key in given})
|
||||
out.append(item)
|
||||
for task_id in changed:
|
||||
ai_tasks._save(ai_tasks._load()[task_id])
|
||||
return out, waiting, messages
|
||||
for tid in changed:
|
||||
ai_tasks._save(tasks[tid])
|
||||
return out, waiting, messages, changed
|
||||
|
||||
|
||||
def get_answers(client, panel_id=None, wait_seconds=0):
|
||||
def get_answers(client, panel_id=None, wait_seconds=0, task_id=None):
|
||||
"""The user's new answers and messages; with `wait_seconds`, waits up to that long (MAX_WAIT)
|
||||
for one: for a panel's question still open, or (no panel named) for anything the user sends."""
|
||||
deadline = time.monotonic() + max(0, min(float(wait_seconds or 0), MAX_WAIT))
|
||||
panel_id = str(panel_id) if panel_id else None
|
||||
notified = set()
|
||||
try:
|
||||
with _answered:
|
||||
if panel_id is not None and _find(panel_id)[1] is None:
|
||||
raise PanelError(f"No panel {panel_id!r}; show_panel returns its id.")
|
||||
while True:
|
||||
answers, waiting, messages, changed = _collect(client, panel_id, task_id=task_id)
|
||||
notified.update(changed)
|
||||
left = deadline - time.monotonic()
|
||||
if answers or messages or left <= 0 or (panel_id is not None and not waiting):
|
||||
answer = {"answers": answers, "still_waiting": waiting}
|
||||
if messages:
|
||||
answer["messages"] = messages
|
||||
return answer
|
||||
_answered.wait(left)
|
||||
finally:
|
||||
for tid in notified:
|
||||
ai_tasks._notify(tid)
|
||||
|
||||
|
||||
def attach_pending(result, client, task_id=None):
|
||||
"""Attaches unread user messages and panel answers to a successful tool result so an AI
|
||||
that does not call get_answers still receives them once."""
|
||||
if not isinstance(result, dict) or result.get("isError"):
|
||||
return result
|
||||
with _answered:
|
||||
if panel_id is not None and _find(panel_id)[1] is None:
|
||||
raise PanelError(f"No panel {panel_id!r}; show_panel returns its id.")
|
||||
while True:
|
||||
answers, waiting, messages = _collect(client, panel_id)
|
||||
left = deadline - time.monotonic()
|
||||
if answers or messages or left <= 0 or (panel_id is not None and not waiting):
|
||||
answer = {"answers": answers, "still_waiting": waiting}
|
||||
if messages:
|
||||
answer["messages"] = messages
|
||||
return answer
|
||||
_answered.wait(left)
|
||||
answers, waiting, messages, changed = _collect(client, None, task_id=task_id)
|
||||
for tid in changed:
|
||||
ai_tasks._notify(tid)
|
||||
if not answers and not messages:
|
||||
return result
|
||||
data = result.get("structuredContent")
|
||||
data = dict(data) if isinstance(data, dict) else {}
|
||||
if messages:
|
||||
data["user_messages"] = messages
|
||||
if answers:
|
||||
data["user_answers"] = answers
|
||||
if waiting:
|
||||
data["still_waiting"] = waiting
|
||||
data["user_notice"] = (
|
||||
"The user sent new input in the AI workbench. Read user_messages / user_answers and follow them."
|
||||
)
|
||||
result["structuredContent"] = data
|
||||
content = list(result.get("content") or [])
|
||||
if content and isinstance(content[0], dict) and content[0].get("type") == "text":
|
||||
content[0] = {"type": "text", "text": json.dumps(data, ensure_ascii=False)}
|
||||
else:
|
||||
content.insert(0, {"type": "text", "text": json.dumps(data, ensure_ascii=False)})
|
||||
result["content"] = content
|
||||
return result
|
||||
|
||||
+139
-74
@@ -2,11 +2,12 @@
|
||||
|
||||
Every tool call an AI app makes (mcp_server.py) becomes a step of a task: a task is one piece
|
||||
of work, named by the AI with begin_task, else the calls of one app that follow each other
|
||||
without a long pause. A step keeps what was done (the tool, a short summary of its arguments),
|
||||
whether it worked, and what came out of it, so the page can show it and lead there: the
|
||||
models it used, the combo it placed, the canvas change it made (by its activity log entry,
|
||||
for undo), the pictures a run made (watched in ComfyUI's history until the run ends), the
|
||||
audio it spoke. The AI can also leave a note (report), with pictures it made another way.
|
||||
without a long pause. A step starts in state "running" when the call begins and is updated in
|
||||
place when it finishes: what was done (the tool, a short summary of its arguments), whether it
|
||||
worked, and what came out of it, so the page can show it live and lead there: the models it
|
||||
used, the combo it placed, the canvas change it made (by its activity log entry, for undo),
|
||||
the pictures a run made (watched in ComfyUI's history until the run ends), the audio it spoke.
|
||||
The AI can also leave a note (report), with pictures it made another way.
|
||||
|
||||
The user can start a task first (New task, POST /anomalous/ai_tasks): it waits for an AI, and
|
||||
the next app that calls takes it over, so the work it does next starts there instead of joining
|
||||
@@ -57,7 +58,7 @@ OWN_STEPS = {"report", "show_panel"} # make their step themselves (with its tex
|
||||
|
||||
_lock = threading.RLock()
|
||||
_tasks = None # id -> task
|
||||
_current = {} # client name -> id of the task its calls go to
|
||||
_current = {} # client/session key -> id of the task its calls go to
|
||||
_waiting = [None] # id of the task the user started that no AI has taken yet
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
@@ -73,11 +74,9 @@ def tasks_dir():
|
||||
os.makedirs(directory, exist_ok=True)
|
||||
return directory
|
||||
|
||||
|
||||
def _path(task_id):
|
||||
return os.path.join(tasks_dir(), f"{task_id}.json")
|
||||
|
||||
|
||||
def _load():
|
||||
global _tasks
|
||||
if _tasks is None:
|
||||
@@ -98,14 +97,12 @@ def _load():
|
||||
_waiting[0] = max(waiting, key=lambda task: task.get("started", 0))["id"]
|
||||
return _tasks
|
||||
|
||||
|
||||
def _save(task):
|
||||
try:
|
||||
atomic_write_json(_path(task["id"]), task, MAX_TASK_BYTES)
|
||||
except (OSError, ValueError) as error:
|
||||
log.warning("Anomalous AI tasks: %s not saved: %s", task["id"], error)
|
||||
|
||||
|
||||
def _drop_oldest():
|
||||
tasks = _load()
|
||||
if len(tasks) <= MAX_TASKS:
|
||||
@@ -117,11 +114,9 @@ def _drop_oldest():
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _notify(task_id, extra=None):
|
||||
try:
|
||||
from server import PromptServer
|
||||
|
||||
PromptServer.instance.send_sync("anomalous.ai_task", {"id": task_id, **(extra or {})})
|
||||
except Exception: # noqa: BLE001 - tests and early start have no server; the page polls on open
|
||||
pass
|
||||
@@ -134,7 +129,6 @@ _CLIENT_NAMES = (
|
||||
("windsurf", "Windsurf"), ("codex", "Codex"), ("gemini", "Gemini"), ("vscode", "VS Code"), ("cline", "Cline"),
|
||||
)
|
||||
|
||||
|
||||
def client_of(headers, params):
|
||||
"""A short name for the AI app making a call: the client info it sends, else its User-Agent."""
|
||||
meta = params.get("_meta") if isinstance(params.get("_meta"), dict) else {}
|
||||
@@ -149,6 +143,27 @@ def client_of(headers, params):
|
||||
clean = re.sub(r"[^\w .-]", "", raw).strip()[:40]
|
||||
return clean or "AI"
|
||||
|
||||
def session_of(headers, params):
|
||||
"""An optional session key when one app runs multiple MCP sessions at once."""
|
||||
meta = params.get("_meta") if isinstance(params.get("_meta"), dict) else {}
|
||||
raw = headers.get("Mcp-Session-Id") or headers.get("X-Anomalous-Session") or meta.get("sessionId") or meta.get("conversationId") or ""
|
||||
return re.sub(r"[^\w .:-]", "", str(raw)).strip()[:64] if isinstance(raw, str) else ""
|
||||
|
||||
def task_of(headers, params, arguments=None):
|
||||
"""An explicit task id from headers, request metadata or tool arguments."""
|
||||
meta = params.get("_meta") if isinstance(params.get("_meta"), dict) else {}
|
||||
arg_task = arguments.get("task") if isinstance(arguments, dict) else None
|
||||
raw = arg_task or headers.get("X-Anomalous-Task") or meta.get("anomalous/task") or meta.get("task") or ""
|
||||
return re.sub(r"[^\w-]", "", str(raw)).strip()[:64] if isinstance(raw, str) else ""
|
||||
|
||||
def _owner_key(client, session=""):
|
||||
return f"{client}::{session}" if session else client
|
||||
|
||||
def _bind_owner(client, session, task_id):
|
||||
_current[_owner_key(client, session)] = task_id
|
||||
if not session or client not in _current:
|
||||
_current[client] = task_id
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- tasks and steps
|
||||
|
||||
@@ -156,34 +171,49 @@ def _short(value, limit=MAX_TEXT):
|
||||
text = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False)
|
||||
return text if len(text) <= limit else text[:limit] + "…"
|
||||
|
||||
|
||||
def _new_task(client, title="", now=None):
|
||||
def _new_task(client, title="", now=None, session=""):
|
||||
now = time.time() if now is None else now
|
||||
task = {"format": FORMAT, "id": f"{int(now)}_{uuid.uuid4().hex[:8]}", "title": _short(title, MAX_TITLE),
|
||||
"title_from": "ai" if title else "auto", "client": client, "started": now, "updated": now,
|
||||
"ended": None, "steps": []}
|
||||
if session:
|
||||
task["session"] = session
|
||||
_load()[task["id"]] = task
|
||||
_current[client] = task["id"]
|
||||
_bind_owner(client, session, task["id"])
|
||||
_drop_oldest()
|
||||
return task
|
||||
|
||||
def _claim_explicit(tasks, task_id, client, session=""):
|
||||
if not task_id or task_id not in tasks or tasks[task_id].get("ended"):
|
||||
return None
|
||||
picked = tasks[task_id]
|
||||
if picked.get("waiting"):
|
||||
if _waiting[0] == task_id:
|
||||
_waiting[0] = None
|
||||
picked["waiting"] = False
|
||||
if not picked.get("client"):
|
||||
picked["client"] = client
|
||||
_bind_owner(client, session, task_id)
|
||||
return picked
|
||||
|
||||
def _take_waiting(client, now):
|
||||
def _take_waiting(client, now, session=""):
|
||||
"""The task the user started, now this app's (its own open task ends), or None."""
|
||||
task = _load().get(_waiting[0])
|
||||
_waiting[0] = None
|
||||
if not task or task.get("ended") or now - task.get("started", 0) > WAITING_SECONDS:
|
||||
return None
|
||||
previous = _load().get(_current.get(client))
|
||||
owner = _owner_key(client, session)
|
||||
previous = _load().get(_current.get(owner) or (None if session else _current.get(client)))
|
||||
if previous and previous is not task and not previous.get("ended"):
|
||||
previous["ended"] = previous.get("updated")
|
||||
_save(previous)
|
||||
task["client"] = client
|
||||
if session:
|
||||
task["session"] = session
|
||||
task["waiting"] = False
|
||||
_current[client] = task["id"]
|
||||
_bind_owner(client, session, task["id"])
|
||||
return task
|
||||
|
||||
|
||||
def start_task(title="", now=None):
|
||||
"""A task the user starts; it waits for the next AI app that calls. Returns the task."""
|
||||
now = time.time() if now is None else now
|
||||
@@ -202,55 +232,68 @@ def start_task(title="", now=None):
|
||||
_notify(task["id"])
|
||||
return task
|
||||
|
||||
def active_task_id(client, session="", task_id=None):
|
||||
"""The active task id for this caller without creating a new task."""
|
||||
with _lock:
|
||||
tasks = _load()
|
||||
if task_id and task_id in tasks:
|
||||
return task_id
|
||||
found = _current.get(_owner_key(client, session)) or (None if session else _current.get(client))
|
||||
return found if found in tasks else None
|
||||
|
||||
def current_task(client, now=None):
|
||||
"""The task this app's calls go to now: one the user started for the next AI, else its open
|
||||
one, unless it paused too long."""
|
||||
def current_task(client, now=None, session="", task_id=None):
|
||||
"""The task this app's calls go to now: an explicit task, one the user started for the next
|
||||
AI, else its open one, unless it paused too long."""
|
||||
now = time.time() if now is None else now
|
||||
with _lock:
|
||||
waiting = _take_waiting(client, now)
|
||||
tasks = _load()
|
||||
picked = _claim_explicit(tasks, task_id, client, session)
|
||||
if picked:
|
||||
return picked
|
||||
waiting = _take_waiting(client, now, session=session)
|
||||
if waiting:
|
||||
return waiting
|
||||
task = _load().get(_current.get(client))
|
||||
task = tasks.get(_current.get(_owner_key(client, session)) or (None if session else _current.get(client)))
|
||||
if task and not task.get("ended") and now - task.get("updated", 0) < IDLE_SECONDS:
|
||||
_bind_owner(client, session, task["id"])
|
||||
return task
|
||||
if task and not task.get("ended"):
|
||||
task["ended"] = task.get("updated")
|
||||
_save(task)
|
||||
return _new_task(client, now=now)
|
||||
return _new_task(client, now=now, session=session)
|
||||
|
||||
|
||||
def begin_task(client, title, now=None, new_task=False):
|
||||
def begin_task(client, title, now=None, new_task=False, session="", task_id=None):
|
||||
"""Name or resume the active task; only an explicit new piece of work splits it."""
|
||||
now = time.time() if now is None else now
|
||||
with _lock:
|
||||
task = _take_waiting(client, now)
|
||||
previous = _load().get(_current.get(client))
|
||||
if task is None and not new_task and previous and not previous.get("ended"):
|
||||
if now - previous.get("updated", 0) < IDLE_SECONDS:
|
||||
task = previous
|
||||
tasks = _load()
|
||||
previous = tasks.get(_current.get(_owner_key(client, session)) or (None if session else _current.get(client)))
|
||||
task = None if new_task else _claim_explicit(tasks, task_id, client, session)
|
||||
if task is None:
|
||||
task = _take_waiting(client, now, session=session)
|
||||
if task is None and not new_task and previous and not previous.get("ended") and now - previous.get("updated", 0) < IDLE_SECONDS:
|
||||
task = previous
|
||||
if task is None:
|
||||
if previous and not previous.get("ended"):
|
||||
previous["ended"] = now
|
||||
_save(previous)
|
||||
task = _new_task(client, title, now)
|
||||
task = _new_task(client, title, now, session=session)
|
||||
elif not task.get("title") or task.get("title_from") == "auto":
|
||||
task["title"] = _short(title, MAX_TITLE)
|
||||
task["title_from"] = "ai"
|
||||
_bind_owner(client, session, task["id"])
|
||||
task["updated"] = now
|
||||
_save(task)
|
||||
_notify(task["id"])
|
||||
return task
|
||||
|
||||
|
||||
def _add_step(task, step):
|
||||
task["steps"].append(step)
|
||||
del task["steps"][:-MAX_STEPS]
|
||||
task["updated"] = step["at"] + step.get("ms", 0) / 1000
|
||||
task["updated"] = max(task.get("updated", 0), step["at"] + step.get("ms", 0) / 1000)
|
||||
if task.get("title_from") == "auto" and not task.get("title") and step["kind"] not in ("look", "note") and step["ok"]:
|
||||
task["title"] = step["summary"][:MAX_TITLE]
|
||||
|
||||
|
||||
def summary_of(tool, arguments):
|
||||
"""A few words on what a call asked for: its main argument."""
|
||||
for key in ("title", "query", "text", "name", "character", "model_id", "url", "what", "names"):
|
||||
@@ -258,10 +301,8 @@ def summary_of(tool, arguments):
|
||||
return _short(f"{tool}: {_short(arguments[key], 120)}", 160)
|
||||
return tool
|
||||
|
||||
|
||||
def _model_result(model_id):
|
||||
from .mcp_tools import _resolve_model
|
||||
|
||||
try:
|
||||
folder_type, path_idx, base_dir, file_path = _resolve_model(model_id)
|
||||
except Exception: # noqa: BLE001 - a model that went away is still worth its id
|
||||
@@ -270,7 +311,6 @@ def _model_result(model_id):
|
||||
return {"type": "model", "model_id": str(model_id), "folder": folder_type, "path_idx": path_idx, "rel": rel,
|
||||
"name": os.path.basename(file_path)}
|
||||
|
||||
|
||||
def results_of(tool, arguments, data):
|
||||
"""What came out of a call, for the page to show and lead to."""
|
||||
data = data if isinstance(data, dict) else {}
|
||||
@@ -289,7 +329,6 @@ def results_of(tool, arguments, data):
|
||||
out.append({"type": "download"})
|
||||
return out[:MAX_RESULTS]
|
||||
|
||||
|
||||
def _count_of(data):
|
||||
"""How many things a read found: the length of its first list."""
|
||||
if isinstance(data, dict):
|
||||
@@ -298,32 +337,63 @@ def _count_of(data):
|
||||
return len(value)
|
||||
return None
|
||||
|
||||
|
||||
def record(client, tool, arguments, result, started, read_only=False, run_before=None):
|
||||
"""One tool call as a step of the app's current task. Never raises."""
|
||||
def begin_step(client, tool, arguments, started=None, read_only=False, session="", task_id=None):
|
||||
"""Starts an in-flight step in state 'running' before a tool executes. Returns (task_id, step_id) or None."""
|
||||
if tool in QUIET or tool in OWN_STEPS:
|
||||
return None
|
||||
try:
|
||||
data = result.get("structuredContent") if isinstance(result, dict) else None
|
||||
failed = not isinstance(result, dict) or result.get("isError")
|
||||
error = ""
|
||||
if failed and isinstance(result, dict):
|
||||
error = _short(" ".join(item.get("text", "") for item in result.get("content") or [] if isinstance(item, dict)))
|
||||
step = {
|
||||
"id": uuid.uuid4().hex[:10], "at": started, "ms": int((time.time() - started) * 1000), "tool": tool,
|
||||
"kind": KINDS.get(tool, "look" if read_only or tool not in KINDS else "act"), "ok": not failed,
|
||||
"summary": summary_of(tool, arguments if isinstance(arguments, dict) else {}),
|
||||
"results": [] if failed else results_of(tool, arguments, data),
|
||||
}
|
||||
if error:
|
||||
step["error"] = error
|
||||
if step["kind"] == "look" and not failed and tool.startswith(("search_", "list_")):
|
||||
step["count"] = _count_of(data)
|
||||
started = time.time() if started is None else started
|
||||
args = arguments if isinstance(arguments, dict) else {}
|
||||
step = {"id": uuid.uuid4().hex[:10], "at": started, "ms": 0, "tool": tool,
|
||||
"kind": KINDS.get(tool, "look" if read_only or tool not in KINDS else "act"),
|
||||
"ok": True, "state": "running", "summary": summary_of(tool, args), "results": []}
|
||||
with _lock:
|
||||
task = current_task(client)
|
||||
task = current_task(client, started, session=session, task_id=task_id)
|
||||
_add_step(task, step)
|
||||
_save(task)
|
||||
_notify(task["id"])
|
||||
return task["id"], step["id"]
|
||||
except Exception: # noqa: BLE001 - the record must never fail the AI's call
|
||||
log.warning("Anomalous AI tasks: begin_step of %s failed", tool, exc_info=True)
|
||||
return None
|
||||
|
||||
def finish_step(client, tool, arguments, result, started, read_only=False, run_before=None,
|
||||
step_ref=None, session="", task_id=None):
|
||||
"""Completes an in-flight step (or creates one if step_ref is None) with its final outcome. Never raises."""
|
||||
if tool in QUIET or tool in OWN_STEPS:
|
||||
return None
|
||||
try:
|
||||
args = arguments if isinstance(arguments, dict) else {}
|
||||
data = result.get("structuredContent") if isinstance(result, dict) else None
|
||||
failed = not isinstance(result, dict) or bool(result.get("isError"))
|
||||
error = _short(" ".join(item.get("text", "") for item in result.get("content") or [] if isinstance(item, dict))) if failed and isinstance(result, dict) else ""
|
||||
ms = max(0, int((time.time() - started) * 1000))
|
||||
kind = KINDS.get(tool, "look" if read_only or tool not in KINDS else "act")
|
||||
state = "error" if failed else ("running" if tool == "run_workflow" else "done")
|
||||
summary = summary_of(tool, args)
|
||||
results = [] if failed else results_of(tool, args, data)
|
||||
with _lock:
|
||||
task = _load().get(step_ref[0]) if isinstance(step_ref, tuple) and len(step_ref) == 2 else None
|
||||
step = next((item for item in (task or {}).get("steps", []) if item.get("id") == step_ref[1]), None) if task else None
|
||||
if task is None:
|
||||
task = current_task(client, started, session=session, task_id=task_id)
|
||||
if step is None:
|
||||
step = {"id": uuid.uuid4().hex[:10], "at": started, "ms": ms, "tool": tool, "kind": kind,
|
||||
"ok": not failed, "state": state, "summary": summary, "results": results}
|
||||
_add_step(task, step)
|
||||
else:
|
||||
step.update(ms=ms, ok=not failed, state=state, summary=summary, results=results)
|
||||
if error:
|
||||
step["error"] = error
|
||||
else:
|
||||
step.pop("error", None)
|
||||
if kind == "look" and not failed and tool.startswith(("search_", "list_")):
|
||||
step["count"] = _count_of(data)
|
||||
task["updated"] = max(task.get("updated", 0), step["at"] + ms / 1000)
|
||||
if task.get("title_from") == "auto" and not task.get("title") and kind not in ("look", "note") and step["ok"]:
|
||||
task["title"] = summary[:MAX_TITLE]
|
||||
_save(task)
|
||||
_notify(task["id"])
|
||||
if tool == "run_workflow" and not failed:
|
||||
_watch_run(task["id"], step["id"], run_before or set())
|
||||
return step
|
||||
@@ -331,6 +401,11 @@ def record(client, tool, arguments, result, started, read_only=False, run_before
|
||||
log.warning("Anomalous AI tasks: step of %s not recorded", tool, exc_info=True)
|
||||
return None
|
||||
|
||||
def record(client, tool, arguments, result, started, read_only=False, run_before=None, session="", task_id=None):
|
||||
"""One tool call as a step of the app's current task. Never raises."""
|
||||
return finish_step(client, tool, arguments, result, started, read_only, run_before,
|
||||
step_ref=None, session=session, task_id=task_id)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- notes from the AI
|
||||
|
||||
@@ -346,8 +421,7 @@ def _output_media(path):
|
||||
subfolder, _, filename = rel.rpartition("/")
|
||||
return {"type": "image", "filename": filename, "subfolder": subfolder, "folder": "output"}
|
||||
|
||||
|
||||
def report(client, text, images=None, done=False, now=None):
|
||||
def report(client, text, images=None, done=False, now=None, session="", task_id=None):
|
||||
"""A note from the AI, with pictures it made (in ComfyUI's output folder). Returns the step."""
|
||||
text = str(text or "").strip()
|
||||
if not text and not images:
|
||||
@@ -358,9 +432,9 @@ def report(client, text, images=None, done=False, now=None):
|
||||
(results.append(item) if item else missing.append(str(path)))
|
||||
now = time.time() if now is None else now
|
||||
step = {"id": uuid.uuid4().hex[:10], "at": now, "ms": 0, "tool": "report", "kind": "note", "ok": True,
|
||||
"summary": "report", "text": _short(text, MAX_NOTE), "results": results}
|
||||
"state": "done", "summary": "report", "text": _short(text, MAX_NOTE), "results": results}
|
||||
with _lock:
|
||||
task = current_task(client, now)
|
||||
task = current_task(client, now, session=session, task_id=task_id)
|
||||
_add_step(task, step)
|
||||
if done:
|
||||
task["ended"] = now
|
||||
@@ -373,10 +447,8 @@ def report(client, text, images=None, done=False, now=None):
|
||||
|
||||
def _queue():
|
||||
from server import PromptServer
|
||||
|
||||
return PromptServer.instance.prompt_queue
|
||||
|
||||
|
||||
def prompt_ids():
|
||||
"""Prompts queued, running or recently done: before and after run_workflow, the new ones are its."""
|
||||
try:
|
||||
@@ -388,7 +460,6 @@ def prompt_ids():
|
||||
except Exception: # noqa: BLE001 - no server (tests): nothing to watch
|
||||
return set()
|
||||
|
||||
|
||||
def outputs_of(record_):
|
||||
"""Saved pictures and videos in one history record."""
|
||||
out = []
|
||||
@@ -400,7 +471,6 @@ def outputs_of(record_):
|
||||
"folder": "output"})
|
||||
return out
|
||||
|
||||
|
||||
def _finish_run(task_id, step_id, prompts, history):
|
||||
images, failed, done = [], False, 0
|
||||
for prompt_id in prompts:
|
||||
@@ -434,7 +504,6 @@ def _finish_run(task_id, step_id, prompts, history):
|
||||
_notify(task_id)
|
||||
return done >= len(prompts)
|
||||
|
||||
|
||||
def _watch_run(task_id, step_id, before):
|
||||
def watch():
|
||||
start = time.monotonic()
|
||||
@@ -480,24 +549,21 @@ def task_summary(task, now=None):
|
||||
"started": task.get("started"), "updated": task.get("updated"), "ended": task.get("ended"),
|
||||
"waiting": bool(task.get("waiting")) and not task.get("ended"),
|
||||
"live": (not task.get("ended") and not task.get("waiting") and now - task.get("updated", 0) < 90)
|
||||
or any(step.get("state") == "running" for step in steps),
|
||||
or any(step.get("state") == "running" and now - step.get("at", 0) < RUN_WATCH_SECONDS for step in steps),
|
||||
"steps": len(steps), "kinds": kinds, "errors": sum(1 for step in steps if not step.get("ok")),
|
||||
"images": len(images), "cover": images[-1] if images else None, "asks": asks,
|
||||
}
|
||||
|
||||
|
||||
def list_tasks(limit=100):
|
||||
with _lock:
|
||||
tasks = sorted(_load().values(), key=lambda task: task.get("updated", 0), reverse=True)
|
||||
return [task_summary(task) for task in tasks[:limit]]
|
||||
|
||||
|
||||
def get_task(task_id):
|
||||
with _lock:
|
||||
task = _load().get(task_id)
|
||||
return json.loads(json.dumps(task)) if task else None
|
||||
|
||||
|
||||
def rename_task(task_id, title):
|
||||
with _lock:
|
||||
task = _load().get(task_id)
|
||||
@@ -508,7 +574,6 @@ def rename_task(task_id, title):
|
||||
_save(task)
|
||||
return task
|
||||
|
||||
|
||||
def remove_task(task_id):
|
||||
with _lock:
|
||||
task = _load().pop(task_id, None)
|
||||
|
||||
+42
-13
@@ -10,8 +10,10 @@ and Host and Origin (when sent) must name a loopback host, which stops DNS rebin
|
||||
Answers are always one JSON object; there are no server-initiated messages. The tools are
|
||||
in mcp_tools.py (reading), mcp_actions.py (acting) and mcp_task_tools.py (the AI workbench's
|
||||
begin_task and report); every call is kept as a step of a task (ai_tasks.py) for the AI
|
||||
workbench page. POST /anomalous/mcp/bridge is where the open ComfyUI page claims and answers
|
||||
canvas actions (mcp_bridge.py).
|
||||
workbench page, starting in state "running" and updated on completion, with unread user
|
||||
messages and panel answers attached to the next successful tool response. POST
|
||||
/anomalous/mcp/bridge is where the open ComfyUI page claims and answers canvas actions
|
||||
(mcp_bridge.py).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
@@ -21,7 +23,7 @@ import urllib.parse
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from . import ai_tasks, mcp_bridge
|
||||
from . import ai_panels, ai_tasks, mcp_bridge
|
||||
from .local_guard import is_local as _is_local, loopback_host as _loopback_host
|
||||
from .mcp_actions import ACTIONS
|
||||
from .mcp_task_tools import TASK_TOOLS
|
||||
@@ -44,9 +46,11 @@ INSTRUCTIONS = (
|
||||
"Combos, Prompts, Voices). The user watches your work in Anomalous's AI workbench: call begin_task "
|
||||
"with a short title when you start what they asked for (it continues the waiting or active task; "
|
||||
"new_task=true is only for a separate user request), and report to explain a choice, hand in "
|
||||
"pictures made another way, or end the task (done=true). The user can write to you there too: call "
|
||||
"get_answers between steps of longer work to read it. Show plans, workflow explanations and results in "
|
||||
"chronological order with show_panel; update_panel appends new content below what the user already saw."
|
||||
"pictures made another way, or end the task (done=true). The user can write to you or answer "
|
||||
"panel questions there: call get_answers to wait for or read them, and any unread user_messages "
|
||||
"or user_answers attached to a tool result must also be honoured. Show plans, workflow explanations "
|
||||
"and results in chronological order with show_panel; update_panel appends new content below what "
|
||||
"the user already saw."
|
||||
)
|
||||
LIST_TTL_MS = 300000
|
||||
# Requests run tools in threads; a few at a time is plenty for one person's AI app.
|
||||
@@ -131,17 +135,42 @@ def _modern_header_problem(request, method, params, version):
|
||||
|
||||
async def _call(params, request):
|
||||
name = params.get("name")
|
||||
arguments = params.get("arguments") or {}
|
||||
if name not in TOOLS or not isinstance(arguments, dict):
|
||||
raw_arguments = params.get("arguments") or {}
|
||||
if name not in TOOLS or not isinstance(raw_arguments, dict):
|
||||
return None
|
||||
arguments = dict(raw_arguments)
|
||||
client = ai_tasks.client_of(request.headers, params)
|
||||
ctx = {"base_url": f"{request.scheme}://{request.host}", "client": client} # loopback, checked in api_mcp
|
||||
session = ai_tasks.session_of(request.headers, params)
|
||||
task_id = ai_tasks.task_of(request.headers, params, arguments)
|
||||
if "task" in arguments and "task" not in TOOLS[name]["spec"]["inputSchema"].get("properties", {}):
|
||||
arguments.pop("task", None)
|
||||
ctx = {"base_url": f"{request.scheme}://{request.host}", "client": client, "session": session, "task": task_id} # loopback, checked in api_mcp
|
||||
started = time.time()
|
||||
async with _slots:
|
||||
before = await asyncio.to_thread(ai_tasks.prompt_ids) if name == "run_workflow" else None
|
||||
result = await asyncio.to_thread(call_tool, TOOLS, name, arguments, ctx)
|
||||
read_only = bool(TOOLS[name]["spec"].get("annotations", {}).get("readOnlyHint", True))
|
||||
await asyncio.to_thread(ai_tasks.record, client, name, arguments, result, started, read_only, before)
|
||||
step_ref = await asyncio.to_thread(
|
||||
ai_tasks.begin_step, client, name, arguments, started, read_only, session=session, task_id=task_id,
|
||||
)
|
||||
active_task = step_ref[0] if step_ref else task_id
|
||||
if active_task:
|
||||
ctx["task"] = active_task
|
||||
try:
|
||||
async with _slots:
|
||||
before = await asyncio.to_thread(ai_tasks.prompt_ids) if name == "run_workflow" else None
|
||||
result = await asyncio.to_thread(call_tool, TOOLS, name, arguments, ctx)
|
||||
except BaseException as error:
|
||||
failed_result = {"content": [{"type": "text", "text": f"{name} failed: {error}"}], "isError": True}
|
||||
await asyncio.to_thread(
|
||||
ai_tasks.finish_step, client, name, arguments, failed_result, started, read_only, None,
|
||||
step_ref=step_ref, session=session, task_id=active_task,
|
||||
)
|
||||
raise
|
||||
await asyncio.to_thread(
|
||||
ai_tasks.finish_step, client, name, arguments, result, started, read_only, before,
|
||||
step_ref=step_ref, session=session, task_id=active_task,
|
||||
)
|
||||
if name != "get_answers" and isinstance(result, dict) and not result.get("isError"):
|
||||
resolved_task = active_task or await asyncio.to_thread(ai_tasks.active_task_id, client, session=session)
|
||||
await asyncio.to_thread(ai_panels.attach_pending, result, client, task_id=resolved_task)
|
||||
return result
|
||||
|
||||
|
||||
|
||||
+26
-12
@@ -11,16 +11,22 @@ def begin_task(ctx, title, new_task=False):
|
||||
title = str(title or "").strip()
|
||||
if not title:
|
||||
raise ToolError("title is empty: a few words on what the user asked for.")
|
||||
task = ai_tasks.begin_task(ctx.get("client") or "AI", title, new_task=bool(new_task))
|
||||
task = ai_tasks.begin_task(
|
||||
ctx.get("client") or "AI", title, new_task=bool(new_task),
|
||||
session=ctx.get("session") or "", task_id=ctx.get("task"),
|
||||
)
|
||||
return {"task": task["id"], "title": task["title"], "note": "Continue this task; do not begin another for its individual steps. "
|
||||
"Every call from now on is a step of this task in Anomalous's AI workbench, "
|
||||
"where the user sees, opens and undoes them. Call report to explain a choice "
|
||||
"or to hand in pictures you made another way; report with done=true at the end."}
|
||||
|
||||
|
||||
def report(ctx, text="", images=None, done=False):
|
||||
def report(ctx, text="", images=None, done=False, task=None):
|
||||
try:
|
||||
step, missing = ai_tasks.report(ctx.get("client") or "AI", text, images or [], bool(done))
|
||||
step, missing = ai_tasks.report(
|
||||
ctx.get("client") or "AI", text, images or [], bool(done),
|
||||
session=ctx.get("session") or "", task_id=task or ctx.get("task"),
|
||||
)
|
||||
except ValueError as error:
|
||||
raise ToolError(str(error))
|
||||
answer = {"noted": True, "pictures": len(step["results"])}
|
||||
@@ -30,9 +36,12 @@ def report(ctx, text="", images=None, done=False):
|
||||
return answer
|
||||
|
||||
|
||||
def show_panel(ctx, title, blocks):
|
||||
def show_panel(ctx, title, blocks, task=None):
|
||||
try:
|
||||
panel = ai_panels.show(ctx.get("client") or "AI", title, blocks)
|
||||
panel = ai_panels.show(
|
||||
ctx.get("client") or "AI", title, blocks,
|
||||
session=ctx.get("session") or "", task_id=task or ctx.get("task"),
|
||||
)
|
||||
except ai_panels.PanelError as error:
|
||||
raise ToolError(str(error))
|
||||
asks = [block["id"] for block in panel["blocks"] if block["type"] in ai_panels.INTERACTIVE]
|
||||
@@ -45,7 +54,8 @@ def show_panel(ctx, title, blocks):
|
||||
return answer
|
||||
|
||||
|
||||
def update_panel(ctx, panel, title=None, blocks=None, closed=None):
|
||||
def update_panel(ctx, panel, title=None, blocks=None, closed=None, task=None):
|
||||
del ctx, task
|
||||
try:
|
||||
updated = ai_panels.update(panel, title, blocks, closed)
|
||||
except ai_panels.PanelError as error:
|
||||
@@ -54,9 +64,9 @@ def update_panel(ctx, panel, title=None, blocks=None, closed=None):
|
||||
for block in updated["blocks"]]}
|
||||
|
||||
|
||||
def get_answers(ctx, panel=None, wait_seconds=0):
|
||||
def get_answers(ctx, panel=None, wait_seconds=0, task=None):
|
||||
try:
|
||||
return ai_panels.get_answers(ctx.get("client") or "AI", panel, wait_seconds)
|
||||
return ai_panels.get_answers(ctx.get("client") or "AI", panel, wait_seconds, task_id=task or ctx.get("task"))
|
||||
except ai_panels.PanelError as error:
|
||||
raise ToolError(str(error))
|
||||
|
||||
@@ -79,6 +89,8 @@ BLOCKS_HELP = (
|
||||
"Plain text only (no markup, no links); up to 30 blocks, 12 pictures or models a block."
|
||||
)
|
||||
|
||||
_TASK_PROP = {"type": "string", "description": "Optional task id returned by begin_task."}
|
||||
|
||||
|
||||
def _task_tool(name, title, description, run, properties, required=()):
|
||||
return name, {"run": run, "context": True, "spec": {
|
||||
@@ -104,14 +116,15 @@ TASK_TOOLS = dict([
|
||||
"make, so they are kept with the task. done=true ends the task.",
|
||||
report, {"text": {"type": "string"},
|
||||
"images": {"type": "array", "items": {"type": "string"}, "maxItems": MAX_LIMIT},
|
||||
"done": {"type": "boolean", "default": False}}),
|
||||
"done": {"type": "boolean", "default": False}, "task": _TASK_PROP}),
|
||||
_task_tool("show_panel", "Show the user a panel",
|
||||
"Draws a panel in the AI workbench, in Anomalous's own style, from blocks: to compare pictures side by "
|
||||
"side, present models with their covers, lay out settings, a table, and to ask (buttons or a line of "
|
||||
"text). Use a new panel for each new phase (plan, workflow, then results), in the order they happen. "
|
||||
"Use it when a choice is the user's, or to show results better than words do. " + BLOCKS_HELP,
|
||||
show_panel, {"title": {"type": "string"},
|
||||
"blocks": {"type": "array", "items": {"type": "object"}, "maxItems": ai_panels.MAX_BLOCKS}},
|
||||
"blocks": {"type": "array", "items": {"type": "object"}, "maxItems": ai_panels.MAX_BLOCKS},
|
||||
"task": _TASK_PROP},
|
||||
["title", "blocks"]),
|
||||
_task_tool("update_panel", "Change a panel",
|
||||
"Adds new output at the BOTTOM of an existing panel, preserving earlier content and its order. "
|
||||
@@ -122,7 +135,7 @@ TASK_TOOLS = dict([
|
||||
"Can also change title, or set closed=true to stop answers; existing answers stay.",
|
||||
update_panel, {"panel": {"type": "string"}, "title": {"type": "string"},
|
||||
"blocks": {"type": "array", "items": {"type": "object"}, "maxItems": ai_panels.MAX_BLOCKS},
|
||||
"closed": {"type": "boolean"}}, ["panel"]),
|
||||
"closed": {"type": "boolean"}, "task": _TASK_PROP}, ["panel"]),
|
||||
_task_tool("get_answers", "Read the user's answers and messages",
|
||||
"What the user picked or typed on your panels, and what they wrote to you in the AI workbench "
|
||||
"(messages: a change of mind, 'stop', 'make it brighter'), since you last asked; each once. For one "
|
||||
@@ -130,5 +143,6 @@ TASK_TOOLS = dict([
|
||||
"yet: for that panel's open questions, or with no panel for anything the user sends. still_waiting "
|
||||
"lists the questions not answered. Check it between steps of longer work.",
|
||||
get_answers, {"panel": {"type": "string"},
|
||||
"wait_seconds": {"type": "number", "minimum": 0, "maximum": ai_panels.MAX_WAIT, "default": 0}}),
|
||||
"wait_seconds": {"type": "number", "minimum": 0, "maximum": ai_panels.MAX_WAIT, "default": 0},
|
||||
"task": _TASK_PROP}),
|
||||
])
|
||||
|
||||
+2
-2
@@ -118,7 +118,7 @@ claude mcp add --transport http anomalous http://127.0.0.1:8188/anomalous/mcp
|
||||
- 每个任务是一条时间线:查了什么(自动折叠成一行,点开看每一次)、画布上改了什么、运行、下载、扫描、配音,还有 AI 自己的留言(`report` 工具,也可以把它用别的方式做出来的图交上来)。
|
||||
- **做出来的东西都留着,点一下就到**:运行出的图会一直等到跑完再收进任务里,最上面的“这次做出来的”可以直接点开看大图;用到的模型点一下打开模型页,搭配打开搭配,配音可以直接听;画布上的改动可以“在画布上找到”,也可以**撤销**(一步,或者整个任务的画布改动一起,从最后一步往前,遇到你自己后来又改过的地方就停下)。
|
||||
- AI 还在干活时,时间线会自己往下长;你在别的页面时,左边栏的 AI 工作台会亮一个小点。首页也有“AI 最近做了什么”。
|
||||
- **给 AI 留言**:任务时间线下面有个留言框,比如“背景再亮一点”“先别跑了”。留言会成为任务里的一步,AI 下次来问(`get_answers`)时就会收到,下面写着它有没有看到。AMB 没法叫醒 AI:它已经停下的话,请回到 AI 软件里说。
|
||||
- **给 AI 留言**:任务时间线下面有个留言框,比如“背景再亮一点”“先别跑了”。留言会成为任务里的一步,AI 下一次调用任何工具(或 `get_answers`)时都会自动收到,下面写着它有没有看到。AMB 没法叫醒已经停下的 AI:它如果已经结束回合,请回到 AI 软件里让它继续。
|
||||
- 左边的任务按日期分组(今天、昨天、某天),前面的小图标表示状态:● 进行中、? 等你回答、! 有一步出错、✓ 完成。
|
||||
- **新任务**:点任务列表上方的 **+ 新任务**,写上要 AI 做什么(也可以空着)。这个任务会显示“等 AI 接手”,下一个来调用的 AI 就会接手它,之后的步骤都记在这里,不会接在它上一个任务后面。AI 即使先查资料再调用 `begin_task`,也会继续这个任务,保留你定的标题;只有另一件事才用 `new_task=true` 明确另开,或先用 `report(done=true)` 结束当前任务。
|
||||
- 任务可以改名、从列表移除(只删记录,图和画布不受影响)。记录存在 ComfyUI 的 user 文件夹里,保留最近 300 个。
|
||||
@@ -153,6 +153,6 @@ their saved layout because earlier revisions were not recorded.
|
||||
|
||||
While ComfyUI runs, Anomalous serves an MCP endpoint at `http://127.0.0.1:8188/anomalous/mcp` (use your ComfyUI port). Add it as a Streamable HTTP server in your AI app (Claude Code: `claude mcp add --transport http anomalous <url>`; Cursor and VS Code: a `url` / `"type": "http"` entry; Claude Desktop: through `npx mcp-remote <url> --transport http-only`). Both the 2026-07-28 protocol and the earlier `initialize`-based versions are served.
|
||||
|
||||
Since 1.62 the **AI workbench** page keeps every step an AI app takes through this endpoint, grouped into tasks (named with `begin_task`, else one app's calls without a 20-minute pause): what it looked up, changed, ran, downloaded, scanned or spoke, its notes (`report`, which also hands in pictures made another way), and what each step made, with links to the picture, the model, the combo, the canvas change (find or undo) and the audio. It updates live; the rail gets a dot while you are elsewhere. With `show_panel` the AI draws a panel there from fixed blocks (text, pictures to compare, your models as cards, tables, settings, progress, buttons and a text line); the user's answers wait for the AI's `get_answers` (which can wait up to 55 s). Blocks are checked on the server and drawn as text; nothing the AI sends runs.
|
||||
Since 1.62 the **AI workbench** page keeps every step an AI app takes through this endpoint, grouped into tasks (named with `begin_task`, else one app's calls without a 20-minute pause): what it is currently running (`running` in real time) and what it looked up, changed, ran, downloaded, scanned or spoke, its notes (`report`, which also hands in pictures made another way), and what each step made, with links to the picture, the model, the combo, the canvas change (find or undo) and the audio. It updates live; the rail gets a dot while you are elsewhere. With `show_panel` the AI draws a panel there from fixed blocks (text, pictures to compare, your models as cards, tables, settings, progress, buttons and a text line); the user's answers and task messages are returned on `get_answers` (which can wait up to 55 s) and are also automatically attached once to the AI's next successful tool call. Blocks are checked on the server and drawn as text; nothing the AI sends runs.
|
||||
|
||||
Reading tools: library overview, model search and details (optionally with the cover), scan status, output-image search and generation settings (optionally with a thumbnail), combos, workflow recipes, saved prompts, generated audio and the activity log. Acting tools: read the open workflow, write prompts, swap a node's model, add a LoRA, place a combo, check and fix missing models, run the workflow, open a model in Anomalous, start scans, list Anomalous TTS voices and have one speak. Canvas actions need ComfyUI open in a browser on this computer; each is one Ctrl+Z step and is logged as the AI's. Nothing deletes or renames files. Only this computer may connect; requests through a proxy, from other hosts or from web pages are refused. When your AI app uses a cloud model, what the tools return (and any attached picture) is sent to that provider.
|
||||
|
||||
@@ -4,13 +4,15 @@ import {
|
||||
audioResult, duration, groupSteps, isVideo, lookTools, outputUrl, stepDetail, taskCounts, taskResults, taskState, toolKey,
|
||||
} from '../web/modules/ai_task_view.js';
|
||||
|
||||
const look = (tool, ok = true) => ({ kind: 'look', tool, ok, summary: tool });
|
||||
const act = (tool, kind = 'canvas') => ({ kind, tool, ok: true, summary: `${tool}: K` });
|
||||
const look = (tool, ok = true, state = 'done') => ({ kind: 'look', tool, ok, state, summary: tool });
|
||||
const act = (tool, kind = 'canvas') => ({ kind, tool, ok: true, state: 'done', summary: `${tool}: K` });
|
||||
|
||||
// Reads that follow each other fold into one row; a failed read and the work stay rows.
|
||||
// Reads that follow each other fold into one row; a running or failed read and the work stay rows.
|
||||
const items = groupSteps([look('search_models'), look('describe_canvas'), act('place_combo'), look('list_combos', false), look('search_images')]);
|
||||
assert.deepEqual(items.map(item => item.type), ['looks', 'step', 'step', 'looks']);
|
||||
assert.equal(items[0].steps.length, 2);
|
||||
const inFlight = groupSteps([look('search_models'), look('describe_canvas', true, 'running')]);
|
||||
assert.deepEqual(inFlight.map(item => item.type), ['looks', 'step'], 'an in-flight read stays visible until it finishes');
|
||||
assert.deepEqual(groupSteps(undefined), []);
|
||||
assert.deepEqual(lookTools([look('a'), look('b'), look('a')]), [{ tool: 'a', count: 2 }, { tool: 'b', count: 1 }]);
|
||||
|
||||
|
||||
@@ -302,5 +302,25 @@ class AnswerTests(unittest.TestCase):
|
||||
self.assertEqual(len({b["id"] for b in changed["blocks"]}), 3)
|
||||
|
||||
|
||||
def test_attach_pending_delivers_once_and_skips_errors(self):
|
||||
shown = ai_panels.show("Claude", "挑一张", [CHOICES])
|
||||
task_id = ai_tasks.list_tasks()[0]["id"]
|
||||
step = ai_tasks.get_task(task_id)["steps"][0]
|
||||
ai_panels.answer(task_id, step["id"], "pick", "a")
|
||||
ai_panels.leave_message(task_id, "再来一张特写")
|
||||
notified = []
|
||||
with mock.patch.object(ai_tasks, "_notify", side_effect=notified.append):
|
||||
failed = {"content": [{"type": "text", "text": "err"}], "structuredContent": {}, "isError": True}
|
||||
self.assertEqual(ai_panels.attach_pending(copy.deepcopy(failed), "Claude", task_id=task_id), failed)
|
||||
self.assertEqual(notified, [], "failed tool calls must not consume pending messages or answers")
|
||||
ok_res = {"content": [{"type": "text", "text": json.dumps({"total": 1})}], "structuredContent": {"total": 1}, "isError": False}
|
||||
delivered = ai_panels.attach_pending(ok_res, "Claude", task_id=task_id)
|
||||
self.assertIn(task_id, notified)
|
||||
self.assertEqual([m["text"] for m in delivered["structuredContent"]["user_messages"]], ["再来一张特写"])
|
||||
self.assertEqual(delivered["structuredContent"]["user_answers"][0]["choice"], ["a"])
|
||||
again = {"content": [{"type": "text", "text": "{}"}], "structuredContent": {}, "isError": False}
|
||||
self.assertNotIn("user_messages", ai_panels.attach_pending(again, "Claude", task_id=task_id)["structuredContent"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -80,6 +80,44 @@ class TaskTests(unittest.TestCase):
|
||||
with self.assertRaises(mcp_task_tools.ToolError):
|
||||
mcp_task_tools.begin_task({"client": "Claude"}, " ")
|
||||
|
||||
def test_in_flight_step_updates_in_place_without_duplicates(self):
|
||||
waiting = ai_tasks.start_task("生成测试", now=50.0)
|
||||
ref = ai_tasks.begin_step("Gemini", "speak", {"character": "Arona", "text": "hello"}, started=60.0)
|
||||
self.assertIsNotNone(ref)
|
||||
task_id, step_id = ref
|
||||
self.assertEqual(task_id, waiting["id"], "starting a tool claims the waiting task immediately")
|
||||
running_task = ai_tasks.get_task(task_id)
|
||||
self.assertEqual(len(running_task["steps"]), 1)
|
||||
self.assertEqual(running_task["steps"][0]["state"], "running")
|
||||
self.assertTrue(ai_tasks.task_summary(running_task, now=61.0)["live"])
|
||||
finished = ai_tasks.finish_step(
|
||||
"Gemini", "speak", {"character": "Arona", "text": "hello"},
|
||||
ok({"done": True, "files": ["audio/Arona/1.flac"]}), 60.0, step_ref=ref,
|
||||
)
|
||||
self.assertEqual(finished["id"], step_id)
|
||||
done_task = ai_tasks.get_task(task_id)
|
||||
self.assertEqual(len(done_task["steps"]), 1, "completed step updates in place, no duplicate")
|
||||
self.assertEqual((done_task["steps"][0]["state"], done_task["steps"][0]["ok"]), ("done", True))
|
||||
self.assertEqual(done_task["steps"][0]["results"], [{"type": "audio", "file": "audio/Arona/1.flac"}])
|
||||
# A failing tool updates its in-flight step to error.
|
||||
err_ref = ai_tasks.begin_step("Gemini", "set_model", {"model_id": "bad"}, started=70.0)
|
||||
ai_tasks.finish_step("Gemini", "set_model", {"model_id": "bad"}, failed("No model file"), 70.0, step_ref=err_ref)
|
||||
err_step = ai_tasks.get_task(task_id)["steps"][1]
|
||||
self.assertEqual((err_step["id"], err_step["state"], err_step["ok"], err_step["error"]),
|
||||
(err_ref[1], "error", False, "No model file"))
|
||||
|
||||
def test_sessions_and_explicit_task_ids_do_not_collide(self):
|
||||
t1 = ai_tasks.begin_task("Claude", "任务一", now=100.0, session="s1")
|
||||
t2 = ai_tasks.begin_task("Claude", "任务二", now=101.0, session="s2")
|
||||
self.assertNotEqual(t1["id"], t2["id"])
|
||||
ai_tasks.record("Claude", "search_models", {"query": "one"}, ok({"models": [1]}), 102.0, True, session="s1")
|
||||
ai_tasks.record("Claude", "search_models", {"query": "two"}, ok({"models": [1, 2]}), 103.0, True, session="s2")
|
||||
self.assertEqual(ai_tasks.get_task(t1["id"])["steps"][0]["summary"], "search_models: one")
|
||||
self.assertEqual(ai_tasks.get_task(t2["id"])["steps"][0]["summary"], "search_models: two")
|
||||
# An explicit task id routes even when called from a fallback client name.
|
||||
ai_tasks.record("AI", "describe_canvas", {}, ok({}), 104.0, True, task_id=t1["id"])
|
||||
self.assertEqual(len(ai_tasks.get_task(t1["id"])["steps"]), 2)
|
||||
|
||||
def test_what_steps_made(self):
|
||||
with mock.patch.object(ai_tasks, "_model_result", side_effect=lambda model_id: {"type": "model", "model_id": model_id}):
|
||||
ai_tasks.record("A", "add_lora", {"model_id": "m1"}, ok({"_activity": "e2"}), 1.0)
|
||||
|
||||
+25
-1
@@ -13,7 +13,7 @@ from aiohttp.test_utils import make_mocked_request
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
|
||||
from api import ai_tasks, lora_info, mcp_actions, mcp_bridge, mcp_server, mcp_tools # noqa: E402
|
||||
from api import ai_panels, ai_tasks, lora_info, mcp_actions, mcp_bridge, mcp_server, mcp_tools # noqa: E402
|
||||
|
||||
MODERN = "2026-07-28"
|
||||
# Every call is kept as an AI workbench step: keep them in a temporary folder, never the user's.
|
||||
@@ -323,5 +323,29 @@ class LoraHeaderTests(unittest.TestCase):
|
||||
self.assertTrue(lora_info.trains_text_encoder(str(Path(self.temp.name) / "old.ckpt"))) # unknown: assume it does
|
||||
|
||||
|
||||
def test_tool_call_records_in_flight_step_and_piggybacks_pending_user_input(self):
|
||||
task = ai_tasks.start_task("画一张白毛魔女")
|
||||
ai_panels.leave_message(task["id"], "背景加点星光")
|
||||
seen_in_flight = []
|
||||
|
||||
def fake_summarize():
|
||||
current = ai_tasks.get_task(task["id"])
|
||||
seen_in_flight.append([s["state"] for s in current["steps"] if s["tool"] == "scan_report"])
|
||||
return dict.fromkeys(("total", "matched", "unmatched", "new", "pending"), 0) | {"unmatched_models": [], "new_models": []}
|
||||
|
||||
call = {"jsonrpc": "2.0", "id": 9, "method": "tools/call",
|
||||
"params": {"name": "scan_report", "arguments": {"task": task["id"]}, "_meta": meta()}}
|
||||
with mock.patch.object(mcp_tools, "_summarize", side_effect=fake_summarize), \
|
||||
mock.patch.object(mcp_tools, "last_scan_path", return_value="no such file"):
|
||||
status, body = post(call, modern_headers("tools/call", "scan_report"))
|
||||
self.assertEqual(status, 200)
|
||||
self.assertEqual(seen_in_flight, [["running"]], "step is visible as running while tool executes")
|
||||
steps = [s for s in ai_tasks.get_task(task["id"])["steps"] if s["tool"] == "scan_report"]
|
||||
self.assertEqual(len(steps), 1, "same step updated in place without duplicates")
|
||||
self.assertEqual(steps[0]["state"], "done")
|
||||
msgs = body["result"]["structuredContent"].get("user_messages", [])
|
||||
self.assertEqual([m["text"] for m in msgs], ["背景加点星光"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -8,14 +8,14 @@
|
||||
export const KINDS = ['panel', 'user', 'canvas', 'run', 'speech', 'download', 'scan', 'show', 'note', 'act', 'look'];
|
||||
|
||||
/**
|
||||
* The timeline: each step, but reads that follow each other fold into one row (`looks`), so
|
||||
* looking around does not push the work out of sight. A failed read stays a row of its own.
|
||||
* The timeline: each step, but completed reads that follow each other fold into one row (`looks`),
|
||||
* so looking around does not push the work out of sight. A running or failed read stays a row of its own.
|
||||
*/
|
||||
export function groupSteps(steps) {
|
||||
const items = [];
|
||||
for (const step of steps || []) {
|
||||
const last = items.at(-1);
|
||||
if (step.kind === 'look' && step.ok) {
|
||||
if (step.kind === 'look' && step.ok && step.state !== 'running') {
|
||||
if (last?.type === 'looks') last.steps.push(step);
|
||||
else items.push({ type: 'looks', steps: [step] });
|
||||
} else {
|
||||
|
||||
@@ -1254,6 +1254,7 @@ export const i18n = {
|
||||
aiFound: "找到 {count} 个",
|
||||
aiMade: "这次做出来的",
|
||||
aiRunRunning: "正在运行工作流…",
|
||||
aiStepRunning: "正在执行…",
|
||||
aiWorking: "AI 正在工作…",
|
||||
aiSeconds: "用时 {count} 秒",
|
||||
aiMinutes: "用时约 {count} 分钟",
|
||||
@@ -1311,7 +1312,7 @@ export const i18n = {
|
||||
aiTool_user_message: "你给 AI 留言",
|
||||
aiMessagePlaceholder: "给 AI 留言,比如:背景再亮一点 / 先别跑了",
|
||||
aiMessageSend: "发送",
|
||||
aiMessageHint: "AI 下次来问时就会收到(它一般在下一步之前来问)。它要是已经停下了,请回到 AI 软件里说。",
|
||||
aiMessageHint: "AI 做下一步或来读取时就会收到。它要是已经停下了,请回到 AI 软件里说。",
|
||||
aiMessageEnded: "这个任务已经结束了,AI 多半不会再来看;新的要求请回到 AI 软件里说。",
|
||||
aiMessageSeen: "AI 已经看到",
|
||||
aiMessageWaiting: "等 AI 来取",
|
||||
@@ -1333,9 +1334,9 @@ export const i18n = {
|
||||
aiCountAsks: "{count} 个问题等你回答",
|
||||
aiPanelSend: "交给 AI",
|
||||
aiPanelTypeHere: "在这里打字",
|
||||
aiPanelHanded: "已经交出去了,AI 下次来问时就会看到。",
|
||||
aiPanelHanded: "已经交出去了,AI 做下一步或来读取时就会看到。",
|
||||
aiPanelSeen: "AI 已经看到了你的回答。",
|
||||
aiPanelHow: "你的回答会留在这里,AI 来问时才拿得到(它没法被叫醒,一般会在下一步之前来问)。",
|
||||
aiPanelHow: "你的回答会留在这里,AI 做下一步或来读取时就会拿到。",
|
||||
aiPanelClosed: "AI 已经关掉了这个面板,不用再回答了。",
|
||||
aiToastAsks: "AI 在问你:",
|
||||
aiToastShows: "AI 给你看:",
|
||||
@@ -4091,6 +4092,7 @@ export const i18n = {
|
||||
aiFound: "found {count}",
|
||||
aiMade: "What it made",
|
||||
aiRunRunning: "Running the workflow…",
|
||||
aiStepRunning: "Running…",
|
||||
aiWorking: "The AI is working…",
|
||||
aiSeconds: "{count} s",
|
||||
aiMinutes: "about {count} min",
|
||||
@@ -4148,7 +4150,7 @@ export const i18n = {
|
||||
aiTool_user_message: "You wrote to the AI",
|
||||
aiMessagePlaceholder: "Write to the AI, e.g. make the background brighter / stop for now",
|
||||
aiMessageSend: "Send",
|
||||
aiMessageHint: "The AI gets it the next time it asks (usually before its next step). If it has stopped, tell it in the AI app.",
|
||||
aiMessageHint: "The AI gets it on its next step or when it asks. If it has stopped, tell it in the AI app.",
|
||||
aiMessageEnded: "This task has ended, so the AI will probably not look here again; ask for more in the AI app.",
|
||||
aiMessageSeen: "The AI has seen it",
|
||||
aiMessageWaiting: "Waiting for the AI to pick it up",
|
||||
@@ -4170,9 +4172,9 @@ export const i18n = {
|
||||
aiCountAsks: "{count} {count|question|questions} for you",
|
||||
aiPanelSend: "Hand it to the AI",
|
||||
aiPanelTypeHere: "Type here",
|
||||
aiPanelHanded: "Handed over: the AI sees it the next time it asks.",
|
||||
aiPanelHanded: "Handed over: the AI sees it on its next step.",
|
||||
aiPanelSeen: "The AI has seen your answer.",
|
||||
aiPanelHow: "Your answer waits here until the AI asks for it (Anomalous cannot wake it; it usually asks before its next step).",
|
||||
aiPanelHow: "Your answer waits here until the AI's next step or check.",
|
||||
aiPanelClosed: "The AI closed this panel; no answer is needed any more.",
|
||||
aiToastAsks: "The AI asks you:",
|
||||
aiToastShows: "The AI shows you:",
|
||||
|
||||
@@ -9,8 +9,8 @@
|
||||
* with a line in the timeline that brings it there; its questions are answered there, and when it
|
||||
* comes while the user is elsewhere a toast says so. The task list can be folded away.
|
||||
* Under the timeline the user can write to the AI on that task; the message waits until the AI
|
||||
* asks (get_answers), and says whether it has. The list groups the tasks by day, each with
|
||||
* an icon for where it stands.
|
||||
* asks (get_answers) or runs its next step, and says whether it has. The list groups the tasks
|
||||
* by day, each with an icon for where it stands.
|
||||
* In a narrow window the list and the task take turns. The rules are in ai_task_view.js.
|
||||
* Also the home page's "What the AI did lately" and the rail entry's dot for news.
|
||||
*/
|
||||
@@ -136,7 +136,8 @@ function renderList(box) {
|
||||
|
||||
/** One step of the timeline; `number` is its place in the task (the dot shows it). */
|
||||
function stepRow(step, number) {
|
||||
const row = el('div', `anomalous-ai-step is-${step.kind}${step.ok ? '' : ' is-failed'}`);
|
||||
const running = step.state === 'running';
|
||||
const row = el('div', `anomalous-ai-step is-${step.kind}${step.ok ? '' : ' is-failed'}${running ? ' is-running' : ''}`);
|
||||
row.append(el('span', 'anomalous-ai-step-dot', number));
|
||||
const body = el('div', 'anomalous-ai-step-body');
|
||||
const line = el('div', 'anomalous-ai-step-line');
|
||||
@@ -158,7 +159,7 @@ function stepRow(step, number) {
|
||||
body.append(panelLine(step, () => showOnStage(step.panel.id)));
|
||||
}
|
||||
if (step.error) body.append(el('div', 'anomalous-ai-step-error', step.error));
|
||||
if (step.state === 'running') body.append(el('div', 'anomalous-ai-step-running', t('aiRunRunning')));
|
||||
if (running) body.append(el('div', 'anomalous-ai-step-running', t(step.kind === 'run' ? 'aiRunRunning' : 'aiStepRunning')));
|
||||
const links = resultLinks(step);
|
||||
if (links) body.append(links);
|
||||
const when = el('div', 'anomalous-ai-step-when');
|
||||
@@ -285,16 +286,17 @@ function renderTask() {
|
||||
if (!items.length && task.waiting) rows.push({ key: `waiting:${task.title}`, make: () => waitingCard(task) });
|
||||
else if (!items.length) rows.push({ key: 'empty', make: () => el('div', 'anomalous-ai-empty-task', t('aiNoSteps')) });
|
||||
if (status === 'live') {
|
||||
// What comes next: the following number, waiting.
|
||||
const running = (task.steps || []).some(step => step.state === 'running');
|
||||
rows.push({
|
||||
key: `next:${items.length}:${running}`,
|
||||
make: () => {
|
||||
const next = el('div', 'anomalous-ai-step is-next');
|
||||
next.append(el('span', 'anomalous-ai-step-dot', items.length + 1), el('div', 'anomalous-ai-working', t(running ? 'aiRunRunning' : 'aiWorking')));
|
||||
return next;
|
||||
},
|
||||
});
|
||||
if (!running) {
|
||||
rows.push({
|
||||
key: `next:${items.length}:waiting`,
|
||||
make: () => {
|
||||
const next = el('div', 'anomalous-ai-step is-next');
|
||||
next.append(el('span', 'anomalous-ai-step-dot', items.length + 1), el('div', 'anomalous-ai-working', t('aiWorking')));
|
||||
return next;
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
syncRows(view.timeline, rows);
|
||||
updateMessageBox(view.message, Boolean(task.ended), status);
|
||||
|
||||
Reference in New Issue
Block a user