diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 7c7ada8..22a90ca 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -232,7 +232,15 @@ covered by the `styles.css` manifest. give the AI the record in a few words, add_nodes a way's or the node packs' own; of a use's ways that can run, the one that did best on the canvas's model is picked (`reference_blocks.pick(order=…)`); the Gates' `nodes` rule (`ai_gates.node_rule`) lets custom nodes in only by the ways - (`tests/test_ai_evidence.py`). + (`tests/test_ai_evidence.py`). A picks panel keeps the AI's own pick (the item it marked + recommended, taken for the user when they leave it to the AI: `ai_answer_wait.take_recommended`) + apart from the user's. `api/ai_training.py` is the training rounds: the user picks a skill (the + AI's prompts with its advice and without) or the ways of one use, the main models (`models()`, + by base model) and the pictures each, and it becomes a task for the next AI with its plan + (`task["training"]`, the models as `about`); a run that ends fills the trial it fits by what it + used, and the AI is told what is next and, once a model's trials are done, to compare them in a + picks panel with its own pick marked (`tell`, in every answer once it changed; `brief` in + begin_task, `progress` in get_situation) (`tests/test_ai_training.py`). `web/modules/mcp_canvas_brief.js` is the canvas in brief for an AI (pure): bounded path chains and node lines, the inputs a run needs that nothing feeds yet with the outputs that fit them, ComfyUI's errors; every canvas change answers with what is still unwired. `api/mcp_situation.py` is @@ -422,7 +430,13 @@ covered by the `styles.css` manifest. model (`ai_evidence.tally`), and a switch that keeps it from the AI (POST, `user/anomalous/ai_blocks.json`); under them the skills (`ai_skills.status`), each with the same switch (`user/anomalous/ai_skills.json`), those of the - task's kind only (a private task's, else the others), as the AI gets. For a run the AI did not wait + task's kind only (a private task's, else the others), as the AI gets. `ui_ai_network.js` is the + side column's training network (GET /anomalous/ai_training): base models left, ways, skills and + node packs right, a line for each pair that ran, coloured by how it turned out (the user's word + first, else the AI's, dashed), drawn over the two columns from where their nodes are (a + ResizeObserver redraws it); a node or line picked shows its record, last pictures and untried + pairs, and `ui_ai_training_setup.js` is the form starting a round (POST), ending in the sentence to + paste to the AI app. For a run the AI did not wait for (`ai_gates.run_wait` "call") the server marks the ended run `call_ai` until the AI is told, and the lines over the message box are about it meanwhile. `ui_ai_side.js` is the stage, the column on the right where the AI's panels are shown (the timeline keeps a card per diff --git a/CHANGELOG.md b/CHANGELOG.md index 0524d53..700f13f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,8 @@ ## v1.63.0 — unreleased +- 新增 **训练网络**：AI 工作台右栏多了一个视图，把跑图记录画成一张网：左边是你的模型（按底模分），右边是做法、skill 和跑过的节点包，一起跑过的连一条线，颜色就是效果（绿好、黄一般、红不太行、灰还没人评，短虚线是只有 AI 评过），线越粗跑得越多；点一个点或一条线看详情和最近的图，选中一个点时虚线连着还没试过的组合。**开一轮训练**：选一个 skill（AI 自己写和照建议写，并排比）或一种活的几块做法，选最多 4 个模型和每组几张，交给 AI；AI 按计划一组组跑，AMB 自己对上每次跑图属于哪一组、告诉它下一组，一个模型跑完它就把图并排放进挑图面板、先推荐一张再让你挑。AI 的推荐和你的挑选分开记进网络，你的算数在先；你没空挑时，关卡里设成“按推荐的继续”，它的推荐就作数（记成 AI 判的）。 +- New: **the training network**, a new view in the AI workbench's side column that draws the run record as a graph: your models on the left (by base model), the ways, skills and node packs that ran on the right, a line for each pair that ran together, its colour how it turned out (green good, yellow so-so, red not quite, grey not judged yet, short dashes when only the AI judged it) and its width how often. A click on a node or a line shows the details and the last pictures; with a node picked, dashed lines lead to what it has not tried yet. **Start a training round**: pick a skill (the AI's own prompts and those written with the advice, side by side) or the ways of one job, up to 4 models and how many pictures each, and hand it to the AI; it runs the trials in turn, Anomalous works out which trial each run was and tells it the next, and once a model's trials are done it puts them side by side in a picks panel, picks one itself, and you pick. Its pick and yours are recorded apart, yours counted first; if you have no time to pick, the Gates window's "go on with the recommended" lets its pick stand (recorded as the AI's). - 新增 **跑图记录**：AMB 记下 AI 每次跑图用了哪些做法、skill 和第三方节点包，按主模型的底模分开（Illustrious、Anima……），效果好不好从你平常的操作里自动来：挑图面板里选中的、收藏的图算效果好，没被选中的算“没被挑中”；跑完的图下面还有 **效果好 / 不太行** 两个按钮，点了最算数，也告诉 AI。做法面板里每块做法和 skill 下面写着它在这台电脑上的记录（跑过几次、几次效果好、哪些 AI 跑的），没跑过的也写明。AI 开工时就拿到这份记录的摘要，加做法时拿到那块做法的记录；同一种活有几块做法能用时，AMB 先给在画布主模型上记录最好的那块。换哪个 AI 来，一看就知道什么试过、哪里还没试。**关卡** 里新增 **第三方节点**：选“只用做法里的”，AI 自己加节点时只能用做法里搭好的组合和你给任务挑的，ComfyUI 自带的节点照常能用。记录只有名字和次数，没有提示词。 - New: **a record of what works**. Anomalous notes the ways, skills and custom node packs of each run the AI starts, by the base model of the run's main model (Illustrious, Anima…), and how it went comes from what you do anyway: a picture picked in a picks panel or kept in the gallery counts the run good, one passed over "not picked"; under a finished run's pictures, **Turned out well / Not quite** is your own word, which counts most and is told to the AI. In the Ways panel each way and skill says how it did on this computer (how often it ran, how often it turned out well, which AI apps ran it), or that it never ran here. The AI gets the record in a few words when it begins a task and a way's own when it adds it, and of the ways that can do a job Anomalous hands it the one with the best record on the canvas's model first, so whichever AI comes next sees what was tried and what was not. The **Gates** window has a new **Custom nodes** row: with "Only the ways'" the AI may add only the nodes of the ready-made ways and what you picked for the task; ComfyUI's built-in nodes stay allowed. The record holds names and counts, never a prompt. - 新增 **隐私模式**：任务标题栏的 **🔒** 一点，这个任务的**图和提示词原文**都不会发给 AI。图一张都不给（放大看图、图片信息、跑完的预览、你递给它的图、AI 软件里任务卡片上的图都一样），它只知道出了几张、成没成功；提示词和图片文件名换成 **[P1]**、**[img1]** 这样的代号：画布上、图片信息里、提示词库、搭配、模型的触发词和介绍、你自己改画布的记录里都一样（只有节点的“文字输入”被换掉，模型名、数字、选项照常给，第三方节点也一样）。AI 拿代号去跑图、写提示词、放挑图面板，AMB 再换回原文，所以它照样能干活，你那边看到的也都是原样。要新的提示词时，AI 会放一个 **提示词框**：你在里面写几条提示词、反向提示词，可以从图库选或拖进参考图，原文只存在 AMB 里；框上方还会按画布主模型的大小提醒你（小模型手容易画坏，可以选简单动作或藏手，只是可选）。新增 skill“隐私模式·怎么干活”，隐私任务里的 AI 开工时会被指过去；隐私任务里 AI 只拿得到这一个 skill，**做法** 面板的 skill 一栏也只显示它，关掉隐私后才换回普通的 skill。**关卡** 窗口里设新任务默认开不开；打开、关掉都记在时间线上并告诉 AI。所有工具的结果出门前都过同一道关，测试会在每个工具的结果里埋“暗号”提示词和图片名，确认一个都漏不出去。 diff --git a/api/ai_answer_wait.py b/api/ai_answer_wait.py index bb313c7..06a543d 100644 --- a/api/ai_answer_wait.py +++ b/api/ai_answer_wait.py @@ -6,7 +6,7 @@ user set in the Gates window (ai_gates.answer_wait, from when it asked), and whi typing an answer (the page posts POST /anomalous/ai_tasks/{id}/typing as they type) until a while after their last key. Then the AI ends its turn and the user brings it back with a line to paste (web/modules/ai_call_lines.js); or, when the user set so (ai_gates.no_answer "recommended"), each -open choice the AI marked a recommended option for takes it, marked as such for the user and the +open choice or picks block the AI marked a recommended option for takes it, marked as such for the user and the AI (recommended_on_timeout), and the AI goes on; not once the user began writing (typed since the question came): they stepped away mid-answer, and it waits for them. Typing is kept in memory only. """ @@ -25,11 +25,11 @@ _typed = {} # task id -> time of the user's last key def take_recommended(task_id, step): - """Answers each open choice of panel step `step` that has a recommended option with it. Returns how many.""" + """Answers each open choice or picks block of panel step `step` that has a recommended option with it. Returns how many.""" taken = 0 for block in step["panel"].get("blocks") or []: - picked = [option["id"] for option in block.get("options") or [] if option.get("recommended")] - if block.get("type") != "choices" or not picked or block.get("id") in (step.get("answers") or {}): + picked = [option["id"] for option in block.get("options") or block.get("items") or [] if option.get("recommended")] + if block.get("type") not in ("choices", "picks") or not picked or block.get("id") in (step.get("answers") or {}): continue try: ai_panels.answer(task_id, step["id"], block["id"], choice=picked if block.get("multiple") else picked[:1], diff --git a/api/ai_evidence.py b/api/ai_evidence.py index 1aac5c4..779d35a 100644 --- a/api/ai_evidence.py +++ b/api/ai_evidence.py @@ -10,7 +10,9 @@ the ComfyUI user folder, the last MAX_RUNS), with the AI app that ran it. How it the user does anyway, nothing to note down by hand: a picture of it picked in a picks panel or kept in the gallery (a recipe, a combo or a prompt made from it, kept_images.py), or their word on the run (set_verdict, told to the AI too), counts it good; their word "not good" counts it so; its pictures -passed over in a picks panel count it "not chosen". The record keeps model, way, skill and pack +passed over in a picks panel count it "not chosen". The AI's own pick in that panel (the picture it +marked recommended, ai_training.py asks for one) is kept apart as the AI's judgement, as is a pick +the user left to it (ai_answer_wait takes the recommended one). The record keeps model, way, skill and pack names, never a prompt; its pictures' names stay on this computer, to match the user's picks. """ @@ -26,6 +28,7 @@ from .path_utils import atomic_write_json MAX_RUNS = 2000 MAX_PICTURES = 8 MAX_BASES = 4 # base models in one item's line for the AI +MAX_SHOWN = 6 # pictures of one item on one base model the network shows OVERVIEW_CHARS = 300 KINDS = ("way", "skill", "pack") VERDICTS = ("good", "bad") @@ -138,20 +141,28 @@ def note_run(task_id, step_id): _save(runs) -def note_picks(items, picked): - """A picks panel answered (its pictures `items`, the ids `picked`): a run with a picture picked counts - good, one whose pictures were only passed over "not chosen".""" +def note_picks(items, picked, taken=False): + """A picks panel answered (its pictures `items`, the ids `picked`): a run with a picture the user picked + counts good, one whose pictures they only passed over "not chosen"; the items the AI marked recommended + count the same way as its own judgement. `taken`: the user left it to the AI (its pick was taken for them): + the AI's judgement only.""" key = lambda item: (str(item.get("subfolder") or ""), str(item.get("filename") or "")) # noqa: E731 - chosen = {key(item) for item in items if item.get("id") in picked} shown = {key(item) for item in items} + chosen = set() if taken else {key(item) for item in items if item.get("id") in picked} + advised = {key(item) for item in items if item.get("recommended")} with _lock: runs, changed = _runs(), False for entry in runs.values(): pictures = {tuple(picture) for picture in entry.get("pictures") or []} - if pictures & chosen and not entry.get("picked"): - entry["picked"], changed = True, True - elif pictures & shown and not pictures & chosen and not entry.get("picked") and not entry.get("passed"): - entry["passed"], changed = True, True + if not pictures & shown: + continue + marks = {} + if not taken: + marks["picked" if pictures & chosen else "passed"] = True + if advised: + marks["ai_picked" if pictures & advised else "ai_passed"] = True + for mark in [mark for mark in marks if not entry.get(mark)]: + entry[mark], changed = True, True if changed: _save(runs) @@ -192,7 +203,8 @@ def set_verdict(task_id, step_id, verdict): def tally(): - """The record summed up: {kind: {name: {base: {runs, good, bad, passed, failed, clients}}}}.""" + """The record summed up: {kind: {name: {base: {runs, good, bad, passed, failed, ai_good, ai_passed, clients, + pictures (the last few, [subfolder, filename])}}}}.""" out = {kind: {} for kind in KINDS} with _lock: entries = list(_runs().values()) @@ -209,12 +221,15 @@ def tally(): for kind in KINDS: for name in entry.get(f"{kind}s") or []: count = out[kind].setdefault(name, {}).setdefault(entry.get("base") or "", { - "runs": 0, "good": 0, "bad": 0, "passed": 0, "failed": 0, "clients": []}) + "runs": 0, "good": 0, "bad": 0, "passed": 0, "failed": 0, "ai_good": 0, "ai_passed": 0, "clients": [], "pictures": []}) count["runs"] += 1 count["good"] += good count["bad"] += verdict == "bad" count["passed"] += not good and verdict != "bad" and bool(entry.get("passed")) count["failed"] += entry.get("state") == "error" + count["ai_good"] += bool(entry.get("ai_picked")) + count["ai_passed"] += bool(entry.get("ai_passed")) and not entry.get("ai_picked") + count["pictures"] = (count["pictures"] + [list(picture) for picture in entry.get("pictures") or []][:2])[-MAX_SHOWN:] if entry.get("client") and entry["client"] not in count["clients"]: count["clients"].append(entry["client"]) return out @@ -222,7 +237,9 @@ def tally(): def _line(base, count): parts = [f"{count[key]} {label}" for key, label in COUNTS if count[key]] - return f"{base or 'other models'} {count['runs']} runs: {', '.join(parts) or 'none judged yet'}" + ai = [f"{count[key]} {label}" for key, label in (("ai_good", "picked"), ("ai_passed", "passed over")) if count[key]] + return (f"{base or 'other models'} {count['runs']} runs: {', '.join(parts) or 'none judged by the user yet'}" + + (f" (the AI's own judgement: {', '.join(ai)})" if ai else "")) def words(kind, name, found=None): diff --git a/api/ai_panels.py b/api/ai_panels.py index 77f0e3e..c1402b8 100644 --- a/api/ai_panels.py +++ b/api/ai_panels.py @@ -323,7 +323,8 @@ def answer(task_id, step_id, block_id, choice=None, text=None, now=None, skip=Fa answers = dict(step["answers"]) ai_tasks._notify(task_id) if block["type"] == "picks" and not skip: - ai_evidence.note_picks(block["items"], entry["choice"]) # the runs whose pictures the user picked did well + # The runs whose pictures the user picked did well; the AI's recommended pick is its judgement. + ai_evidence.note_picks(block["items"], entry["choice"], bool((extra or {}).get("recommended_on_timeout"))) return answers diff --git a/api/ai_task_runs.py b/api/ai_task_runs.py index 8dfd92b..10b1875 100644 --- a/api/ai_task_runs.py +++ b/api/ai_task_runs.py @@ -16,7 +16,7 @@ a run the user or the AI stopped (ai_run_control.py) ends "cancelled".""" import threading import time -from . import ai_evidence, ai_gates, ai_run_map, ai_run_watch, ai_tasks +from . import ai_evidence, ai_gates, ai_run_map, ai_run_watch, ai_tasks, ai_training from .mcp_tools import ToolError, _image_content, _resolve_image, retext, tool_text RUN_POLL_SECONDS = 2.0 @@ -177,6 +177,7 @@ def _finish_run(task_id, step_id, prompts, history, queued=None): ai_tasks._notify(task_id) if done >= len(prompts): ai_evidence.note_run(task_id, step_id) # what it used and how it went, in the record of what works + ai_training.note_run(task_id, step_id) # a trial of a training round, when it fits one return done >= len(prompts) diff --git a/api/ai_tasks_routes.py b/api/ai_tasks_routes.py index bf82d1e..7f967f6 100644 --- a/api/ai_tasks_routes.py +++ b/api/ai_tasks_routes.py @@ -2,7 +2,7 @@ messages): the list and one task (reads, open like the other pages), and the writes the page makes - start a task, rename or remove one, answer a panel, write to the AI, answer a gate and set the gates' policies (ai_gates.py), stop a run (ai_run_control.py), say how a run turned out -(ai_evidence.py), hand the AI pictures (ai_pictures.py) - (local_guard.py keeps the writes to this computer).""" +(ai_evidence.py), start a training round (ai_training.py), hand the AI pictures (ai_pictures.py) - (local_guard.py keeps the writes to this computer).""" import asyncio @@ -323,6 +323,32 @@ async def api_blocks(request): "skills": ai_skills.status(), "record": await asyncio.to_thread(ai_evidence.tally)}) +async def api_training(request): + """GET /anomalous/ai_training: the training network (ai_training.py): the record summed up by base model, + the main models here, the ways and skills; POST {title, skill | use, models, each, subject?, picture?} + starts a training round, a task for the next AI that calls.""" + from . import ai_skills, ai_training, mcp_canvas_edit, reference_blocks + from .mcp_tools import ToolError + + try: + info = await asyncio.to_thread(mcp_canvas_edit.object_info, {"base_url": f"{request.scheme}://{request.host}"}) + except ToolError as error: + return web.json_response({"error": str(error)}, status=502) + if request.method == "POST": + body = await _body(request) or {} + try: + task = await asyncio.to_thread(ai_training.start, str(body.get("title") or ""), body.get("models"), body.get("skill") or "", + body.get("use") or "", body.get("each", 2), body.get("subject") or "", body.get("picture") or "", info) + except ai_training.TrainingError as error: + return web.json_response({"error": str(error)}, status=400) + return web.json_response({"task": ai_tasks.task_summary(task)}) + blocks = [{**{key: block[key] for key in ("name", "job", "title", "usable", "on")}, + "takes": bool(reference_blocks.BLOCKS[block["name"]].get("takes"))} for block in reference_blocks.status(info)] + skills = [skill for skill in ai_skills.status() if not skill["private_only"]] + return web.json_response({"record": await asyncio.to_thread(ai_evidence.tally), "models": await asyncio.to_thread(ai_training.models), + "blocks": blocks, "skills": skills, "jobs": list(reference_blocks.JOBS)}) + + def register_routes(app): activity_log.add_listener(ai_task_changes.note_activity) # the user's changes while an AI works app.router.add_get("/anomalous/ai_tasks", api_list) @@ -345,5 +371,7 @@ def register_routes(app): app.router.add_post("/anomalous/ai_toolbox/favorites", api_toolbox) app.router.add_get("/anomalous/ai_blocks", api_blocks) app.router.add_post("/anomalous/ai_blocks", api_blocks) + app.router.add_get("/anomalous/ai_training", api_training) + app.router.add_post("/anomalous/ai_training", api_training) app.router.add_get("/anomalous/ai_gates", api_gates) app.router.add_post("/anomalous/ai_gates", api_gates) diff --git a/api/ai_training.py b/api/ai_training.py new file mode 100644 index 0000000..be6f98a --- /dev/null +++ b/api/ai_training.py @@ -0,0 +1,203 @@ +"""Training rounds of the training network (web/modules/ui_ai_network.js): the user picks what to +try, a skill (prompts written with its advice, and without it) or the ready-made ways of one use, +on which of their models and how many pictures each, and Anomalous makes it a task for the next AI +that calls, with the plan on it ("training"). The AI does the trials with its usual tools; each +run that ends is matched to a trial by what it used (ai_evidence.py: the model, the ways it still +has, the skills fetched before it in the task), and the AI is told in its answers how far it is +and what is next (tell). Once a model's trials are done it is asked to show them side by side in a +picks panel with its own pick marked recommended, for the user to pick: both picks go into the +record, the user's counted first. The network itself is ai_evidence's record summed up, beside +the main models here by base model (models), the ways and the skills. +""" + +import os +import random + +from . import ai_skills, ai_tasks, model_advice, reference_blocks +from .metadata import get_metadata +from .mcp_tools import ToolError, _each_model, _model_id, _resolve_model, retext + +MAX_MODELS = 4 +MAX_EACH = 3 # pictures of one trial set (seeds) +MAX_LISTED = 400 # main models the network lists +MAX_SUBJECT = 300 +WITHOUT = "" # a skill's trials without it + + +class TrainingError(ValueError): + pass + + +def models(): + """The main models here (checkpoints, diffusion models), [{model_id, name, file, base}], at most MAX_LISTED.""" + out = [] + for folder_type, path_idx, base_dir, file_path in _each_model(): + if folder_type not in model_advice.MAIN_TYPES: + continue + meta = get_metadata(file_path) + out.append({"model_id": _model_id(folder_type, path_idx, base_dir, file_path), "file": os.path.basename(file_path), + "name": meta.get("custom_name") or meta.get("name") or os.path.splitext(os.path.basename(file_path))[0], + "base": meta.get("baseModel") or meta.get("base_guess") or ""}) + if len(out) >= MAX_LISTED: + break + return out + + +def _model(model_id): + try: + folder_type, _path_idx, _base_dir, file_path = _resolve_model(model_id) + except ToolError as error: + raise TrainingError(str(error)) + if folder_type not in model_advice.MAIN_TYPES: + raise TrainingError(f"{model_id} is not a main model (a checkpoint or a diffusion model)") + meta = get_metadata(file_path) + return {"model_id": model_id, "file": os.path.basename(file_path), "base": meta.get("baseModel") or meta.get("base_guess") or "", + "name": meta.get("custom_name") or meta.get("name") or os.path.splitext(os.path.basename(file_path))[0]} + + +def _items(skill, use, info): + """What a round compares: a skill and its trials without it, or the ways of `use` that run here and are on.""" + if skill: + found = ai_skills.SKILLS.get(skill) + if not found or found.get("private_only") or skill in ai_skills.turned_off(): + raise TrainingError(f"No skill {skill!r} to train") + return "skill", [WITHOUT, skill] # without it first: a skill counts for every run after it is fetched + blocks = [block for block in reference_blocks.status(info) if block["job"] == use and block["usable"] and block["on"]] + if not blocks: + raise TrainingError(f"No way of {use!r} runs here") + return "use", [block["name"] for block in blocks] + + +def start(title, model_ids, skill="", use="", each=2, subject="", picture="", info=None): + """A training round as a task for the next AI that calls (ai_tasks.start_task, the models as what it + is about). Returns the task.""" + kind, items = _items(str(skill or ""), str(use or ""), info or {}) + ids = list(dict.fromkeys(model_ids if isinstance(model_ids, list) else [])) + if not 1 <= len(ids) <= MAX_MODELS: + raise TrainingError(f"Pick 1 to {MAX_MODELS} models") + if not isinstance(each, int) or isinstance(each, bool) or not 1 <= each <= MAX_EACH: + raise TrainingError(f"each is 1 to {MAX_EACH}") + subject = str(subject or "").strip()[:MAX_SUBJECT] + if kind == "skill" and not subject: + raise TrainingError("Say what to draw (subject)") + takes = kind == "use" and any(reference_blocks.BLOCKS[name].get("takes") for name in items) + if takes and not ai_tasks._output_media(str(picture or "")): + raise TrainingError("These ways work on a picture: pick one from the gallery") + chosen = [_model(str(model_id)) for model_id in ids] + plan = {"kind": kind, "skill": skill if kind == "skill" else "", "use": use if kind == "use" else "", "items": items, + "models": chosen, "seeds": [random.randint(1, 2 ** 31 - 1) for _ in range(each)], "subject": subject, + "picture": str(picture or "") if takes else "", "told": 0, + "trials": [{"model": at, "item": item, "run": None} for item in items for at in range(len(chosen)) for _ in range(each)]} + task = ai_tasks.start_task(title, about=[{"type": "model", "model_id": model["model_id"], "label": model["name"]} for model in chosen]) + with ai_tasks._lock: + stored = ai_tasks._load()[task["id"]] + stored["training"] = plan + ai_tasks._save(stored) + ai_tasks._notify(task["id"]) + return stored + + +def _uses(plan, item, used): + if plan["kind"] == "skill": + return (plan["skill"] in used["skills"]) == (item != WITHOUT) + return item in [way["name"] for way in used["ways"]] + + +def note_run(task_id, step_id): + """Run `step_id` of task `task_id` ended: the first open trial of its plan the run fits (its model and + what it used) takes it.""" + with ai_tasks._lock: + task = ai_tasks._load().get(task_id) + plan = (task or {}).get("training") + step = next((item for item in (task or {}).get("steps", []) if item["id"] == step_id), None) + if not plan or not step or step.get("state") != "done" or not step.get("used"): + return + if any(trial["run"] == step_id for trial in plan["trials"]): + return + for trial in plan["trials"]: + if not trial["run"] and step["used"]["model"] == plan["models"][trial["model"]]["file"] and _uses(plan, trial["item"], step["used"]): + trial["run"] = step_id + ai_tasks._save(task) + break + + +def _item_words(plan, item): + if plan["kind"] == "skill": + return f"with the skill (get_toolbox(skill={plan['skill']}) first)" if item else "without the skill" + return f"way {item} (add_nodes(block={item}))" + + +def _pictures(task, runs): + """The output pictures of runs `runs` of `task`, as show_panel takes them.""" + out = [] + for step in task.get("steps") or []: + if step["id"] in runs: + out += ["/".join(part for part in (item.get("subfolder"), item["filename"]) if part) + for item in step.get("results") or [] if item.get("type") == "image"] + return out + + +def progress(task): + """Where the round stands, for the AI: how many trials are done and the next, or the trials of a + model just done to compare, or that all are done.""" + plan = task["training"] + trials = plan["trials"] + parts = [f"Training: {sum(1 for trial in trials if trial['run'])} of {len(trials)} trials done."] + for at, model in enumerate(plan["models"]): + own = [trial for trial in trials if trial["model"] == at] + if all(trial["run"] for trial in own) and at not in plan.get("compared", []): + parts.append(f"The trials on {model['name']} are done: show_panel one picks block of their pictures " + f"{_pictures(task, {trial['run'] for trial in own})}, each labelled with its trial, recommended=true on " + "the one you judge best, asking which turned out best. The user picks; their pick and yours both go into the record.") + break + following = next((trial for trial in trials if not trial["run"]), None) + if following: + model = plan["models"][following["model"]] + parts.append(f"Next: {model['name']} (set_model {model['model_id']}), {_item_words(plan, following['item'])}, seeds {plan['seeds']}.") + else: + parts.append("Then: report what did best on which model (done=true).") + return " ".join(parts) + + +def brief(task): + """The round for begin_task: what to compare, on which models, how, and where it stands.""" + plan = task["training"] + if plan["kind"] == "skill": + goal = (f"A training round of the skill {plan['skill']}: draw \"{plan['subject']}\" on each model below, the prompt written " + f"by you without the skill, then with its advice; one picture for each of the seeds {plan['seeds']}, the same " + "seeds both ways, so the two compare. Do every trial without it first: a skill counts for every run after it is fetched.") + how = "A plain text-to-image workflow (the canvas's, or new_workflow); per trial set_model, set_prompt, run_workflow(seed=…)." + else: + ways = ", ".join(plan["items"]) + goal = (f"A training round of the ways of {plan['use']} ({ways}): run each on each model below" + + (f" on the picture {plan['picture']}" if plan["picture"] else "") + f", one picture for each of the seeds {plan['seeds']}.") + how = ("Per trial set_model, then add_nodes(block=…) on a workflow without another of these ways (open the same base " + "workflow again, or remove the last way), then run_workflow(seed=…). Ask the user (ask_user) when a way needs an area marked.") + return {"goal": goal, "how": how, "models": [{key: model[key] for key in ("model_id", "name", "base")} for model in plan["models"]], + "now": progress(task), "note": "Anomalous matches each run to its trial by what it used and tells you what is next."} + + +def tell(result, task_id): + """Puts the round's progress into a tool result once it changed (a trial done, a model to compare).""" + if not isinstance(result, dict) or result.get("isError") or not task_id or "training" in (result.get("structuredContent") or {}): + return result # begin_task's answer has the whole round (brief) + with ai_tasks._lock: + task = ai_tasks._load().get(task_id) + plan = (task or {}).get("training") + if not plan: + return result + done = sum(1 for trial in plan["trials"] if trial["run"]) + if done == plan.get("told"): + return result + plan["told"] = done + words = progress(task) + for at in range(len(plan["models"])): + own = [trial for trial in plan["trials"] if trial["model"] == at] + if own and all(trial["run"] for trial in own) and at not in plan.setdefault("compared", []): + plan["compared"].append(at) # asked once + break + ai_tasks._save(task) + data = dict(result.get("structuredContent") or {}) + data["training"] = words + retext(result, data) + return result diff --git a/api/mcp_server.py b/api/mcp_server.py index 16696c6..7f93256 100644 --- a/api/mcp_server.py +++ b/api/mcp_server.py @@ -24,7 +24,7 @@ import urllib.parse from aiohttp import web -from . import ai_callers, ai_gates, ai_outside, ai_privacy, ai_sealed, ai_panels, ai_run_tracking, ai_task_canvas, ai_task_runs, ai_tasks, mcp_apps, mcp_bridge, mcp_directory, mcp_prompts +from . import ai_callers, ai_gates, ai_outside, ai_privacy, ai_sealed, ai_panels, ai_run_tracking, ai_task_canvas, ai_task_runs, ai_tasks, ai_training, mcp_apps, mcp_bridge, mcp_directory, mcp_prompts from .local_guard import is_local as _is_local, loopback_host as _loopback_host from .mcp_actions import ACTIONS from .mcp_canvas_edit import CANVAS_TOOLS, object_info @@ -225,6 +225,7 @@ async def _call(params, request): if name not in ("run_workflow", "run_status", "stop_run") and resolved_task: await asyncio.to_thread(ai_task_runs.tell, result, resolved_task) await asyncio.to_thread(ai_task_runs.show_pictures, result, resolved_task) + await asyncio.to_thread(ai_training.tell, result, resolved_task) # a training round's progress, once it changed if name != "get_answers" and name not in mcp_apps.APP_TOOLS and isinstance(result, dict) and not result.get("isError"): await asyncio.to_thread(ai_panels.attach_pending, result, client, task_id=resolved_task or ("" if session else None)) mcp_directory.attach(result, name, {key: tool["spec"] for key, tool in TOOLS.items() if key not in mcp_apps.APP_TOOLS}) diff --git a/api/mcp_situation.py b/api/mcp_situation.py index c64667a..f8810fc 100644 --- a/api/mcp_situation.py +++ b/api/mcp_situation.py @@ -9,7 +9,7 @@ and the task's history. Unread messages of the user's come with it as with every (ai_panels.attach_pending). """ -from . import ai_decisions, ai_task_runs, ai_tasks +from . import ai_decisions, ai_task_runs, ai_tasks, ai_training from .mcp_bridge import PageRefused, PageUnavailable, ask_page from .mcp_tools import _schema, tool_text @@ -64,6 +64,8 @@ def get_situation(ctx, task=None): "asked": _last_asked(record), "waiting_for_user": summary["asks"], "decisions": ai_decisions.lines(record, limit=4), "pictures": pictures[-3:], "outside_anomalous": summary["kinds"].get("outside", 0)} + if record.get("training"): + answer["task"]["training"] = ai_training.progress(record) # a training round: how far, what next status = ai_task_runs.status(client, session, task_id) run = {key: status[key] for key in ("state", "summary", "pictures") if status.get(key)} else: diff --git a/api/mcp_task_tools.py b/api/mcp_task_tools.py index 271b904..bb1db78 100644 --- a/api/mcp_task_tools.py +++ b/api/mcp_task_tools.py @@ -4,7 +4,7 @@ show_panel / update_panel draw a panel from Anomalous's blocks (ai_panels.py), g reads what the user picked or typed on it, log_outside keeps what the AI did with other tools (ai_outside.py).""" -from . import ai_answer_wait, ai_decisions, ai_evidence, ai_marks, ai_message_refs, ai_outside, ai_panels, ai_repaint, ai_run_control, ai_run_tracking, ai_task_links, ai_task_runs, ai_tasks, ai_toolbox, ai_privacy, ai_skills, reference_blocks +from . import ai_answer_wait, ai_decisions, ai_evidence, ai_marks, ai_training, ai_message_refs, ai_outside, ai_panels, ai_repaint, ai_run_control, ai_run_tracking, ai_task_links, ai_task_runs, ai_tasks, ai_toolbox, ai_privacy, ai_skills, reference_blocks from .mcp_tools import MAX_LIMIT, ToolError, _schema PACK_TYPES = 150 # node types listed per picked pack @@ -40,6 +40,8 @@ def begin_task(ctx, title, new_task=False): record = ai_evidence.overview() if record: answer["record"] = record # what ran here how often and how it went, by model: any AI's runs + if task.get("training"): + answer["training"] = ai_training.brief(ai_tasks.get_task(task["id"])) # a round the user started in the training network box = ai_toolbox.effective(ai_tasks.get_task(task["id"])) if box: answer["toolbox"] = ai_toolbox.words(box) diff --git a/docs/architecture/backend.md b/docs/architecture/backend.md index d13039f..fd948fa 100644 --- a/docs/architecture/backend.md +++ b/docs/architecture/backend.md @@ -298,6 +298,23 @@ AI apps that ran it so the next one knows what is untried. `add_nodes(use=…)` runnable ways in order by their record on the canvas's base model (asked of the page only when one has a record). With the Gates' "custom nodes: only the ways'" (`ai_tools.json` `nodes`), `mcp_canvas_edit._allowed` refuses a custom node no way builds and the task did not pick. +A picks panel answer also records the AI's judgement: the items it marked recommended count as +its pick (`ai_picked`), the others shown as passed over by it; when the user left the panel to the +AI (`no_answer` "recommended", `ai_answer_wait.take_recommended`, now for picks blocks too) the +choice is the AI's only, and nothing is counted as the user's. + +Training rounds (`api/ai_training.py`, `GET/POST /anomalous/ai_training`): the network's data is +the record summed up (`ai_evidence.tally`, with each pair's last pictures), the main models here +with their base models (`models()`, at most 400, read from their metadata), the ways with whether +they take a picture, and the skills. A round is a skill (trials without it first, as a skill +counts for every run after it is fetched, then with it) or the ways of one use that run here, +on 1-4 main models, 1-3 pictures each with the same seeds; it is started as a waiting task +(`ai_tasks.start_task`, the models as `about`) carrying `training` {kind, items, models, seeds, +subject, picture, trials}. When a run of that task ends (`ai_task_runs._finish_run`), the first +open trial whose model file and item fit its `used` takes it; `tell` (in `mcp_server._call` +after the run's pictures) puts the progress into the AI's answer once the count of done trials +changed: the next trial, or, when a model's trials are all done, the picks panel to show with +its pictures and its own pick marked recommended (asked once a model), then the report. `track_run(prompt_ids, task, wait_seconds)` attaches existing submissions from this server to an open task without running anything. IDs must exist in its queue/history; each is owned by diff --git a/docs/architecture/ui-flows.md b/docs/architecture/ui-flows.md index d83bf2b..d4318ef 100644 --- a/docs/architecture/ui-flows.md +++ b/docs/architecture/ui-flows.md @@ -75,6 +75,7 @@ AMB 更新以后，AI 软件还拿着旧的工具列表来调用时，标题栏 - 从上往下读，编号 1、2、3……；AI 连着查资料、连着改画布的几步折成一行（`ui_ai_chain.js`），点开看每一步。 - 每一步下面写它做了什么（`ui_ai_step_body.js`），出的东西能直接去：图片进看图器、模型进模型页、画布改动能定位和撤销（`ui_ai_results.js`）。 - 跑完的图下面是「这次效果：效果好 / 不太行」（`ui_ai_verdict.js`），再点一下收回。点了记进跑图记录，也告诉 AI；不点也行，挑图、收藏会自己记。跑图那一步还写着这次用到的做法。做法面板里每块做法和 skill 下面是它在这台电脑上按模型分的记录（`api/ai_evidence.py`）。 +- 右栏「训练网络」（`ui_ai_network.js`）：左模型、右做法 / skill / 节点包，连线颜色是效果；点点或线看详情。「开一轮训练」（`ui_ai_training_setup.js`）选练什么、哪些模型、每组几张，「交给 AI 开练」建一个等 AI 接的任务并给出要贴的那句话。 - 图片上鼠标移上去有工具（`ui_ai_images.js`）：「图片与参数」、「保存为…」、勾两张对比，还有 ✎ 标注。 - 时间线和展台里的图先用缩略图（`/anomalous/output_thumbnail`），显示得比缩略图大时才换原图（`/anomalous/output_preview`：超大图换的是 4096 像素的预览），打开任务不用等大图。 diff --git a/docs/guides/mcp.md b/docs/guides/mcp.md index e2ca649..7cc567a 100644 --- a/docs/guides/mcp.md +++ b/docs/guides/mcp.md @@ -158,6 +158,8 @@ claude mcp add --transport http anomalous http://127.0.0.1:8188/anomalous/mcp **跑图记录（哪套组合在哪个模型上好用）**：AI 每次跑图，AMB 都记下这次用了哪些做法、哪些 skill、哪些第三方节点包，主模型是什么底模（比如 Illustrious、Anima）。效果好不好不用你专门去记：在挑图面板选中的图、收藏的图（存成工作流、搭配、提示词）算 **效果好**；挑图时没被选的算 **没被挑中**；跑完的图下面也有 **这次效果：效果好 / 不太行** 两个按钮，你点的那个最算数，还会告诉 AI（点“不太行”它就该换做法、设置或提示词再跑），再点一下收回。做法面板里每块做法和 skill 下面写着它的记录，比如“Illustrious：跑过 4 次，3 次效果好”；没跑过的写“这台电脑上还没跑过”，鼠标停在一行上能看到是哪些 AI 跑的。AI 开工时（`begin_task`）就能看到这份记录的摘要，加做法时也会告诉它这块做法的记录；同一种活有几块做法都能用时，AMB 先给它在画布主模型上记录最好的那块。所以下一个 AI，不管是 Claude、Gemini 还是 GPT，一看就知道什么在这台电脑上试过、效果怎样、哪里还没试。记录只有名字和次数，没有提示词；存在 ComfyUI 的 user 文件夹里（anomalous/ai_evidence.json，最近 2000 次）。**关卡** 窗口里的 **第三方节点** 选“只用做法里的”，AI 自己加节点时就只能用做法里那些搭好的组合，加上你给这个任务挑的；ComfyUI 自带的节点照常能用。 +**训练网络**：AI 工作台右栏的 **训练网络**（脉搏图标）把这份记录画成一张网：左边是你的模型（按底模分，写着各有几个），右边是做法、skill 和跑过的节点包，一起跑过的连一条线，颜色是效果（绿好、黄一般、红不太行、灰跑过还没人评），短虚线表示只有 AI 评过，线越粗跑得越多。点一个模型、一块做法或一条线，下面写着它的记录：跑过几次、你怎么判的、AI 怎么判的、哪些 AI 跑过、最近的几张图；选中一个点时，浅虚线连着它还没试过的组合。**开一轮训练**：选练什么（一个 skill：同一个题材，AI 自己写的提示词和照 skill 建议写的并排比；或者一种活的几块做法并排比）、在哪几个模型上（最多 4 个）、每组几张（同样的种子），点 **交给 AI 开练**，就建好一个等 AI 来接的任务，把显示的那句话贴给你的 AI。AI 按计划一组组跑，AMB 自己认出每次跑图对应计划里哪一组，并在回答里告诉它下一组；一个模型的几组跑完，AI 会把这几张并排放进挑图面板，先推荐一张，再由你挑。你的挑选和 AI 的推荐都记进网络，分开记、你的算数在先；你没空挑时，关卡里“等不到回答时”设成按推荐的继续，AI 的推荐就作数（只记成 AI 判的）。 + **隐私模式**：有些图和提示词不方便给 AI 看时，点任务标题栏的 **🔒**，这个任务就成了隐私任务。AMB 一张图都不发给 AI（放大看图、图片信息里的图、跑完附带的预览、你递给它的图、AI 软件里任务卡片上的图都一样），它只知道出了几张、成没成功。提示词原文和图片文件名换成 **[P1]**、**[img1]** 这样的代号：画布上节点的文字输入（按 ComfyUI 自己的节点定义认，第三方节点也一样；模型名、数字、选项照常给）、图片信息、提示词库、搭配、工作流的设置、模型的触发词和介绍、你自己改画布的记录都一样，同一段文字永远是同一个代号。AI 用代号去写提示词、跑图、填加载图片、放挑图面板，AMB 在工具执行前换回原文，所以画布和出图都是原样，你看到的也都是原样；时间线上记的是代号。要新提示词时，AI 会放一个 **提示词框**：写几条提示词（可以再加）、反向提示词，可以从图库选或把图拖进来当参考图，点 **交给 AI（只给代号）**；原文只存在 AMB 里，交出后框里显示代号和你写的原文。框上方按画布主模型的大小提醒你（来自 skill“写提示词·动作适合模型”）。AI 怎么在隐私任务里干活写在 skill“隐私模式·怎么干活”里，开工时会被指过去。隐私任务里 AI 只拿得到这一个 skill，做法面板的 skill 一栏也只显示它；普通任务正好反过来，只显示、只给普通的 skill。**关卡** 窗口里的“隐私模式”设新任务默认开不开；默认改成关时，原来按默认开着的任务仍然保持隐私。AI 的某次调用认不出属于哪个任务时，只要默认开着或者还有没做完的隐私任务，就按隐私处理。注意：任务做到一半才打开时，之前已经给过 AI 的收不回来；你自己在留言或 AI 聊天框里写的文字和贴的图，AMB 不拦（那是你主动给的）；模型名字 AI 是看得到的。 **这次跑的节点结构**：每次运行那一步下面写着一行“底模 · 几个 LoRA · 尺寸 · 步数 · 几个节点”，点开有 **简单** 和 **完整** 两种看法（AMB 记得你上次选的）。简单：按每一段采样列出“用哪个模型加哪些 LoRA、提示词是什么、从空白画布还是从上一段的结果（放大多少）开始”，最后保存在哪；后一段和前一段一样的就写“和第 1 段一样”。完整：方块图，哪个底模、LoRA 和权重、正负提示词开头、尺寸、采样器设置、保存，方块之间的线标着类型。两种看法里点任意一项都能定位到画布上的节点。这张图是 AMB 按 ComfyUI 实际收到的工作流画的，不靠 AI 描述。上面的黄色小块是 **比上次改了什么**（比如“K采样器·步数：20 → 30”“新增「加载LoRA」”）。AI 讲解时可以用 `show_run_map` 一次把同一张图放上展台，不用自己拼。 diff --git a/tests/test_ai_training.py b/tests/test_ai_training.py new file mode 100644 index 0000000..721f970 --- /dev/null +++ b/tests/test_ai_training.py @@ -0,0 +1,121 @@ +"""Training rounds (api/ai_training.py): a round the user starts is a task waiting for an AI with +its plan; each run that ends fills the trial it fits; the AI is told what is next once, and to +compare a model's trials in a picks panel with its own pick, where the user's pick and the AI's +are kept apart (api/ai_evidence.py). Temporary folders only.""" +import os +import sys +import time +import unittest +from pathlib import Path +from unittest import mock + +PLUGIN = Path(__file__).resolve().parents[1] +sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1]), str(PLUGIN / "tests")] + +from api import ai_answer_wait, ai_evidence, ai_gates, ai_panels, ai_training, ai_tasks, mcp_task_tools # noqa: E402 +from test_ai_panels import OUTPUT # noqa: E402 (a.png and b.png in a temporary output folder) +from test_ai_tasks import isolate, ok # noqa: E402 + +MODELS = {"checkpoints:0:il.safetensors": ("il.safetensors", "Illustrious"), "checkpoints:0:anima.safetensors": ("anima.safetensors", "Anima")} + + +def resolve(model_id): + file, _base = MODELS[model_id] + return "checkpoints", 0, "/models", f"/models/{file}" + + +class TrainingTests(unittest.TestCase): + def setUp(self): + isolate() + for patcher in (mock.patch.object(ai_training, "_resolve_model", side_effect=resolve), + mock.patch.object(ai_training, "get_metadata", side_effect=lambda path: { + "baseModel": next(base for file, base in MODELS.values() if path.endswith(file))}), + mock.patch.object(ai_evidence.kept_images, "kept_keys", return_value=set()), + mock.patch.object(ai_tasks.folder_paths, "get_output_directory", return_value=OUTPUT), + mock.patch.object(mcp_task_tools, "_canvas_model", return_value=None), + mock.patch.object(ai_gates, "_tools_path", lambda: os.path.join(os.path.dirname(ai_evidence._path()), "ai_tools.json"))): + patcher.start() + self.addCleanup(patcher.stop) + + def start(self, each=1): + return ai_training.start("Training", list(MODELS), skill="prompt_pose", each=each, subject="a girl holding a cup") + + def finish(self, task_id, model, skills, picture): + """A run of the task that ended with `picture`, on `model`, the skills `skills` fetched before it.""" + step = ai_tasks.record("Gemini", "run_workflow", {}, ok({"queued": 1}), time.time(), task_id=task_id) + with ai_tasks._lock: + task = ai_tasks._load()[task_id] + stored = next(item for item in task["steps"] if item["id"] == step["id"]) + stored.update(state="done", used={"model": model, "base": "", "ways": [], "skills": skills, "packs": []}, + results=[{"type": "image", "filename": picture, "subfolder": "", "folder": "output"}]) + ai_tasks._save(task) + ai_evidence.note_run(task_id, step["id"]) + ai_training.note_run(task_id, step["id"]) + return step["id"] + + def test_a_round_is_a_task_waiting_for_an_ai_with_its_plan(self): + task = self.start(each=2) + plan = task["training"] + self.assertTrue(task["waiting"]) + self.assertEqual([item["model_id"] for item in task["about"]], list(MODELS), "the AI gets the models' ids") + self.assertEqual([model["base"] for model in plan["models"]], ["Illustrious", "Anima"]) + self.assertEqual(len(plan["trials"]), 8, "2 models × with and without × 2 pictures") + self.assertEqual({trial["item"] for trial in plan["trials"][:4]}, {ai_training.WITHOUT}, "every trial without the skill first") + begun = mcp_task_tools.begin_task({"client": "Gemini"}, "training") + self.assertEqual(begun["task"], task["id"], "the next AI that calls takes it") + self.assertIn("without the skill, then with its advice", begun["training"]["goal"]) + self.assertIn("Next: il (set_model checkpoints:0:il.safetensors), without the skill", begun["training"]["now"]) + self.assertEqual(ai_training.tell(begun_result := ok(begun), task["id"]), begun_result, "the brief is not written over") + self.assertNotIn("training", ai_training.tell(ok({}), task["id"])["structuredContent"], "nothing new before a trial is done") + + def test_wrong_rounds_are_refused(self): + for kwargs in ({"skill": "privacy", "subject": "x"}, {"skill": "prompt_pose", "subject": ""}, {"skill": "nope", "subject": "x"}, + {"use": "repaint", "info": {}}): + with self.subTest(**{key: str(value) for key, value in kwargs.items()}), self.assertRaises(ai_training.TrainingError): + ai_training.start("Training", list(MODELS), **kwargs) + with self.assertRaises(ai_training.TrainingError): + ai_training.start("Training", [], skill="prompt_pose", subject="x") + + def test_runs_fill_the_trials_they_fit_and_the_ai_hears_what_is_next(self): + task = self.start() + self.finish(task["id"], "other.safetensors", [], "a.png") + self.assertFalse(any(trial["run"] for trial in ai_tasks.get_task(task["id"])["training"]["trials"]), "another model fits no trial") + first = self.finish(task["id"], "il.safetensors", [], "a.png") + trials = ai_tasks.get_task(task["id"])["training"]["trials"] + self.assertEqual((trials[0]["run"], trials[0]["item"]), (first, ai_training.WITHOUT)) + told = ai_training.tell(ok({}), task["id"])["structuredContent"]["training"] + self.assertIn("1 of 4 trials done", told) + self.assertNotIn("training", ai_training.tell(ok({}), task["id"])["structuredContent"], "told once") + self.finish(task["id"], "anima.safetensors", [], "b.png") + self.finish(task["id"], "il.safetensors", ["prompt_pose"], "b.png") + told = ai_training.tell(ok({}), task["id"])["structuredContent"]["training"] + self.assertIn("The trials on il are done", told) + self.assertIn("'a.png', 'b.png'", told, "the pictures to put side by side") + self.assertIn("recommended=true", told) + self.finish(task["id"], "anima.safetensors", ["prompt_pose"], "a.png") + told = ai_training.tell(ok({}), task["id"])["structuredContent"]["training"] + self.assertIn("4 of 4", told) + self.assertIn("report what did best", told) + + def test_the_ais_pick_and_the_users_are_kept_apart(self): + task = self.start() + first = self.finish(task["id"], "il.safetensors", ["prompt_pose"], "a.png") + self.finish(task["id"], "anima.safetensors", ["prompt_pose"], "b.png") + shown = ai_panels.show("Gemini", "Which?", [{"type": "picks", "id": "p", "question": "Which turned out best?", + "items": [{"id": "a", "image": "a.png", "recommended": True}, {"id": "b", "image": "b.png"}]}], + task_id=task["id"]) + step = next(item for item in ai_tasks.get_task(task["id"])["steps"] if item.get("panel", {}).get("id") == shown["id"]) + with mock.patch.object(ai_answer_wait.ai_gates, "no_answer", return_value="recommended"): + self.assertEqual(ai_answer_wait.take_recommended(task["id"], step), 1, "a picks block left to the AI takes its pick") + answered = next(item for item in ai_tasks.get_task(task["id"])["steps"] if item["id"] == step["id"])["answers"]["p"] + self.assertEqual(answered["choice"], ["a"]) + runs = ai_evidence._runs() + self.assertTrue(runs[first].get("ai_picked")) + self.assertFalse(runs[first].get("picked"), "the user did not pick: no word of theirs") + count = ai_evidence.tally()["skill"]["prompt_pose"][""] + self.assertEqual((count["good"], count["ai_good"], count["ai_passed"]), (0, 1, 1)) + self.assertIn("the AI's own judgement: 1 picked, 1 passed over", ai_evidence.words("skill", "prompt_pose")) + + +if __name__ == "__main__": + unittest.main() diff --git a/web/modules/locales.js b/web/modules/locales.js index ff900ed..9f110d2 100644 --- a/web/modules/locales.js +++ b/web/modules/locales.js @@ -1466,6 +1466,59 @@ export const i18n = { aiGateWhy_nodes: "AI 自己往画布上加节点时，第三方节点包的节点能不能随便用。选“只用做法里的”，它就只能用做法面板里那些搭好的组合，加上你给这个任务挑的；ComfyUI 自带的节点照常能用。", aiNodeRule_any: "随便用", aiNodeRule_ways: "只用做法里的", + aiRecord_ai_good: "，AI 推荐 {n} 次", + aiSideNetwork: "训练网络", + aiNetIntro: "左边是你的模型（按底模分），右边是做法、skill 和跑过的节点包。连线是一起跑过的，颜色是效果：绿＝好，黄＝一般，红＝不太行，灰＝跑过还没人评；短虚线是只有 AI 评过。线越粗跑得越多。", + aiNetTrain: "开一轮训练", + aiNetModels: "模型", + aiNetItems: "做法 / skill", + aiNetModelsHere: "{count} 个模型", + aiNetNotHere: "这里没有这种模型", + aiNetPack: "节点包", + aiNetHint: "点一个模型、一块做法或一条线看它的记录。选中一个点后，浅虚线是它还没试过的组合。", + aiNetEmpty: "还没有记录。开一轮训练，或者让 AI 照常干活，记录会自己长出来。", + aiNetReading: "正在读取训练网络…", + aiNetUnread: "读不到训练网络：{error}", + aiNetRuns: "一起跑过 {runs} 次", + aiNetUser: "你判的：{parts}", + aiNetUserNone: "你还没评过", + aiNetAi: "AI 自己判的：{parts}", + aiNetCount_good: "{n} 次效果好", + aiNetCount_bad: "{n} 次不太行", + aiNetCount_passed: "{n} 次没被挑中", + aiNetCount_failed: "{n} 次出错", + aiNetCount_ai_good: "推荐了 {n} 次", + aiNetCount_ai_passed: "没推荐 {n} 次", + aiNetShort: "跑过 {runs} 次 · {good} 次好", + aiNetShortAi: "跑过 {runs} 次 · AI 推荐 {good} 次", + aiNetUntried: "还没试过：{names}", + aiNetTrainThis: "在这一对上练一轮", + aiNetTrainBase: "在这个底模上练一轮", + aiNetTrainItem: "练一轮", + aiTrainHead: "开一轮训练", + aiTrainBack: "← 回到网络", + aiTrainWhat: "练什么", + aiTrainSkill: "skill：{name}（照建议写 和 AI 自己写，并排比）", + aiTrainUse: "做法：{job}（{count} 块并排比）", + aiTrainModels: "在哪些模型上（最多 {max} 个）", + aiTrainModelsNone: "没找到主模型（checkpoints 或 diffusion_models 里的模型）。", + aiTrainNothing: "现在没有能练的：做法面板里开着又能用的做法和 skill 才能练。", + aiTrainSubject: "画什么", + aiTrainSubjectDefault: "一个女孩坐在窗边，双手捧着一杯热茶，半身像", + aiTrainPicture: "在哪张图上做", + aiTrainPick: "从图库选一张", + aiTrainPicsHint: "点一张图来选。", + aiTrainPicsFull: "只要一张。", + aiTrainPicsAdd: "用这张", + aiTrainEach: "每组几张（不同种子）", + aiTrainCount: "一共 {total} 张图：{models} 个模型 × {kinds} 种 × 每组 {each} 张。", + aiTrainHow: "AI 来跑：每个模型跑完，它把这几张并排放进挑图面板，先推荐一张，再由你挑。你的挑选和它的推荐都记进网络，你的算数在先；你没空挑时，可以在关卡里设成按 AI 推荐的继续。", + aiTrainStart: "交给 AI 开练", + aiTrainNeedModel: "至少选一个模型", + aiTrainNeedPicture: "这些做法要在一张图上做：先选一张图", + aiTrainTaskTitle: "训练：{what} × {count} 个模型", + aiTrainMade: "训练任务建好了，正等 AI 来接。把下面这句话贴给你的 AI：", + aiTrainOpen: "去看这个任务", modelVersionsOpen: "展开这个模型的 {count} 个版本（来自同一个 Civitai 模型页）", modelVersionsFold: "收起，只留图钉那个版本", modelVersionsPin: "收起时显示这个版本", @@ -4999,6 +5052,59 @@ export const i18n = { aiGateWhy_nodes: "Whether the AI may add nodes of any custom node pack when it adds nodes itself. With \"Only the ways'\" it may use only the ready-made combinations of the Ways panel and what you picked for the task; ComfyUI's built-in nodes stay allowed.", aiNodeRule_any: "Any", aiNodeRule_ways: "Only the ways'", + aiRecord_ai_good: ", the AI's pick {n} times", + aiSideNetwork: "Training network", + aiNetIntro: "Left are your models (by base model), right the ways, skills and the node packs that ran. A line joins what ran together, its colour how it turned out: green good, yellow so-so, red not quite, grey ran but nobody judged it; short dashes mean only the AI judged it. The thicker, the more it ran.", + aiNetTrain: "Start a training round", + aiNetModels: "Models", + aiNetItems: "Ways / skills", + aiNetModelsHere: "{count} models", + aiNetNotHere: "No such model here", + aiNetPack: "Node pack", + aiNetHint: "Click a model, a way or a line to see its record. With a node picked, faint dashes are what it has not tried yet.", + aiNetEmpty: "No record yet. Start a training round, or let the AI work as usual: the record grows by itself.", + aiNetReading: "Reading the training network…", + aiNetUnread: "Could not read the training network: {error}", + aiNetRuns: "Ran together {runs} times", + aiNetUser: "Your judgement: {parts}", + aiNetUserNone: "You have not judged it yet", + aiNetAi: "The AI's own judgement: {parts}", + aiNetCount_good: "{n} turned out well", + aiNetCount_bad: "{n} not quite", + aiNetCount_passed: "{n} not picked", + aiNetCount_failed: "{n} failed", + aiNetCount_ai_good: "picked {n} times", + aiNetCount_ai_passed: "passed over {n} times", + aiNetShort: "{runs} runs · {good} good", + aiNetShortAi: "{runs} runs · the AI picked {good}", + aiNetUntried: "Not tried yet: {names}", + aiNetTrainThis: "Train this pair", + aiNetTrainBase: "Train on this base model", + aiNetTrainItem: "Train it", + aiTrainHead: "Start a training round", + aiTrainBack: "← Back to the network", + aiTrainWhat: "What to train", + aiTrainSkill: "Skill: {name} (written with its advice and by the AI alone, side by side)", + aiTrainUse: "Ways: {job} ({count} side by side)", + aiTrainModels: "On which models (up to {max})", + aiTrainModelsNone: "No main models found (in checkpoints or diffusion_models).", + aiTrainNothing: "Nothing to train now: ways and skills that are on and can run in the Ways panel can be trained.", + aiTrainSubject: "What to draw", + aiTrainSubjectDefault: "A girl sitting by a window, holding a cup of hot tea in both hands, upper body", + aiTrainPicture: "On which picture", + aiTrainPick: "Pick from the gallery", + aiTrainPicsHint: "Click a picture to pick it.", + aiTrainPicsFull: "Just one.", + aiTrainPicsAdd: "Use this one", + aiTrainEach: "Pictures each (different seeds)", + aiTrainCount: "{total} pictures in all: {models} models × {kinds} kinds × {each} each.", + aiTrainHow: "The AI runs it: once a model's trials are done it puts their pictures side by side in a picks panel, picks one itself, and you pick. Your pick and its own both go into the network, yours counted first; when you have no time to pick, the Gates window can let its pick go on.", + aiTrainStart: "Hand it to the AI", + aiTrainNeedModel: "Pick at least one model", + aiTrainNeedPicture: "These ways work on a picture: pick one first", + aiTrainTaskTitle: "Training: {what} × {count} models", + aiTrainMade: "The training task is made and waits for an AI. Paste this line to your AI:", + aiTrainOpen: "Open the task", modelVersionsOpen: "Show this model's {count} versions (from one Civitai model page)", modelVersionsFold: "Fold, keeping the pinned version", modelVersionsPin: "Show this version when folded", diff --git a/web/modules/ui_ai_blocks.js b/web/modules/ui_ai_blocks.js index 522924b..15f444d 100644 --- a/web/modules/ui_ai_blocks.js +++ b/web/modules/ui_ai_blocks.js @@ -74,7 +74,7 @@ function record(bases) { return box; } for (const [base, count] of lines) { - const parts = ['good', 'bad', 'passed', 'failed'].filter(key => count[key]).map(key => t(`aiRecord_${key}`, { n: count[key] })); + const parts = ['good', 'bad', 'passed', 'failed', 'ai_good'].filter(key => count[key]).map(key => t(`aiRecord_${key}`, { n: count[key] })); const line = el('div', 'anomalous-ai-block-record-line', t('aiRecordLine', { base: base || t('aiRecordOther'), runs: count.runs, parts: parts.join('') })); if (count.clients?.length) line.title = t('aiRecordBy', { names: count.clients.join(', ') }); box.append(line); diff --git a/web/modules/ui_ai_network.js b/web/modules/ui_ai_network.js new file mode 100644 index 0000000..df32da1 --- /dev/null +++ b/web/modules/ui_ai_network.js @@ -0,0 +1,256 @@ +/** + * The side column's training network (训练网络, ui_ai_side_views.js): what the runs here showed, as a + * graph. Left the base models (those of the main models here, and any the record names), right the + * ready-made ways, the skills and the node packs that ran; a line joins a model and an item that ran + * together, its colour how that turned out (api/ai_evidence.py: the user's picks, keeps and word, + * else, dashed, the AI's own pick), its width how often. A click on a node or a line shows it below: + * the counts, who judged, which AI apps ran it, its last pictures, and, for a node, the pairs not + * tried yet (dashed in the graph); a training round (ui_ai_training_setup.js) starts from there or + * from the top. GET /anomalous/ai_training, read again each time the view is opened. + */ + +import { resolveLocale, translate as t } from './locales.js'; +import { sideHead } from './ui_ai_side.js'; +import { imageResults } from './ui_ai_images.js'; +import { trainingSetup } from './ui_ai_training_setup.js'; + +const SVG = 'http://www.w3.org/2000/svg'; +const COUNTS = ['good', 'bad', 'passed', 'failed']; +let shown = null; // the network last read, or {error} +let reading = null; +let chosen = null; // {base} | {kind, name} | {base, kind, name} +let setup = null; // the training round being set up: its preset, or null +let watcher = null; // redraws the lines when the graph's size changes + +const words = value => (resolveLocale() === 'zh' ? value?.zh : value?.en) || value?.en || value?.zh || ''; +const baseLabel = base => base || t('aiRecordOther'); + +function el(tag, className, text) { + const node = document.createElement(tag); + if (className) node.className = className; + if (text !== undefined) node.textContent = String(text); + return node; +} + +function button(className, label, onClick) { + const node = el('button', className, label); + node.type = 'button'; + node.onclick = onClick; + return node; +} + +/** The items: the ways and skills on (and any with a record) and the node packs that ran, those that ran most first. */ +function itemsOf(data) { + const record = data.record || {}; + const out = []; + for (const block of data.blocks) { + const usable = block.usable && block.on; + if (usable || record.way?.[block.name]) out.push({ kind: 'way', name: block.name, title: words(block.title), sub: t(`aiBlocksJob_${block.job}`), usable }); + } + for (const skill of data.skills) { + if (skill.on || record.skill?.[skill.name]) out.push({ kind: 'skill', name: skill.name, title: words(skill.title), sub: 'Skill', usable: skill.on }); + } + for (const pack of Object.keys(record.pack || {}).sort()) out.push({ kind: 'pack', name: pack, title: pack, sub: t('aiNetPack'), usable: false }); + const runs = item => Object.values(record[item.kind]?.[item.name] || {}).reduce((sum, count) => sum + count.runs, 0); + return out.map(item => ({ ...item, key: `${item.kind}:${item.name}`, runs: runs(item) })).sort((a, b) => b.runs - a.runs); +} + +/** The base models with how many main models here have each and how often they ran, most run first. */ +function basesOf(data, edges) { + const bases = new Map(data.models.map(model => [model.base, 0])); + for (const model of data.models) bases.set(model.base, bases.get(model.base) + 1); + const runs = new Map(); + for (const edge of edges) { + runs.set(edge.base, (runs.get(edge.base) || 0) + edge.count.runs); + if (!bases.has(edge.base)) bases.set(edge.base, 0); + } + return [...bases].map(([base, models]) => ({ base, models, runs: runs.get(base) || 0 })) + .sort((a, b) => b.runs - a.runs || b.models - a.models || a.base.localeCompare(b.base)); +} + +/** How a pair turned out: the user's judgement first, else the AI's (`ai`), else only that it ran. */ +function look(count) { + const share = (good, all) => (good / all >= 0.6 ? 'good' : good / all >= 0.3 ? 'fair' : 'bad'); + const user = count.good + count.bad + count.passed; + const ai = count.ai_good + count.ai_passed; + if (user) return { tone: share(count.good, user), ai: false }; + if (ai) return { tone: share(count.ai_good, ai), ai: true }; + return { tone: 'ran', ai: false }; +} + +function edgesOf(data, items) { + return items.flatMap(item => Object.entries(data.record?.[item.kind]?.[item.name] || {}) + .map(([base, count]) => ({ base, item, count, look: look(count) }))); +} + +const isBase = pick => pick && pick.base !== undefined && !pick.kind; +const isItem = pick => pick && pick.kind && pick.base === undefined; +const touches = (edge, pick) => !pick || (pick.base === undefined || pick.base === edge.base) && (!pick.kind || pick.kind === edge.item.kind && pick.name === edge.item.name); + +/** The pairs of the chosen node not tried yet: [base, item], items that can run, bases with models here. */ +function untried(bases, items, edges) { + const tried = new Set(edges.map(edge => `${edge.base}|${edge.item.key}`)); + if (isBase(chosen)) return items.filter(item => item.usable && !tried.has(`${chosen.base}|${item.key}`)).map(item => [chosen.base, item]); + const item = isItem(chosen) && items.find(entry => entry.kind === chosen.kind && entry.name === chosen.name); + return item?.usable ? bases.filter(base => base.models && !tried.has(`${base.base}|${item.key}`)).map(base => [base.base, item]) : []; +} + +function choose(pick, redraw) { + chosen = JSON.stringify(chosen) === JSON.stringify(pick) ? null : pick; + redraw(); +} + +/** The lines between the columns, drawn from where the nodes are now. */ +function drawLines(svg, graph, nodes, edges, open) { + const box = graph.getBoundingClientRect(); + svg.setAttribute('width', String(box.width)); + svg.setAttribute('height', String(box.height)); + svg.replaceChildren(); + const at = (node, right) => { + const rect = node.getBoundingClientRect(); + return [(right ? rect.right : rect.left) - box.left, rect.top + rect.height / 2 - box.top]; + }; + const line = (base, item, className, width, onClick, title) => { + const from = nodes.get(`base:${base}`); + const to = nodes.get(item.key); + if (!from || !to) return; + const [x1, y1] = at(from, true); + const [x2, y2] = at(to, false); + const middle = (x1 + x2) / 2; + const shape = `M${x1} ${y1} C${middle} ${y1} ${middle} ${y2} ${x2} ${y2}`; + const path = document.createElementNS(SVG, 'path'); + path.setAttribute('d', shape); + path.setAttribute('class', className); + path.style.strokeWidth = `${width}px`; + svg.append(path); + if (!onClick) return; + const hit = document.createElementNS(SVG, 'path'); + hit.setAttribute('d', shape); + hit.setAttribute('class', 'anomalous-ai-net-hit'); + hit.addEventListener('click', onClick); + const tip = document.createElementNS(SVG, 'title'); + tip.textContent = title; + hit.append(tip); + svg.append(hit); + }; + for (const [base, item] of open) line(base, item, 'anomalous-ai-net-line is-untried', 1.5); + for (const edge of edges) { + const own = touches(edge, chosen); + const picked = chosen?.kind && chosen.base !== undefined && own; + line(edge.base, edge.item, `anomalous-ai-net-line is-${edge.look.tone}${edge.look.ai ? ' is-ai' : ''}${own ? '' : ' is-dim'}${picked ? ' is-chosen' : ''}`, + 1.5 + Math.min(edge.count.runs, 10) * 0.35, edge.onClick, `${baseLabel(edge.base)} × ${edge.item.title}: ${short(edge.count)}`); + } +} + +/** "4 runs · 3 good", or the AI's word when the user gave none. */ +function short(count) { + const judged = look(count); + return t(judged.ai ? 'aiNetShortAi' : 'aiNetShort', { runs: count.runs, good: judged.ai ? count.ai_good : count.good }); +} + +function countWords(count, keys) { + return keys.filter(key => count[key]).map(key => t(`aiNetCount_${key}`, { n: count[key] })).join(' · '); +} + +/** What is shown below the graph for the chosen node or line. */ +function detailBox(data, bases, items, edges, ctx, redraw) { + const box = el('div', 'anomalous-ai-net-detail'); + if (!chosen) { + box.append(el('div', 'anomalous-ai-net-hint', t(edges.length ? 'aiNetHint' : 'aiNetEmpty'))); + return box; + } + const train = (label, preset) => button('anomalous-ai-net-train is-small', label, () => { setup = preset; redraw(); }); + if (chosen.kind && chosen.base !== undefined) { + const edge = edges.find(item => touches(item, chosen)); + if (!edge) return box; + const count = edge.count; + box.append(el('div', 'anomalous-ai-net-detail-title', `${baseLabel(edge.base)} × ${edge.item.title}`), + el('div', 'anomalous-ai-net-detail-line', t('aiNetRuns', { runs: count.runs })), + el('div', 'anomalous-ai-net-detail-line', countWords(count, ['good', 'bad', 'passed']) ? t('aiNetUser', { parts: countWords(count, ['good', 'bad', 'passed']) }) : t('aiNetUserNone'))); + if (count.ai_good || count.ai_passed) box.append(el('div', 'anomalous-ai-net-detail-line', t('aiNetAi', { parts: countWords(count, ['ai_good', 'ai_passed']) }))); + if (count.failed) box.append(el('div', 'anomalous-ai-net-detail-line', countWords(count, ['failed']))); + if (count.clients.length) box.append(el('div', 'anomalous-ai-net-detail-line is-soft', t('aiRecordBy', { names: count.clients.join(', ') }))); + const pictures = [...count.pictures].reverse().map(([subfolder, filename]) => ({ type: 'image', folder: 'output', filename, subfolder })); + if (pictures.length) box.append(imageResults(pictures, ctx.owner, { compact: true })); + if (edge.item.usable) box.append(train(t('aiNetTrainThis'), { item: edge.item, base: edge.base })); + return box; + } + const base = isBase(chosen) ? bases.find(item => item.base === chosen.base) : null; + const item = base ? null : items.find(entry => entry.kind === chosen.kind && entry.name === chosen.name); + if (!base && !item) return box; + box.append(el('div', 'anomalous-ai-net-detail-title', base ? baseLabel(base.base) : item.title), + el('div', 'anomalous-ai-net-detail-line is-soft', base ? (base.models ? t('aiNetModelsHere', { count: base.models }) : t('aiNetNotHere')) : item.sub)); + for (const edge of edges.filter(entry => touches(entry, chosen))) { + const label = base ? edge.item.title : baseLabel(edge.base); + const row = button(`anomalous-ai-net-row is-${edge.look.tone}${edge.look.ai ? ' is-ai' : ''}`, '', () => choose({ base: edge.base, kind: edge.item.kind, name: edge.item.name }, redraw)); + row.append(el('span', 'anomalous-ai-net-row-name', label), el('span', 'anomalous-ai-net-row-count', short(edge.count))); + box.append(row); + } + const open = untried(bases, items, edges); + if (open.length) box.append(el('div', 'anomalous-ai-net-detail-line is-soft', t('aiNetUntried', { names: open.map(([key, entry]) => (base ? entry.title : baseLabel(key))).join('、') }))); + if (base?.models) box.append(train(t('aiNetTrainBase'), { base: base.base })); + if (item?.usable) box.append(train(t('aiNetTrainItem'), { item })); + return box; +} + +function graphBox(data, ctx, redraw) { + const items = itemsOf(data); + const edges = edgesOf(data, items); + const bases = basesOf(data, edges); + for (const edge of edges) edge.onClick = () => choose({ base: edge.base, kind: edge.item.kind, name: edge.item.name }, redraw); + const graph = el('div', 'anomalous-ai-net-graph'); + const svg = document.createElementNS(SVG, 'svg'); + svg.setAttribute('class', 'anomalous-ai-net-lines'); + svg.setAttribute('aria-hidden', 'true'); + const nodes = new Map(); + const column = (head, entries, keyOf, labelOf, subOf, pickOf, extra) => { + const box = el('div', `anomalous-ai-net-col ${extra}`); + box.append(el('div', 'anomalous-ai-net-head', head)); + for (const entry of entries) { + const pick = pickOf(entry); + const lit = JSON.stringify(chosen) === JSON.stringify(pick); + const dim = chosen && !lit && !edges.some(edge => touches(edge, chosen) && touches(edge, pick)) && !untried(bases, items, edges).some(([base, item]) => keyOf(entry) === `base:${base}` || keyOf(entry) === item.key); + const node = button(`anomalous-ai-net-node${lit ? ' is-chosen' : ''}${dim ? ' is-dim' : ''}`, '', () => choose(pick, redraw)); + node.setAttribute('aria-pressed', String(lit)); + node.append(el('span', 'anomalous-ai-net-node-name', labelOf(entry)), el('span', 'anomalous-ai-net-node-sub', subOf(entry))); + nodes.set(keyOf(entry), node); + box.append(node); + } + return box; + }; + graph.append(svg, + column(t('aiNetModels'), bases, base => `base:${base.base}`, base => baseLabel(base.base), + base => (base.models ? t('aiNetModelsHere', { count: base.models }) : t('aiNetNotHere')), base => ({ base: base.base }), 'is-bases'), + column(t('aiNetItems'), items, item => item.key, item => item.title, item => item.sub, item => ({ kind: item.kind, name: item.name }), 'is-items')); + const draw = () => drawLines(svg, graph, nodes, edges, untried(bases, items, edges)); + watcher?.disconnect(); + watcher = new ResizeObserver(draw); + watcher.observe(graph); + return [graph, detailBox(data, bases, items, edges, ctx, redraw)]; +} + +function draw(pane, ctx, redraw) { + const body = el('div', 'anomalous-ai-stage-body anomalous-ai-net'); + if (!shown || shown.error) { + body.append(el('div', 'anomalous-ai-stage-empty', shown ? t('aiNetUnread', { error: shown.error }) : t('aiNetReading'))); + } else if (setup) { + body.append(trainingSetup(shown, setup, { back: () => { setup = null; redraw(); }, openTask: ctx.openTask })); + } else { + body.append(el('div', 'anomalous-ai-blocks-note', t('aiNetIntro')), + button('anomalous-ai-net-train', t('aiNetTrain'), () => { setup = {}; redraw(); }), ...graphBox(shown, ctx, redraw)); + } + pane.replaceChildren(sideHead(t('aiSideNetwork'), ctx.close), body); +} + +/** The view into `pane`; `ctx`: {close, owner, openTask(id)}. `fresh`: just opened, so the network is read again. */ +export function renderNetworkView(pane, ctx, fresh = false) { + const redraw = () => draw(pane, ctx, redraw); + if ((fresh || !shown) && !reading) { + reading = fetch('/anomalous/ai_training').then(async (response) => { + const data = await response.json().catch(() => ({})); + if (!response.ok) throw new Error(data.error || response.statusText); + shown = data; + }).catch((error) => { shown = { error: error.message }; }).finally(() => { reading = null; redraw(); }); + } + redraw(); +} diff --git a/web/modules/ui_ai_side_views.js b/web/modules/ui_ai_side_views.js index 79c8704..9ebfbb8 100644 --- a/web/modules/ui_ai_side_views.js +++ b/web/modules/ui_ai_side_views.js @@ -3,8 +3,8 @@ * of the window (its title bar, on the AI page only), as an app's side panel is: the stage (the AI's panels, ui_ai_side.js), the * shape of a run at full size (ui_ai_run_map.js; the newest run, or one picked from the list or * from its step in the timeline), the task's decisions (ui_ai_decisions.js), the materials - * picker (ui_ai_toolbox.js) docked there, what the task made (ui_ai_outputs.js), or the ready-made - * ways the AI may use (ui_ai_blocks.js). + * picker (ui_ai_toolbox.js) docked there, what the task made (ui_ai_outputs.js), the ready-made + * ways the AI may use (ui_ai_blocks.js), or the training network (ui_ai_network.js). * Picking the view that is shown closes the column. Each view keeps its own pane, so going back * finds it as it was: a scrolled stage, materials picked but not handed over yet. The column is a * card of its own beside the timeline; its left edge drags it wider or narrower (kept on this @@ -23,6 +23,7 @@ import { openQuestions, panelSteps, renderStage, sideHead, tabBar } from './ui_a import { renderDecisionsView } from './ui_ai_decisions.js'; import { renderOutputsView } from './ui_ai_outputs.js'; import { renderBlocksView } from './ui_ai_blocks.js'; +import { renderNetworkView } from './ui_ai_network.js'; const WIDTH_KEY = 'anomalous.aiSideWidth'; const WIDTH = { min: 340, start: 440 }; @@ -41,7 +42,7 @@ export function sideWidth() { export const VIEWS = { stage: ['panel', 'aiStage'], map: ['nodes', 'aiSideMap'], decisions: ['pin', 'aiSideDecisions'], toolbox: ['sparkle', 'aiToolbox'], outputs: ['tray', 'aiSideOutputs'], - blocks: ['blocks', 'aiSideBlocks'], + blocks: ['blocks', 'aiSideBlocks'], network: ['pulse', 'aiSideNetwork'], }; function el(tag, className, text) { @@ -212,6 +213,7 @@ export function drawSide(box, task, side, ctx, pickTab, arrived = null) { const panes = sidePanes(box); if (panes.toolbox.dataset.task && panes.toolbox.dataset.task !== task.id) closeToolbox(); const opened = panes.blocks.hidden && side.view === 'blocks'; // the ways are read again when the view comes up + const netOpened = panes.network.hidden && side.view === 'network'; // so is the training network for (const [view, pane] of Object.entries(panes)) pane.hidden = view !== side.view; const asks = panelSteps(task).reduce((sum, step) => sum + openQuestions(step), 0); box.querySelector(':scope > .anomalous-ai-tabs').replaceWith(tabBar(side, asks, pickTab, viewLabel(side.view))); @@ -230,6 +232,7 @@ export function drawSide(box, task, side, ctx, pickTab, arrived = null) { panes.blocks.dataset.private = privacy; renderBlocksView(panes.blocks, ctx, opened, task.private === true); } + if (side.view === 'network' && (netOpened || !panes.network.childElementCount)) renderNetworkView(panes.network, ctx, netOpened); } /** diff --git a/web/modules/ui_ai_training_setup.js b/web/modules/ui_ai_training_setup.js new file mode 100644 index 0000000..8480b47 --- /dev/null +++ b/web/modules/ui_ai_training_setup.js @@ -0,0 +1,173 @@ +/** + * Starting a training round (api/ai_training.py) from the training network (ui_ai_network.js): what + * to try, a skill (the AI's prompts with its advice and without, side by side) or the ready-made ways + * of one use; on which main models, up to four, listed by base model; how many pictures each (seeds); + * what to draw, or the picture the ways work on (from the gallery). Then the task waits for the AI + * and the sentence to paste into the AI app is shown, as handing a task over does (ui_ai_handoff.js). + * POST /anomalous/ai_training. + */ + +import { resolveLocale, translate as t } from './locales.js'; +import { copyLine } from './ui_ai_connect.js'; +import { galleryPicker, thumbUrl } from './ui_ai_gallery_pick.js'; + +const MAX_MODELS = 4; +const words = value => (resolveLocale() === 'zh' ? value?.zh : value?.en) || value?.en || value?.zh || ''; + +function el(tag, className, text) { + const node = document.createElement(tag); + if (className) node.className = className; + if (text !== undefined) node.textContent = String(text); + return node; +} + +function button(className, label, onClick) { + const node = el('button', className, label); + node.type = 'button'; + node.onclick = onClick; + return node; +} + +function field(label, ...content) { + const box = el('div', 'anomalous-ai-train-field'); + box.append(el('div', 'anomalous-ai-train-label', label), ...content); + return box; +} + +/** What a round can try: each skill on, each use with a way that runs here. */ +function targetsOf(data) { + const out = data.skills.filter(skill => skill.on).map(skill => ({ value: `skill:${skill.name}`, skill: skill.name, title: words(skill.title), + label: t('aiTrainSkill', { name: words(skill.title) }) })); + for (const job of data.jobs) { + const ways = data.blocks.filter(block => block.job === job && block.usable && block.on); + if (ways.length) out.push({ value: `use:${job}`, use: job, title: t(`aiBlocksJob_${job}`), takes: ways.some(way => way.takes), + label: t('aiTrainUse', { job: t(`aiBlocksJob_${job}`), count: ways.length }), kinds: ways.length }); + } + return out; +} + +/** + * The form for `data` (the network, GET /anomalous/ai_training); `preset`: {item?, base?} picked in the + * network; `back()` returns to it, `openTask(id)` shows the task made. + */ +export function trainingSetup(data, preset, { back, openTask }) { + const box = el('div', 'anomalous-ai-train'); + const targets = targetsOf(data); + const item = preset.item; + const wanted = item?.kind === 'skill' ? `skill:${item.name}` : item?.kind === 'way' ? `use:${data.blocks.find(block => block.name === item.name)?.job}` : ''; + let target = targets.find(entry => entry.value === wanted) || targets[0]; + const models = new Set(data.models.filter(model => model.base === preset.base).slice(0, 1).map(model => model.model_id)); + let each = 2; + let picture = null; + const note = el('div', 'anomalous-ai-train-note'); + box.append(button('anomalous-ai-train-back', t('aiTrainBack'), back), el('div', 'anomalous-ai-net-detail-title', t('aiTrainHead'))); + if (!targets.length || !data.models.length) { + box.append(el('div', 'anomalous-ai-stage-empty', t(data.models.length ? 'aiTrainNothing' : 'aiTrainModelsNone'))); + return box; + } + + const pick = el('select', 'anomalous-ai-train-select'); + for (const entry of targets) { + const option = el('option', '', entry.label); + option.value = entry.value; + option.selected = entry === target; + pick.append(option); + } + const list = el('div', 'anomalous-ai-train-models'); + const byBase = new Map(); + for (const model of data.models) byBase.set(model.base, [...(byBase.get(model.base) || []), model]); + const boxes = []; + for (const [base, entries] of [...byBase].sort((a, b) => (b[0] === preset.base) - (a[0] === preset.base) || a[0].localeCompare(b[0]))) { + list.append(el('div', 'anomalous-ai-train-base', base || t('aiRecordOther'))); + for (const model of entries) { + const row = el('label', 'anomalous-ai-train-model'); + const check = el('input'); + check.type = 'checkbox'; + check.checked = models.has(model.model_id); + check.onchange = () => { + if (check.checked) models.add(model.model_id); + else models.delete(model.model_id); + update(); + }; + boxes.push([check, model]); + row.append(check, el('span', '', model.name)); + list.append(row); + } + } + const subject = el('textarea', 'anomalous-ai-prompt-field'); + subject.rows = 2; + subject.value = t('aiTrainSubjectDefault'); + const shot = el('div', 'anomalous-ai-train-picture'); + const tray = el('div', 'anomalous-ai-prompt-tray'); + const choose = button('anomalous-ai-prompt-pick', t('aiTrainPick'), () => picker.toggle()); + const picker = galleryPicker(shot, () => 1, (items) => { + const image = items[0]?.image || items[0]; + if (!image) return; + picture = [image.subfolder, image.filename].filter(Boolean).join('/'); + const tile = el('span', 'anomalous-ai-prompt-pic'); + const img = el('img'); + img.src = thumbUrl(image); + img.alt = ''; + tile.append(img); + tray.replaceChildren(tile); + update(); + }, choose, { hint: 'aiTrainPicsHint', full: 'aiTrainPicsFull', add: 'aiTrainPicsAdd' }); + shot.append(tray, choose); + const counts = el('div', 'anomalous-ai-gates-choice anomalous-ai-train-seg'); + const segment = [1, 2, 3].map((value) => { + const node = button('anomalous-ai-gates-option', String(value), () => { each = value; update(); }); + counts.append(node); + return [node, value]; + }); + const sum = el('div', 'anomalous-ai-train-sum'); + const send = button('anomalous-ai-net-train', t('aiTrainStart'), start); + const what = field(t('aiTrainSubject'), subject); + const where = field(t('aiTrainPicture'), shot); + + function update() { + target = targets.find(entry => entry.value === pick.value) || targets[0]; + what.hidden = !target.skill; + where.hidden = !target.takes; + for (const [check, model] of boxes) check.disabled = !models.has(model.model_id) && models.size >= MAX_MODELS; + segment.forEach(([node, value]) => node.classList.toggle('is-on', value === each)); + const kinds = target.skill ? 2 : target.kinds; + sum.textContent = t('aiTrainCount', { total: models.size * kinds * each, models: models.size, kinds, each }); + note.textContent = ''; + } + pick.onchange = update; + + async function start() { + if (!models.size) { + note.textContent = t('aiTrainNeedModel'); + return; + } + if (target.takes && !picture) { + note.textContent = t('aiTrainNeedPicture'); + return; + } + const title = t('aiTrainTaskTitle', { what: target.title, count: models.size }); + send.disabled = true; + try { + const response = await fetch('/anomalous/ai_training', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ title, models: [...models], each, skill: target.skill || '', use: target.use || '', + subject: target.skill ? subject.value.trim() : '', picture: target.takes ? picture : '' }), + }); + const answer = await response.json().catch(() => ({})); + if (!response.ok) throw new Error(answer.error || response.statusText); + const done = el('div', 'anomalous-ai-train-done'); + done.append(el('p', 'anomalous-ai-handoff-text', t('aiTrainMade')), + copyLine(t('aiHandSentence', { task: title }), { code: false, label: t('aiConnectCopySentence') }), + button('anomalous-ai-net-train is-small', t('aiTrainOpen'), () => openTask?.(answer.task.id))); + box.replaceChildren(button('anomalous-ai-train-back', t('aiTrainBack'), back), done); + } catch (error) { + send.disabled = false; + note.textContent = error.message; + } + } + + box.append(field(t('aiTrainWhat'), pick), field(t('aiTrainModels', { max: MAX_MODELS }), list), what, where, + field(t('aiTrainEach'), counts), sum, el('div', 'anomalous-ai-blocks-note', t('aiTrainHow')), send, note); + update(); + return box; +} diff --git a/web/modules/ui_ai_workbench.js b/web/modules/ui_ai_workbench.js index aa04e92..9b30e8d 100644 --- a/web/modules/ui_ai_workbench.js +++ b/web/modules/ui_ai_workbench.js @@ -327,6 +327,7 @@ function drawStage(task, arrived = null) { changed: () => void refresh(), numberOf: id => state.numbers.get(id) || '', showCanvas: () => state.owner?.close?.(), + openTask: id => void select(id), close: () => { state.stage.hidden = true; state.stage.tab = 'timeline'; diff --git a/web/styles.css b/web/styles.css index 4514c0c..1a90fb6 100644 --- a/web/styles.css +++ b/web/styles.css @@ -48,3 +48,4 @@ @import url("./styles/47-ai-decisions.css?v=20261024-feel"); @import url("./styles/48-ai-blocks.css?v=20261027-ways"); @import url("./styles/49-model-versions.css?v=20261030-versions"); +@import url("./styles/50-ai-network.css?v=20261101-network"); diff --git a/web/styles/50-ai-network.css b/web/styles/50-ai-network.css new file mode 100644 index 0000000..90e57d5 --- /dev/null +++ b/web/styles/50-ai-network.css @@ -0,0 +1,124 @@ +/* The side column's training network (ui_ai_network.js): models left, ways and skills right, the + lines between them coloured by how they turned out; the training round's form under it + (ui_ai_training_setup.js). */ + +.anomalous-ai-net { display: flex; flex-direction: column; gap: 12px; } +.anomalous-ai-net-train { + align-self: flex-start; + padding: 6px 14px; + border: none; + border-radius: var(--amb-radius-control, 7px); + background: var(--amb-btn-primary-bg); + color: var(--amb-btn-primary-text); + font: inherit; + font-size: 13px; + cursor: pointer; +} +.anomalous-ai-net-train:hover { background: var(--amb-btn-primary-hover); } +.anomalous-ai-net-train:disabled { cursor: progress; opacity: 0.7; } +.anomalous-ai-net-train.is-small { padding: 4px 11px; font-size: 12.5px; } + +.anomalous-ai-net-graph { + position: relative; + display: grid; + grid-template-columns: minmax(0, 40%) minmax(40px, 1fr) minmax(0, 44%); +} +.anomalous-ai-net-lines { position: absolute; inset: 0; z-index: 0; overflow: visible; pointer-events: none; } +.anomalous-ai-net-col { position: relative; z-index: 1; display: flex; flex-direction: column; gap: 6px; min-width: 0; } +.anomalous-ai-net-col.is-bases { grid-column: 1; } +.anomalous-ai-net-col.is-items { grid-column: 3; } +.anomalous-ai-net-head { margin-bottom: 2px; color: var(--amb-text-soft); font-size: 12px; font-weight: 600; } +.anomalous-ai-net-node { + display: flex; + flex-direction: column; + align-items: flex-start; + gap: 1px; + min-width: 0; + padding: 6px 9px; + border: 1px solid var(--amb-border); + border-radius: 8px; + background: var(--amb-bg-card); + color: var(--amb-text-main); + font: inherit; + text-align: left; + cursor: pointer; + transition: opacity 0.15s, border-color 0.15s; +} +.anomalous-ai-net-node:hover { border-color: var(--amb-border-hover); } +.anomalous-ai-net-node.is-chosen { border-color: var(--amb-link); box-shadow: 0 0 0 1px var(--amb-link); } +.anomalous-ai-net-node.is-dim { opacity: 0.45; } +.anomalous-ai-net-node-name { max-width: 100%; overflow: hidden; font-size: 12.5px; font-weight: 600; text-overflow: ellipsis; white-space: nowrap; } +.anomalous-ai-net-node-sub { max-width: 100%; overflow: hidden; color: var(--amb-text-dim); font-size: 11px; text-overflow: ellipsis; white-space: nowrap; } + +.anomalous-ai-net-line { fill: none; stroke-linecap: round; opacity: 0.85; transition: opacity 0.15s; } +.anomalous-ai-net-line.is-good { stroke: var(--amb-ok); } +.anomalous-ai-net-line.is-fair { stroke: var(--amb-warn); } +.anomalous-ai-net-line.is-bad { stroke: var(--amb-danger); } +.anomalous-ai-net-line.is-ran { stroke: rgba(var(--amb-ink-rgb), 0.35); } +.anomalous-ai-net-line.is-ai { stroke-dasharray: 2 4; } +.anomalous-ai-net-line.is-untried { stroke: rgba(var(--amb-ink-rgb), 0.3); stroke-dasharray: 5 5; opacity: 1; } +.anomalous-ai-net-line.is-dim { opacity: 0.12; } +.anomalous-ai-net-line.is-chosen { opacity: 1; filter: drop-shadow(0 0 2px currentColor); } +.anomalous-ai-net-hit { fill: none; stroke: transparent; stroke-width: 12px; pointer-events: stroke; cursor: pointer; } + +/* Kept in sight at the column's foot while the graph scrolls. */ +.anomalous-ai-net-detail { + position: sticky; + bottom: 0; + z-index: 2; + display: flex; + flex-direction: column; + gap: 5px; + flex: none; + max-height: 40vh; + overflow-y: auto; + padding: 10px 12px; + border: 1px solid var(--amb-border-strong); + border-radius: 9px; + background: var(--amb-bg-panel); + box-shadow: 0 -6px 16px rgba(0, 0, 0, 0.18); +} +.anomalous-ai-net-hint { color: var(--amb-text-soft); font-size: 12.5px; line-height: 1.5; } +.anomalous-ai-net-detail-title { color: var(--amb-text-main); font-size: 13.5px; font-weight: 600; } +.anomalous-ai-net-detail-line { color: var(--amb-text-main); font-size: 12.5px; line-height: 1.5; } +.anomalous-ai-net-detail-line.is-soft { color: var(--amb-text-soft); } +.anomalous-ai-net-row { + display: flex; + justify-content: space-between; + gap: 10px; + padding: 4px 8px; + border: none; + border-left: 3px solid rgba(var(--amb-ink-rgb), 0.3); + border-radius: 0; + background: none; + color: var(--amb-text-main); + font: inherit; + font-size: 12.5px; + text-align: left; + cursor: pointer; +} +.anomalous-ai-net-row:hover { background: rgba(var(--amb-ink-rgb), 0.05); } +.anomalous-ai-net-row.is-good { border-left-color: var(--amb-ok); } +.anomalous-ai-net-row.is-fair { border-left-color: var(--amb-warn); } +.anomalous-ai-net-row.is-bad { border-left-color: var(--amb-danger); } +.anomalous-ai-net-row.is-ai { border-left-style: dashed; } +.anomalous-ai-net-row-name { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +.anomalous-ai-net-row-count { flex: none; color: var(--amb-text-soft); } + +/* The training round's form. */ +.anomalous-ai-train { display: flex; flex-direction: column; gap: 10px; } +.anomalous-ai-train-back { align-self: flex-start; padding: 0; border: none; background: none; color: var(--amb-link); font: inherit; font-size: 12.5px; cursor: pointer; } +.anomalous-ai-train-field { display: flex; flex-direction: column; gap: 5px; } +.anomalous-ai-train-field[hidden] { display: none; } +.anomalous-ai-train-label { color: var(--amb-text-soft); font-size: 12px; font-weight: 600; } +.anomalous-ai-train-select { padding: 5px 8px; border: 1px solid var(--amb-border-strong); border-radius: 7px; background: var(--amb-bg-input); color: var(--amb-text-main); font: inherit; font-size: 13px; } +.anomalous-ai-train-models { display: flex; flex-direction: column; gap: 2px; max-height: 220px; overflow-y: auto; padding: 4px 8px; border: 1px solid var(--amb-border); border-radius: 8px; } +.anomalous-ai-train-base { margin-top: 4px; color: var(--amb-text-dim); font-size: 11.5px; font-weight: 600; } +.anomalous-ai-train-model { display: flex; align-items: center; gap: 6px; color: var(--amb-text-main); font-size: 12.5px; cursor: pointer; } +.anomalous-ai-train-model input { accent-color: var(--amb-link); } +.anomalous-ai-train-picture { position: relative; display: flex; flex-wrap: wrap; align-items: center; gap: 8px; } +.anomalous-ai-train-seg { align-self: flex-start; } +.anomalous-ai-train-sum { color: var(--amb-text-main); font-size: 12.5px; } +.anomalous-ai-train-note { color: var(--amb-warn); font-size: 12.5px; } +.anomalous-ai-train-note:empty { display: none; } +.anomalous-ai-train-done { display: flex; flex-direction: column; gap: 10px; }