Repaint an area: the marker's brush makes an inpaint mask for the AI
The marker gains a repaint brush (size adjustable) and "make a repaint mask": the page sends a mask (white where boxes, circles, lines and brush strokes are) and the server writes the picture with that area as its alpha to ComfyUI's input folder (anomalous_inpaint_<step>.png, a new file each time), so LoadImage gives the picture and the area as MASK. The AI's marks message carries the file and how to inpaint with it; the timeline says the mask is ready. Canvas tools read ComfyUI's lists anew once when a value is not among a drop-down's choices, so they take the new file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
353b5d9ed5
commit
bb744a8c38
+8
-2
@@ -205,8 +205,14 @@ covered by the `styles.css` manifest.
|
||||
`anomalous/ai_marks/<task>/`, never the output folder, gone with the task; the marks as
|
||||
fractions with notes): a `user_marks` step the AI receives like a message (kind `marks`) with
|
||||
the marked picture attached as an image to the same answer. `web/modules/ui_ai_marker.js` is
|
||||
the marker window (box, circle, arrow, pen; three colours; a note per mark) opened from a
|
||||
picture's ✎ on the AI page, and the timeline row of marks sent.
|
||||
the marker window (box, circle, arrow, pen; three colours; a note per mark; a brush for the
|
||||
area to repaint) opened from a picture's ✎ on the AI page, and the timeline row of marks sent.
|
||||
With "make a repaint mask" the page also sends a mask (white where boxes, circles, lines and
|
||||
brush strokes are); the server writes the picture with that area as its alpha to ComfyUI's
|
||||
input folder as `anomalous_inpaint_<step>.png` (a new file, never replacing one; kept when the
|
||||
task goes, as workflows may load it), and the AI's marks message carries its name with how to
|
||||
inpaint with it (LoadImage gives the MASK). The canvas tools read ComfyUI's lists anew once
|
||||
when a value is not among a drop-down's choices, so they take such a new file.
|
||||
`web/modules/ui_ai_gate.js` draws the card under a step that asks and the Gates window;
|
||||
`ui_ai_flow.js` the flow over a task (its stages in order, each done, running, waiting or
|
||||
failed; a stage leads to its first step). `ui_ai_handoff.js` is "Hand to AI" on a model's
|
||||
|
||||
@@ -6,6 +6,10 @@
|
||||
- AI 可以在你眼前的画布上搭工作流了:查这台 ComfyUI 有哪些节点(包括自定义节点)、每个节点有哪些输入输出和可选项;新建一个工作流标签页(空白的,或者一次搭好整套并自动排好位置,不动你正在开的那个);加节点、连线、改参数、删节点。每一步都在画布上当场发生,是一次 Ctrl+Z,记在操作记录里,并在 AI 工作台那一步下面显示改了什么,连线也会写成“某节点的输入 model:(空)→ 加载模型 #4 · MODEL”。连不上的类型、不存在的选项、写错的参数会先被拦下并告诉 AI 正确的写法,画布不会留下半截。新建的标签页名字带时间,不会打开或覆盖你存过的工作流。连线会绕成死循环时也会先拦下。运行被 ComfyUI 拒绝时,AI 收到的是 ComfyUI 给的具体原因(哪个节点、什么问题),不再只有一句“运行失败”。
|
||||
- The AI can build workflows on the canvas the user sees: look up this ComfyUI's node types (custom nodes too) with their inputs, outputs and choices; open a new tab, blank or built and laid out from a workflow in API form (the open one is left alone); add, wire, set and remove nodes. Each is one Ctrl+Z step, logged as the AI's and shown under its AI workbench step, rewiring included. Wrong types, choices and settings are refused with what fits, before anything changes. Wiring that would loop is refused too, and a run ComfyUI rejects reports ComfyUI's own reason.
|
||||
|
||||
### 局部重绘:涂一涂就交给 AI / Repaint an area: paint it, hand it to the AI
|
||||
- 标注窗口多了“▧ 涂重绘区”笔刷(粗细可调)和“同时做成重绘遮罩”开关:涂上要重画的地方(框、圈、画笔圈的地方也算)发给 AI,AMB 就在 ComfyUI 的 input 文件夹里做好一张带遮罩的原图(`anomalous_inpaint_….png`),AI 用 LoadImage 读进来就能局部重绘;AI 同时收到怎么接、重绘强度怎么选的说明。时间线上你的标注下面写着“已做成重绘遮罩”。AI 搭工作流时如果刚生成的文件还不在下拉列表里,AMB 会自动刷新一次列表。
|
||||
- The marker's repaint brush and "make a repaint mask": the picture with the painted area as its alpha goes to ComfyUI's input folder for LoadImage, and the AI is told how to inpaint with it. Canvas tools refresh ComfyUI's lists once when a choice is missing.
|
||||
|
||||
### AI 的问题在 AMB 里问 / The AI asks in Anomalous
|
||||
- 你在图上标注或给 AI 留言后,AI 的分析和修改方案也回在 AMB 里:它的留言会写明“回复第几步”,你那条标注下面出现“AI 已回复 · 第几步”,两边点一下就互相跳过去。AI 的留言现在有排版:小标题、列表、加粗、`代码` 都会正常显示(只认这几种写法,不会运行网页代码)。AMB 告诉 AI:收到标注和留言就在 AMB 里回复,要你选修法就用选择题问。
|
||||
- The AI replies in Anomalous: `report` takes `reply_to` (the user's message or marks), both steps link to each other; its notes are laid out (headings, lists, bold, code) with text only.
|
||||
|
||||
+40
-6
@@ -7,6 +7,10 @@ like a message (ai_panels._collect, kind "marks": the notes with where each mark
|
||||
fractions of the picture) and, attached to the same answer, the marked picture as a thumbnail
|
||||
(image_items), so an AI that can see pictures sees the marks. Only an output picture can be
|
||||
marked; the PNG is checked (signature, size, dimensions) before it is kept.
|
||||
With a mask (the marker's "repaint" brush, and the boxes, circles and lines; white where to
|
||||
repaint), the picture is also written to ComfyUI's input folder as anomalous_inpaint_<step>.png
|
||||
with that area as its alpha, so LoadImage gives the picture and the area as its MASK for an
|
||||
inpaint workflow; the AI gets the file name with the marks.
|
||||
"""
|
||||
|
||||
import base64
|
||||
@@ -16,6 +20,8 @@ import re
|
||||
import time
|
||||
import uuid
|
||||
|
||||
import folder_paths
|
||||
|
||||
from . import ai_tasks
|
||||
from .mcp_tools import _image_content
|
||||
|
||||
@@ -71,8 +77,8 @@ def _fraction(value):
|
||||
raise MarksError("Mark positions are fractions of the picture")
|
||||
|
||||
|
||||
def _clean(marks):
|
||||
if not isinstance(marks, list) or not marks or len(marks) > MAX_MARKS:
|
||||
def _clean(marks, masked=False):
|
||||
if not isinstance(marks, list) or (not marks and not masked) or len(marks) > MAX_MARKS:
|
||||
raise MarksError(f"Between 1 and {MAX_MARKS} marks")
|
||||
out = []
|
||||
for mark in marks:
|
||||
@@ -94,8 +100,32 @@ def _words(marks, text):
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def save(task_id, image, png, marks, text="", now=None):
|
||||
"""Keeps the marked picture and adds the user's marks to the task. Returns the step."""
|
||||
def _inpaint_file(picture, mask, step_id):
|
||||
"""The picture with mask `mask` (PNG bytes, white where to repaint) as its alpha, in ComfyUI's
|
||||
input folder (a new file, never one that is there): its name for LoadImage."""
|
||||
from PIL import Image, ImageOps
|
||||
|
||||
source = os.path.join(folder_paths.get_output_directory(), picture["subfolder"], picture["filename"])
|
||||
try:
|
||||
with Image.open(source) as original, Image.open(io.BytesIO(mask)) as drawn:
|
||||
area = drawn.convert("L").resize(original.size, Image.NEAREST).point(lambda value: 255 if value > 127 else 0)
|
||||
if not area.getbbox():
|
||||
raise MarksError("The repaint area is empty")
|
||||
out = original.convert("RGB")
|
||||
out.putalpha(ImageOps.invert(area)) # ComfyUI's LoadImage: MASK = 1 - alpha
|
||||
except MarksError:
|
||||
raise
|
||||
except Exception: # noqa: BLE001 - a picture PIL cannot read cannot be repainted either
|
||||
raise MarksError("The picture cannot be read for repainting")
|
||||
name = f"anomalous_inpaint_{step_id}.png"
|
||||
with open(os.path.join(folder_paths.get_input_directory(), name), "xb") as target: # never replaces a file
|
||||
out.save(target, "PNG")
|
||||
return name
|
||||
|
||||
|
||||
def save(task_id, image, png, marks, text="", now=None, mask=None):
|
||||
"""Keeps the marked picture and adds the user's marks to the task; with `mask` (a PNG data
|
||||
URL), also the picture ready to repaint (_inpaint_file). Returns the step."""
|
||||
try:
|
||||
picture = ai_tasks._output_media(image)
|
||||
except (ValueError, OSError):
|
||||
@@ -103,10 +133,14 @@ def save(task_id, image, png, marks, text="", now=None):
|
||||
if not picture:
|
||||
raise MarksError("Only a picture in the output folder can be marked")
|
||||
data = _png(png)
|
||||
marks = _clean(marks)
|
||||
mask = _png(mask) if mask else None
|
||||
marks = _clean(marks, masked=mask is not None)
|
||||
text = ai_tasks._short(str(text or "").strip(), 2000)
|
||||
step_id = uuid.uuid4().hex[:10]
|
||||
now = time.time() if now is None else now
|
||||
if not ai_tasks.get_task(task_id):
|
||||
raise MarksError("No such task")
|
||||
inpaint = _inpaint_file(picture, mask, step_id) if mask else None # outside the lock: a picture takes a moment
|
||||
with ai_tasks._lock:
|
||||
task = ai_tasks._load().get(task_id)
|
||||
if not task:
|
||||
@@ -116,7 +150,7 @@ def save(task_id, image, png, marks, text="", now=None):
|
||||
target.write(data)
|
||||
step = {"id": step_id, "at": now, "ms": 0, "tool": "user_marks", "kind": "user", "ok": True, "state": "done",
|
||||
"summary": "user_marks", "text": _words(marks, text), "note": text, "image": picture, "marks": marks,
|
||||
"marked": f"{step_id}.png", "delivered": False, "results": []}
|
||||
"marked": f"{step_id}.png", "delivered": False, "results": [], **({"inpaint": inpaint} if inpaint else {})}
|
||||
task["steps"].append(step)
|
||||
del task["steps"][:-ai_tasks.MAX_STEPS]
|
||||
task["updated"] = max(task.get("updated", 0), now)
|
||||
|
||||
@@ -37,6 +37,14 @@ MAX_WAIT = 55
|
||||
TONES = ("normal", "muted", "tip", "warning")
|
||||
LAYOUTS = ("row", "grid", "compare")
|
||||
INTERACTIVE = ("choices", "input", "picks")
|
||||
INPAINT_NOTE = (
|
||||
"The user painted where to repaint. LoadImage(image=inpaint_image) gives the picture and, as MASK, that area. "
|
||||
"Use the picture's own model and prompt (get_image_info on image), e.g. new_workflow: CheckpointLoaderSimple, "
|
||||
"LoadImage, CLIPTextEncode x2, VAEEncode(pixels, vae) -> SetLatentNoiseMask(samples, mask) -> KSampler "
|
||||
"(denoise 0.5-0.8 keeps the look; 1.0 with VAEEncodeForInpaint(grow_mask_by 8) replaces it) -> VAEDecode -> "
|
||||
"SaveImage; GrowMask softens the edge. Say in report what you set and why, ask_user the strength if unsure, "
|
||||
"and show the old and new picture with a slider panel."
|
||||
)
|
||||
COLORS = ("red", "green", "blue", "yellow", "purple")
|
||||
MAX_MARKS = 8
|
||||
MAX_NODES = 24
|
||||
@@ -446,6 +454,8 @@ def _collect(client, panel_id, task_id=None):
|
||||
# Marks on a picture: the picture's path for the AI, and the marked one attached (ai_marks.image_items).
|
||||
message.update(kind="marks", image="/".join(part for part in (step["image"].get("subfolder"), step["image"]["filename"]) if part),
|
||||
_marked=ai_marks.path_of(task["id"], step.get("marked")))
|
||||
if step.get("inpaint"):
|
||||
message.update(inpaint_image=step["inpaint"], inpaint_note=INPAINT_NOTE)
|
||||
messages.append(message)
|
||||
continue
|
||||
asked = step.get("gate") or {}
|
||||
|
||||
@@ -129,7 +129,7 @@ async def api_marks(request):
|
||||
return web.json_response({"error": "Expected {image, png, marks}"}, status=400)
|
||||
try:
|
||||
step = await asyncio.to_thread(ai_marks.save, request.match_info["task_id"], body.get("image"),
|
||||
body.get("png"), body.get("marks"), body.get("text"))
|
||||
body.get("png"), body.get("marks"), body.get("text"), None, body.get("mask"))
|
||||
except ai_marks.MarksError as error:
|
||||
return web.json_response({"error": str(error)}, status=400)
|
||||
ai_panels.wake()
|
||||
|
||||
@@ -129,6 +129,8 @@ claude mcp add --transport http anomalous http://127.0.0.1:8188/anomalous/mcp
|
||||
|
||||
**在图上标给 AI 看**:工作台里图片右上角的 **✎** 打开标注:框、圈、箭头、画笔,三种颜色,每个标记写一句要怎么改,再写整体的想法,点 **发给 AI**。AI 下一次调用工具(或 `get_answers`)时收到这些说明和标好的图;能看图的 AI 直接看得到标在哪里。标好的图存在 AMB 自己的记录里,不进图库,任务删掉时一起删。
|
||||
|
||||
**让 AI 局部重绘**:标注里选 **▧ 涂重绘区**,用笔刷把要重画的地方涂上(笔刷粗细可调;框、圈、画笔圈出的地方也算),勾上“同时做成重绘遮罩”再发给 AI。AMB 会在 ComfyUI 的 input 文件夹里放一张 `anomalous_inpaint_….png`:原图加上你涂的区域做遮罩。AI 收到时会被告诉怎么用它(LoadImage 读进来就带遮罩,用原图的模型和提示词,重绘强度 0.5~0.8 保留原样子,1.0 完全重画),不确定时用选择题问你强度,画完用前后对比给你看。这张文件删任务时不会删,因为你的工作流可能还在用它。
|
||||
|
||||
**让 AI 在画布上搭工作流**:AI 可以先用 `find_nodes` 查这台 ComfyUI 有哪些节点、每个节点要接什么,然后用 `new_workflow` 新开一个标签页(空白,或者一次把整套工作流搭好并排好位置;你正开着的工作流不受影响),再用 `add_node`、`connect_nodes`、`set_values`、`remove_nodes` 加节点、连线、改参数、删节点。每一步你都能在画布上看到,按 Ctrl+Z 能撤回;AI 工作台那一步下面写着改了什么(包括哪根线从哪里连到哪里)。类型接不上、选项不存在时,画布不会动,AI 会收到能用的写法。
|
||||
|
||||
**给 AI 的素材**:任务标题旁边的 **✦ 素材** 打开一个大窗口:上面搜索所有素材,左边选种类,节点可以按来源筛(ComfyUI 自带、你装的每个第三方节点包、付费 API 节点),卡片上看得到节点的输入输出、搭配和工作流的封面、提示词原文;鼠标移到卡片上右边显示完整说明。提示词里也能挑搭配和工作流里的提示词。勾选后点 **交给 AI**,选“优先用这些”或“只能用这些节点”(自带节点照常可用,别的第三方和付费节点会被拦下)。☆ 设为常用:没挑素材的任务自动带上常用的。AI 用 `get_toolbox` 看到你挑的东西的详细说明,用 `open_workflow` 在新标签页打开你存的工作流。工作流详情页的 **✦ 交给 AI** 会新开一个带着这个工作流的任务。
|
||||
|
||||
@@ -80,6 +80,29 @@ class MarkTests(unittest.TestCase):
|
||||
with self.assertRaises(ToolError):
|
||||
mcp_task_tools.report({"client": "Claude", "task": self.task}, "x", reply_to=note["id"]) # not the user's
|
||||
|
||||
def test_a_repaint_mask_becomes_the_picture_with_that_area_as_its_mask(self):
|
||||
inputs = tempfile.mkdtemp(prefix="amb_ai_marks_input_")
|
||||
buffer = io.BytesIO()
|
||||
mask = Image.new("L", (64, 48), 0)
|
||||
mask.paste(255, (0, 0, 32, 48)) # the left half
|
||||
mask.save(buffer, "PNG")
|
||||
data = "data:image/png;base64," + base64.b64encode(buffer.getvalue()).decode()
|
||||
with mock.patch.object(ai_marks.folder_paths, "get_input_directory", return_value=inputs):
|
||||
step = ai_marks.save(self.task, "a.png", png(), [], "去掉多余的手", mask=data)
|
||||
with self.assertRaises(ai_marks.MarksError):
|
||||
blank = io.BytesIO()
|
||||
Image.new("L", (8, 8), 0).save(blank, "PNG")
|
||||
ai_marks.save(self.task, "a.png", png(), [], "", mask="data:image/png;base64," + base64.b64encode(blank.getvalue()).decode())
|
||||
self.assertRegex(step["inpaint"], r"^anomalous_inpaint_[0-9a-f]{10}\.png$")
|
||||
with Image.open(os.path.join(inputs, step["inpaint"])) as made:
|
||||
self.assertEqual((made.mode, made.size), ("RGBA", (8, 8)))
|
||||
self.assertEqual((made.getpixel((1, 4))[3], made.getpixel((6, 4))[3]), (0, 255), "marked: transparent, so LoadImage's MASK is 1 there")
|
||||
with mock.patch.object(ai_marks, "_image_content", return_value=None):
|
||||
message = ai_panels.attach_pending(ok({}), "Claude", task_id=self.task)["structuredContent"]["user_messages"][0]
|
||||
self.assertEqual(message["inpaint_image"], step["inpaint"])
|
||||
self.assertIn("SetLatentNoiseMask", message["inpaint_note"])
|
||||
self.assertEqual(os.listdir(inputs), [step["inpaint"]], "an empty mask writes nothing")
|
||||
|
||||
def test_what_is_refused(self):
|
||||
for image, picture, marks in [("../a.png", png(), MARKS), ("nope.png", png(), MARKS), ("a.png", "data:image/jpeg;base64,AAAA", MARKS),
|
||||
("a.png", "data:image/png;base64," + base64.b64encode(b"not a png").decode(), MARKS),
|
||||
|
||||
@@ -1263,6 +1263,10 @@ export const i18n = {
|
||||
aiMarkTool_circle: "圈",
|
||||
aiMarkTool_arrow: "箭头",
|
||||
aiMarkTool_pen: "画笔",
|
||||
aiMarkTool_mask: "涂重绘区",
|
||||
aiMarkBrush: "笔刷",
|
||||
aiMarkRepaint: "同时做成重绘遮罩:框、圈、画笔和涂抹的地方让 AI 局部重绘",
|
||||
aiMarkInpaintReady: "已做成重绘遮罩({file},在 ComfyUI 的 input 文件夹)",
|
||||
aiMarkColor: "颜色",
|
||||
aiMarkUndo: "撤销",
|
||||
aiMarkClear: "清空",
|
||||
@@ -4304,6 +4308,10 @@ export const i18n = {
|
||||
aiMarkTool_circle: "Circle",
|
||||
aiMarkTool_arrow: "Arrow",
|
||||
aiMarkTool_pen: "Pen",
|
||||
aiMarkTool_mask: "Repaint area",
|
||||
aiMarkBrush: "Brush",
|
||||
aiMarkRepaint: "Also make a repaint mask: boxes, circles, lines and painted areas for the AI to repaint",
|
||||
aiMarkInpaintReady: "Repaint mask made ({file}, in ComfyUI’s input folder)",
|
||||
aiMarkColor: "Colour",
|
||||
aiMarkUndo: "Undo",
|
||||
aiMarkClear: "Clear",
|
||||
|
||||
@@ -133,13 +133,29 @@ function prepare(nodes) {
|
||||
return { made, plans, links };
|
||||
}
|
||||
|
||||
/**
|
||||
* `make()` again after ComfyUI's lists were read anew, when a value was not among a drop-down's
|
||||
* choices: a file that has just appeared (a picture Anomalous put in the input folder to repaint).
|
||||
*/
|
||||
async function withFreshChoices(make) {
|
||||
try {
|
||||
return make();
|
||||
} catch (error) {
|
||||
if (!/has no choice/.test(error.message) || typeof app.refreshComboInNodes !== 'function') throw error;
|
||||
await app.refreshComboInNodes();
|
||||
return make();
|
||||
}
|
||||
}
|
||||
|
||||
/** The canvas-building actions, given mcp_bridge.js's own helpers. */
|
||||
export function canvasEditActions({ recorded, nodeById }) {
|
||||
function addNode({ type, title, values, near_node_id: nearId }, getOwner) {
|
||||
return recorded(getOwner, 'mcpDidNode', (g) => {
|
||||
const node = newNode(String(type));
|
||||
return recorded(getOwner, 'mcpDidNode', async (g) => {
|
||||
const { node, plan } = await withFreshChoices(() => {
|
||||
const made = newNode(String(type));
|
||||
return { node: made, plan: valuePlan(made, values) };
|
||||
});
|
||||
if (title) node.title = String(title).slice(0, 120);
|
||||
const plan = valuePlan(node, values);
|
||||
const near = nearId != null ? nodeById(nearId) : null;
|
||||
g.beforeChange?.();
|
||||
node.pos = freeSpot(node, (g._nodes || []), near);
|
||||
@@ -176,9 +192,9 @@ export function canvasEditActions({ recorded, nodeById }) {
|
||||
}
|
||||
|
||||
function setValues({ node_id: nodeId, values }, getOwner) {
|
||||
return recorded(getOwner, 'mcpDidValues', (g) => {
|
||||
return recorded(getOwner, 'mcpDidValues', async (g) => {
|
||||
const node = nodeById(nodeId);
|
||||
const plan = valuePlan(node, values);
|
||||
const plan = await withFreshChoices(() => valuePlan(node, values));
|
||||
if (!plan.length) throw new Error('values is empty: {setting name: value}.');
|
||||
g.beforeChange?.(node);
|
||||
apply(node, plan);
|
||||
@@ -203,7 +219,7 @@ export function canvasEditActions({ recorded, nodeById }) {
|
||||
}
|
||||
|
||||
async function newWorkflow({ name, nodes }, getOwner) {
|
||||
const built = nodes ? prepare(nodes) : null; // nothing opens unless all of it fits
|
||||
const built = nodes ? await withFreshChoices(() => prepare(nodes)) : null; // nothing opens unless all of it fits
|
||||
const tab = tabName(name);
|
||||
getOwner()?.flushCanvasActivity?.(); // what was pending belongs to the tab being left
|
||||
await app.loadGraphData(structuredClone(BLANK), true, true, tab);
|
||||
|
||||
+106
-23
@@ -1,17 +1,23 @@
|
||||
/**
|
||||
* Marking a picture for the AI (api/ai_marks.py): a window over the page with the picture and four
|
||||
* simple tools - box, circle, arrow, pen - in three colours, each mark numbered with a note of what
|
||||
* to change there, plus a line for the whole picture. "Send to the AI" draws the marks onto the
|
||||
* picture (at most MAX_EDGE px), and sends that PNG with the marks (fractions of the picture) to
|
||||
* the task; the AI gets the notes and the marked picture with its next step. Also the timeline's
|
||||
* row for marks sent (marksStep). Every listener lives as long as the window and goes with it.
|
||||
* to change there, plus a line for the whole picture, and a brush that paints the area to repaint.
|
||||
* "Send to the AI" draws the marks onto the picture (at most MAX_EDGE px), and sends that PNG with
|
||||
* the marks (fractions of the picture) to the task; with "make a repaint mask" on, also a mask
|
||||
* (white where boxes, circles, lines and brush strokes are) that the server turns into the picture
|
||||
* ready to repaint. The AI gets the notes and the marked picture with its next step. Also the
|
||||
* timeline's row for marks sent (marksStep). Every listener lives as long as the window and goes with it.
|
||||
*/
|
||||
|
||||
import { translate as t } from './locales.js';
|
||||
import { anomalousAlert } from './ui_dialog.js';
|
||||
import { outputUrl } from './ai_task_view.js';
|
||||
|
||||
const TOOLS = [['box', '□'], ['circle', '○'], ['arrow', '↗'], ['pen', '✎']];
|
||||
const TOOLS = [['box', '□'], ['circle', '○'], ['arrow', '↗'], ['pen', '✎'], ['mask', '▧']];
|
||||
const MASK_COLOR = 'rgba(255, 64, 160, 0.45)';
|
||||
const MAX_STROKES = 300;
|
||||
const MASK_EDGE = 4096;
|
||||
const AREAS = new Set(['box', 'circle', 'pen', 'mask']);
|
||||
const COLORS = ['#ff4d4f', '#fadb14', '#36cfc9'];
|
||||
const MAX_EDGE = 2048;
|
||||
const MAX_MARKS = 20;
|
||||
@@ -43,8 +49,50 @@ function bounds(mark) {
|
||||
return { x, y, w: Math.max(...xs) - x, h: Math.max(...ys) - y };
|
||||
}
|
||||
|
||||
/** One mark on a 2D context of `w` × `h` px, with its number. */
|
||||
/** A brush stroke of the repaint area on a 2D context of `w` × `h` px, in `color`. */
|
||||
function stroke(ctx, mark, w, h, color = MASK_COLOR) {
|
||||
ctx.strokeStyle = color;
|
||||
ctx.lineWidth = Math.max(2, mark.size * Math.min(w, h));
|
||||
ctx.lineCap = 'round';
|
||||
ctx.lineJoin = 'round';
|
||||
ctx.beginPath();
|
||||
const points = mark.points.length > 1 ? mark.points : [mark.points[0], mark.points[0]];
|
||||
points.forEach(([x, y], index) => (index ? ctx.lineTo(x * w, y * h) : ctx.moveTo(x * w, y * h)));
|
||||
ctx.stroke();
|
||||
}
|
||||
|
||||
/** The repaint mask of `marks` on `w` × `h` px: white where boxes, circles, lines and strokes are. */
|
||||
function maskOf(marks, w, h) {
|
||||
const out = document.createElement('canvas');
|
||||
out.width = w;
|
||||
out.height = h;
|
||||
const ctx = out.getContext('2d');
|
||||
ctx.fillStyle = '#000';
|
||||
ctx.fillRect(0, 0, w, h);
|
||||
ctx.fillStyle = '#fff';
|
||||
for (const mark of marks) {
|
||||
const [start, end] = [mark.points[0], mark.points.at(-1)];
|
||||
if (mark.shape === 'mask') {
|
||||
stroke(ctx, mark, w, h, '#fff');
|
||||
} else if (mark.shape === 'pen') {
|
||||
stroke(ctx, { ...mark, size: 0.03 }, w, h, '#fff');
|
||||
} else if (mark.shape === 'box') {
|
||||
ctx.fillRect(Math.min(start[0], end[0]) * w, Math.min(start[1], end[1]) * h, Math.abs(end[0] - start[0]) * w, Math.abs(end[1] - start[1]) * h);
|
||||
} else if (mark.shape === 'circle') {
|
||||
ctx.beginPath();
|
||||
ctx.ellipse((start[0] + end[0]) / 2 * w, (start[1] + end[1]) / 2 * h, Math.abs(end[0] - start[0]) / 2 * w, Math.abs(end[1] - start[1]) / 2 * h, 0, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
}
|
||||
}
|
||||
return out.toDataURL('image/png');
|
||||
}
|
||||
|
||||
/** One mark on a 2D context of `w` × `h` px, with its number (a brush stroke has none). */
|
||||
function paint(ctx, mark, number, w, h) {
|
||||
if (mark.shape === 'mask') {
|
||||
stroke(ctx, mark, w, h);
|
||||
return;
|
||||
}
|
||||
const line = Math.max(2, Math.round(Math.min(w, h) * 0.006));
|
||||
const [start, end] = [mark.points[0], mark.points.at(-1)];
|
||||
ctx.strokeStyle = mark.color;
|
||||
@@ -94,7 +142,9 @@ export function openMarker(image, taskId, sent) {
|
||||
const marks = [];
|
||||
let tool = 'box';
|
||||
let color = COLORS[0];
|
||||
let brush = 0.05;
|
||||
let drawing = null;
|
||||
const noted = () => marks.filter(mark => mark.shape !== 'mask');
|
||||
|
||||
const overlay = el('div', 'anomalous-ai-marker-overlay');
|
||||
const dialog = el('div', 'anomalous-ai-marker');
|
||||
@@ -105,9 +155,10 @@ export function openMarker(image, taskId, sent) {
|
||||
|
||||
const bar = el('div', 'anomalous-ai-marker-tools');
|
||||
const toolButtons = TOOLS.map(([id, glyph]) => {
|
||||
const pick = button(`anomalous-ai-marker-tool${id === tool ? ' is-on' : ''}`, `${glyph} ${t(`aiMarkTool_${id}`)}`, () => {
|
||||
const pick = button(`anomalous-ai-marker-tool${id === tool ? ' is-on' : ''}${id === 'mask' ? ' is-mask' : ''}`, `${glyph} ${t(`aiMarkTool_${id}`)}`, () => {
|
||||
tool = id;
|
||||
toolButtons.forEach(item => item.classList.toggle('is-on', item === pick));
|
||||
size.hidden = id !== 'mask';
|
||||
});
|
||||
return pick;
|
||||
});
|
||||
@@ -119,7 +170,16 @@ export function openMarker(image, taskId, sent) {
|
||||
swatch.style.background = value;
|
||||
return swatch;
|
||||
});
|
||||
bar.append(...toolButtons, el('span', 'anomalous-ai-marker-gap'), ...colorButtons, el('span', 'anomalous-ai-marker-gap'),
|
||||
const size = el('label', 'anomalous-ai-marker-size');
|
||||
const sizeInput = el('input');
|
||||
sizeInput.type = 'range';
|
||||
sizeInput.min = '1';
|
||||
sizeInput.max = '15';
|
||||
sizeInput.value = String(brush * 100);
|
||||
sizeInput.addEventListener('input', () => { brush = Number(sizeInput.value) / 100; });
|
||||
size.append(el('span', '', t('aiMarkBrush')), sizeInput);
|
||||
size.hidden = true;
|
||||
bar.append(...toolButtons, size, el('span', 'anomalous-ai-marker-gap'), ...colorButtons, el('span', 'anomalous-ai-marker-gap'),
|
||||
button('anomalous-ai-marker-tool', `↶ ${t('aiMarkUndo')}`, () => { marks.pop(); redraw(); }),
|
||||
button('anomalous-ai-marker-tool', t('aiMarkClear'), () => { marks.length = 0; redraw(); }));
|
||||
|
||||
@@ -137,7 +197,11 @@ export function openMarker(image, taskId, sent) {
|
||||
whole.placeholder = t('aiMarkWhole');
|
||||
whole.rows = 3;
|
||||
const send = button('anomalous-ai-marker-send', t('aiMarkSend'), () => void submit());
|
||||
side.append(empty, list, whole, send, el('div', 'anomalous-ai-marker-note', t('aiMarkNote')));
|
||||
const repaint = el('label', 'anomalous-ai-marker-repaint');
|
||||
const repaintBox = el('input');
|
||||
repaintBox.type = 'checkbox';
|
||||
repaint.append(repaintBox, el('span', '', t('aiMarkRepaint')));
|
||||
side.append(empty, list, whole, repaint, send, el('div', 'anomalous-ai-marker-note', t('aiMarkNote')));
|
||||
|
||||
const body = el('div', 'anomalous-ai-marker-body');
|
||||
const left = el('div', 'anomalous-ai-marker-left');
|
||||
@@ -154,19 +218,23 @@ export function openMarker(image, taskId, sent) {
|
||||
function redraw() {
|
||||
const ctx = canvas.getContext('2d');
|
||||
ctx.clearRect(0, 0, canvas.width, canvas.height);
|
||||
[...marks, ...(drawing ? [drawing] : [])].forEach((mark, index) => paint(ctx, mark, index + 1, canvas.width, canvas.height));
|
||||
let number = 0;
|
||||
[...marks, ...(drawing ? [drawing] : [])].forEach(mark => paint(ctx, mark, mark.shape === 'mask' ? 0 : ++number, canvas.width, canvas.height));
|
||||
empty.hidden = marks.length > 0;
|
||||
// The notes, one per mark, kept as typed.
|
||||
while (list.children.length > marks.length) list.lastElementChild.remove();
|
||||
marks.forEach((mark, index) => {
|
||||
repaintBox.disabled = !marks.some(mark => AREAS.has(mark.shape));
|
||||
if (repaintBox.disabled) repaintBox.checked = false;
|
||||
// The notes, one per mark (not per brush stroke), kept as typed.
|
||||
const notes = noted();
|
||||
while (list.children.length > notes.length) list.lastElementChild.remove();
|
||||
notes.forEach((mark, index) => {
|
||||
let row = list.children[index];
|
||||
if (!row) {
|
||||
row = el('li', 'anomalous-ai-marker-row');
|
||||
const note = el('input', 'anomalous-ai-marker-input');
|
||||
note.placeholder = t('aiMarkNotePlaceholder');
|
||||
note.addEventListener('input', () => { marks[[...list.children].indexOf(row)].note = note.value; });
|
||||
note.addEventListener('input', () => { noted()[[...list.children].indexOf(row)].note = note.value; });
|
||||
row.append(el('span', 'anomalous-ai-marker-badge'), el('span', 'anomalous-ai-marker-shape'), note,
|
||||
button('anomalous-ai-marker-remove', '×', () => { marks.splice([...list.children].indexOf(row), 1); redraw(); }, t('aiMarkRemove')));
|
||||
button('anomalous-ai-marker-remove', '×', () => { marks.splice(marks.indexOf(noted()[[...list.children].indexOf(row)]), 1); redraw(); }, t('aiMarkRemove')));
|
||||
list.append(row);
|
||||
requestAnimationFrame(() => note.focus());
|
||||
}
|
||||
@@ -188,21 +256,27 @@ export function openMarker(image, taskId, sent) {
|
||||
observer?.observe(picture);
|
||||
|
||||
canvas.addEventListener('pointerdown', (event) => {
|
||||
if (marks.length >= MAX_MARKS) return;
|
||||
if (tool === 'mask' ? marks.length - noted().length >= MAX_STROKES : noted().length >= MAX_MARKS) return;
|
||||
canvas.setPointerCapture(event.pointerId);
|
||||
drawing = { shape: tool, color, points: [point(event)], note: '' };
|
||||
drawing = tool === 'mask' ? { shape: 'mask', size: brush, points: [point(event)] } : { shape: tool, color, points: [point(event)], note: '' };
|
||||
if (tool === 'mask') redraw();
|
||||
});
|
||||
canvas.addEventListener('pointermove', (event) => {
|
||||
if (!drawing) return;
|
||||
if (drawing.shape === 'pen') drawing.points.push(point(event));
|
||||
if (drawing.shape === 'pen' || drawing.shape === 'mask') drawing.points.push(point(event));
|
||||
else drawing.points[1] = point(event);
|
||||
redraw();
|
||||
});
|
||||
const finish = () => {
|
||||
if (!drawing) return;
|
||||
const box = bounds(drawing);
|
||||
// A click without a drag makes no mark.
|
||||
if (drawing.points.length > 1 && (box.w > 0.01 || box.h > 0.01)) marks.push(drawing);
|
||||
// A click without a drag makes no mark; a brush click is a dot of the repaint area.
|
||||
if (drawing.shape === 'mask') {
|
||||
marks.push(drawing);
|
||||
repaintBox.checked = true;
|
||||
} else if (drawing.points.length > 1 && (box.w > 0.01 || box.h > 0.01)) {
|
||||
marks.push(drawing);
|
||||
}
|
||||
drawing = null;
|
||||
redraw();
|
||||
};
|
||||
@@ -236,12 +310,19 @@ export function openMarker(image, taskId, sent) {
|
||||
out.height = Math.round(picture.naturalHeight * scale);
|
||||
const ctx = out.getContext('2d');
|
||||
ctx.drawImage(picture, 0, 0, out.width, out.height);
|
||||
marks.forEach((mark, index) => paint(ctx, mark, index + 1, out.width, out.height));
|
||||
let number = 0;
|
||||
marks.forEach(mark => paint(ctx, mark, mark.shape === 'mask' ? 0 : ++number, out.width, out.height));
|
||||
return out.toDataURL('image/png');
|
||||
}
|
||||
|
||||
/** The repaint mask at the picture's own size (at most MASK_EDGE px; the server fits it to the picture). */
|
||||
function repaintMask() {
|
||||
const scale = Math.min(1, MASK_EDGE / Math.max(picture.naturalWidth, picture.naturalHeight));
|
||||
return maskOf(marks, Math.round(picture.naturalWidth * scale), Math.round(picture.naturalHeight * scale));
|
||||
}
|
||||
|
||||
async function submit() {
|
||||
if (!marks.length) {
|
||||
if (!marks.length || (!noted().length && !repaintBox.checked)) {
|
||||
await anomalousAlert(t('aiMarkNeedOne'));
|
||||
return;
|
||||
}
|
||||
@@ -251,7 +332,8 @@ export function openMarker(image, taskId, sent) {
|
||||
method: 'POST', headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
image: [image.subfolder, image.filename].filter(Boolean).join('/'), png: composite(), text: whole.value.trim(),
|
||||
marks: marks.map(mark => ({ shape: mark.shape, note: (mark.note || '').trim(), ...bounds(mark) })),
|
||||
marks: noted().map(mark => ({ shape: mark.shape, note: (mark.note || '').trim(), ...bounds(mark) })),
|
||||
...(repaintBox.checked ? { mask: repaintMask() } : {}),
|
||||
}),
|
||||
});
|
||||
const data = await response.json().catch(() => ({}));
|
||||
@@ -284,6 +366,7 @@ export function marksStep(step, taskId, owner) {
|
||||
box.append(open, el('div', 'anomalous-ai-marks-sent-words'));
|
||||
const words = box.lastChild;
|
||||
if (step.note) words.append(el('div', 'anomalous-ai-user-text', step.note));
|
||||
if (step.inpaint) words.append(el('div', 'anomalous-ai-marks-inpaint', t('aiMarkInpaintReady', { file: step.inpaint })));
|
||||
words.append(notes, el('div', 'anomalous-ai-user-state', t(step.delivered ? 'aiMarksSeen' : 'aiMarksWaiting')));
|
||||
return box;
|
||||
}
|
||||
|
||||
+1
-1
@@ -40,5 +40,5 @@
|
||||
@import url("./styles/39-ai-stage.css?v=20261021-stage");
|
||||
@import url("./styles/40-ai-side.css?v=20261023-look");
|
||||
@import url("./styles/41-ai-results.css?v=20261023-tags");
|
||||
@import url("./styles/42-ai-flow.css?v=20261023-percent");
|
||||
@import url("./styles/42-ai-flow.css?v=20261023-inpaint");
|
||||
@import url("./styles/43-ai-toolbox.css?v=20261023-art");
|
||||
|
||||
@@ -201,6 +201,13 @@
|
||||
}
|
||||
.anomalous-ai-marker-tool:hover { border-color: var(--amb-link); color: var(--amb-text-main); }
|
||||
.anomalous-ai-marker-tool.is-on { border-color: var(--amb-link); background: color-mix(in srgb, var(--amb-link) 20%, transparent); color: var(--amb-text-main); }
|
||||
.anomalous-ai-marker-tool.is-mask.is-on { border-color: #ff40a0; background: rgba(255, 64, 160, 0.18); }
|
||||
.anomalous-ai-marker-size { display: inline-flex; align-items: center; gap: 6px; color: var(--amb-text-soft); font-size: 12px; }
|
||||
.anomalous-ai-marker-size input { width: 90px; }
|
||||
.anomalous-ai-marker-repaint { display: flex; align-items: flex-start; gap: 8px; color: var(--amb-text-soft); font-size: 12px; line-height: 1.5; cursor: pointer; }
|
||||
.anomalous-ai-marker-repaint input { margin-top: 2px; accent-color: #ff40a0; }
|
||||
.anomalous-ai-marker-repaint:has(input:disabled) { opacity: 0.5; cursor: default; }
|
||||
.anomalous-ai-marks-inpaint { align-self: flex-start; padding: 2px 9px; border-radius: 999px; background: rgba(255, 64, 160, 0.16); color: var(--amb-text-main); font-size: 11.5px; }
|
||||
.anomalous-ai-marker-color { width: 22px; height: 22px; padding: 0; border: 2px solid transparent; border-radius: 50%; cursor: pointer; }
|
||||
.anomalous-ai-marker-color.is-on { border-color: var(--amb-text-main); }
|
||||
.anomalous-ai-marker-stage { position: relative; align-self: center; max-width: 100%; line-height: 0; }
|
||||
|
||||
Reference in New Issue
Block a user