Compare commits
18
Commits
fix/second-copy
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
89bda6ba11 | ||
|
|
47d1c4f739 | ||
|
|
799d731ab5 | ||
|
|
5fc8cffc9f | ||
|
|
4dffaf3991 | ||
|
|
bc474f1532 | ||
|
|
c206d9e9da | ||
|
|
eba880cdee | ||
|
|
b6a67a4c34 | ||
|
|
27525ffdd8 | ||
|
|
eb668c448c | ||
|
|
9b3cd7c769 | ||
|
|
13c303178b | ||
|
|
f945feabb6 | ||
|
|
24ce4ffb06 | ||
|
|
980ed5db8b | ||
|
|
bf59887643 | ||
|
|
60be0ee865 |
@@ -2,6 +2,66 @@
|
||||
|
||||
All notable changes to ComfyUI-AusBoss are documented here.
|
||||
|
||||
## Unreleased
|
||||
|
||||
- **New node: Edit Lock.** Keeps every part of your picture that an edit
|
||||
did not change exactly where it was, with the colors it had. Edit models
|
||||
redraw the whole picture, so untouched parts often come back a little
|
||||
zoomed, shifted or tinted. Put Edit Lock between the model and the
|
||||
sampler and give it the picture you are editing. It needs no mask. It
|
||||
steps aside when the edit changes the whole picture, and lets rain,
|
||||
snow and haze through while holding the picture in place. Tested on
|
||||
MiniMax-H3-Image.
|
||||
|
||||
- **New node: Edit Lock Restore.** Puts your picture's own pixels back in
|
||||
every part the edit did not change. Add it after the VAE Decode with
|
||||
Edit Lock's `edit_lock` output. It also gives a mask of what the edit
|
||||
changed and a one-line report.
|
||||
|
||||
- **New node: Text Lines.** It runs what comes after once for each line of
|
||||
your text. Type five prompts, one on each line, and one run makes five
|
||||
pictures. It also takes a list a text model wrote: empty lines are
|
||||
skipped, a leading number or bullet comes off, and **Limit** stops a list
|
||||
that ran long.
|
||||
|
||||
- **Mask by Touch: type a name to take it out of the mask.** The name field
|
||||
under the picture now has a sign in front. Click it to switch between **+**
|
||||
(add by name) and **−** (take away by name), then type "face" or "hair" and
|
||||
press Enter. Typing "-face" takes it away whatever the sign says. Before,
|
||||
a typed name was only taken away while Erase was chosen, which nothing on
|
||||
the node said.
|
||||
|
||||
- **Mask by Touch: Ctrl-click only takes away what you point at.** With a
|
||||
sweater picked, a Ctrl-click on the face offered the whole person, and the
|
||||
sweater went with it. Now a spot where nothing is picked says so and
|
||||
changes nothing.
|
||||
|
||||
- **Mask by Touch works on big pictures.** A look at a 3840 pixel picture took
|
||||
twice as long as at 2048, and at 8192 pixels it ran a 32 GB graphics card
|
||||
out of memory. The node now asks SAM 3 about a copy of at most 2048 pixels
|
||||
and brings the answer back to full size. Pictures up to 2048 pixels come
|
||||
out exactly as before. On bigger ones a mask's edge can sit a pixel or two
|
||||
differently.
|
||||
|
||||
- **Mask by Touch no longer fills the queue.** With Image Crop + Rotate + Pad
|
||||
or Load Image + Pad wired in, every change to the loader (each step of its
|
||||
megapixels, for one) queued one more look at a picture that was already
|
||||
replaced. Now only the newest look waits. The button that pauses the
|
||||
looking also says **Pause** and **Look on**.
|
||||
|
||||
- **Two errors from outside a workflow now say what to do.** When a MiniMax H3
|
||||
run stops with `KeyError: 'latent'` because the Muse MiniMax Director pack is
|
||||
installed, a message names that pack and says to turn it off. When an older
|
||||
ComfyUI cannot read a `w6a8` model file, a message says to update ComfyUI.
|
||||
ComfyUI's own error report stays as it is.
|
||||
|
||||
- **ComfyUI starts when the pack is installed twice.** A Manager install
|
||||
(`custom_nodes/ausboss-nodes`) beside a git clone
|
||||
(`custom_nodes/ComfyUI-AusBoss`) stopped ComfyUI while it started. Now one
|
||||
copy loads and the other steps aside, and the console says which folder to
|
||||
delete. A copy from 2.23.0 or older is the one that loads when it sits
|
||||
beside a newer one, so delete the older folder.
|
||||
|
||||
## 2.23.0 - 2026-11-09
|
||||
|
||||
- **Mask by Touch keeps your picks when the picture changes.** Your clicks
|
||||
|
||||
@@ -20,6 +20,7 @@ Most of them are for image and video work. ComfyUI's core nodes already cover th
|
||||
- **[Mask by Touch](#mask-by-touch):** click a thing on the picture and get its mask. Hover shows what a click would take, and **Fine edges** follows hair and fur.
|
||||
- **[Mask by Name](#mask-by-name):** type "the red jacket" and get its mask and a cut-out.
|
||||
- **[Image Crop + Rotate + Pad](#image-crop--rotate--pad):** turn, crop and pad a picture by dragging handles on the node, for outpainting.
|
||||
- **[Edit Lock](#edit-lock):** an edit changes what you asked for, and the rest of the picture stays exactly where it was. No mask needed.
|
||||
- **[Stitch Inpaint](#stitch-inpaint):** after an inpaint or outpaint, everything you didn't ask to change is your original pixels again.
|
||||
- **[Video Crop + Rotate + Pad → Clip](#video-crop--rotate--pad--clip):** trim a clip on a timeline, and pad or crop it to a new shape.
|
||||
- **[Tiled Upscale](#tiled-upscale):** upscale a picture in tiles to the megapixels you set.
|
||||
@@ -42,6 +43,8 @@ git clone https://github.com/ausboss/ComfyUI-AusBoss.git
|
||||
|
||||
Restart ComfyUI and search the node library for **AusBoss**.
|
||||
|
||||
Install it one way, not both. Manager puts the pack in `custom_nodes/ausboss-nodes` and a clone in `custom_nodes/ComfyUI-AusBoss`. They are the same pack, so if both folders are there, delete one. The console says which copy it skipped.
|
||||
|
||||
After an update, through Manager or `git pull`, restart ComfyUI and hard-refresh the browser with **Ctrl+Shift+R**. Otherwise the browser keeps the old JavaScript, and the pack warns you about it. Manager can install a version or two behind while the Comfy Registry reviews a new release. To get the newest one, pick it in Manager's version list, or update by hand: delete the pack's folder in `ComfyUI/custom_nodes` and clone the repository as shown above.
|
||||
|
||||
The pack uses the Pillow, NumPy, Torch and PyAV that come with ComfyUI. Each example lists the models it needs on its Workflow Note. LaMa Inpaint also needs [`big-lama.pt`](https://github.com/Sanster/models/releases/download/add_big_lama/big-lama.pt) in `ComfyUI/models/lama/`. The pack never downloads weights by itself.
|
||||
@@ -137,6 +140,18 @@ The same watercolor edit without and with Realign to Source: 61 px off before, 0
|
||||
|
||||
Qwen drew this watercolor 5.6% taller, and Realign to Source lined it back up with the original.
|
||||
|
||||
### Edit Lock
|
||||
|
||||
Keeps every part of your picture that an edit did not change exactly where it was, with the colors it had. An edit model redraws the whole picture, so the parts you did not ask about often come back a little zoomed in, shifted or tinted, and the picture jumps when you flip between before and after. Put Edit Lock between the model and the sampler and give it the picture you are editing. It needs no mask: it sees what the edit changes while the picture is being made, and holds on to everything else.
|
||||
|
||||
When the edit changes the whole picture (a new style, new light), there is nothing to keep, so the node steps aside and the edit runs as usual. A soft effect over the whole picture, such as rain or falling snow, is let through as well: the node then only holds the picture in place. **Lock** (`lock`) set to `off` does nothing at all, `always` locks even a whole-picture edit, and an optional `mask` says where the edit may happen. **Tolerance** (`tolerance`) sets how big a change must be to count as part of the edit.
|
||||
|
||||
On MiniMax-H3-Image, 4 megapixel edits moved the picture by up to 14 px without the lock (measured at 1 megapixel) and by 0.2 px or less with it (0.08 px or less on 35 of the 36 locked runs), on 19 local edits at two seeds each. After five edits in a row on one picture the plain chain had drifted 1.65 px and turned visibly pink; the locked chain sat at 0.04 px with no tint.
|
||||
|
||||
### Edit Lock Restore
|
||||
|
||||
Puts your picture's own pixels back in every part the edit did not change. Edit Lock keeps those parts in place, but they still pass through the VAE, which softens fine detail a little. Add this right after the VAE Decode and wire in Edit Lock's `edit_lock` output. **Feather** (`feather`) sets how softly the original fades into the edit. It also returns `edit_mask`, white where the edit changed the picture, and a one-line `report` of how much was kept.
|
||||
|
||||
### Image Compare A/B
|
||||
|
||||
Compares two pictures with a slider, or flips between them whole. Nothing covers the picture, and its size shows underneath. The panel grows with the node, and images A and B pass through, so you can put it anywhere on an image wire. **Erase** mixes the two by hand: rub parts of one picture away with a brush so the other shows through, then save the mix as a PNG under a name you type, with a layered PSD for Photoshop or GIMP if you want it.
|
||||
@@ -342,6 +357,10 @@ Puts batch `b` after `a`. When the sizes match, nothing is resized. When they di
|
||||
|
||||
A text box that grows with the node and sends its text, unchanged, down a `STRING` wire. Use it for a prompt, caption, path or other text that several nodes share. Spaces and line breaks are kept. Ctrl + Up and Ctrl + Down change the weight of the selected words, as in ComfyUI's own prompt boxes.
|
||||
|
||||
### Text Lines
|
||||
|
||||
Runs what comes after once for each line of your text. Type five prompts, one on each line, and one run makes five pictures. It also takes a list another node wrote, skips empty lines, and can take a leading `1.` or `-` off each line. `number` counts the lines from 1, to give each its own seed.
|
||||
|
||||
### H3 Prompt Writer
|
||||
|
||||
Type what happens, one line for each moment, and leave a blank line to cut to a new place. Connect up to four pictures and give each one a job: a person, a place, an outfit, a thing or a style. Copy the moves from a video, keep a clip going, or add a voice or a song. A timeline on the node shows when each moment plays before you run it. It needs no extra model and you never see H3's prompt format. Connect a small text model and it fills in a one-line idea.
|
||||
|
||||
+36
-7
@@ -22,6 +22,7 @@ NODE_MODULES = [
|
||||
"node_callout",
|
||||
"node_color_match",
|
||||
"node_compare",
|
||||
"node_edit_lock",
|
||||
"node_frame_interpolate",
|
||||
"node_free_memory",
|
||||
"node_h3_prompt_writer",
|
||||
@@ -52,6 +53,7 @@ NODE_MODULES = [
|
||||
"node_select_frame",
|
||||
"node_show_text",
|
||||
"node_step_preview",
|
||||
"node_text_lines",
|
||||
"node_tiled_upscale",
|
||||
"node_video_crop_rotate_pad",
|
||||
"node_video_crop_rotate_pad_clip",
|
||||
@@ -63,7 +65,19 @@ NODE_CLASS_MAPPINGS = {}
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {}
|
||||
_failed_modules = []
|
||||
|
||||
# Two copies of the pack (a Manager install beside a git clone) add every
|
||||
# route twice and ComfyUI stops while starting. Only one copy loads.
|
||||
_other_copy = None
|
||||
try:
|
||||
from .nodes._single_copy import other_copy_that_loads
|
||||
|
||||
_other_copy = other_copy_that_loads(os.path.dirname(__file__))
|
||||
except Exception:
|
||||
traceback.print_exc()
|
||||
|
||||
for _module_name in NODE_MODULES:
|
||||
if _other_copy:
|
||||
break
|
||||
try:
|
||||
_module = importlib.import_module(f".nodes.{_module_name}", __name__)
|
||||
NODE_CLASS_MAPPINGS.update(getattr(_module, "NODE_CLASS_MAPPINGS", {}))
|
||||
@@ -76,7 +90,7 @@ for _module_name in NODE_MODULES:
|
||||
|
||||
# Folder ComfyUI serves to the browser; every .js file in it is loaded as a
|
||||
# frontend extension. Shared modules use .mjs so they are import-only.
|
||||
WEB_DIRECTORY = "./js"
|
||||
WEB_DIRECTORY = None if _other_copy else "./js"
|
||||
|
||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"]
|
||||
|
||||
@@ -112,11 +126,26 @@ def _print_banner():
|
||||
print(bar)
|
||||
|
||||
|
||||
try:
|
||||
from .nodes._frontend_cache import register_frontend_cache_guard
|
||||
def _print_second_copy():
|
||||
# ASCII only, as in the banner.
|
||||
red = "\033[0;31m"
|
||||
reset = "\033[0m"
|
||||
own = os.path.basename(os.path.dirname(os.path.realpath(__file__)))
|
||||
other = os.path.basename(_other_copy["folder"])
|
||||
print(f"{red}AusBoss: this pack is installed twice.{reset}")
|
||||
print(f" Using: custom_nodes/{other} (v{_other_copy['version']})")
|
||||
print(f" Skipping: custom_nodes/{own} (v{_read_version()})")
|
||||
print(" Delete one of the two folders, then restart ComfyUI.")
|
||||
|
||||
register_frontend_cache_guard()
|
||||
except Exception:
|
||||
traceback.print_exc()
|
||||
|
||||
_print_banner()
|
||||
if _other_copy:
|
||||
_print_second_copy()
|
||||
else:
|
||||
try:
|
||||
from .nodes._frontend_cache import register_frontend_cache_guard
|
||||
|
||||
register_frontend_cache_guard()
|
||||
except Exception:
|
||||
traceback.print_exc()
|
||||
|
||||
_print_banner()
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
# Edit Lock
|
||||
|
||||
Keeps every part of your picture that an edit did not change exactly where
|
||||
it was, with the colors it had.
|
||||
|
||||
An edit model redraws the whole picture, also the parts you did not ask it
|
||||
to change. Those parts often come back a little zoomed in, shifted or
|
||||
tinted. Flip between the original and the edit and the whole picture
|
||||
jumps. Edit Lock stops that: only the edit is new.
|
||||
|
||||
It needs no mask. It sees what the edit changes while the picture is being
|
||||
made, and holds on to everything else.
|
||||
|
||||
## Quick start
|
||||
|
||||
1. Put **Edit Lock 🆎** between the model and the sampler: the model (after
|
||||
any LoRA loader) goes in, and its `model` output goes to the sampler or
|
||||
guider.
|
||||
2. Wire the VAE into `vae` and the picture you are editing into `image`.
|
||||
Use the same picture the edit model gets, at the size of the result.
|
||||
3. After the VAE Decode, add **Edit Lock Restore 🆎**. Wire the decoded
|
||||
picture into `image` and Edit Lock's `edit_lock` output into
|
||||
`edit_lock`. This puts your own pixels back in the untouched parts.
|
||||
4. Wire Restore's `report` into **Show Text 🆎** to read how much was kept.
|
||||
|
||||
Run your edit as usual. Prompt, steps and seed stay as they were.
|
||||
|
||||
## What it does with each kind of edit
|
||||
|
||||
- **A local edit** (add glasses, repaint the car, change the sign): the
|
||||
edit happens, and the rest of the picture stays put.
|
||||
- **An edit of the whole picture** (a new style, new light, fog):
|
||||
there is nothing to keep, so Edit Lock steps aside and the edit runs as
|
||||
if the node were not there. The report says so.
|
||||
- **A soft effect over the whole picture** (rain, falling snow, haze):
|
||||
it touches everything a little, so there is no untouched part to put
|
||||
back. Edit Lock lets the effect through and only holds the picture in
|
||||
place, so it does not come back zoomed or shifted. The report says so.
|
||||
- **A soft effect and a clear change in one edit** (rain and an umbrella):
|
||||
the node may lock on the umbrella and hold the rain back everywhere
|
||||
else. If the effect is missing outside the changed part, set **Lock** to
|
||||
`off`.
|
||||
- **Something in between** (a new hair color on a close-up, a new
|
||||
background behind a person): it may lock or step aside. The report says
|
||||
which. These edits often change the light on the face too. `auto` steps
|
||||
aside for most of them, so the new light comes with the new hair.
|
||||
`always` keeps the old light on everything it keeps, which can look
|
||||
wrong on a face. A mask settles it: paint the hair and the face, and
|
||||
the rest is locked.
|
||||
- **The model ignores your picture and draws a new one:** that reads as a
|
||||
whole-picture edit, so `auto` steps aside. `always` holds the model to
|
||||
your picture.
|
||||
|
||||
## Controls
|
||||
|
||||
- **model**: the edit model. Patch last, after any LoRAs.
|
||||
- **vae**: the same VAE the workflow decodes with.
|
||||
- **image**: the picture you are editing, at the size of the result. If
|
||||
the sizes differ by more than a few percent the lock does nothing, and
|
||||
the report says so.
|
||||
- **Lock** (`lock`): `auto` (the default) locks the untouched parts and
|
||||
steps aside when the whole picture changes. `always` locks even then.
|
||||
`off` does nothing, which makes a quick before and after.
|
||||
- **Tolerance** (`tolerance`): how big a change must be to count as part of
|
||||
the edit. 1 is the default. Raise it if small unwanted changes get
|
||||
through. Lower it if the lock holds back part of your edit.
|
||||
- **mask** (optional): white marks where the edit may happen; everything
|
||||
else is locked. With a mask the node does not look for the change
|
||||
itself. A mask from **Mask by Touch 🆎** or **Mask by Name 🆎** works.
|
||||
|
||||
## Outputs
|
||||
|
||||
- **model**: the model with the lock on.
|
||||
- **edit_lock**: what the lock kept, for Edit Lock Restore 🆎.
|
||||
|
||||
## Good to know
|
||||
|
||||
- The lock belongs to one picture. For an edit with two pictures, give it
|
||||
the one the result should line up with.
|
||||
- It works on single pictures, not video.
|
||||
- The node watches the first steps of a run and locks for the rest, so
|
||||
the model finishes the edit on top of your real picture. Shadows and
|
||||
reflections the edit adds next to the changed part still come through.
|
||||
- If your edit is missing and the report says nearly all of the picture
|
||||
was kept, set **Lock** to `off`.
|
||||
- Tested on MiniMax-H3-Image. The lock itself is not tied to one model, but
|
||||
the line between "local edit" and "whole picture" was measured there.
|
||||
|
||||
## Technical details
|
||||
|
||||
At every step an edit model predicts the finished picture. Edit Lock
|
||||
compares that prediction with your picture in latent space.
|
||||
|
||||
**Watching.** The first prediction of a run is a blur, and rain, snow or
|
||||
the light a new hair color throws on a face only show a few steps in. So
|
||||
the node watches until the noise is down to 0.8 of where it began: step 4
|
||||
of a 6-step MiniMax-H3-Image run. A run that never gets that low decides
|
||||
at its last step. While it watches, the model draws its own picture and
|
||||
only one thing is corrected. The model often draws the whole picture about
|
||||
one percent larger, or a little shifted. The node measures that zoom and
|
||||
shift at every step, on the calmer half of the picture, and moves the
|
||||
prediction back so it sits where your picture does.
|
||||
|
||||
**Deciding.** Change is measured against the quiet part of the picture:
|
||||
the tenth where the prediction differs least. An even tint over the whole
|
||||
picture is taken out first, so a color cast is not read as a change. A
|
||||
spot counts as changed when it differs by 3 times the quiet level, or by 2
|
||||
times while it touches a spot that does. So a faint change beside a clear
|
||||
one belongs to the edit, and a faint change alone is redraw noise.
|
||||
**Tolerance** multiplies both numbers.
|
||||
|
||||
**Locking.** From the decision on, wherever the picture did not change,
|
||||
the original latent is put back before the sampler takes its next step. A
|
||||
part once read as changed stays changed for the rest of the run. The last
|
||||
step ends on the original there, so what was kept is the original itself,
|
||||
not a redrawn copy.
|
||||
|
||||
**Stepping aside.** `auto` does not lock in three cases:
|
||||
|
||||
- At the first step the whole picture is already changing (a new style,
|
||||
new light). The node reads the calmest fifth of a smoothed copy, where a
|
||||
slightly larger redraw nearly vanishes and a new style does not. On
|
||||
MiniMax-H3-Image, local edits at 1, 4 and 16 megapixels read 0.22 or
|
||||
less there and whole-picture edits 0.27 or more; the line sits at 0.243.
|
||||
The model then runs untouched.
|
||||
- At the decision no place has changed clearly: the strongest spot of the
|
||||
picture is under 4 times the quiet level. That is light rain or fog.
|
||||
Local edits read 4.7 or more there, glasses and makeup included.
|
||||
- At the decision the lock would hand a tenth of the picture or more to
|
||||
the edit, yet the strongest hundredth of the picture is under 4 times
|
||||
the quiet level. That is heavier rain or snow, and a lock would let it
|
||||
through in patches. Local edits that wide read 4.4 or more; rain and
|
||||
snow read 3.6 or less.
|
||||
|
||||
In the last two cases the node keeps undoing the zoom and shift to the end
|
||||
of the run, and puts nothing back. An edit close to one of these lines (a
|
||||
hair color change on a close-up, a new background, smoke) can fall on
|
||||
either side. The report says which, and **Lock** overrides it.
|
||||
|
||||
This is the target the TailSteer paper trains a model toward (the source
|
||||
where nothing changed, the model's own edit elsewhere), applied while
|
||||
sampling. It needs no LoRA and no training.
|
||||
@@ -0,0 +1,48 @@
|
||||
# Edit Lock Restore
|
||||
|
||||
Puts your picture's own pixels back in every part the edit did not change.
|
||||
|
||||
**Edit Lock 🆎** keeps the untouched parts in place while the picture is
|
||||
made. They still pass through the VAE, which softens fine detail a little.
|
||||
This node swaps your original pixels in for those parts, so they match
|
||||
your picture pixel for pixel. Only the edit is new.
|
||||
|
||||
## Quick start
|
||||
|
||||
1. Add it right after the VAE Decode, before any upscaler.
|
||||
2. Wire the decoded picture into `image`.
|
||||
3. Wire the `edit_lock` output of Edit Lock 🆎 into `edit_lock`.
|
||||
4. Wire `report` into **Show Text 🆎** to read what was kept.
|
||||
|
||||
## Controls
|
||||
|
||||
- **image**: the edited picture, straight from the VAE Decode.
|
||||
- **edit_lock**: from Edit Lock 🆎. It carries your original picture and
|
||||
the map of what the edit changed, so there is nothing else to wire.
|
||||
- **Feather** (`feather`): how softly your original fades into the edit,
|
||||
in pixels. 16 is the default. The fade lies outside the edit, so the
|
||||
edit itself is never mixed with the old picture.
|
||||
|
||||
## Outputs
|
||||
|
||||
- **image**: the edit, with your original pixels in the untouched parts.
|
||||
- **edit_mask**: white where the edit changed the picture. Use it to see
|
||||
what the lock decided, or for a later step that should only touch the
|
||||
edit.
|
||||
- **report**: one line, such as "Edit Lock kept 82% of your picture as it
|
||||
was and left 18% to the edit".
|
||||
|
||||
## When nothing was locked
|
||||
|
||||
The picture passes through unchanged, the mask is all white, and the
|
||||
report says why: the edit changed the whole picture, the lock was off, or
|
||||
the picture and the result were different sizes.
|
||||
|
||||
## Technical details
|
||||
|
||||
The map of what changed is made while sampling, on the latent grid. Here
|
||||
it is enlarged to the picture's size, widened by the feather and blurred
|
||||
by the same amount, and used to blend the decoded picture with your
|
||||
original. If the decoded picture is not the size of your original (after
|
||||
an upscale), the original is resized to fit first, which costs it some
|
||||
sharpness. That is why this node goes before an upscaler.
|
||||
@@ -19,8 +19,8 @@ What is lit in teal is what the mask will be. When it is right, press Run
|
||||
for the whole workflow.
|
||||
|
||||
The first time the pointer rests on a picture, the node looks at it. A
|
||||
small button under the picture shows how far along it is, **❚❚ 40/104**,
|
||||
filling up as it goes. The first things light up after a few seconds, and
|
||||
button under the picture shows how far along it is, **❚❚ Pause · looking
|
||||
40/104**, filling up as it goes. The first things light up after a few seconds, and
|
||||
it has seen the whole picture after about 20.
|
||||
|
||||
You do not have to wait for that. Whatever is lit under the pointer can be
|
||||
@@ -28,7 +28,7 @@ picked: a click takes it at once, also while the node is still looking.
|
||||
|
||||
Click that button to pause the looking when what you need is already lit:
|
||||
what is lit stays, and you go on picking and painting. It turns into
|
||||
**▶**; click it to look on from where it stopped. A click on a spot where
|
||||
**▶ Look on**; click it to look on from where it stopped. A click on a spot where
|
||||
nothing is lit yet looks on as well.
|
||||
|
||||
A click on a spot where nothing is lit yet is not lost: a dot marks it, the
|
||||
@@ -50,6 +50,9 @@ at that spot.
|
||||
A stroke across two things, like both shoes, takes both.
|
||||
- Did the thing not light up at all? Type its name into the field under the
|
||||
picture, like "screen" or "earring", and press Enter.
|
||||
- Is something in the mask that should not be, like the face or the hair of
|
||||
a person you picked? Click the **+** in front of that field so it shows
|
||||
**−**, type "face" and press Enter. Or hold Ctrl and click the face.
|
||||
- When nothing fits, switch to **Paint** and paint the area yourself.
|
||||
|
||||
## The whole subject, or the background
|
||||
@@ -59,6 +62,10 @@ click: the person, the animal, the product. **Background** picks everything
|
||||
else. They are picks like any other: click more things to add them, or
|
||||
Ctrl-click a thing to take it away again.
|
||||
|
||||
Ctrl-click takes away the thing under the pointer, where it is in the mask.
|
||||
On a spot where nothing is picked it changes nothing, and the picture says
|
||||
so.
|
||||
|
||||
Both need ComfyUI's background-removal model, see "The model for Fine
|
||||
edges" below.
|
||||
|
||||
@@ -115,8 +122,11 @@ or an area you painted.
|
||||
it.
|
||||
- **all 3** on a chip takes every thing of that name instead of the one you
|
||||
clicked: all five birds, both shoes.
|
||||
- The field at the end, **+ type a name**, picks every thing of the name
|
||||
you type. With Erase chosen it takes them away.
|
||||
- The field at the end picks every thing of the name you type. The sign in
|
||||
front of it says what happens to the name: **+** adds it, **−** takes it
|
||||
away. Click the sign to change it. A name typed with a minus in front,
|
||||
like "-face", is taken away whatever the sign shows. With Erase chosen the
|
||||
sign starts on **−**.
|
||||
|
||||
## The tools on the node
|
||||
|
||||
@@ -273,6 +283,15 @@ then.
|
||||
|
||||
### How SAM 3 is asked
|
||||
|
||||
- A picture whose long side is over 2048 pixels is asked about on a copy
|
||||
with its long side at 2048. SAM 3 reads every picture at 1008 pixels
|
||||
anyway, and returns each answer at the size it was given, on the graphics
|
||||
card: at 8192 pixels one word with many matches needed over 27 GB. The
|
||||
answer is brought back to the picture's own size, so the mask, the cut-out
|
||||
and your touches stay in pixels of the picture. Paint is never scaled.
|
||||
Measured on a 3840 pixel picture, five words: the new masks share 95 to
|
||||
99.6 % of their area with the old ones, the edge sits a median 0 to 2
|
||||
pixels differently, and a look takes 31 seconds in place of 53.
|
||||
- The search is ComfyUI's own SAM 3: SAM3 Detect, asked about points, or
|
||||
about a word that CLIP Text Encode has read. Both use ComfyUI's own
|
||||
settings (threshold 0.5, two edge passes). The node needs ComfyUI 0.20 or
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
# Text Lines
|
||||
|
||||
Runs what comes after once for each line of your text. Type five prompts,
|
||||
one on each line, press Run once, and you get five pictures.
|
||||
|
||||
Use it for a list of prompts you typed, or for a list another node wrote,
|
||||
such as a text model asked for "five different poses".
|
||||
|
||||
## How to use it
|
||||
|
||||
1. Type your lines in the box, or wire text into **text**.
|
||||
2. Wire **line** to where one prompt would go, such as a text encoder's
|
||||
prompt.
|
||||
3. Run. Every node after it runs once for each line, and a Save Image at the
|
||||
end saves one picture for each.
|
||||
|
||||
Empty lines are skipped. Models and pictures that do not depend on the line
|
||||
are loaded once, not once for each line.
|
||||
|
||||
## Controls
|
||||
|
||||
- The text box (`text`): One line for each run.
|
||||
- **Split at** (`split`): **each line** makes every line its own run.
|
||||
**blank line** keeps lines together and starts the next run at an empty
|
||||
line. Pick it when one prompt is several lines long.
|
||||
- **Numbering** (`remove_numbering`): **remove** takes a number or bullet
|
||||
off the start of each line, such as `1.`, `2)`, `-` or `[prompt 3]`.
|
||||
Lists a text model writes usually have them. **keep** leaves every line
|
||||
as written.
|
||||
- **Limit** (`limit`): The most lines to run. 0 runs them all. Set it when
|
||||
a text model might write more lines than you asked for.
|
||||
|
||||
## Outputs
|
||||
|
||||
- **line**: Each line, one at a time.
|
||||
- **number**: The line's place in the list, starting at 1. Add it to a seed
|
||||
with Math Expression (`a + b`) to give each line its own seed.
|
||||
- **count**: How many lines run.
|
||||
|
||||
## Notes
|
||||
|
||||
- With no lines at all, the run stops at this node and says so.
|
||||
- Every run uses the same seed unless you add **number** to it. Different
|
||||
prompts still give different pictures.
|
||||
- A line such as "2 women on a beach" or "1.5x zoom" keeps its number: only
|
||||
a number followed by a period, bracket or colon counts as numbering.
|
||||
|
||||
## Technical details
|
||||
|
||||
**line** and **number** are ComfyUI lists (`OUTPUT_IS_LIST`), the same way
|
||||
Image Folder hands out pictures. ComfyUI runs each later node once for each
|
||||
item and keeps the results in order. A node that takes a whole list at once,
|
||||
such as core Rebatch Images, gets all of them together.
|
||||
|
||||
Numbering that is removed: a number in square, round or CJK brackets with an
|
||||
optional word before it (`[prompt 1]`, `(2)`), a word and a number followed by
|
||||
a colon, period, bracket or dash (`Shot 3:`), a bare number followed by a
|
||||
period, bracket or colon that has no digit right after it, and the bullets
|
||||
`-`, `*` and `•` followed by a space. Only one marker is removed from each
|
||||
line.
|
||||
@@ -24,6 +24,7 @@ help panel.
|
||||
| Get a mask by typing what to find | Mask by Name (`AUSBOSS_NODES_MaskByName`) |
|
||||
| Get a mask by clicking the thing | Mask by Touch (`AUSBOSS_NODES_MaskByTouch`) |
|
||||
| Grow, shrink, fill or smooth a mask | Mask Refine (`AUSBOSS_NODES_RefineMask`) |
|
||||
| Keep the untouched parts of a picture exactly as they were while an edit model edits it | Edit Lock (`AUSBOSS_NODES_EditLock`) then Edit Lock Restore (`AUSBOSS_NODES_EditLockRestore`) |
|
||||
| Line an edited picture back up with the original | Realign to Source (`AUSBOSS_NODES_RealignToSource`, experimental) |
|
||||
| Match one picture's colors to another | Color Match (`AUSBOSS_NODES_ColorMatch`) |
|
||||
| Upscale big pictures in tiles | Tiled Upscale (`AUSBOSS_NODES_TiledUpscale`) then Tiled Upscale Stitch (`AUSBOSS_NODES_TiledUpscaleStitch`) |
|
||||
@@ -38,6 +39,7 @@ help panel.
|
||||
| Turn, crop and pad a video: one frame, or the whole clip | Video Crop + Rotate + Pad → Frame (`AUSBOSS_NODES_VideoCropRotatePad`), → Clip (`AUSBOSS_NODES_VideoCropRotatePadClip`) |
|
||||
| Change a clip's frame rate | Frame Interpolate (`AUSBOSS_NODES_FrameInterpolate`) |
|
||||
| Work with frame batches | Select Frame, Select Every Nth, Split Batch, Merge Batches (`AUSBOSS_NODES_SelectFrame`, `_SelectEveryNth`, `_SplitBatch`, `_MergeBatches`) |
|
||||
| Run a workflow once for each line of text, such as a list of prompts | Text Lines (`AUSBOSS_NODES_TextLines`) |
|
||||
| One shared seed, number or text | Seed, Integer, Float, Text (`AUSBOSS_NODES_Seed`, `_Integer`, `_Float`, `_Text`) |
|
||||
| Do maths on numbers | Math Expression (`AUSBOSS_NODES_MathExpression`) |
|
||||
| Show text on the node, or the picture while it is made | Show Text (`AUSBOSS_NODES_ShowText`), Step Preview (`AUSBOSS_NODES_StepPreview`) |
|
||||
|
||||
@@ -47,6 +47,24 @@ async function stopLook(node) {
|
||||
return Boolean(running || waiting.length);
|
||||
}
|
||||
|
||||
// Take this node's own runs that still wait out of the queue: the looks and
|
||||
// touches its picture asked for (they carry an `ask`), never a run of the
|
||||
// workflow. Says how many went. The one that is running is left alone.
|
||||
async function dropWaiting(node) {
|
||||
const id = executionIdOf(app.rootGraph ?? app.graph, node);
|
||||
if (!id) return 0;
|
||||
const queue = await (await api.fetchApi("/queue")).json();
|
||||
const ours = (entry) => {
|
||||
const step = entry?.[2]?.[id];
|
||||
return step?.class_type === NODE_CLASS && String(step.inputs?.ask ?? "") !== "";
|
||||
};
|
||||
const waiting = (queue?.queue_pending ?? []).filter(ours).map((entry) => entry[1]);
|
||||
if (waiting.length) {
|
||||
await api.fetchApi("/queue", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ delete: waiting }) });
|
||||
}
|
||||
return waiting.length;
|
||||
}
|
||||
|
||||
// The picture goes above the settings: the stage widget sits ahead of the
|
||||
// node's card, whichever of the two was built first.
|
||||
function stageFirst(node) {
|
||||
@@ -92,6 +110,7 @@ function build(node) {
|
||||
const face = mountTouchStage(node, {
|
||||
lookOnButton: () => lookOnButton,
|
||||
stopLook: () => stopLook(node).catch(() => false),
|
||||
dropWaiting: () => dropWaiting(node).catch(() => 0),
|
||||
// A workflow is being opened: nothing is run on the person's behalf.
|
||||
loading: () => Boolean(app.configuringGraph),
|
||||
// Bypassed or muted, or inside a box that is: nothing is run for it.
|
||||
|
||||
@@ -26,6 +26,7 @@ import { nodeByExecutionId } from "../shared/graph_ids.mjs";
|
||||
import {
|
||||
collapseRepeatedErrors,
|
||||
emptySourceMessage,
|
||||
explainRunError,
|
||||
findEmptySources,
|
||||
nodeLacksSource,
|
||||
} from "../shared/run_check.mjs";
|
||||
@@ -174,10 +175,20 @@ function collapseAnswers() {
|
||||
api.queuePrompt = collapsing;
|
||||
}
|
||||
|
||||
// A run that stops on an error the pack knows the cause of gets one toast
|
||||
// that says what to do. ComfyUI's own error report is left as it is.
|
||||
function explainKnownErrors() {
|
||||
api.addEventListener("execution_error", ({ detail }) => {
|
||||
const known = explainRunError(detail);
|
||||
if (known) showToast({ severity: "warn", summary: known.summary, detail: known.detail, life: 30000 });
|
||||
});
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "ausboss.run_check",
|
||||
setup() {
|
||||
guardRuns();
|
||||
collapseAnswers();
|
||||
explainKnownErrors();
|
||||
},
|
||||
});
|
||||
|
||||
@@ -160,3 +160,51 @@ export function collapseRepeatedErrors(nodeErrors) {
|
||||
}
|
||||
return nodeErrors;
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------ known run errors
|
||||
//
|
||||
// Some errors that stop a run come from outside the workflow and say nothing
|
||||
// a person can act on ("KeyError: 'latent'"). Each entry here names one such
|
||||
// error by what is in ComfyUI's own report, and says in plain words what to
|
||||
// do. An entry is added only after the cause was found for real.
|
||||
const KNOWN_RUN_ERRORS = Object.freeze([
|
||||
{
|
||||
// Muse MiniMax Director replaces part of H3 for every workflow, and that
|
||||
// part reads a picture from every guide. ComfyUI's own H3 nodes also make
|
||||
// guides that hold sound alone, which is how a clip keeps its sound.
|
||||
matches: ({ type, message, traceback }) =>
|
||||
type === "KeyError" && message.includes("'latent'") && traceback.includes("muse_minimax_hybrid_patch"),
|
||||
summary: "Another node pack stopped this run",
|
||||
detail:
|
||||
"This error comes from the Muse MiniMax Director pack, not from this workflow. Turn that pack off in "
|
||||
+ "Manager, restart ComfyUI and run again.",
|
||||
},
|
||||
{
|
||||
// The w6a8 H3 files load from ComfyUI 0.38.0 on; older versions stop in
|
||||
// the model loader on the name of the format.
|
||||
matches: ({ message }) => message.includes("w6a8_int8"),
|
||||
summary: "This model needs a newer ComfyUI",
|
||||
detail:
|
||||
"Your ComfyUI is too old to read this model file. Update ComfyUI to 0.38 or newer and run again.",
|
||||
},
|
||||
]);
|
||||
|
||||
/**
|
||||
* Plain advice for a run error the pack knows the cause of, from the
|
||||
* `execution_error` event's detail. `null` for every other error.
|
||||
*/
|
||||
export function explainRunError(detail) {
|
||||
const report = {
|
||||
type: String(detail?.exception_type ?? "").split(".").pop(),
|
||||
message: String(detail?.exception_message ?? ""),
|
||||
traceback: [].concat(detail?.traceback ?? []).join("\n"),
|
||||
};
|
||||
for (const known of KNOWN_RUN_ERRORS) {
|
||||
try {
|
||||
if (known.matches(report)) return { summary: known.summary, detail: known.detail };
|
||||
} catch (_error) {
|
||||
// A report in a shape this entry did not expect is simply not a match.
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -74,6 +74,25 @@ export function loaderSignature(widgets) {
|
||||
return JSON.stringify(settings);
|
||||
}
|
||||
|
||||
// What the picture says when a run stopped at the node. ComfyUI's own text
|
||||
// for a full graphics card is ten lines of numbers; here it is one plain
|
||||
// line. Anything else is said as it came.
|
||||
export const OUT_OF_MEMORY = "The graphics card ran out of memory, so the node could not look. Free some memory on it, then click again.";
|
||||
export function stopMessage(message) {
|
||||
const text = String(message ?? "");
|
||||
return /out of memory|OutOfMemory/i.test(text) ? OUT_OF_MEMORY : text;
|
||||
}
|
||||
|
||||
// A name as it is typed under the picture: "-face" (or the minus sign a
|
||||
// phone types) takes the name away, "+face" adds it; without a sign it does
|
||||
// what `takes` says. Returns { word, add }, the word cleaned.
|
||||
export function typedName(text, takes = false) {
|
||||
const typed = String(text ?? "").trim();
|
||||
const sign = /^[-\u2212\u2013+]/.exec(typed)?.[0] ?? "";
|
||||
const word = cleanWord(sign ? typed.slice(1) : typed);
|
||||
return { word, add: sign ? sign === "+" : !takes };
|
||||
}
|
||||
|
||||
// A word as the node sends it to SAM 3: commas, colons and brackets are the
|
||||
// text reader's own syntax and go; spaces are collapsed.
|
||||
export function cleanWord(word) {
|
||||
@@ -1044,19 +1063,46 @@ export function strokePieces(planes, list, sizes, pixels, stroke, ranks, wholes
|
||||
// picked area, and as untouched while almost none of it is.
|
||||
const NEARLY = 0.03;
|
||||
|
||||
// Which pieces hold a whole pick: nearly all of one picked area (`areas`,
|
||||
// on/off masks, one per chip) lies inside the piece. A person holds the
|
||||
// sweater picked on them; hair that a picked jacket only laps over does not.
|
||||
export const HOLDS = 0.9;
|
||||
export function piecesHolding(planes, count, areas) {
|
||||
const holds = new Uint8Array(count);
|
||||
for (const area of areas ?? []) {
|
||||
if (!area) continue;
|
||||
let size = 0;
|
||||
for (let index = 0; index < area.length; index += 1) if (area[index]) size += 1;
|
||||
if (!size) continue;
|
||||
const inside = piecesInside(planes, count, area);
|
||||
for (let piece = 0; piece < count; piece += 1) if (inside[piece] >= HOLDS * size) holds[piece] = 1;
|
||||
}
|
||||
return holds;
|
||||
}
|
||||
|
||||
// Which piece a pointer at pixel `index` lights up: the first piece there,
|
||||
// in the order they are offered (`ranks`; without it, smallest first), that
|
||||
// would still change something. Adding, that is a piece not yet (nearly)
|
||||
// all picked, so clicking the same spot again goes on from the glasses to
|
||||
// the face to the person. Taking away, it is a piece that has some of the
|
||||
// picked area in it. -1 when there is none.
|
||||
export function pieceAt(planes, index, { sizes, inside, adding = true, ranks = null } = {}) {
|
||||
//
|
||||
// Taking away from a spot that is not picked itself (`here` false) never
|
||||
// means a thing that holds a whole pick somewhere else (`holds`, from
|
||||
// piecesHolding): with a sweater picked, pointing at the face does not
|
||||
// offer the person, which would take the sweater along.
|
||||
export function pieceAt(planes, index, { sizes, inside, adding = true, ranks = null, here = true, holds = null } = {}) {
|
||||
for (const piece of candidatesAt(planes, index, ranks)) {
|
||||
const size = sizes?.[piece] ?? 0;
|
||||
if (!size) continue;
|
||||
const picked = inside?.[piece] ?? 0;
|
||||
const slack = Math.max(4, size * NEARLY);
|
||||
if (adding ? size - picked > slack : picked > slack) return piece;
|
||||
if (adding) {
|
||||
if (size - picked > slack) return piece;
|
||||
continue;
|
||||
}
|
||||
if (picked <= slack || (!here && holds?.[piece])) continue;
|
||||
return piece;
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
+106
-19
@@ -77,6 +77,7 @@ import {
|
||||
pieceRanks,
|
||||
pieceWholes,
|
||||
pieceSizes,
|
||||
piecesHolding,
|
||||
piecesInside,
|
||||
prefixStamps,
|
||||
previewPixels,
|
||||
@@ -85,9 +86,11 @@ import {
|
||||
sharedPixels,
|
||||
shrinkMask,
|
||||
stampOf,
|
||||
stopMessage,
|
||||
strokePieces,
|
||||
thinStroke,
|
||||
touchFits,
|
||||
typedName,
|
||||
} from "./touch_math.mjs";
|
||||
|
||||
const CSS_ID = "ausboss-touch-css";
|
||||
@@ -190,9 +193,17 @@ function ensureCss() {
|
||||
.ausboss-touch-chip>button:hover{opacity:1;background:rgba(255,255,255,.14);}
|
||||
.ausboss-touch-chip>button.more{padding:0 6px;background:rgba(0,0,0,.35);opacity:.95;}
|
||||
.ausboss-touch-chip.open>button.more{background:rgba(0,0,0,.2);}
|
||||
.ausboss-touch-name{box-sizing:border-box;flex:none;width:104px;height:20px;padding:0 9px;border:1px dashed rgba(0,180,170,.5);border-radius:10px;background:transparent;color:#dffcf9;font:600 10px/18px -apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;outline:none;}
|
||||
.ausboss-touch-namebox{box-sizing:border-box;flex:none;display:inline-flex;align-items:stretch;height:20px;border:1px dashed rgba(0,180,170,.5);border-radius:10px;overflow:hidden;}
|
||||
.ausboss-touch-namebox:focus-within{border-style:solid;border-color:${BRAND};background:rgba(0,0,0,.4);}
|
||||
.ausboss-touch-namebox.away{border-color:rgba(${LIT_AWAY.join(",")},.6);}
|
||||
.ausboss-touch-namebox.away:focus-within{border-color:rgb(${LIT_AWAY.join(",")});}
|
||||
.ausboss-touch-sign{box-sizing:border-box;flex:none;width:22px;padding:0;border:0;border-right:1px solid rgba(0,180,170,.35);background:rgba(0,180,170,.16);color:#dffcf9;font:700 13px/18px -apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;cursor:pointer;}
|
||||
.ausboss-touch-sign:hover{background:rgba(0,180,170,.34);color:#fff;}
|
||||
.ausboss-touch-sign:disabled{opacity:.4;cursor:default;}
|
||||
.ausboss-touch-namebox.away .ausboss-touch-sign{border-right-color:rgba(${LIT_AWAY.join(",")},.4);background:rgba(${LIT_AWAY.join(",")},.24);color:#ffe9e2;}
|
||||
.ausboss-touch-namebox.away .ausboss-touch-sign:hover{background:rgba(${LIT_AWAY.join(",")},.42);}
|
||||
.ausboss-touch-name{box-sizing:border-box;flex:none;width:118px;height:18px;padding:0 8px 0 6px;border:0;background:transparent;color:#dffcf9;font:600 10px/18px -apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;outline:none;}
|
||||
.ausboss-touch-name::placeholder{color:#78908e;font-weight:500;}
|
||||
.ausboss-touch-name:focus{border-style:solid;border-color:${BRAND};background:rgba(0,0,0,.4);}
|
||||
.ausboss-touch-gap{flex:1 1 auto;min-width:0;}
|
||||
.ausboss-touch-stage{position:relative;flex:1 1 auto;min-height:0;width:100%;overflow:hidden;border:1px solid rgba(0,180,170,.27);border-radius:6px;background:rgba(0,0,0,.28);}
|
||||
[data-node-id] .ausboss-touch-stage{box-sizing:border-box;min-height:${STAGE_MIN}px;}
|
||||
@@ -224,9 +235,12 @@ function ensureCss() {
|
||||
.ausboss-touch.fining .ausboss-touch-fining{display:flex;}
|
||||
.ausboss-touch-fining::before{content:"";flex:none;width:7px;height:7px;border-radius:50%;background:${BRAND};animation:ausboss-touch-pulse 1s ease-in-out infinite;}
|
||||
@keyframes ausboss-touch-pulse{0%,100%{opacity:.25;}50%{opacity:1;}}
|
||||
.ausboss-touch-btn.lookpill{display:inline-flex;align-items:center;gap:5px;height:${BAR_HEIGHT}px;padding:0 9px 0 8px;border:1px solid rgba(0,180,170,.6);border-radius:10px;letter-spacing:0;text-transform:none;font:600 10px/18px -apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;color:#dffcf9;background:linear-gradient(90deg,rgba(0,180,170,.38) var(--done,0%),rgba(0,180,170,.08) var(--done,0%));}
|
||||
.ausboss-touch-btn.lookpill{display:inline-flex;align-items:center;gap:5px;height:${BAR_HEIGHT}px;padding:0 9px 0 0;overflow:hidden;border:1px solid rgba(0,180,170,.8);border-radius:10px;letter-spacing:0;text-transform:none;font:600 10px/18px -apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;color:#dffcf9;background:linear-gradient(90deg,rgba(0,180,170,.38) var(--done,0%),rgba(0,180,170,.08) var(--done,0%));}
|
||||
.ausboss-touch-btn.lookpill:hover{border-color:${BRAND};color:#fff;background:linear-gradient(90deg,rgba(0,180,170,.5) var(--done,0%),rgba(0,180,170,.16) var(--done,0%));}
|
||||
.ausboss-touch-btn.lookpill .icon{flex:none;font-size:8px;letter-spacing:-1px;opacity:.9;}
|
||||
.ausboss-touch-btn.lookpill .key{flex:none;display:inline-flex;align-items:center;gap:4px;align-self:stretch;padding:0 7px 0 8px;background:${BRAND};color:#04201e;font-weight:700;}
|
||||
.ausboss-touch-btn.lookpill:hover .key{background:#35d6cb;}
|
||||
.ausboss-touch-btn.lookpill.paused .key{background:#9fb4b2;}
|
||||
.ausboss-touch-btn.lookpill .icon{flex:none;font-size:8px;letter-spacing:-1px;}
|
||||
.ausboss-touch-btn.lookpill.paused{border-color:rgba(120,144,142,.6);color:#c8dddd;background:linear-gradient(90deg,rgba(120,144,142,.3) var(--done,0%),rgba(120,144,142,.06) var(--done,0%));}
|
||||
.ausboss-touch-btn.lookpill.paused .icon{letter-spacing:0;}
|
||||
.ausboss-touch-btn.lookpill .count{opacity:.8;}
|
||||
@@ -527,6 +541,7 @@ function resolve(face) {
|
||||
face.raw = raw;
|
||||
face.exact = exact;
|
||||
face.chipAreas = null;
|
||||
face.holds = null;
|
||||
face.inside = pic.pieces ? piecesInside(pic.pieces.planes, pic.pieces.list.length, raw) : null;
|
||||
face.hoverMask = null;
|
||||
}
|
||||
@@ -697,6 +712,11 @@ function requestRun(face, ask) {
|
||||
return false;
|
||||
}
|
||||
const run = { ask, stamp: face.stamp, epoch: face.sourceEpoch, queued: false };
|
||||
// Runs of this node that are at a picture it no longer has. The one that
|
||||
// is running ends by itself once the next job waits; the ones that still
|
||||
// wait in the queue would each run for nothing, one after the other, so
|
||||
// they are taken out before this one is sent.
|
||||
const stale = face.flying.filter((other) => other.queued && other.epoch !== run.epoch);
|
||||
face.flying.push(run);
|
||||
face.inFlight = true;
|
||||
face.askedAt = performance.now();
|
||||
@@ -707,7 +727,15 @@ function requestRun(face, ask) {
|
||||
face.flying = face.flying.filter((other) => other !== run);
|
||||
if (!face.flying.length) settle(face);
|
||||
};
|
||||
Promise.resolve(face.host.run(ask === "scope" ? scopeAsk(face) : ask)).then((queued) => {
|
||||
const cleared = stale.length && face.host.dropWaiting
|
||||
? Promise.resolve(face.host.dropWaiting()).then((gone) => {
|
||||
// What was taken out never answers: the newest of the stale ones,
|
||||
// because a run of this node goes to the front of the queue.
|
||||
const out = new Set(stale.slice(Math.max(0, stale.length - (Number(gone) || 0))));
|
||||
if (out.size) face.flying = face.flying.filter((other) => !out.has(other));
|
||||
}, () => {})
|
||||
: Promise.resolve();
|
||||
cleared.then(() => face.host.run(ask === "scope" ? scopeAsk(face) : ask)).then((queued) => {
|
||||
if (queued === false) {
|
||||
failed();
|
||||
return;
|
||||
@@ -1258,11 +1286,23 @@ function takeAll(face, index) {
|
||||
refreshHover(face);
|
||||
}
|
||||
|
||||
// Whether a typed name is taken away: what the sign in front of the field
|
||||
// was set to, or else what the tool does right now (Erase, or Ctrl held).
|
||||
function nameTakes(face) {
|
||||
return face.nameAway ?? face.away;
|
||||
}
|
||||
|
||||
// A typed name, as every thing of that name. "-face" takes it away and
|
||||
// "+face" adds it, whatever the sign says.
|
||||
function addName(face, text) {
|
||||
const word = cleanWord(text);
|
||||
if (!word || !face.pic || face.held) return false;
|
||||
const typed = typedName(text, nameTakes(face));
|
||||
if (!typed.word || !face.pic || face.held) return false;
|
||||
flushPending(face);
|
||||
addItem(face, { t: "all", op: face.away ? 0 : 1, w: word });
|
||||
if (!typed.add && !face.items.some((item) => item.op === 1)) {
|
||||
setNote(face, "Nothing is picked yet. Pick something first, then take a name away from it.", { life: 3200 });
|
||||
return false;
|
||||
}
|
||||
addItem(face, { t: "all", op: typed.add ? 1 : 0, w: typed.word });
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -1413,9 +1453,19 @@ function syncTools(face) {
|
||||
face.lookButton.style.display = offer ? "" : "none";
|
||||
}
|
||||
if (face.nameField) {
|
||||
face.nameField.disabled = !live || face.held > 0;
|
||||
// A typed name is added, or taken away while the tool takes away.
|
||||
face.nameField.placeholder = face.away ? "\u2212 type a name" : "+ type a name";
|
||||
const off = !live || face.held > 0;
|
||||
const takes = nameTakes(face);
|
||||
face.nameField.disabled = off;
|
||||
face.nameSign.disabled = off;
|
||||
face.nameBox.classList.toggle("away", takes);
|
||||
face.nameSign.textContent = takes ? "\u2212" : "+";
|
||||
face.nameSign.title = takes
|
||||
? "A name you type is taken away from the mask. Click to add by name instead."
|
||||
: "A name you type is added to the mask. Click to take away by name instead.";
|
||||
face.nameField.placeholder = takes ? "take away by name" : "add by name";
|
||||
face.nameField.title = takes
|
||||
? "Type what to take out of the mask, like \"face\" or \"hair\", and press Enter. Every thing of that name is taken away."
|
||||
: "Type what to pick, like \"screen\" or \"earring\", and press Enter. Every thing of that name is taken. Start with a minus, like \"-face\", to take it away instead.";
|
||||
}
|
||||
if (face.newButton) face.newButton.disabled = !live || face.held > 0 || Boolean(face.fresh);
|
||||
// The switch shows what a press does right now: with Ctrl held, Paint
|
||||
@@ -1949,12 +1999,31 @@ function pointOf(face, event) {
|
||||
return [clamp(x, 0, face.pic.width - 1), clamp(y, 0, face.pic.height - 1)];
|
||||
}
|
||||
|
||||
// The pieces that hold a whole pick, worked out once per change to the
|
||||
// picks and only when something is being taken away.
|
||||
function holdsOf(face) {
|
||||
const pic = face.pic;
|
||||
if (!pic?.pieces || !face.raw) return null;
|
||||
if (!face.holds || face.holds.length !== pic.pieces.list.length) {
|
||||
const areas = chipGroups(face.items).filter((chip) => !chip.away).map((chip) => chipArea(face, chip));
|
||||
face.holds = piecesHolding(pic.pieces.planes, pic.pieces.list.length, areas);
|
||||
}
|
||||
return face.holds;
|
||||
}
|
||||
|
||||
function pieceUnder(face, point, adding) {
|
||||
const pic = face.pic;
|
||||
if (!pic?.pieces || !point) return -1;
|
||||
const x = clamp(Math.floor(point[0] * pic.scale), 0, pic.pw - 1);
|
||||
const y = clamp(Math.floor(point[1] * pic.scale), 0, pic.ph - 1);
|
||||
return pieceAt(pic.pieces.planes, y * pic.pw + x, { sizes: pic.pieces.sizes, inside: face.inside, adding, ranks: pic.pieces.ranks });
|
||||
const at = y * pic.pw + x;
|
||||
if (adding) return pieceAt(pic.pieces.planes, at, { sizes: pic.pieces.sizes, inside: face.inside, ranks: pic.pieces.ranks });
|
||||
// Taking away from a spot that is not picked: never a thing that holds a
|
||||
// whole pick somewhere else (the person, for the sweater picked on them).
|
||||
return pieceAt(pic.pieces.planes, at, {
|
||||
sizes: pic.pieces.sizes, inside: face.inside, adding, ranks: pic.pieces.ranks,
|
||||
here: face.raw?.[at] >= 0.5, holds: holdsOf(face),
|
||||
});
|
||||
}
|
||||
|
||||
// The pixel of the small copy under a point of the picture.
|
||||
@@ -2324,7 +2393,8 @@ function syncLooking(face) {
|
||||
pill.classList.toggle("paused", paused);
|
||||
pill.style.setProperty("--done", `${total ? Math.round((done / total) * 100) : 0}%`);
|
||||
face.lookIcon.textContent = paused ? "\u25b6" : "\u275a\u275a";
|
||||
face.lookWord.textContent = paused ? "Paused" : "Looking";
|
||||
face.lookVerb.textContent = paused ? "Look on" : "Pause";
|
||||
face.lookWord.textContent = paused ? "paused" : "looking";
|
||||
face.lookCount.textContent = total ? `${done}/${total}` : "\u2026";
|
||||
const far = total ? ` ${done} of ${total}` : "";
|
||||
pill.title = paused
|
||||
@@ -2717,6 +2787,8 @@ function setSoft(face, soft) {
|
||||
|
||||
// Pick takes whole things; Paint and Erase are the brush.
|
||||
function setTool(face, tool) {
|
||||
// The sign in front of the name field follows the tool again.
|
||||
face.nameAway = null;
|
||||
if (!TOOLS.includes(tool)) return;
|
||||
face.tool = tool;
|
||||
face.snap = tool === "pick";
|
||||
@@ -2773,16 +2845,21 @@ export function mountTouchStage(node, host) {
|
||||
const bar = el("div", "ausboss-touch-bar");
|
||||
const chips = el("div", "ausboss-touch-chips");
|
||||
const chipList = el("span", "ausboss-touch-chiplist");
|
||||
// A name typed here picks every thing of that name, or takes it away:
|
||||
// the sign in front says which, and a press on it changes it.
|
||||
const nameBox = el("span", "ausboss-touch-namebox");
|
||||
const nameSign = el("button", "ausboss-touch-sign", "+");
|
||||
nameSign.type = "button";
|
||||
const nameField = el("input", "ausboss-touch-name");
|
||||
nameField.type = "text";
|
||||
nameField.placeholder = "+ type a name";
|
||||
nameField.placeholder = "add by name";
|
||||
nameField.maxLength = 60;
|
||||
nameField.spellcheck = false;
|
||||
nameField.title = "Type what to pick, like \"screen\" or \"earring\", and press Enter. Every thing of that name is taken.";
|
||||
nameBox.append(nameSign, nameField);
|
||||
const newButton = el("button", "ausboss-touch-new", "+ new");
|
||||
newButton.type = "button";
|
||||
newButton.title = "Start a new area: what you paint next gets a chip of its own, also where it touches another area.";
|
||||
chips.append(chipList, newButton, nameField);
|
||||
chips.append(chipList, newButton, nameBox);
|
||||
root.append(tools, stage, bar, chips);
|
||||
|
||||
const frame = el("div", "ausboss-touch-frame");
|
||||
@@ -2827,7 +2904,7 @@ export function mountTouchStage(node, host) {
|
||||
|
||||
const face = node.__ausbossTouch = {
|
||||
node, host, root, stage, frame, img, tint, lit, ink, path, dots, ringBack, ring, coreRing, dab, brushLabel, tag, hint, noteEl, abort,
|
||||
chips, chipList, newButton, nameField, chipsShown: null,
|
||||
chips, chipList, newButton, nameField, nameBox, nameSign, nameAway: null, chipsShown: null,
|
||||
flying: [],
|
||||
pendingTimer: 0,
|
||||
alive: true,
|
||||
@@ -3031,10 +3108,15 @@ export function mountTouchStage(node, host) {
|
||||
if (face.pic?.paused && !face.inFlight) resumeLook(face);
|
||||
else pauseLook(face);
|
||||
}, signal);
|
||||
// The press it takes comes first, as a button of its own look: "Pause"
|
||||
// while the node looks, "Look on" once it is paused.
|
||||
face.lookIcon = el("span", "icon");
|
||||
face.lookVerb = el("span", "verb");
|
||||
face.lookWord = el("span", "word");
|
||||
face.lookCount = el("span", "count");
|
||||
face.lookPill.append(face.lookIcon, face.lookWord, face.lookCount);
|
||||
const lookKey = el("span", "key");
|
||||
lookKey.append(face.lookIcon, face.lookVerb);
|
||||
face.lookPill.append(lookKey, face.lookWord, face.lookCount);
|
||||
face.lookPill.style.display = "none";
|
||||
// First in the line under the picture: what the node is doing with the
|
||||
// picture, then what is picked; the buttons that pick come after.
|
||||
@@ -3044,6 +3126,11 @@ export function mountTouchStage(node, host) {
|
||||
for (const type of ["keydown", "keyup", "keypress", "pointerdown", "mousedown", "dblclick"]) {
|
||||
chips.addEventListener(type, (event) => event.stopPropagation(), { signal });
|
||||
}
|
||||
nameSign.addEventListener("click", () => {
|
||||
face.nameAway = !nameTakes(face);
|
||||
syncTools(face);
|
||||
nameField.focus();
|
||||
}, { signal });
|
||||
nameField.addEventListener("keydown", (event) => {
|
||||
if (event.key === "Enter") {
|
||||
event.preventDefault();
|
||||
@@ -3096,7 +3183,7 @@ export function mountTouchStage(node, host) {
|
||||
// The run stopped at this node with a message for the person.
|
||||
face.stopped = (message) => {
|
||||
settle(face);
|
||||
setNote(face, String(message ?? ""));
|
||||
setNote(face, stopMessage(message));
|
||||
// It stays up until something else is said or the next result comes in.
|
||||
face.stopNote = true;
|
||||
};
|
||||
|
||||
@@ -121,6 +121,15 @@ const CARDS = {
|
||||
{ widget: "interpolation", label: "Filter", kind: "select" },
|
||||
],
|
||||
},
|
||||
AUSBOSS_NODES_EditLock: {
|
||||
minWidth: 300,
|
||||
rows: [
|
||||
{ widget: "lock", label: "Lock", kind: "segment",
|
||||
titles: { auto: "Lock the untouched parts; step aside when the whole picture changes", always: "Lock even when the whole picture changes", off: "Do nothing" } },
|
||||
{ widget: "tolerance", label: "Tolerance", decimals: 2, suffix: "×" },
|
||||
],
|
||||
},
|
||||
AUSBOSS_NODES_EditLockRestore: { minWidth: 300, rows: [{ widget: "feather", label: "Feather", suffix: "px" }] },
|
||||
AUSBOSS_NODES_ColorMatch: {
|
||||
minWidth: 300,
|
||||
rows: [
|
||||
@@ -331,6 +340,17 @@ const CARDS = {
|
||||
rows: [{ widget: "text", kind: "textarea", placeholder: "Prompt, caption, or shared text", height: 100, grow: true,
|
||||
looks: { big: { height: 62, placeholder: "type here" } } }],
|
||||
},
|
||||
// One line for each run. The three settings under the box say where a run
|
||||
// ends, whether a leading "1." comes off, and how many lines run at most.
|
||||
AUSBOSS_NODES_TextLines: {
|
||||
minWidth: 320,
|
||||
rows: [
|
||||
{ widget: "text", kind: "textarea", placeholder: "One line for each run", height: 100, grow: true },
|
||||
{ widget: "split", label: "Split at", kind: "segment" },
|
||||
{ widget: "remove_numbering", label: "Numbering", onText: "remove", offText: "keep" },
|
||||
{ widget: "limit", label: "Limit" },
|
||||
],
|
||||
},
|
||||
// All the words go in the box, and the strip under it shows when each
|
||||
// moment plays. The pictures, the video, the voice and the song arrive on
|
||||
// sockets, and each picture's job sits under the words.
|
||||
|
||||
@@ -172,6 +172,64 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"AUSBOSS_NODES_EditLock": {
|
||||
"display_name": "编辑锁定 🆎",
|
||||
"description": "让图片里编辑没有改动的部分保持在原位,颜色也不变。编辑模型会把整张图重画一遍,没改的部分常常略微放大、偏移或偏色。把这个节点放在模型和采样器之间,并接上你要编辑的图片。不需要遮罩:它在出图过程中看出编辑改了哪里。如果编辑改变的是整张图(新风格、新光线、新天气),它会自动让开。解码后再接上编辑锁定还原 🆎,未改动的部分就能换回你原图的像素。",
|
||||
"inputs": {
|
||||
"model": {
|
||||
"name": "模型"
|
||||
},
|
||||
"vae": {
|
||||
"name": "vae"
|
||||
},
|
||||
"image": {
|
||||
"name": "图像"
|
||||
},
|
||||
"lock": {
|
||||
"name": "锁定"
|
||||
},
|
||||
"tolerance": {
|
||||
"name": "容差"
|
||||
},
|
||||
"mask": {
|
||||
"name": "遮罩"
|
||||
}
|
||||
},
|
||||
"outputs": {
|
||||
"0": {
|
||||
"name": "模型"
|
||||
},
|
||||
"1": {
|
||||
"name": "编辑锁"
|
||||
}
|
||||
}
|
||||
},
|
||||
"AUSBOSS_NODES_EditLockRestore": {
|
||||
"display_name": "编辑锁定还原 🆎",
|
||||
"description": "把编辑没有改动的部分换回你原图自己的像素。放在 VAE 解码之后,并接上编辑锁定 🆎 的 edit_lock 输出。未改动的部分与原图逐像素一致,只有编辑的部分是新的。如果锁定让开了,图片原样通过。",
|
||||
"inputs": {
|
||||
"image": {
|
||||
"name": "图像"
|
||||
},
|
||||
"edit_lock": {
|
||||
"name": "编辑锁"
|
||||
},
|
||||
"feather": {
|
||||
"name": "羽化"
|
||||
}
|
||||
},
|
||||
"outputs": {
|
||||
"0": {
|
||||
"name": "图像"
|
||||
},
|
||||
"1": {
|
||||
"name": "编辑遮罩"
|
||||
},
|
||||
"2": {
|
||||
"name": "报告"
|
||||
}
|
||||
}
|
||||
},
|
||||
"AUSBOSS_NODES_Float": {
|
||||
"display_name": "浮点数 🆎",
|
||||
"description": "一个独立的 FLOAT 值。在多个节点之间共用同一个强度、缩放或 CFG,只需在一处修改。",
|
||||
@@ -1240,6 +1298,35 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"AUSBOSS_NODES_TextLines": {
|
||||
"display_name": "逐行文本 🆎",
|
||||
"description": "文本的每一行都会让后面的节点运行一次。输入五条提示词,每行一条,运行一次就得到五张图。空行会跳过,行首的序号或项目符号可以去掉。",
|
||||
"inputs": {
|
||||
"text": {
|
||||
"name": "文本"
|
||||
},
|
||||
"split": {
|
||||
"name": "分隔方式"
|
||||
},
|
||||
"remove_numbering": {
|
||||
"name": "去掉序号"
|
||||
},
|
||||
"limit": {
|
||||
"name": "上限"
|
||||
}
|
||||
},
|
||||
"outputs": {
|
||||
"0": {
|
||||
"name": "行"
|
||||
},
|
||||
"1": {
|
||||
"name": "序号"
|
||||
},
|
||||
"2": {
|
||||
"name": "数量"
|
||||
}
|
||||
}
|
||||
},
|
||||
"AUSBOSS_NODES_TiledUpscale": {
|
||||
"display_name": "分块放大 🆎",
|
||||
"description": "以百万像素设置成品尺寸,并把图片切成模型可以逐块重画的图块。一块就能装下的图片保持完整。把图块送进采样器,再和 stitcher 一起送进分块放大拼合 🆎。小图不会被重画到远超其细节的程度:最多重画到原百万像素的 max_growth 倍,再用普通方式放大。report 会说明对你的图片做了什么。",
|
||||
|
||||
@@ -0,0 +1,455 @@
|
||||
"""Edit Lock: keep what an edit did not change. Pure torch, no ComfyUI imports.
|
||||
|
||||
At every sampling step an edit model predicts the finished picture. Where
|
||||
that prediction sits close to the original, the edit is not happening
|
||||
there, so the original is put back and the model carries on from the real
|
||||
picture. This is the target TailSteer trains a model toward
|
||||
(original where nothing changed, the model's edit elsewhere), applied while
|
||||
sampling instead of through training.
|
||||
|
||||
What counts as changed is measured against the quiet part of the picture:
|
||||
the tenth of it where the prediction differs least, which a local edit
|
||||
leaves near the level of the model's own redraw noise. The node watches the
|
||||
first steps, where it only undoes the model's zoom and shift, and decides
|
||||
once the picture has formed. An edit of the whole picture (a style, the
|
||||
light) has no quiet part: the first look checks for that and the lock steps
|
||||
aside. A soft change over the whole picture (rain, snow, haze) has no clear
|
||||
peak: the decision checks for that, and the picture is only held in place.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import torch
|
||||
import torch.nn.functional as F
|
||||
|
||||
LOCK_MODES = ("auto", "always", "off")
|
||||
|
||||
# A spot counts as changed when it differs from the original by STRONG times
|
||||
# the quiet level, or by WEAK times while it touches a spot that does. So a
|
||||
# faint change next to a clear one belongs to the edit, and a faint change
|
||||
# alone is redraw noise. Set against what 14 plain MiniMax-H3-Image edits
|
||||
# changed in the end: this frees about 80% of it at the first look and 3%
|
||||
# of what stayed.
|
||||
WEAK = 2.0
|
||||
STRONG = 3.0
|
||||
# The model often draws the whole picture a little larger or shifted. That
|
||||
# is measured and discounted before anything is compared, up to these
|
||||
# limits; a bigger move is treated as part of the edit.
|
||||
# The node watches the start of a run and decides once the noise is down to
|
||||
# this share of where it began. The first prediction is a blur: rain, snow,
|
||||
# smoke and the light a new hair color throws on a face only show a few
|
||||
# steps in. While it watches it only undoes the zoom and shift.
|
||||
DECIDE_AT = 0.8
|
||||
# A local edit has a peak: somewhere the picture clearly changes. Rain, snow
|
||||
# and haze change the whole picture a little and no part clearly. A lock
|
||||
# would hold them back, or let them through in patches where it hands a wide
|
||||
# part of the picture to the edit. So two things are read at the decision,
|
||||
# in units of the quiet level: the strongest spot, and the strongest
|
||||
# hundredth when the lock would free SOFT_WIDE of the picture or more. On
|
||||
# recorded MiniMax-H3-Image runs the strongest spot read 4.7 or more for
|
||||
# every local edit, glasses and makeup included, and 3.7 or less for light
|
||||
# rain, fog and snow; with a wide part freed, the strongest hundredth read
|
||||
# 4.4 or more for local edits and 3.6 or less for rain and snow.
|
||||
SOFT_WIDE = 0.10
|
||||
SOFT_PEAK = 4.0
|
||||
# Kept this much or more: the report adds what to do if the edit is missing.
|
||||
# A soft effect over the whole picture (rain, snow, smoke) looks like a small
|
||||
# edit at the first look, and under the lock the model then drops it.
|
||||
NEARLY_ALL = 0.95
|
||||
MAX_ZOOM = 0.05
|
||||
MAX_SHIFT = 4.0
|
||||
# The whole picture is changing when even its calmest fifth sits this far
|
||||
# from the original at the first look, in units of the original's own
|
||||
# spread. Measured on MiniMax-H3-Image: 28 local edits at 1, 4 and 16 MP came
|
||||
# in at 0.22 or less (zoomed ones included), 12 whole-picture edits (styles,
|
||||
# light, weather) at 0.27 or more.
|
||||
WHOLE_PICTURE = 0.243
|
||||
# Blur and growth of the change map, as a share of the short side.
|
||||
REACH = 1.0 / 54.0
|
||||
# A result this much larger or smaller than the picture is another picture.
|
||||
SIZE_SLACK = 0.04
|
||||
|
||||
|
||||
def box_blur(x: torch.Tensor, radius: int) -> torch.Tensor:
|
||||
if radius <= 0:
|
||||
return x
|
||||
size = 2 * radius + 1
|
||||
return F.avg_pool2d(F.pad(x, (radius,) * 4, mode="replicate"), size, stride=1)
|
||||
|
||||
|
||||
def reach(height: int, width: int) -> int:
|
||||
return max(1, round(min(height, width) * REACH))
|
||||
|
||||
|
||||
def _moved(source: torch.Tensor, move: torch.Tensor) -> torch.Tensor:
|
||||
"""The original sampled at (zoom_x * x + shift_x, zoom_y * y + shift_y), x and y in cells from the centre.
|
||||
|
||||
move = [zoom_x - 1, zoom_y - 1, shift_x, shift_y].
|
||||
"""
|
||||
height, width = source.shape[-2:]
|
||||
theta = torch.zeros(source.shape[0], 2, 3, device=source.device, dtype=source.dtype)
|
||||
theta[:, 0, 0] = 1.0 + move[0]
|
||||
theta[:, 1, 1] = 1.0 + move[1]
|
||||
theta[:, 0, 2] = 2.0 * move[2] / width
|
||||
theta[:, 1, 2] = 2.0 * move[3] / height
|
||||
grid = F.affine_grid(theta, list(source.shape), align_corners=False)
|
||||
return F.grid_sample(source, grid, mode="bilinear", padding_mode="border", align_corners=False)
|
||||
|
||||
|
||||
def measure_redraw(pred: torch.Tensor, source: torch.Tensor) -> torch.Tensor:
|
||||
"""How much larger and how far shifted the prediction is drawn, as the move that lays the original over it.
|
||||
|
||||
pred and source are [1, C, H, W]. Fitted on the calmer half of the
|
||||
picture, so the edit itself does not pull the answer, from a coarse grid
|
||||
down to the full one. Returns [zoom_x - 1, zoom_y - 1, shift_x, shift_y];
|
||||
zeros when nothing can be measured.
|
||||
"""
|
||||
pred, source = pred.float(), source.float()
|
||||
move = torch.zeros(4, device=pred.device, dtype=torch.float32)
|
||||
limit = torch.tensor([MAX_ZOOM, MAX_ZOOM, MAX_SHIFT, MAX_SHIFT], device=pred.device)
|
||||
for level in (4, 2, 1):
|
||||
if min(pred.shape[-2:]) // level < 16:
|
||||
continue
|
||||
a = box_blur(F.avg_pool2d(pred, level) if level > 1 else pred, 1)
|
||||
b = box_blur(F.avg_pool2d(source, level) if level > 1 else source, 1)
|
||||
height, width = a.shape[-2:]
|
||||
ys = (torch.arange(height, device=a.device, dtype=torch.float32) + 0.5 - height / 2)[:, None].expand(height, width)
|
||||
xs = (torch.arange(width, device=a.device, dtype=torch.float32) + 0.5 - width / 2)[None, :].expand(height, width)
|
||||
here = move.clone()
|
||||
here[2:] = here[2:] / level
|
||||
for _ in range(6):
|
||||
laid = _moved(b, here)
|
||||
rest = a - laid
|
||||
rest = rest - rest.flatten(2).median(dim=2).values[:, :, None, None]
|
||||
wide = F.pad(laid, (1, 1, 1, 1), mode="replicate")
|
||||
gx = (wide[..., 1:-1, 2:] - wide[..., 1:-1, :-2]) / 2.0
|
||||
gy = (wide[..., 2:, 1:-1] - wide[..., :-2, 1:-1]) / 2.0
|
||||
size = rest.abs().mean(dim=1, keepdim=True)
|
||||
calm = (size <= size.flatten().median()).float()
|
||||
calm[..., :2, :] = 0.0
|
||||
calm[..., -2:, :] = 0.0
|
||||
calm[..., :, :2] = 0.0
|
||||
calm[..., :, -2:] = 0.0
|
||||
jac = (torch.stack([gx * xs, gy * ys, gx, gy], dim=-1) * calm[..., None]).reshape(-1, 4)
|
||||
normal = jac.T @ jac + 1e-6 * torch.eye(4, device=a.device)
|
||||
try:
|
||||
change = torch.linalg.solve(normal, jac.T @ (rest * calm).reshape(-1))
|
||||
except RuntimeError:
|
||||
return torch.zeros(4, device=pred.device, dtype=torch.float32)
|
||||
if not bool(torch.isfinite(change).all()):
|
||||
return torch.zeros(4, device=pred.device, dtype=torch.float32)
|
||||
here = here + change
|
||||
if float(change.abs().max()) < 1e-4:
|
||||
break
|
||||
move = here.clone()
|
||||
move[2:] = move[2:] * level
|
||||
move = torch.maximum(torch.minimum(move, limit), -limit)
|
||||
return move
|
||||
|
||||
|
||||
def redraws(pred: torch.Tensor, source: torch.Tensor) -> list[torch.Tensor]:
|
||||
"""measure_redraw for each picture of a batch."""
|
||||
return [measure_redraw(pred[index:index + 1], source[:1]) for index in range(pred.shape[0])]
|
||||
|
||||
|
||||
def difference(pred: torch.Tensor, source: torch.Tensor, moves: list[torch.Tensor] | None = None) -> torch.Tensor:
|
||||
"""How far each spot of the prediction is from the original, [B, 1, H, W].
|
||||
|
||||
Two things are not a change. A redraw of the whole picture slightly
|
||||
larger or shifted: the original is laid over the prediction first. And
|
||||
an even tint: each channel's middle difference is taken out. The result
|
||||
is in units of the original's own spread, which keeps the numbers alike
|
||||
between models.
|
||||
"""
|
||||
pred = pred.float()
|
||||
source = source.float()
|
||||
moves = moves if moves is not None else redraws(pred, source)
|
||||
out = []
|
||||
for index in range(pred.shape[0]):
|
||||
delta = pred[index:index + 1] - _moved(source[:1], moves[index])
|
||||
delta = delta - delta.flatten(2).median(dim=2).values[:, :, None, None]
|
||||
out.append(delta.abs().mean(dim=1, keepdim=True))
|
||||
magnitude = torch.cat(out) / source.std().clamp_min(1e-6)
|
||||
return box_blur(magnitude, reach(*magnitude.shape[-2:]))
|
||||
|
||||
|
||||
def moved_back(pred: torch.Tensor, moves: list[torch.Tensor]) -> torch.Tensor:
|
||||
"""The prediction with its zoom and shift undone, so it sits where the original does.
|
||||
|
||||
Resampling softens a latent, so a move under a tenth of a cell at the
|
||||
corner is left alone, and the rest is resampled bicubic: on recorded
|
||||
runs that kept 85% of the fine detail where bilinear kept 65%.
|
||||
"""
|
||||
out = []
|
||||
for index in range(pred.shape[0]):
|
||||
one = pred[index:index + 1]
|
||||
move = moves[index]
|
||||
height, width = one.shape[-2:]
|
||||
corner = (move[0].abs() * width / 2 + move[2].abs()).maximum(move[1].abs() * height / 2 + move[3].abs())
|
||||
if float(corner) < 0.1:
|
||||
out.append(one)
|
||||
continue
|
||||
theta = torch.zeros(1, 2, 3, device=one.device, dtype=torch.float32)
|
||||
theta[:, 0, 0] = 1.0 / (1.0 + move[0])
|
||||
theta[:, 1, 1] = 1.0 / (1.0 + move[1])
|
||||
theta[:, 0, 2] = -2.0 * move[2] / (1.0 + move[0]) / width
|
||||
theta[:, 1, 2] = -2.0 * move[3] / (1.0 + move[1]) / height
|
||||
grid = F.affine_grid(theta, list(one.shape), align_corners=False)
|
||||
back = F.grid_sample(one.float(), grid, mode="bicubic", padding_mode="border", align_corners=False)
|
||||
out.append(back.to(one.dtype))
|
||||
return torch.cat(out)
|
||||
|
||||
|
||||
def broad_change(pred: torch.Tensor, source: torch.Tensor) -> torch.Tensor:
|
||||
"""How far the calmest fifth of each picture sits from the original, [B].
|
||||
|
||||
Read on a smoothed copy, where a redraw one percent larger nearly
|
||||
vanishes and a new style, new light or new weather does not. An even
|
||||
tint is taken out first: the model adds one by itself, a stronger one
|
||||
on large pictures, and that is no edit.
|
||||
"""
|
||||
delta = (pred - source).float()
|
||||
delta = delta - delta.flatten(2).median(dim=2).values[:, :, None, None]
|
||||
delta = box_blur(delta, max(1, round(min(delta.shape[-2:]) / 30.0)))
|
||||
level = (delta * delta).mean(dim=1).sqrt().flatten(1)
|
||||
stride = max(1, level.shape[1] // 200000)
|
||||
spread = source.float().std().clamp_min(1e-6)
|
||||
return torch.quantile(level[:, ::stride], 0.2, dim=1) / spread
|
||||
|
||||
|
||||
def quiet_level(magnitude: torch.Tensor) -> torch.Tensor:
|
||||
"""The tenth of each picture that differs least, [B]."""
|
||||
flat = magnitude.flatten(1)
|
||||
stride = max(1, flat.shape[1] // 200000)
|
||||
return torch.quantile(flat[:, ::stride], 0.1, dim=1)
|
||||
|
||||
|
||||
def peaks(magnitude: torch.Tensor, quiet: torch.Tensor) -> tuple[float, float]:
|
||||
"""How clear the clearest change is, in units of the quiet level: (strongest spot, strongest hundredth)."""
|
||||
ratio = (magnitude / quiet.clamp_min(1e-6)[:, None, None, None]).flatten(1)
|
||||
stride = max(1, ratio.shape[1] // 200000)
|
||||
return float(ratio.max(dim=1).values.min()), float(torch.quantile(ratio[:, ::stride], 0.99, dim=1).min())
|
||||
|
||||
|
||||
def change_core(magnitude: torch.Tensor, quiet: torch.Tensor, tolerance: float = 1.0,
|
||||
held: torch.Tensor | None = None) -> torch.Tensor:
|
||||
"""1 where the edit changed the picture, 0 where it did not, hard-edged, [B, 1, H, W].
|
||||
|
||||
`held` is what earlier steps called changed: it stays changed, and
|
||||
faint change that touches it joins it.
|
||||
"""
|
||||
ratio = magnitude / quiet.clamp_min(1e-6)[:, None, None, None]
|
||||
weak = (ratio > WEAK * tolerance).float()
|
||||
core = (ratio > STRONG * tolerance).float()
|
||||
if held is not None:
|
||||
weak = torch.maximum(weak, held)
|
||||
core = torch.maximum(core, held)
|
||||
for _ in range(sum(core.shape[-2:])):
|
||||
grown = F.max_pool2d(core, 3, stride=1, padding=1) * weak
|
||||
if torch.equal(grown, core):
|
||||
break
|
||||
core = grown
|
||||
return core
|
||||
|
||||
|
||||
def soften(core: torch.Tensor) -> torch.Tensor:
|
||||
"""The hard change map with a margin and a soft edge, so the edit has room to blend in."""
|
||||
grow = reach(*core.shape[-2:])
|
||||
return box_blur(F.max_pool2d(core, 2 * grow + 1, stride=1, padding=grow), grow)
|
||||
|
||||
|
||||
def fit_latent(latent: torch.Tensor, height: int, width: int) -> torch.Tensor | None:
|
||||
"""The original's latent on the result's grid, or None when they are different pictures."""
|
||||
h, w = latent.shape[-2:]
|
||||
if (h, w) == (height, width):
|
||||
return latent
|
||||
if abs(h / height - 1.0) > SIZE_SLACK or abs(w / width - 1.0) > SIZE_SLACK:
|
||||
return None
|
||||
return F.interpolate(latent, size=(height, width), mode="bilinear", align_corners=False)
|
||||
|
||||
|
||||
class EditLock:
|
||||
"""One lock: the original, the settings, and what the last run kept."""
|
||||
|
||||
def __init__(self, mode: str = "auto", tolerance: float = 1.0):
|
||||
self.mode = mode
|
||||
self.tolerance = tolerance
|
||||
self.source: torch.Tensor | None = None # IMAGE, [1, H, W, 3]
|
||||
self.latent: torch.Tensor | None = None # [1, C, H, W] or [1, C, 1, H, W], model space
|
||||
self.user_mask: torch.Tensor | None = None # MASK, white = the edit may happen
|
||||
self.status = "waiting"
|
||||
self.mask: torch.Tensor | None = None # [B, 1, h, w], 1 = changed, soft-edged
|
||||
self.core: torch.Tensor | None = None # the same before its margin, hard-edged
|
||||
self.quiet: torch.Tensor | None = None # [B], the quiet level at the decision
|
||||
self.broad: float | None = None # the first look's whole-picture reading
|
||||
self.wide: float | None = None # share of the picture the lock would free, at the decision
|
||||
self.top: float | None = None # the strongest spot then, in units of the quiet level
|
||||
self.peak: float | None = None # and the strongest hundredth
|
||||
self._sigma: float | None = None
|
||||
self._first = True
|
||||
self._first_sigma = 1.0
|
||||
self._decided = False
|
||||
|
||||
def _begin(self) -> None:
|
||||
self.status = "waiting"
|
||||
self.mask = None
|
||||
self.core = None
|
||||
self.quiet = None
|
||||
self.broad = None
|
||||
self.wide = None
|
||||
self.top = None
|
||||
self.peak = None
|
||||
self._first = True
|
||||
self._decided = False
|
||||
|
||||
def _decides_now(self, args: dict, sigma: float) -> bool:
|
||||
"""True once the noise is down far enough, or this is the run's last step."""
|
||||
steps = (args.get("model_options") or {}).get("transformer_options", {}).get("sample_sigmas")
|
||||
if steps is None or len(steps) < 2:
|
||||
return True # the sampler does not say where the run ends: decide at once
|
||||
if sigma <= float(steps[-2]) * 1.0001:
|
||||
return True
|
||||
return sigma <= DECIDE_AT * self._first_sigma
|
||||
|
||||
def step(self, args: dict) -> torch.Tensor:
|
||||
"""ComfyUI's post-CFG hook: the prediction comes in, the locked prediction goes out."""
|
||||
pred = args["denoised"]
|
||||
sigma = float(args["sigma"].flatten()[0])
|
||||
if self._sigma is None or sigma >= self._sigma:
|
||||
self._begin() # the noise level went up: a new run
|
||||
self._sigma = sigma
|
||||
if self.status in ("whole picture", "size", "video", "vae") or self.latent is None:
|
||||
return pred
|
||||
video = pred.ndim == 5
|
||||
if video and pred.shape[2] != 1:
|
||||
self.status = "video"
|
||||
return pred
|
||||
flat = pred[:, :, 0] if video else pred
|
||||
latent = self.latent[:, :, 0] if self.latent.ndim == 5 else self.latent
|
||||
if latent.shape[1] != flat.shape[1]:
|
||||
self.status = "vae"
|
||||
return pred
|
||||
source = fit_latent(latent.to(device=flat.device, dtype=torch.float32), *flat.shape[-2:])
|
||||
if source is None:
|
||||
self.status = "size"
|
||||
return pred
|
||||
if self.status == "soft":
|
||||
# nothing is put back, the picture is only held where the original sits
|
||||
back = moved_back(flat, redraws(flat, source))
|
||||
return back[:, :, None] if video else back
|
||||
if self.user_mask is not None:
|
||||
mask = self._given_mask(flat)
|
||||
else:
|
||||
if self._first:
|
||||
self._first_sigma = sigma
|
||||
self.broad = float(broad_change(flat, source).max())
|
||||
if self.mode == "auto" and self.broad > WHOLE_PICTURE:
|
||||
self.status = "whole picture"
|
||||
return pred
|
||||
moves = redraws(flat, source)
|
||||
magnitude = difference(flat, source, moves)
|
||||
if not self._decided and not self._decides_now(args, sigma):
|
||||
# still watching: the model draws its own picture, only its zoom and shift are undone
|
||||
self._first = False
|
||||
self.status = "watching"
|
||||
if self.broad > WHOLE_PICTURE:
|
||||
return pred # too much is changing to trust the measured move
|
||||
back = moved_back(flat, moves)
|
||||
return back[:, :, None] if video else back
|
||||
if not self._decided:
|
||||
self._decided = True
|
||||
# the yardstick is set once, here: later steps are read against it
|
||||
self.quiet = quiet_level(magnitude)
|
||||
self.top, self.peak = peaks(magnitude, self.quiet)
|
||||
self.wide = float(change_core(magnitude, self.quiet, self.tolerance).flatten(1).mean(dim=1).max())
|
||||
thin = self.top < SOFT_PEAK or (self.wide >= SOFT_WIDE and self.peak < SOFT_PEAK)
|
||||
if self.mode == "auto" and not self._first and thin:
|
||||
# a soft change over the whole picture: a lock would hold it back or let it through in patches
|
||||
self.status = "soft"
|
||||
back = moved_back(flat, moves)
|
||||
return back[:, :, None] if video else back
|
||||
held = self.core if self.core is not None and self.core.shape[0] == flat.shape[0] else None
|
||||
self.core = change_core(magnitude, self.quiet.to(magnitude.device), self.tolerance,
|
||||
held.to(magnitude.device) if held is not None else None).detach()
|
||||
mask = soften(self.core)
|
||||
self._first = False
|
||||
self.mask = mask.detach()
|
||||
self.status = "locked"
|
||||
keep = (1.0 - mask).to(flat.dtype)
|
||||
locked = flat * (1.0 - keep) + source.to(flat.dtype) * keep
|
||||
return locked[:, :, None] if video else locked
|
||||
|
||||
def _given_mask(self, flat: torch.Tensor) -> torch.Tensor:
|
||||
mask = self.user_mask
|
||||
if mask.ndim == 2:
|
||||
mask = mask[None]
|
||||
mask = mask[:1, None].to(device=flat.device, dtype=torch.float32)
|
||||
mask = F.interpolate(mask, size=flat.shape[-2:], mode="bilinear", align_corners=False).clamp(0.0, 1.0)
|
||||
return soften(mask).expand(flat.shape[0], -1, -1, -1)
|
||||
|
||||
def kept_share(self) -> float | None:
|
||||
if self.status != "locked" or self.mask is None:
|
||||
return None
|
||||
return float((self.mask < 0.5).float().mean())
|
||||
|
||||
def pixel_mask(self, height: int, width: int) -> torch.Tensor | None:
|
||||
"""The change mask at the picture's size, [B, H, W], or None when nothing was locked."""
|
||||
if self.status != "locked" or self.mask is None:
|
||||
return None
|
||||
mask = F.interpolate(self.mask.float().cpu(), size=(height, width), mode="bilinear", align_corners=False)
|
||||
return mask[:, 0].clamp(0.0, 1.0)
|
||||
|
||||
|
||||
def restore_pixels(result: torch.Tensor, source: torch.Tensor, mask: torch.Tensor,
|
||||
feather: int) -> tuple[torch.Tensor, torch.Tensor]:
|
||||
"""The original's own pixels where the mask is black. result [B, H, W, C], source [1, h, w, 3], mask [b, H, W].
|
||||
|
||||
The fade lies outside the edit: the mask is widened by the feather first.
|
||||
"""
|
||||
frames, height, width = int(result.shape[0]), int(result.shape[1]), int(result.shape[2])
|
||||
edit = mask[:, None].float()
|
||||
if edit.shape[0] not in (1, frames):
|
||||
edit = edit[:1]
|
||||
if feather > 0:
|
||||
size = 2 * feather + 1
|
||||
edit = F.max_pool2d(edit, size, stride=1, padding=feather)
|
||||
edit = box_blur(edit, feather)
|
||||
original = source[..., :3].float()
|
||||
if tuple(original.shape[1:3]) != (height, width):
|
||||
original = F.interpolate(original.movedim(-1, 1), size=(height, width), mode="bicubic",
|
||||
align_corners=False, antialias=True).movedim(1, -1).clamp(0.0, 1.0)
|
||||
weight = edit.movedim(1, -1).to(result.device, result.dtype)
|
||||
out = result.clone()
|
||||
out[..., :3] = result[..., :3] * weight + original.to(result.device, result.dtype) * (1.0 - weight)
|
||||
edit_mask = edit[:, 0].expand(frames, -1, -1).clone()
|
||||
return out, edit_mask
|
||||
|
||||
|
||||
def describe(lock: EditLock) -> str:
|
||||
if lock.status == "locked":
|
||||
kept = lock.kept_share() or 0.0
|
||||
how = "your mask" if lock.user_mask is not None else "what the edit changed"
|
||||
text = (f"Edit Lock kept {kept * 100:.0f}% of your picture as it was and left "
|
||||
f"{(1.0 - kept) * 100:.0f}% to the edit ({how}).")
|
||||
if kept >= NEARLY_ALL and lock.user_mask is None:
|
||||
# rain, snow or smoke over the whole picture reads as a small edit and is held back
|
||||
text += (" If your edit is missing, it is one that spreads thinly over the whole picture "
|
||||
"(rain, snow, smoke): set lock to off.")
|
||||
return text
|
||||
if lock.status == "whole picture":
|
||||
return ("Edit Lock stepped aside: the edit changes the whole picture, so there was nothing to keep. "
|
||||
"Set lock to always to lock anyway.")
|
||||
if lock.status == "soft":
|
||||
return ("Edit Lock stepped aside: the edit spreads thinly over the whole picture (rain, snow, haze), "
|
||||
"so no part of it stayed as it was. It only held the picture in place. "
|
||||
"Set lock to always to lock anyway.")
|
||||
if lock.status == "off":
|
||||
return "Edit Lock is off."
|
||||
if lock.status == "size":
|
||||
return ("Edit Lock did nothing: the picture and the result are different sizes. "
|
||||
"Give it the picture at the size of the result.")
|
||||
if lock.status == "video":
|
||||
return "Edit Lock did nothing: it works on single pictures, not video."
|
||||
if lock.status == "vae":
|
||||
return "Edit Lock did nothing: this VAE does not belong to this model."
|
||||
return "Edit Lock has not run yet: plug its model output into the sampler."
|
||||
@@ -41,6 +41,16 @@ RADIUS_MAX = 4096.0
|
||||
# The face's own picture: the long side is at most this.
|
||||
SMALL_SIDE = 1024
|
||||
|
||||
# SAM 3 is asked about a copy of a big picture: its long side is at most
|
||||
# this. ComfyUI's SAM 3 reads every picture at 1008 pixels and returns each
|
||||
# answer at the size of the picture it was given, on the graphics card. At
|
||||
# 8192 pixels one word with sixteen matches needed over 27 GB there, and a
|
||||
# look at 3840 took twice as long as at 2048, all for outlines the model
|
||||
# never saw that finely. Twice what the model reads keeps its second pass
|
||||
# (a close-up of each thing it found) as sharp as before on pictures up to
|
||||
# this size, which come out exactly as they did.
|
||||
WORK_SIDE = 2048
|
||||
|
||||
# How a find stroke is asked (find_points, choose_found). Measured on 27
|
||||
# strokes over three pictures, each drawn 13 slightly different ways: four
|
||||
# points alone returned the stroked thing in 87 % of 351 tries, two points
|
||||
@@ -386,6 +396,73 @@ def read_ask(ask, height: int, width: int) -> tuple:
|
||||
return "look", None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The copy of a big picture that SAM 3 is asked about
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class WorkView:
|
||||
"""One picture as SAM 3 is asked about it.
|
||||
|
||||
A picture whose long side is over WORK_SIDE is asked about on a smaller
|
||||
copy (`frame`, averaged down, so nothing shimmers), and what SAM 3
|
||||
answers there is brought back to the picture's own size with `full`.
|
||||
A smaller picture is its own copy: `same` is true and nothing changes.
|
||||
|
||||
Touches and pieces keep their places in pixels of the picture. `point`
|
||||
gives a place on the copy, `full_point` a place on the picture that
|
||||
`point` leads back to the same pixel of the copy.
|
||||
"""
|
||||
|
||||
def __init__(self, frame: torch.Tensor):
|
||||
self.height, self.width = int(frame.shape[1]), int(frame.shape[2])
|
||||
long_side = max(self.height, self.width)
|
||||
self.same = long_side <= WORK_SIDE
|
||||
if self.same:
|
||||
self.frame, self.work_height, self.work_width = frame, self.height, self.width
|
||||
return
|
||||
scale = WORK_SIDE / long_side
|
||||
self.work_height = max(1, int(math.floor(self.height * scale + 0.5)))
|
||||
self.work_width = max(1, int(math.floor(self.width * scale + 0.5)))
|
||||
small = F.interpolate(
|
||||
frame.detach().float().cpu().movedim(-1, 1), size=(self.work_height, self.work_width), mode="area",
|
||||
)
|
||||
self.frame = small.movedim(1, -1).clamp(0.0, 1.0).contiguous()
|
||||
|
||||
def point(self, point) -> tuple:
|
||||
x = min(self.work_width - 1, max(0, int(point[0]) * self.work_width // self.width))
|
||||
y = min(self.work_height - 1, max(0, int(point[1]) * self.work_height // self.height))
|
||||
return (x, y)
|
||||
|
||||
def full_point(self, point) -> tuple:
|
||||
x = min(self.width - 1, -(-int(point[0]) * self.width // self.work_width))
|
||||
y = min(self.height - 1, -(-int(point[1]) * self.height // self.work_height))
|
||||
return (x, y)
|
||||
|
||||
def points(self, points) -> tuple:
|
||||
return tuple(self.point(point) for point in points)
|
||||
|
||||
def length(self, value: float) -> float:
|
||||
"""A length of the picture (a brush's radius) on the copy."""
|
||||
return float(value) * self.work_width / self.width
|
||||
|
||||
def full_box(self, box) -> list:
|
||||
if self.same:
|
||||
return list(box)
|
||||
x0, y0, x1, y1 = (int(value) for value in box)
|
||||
return [
|
||||
x0 * self.width // self.work_width, y0 * self.height // self.work_height,
|
||||
min(self.width, -(-x1 * self.width // self.work_width)), min(self.height, -(-y1 * self.height // self.work_height)),
|
||||
]
|
||||
|
||||
def full(self, mask: torch.Tensor) -> torch.Tensor:
|
||||
"""An answer from the copy at the picture's own size (bool)."""
|
||||
if self.same or tuple(mask.shape) == (self.height, self.width):
|
||||
return mask
|
||||
soft = F.interpolate(mask[None, None].float(), size=(self.height, self.width), mode="bilinear", align_corners=False)
|
||||
return soft[0, 0] >= 0.5
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Strokes
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
"""Only one copy of the pack loads.
|
||||
|
||||
A Manager install lands in ``custom_nodes/ausboss-nodes`` and a git clone in
|
||||
``custom_nodes/ComfyUI-AusBoss``. With both in place every route is added
|
||||
twice and ComfyUI stops while starting. So each copy looks for the others
|
||||
and all of them agree on the one that loads; the rest step aside and say
|
||||
which folder to delete.
|
||||
|
||||
A copy from before this file existed cannot step aside, so it always loads
|
||||
and a newer copy yields to it. Among copies that can, the newest version
|
||||
loads.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import tomllib
|
||||
|
||||
PACK_NAME = "ausboss-nodes"
|
||||
# A copy that has this file knows how to step aside.
|
||||
_GUARD_FILE = os.path.join("nodes", "_single_copy.py")
|
||||
|
||||
|
||||
def _read_pack(folder: str) -> dict | None:
|
||||
"""Name the pack copy in ``folder``, or None when it is another pack."""
|
||||
try:
|
||||
with open(os.path.join(folder, "pyproject.toml"), "rb") as f:
|
||||
project = tomllib.load(f).get("project", {})
|
||||
except Exception:
|
||||
return None
|
||||
if project.get("name") != PACK_NAME:
|
||||
return None
|
||||
if not os.path.isfile(os.path.join(folder, "__init__.py")):
|
||||
return None
|
||||
version = project.get("version")
|
||||
return {
|
||||
"folder": folder,
|
||||
"version": version if isinstance(version, str) else "unknown",
|
||||
"guarded": os.path.isfile(os.path.join(folder, _GUARD_FILE)),
|
||||
}
|
||||
|
||||
|
||||
def _version_key(version: str) -> tuple:
|
||||
parts = []
|
||||
for part in version.split("."):
|
||||
digits = "".join(ch for ch in part if ch.isdigit())
|
||||
parts.append(int(digits) if digits else -1)
|
||||
return tuple(parts)
|
||||
|
||||
|
||||
def find_copies(own_folder: str, roots: list[str]) -> list[dict]:
|
||||
"""Every copy of the pack below ``roots``, this one included."""
|
||||
own = os.path.realpath(own_folder)
|
||||
seen = {own}
|
||||
copies = [_read_pack(own) or {"folder": own, "version": "unknown", "guarded": True}]
|
||||
for root in roots:
|
||||
try:
|
||||
names = sorted(os.listdir(root))
|
||||
except OSError:
|
||||
continue
|
||||
for name in names:
|
||||
# ComfyUI skips these too.
|
||||
if name.endswith(".disabled") or name.startswith("."):
|
||||
continue
|
||||
folder = os.path.realpath(os.path.join(root, name))
|
||||
if folder in seen or not os.path.isdir(folder):
|
||||
continue
|
||||
seen.add(folder)
|
||||
pack = _read_pack(folder)
|
||||
if pack:
|
||||
copies.append(pack)
|
||||
return copies
|
||||
|
||||
|
||||
def pick_copy(copies: list[dict]) -> dict:
|
||||
"""The one copy that loads. Every copy reaches the same answer."""
|
||||
unguarded = [c for c in copies if not c["guarded"]]
|
||||
pool = unguarded or copies
|
||||
return max(pool, key=lambda c: (_version_key(c["version"]), c["folder"]))
|
||||
|
||||
|
||||
def other_copy_that_loads(own_folder: str, roots: list[str] | None = None) -> dict | None:
|
||||
"""The copy this one steps aside for, or None when this one loads."""
|
||||
if roots is None:
|
||||
# Outside ComfyUI nothing is served, so there is nothing to clash.
|
||||
try:
|
||||
import folder_paths
|
||||
|
||||
roots = list(folder_paths.get_folder_paths("custom_nodes"))
|
||||
except Exception:
|
||||
return None
|
||||
roots = list(dict.fromkeys(os.path.realpath(r) for r in roots))
|
||||
copies = find_copies(own_folder, roots)
|
||||
if len(copies) < 2:
|
||||
return None
|
||||
chosen = pick_copy(copies)
|
||||
return None if chosen["folder"] == os.path.realpath(own_folder) else chosen
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Cut a block of text into the items Text Lines runs one at a time.
|
||||
|
||||
Pure text handling with no ComfyUI imports, so it runs under the
|
||||
standard-library tests.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
SPLIT_EACH_LINE = "each line"
|
||||
SPLIT_BLANK_LINE = "blank line"
|
||||
SPLIT_MODES = (SPLIT_EACH_LINE, SPLIT_BLANK_LINE)
|
||||
|
||||
# The words a list writer puts before an item's number: "Prompt 3:", "Shot 2 -".
|
||||
# Not "image" or "picture": an edit prompt may open with "Image 1: ...".
|
||||
_LABEL = r"(?:prompt|shot|scene|line|step|item)"
|
||||
|
||||
# A marker at the start of an item that only counts or bullets it. A bare
|
||||
# number needs punctuation after it, and that punctuation must not be followed
|
||||
# by a digit, so "2 women on a beach", "1.5x zoom" and "10:30 at night" stay
|
||||
# whole. A bullet needs a space after it, so "*bold* text" stays whole.
|
||||
_MARKER = re.compile(
|
||||
r"""^\s*(?:
|
||||
[\[【(]\s*(?:%(label)s[\s_#-]*)?\d+\s*[\]】)]\s*[:.\-–—]? # [prompt 1] (2)
|
||||
| %(label)s[\s_#-]*\d+\s*[:.)\-–—] # Prompt 1: Shot 2 -
|
||||
| \#?\d+\s*[.):](?!\d) # 1. 2) #3:
|
||||
| [-*•·–—](?=\s|$) # - bullet
|
||||
)\s*"""
|
||||
% {"label": _LABEL},
|
||||
re.IGNORECASE | re.VERBOSE,
|
||||
)
|
||||
|
||||
_BLANK_LINE = re.compile(r"\n[ \t]*(?:\n[ \t]*)+")
|
||||
|
||||
|
||||
def strip_marker(item: str) -> str:
|
||||
"""Take one leading number or bullet off an item."""
|
||||
return _MARKER.sub("", item, count=1).strip()
|
||||
|
||||
|
||||
def split_items(text, split=SPLIT_EACH_LINE, remove_numbering=True, limit=0) -> list[str]:
|
||||
"""The items of `text`, in order, with empty ones left out.
|
||||
|
||||
`split` picks where one item ends: at every line break, or only at an
|
||||
empty line (so an item can hold several lines). `remove_numbering` takes
|
||||
a leading "1." or "-" off each item. `limit` keeps the first so many;
|
||||
0 keeps them all.
|
||||
"""
|
||||
text = str(text or "").replace("\r\n", "\n").replace("\r", "\n")
|
||||
if split == SPLIT_BLANK_LINE:
|
||||
parts = _BLANK_LINE.split(text)
|
||||
else:
|
||||
parts = text.split("\n")
|
||||
|
||||
items = []
|
||||
for part in parts:
|
||||
item = part.strip()
|
||||
if remove_numbering:
|
||||
item = strip_marker(item)
|
||||
if item:
|
||||
items.append(item)
|
||||
|
||||
limit = int(limit or 0)
|
||||
if limit > 0:
|
||||
items = items[:limit]
|
||||
return items
|
||||
@@ -0,0 +1,213 @@
|
||||
"""Edit Lock 🆎 and Edit Lock Restore 🆎: an edit leaves the rest of the picture alone.
|
||||
|
||||
An edit model redraws the whole picture, so the parts nobody asked to change
|
||||
come back a little moved and a little tinted. Edit Lock watches the model's
|
||||
own prediction at every step, reads where it differs from the original, and
|
||||
puts the original back everywhere else, so the model finishes the edit on
|
||||
top of the real picture. Edit Lock Restore then swaps the original's own
|
||||
pixels in for those parts after the decode. The deciding and blending live
|
||||
in _edit_lock_helpers.py.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import torch
|
||||
|
||||
from ._edit_lock_helpers import (
|
||||
LOCK_MODES,
|
||||
EditLock,
|
||||
describe,
|
||||
restore_pixels,
|
||||
)
|
||||
|
||||
|
||||
class AusBossEditLock:
|
||||
CATEGORY = "🆎 AusBoss/Image"
|
||||
DESCRIPTION = (
|
||||
"Keeps every part of your picture that an edit did not change "
|
||||
"exactly where it was, with the colors it had. Edit models redraw "
|
||||
"the whole picture, so untouched parts often come back slightly "
|
||||
"zoomed, shifted or tinted. Put this between the model and the "
|
||||
"sampler and give it the picture you are editing. It needs no "
|
||||
"mask: it sees what the edit changes while the picture is made. "
|
||||
"When the edit changes the whole picture (a new style, new light, "
|
||||
"new weather) it steps aside. Add Edit Lock Restore 🆎 after the "
|
||||
"decode to get your own pixels back in the untouched parts."
|
||||
)
|
||||
SEARCH_ALIASES = [
|
||||
"edit lock",
|
||||
"pixel drift",
|
||||
"pixel shift",
|
||||
"color shift",
|
||||
"consistency",
|
||||
"keep unchanged",
|
||||
"preserve original",
|
||||
"tail steer",
|
||||
"no drift edit",
|
||||
"ausboss",
|
||||
]
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"model": (
|
||||
"MODEL",
|
||||
{"tooltip": "The edit model. Patch last, after any LoRAs."},
|
||||
),
|
||||
"vae": (
|
||||
"VAE",
|
||||
{"tooltip": "The same VAE the workflow decodes with."},
|
||||
),
|
||||
"image": (
|
||||
"IMAGE",
|
||||
{
|
||||
"tooltip": (
|
||||
"The picture you are editing, at the size of the "
|
||||
"result. Use the same picture the edit model gets."
|
||||
),
|
||||
},
|
||||
),
|
||||
"lock": (
|
||||
list(LOCK_MODES),
|
||||
{
|
||||
"default": LOCK_MODES[0],
|
||||
"tooltip": (
|
||||
"auto locks the untouched parts and steps aside "
|
||||
"when the whole picture changes. always locks "
|
||||
"even then. off does nothing."
|
||||
),
|
||||
},
|
||||
),
|
||||
"tolerance": (
|
||||
"FLOAT",
|
||||
{
|
||||
"default": 1.0,
|
||||
"min": 0.25,
|
||||
"max": 4.0,
|
||||
"step": 0.05,
|
||||
"tooltip": (
|
||||
"How big a change must be to count as part of the "
|
||||
"edit. Raise it if the lock lets small unwanted "
|
||||
"changes through; lower it if it holds back part "
|
||||
"of your edit."
|
||||
),
|
||||
},
|
||||
),
|
||||
},
|
||||
"optional": {
|
||||
"mask": (
|
||||
"MASK",
|
||||
{
|
||||
"tooltip": (
|
||||
"Optional. White marks where the edit may happen; "
|
||||
"everything else is locked. With a mask the node "
|
||||
"does not look for the change itself."
|
||||
),
|
||||
},
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("MODEL", "AUSBOSS_EDIT_LOCK")
|
||||
RETURN_NAMES = ("model", "edit_lock")
|
||||
OUTPUT_TOOLTIPS = (
|
||||
"The model with the lock on. Plug it into the sampler.",
|
||||
"What the lock kept. Plug it into Edit Lock Restore 🆎 after the decode.",
|
||||
)
|
||||
FUNCTION = "patch"
|
||||
|
||||
def patch(self, model, vae, image, lock, tolerance, mask=None):
|
||||
if image.ndim != 4:
|
||||
raise ValueError("Edit Lock: image must be an IMAGE batch.")
|
||||
state = EditLock(mode=str(lock), tolerance=float(tolerance))
|
||||
source = image[:1, ..., :3]
|
||||
state.source = source.detach().float().cpu()
|
||||
if mask is not None:
|
||||
state.user_mask = mask.detach().float().cpu()
|
||||
if state.mode == "off":
|
||||
state.status = "off"
|
||||
return (model, state)
|
||||
latent = vae.encode(source)
|
||||
patched = model.clone()
|
||||
state.latent = patched.model.process_latent_in(latent).detach().float().cpu()
|
||||
patched.set_model_sampler_post_cfg_function(state.step)
|
||||
return (patched, state)
|
||||
|
||||
|
||||
class AusBossEditLockRestore:
|
||||
CATEGORY = "🆎 AusBoss/Image"
|
||||
DESCRIPTION = (
|
||||
"Puts your picture's own pixels back in every part the edit did not "
|
||||
"change. Use it right after the VAE decode, with the edit_lock "
|
||||
"output of Edit Lock 🆎. The untouched parts then match your "
|
||||
"original pixel for pixel; only the edit is new. When the lock "
|
||||
"stepped aside the picture passes through unchanged."
|
||||
)
|
||||
SEARCH_ALIASES = [
|
||||
"edit lock restore",
|
||||
"restore original pixels",
|
||||
"paste back",
|
||||
"keep unchanged",
|
||||
"ausboss",
|
||||
]
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"image": (
|
||||
"IMAGE",
|
||||
{"tooltip": "The edited picture, straight from the VAE decode."},
|
||||
),
|
||||
"edit_lock": (
|
||||
"AUSBOSS_EDIT_LOCK",
|
||||
{"tooltip": "From Edit Lock 🆎."},
|
||||
),
|
||||
"feather": (
|
||||
"INT",
|
||||
{
|
||||
"default": 16,
|
||||
"min": 0,
|
||||
"max": 256,
|
||||
"step": 1,
|
||||
"tooltip": (
|
||||
"How softly your original fades into the edit, in "
|
||||
"pixels. The fade stays outside the edit."
|
||||
),
|
||||
},
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("IMAGE", "MASK", "STRING")
|
||||
RETURN_NAMES = ("image", "edit_mask", "report")
|
||||
OUTPUT_TOOLTIPS = (
|
||||
"The edit with your original pixels in the untouched parts.",
|
||||
"White where the edit changed the picture.",
|
||||
"How much of the picture was kept. Plug it into Show Text 🆎 to read it.",
|
||||
)
|
||||
FUNCTION = "restore"
|
||||
|
||||
def restore(self, image, edit_lock, feather):
|
||||
if image.ndim != 4:
|
||||
raise ValueError("Edit Lock Restore: image must be an IMAGE batch.")
|
||||
if not isinstance(edit_lock, EditLock):
|
||||
raise ValueError("Edit Lock Restore: edit_lock needs the output of Edit Lock.")
|
||||
height, width = int(image.shape[1]), int(image.shape[2])
|
||||
mask = edit_lock.pixel_mask(height, width)
|
||||
if mask is None or edit_lock.source is None:
|
||||
return (image, torch.ones((int(image.shape[0]), height, width)), describe(edit_lock))
|
||||
result, edit_mask = restore_pixels(image, edit_lock.source, mask, int(feather))
|
||||
return (result, edit_mask, describe(edit_lock))
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"AUSBOSS_NODES_EditLock": AusBossEditLock,
|
||||
"AUSBOSS_NODES_EditLockRestore": AusBossEditLockRestore,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"AUSBOSS_NODES_EditLock": "Edit Lock 🆎",
|
||||
"AUSBOSS_NODES_EditLockRestore": "Edit Lock Restore 🆎",
|
||||
}
|
||||
+65
-30
@@ -42,6 +42,7 @@ from ._mask_by_touch_helpers import (
|
||||
WORDS_MORE,
|
||||
Memory,
|
||||
PieceSieve,
|
||||
WorkView,
|
||||
apply_item,
|
||||
box_of,
|
||||
choose_found,
|
||||
@@ -126,6 +127,8 @@ QUICK = True
|
||||
|
||||
# Shared by every Mask by Touch node: what SAM 3 answered, per picture.
|
||||
_MEMORY = Memory()
|
||||
# How many of those smaller copies a node keeps: this picture's and the last one's.
|
||||
VIEWS_KEPT = 2
|
||||
# What the text reader made of each word, per text reader. A word reads the
|
||||
# same for every picture, so it is read once.
|
||||
_WORDS = OrderedDict()
|
||||
@@ -836,25 +839,44 @@ class AusBossMaskByTouch:
|
||||
# What Fine edges is set to in the run at hand: the subject is asked
|
||||
# of that model.
|
||||
self._edge_choice = FINE_OFF
|
||||
# The smaller copies of big pictures SAM 3 is asked about, newest last.
|
||||
self._views = OrderedDict()
|
||||
|
||||
# -- what SAM 3 finds ---------------------------------------------------
|
||||
|
||||
def _tap(self, session: _Session, frame, frame_key: str, point):
|
||||
"""The thing at one point, without its stray specks. None for nothing.
|
||||
def _view(self, frame, frame_key: str) -> WorkView:
|
||||
"""The picture as SAM 3 is asked about it: itself, or a smaller copy
|
||||
of a big one, kept for the next run on the same picture."""
|
||||
held = self._views.get(frame_key)
|
||||
if held is not None and (held.height, held.width) == (int(frame.shape[1]), int(frame.shape[2])):
|
||||
self._views.move_to_end(frame_key)
|
||||
return held
|
||||
view = WorkView(frame)
|
||||
if not view.same:
|
||||
self._views[frame_key] = view
|
||||
while len(self._views) > VIEWS_KEPT:
|
||||
self._views.popitem(last=False)
|
||||
return view
|
||||
|
||||
def _tap(self, session: _Session, view: WorkView, frame_key: str, point):
|
||||
"""The thing at one point, without its stray specks, at the size SAM 3
|
||||
was asked at (`view`). None for nothing. `point` is a pixel of the
|
||||
picture.
|
||||
|
||||
Hovering's pieces and taps share this, and what it returned is
|
||||
remembered, so a tap on a piece that was shown gives that piece."""
|
||||
key = (session.token, frame_key, ("tap", int(point[0]), int(point[1])))
|
||||
found, answer = _MEMORY.get(key)
|
||||
if not found:
|
||||
answer = drop_specks(session.ask(frame, [point]), point)
|
||||
spot = view.point(point)
|
||||
answer = drop_specks(session.ask(view.frame, [spot]), spot)
|
||||
answer = answer if bool(answer.any()) else None
|
||||
_MEMORY.put(key, answer)
|
||||
return answer
|
||||
|
||||
def _named(self, session: _Session, frame, frame_key: str, word: str) -> list:
|
||||
def _named(self, session: _Session, view: WorkView, frame_key: str, word: str) -> list:
|
||||
"""Everything in one picture that SAM 3 calls by a word, tidied:
|
||||
[(H x W bool, how sure), ...].
|
||||
[(bool at the size SAM 3 was asked at, how sure), ...].
|
||||
|
||||
Hovering's named pieces and `name` touches share this, and what it
|
||||
returned is remembered, so a touch on a named piece that was shown
|
||||
@@ -862,7 +884,7 @@ class AusBossMaskByTouch:
|
||||
key = (session.token, session.reader, frame_key, ("word", word))
|
||||
found, matches = _MEMORY.get_matches(key)
|
||||
if not found:
|
||||
matches = tidy_matches(*session.ask_word(frame, word))
|
||||
matches = tidy_matches(*session.ask_word(view.frame, word))
|
||||
_MEMORY.put_matches(key, matches)
|
||||
return matches
|
||||
|
||||
@@ -873,6 +895,9 @@ class AusBossMaskByTouch:
|
||||
The area of a `paint` touch made with a soft brush is float, 0..1.
|
||||
"""
|
||||
height, width = int(frame.shape[1]), int(frame.shape[2])
|
||||
# What SAM 3 is asked comes back at the size it was asked at, and is
|
||||
# brought to the picture's own size here.
|
||||
view = self._view(frame, frame_key)
|
||||
if item.kind in ("subject", "background"):
|
||||
# The picture's main subject, as ComfyUI's background-removal
|
||||
# model sees it, or everything but it.
|
||||
@@ -881,25 +906,25 @@ class AusBossMaskByTouch:
|
||||
return (area, "thing", ()) if bool(area.any()) else (None, "none", ())
|
||||
if item.kind == "all":
|
||||
# Every thing called by that word, joined.
|
||||
matches = self._named(session, frame, frame_key, item.word) if item.word else []
|
||||
matches = self._named(session, view, frame_key, item.word) if item.word else []
|
||||
if not matches:
|
||||
return None, "none", ()
|
||||
area = torch.zeros(height, width, dtype=torch.bool)
|
||||
area = torch.zeros(view.work_height, view.work_width, dtype=torch.bool)
|
||||
for mask, _ in matches:
|
||||
area |= mask
|
||||
return area, "thing", tuple(range(len(matches)))
|
||||
return view.full(area), "thing", tuple(range(len(matches)))
|
||||
if item.kind in ("tap", "name"):
|
||||
if not (0 <= item.x < width and 0 <= item.y < height):
|
||||
return None, "none", ()
|
||||
if item.kind == "tap":
|
||||
answer = self._tap(session, frame, frame_key, (item.x, item.y))
|
||||
return (answer, "thing", ()) if answer is not None else (None, "none", ())
|
||||
answer = self._tap(session, view, frame_key, (item.x, item.y))
|
||||
return (view.full(answer), "thing", ()) if answer is not None else (None, "none", ())
|
||||
if not item.word:
|
||||
return None, "none", ()
|
||||
# Of the things called by that word, the smallest at the point.
|
||||
matches = self._named(session, frame, frame_key, item.word)
|
||||
index = match_under(matches, (item.x, item.y))
|
||||
return (None, "none", ()) if index is None else (matches[index][0], "thing", (index,))
|
||||
matches = self._named(session, view, frame_key, item.word)
|
||||
index = match_under(matches, view.point((item.x, item.y)))
|
||||
return (None, "none", ()) if index is None else (view.full(matches[index][0]), "thing", (index,))
|
||||
if item.kind not in ("find", "paint"):
|
||||
return None, "none", ()
|
||||
if item.kind == "paint" and item.soft > 0:
|
||||
@@ -918,14 +943,18 @@ class AusBossMaskByTouch:
|
||||
found, answer = _MEMORY.get(key)
|
||||
if not found:
|
||||
many, pair = find_points(item.pts, height, width)
|
||||
# The stroke as it lies on the copy SAM 3 is asked about.
|
||||
under = stroke if view.same else stroke_area(
|
||||
view.points(item.pts), max(1.0, view.length(item.r)), view.work_height, view.work_width,
|
||||
)
|
||||
answer = choose_found(
|
||||
session.ask(frame, many) if many else None,
|
||||
session.ask(frame, pair) if pair else None,
|
||||
stroke,
|
||||
session.ask(view.frame, view.points(many)) if many else None,
|
||||
session.ask(view.frame, view.points(pair)) if pair else None,
|
||||
under,
|
||||
)
|
||||
_MEMORY.put(key, answer)
|
||||
# Over empty space the paint is kept as it is.
|
||||
return (answer, "thing", ()) if answer is not None else (stroke, "paint", ())
|
||||
return (view.full(answer), "thing", ()) if answer is not None else (stroke, "paint", ())
|
||||
|
||||
def _look(self, session: _Session, frame, key: str, pieces: dict, pointer=None, tell=None) -> dict:
|
||||
"""Find a picture's pieces for hovering.
|
||||
@@ -943,6 +972,7 @@ class AusBossMaskByTouch:
|
||||
piece.
|
||||
"""
|
||||
height, width = int(frame.shape[1]), int(frame.shape[2])
|
||||
view = self._view(frame, key)
|
||||
queue = _scope_queue(width, height)
|
||||
sieve = pieces["sieve"]
|
||||
fresh = {} # (name or None, point) -> (area, is it tidy and remembered)
|
||||
@@ -967,7 +997,7 @@ class AusBossMaskByTouch:
|
||||
if pointer is not None:
|
||||
since = time.perf_counter()
|
||||
known = _MEMORY.has((session.token, key, ("tap", pointer[0], pointer[1])))
|
||||
answer = self._tap(session, frame, key, pointer)
|
||||
answer = self._tap(session, view, key, pointer)
|
||||
answered(known, since)
|
||||
if answer is not None and sieve.offer(pointer, answer, pointed=True):
|
||||
fresh[(None, pointer)] = (answer, True)
|
||||
@@ -993,8 +1023,10 @@ class AusBossMaskByTouch:
|
||||
break
|
||||
ahead.append(later)
|
||||
session.read_ahead(ahead, stop=_jobs_waiting)
|
||||
matches = self._named(session, frame, key, what)
|
||||
for (area, sure), point in zip(matches, match_points(matches)):
|
||||
matches = self._named(session, view, key, what)
|
||||
for (area, sure), spot in zip(matches, match_points(matches)):
|
||||
# The match's own point, as a pixel of the picture.
|
||||
point = None if spot is None else view.full_point(spot)
|
||||
if point is not None and sieve.offer(point, area, name=what, sure=sure):
|
||||
fresh[(what, point)] = (area, True)
|
||||
pieces["words"] += 1
|
||||
@@ -1002,7 +1034,7 @@ class AusBossMaskByTouch:
|
||||
memory_key = (session.token, key, ("tap", what[0], what[1]))
|
||||
known, answer = _MEMORY.get(memory_key)
|
||||
if not known:
|
||||
answer = session.ask(frame, [what])
|
||||
answer = session.ask(view.frame, [view.point(what)])
|
||||
if not bool(answer.any()):
|
||||
_MEMORY.put(memory_key, None)
|
||||
answer = None
|
||||
@@ -1025,6 +1057,7 @@ class AusBossMaskByTouch:
|
||||
first, and the small copies packed into planes. `fresh` holds the
|
||||
areas found since the last time, and is emptied."""
|
||||
height, width = int(frame.shape[1]), int(frame.shape[2])
|
||||
view = self._view(frame, key)
|
||||
small_width, small_height, _ = small_size(width, height)
|
||||
sieve = pieces["sieve"]
|
||||
before = {ident: slot for slot, ident in enumerate(pieces["ids"])}
|
||||
@@ -1036,18 +1069,19 @@ class AusBossMaskByTouch:
|
||||
else:
|
||||
# A piece kept by a look that was stopped half way is asked
|
||||
# for again: its answer was not held on to.
|
||||
answer, tidy = fresh.get(ident) or (self._piece(session, frame, key, name, point), True)
|
||||
answer, tidy = fresh.get(ident) or (self._piece(session, view, key, name, point), True)
|
||||
if answer is None:
|
||||
continue
|
||||
if not tidy:
|
||||
# What a tap at this point gives: the same answer, tidied
|
||||
# the same way, remembered so the tap asks nothing.
|
||||
answer = drop_specks(answer, point)
|
||||
answer = drop_specks(answer, view.point(point))
|
||||
_MEMORY.put((session.token, key, ("tap", point[0], point[1])), answer)
|
||||
sure = sieve.sure_of(name, point)
|
||||
entry = {
|
||||
"x": point[0], "y": point[1],
|
||||
"share": round(float(answer.sum()) / (height * width), 5), "box": box_of(answer),
|
||||
"share": round(float(answer.sum()) / (view.work_height * view.work_width), 5),
|
||||
"box": view.full_box(box_of(answer)),
|
||||
"kind": "point" if name is None else "name", "name": name,
|
||||
"sure": None if sure is None else round(sure, 3),
|
||||
}
|
||||
@@ -1064,13 +1098,13 @@ class AusBossMaskByTouch:
|
||||
pieces["list"] = [row[4] for row in rows]
|
||||
pieces["planes"] = pack_planes(masks)
|
||||
|
||||
def _piece(self, session: _Session, frame, key: str, name, point):
|
||||
def _piece(self, session: _Session, view: WorkView, key: str, name, point):
|
||||
"""The area of a piece from what is remembered: a point's answer, or
|
||||
the match of a word that the piece's own point stands for."""
|
||||
if name is None:
|
||||
return self._tap(session, frame, key, point)
|
||||
matches = self._named(session, frame, key, name)
|
||||
index = match_under(matches, point)
|
||||
return self._tap(session, view, key, point)
|
||||
matches = self._named(session, view, key, name)
|
||||
index = match_under(matches, view.point(point))
|
||||
return None if index is None else matches[index][0]
|
||||
|
||||
# -- files for the node's face -------------------------------------------
|
||||
@@ -1385,8 +1419,9 @@ class AusBossMaskByTouch:
|
||||
if asked == "scope" and (pieces is None or pointer is not None or pieces["next"] < pieces["total"]):
|
||||
if pieces is None:
|
||||
words = len(WORDS_FIRST) + len(WORDS_MORE)
|
||||
view = self._view(image[:1], key)
|
||||
pieces = {
|
||||
"sieve": PieceSieve(height, width), "next": 0, "total": len(_scope_queue(width, height)),
|
||||
"sieve": PieceSieve(view.work_height, view.work_width), "next": 0, "total": len(_scope_queue(width, height)),
|
||||
"words": 0, "words_total": words, "number": 0, "ids": [], "list": [], "planes": [],
|
||||
}
|
||||
# Remembered from the start: a look that is stopped half way
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
"""Text Lines 🆎 — run what comes after once for each line of text."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from ._text_lines_helpers import SPLIT_EACH_LINE, SPLIT_MODES, split_items
|
||||
|
||||
MAX_LIMIT = 1000
|
||||
|
||||
|
||||
class AusBossTextLines:
|
||||
CATEGORY = "🆎 AusBoss/Utility"
|
||||
DESCRIPTION = (
|
||||
"Runs what comes after once for each line of your text. Type five "
|
||||
"prompts, one on each line, and one run makes five pictures. Empty "
|
||||
"lines are skipped, and a number or bullet at the start of a line "
|
||||
"can be taken off."
|
||||
)
|
||||
SEARCH_ALIASES = [
|
||||
"prompt list",
|
||||
"one prompt per line",
|
||||
"each line",
|
||||
"split text",
|
||||
"text to list",
|
||||
"batch prompts",
|
||||
"queue prompts",
|
||||
"ausboss",
|
||||
]
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"text": (
|
||||
"STRING",
|
||||
{
|
||||
"default": "",
|
||||
"multiline": True,
|
||||
"tooltip": "One line for each run. Type it here, or wire in text another node wrote.",
|
||||
},
|
||||
),
|
||||
"split": (
|
||||
list(SPLIT_MODES),
|
||||
{
|
||||
"default": SPLIT_EACH_LINE,
|
||||
"tooltip": (
|
||||
"each line: every line is its own run. blank line: lines stay "
|
||||
"together, and an empty line starts the next run."
|
||||
),
|
||||
},
|
||||
),
|
||||
"remove_numbering": (
|
||||
"BOOLEAN",
|
||||
{
|
||||
"default": True,
|
||||
"tooltip": (
|
||||
'On: a number or bullet at the start is taken off, like "1.", "-" '
|
||||
'or "[prompt 2]". Off: every line stays as written.'
|
||||
),
|
||||
},
|
||||
),
|
||||
"limit": (
|
||||
"INT",
|
||||
{
|
||||
"default": 0,
|
||||
"min": 0,
|
||||
"max": MAX_LIMIT,
|
||||
"step": 1,
|
||||
"tooltip": "The most lines to run. 0 runs them all.",
|
||||
},
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING", "INT", "INT")
|
||||
RETURN_NAMES = ("line", "number", "count")
|
||||
OUTPUT_IS_LIST = (True, True, False)
|
||||
OUTPUT_TOOLTIPS = (
|
||||
"Each line, one at a time. What comes after runs once for each.",
|
||||
"The line's place in the list, starting at 1. Add it to a seed to give each run its own.",
|
||||
"How many lines run.",
|
||||
)
|
||||
FUNCTION = "split"
|
||||
|
||||
def split(self, text, split=SPLIT_EACH_LINE, remove_numbering=True, limit=0):
|
||||
items = split_items(text, split, remove_numbering, limit)
|
||||
if not items:
|
||||
raise ValueError(
|
||||
"Text Lines has no lines to run. Type at least one line, or check the node that writes its text."
|
||||
)
|
||||
return (items, list(range(1, len(items) + 1)), len(items))
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"AUSBOSS_NODES_TextLines": AusBossTextLines,
|
||||
}
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"AUSBOSS_NODES_TextLines": "Text Lines 🆎",
|
||||
}
|
||||
|
||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
|
||||
@@ -56,6 +56,8 @@ PUBLIC_NODE_IDS = {
|
||||
"AUSBOSS_NODES_ColorMatch",
|
||||
"AUSBOSS_NODES_Compare",
|
||||
"AUSBOSS_NODES_CropForInpaint",
|
||||
"AUSBOSS_NODES_EditLock",
|
||||
"AUSBOSS_NODES_EditLockRestore",
|
||||
"AUSBOSS_NODES_Float",
|
||||
"AUSBOSS_NODES_FrameInterpolate",
|
||||
"AUSBOSS_NODES_FreeMemory",
|
||||
@@ -92,6 +94,7 @@ PUBLIC_NODE_IDS = {
|
||||
"AUSBOSS_NODES_StitchInpaint",
|
||||
"AUSBOSS_NODES_TiledUpscaleStitch",
|
||||
"AUSBOSS_NODES_Text",
|
||||
"AUSBOSS_NODES_TextLines",
|
||||
"AUSBOSS_NODES_TiledUpscale",
|
||||
"AUSBOSS_NODES_VideoCropRotatePad",
|
||||
"AUSBOSS_NODES_VideoCropRotatePadClip",
|
||||
|
||||
Vendored
+67
@@ -116,6 +116,50 @@
|
||||
]
|
||||
]
|
||||
},
|
||||
"AUSBOSS_NODES_EditLock": {
|
||||
"required": [
|
||||
"model",
|
||||
"vae",
|
||||
"image",
|
||||
"lock",
|
||||
"tolerance"
|
||||
],
|
||||
"optional": [
|
||||
"mask"
|
||||
],
|
||||
"outputs": [
|
||||
[
|
||||
"MODEL",
|
||||
"model"
|
||||
],
|
||||
[
|
||||
"AUSBOSS_EDIT_LOCK",
|
||||
"edit_lock"
|
||||
]
|
||||
]
|
||||
},
|
||||
"AUSBOSS_NODES_EditLockRestore": {
|
||||
"required": [
|
||||
"image",
|
||||
"edit_lock",
|
||||
"feather"
|
||||
],
|
||||
"optional": [],
|
||||
"outputs": [
|
||||
[
|
||||
"IMAGE",
|
||||
"image"
|
||||
],
|
||||
[
|
||||
"MASK",
|
||||
"edit_mask"
|
||||
],
|
||||
[
|
||||
"STRING",
|
||||
"report"
|
||||
]
|
||||
]
|
||||
},
|
||||
"AUSBOSS_NODES_Float": {
|
||||
"required": [
|
||||
"value"
|
||||
@@ -992,6 +1036,29 @@
|
||||
]
|
||||
]
|
||||
},
|
||||
"AUSBOSS_NODES_TextLines": {
|
||||
"required": [
|
||||
"text",
|
||||
"split",
|
||||
"remove_numbering",
|
||||
"limit"
|
||||
],
|
||||
"optional": [],
|
||||
"outputs": [
|
||||
[
|
||||
"STRING",
|
||||
"line"
|
||||
],
|
||||
[
|
||||
"INT",
|
||||
"number"
|
||||
],
|
||||
[
|
||||
"INT",
|
||||
"count"
|
||||
]
|
||||
]
|
||||
},
|
||||
"AUSBOSS_NODES_TiledUpscale": {
|
||||
"required": [
|
||||
"image",
|
||||
|
||||
@@ -4,6 +4,7 @@ import test from "node:test";
|
||||
import {
|
||||
collapseRepeatedErrors,
|
||||
emptySourceMessage,
|
||||
explainRunError,
|
||||
findEmptySources,
|
||||
nodeLacksSource,
|
||||
nodesThatRun,
|
||||
@@ -199,3 +200,40 @@ test("an empty Load Audio asks for a sound file", () => {
|
||||
prompt[1].inputs.audio = "song.mp3";
|
||||
assert.deepEqual(findEmptySources(prompt, outputs), []);
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------ known run errors
|
||||
|
||||
// The report a user posted: ComfyUI 0.44.0, H3 keeping the clip's sound, with
|
||||
// Muse MiniMax Director installed.
|
||||
const MUSE_REPORT = {
|
||||
node_type: "SamplerCustomAdvanced",
|
||||
exception_type: "KeyError",
|
||||
exception_message: "'latent'",
|
||||
traceback: [
|
||||
' File "G:\\ComfyUI\\ComfyUI\\comfy\\samplers.py", line 953, in encode_model_conds\n out = model_function(**params)\n',
|
||||
' File "G:\\ComfyUI\\ComfyUI\\custom_nodes\\muse-minimax-director-v1-2\\muse_minimax_hybrid_patch.py", line 30, in extra_conds_with_hybrid\n visual_latents.extend(item["latent"] for item in keyframes)\n',
|
||||
],
|
||||
};
|
||||
|
||||
test("the Muse MiniMax Director error is explained and names the pack", () => {
|
||||
const known = explainRunError(MUSE_REPORT);
|
||||
assert.match(known.detail, /Muse MiniMax Director/);
|
||||
assert.match(known.detail, /Turn that pack off/);
|
||||
});
|
||||
|
||||
test("the same KeyError from anywhere else is left alone", () => {
|
||||
assert.equal(explainRunError({ ...MUSE_REPORT, traceback: [' File "comfy/samplers.py", line 953\n'] }), null);
|
||||
assert.equal(explainRunError({ ...MUSE_REPORT, exception_message: "'samples'" }), null);
|
||||
});
|
||||
|
||||
test("an old ComfyUI loading a w6a8 model is told to update", () => {
|
||||
const known = explainRunError({ exception_type: "KeyError", exception_message: "'w6a8_int8'", traceback: [] });
|
||||
assert.match(known.detail, /0\.38/);
|
||||
});
|
||||
|
||||
test("an unknown error, or no report at all, gives nothing", () => {
|
||||
assert.equal(explainRunError({ exception_type: "RuntimeError", exception_message: "CUDA out of memory", traceback: [] }), null);
|
||||
assert.equal(explainRunError(null), null);
|
||||
assert.equal(explainRunError({}), null);
|
||||
assert.equal(explainRunError({ exception_type: "KeyError", exception_message: "'latent'", traceback: "muse_minimax_hybrid_patch" }) !== null, true);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,373 @@
|
||||
"""Edit Lock: a local edit keeps the rest of the picture, a whole-picture edit is left alone."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
import torch
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
if "nodes" in sys.modules and not hasattr(sys.modules["nodes"], "__path__"):
|
||||
del sys.modules["nodes"]
|
||||
|
||||
from nodes._edit_lock_helpers import ( # noqa: E402
|
||||
WHOLE_PICTURE,
|
||||
EditLock,
|
||||
box_blur,
|
||||
broad_change,
|
||||
change_core,
|
||||
describe,
|
||||
difference,
|
||||
measure_redraw,
|
||||
quiet_level,
|
||||
restore_pixels,
|
||||
soften,
|
||||
)
|
||||
from nodes.node_edit_lock import ( # noqa: E402
|
||||
NODE_CLASS_MAPPINGS,
|
||||
NODE_DISPLAY_NAME_MAPPINGS,
|
||||
AusBossEditLock,
|
||||
AusBossEditLockRestore,
|
||||
)
|
||||
|
||||
H, W = 96, 80
|
||||
BOX = (slice(30, 60), slice(20, 50)) # where the planted edit sits
|
||||
|
||||
|
||||
def latents(seed: int = 0, video: bool = True):
|
||||
gen = torch.Generator().manual_seed(seed)
|
||||
# a real latent is smooth from cell to cell, so these are too
|
||||
source, other = (box_blur(torch.randn(1, 24, H, W, generator=gen), 4) for _ in range(2))
|
||||
source, other = source / source.std(), other / other.std()
|
||||
redraw = 0.15 * torch.randn(1, 24, H, W, generator=gen) # the model's own redraw noise
|
||||
if video:
|
||||
return source[:, :, None], other[:, :, None], redraw[:, :, None]
|
||||
return source, other, redraw
|
||||
|
||||
|
||||
def local_prediction(source, other, redraw, tint: float = 0.0):
|
||||
pred = source + redraw + tint
|
||||
pred[..., BOX[0], BOX[1]] = other[..., BOX[0], BOX[1]]
|
||||
return pred
|
||||
|
||||
|
||||
def lock_for(source, mode="auto", tolerance=1.0):
|
||||
lock = EditLock(mode=mode, tolerance=tolerance)
|
||||
lock.latent = source.clone()
|
||||
lock.source = torch.zeros(1, H * 16, W * 16, 3)
|
||||
return lock
|
||||
|
||||
|
||||
def run(lock, pred, sigma=1.0, steps=None):
|
||||
args = {"denoised": pred, "sigma": torch.tensor([sigma])}
|
||||
if steps is not None:
|
||||
# the sampler's own list of noise levels, the way ComfyUI hands it to the hook
|
||||
args["model_options"] = {"transformer_options": {"sample_sigmas": torch.tensor(steps)}}
|
||||
return lock.step(args)
|
||||
|
||||
|
||||
SIX = [1.0, 0.97, 0.89, 0.75, 0.53, 0.23, 0.0]
|
||||
|
||||
|
||||
class EditLockTests(unittest.TestCase):
|
||||
def test_local_edit_keeps_the_rest(self):
|
||||
for video in (True, False):
|
||||
source, other, redraw = latents(video=video)
|
||||
pred = local_prediction(source, other, redraw, tint=0.2)
|
||||
lock = lock_for(source)
|
||||
out = run(lock, pred)
|
||||
self.assertEqual(lock.status, "locked")
|
||||
self.assertEqual(out.shape, pred.shape)
|
||||
# far from the edit the original is back exactly, tint and redraw noise gone
|
||||
self.assertTrue(torch.equal(out[..., :12, :], source[..., :12, :].to(out.dtype)))
|
||||
self.assertTrue(torch.equal(out[..., 80:, :], source[..., 80:, :].to(out.dtype)))
|
||||
# inside the edit the model's prediction is untouched
|
||||
inner = (slice(36, 54), slice(26, 44))
|
||||
self.assertTrue(torch.equal(out[..., inner[0], inner[1]], pred[..., inner[0], inner[1]]))
|
||||
kept = lock.kept_share()
|
||||
self.assertGreater(kept, 0.7)
|
||||
self.assertLess(kept, 0.92)
|
||||
self.assertIn("kept", describe(lock))
|
||||
|
||||
def test_report_says_what_to_do_when_nearly_all_was_kept(self):
|
||||
source, other, redraw = latents(video=False)
|
||||
lock = lock_for(source)
|
||||
run(lock, local_prediction(source, other, redraw))
|
||||
self.assertNotIn("set lock to off", describe(lock))
|
||||
# a faint change everywhere (rain) and one small clear spot: nearly all of it is kept
|
||||
pred = source + redraw
|
||||
pred[..., 40:46, 30:36] = other[..., 40:46, 30:36]
|
||||
lock = lock_for(source)
|
||||
run(lock, pred)
|
||||
self.assertGreater(lock.kept_share(), 0.95)
|
||||
self.assertIn("set lock to off", describe(lock))
|
||||
|
||||
def test_it_watches_first_and_locks_when_the_noise_is_down(self):
|
||||
source, other, redraw = latents(video=False)
|
||||
lock = lock_for(source)
|
||||
# the model's own redraw noise fades from step to step, as it does in a real run
|
||||
for sigma, fade in ((1.0, 1.0), (0.97, 0.8), (0.89, 0.7)):
|
||||
pred = local_prediction(source, other, redraw * fade)
|
||||
out = run(lock, pred, sigma, SIX)
|
||||
self.assertEqual(lock.status, "watching")
|
||||
self.assertIsNone(lock.pixel_mask(H * 16, W * 16))
|
||||
# nothing is put back yet, and with no zoom to undo the prediction passes as it is
|
||||
self.assertTrue(torch.equal(out, pred))
|
||||
pred = local_prediction(source, other, redraw * 0.65)
|
||||
out = run(lock, pred, 0.75, SIX)
|
||||
self.assertEqual(lock.status, "locked")
|
||||
self.assertGreater(lock.top, 4.0)
|
||||
self.assertTrue(torch.equal(out[..., :12, :], source[..., :12, :]))
|
||||
inner = (slice(36, 54), slice(26, 44))
|
||||
self.assertTrue(torch.equal(out[..., inner[0], inner[1]], pred[..., inner[0], inner[1]]))
|
||||
|
||||
def test_a_faint_change_everywhere_is_let_through(self):
|
||||
source, _, redraw = latents(video=False)
|
||||
rain = 0.25 * box_blur(torch.randn(1, 24, H, W, generator=torch.Generator().manual_seed(5)), 1)
|
||||
for mode, status in (("auto", "soft"), ("always", "locked")):
|
||||
lock = lock_for(source, mode=mode)
|
||||
# nothing shows at the first look; then the rain comes in everywhere, and no spot changes clearly
|
||||
for sigma, fade, wet in ((1.0, 1.0, 0.0), (0.97, 0.9, 0.5), (0.89, 0.8, 1.0)):
|
||||
run(lock, source + redraw * fade + rain * wet, sigma, SIX)
|
||||
self.assertEqual(lock.status, "watching")
|
||||
pred = source + redraw * 0.7 + rain * 1.5
|
||||
out = run(lock, pred, 0.75, SIX)
|
||||
self.assertEqual(lock.status, status)
|
||||
self.assertLess(lock.top, 4.0)
|
||||
if mode == "auto":
|
||||
self.assertLess(float((out - pred).abs().max()), 1e-4)
|
||||
self.assertIn("stepped aside", describe(lock))
|
||||
|
||||
def test_a_wide_faint_change_with_no_peak_is_not_locked_in_patches(self):
|
||||
source, _, redraw = latents(video=False)
|
||||
gen = torch.Generator().manual_seed(9)
|
||||
snow = torch.zeros(1, 24, H, W)
|
||||
snow[..., :, :34] = 0.25 * torch.randn(1, 24, H, 34, generator=gen) # faint, over two fifths of the picture
|
||||
theta = torch.tensor([[[1 / 1.015, 0.0, 0.0], [0.0, 1 / 1.015, 0.0]]])
|
||||
grid = torch.nn.functional.affine_grid(theta, list(source.shape), align_corners=False)
|
||||
zoomed = torch.nn.functional.grid_sample(source, grid, mode="bilinear", padding_mode="border", align_corners=False)
|
||||
for mode, status in (("auto", "soft"), ("always", "locked")):
|
||||
lock = lock_for(source, mode=mode)
|
||||
for sigma, fade in ((1.0, 1.0), (0.97, 0.8), (0.89, 0.7)):
|
||||
run(lock, source + redraw * fade, sigma, SIX)
|
||||
pred = source + redraw * 0.65 + snow
|
||||
out = run(lock, pred, 0.75, SIX)
|
||||
self.assertEqual(lock.status, status)
|
||||
self.assertGreater(lock.wide, 0.10)
|
||||
self.assertLess(lock.peak, 4.0)
|
||||
if mode == "always":
|
||||
self.assertLess(lock.kept_share(), 0.8)
|
||||
continue
|
||||
self.assertIsNone(lock.pixel_mask(H * 16, W * 16))
|
||||
self.assertLess(float((out - pred).abs().max()), 1e-4)
|
||||
self.assertIn("held the picture in place", describe(lock))
|
||||
# for the rest of the run only the zoom is undone
|
||||
out = run(lock, zoomed, 0.53, SIX)
|
||||
self.assertEqual(lock.status, "soft")
|
||||
smooth = lambda x: box_blur(x, 2)[..., 6:-6, 6:-6]
|
||||
self.assertLess(float((smooth(out) - smooth(source)).abs().mean()),
|
||||
float((smooth(zoomed) - smooth(source)).abs().mean()) * 0.35)
|
||||
|
||||
def test_the_last_step_decides_when_the_noise_never_gets_that_low(self):
|
||||
source, other, redraw = latents(video=False)
|
||||
lock = lock_for(source)
|
||||
two = [1.0, 0.92, 0.0]
|
||||
run(lock, local_prediction(source, other, redraw), 1.0, two)
|
||||
self.assertEqual(lock.status, "watching")
|
||||
run(lock, local_prediction(source, other, redraw * 0.8), 0.92, two)
|
||||
self.assertEqual(lock.status, "locked")
|
||||
# a one-step run decides at once
|
||||
lock = lock_for(source)
|
||||
run(lock, local_prediction(source, other, redraw), 1.0, [1.0, 0.0])
|
||||
self.assertEqual(lock.status, "locked")
|
||||
|
||||
def test_the_zoom_is_undone_while_it_watches(self):
|
||||
source, other, redraw = latents(video=False)
|
||||
theta = torch.tensor([[[1 / 1.015, 0.0, 0.0], [0.0, 1 / 1.015, 0.0]]])
|
||||
grid = torch.nn.functional.affine_grid(theta, list(source.shape), align_corners=False)
|
||||
zoomed = torch.nn.functional.grid_sample(source, grid, mode="bilinear", padding_mode="border", align_corners=False)
|
||||
lock = lock_for(source)
|
||||
out = run(lock, zoomed, 1.0, SIX)
|
||||
self.assertEqual(lock.status, "watching")
|
||||
# read on smoothed copies: this is about where things sit, and resampling costs fine detail either way
|
||||
smooth = lambda x: box_blur(x, 2)[..., 6:-6, 6:-6]
|
||||
off_before = float((smooth(zoomed) - smooth(source)).abs().mean())
|
||||
off_after = float((smooth(out) - smooth(source)).abs().mean())
|
||||
self.assertLess(off_after, off_before * 0.35)
|
||||
|
||||
def test_whole_picture_edit_is_left_alone(self):
|
||||
source, other, _ = latents()
|
||||
pred = 0.3 * source + other # a new picture over the old one
|
||||
lock = lock_for(source)
|
||||
out = run(lock, pred)
|
||||
self.assertEqual(lock.status, "whole picture")
|
||||
self.assertTrue(torch.equal(out, pred))
|
||||
self.assertIsNone(lock.pixel_mask(H * 16, W * 16))
|
||||
# and it stays aside for the rest of the run
|
||||
self.assertTrue(torch.equal(run(lock, pred, 0.8), pred))
|
||||
self.assertIn("stepped aside", describe(lock))
|
||||
|
||||
def test_a_slightly_larger_redraw_is_measured_and_is_not_an_edit(self):
|
||||
source, other, redraw = latents(video=False)
|
||||
# drawn 1.5% wider, 1% taller and half a cell to the right
|
||||
theta = torch.tensor([[[1 / 1.015, 0.0, -1.0 / W], [0.0, 1 / 1.01, 0.0]]])
|
||||
grid = torch.nn.functional.affine_grid(theta, list(source.shape), align_corners=False)
|
||||
zoomed = torch.nn.functional.grid_sample(source, grid, mode="bilinear", padding_mode="border", align_corners=False)
|
||||
move = measure_redraw(local_prediction(zoomed, other, redraw), source)
|
||||
self.assertAlmostEqual(float(move[0]), 1 / 1.015 - 1, delta=0.002)
|
||||
self.assertAlmostEqual(float(move[1]), 1 / 1.01 - 1, delta=0.002)
|
||||
self.assertAlmostEqual(float(move[2]), -0.5, delta=0.15)
|
||||
self.assertAlmostEqual(float(move[3]), 0.0, delta=0.15)
|
||||
self.assertLess(float(measure_redraw(source + redraw, source).abs().max()), 0.02)
|
||||
self.assertLess(float(broad_change(zoomed + redraw, source)), WHOLE_PICTURE / 2)
|
||||
lock = lock_for(source)
|
||||
out = run(lock, local_prediction(zoomed, other, redraw))
|
||||
self.assertEqual(lock.status, "locked")
|
||||
# the zoom did not read as an edit: only the planted box is free, and the rest is the original, unzoomed
|
||||
self.assertGreater(lock.kept_share(), 0.7)
|
||||
self.assertTrue(torch.equal(out[..., :12, :], source[..., :12, :]))
|
||||
# an even tint is not one either; new light across the picture is
|
||||
self.assertLess(float(broad_change(source + redraw + 0.6, source)), WHOLE_PICTURE / 2)
|
||||
self.assertGreater(float(broad_change(source + redraw + 0.6 * other, source)), WHOLE_PICTURE)
|
||||
|
||||
def test_always_locks_a_whole_picture_edit(self):
|
||||
source, other, _ = latents()
|
||||
lock = lock_for(source, mode="always")
|
||||
run(lock, 0.3 * source + other)
|
||||
self.assertEqual(lock.status, "locked")
|
||||
|
||||
def test_changed_stays_changed_and_a_new_run_starts_clean(self):
|
||||
source, other, redraw = latents()
|
||||
lock = lock_for(source)
|
||||
run(lock, local_prediction(source, other, redraw), 1.0)
|
||||
first = lock.mask.clone()
|
||||
run(lock, source + redraw, 0.8) # a later step that agrees with the original everywhere
|
||||
self.assertTrue(torch.all(lock.mask >= first))
|
||||
run(lock, source + redraw, 1.0) # the noise level went up: a new run
|
||||
self.assertLess(float(lock.mask.max()), 0.5)
|
||||
|
||||
def test_given_mask_is_used_as_is(self):
|
||||
source, other, redraw = latents()
|
||||
lock = lock_for(source)
|
||||
mask = torch.zeros(1, H * 16, W * 16)
|
||||
mask[:, : H * 8] = 1.0 # the top half may change
|
||||
lock.user_mask = mask
|
||||
pred = 0.3 * source + other # would count as a whole-picture edit
|
||||
out = run(lock, pred)
|
||||
self.assertEqual(lock.status, "locked")
|
||||
self.assertTrue(torch.equal(out[..., :40, :], pred[..., :40, :]))
|
||||
self.assertTrue(torch.equal(out[..., 56:, :], source[..., 56:, :]))
|
||||
self.assertIn("your mask", describe(lock))
|
||||
|
||||
def test_other_sizes_video_and_wrong_vae_do_nothing(self):
|
||||
source, other, redraw = latents()
|
||||
pred = local_prediction(source, other, redraw)
|
||||
lock = lock_for(source[..., :48, :40].clone())
|
||||
self.assertTrue(torch.equal(run(lock, pred), pred))
|
||||
self.assertEqual(lock.status, "size")
|
||||
lock = lock_for(source)
|
||||
clip = pred.expand(-1, -1, 5, -1, -1)
|
||||
self.assertTrue(torch.equal(run(lock, clip), clip))
|
||||
self.assertEqual(lock.status, "video")
|
||||
lock = lock_for(source[:, :16].clone())
|
||||
self.assertTrue(torch.equal(run(lock, pred), pred))
|
||||
self.assertEqual(lock.status, "vae")
|
||||
for status in ("size", "video", "vae"):
|
||||
lock.status = status
|
||||
self.assertIn("did nothing", describe(lock))
|
||||
|
||||
def test_a_grid_one_cell_off_still_locks(self):
|
||||
source, other, redraw = latents(video=False)
|
||||
lock = lock_for(torch.nn.functional.interpolate(source, size=(H + 2, W + 1), mode="bilinear"))
|
||||
run(lock, local_prediction(source, other, redraw))
|
||||
self.assertEqual(lock.status, "locked")
|
||||
self.assertGreater(lock.kept_share(), 0.7)
|
||||
|
||||
def test_tolerance_moves_the_line(self):
|
||||
source, other, redraw = latents(video=False)
|
||||
pred = source + redraw
|
||||
pred[..., BOX[0], BOX[1]] += 0.45 * other[..., BOX[0], BOX[1]] # a faint change
|
||||
magnitude = difference(pred, source)
|
||||
quiet = quiet_level(magnitude)
|
||||
loose = change_core(magnitude, quiet, 0.5)
|
||||
strict = change_core(magnitude, quiet, 2.0)
|
||||
self.assertGreater(float(loose[..., 40:50, 30:40].mean()), 0.9)
|
||||
self.assertEqual(float(strict.max()), 0.0)
|
||||
|
||||
def test_faint_change_counts_only_beside_a_clear_one(self):
|
||||
quiet = torch.tensor([1.0])
|
||||
magnitude = torch.ones(1, 1, 40, 40)
|
||||
magnitude[..., 5:15, 5:15] = 2.2 # faint, alone: redraw noise
|
||||
magnitude[..., 22:34, 22:34] = 2.2 # faint ...
|
||||
magnitude[..., 26:30, 26:30] = 4.0 # ... around a clear change
|
||||
core = change_core(magnitude, quiet)
|
||||
self.assertEqual(float(core[..., 5:15, 5:15].max()), 0.0)
|
||||
self.assertEqual(float(core[..., 22:34, 22:34].min()), 1.0)
|
||||
self.assertEqual(float(core.sum()), 144.0)
|
||||
# what an earlier step held stays, and faint change that touches it joins
|
||||
held = torch.zeros_like(core)
|
||||
held[..., 8:10, 8:10] = 1.0
|
||||
again = change_core(torch.ones(1, 1, 40, 40), quiet, held=held)
|
||||
self.assertEqual(float(again.sum()), 4.0)
|
||||
joined = change_core(magnitude, quiet, held=held)
|
||||
self.assertEqual(float(joined[..., 5:15, 5:15].min()), 1.0)
|
||||
soft = soften(core)
|
||||
self.assertTrue(torch.all(soft[..., 22:34, 22:34] == 1.0))
|
||||
self.assertGreater(float(soft[0, 0, 21, 28]), 0.5)
|
||||
self.assertEqual(float(soft[0, 0, 2, 2]), 0.0)
|
||||
|
||||
def test_restore_gives_the_original_pixels_back(self):
|
||||
gen = torch.Generator().manual_seed(1)
|
||||
original = torch.rand(1, 128, 160, 3, generator=gen)
|
||||
result = torch.rand(2, 128, 160, 3, generator=gen)
|
||||
mask = torch.zeros(1, 128, 160)
|
||||
mask[:, 40:80, 60:100] = 1.0
|
||||
out, edit = restore_pixels(result, original, mask, 8)
|
||||
self.assertEqual(out.shape, result.shape)
|
||||
self.assertEqual(edit.shape, (2, 128, 160))
|
||||
self.assertTrue(torch.equal(out[:, :16], original[:, :16].expand(2, -1, -1, -1)))
|
||||
self.assertTrue(torch.equal(out[:, 40:80, 60:100], result[:, 40:80, 60:100]))
|
||||
# the fade lies outside the edit
|
||||
self.assertTrue(torch.all(edit[:, 40:80, 60:100] == 1.0))
|
||||
self.assertGreater(float(edit[0, 36, 80]), 0.0)
|
||||
# another size: the original is resized to the result
|
||||
small = torch.rand(1, 64, 80, 3, generator=gen)
|
||||
out, _ = restore_pixels(result, small, mask, 0)
|
||||
self.assertEqual(out.shape, result.shape)
|
||||
|
||||
def test_restore_node_passes_through_when_nothing_was_locked(self):
|
||||
lock = EditLock()
|
||||
lock.status = "whole picture"
|
||||
image = torch.rand(1, 32, 48, 3)
|
||||
out, mask, report = AusBossEditLockRestore().restore(image, lock, 16)
|
||||
self.assertTrue(torch.equal(out, image))
|
||||
self.assertEqual(mask.shape, (1, 32, 48))
|
||||
self.assertTrue(torch.all(mask == 1.0))
|
||||
self.assertIn("stepped aside", report)
|
||||
|
||||
def test_restore_node_after_a_locked_run(self):
|
||||
source, other, redraw = latents()
|
||||
lock = lock_for(source)
|
||||
lock.source = torch.rand(1, H * 4, W * 4, 3)
|
||||
run(lock, local_prediction(source, other, redraw))
|
||||
image = torch.rand(1, H * 4, W * 4, 3)
|
||||
out, mask, report = AusBossEditLockRestore().restore(image, lock, 4)
|
||||
self.assertTrue(torch.equal(out[:, :24], lock.source[:, :24]))
|
||||
self.assertTrue(torch.equal(out[:, 150:210, 110:170], image[:, 150:210, 110:170]))
|
||||
self.assertGreater(float(mask[0, 180, 140]), 0.99)
|
||||
self.assertIn("kept", report)
|
||||
|
||||
def test_mappings(self):
|
||||
keys = {"AUSBOSS_NODES_EditLock", "AUSBOSS_NODES_EditLockRestore"}
|
||||
self.assertEqual(set(NODE_CLASS_MAPPINGS), keys)
|
||||
self.assertEqual(set(NODE_DISPLAY_NAME_MAPPINGS), keys)
|
||||
self.assertIs(NODE_CLASS_MAPPINGS["AUSBOSS_NODES_EditLock"], AusBossEditLock)
|
||||
for name in NODE_DISPLAY_NAME_MAPPINGS.values():
|
||||
self.assertTrue(name.endswith(" 🆎"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,235 @@
|
||||
"""Mask by Touch on a big picture: SAM 3 is asked about a smaller copy, and
|
||||
what it answers comes back at the picture's own size. ComfyUI's detector and
|
||||
text reader are stood in for by fakes, so this runs without ComfyUI and
|
||||
without a model."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
import unittest
|
||||
|
||||
import torch
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
if "nodes" in sys.modules and not hasattr(sys.modules["nodes"], "__path__"):
|
||||
del sys.modules["nodes"]
|
||||
|
||||
from nodes import _mask_by_touch_helpers as helpers # noqa: E402
|
||||
from nodes import node_mask_by_touch as node_module # noqa: E402
|
||||
from nodes._mask_by_name_helpers import IF_NOTHING # noqa: E402
|
||||
from nodes._mask_by_touch_helpers import WORK_SIDE, WorkView # noqa: E402
|
||||
|
||||
UI = "ausboss_mask_by_touch"
|
||||
# The big picture, and the long side of the copy SAM 3 is asked about here.
|
||||
H, W = 300, 240
|
||||
SIDE = 60
|
||||
# Things as shares of the picture: (left, top, right, bottom).
|
||||
PERSON = (0.25, 0.1, 0.75, 0.95)
|
||||
FACE = (0.4, 0.15, 0.6, 0.3)
|
||||
THINGS = {"person": [(PERSON, 0.98)], "face": [(FACE, 0.9)]}
|
||||
|
||||
|
||||
def box(thing, height, width):
|
||||
left, top, right, bottom = thing
|
||||
mask = torch.zeros(height, width)
|
||||
mask[round(top * height):round(bottom * height), round(left * width):round(right * width)] = 1.0
|
||||
return mask
|
||||
|
||||
|
||||
def same_area(one, other):
|
||||
one, other = one.bool(), other.bool()
|
||||
return float((one & other).sum()) / max(1.0, float((one | other).sum()))
|
||||
|
||||
|
||||
class SizedSam:
|
||||
"""Stands in for ComfyUI's SAM3_Detect. It answers at the size of the
|
||||
picture it is given, as the real one does."""
|
||||
|
||||
calls: list = []
|
||||
|
||||
@classmethod
|
||||
def execute(cls, model, image, conditioning=None, positive_coords=None, negative_coords=None, threshold=0.5, refine_iterations=2, individual_masks=False):
|
||||
height, width = int(image.shape[1]), int(image.shape[2])
|
||||
call = {"size": (height, width)}
|
||||
cls.calls.append(call)
|
||||
if conditioning is not None:
|
||||
call["word"] = conditioning[0][2].rpartition(":")[0]
|
||||
found = THINGS.get(call["word"], [])
|
||||
masks = torch.stack([box(thing, height, width) for thing, _ in found]) if found else torch.zeros(0, height, width)
|
||||
return types.SimpleNamespace(result=(masks, [[{"score": sure} for _, sure in found]]))
|
||||
call["points"] = points = [(point["x"], point["y"]) for point in json.loads(positive_coords)]
|
||||
answer = torch.zeros(height, width)
|
||||
for thing in (FACE, PERSON): # the smallest thing that holds every point
|
||||
area = box(thing, height, width)
|
||||
if all(0 <= y < height and 0 <= x < width and float(area[y, x]) > 0 for x, y in points):
|
||||
answer = area
|
||||
break
|
||||
return types.SimpleNamespace(result=(answer[None], [[]]))
|
||||
|
||||
|
||||
class Encode:
|
||||
def encode(self, clip, text):
|
||||
return ([["read by", clip, text]],)
|
||||
|
||||
|
||||
class Folders:
|
||||
def __init__(self, folder):
|
||||
self.folder = folder
|
||||
|
||||
def get_temp_directory(self):
|
||||
return self.folder
|
||||
|
||||
def get_save_image_path(self, prefix, folder, _width, _height):
|
||||
counter = 1 + sum(1 for name in Path(folder).iterdir() if name.name.startswith(prefix + "_"))
|
||||
return folder, prefix, counter, "", prefix
|
||||
|
||||
|
||||
class WorkViewTests(unittest.TestCase):
|
||||
def test_a_picture_up_to_the_work_side_is_asked_about_as_it_is(self):
|
||||
frame = torch.rand(1, 40, WORK_SIDE, 3)
|
||||
view = WorkView(frame)
|
||||
self.assertTrue(view.same)
|
||||
self.assertIs(view.frame, frame)
|
||||
self.assertEqual((view.work_height, view.work_width), (40, WORK_SIDE))
|
||||
self.assertEqual(view.point((17, 5)), (17, 5))
|
||||
self.assertEqual(view.full_point((17, 5)), (17, 5))
|
||||
mask = torch.zeros(40, WORK_SIDE, dtype=torch.bool)
|
||||
self.assertIs(view.full(mask), mask)
|
||||
self.assertEqual(view.full_box([1, 2, 3, 4]), [1, 2, 3, 4])
|
||||
|
||||
def test_a_bigger_picture_gets_a_copy_with_the_long_side_at_the_work_side(self):
|
||||
frame = torch.rand(1, WORK_SIDE * 2, WORK_SIDE + 500, 3)
|
||||
view = WorkView(frame)
|
||||
self.assertFalse(view.same)
|
||||
self.assertEqual(view.work_height, WORK_SIDE)
|
||||
self.assertEqual(tuple(view.frame.shape), (1, view.work_height, view.work_width, 3))
|
||||
self.assertEqual(view.work_width, round((WORK_SIDE + 500) / 2))
|
||||
# Averaged down: the copy's mean is the picture's.
|
||||
self.assertAlmostEqual(float(view.frame.mean()), float(frame.mean()), places=3)
|
||||
|
||||
def test_a_place_on_the_copy_leads_back_to_itself(self):
|
||||
for height, width in ((4096, 4096), (3840, 2160), (2049, 8192), (5000, 3333)):
|
||||
view = WorkView(torch.zeros(1, height, width, 1))
|
||||
for x in (0, 1, 7, view.work_width // 3, view.work_width - 2, view.work_width - 1):
|
||||
for y in (0, 1, 5, view.work_height // 2, view.work_height - 1):
|
||||
full = view.full_point((x, y))
|
||||
self.assertTrue(0 <= full[0] < width and 0 <= full[1] < height)
|
||||
self.assertEqual(view.point(full), (x, y), f"{width}x{height} at {x},{y}")
|
||||
self.assertEqual(view.point((width - 1, height - 1)), (view.work_width - 1, view.work_height - 1))
|
||||
|
||||
def test_an_answer_comes_back_at_the_pictures_own_size(self):
|
||||
view = WorkView(torch.zeros(1, 4096, 3000, 3))
|
||||
small = box(FACE, view.work_height, view.work_width) > 0.5
|
||||
full = view.full(small)
|
||||
self.assertEqual(tuple(full.shape), (4096, 3000))
|
||||
self.assertEqual(full.dtype, torch.bool)
|
||||
self.assertGreater(same_area(full, box(FACE, 4096, 3000)), 0.98)
|
||||
x0, y0, x1, y1 = view.full_box(helpers.box_of(small))
|
||||
wanted = helpers.box_of(box(FACE, 4096, 3000) > 0.5)
|
||||
for got, want in zip((x0, y0, x1, y1), wanted):
|
||||
self.assertLessEqual(abs(got - want), 3)
|
||||
|
||||
|
||||
class BigPictureTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
SizedSam.calls = []
|
||||
node_module._MEMORY.clear()
|
||||
node_module._WORDS.clear()
|
||||
node_module._TRUST.clear()
|
||||
node_module._UNSTEADY.clear()
|
||||
node_module._NOTED.clear()
|
||||
self.swap(node_module, "_core_nodes", lambda: {"SAM3_Detect": SizedSam, "CLIPTextEncode": Encode})
|
||||
self.swap(node_module, "SCOPE_SPACING", 12)
|
||||
self.swap(node_module, "SCOPE_SECONDS_BLIND", 1e9)
|
||||
self.swap(helpers, "WORK_SIDE", SIDE)
|
||||
folder = tempfile.mkdtemp(prefix="ausboss_touch_big_")
|
||||
self.addCleanup(shutil.rmtree, folder, True)
|
||||
self.swap(node_module, "folder_paths", Folders(folder))
|
||||
self.picture = torch.rand(1, H, W, 3) * 0.9
|
||||
self.node = node_module.AusBossMaskByTouch()
|
||||
|
||||
def swap(self, module, attribute, value):
|
||||
self.addCleanup(setattr, module, attribute, getattr(module, attribute))
|
||||
setattr(module, attribute, value)
|
||||
|
||||
def run_node(self, text="", image=None, **overrides):
|
||||
values = {"grow": 0, "soften": 0.0, "smooth": 0, "fill_holes": False, "if_nothing": IF_NOTHING[1]}
|
||||
values.update(overrides)
|
||||
return self.node.pick(self.picture if image is None else image, "sam3 model", "sam3 clip", text, **values)
|
||||
|
||||
def touches(self, *items):
|
||||
return json.dumps({"v": 1, "items": list(items)})
|
||||
|
||||
def test_sam3_is_never_given_the_big_picture(self):
|
||||
self.run_node(self.touches({"t": "tap", "op": 1, "x": 120, "y": 66}, {"t": "all", "op": 1, "w": "person"}))
|
||||
self.assertTrue(SizedSam.calls)
|
||||
for call in SizedSam.calls:
|
||||
self.assertEqual(call["size"], (SIDE, 48), "the copy: 300 x 240 with its long side at 60")
|
||||
|
||||
def test_a_tap_takes_the_thing_at_its_place_and_the_mask_is_full_size(self):
|
||||
out = self.run_node(self.touches({"t": "tap", "op": 1, "x": 120, "y": 66})) # on the face
|
||||
mask = out["result"][0]
|
||||
self.assertEqual(tuple(mask.shape), (1, H, W))
|
||||
self.assertGreater(same_area(mask[0] > 0.5, box(FACE, H, W)), 0.9)
|
||||
# The point was asked where it lies on the copy.
|
||||
self.assertEqual(SizedSam.calls[0]["points"], [(120 * 48 // W, 66 * SIDE // H)])
|
||||
|
||||
def test_a_word_takes_every_thing_of_that_name_at_full_size(self):
|
||||
out = self.run_node(self.touches({"t": "all", "op": 1, "w": "person"}, {"t": "all", "op": 0, "w": "face"}))
|
||||
mask = out["result"][0][0] > 0.5
|
||||
wanted = (box(PERSON, H, W) > 0.5) & ~(box(FACE, H, W) > 0.5)
|
||||
self.assertGreater(same_area(mask, wanted), 0.93)
|
||||
self.assertEqual(out["result"][2], "person")
|
||||
|
||||
def test_a_stroke_is_asked_about_on_the_copy(self):
|
||||
stroke = {"t": "find", "op": 1, "r": 8, "pts": [[110, 60], [130, 70]]} # across the face
|
||||
out = self.run_node(self.touches(stroke))
|
||||
self.assertGreater(same_area(out["result"][0][0] > 0.5, box(FACE, H, W)), 0.9)
|
||||
for call in SizedSam.calls:
|
||||
for x, y in call.get("points", []):
|
||||
self.assertTrue(0 <= x < 48 and 0 <= y < SIDE)
|
||||
|
||||
def test_what_a_look_lights_up_is_what_a_click_on_it_takes(self):
|
||||
note = self.run_node(ask="scope")["ui"][UI][0]
|
||||
self.assertEqual((note["width"], note["height"]), (W, H))
|
||||
pieces = {piece["name"]: piece for piece in note["pieces"]["list"] if piece["name"]}
|
||||
self.assertEqual(set(pieces), {"person", "face"})
|
||||
face = pieces["face"]
|
||||
# Its own point is a pixel of the picture, inside the thing, and its box is the picture's.
|
||||
self.assertGreater(float(box(FACE, H, W)[face["y"], face["x"]]), 0)
|
||||
for got, want in zip(face["box"], helpers.box_of(box(FACE, H, W) > 0.5)):
|
||||
self.assertLessEqual(abs(got - want), 6)
|
||||
self.assertAlmostEqual(face["share"], float(box(FACE, H, W).mean()), delta=0.01)
|
||||
# The face picks it as a named touch at that point: nothing is asked again.
|
||||
before = len(SizedSam.calls)
|
||||
out = self.run_node(self.touches({"t": "name", "op": 1, "w": "face", "x": face["x"], "y": face["y"]}), ask="look")
|
||||
self.assertEqual(len(SizedSam.calls), before)
|
||||
self.assertGreater(same_area(out["result"][0][0] > 0.5, box(FACE, H, W)), 0.9)
|
||||
# A point piece is remembered under the picture's own pixel as well.
|
||||
spot = next((piece for piece in note["pieces"]["list"] if not piece["name"]), None)
|
||||
if spot is not None:
|
||||
self.run_node(self.touches({"t": "tap", "op": 1, "x": spot["x"], "y": spot["y"]}), ask="look")
|
||||
self.assertEqual(len(SizedSam.calls), before)
|
||||
|
||||
def test_a_small_picture_is_asked_about_as_it_is(self):
|
||||
small = torch.rand(1, 50, 40, 3) * 0.9
|
||||
out = self.run_node(self.touches({"t": "tap", "op": 1, "x": 20, "y": 11}), image=small)
|
||||
self.assertEqual([call["size"] for call in SizedSam.calls], [(50, 40)])
|
||||
self.assertTrue(torch.equal(out["result"][0][0] > 0.5, box(FACE, 50, 40) > 0.5), "exactly what SAM 3 answered")
|
||||
|
||||
def test_paint_stays_exactly_as_painted_on_the_big_picture(self):
|
||||
stroke = [[20, 20], [200, 20]]
|
||||
out = self.run_node(self.touches({"t": "paint", "op": 1, "r": 5, "pts": stroke}))
|
||||
wanted = helpers.stroke_area(tuple(tuple(point) for point in stroke), 5.0, H, W)
|
||||
self.assertTrue(torch.equal(out["result"][0][0] > 0.5, wanted))
|
||||
self.assertEqual(SizedSam.calls, [], "paint asks SAM 3 nothing")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,79 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
if "nodes" in sys.modules and not hasattr(sys.modules["nodes"], "__path__"):
|
||||
del sys.modules["nodes"]
|
||||
|
||||
from nodes._single_copy import other_copy_that_loads
|
||||
|
||||
|
||||
def make_copy(root: Path, name: str, version: str, guarded: bool = True, pack: str = "ausboss-nodes") -> str:
|
||||
folder = root / name
|
||||
(folder / "nodes").mkdir(parents=True)
|
||||
(folder / "__init__.py").write_text("")
|
||||
(folder / "pyproject.toml").write_text(f'[project]\nname = "{pack}"\nversion = "{version}"\n')
|
||||
if guarded:
|
||||
(folder / "nodes" / "_single_copy.py").write_text("")
|
||||
return str(folder)
|
||||
|
||||
|
||||
def loads(folder: str, root: Path) -> bool:
|
||||
return other_copy_that_loads(folder, [str(root)]) is None
|
||||
|
||||
|
||||
class SingleCopyTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory()
|
||||
self.root = Path(self._tmp.name).resolve()
|
||||
|
||||
def tearDown(self):
|
||||
self._tmp.cleanup()
|
||||
|
||||
def test_a_lone_copy_loads(self):
|
||||
a = make_copy(self.root, "ComfyUI-AusBoss", "2.24.0")
|
||||
make_copy(self.root, "some-other-pack", "9.0.0", pack="other")
|
||||
self.assertTrue(loads(a, self.root))
|
||||
|
||||
def test_exactly_one_of_two_copies_loads_and_it_is_the_newer(self):
|
||||
a = make_copy(self.root, "ComfyUI-AusBoss", "2.24.0")
|
||||
b = make_copy(self.root, "ausboss-nodes", "2.25.0")
|
||||
self.assertFalse(loads(a, self.root))
|
||||
self.assertTrue(loads(b, self.root))
|
||||
self.assertEqual(other_copy_that_loads(a, [str(self.root)])["version"], "2.25.0")
|
||||
|
||||
def test_same_version_still_picks_one(self):
|
||||
a = make_copy(self.root, "ComfyUI-AusBoss", "2.24.0")
|
||||
b = make_copy(self.root, "ausboss-nodes", "2.24.0")
|
||||
self.assertEqual([loads(a, self.root), loads(b, self.root)].count(True), 1)
|
||||
|
||||
def test_yields_to_an_older_copy_that_cannot_step_aside(self):
|
||||
new = make_copy(self.root, "ComfyUI-AusBoss", "2.24.0")
|
||||
make_copy(self.root, "ausboss-nodes", "2.22.0", guarded=False)
|
||||
self.assertFalse(loads(new, self.root))
|
||||
|
||||
def test_a_disabled_copy_does_not_count(self):
|
||||
a = make_copy(self.root, "ComfyUI-AusBoss", "2.24.0")
|
||||
make_copy(self.root, "ausboss-nodes.disabled", "2.25.0")
|
||||
self.assertTrue(loads(a, self.root))
|
||||
|
||||
def test_a_link_to_the_same_folder_is_one_copy(self):
|
||||
a = make_copy(self.root, "ComfyUI-AusBoss", "2.24.0")
|
||||
(self.root / "link").symlink_to(a)
|
||||
self.assertTrue(loads(a, self.root))
|
||||
self.assertTrue(loads(str(self.root / "link"), self.root))
|
||||
|
||||
def test_ten_is_newer_than_nine(self):
|
||||
a = make_copy(self.root, "ComfyUI-AusBoss", "2.9.0")
|
||||
b = make_copy(self.root, "ausboss-nodes", "2.10.0")
|
||||
self.assertFalse(loads(a, self.root))
|
||||
self.assertTrue(loads(b, self.root))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,100 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
if "nodes" in sys.modules and not hasattr(sys.modules["nodes"], "__path__"):
|
||||
del sys.modules["nodes"]
|
||||
|
||||
from nodes._text_lines_helpers import split_items, strip_marker
|
||||
from nodes.node_text_lines import AusBossTextLines
|
||||
|
||||
|
||||
class SplitItemsTests(unittest.TestCase):
|
||||
def test_each_line_is_an_item_and_empty_lines_are_skipped(self):
|
||||
text = "a red car\n\n \na blue boat \r\n\tgreen hills\n"
|
||||
self.assertEqual(split_items(text), ["a red car", "a blue boat", "green hills"])
|
||||
|
||||
def test_blank_line_keeps_lines_together(self):
|
||||
text = "first line\nsecond line\n\n\nnext item\n \nlast"
|
||||
self.assertEqual(
|
||||
split_items(text, "blank line"),
|
||||
["first line\nsecond line", "next item", "last"],
|
||||
)
|
||||
|
||||
def test_limit_keeps_the_first_items(self):
|
||||
self.assertEqual(split_items("a\nb\nc\nd", limit=2), ["a", "b"])
|
||||
self.assertEqual(split_items("a\nb", limit=0), ["a", "b"])
|
||||
self.assertEqual(split_items("a\nb", limit=9), ["a", "b"])
|
||||
|
||||
def test_numbers_and_bullets_come_off(self):
|
||||
cases = {
|
||||
"1. close-up, soft smile": "close-up, soft smile",
|
||||
"2) medium shot": "medium shot",
|
||||
"#3: from above": "from above",
|
||||
"- a bullet": "a bullet",
|
||||
"* a star": "a star",
|
||||
"• a dot": "a dot",
|
||||
"[prompt 4] from the side": "from the side",
|
||||
"【prompt_5】 looking back": "looking back",
|
||||
"(6) arms crossed": "arms crossed",
|
||||
"Prompt 7: laughing": "laughing",
|
||||
"Shot 8 - wide": "wide",
|
||||
"12. twelve": "twelve",
|
||||
}
|
||||
for line, want in cases.items():
|
||||
self.assertEqual(strip_marker(line), want, line)
|
||||
|
||||
def test_text_that_only_starts_like_a_marker_stays(self):
|
||||
for line in [
|
||||
"2 women on a beach",
|
||||
"1.5x zoom on the face",
|
||||
"10:30 at night, a diner",
|
||||
"3D render of a robot",
|
||||
"*bold* light from the left",
|
||||
"-90 degrees, seen from below",
|
||||
"1920s street, a man in a hat",
|
||||
"shot from below",
|
||||
"Image 1: the woman, in the coat from image 2",
|
||||
]:
|
||||
self.assertEqual(strip_marker(line), line, line)
|
||||
|
||||
def test_numbering_stays_when_asked(self):
|
||||
self.assertEqual(split_items("1. a\n- b", remove_numbering=False), ["1. a", "- b"])
|
||||
|
||||
def test_a_line_that_is_only_a_marker_is_skipped(self):
|
||||
self.assertEqual(split_items("1.\n2. real\n-"), ["real"])
|
||||
|
||||
def test_nothing_gives_nothing(self):
|
||||
self.assertEqual(split_items(""), [])
|
||||
self.assertEqual(split_items(None), [])
|
||||
self.assertEqual(split_items("\n \n"), [])
|
||||
|
||||
|
||||
class TextLinesNodeTests(unittest.TestCase):
|
||||
def test_outputs_lines_numbers_and_count(self):
|
||||
lines, numbers, count = AusBossTextLines().split("1. one\n2. two\n\n3. three")
|
||||
self.assertEqual(lines, ["one", "two", "three"])
|
||||
self.assertEqual(numbers, [1, 2, 3])
|
||||
self.assertEqual(count, 3)
|
||||
|
||||
def test_the_line_and_number_outputs_are_lists(self):
|
||||
self.assertEqual(AusBossTextLines.OUTPUT_IS_LIST, (True, True, False))
|
||||
self.assertEqual(AusBossTextLines.RETURN_NAMES, ("line", "number", "count"))
|
||||
|
||||
def test_no_lines_stops_the_run_with_plain_words(self):
|
||||
with self.assertRaises(ValueError) as caught:
|
||||
AusBossTextLines().split(" \n")
|
||||
self.assertIn("no lines to run", str(caught.exception))
|
||||
|
||||
def test_inputs_and_their_order(self):
|
||||
inputs = AusBossTextLines.INPUT_TYPES()
|
||||
self.assertEqual(list(inputs), ["required"])
|
||||
self.assertEqual(list(inputs["required"]), ["text", "split", "remove_numbering", "limit"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -60,6 +60,7 @@ import {
|
||||
thinStroke,
|
||||
touchFits,
|
||||
} from "../js/shared/touch_math.mjs";
|
||||
import { OUT_OF_MEMORY, goesBehind as goesBehindAgain, piecesHolding, stopMessage, typedName } from "../js/shared/touch_math.mjs";
|
||||
import {
|
||||
GROUP_MAX,
|
||||
areaKey,
|
||||
@@ -1062,3 +1063,68 @@ test("picks the node moved onto a new picture are taken only while nothing was p
|
||||
assert.equal(movedTouches({ moved: { from: 7, touches: after } }, before), null, "a note that is not this shape moves nothing");
|
||||
assert.equal(movedTouches(null, before), null);
|
||||
});
|
||||
|
||||
test("taking away from a spot that is not picked never offers a thing that holds a whole pick", () => {
|
||||
const { width, masks, planes, sizes, ranks } = scene();
|
||||
// The jacket is picked. The pointer is on the person's legs, which are not.
|
||||
const inside = piecesInside(planes, 5, masks.jacket);
|
||||
const holds = piecesHolding(planes, 5, [masks.jacket]);
|
||||
assert.deepEqual([...holds], [0, 1, 1, 1, 1], "the jacket, the dress over it, the person and the wall hold all of it");
|
||||
const legs = 32 * width + 14;
|
||||
assert.equal(pieceAt(planes, legs, { sizes, inside, ranks, adding: false }), 3, "as before, when nothing is said about the spot");
|
||||
assert.equal(pieceAt(planes, legs, { sizes, inside, ranks, adding: false, here: false, holds }), -1, "not the person: the jacket would go with it");
|
||||
// On the jacket itself the spot is picked: the jacket is offered.
|
||||
const onJacket = 14 * width + 14;
|
||||
assert.equal(pieceAt(planes, onJacket, { sizes, inside, ranks, adding: false, here: true, holds }), 1);
|
||||
// The person is picked, and the pointer is on the jacket: the jacket is a part of the pick, and is offered.
|
||||
const person = piecesInside(planes, 5, masks.person);
|
||||
const personHolds = piecesHolding(planes, 5, [masks.person]);
|
||||
assert.equal(pieceAt(planes, onJacket, { sizes, inside: person, ranks, adding: false, here: true, holds: personHolds }), 1);
|
||||
// A thing a pick only laps over is still offered from a spot that is not picked.
|
||||
const lap = Uint8Array.from(masks.person, (value, index) => (value && Math.floor(index / width) >= 30 ? 1 : 0));
|
||||
const lapInside = piecesInside(planes, 5, lap);
|
||||
const lapHolds = piecesHolding(planes, 5, [masks.wall]);
|
||||
assert.equal(lapHolds[3], 0, "the person does not hold a pick of the wall");
|
||||
assert.equal(pieceAt(planes, onJacket + 0, { sizes, inside: lapInside, ranks, adding: false, here: false, holds: lapHolds }), 3, "the person has picked area in it and holds no whole pick");
|
||||
// Adding is not changed by any of it.
|
||||
assert.equal(pieceAt(planes, legs, { sizes, inside, ranks, here: false, holds }), 3);
|
||||
assert.deepEqual([...piecesHolding(planes, 5, [null, new Uint8Array(masks.jacket.length)])], [0, 0, 0, 0, 0], "no area, nothing held");
|
||||
});
|
||||
|
||||
test("a typed name is added or taken away by the sign, and a sign in the text wins", () => {
|
||||
assert.deepEqual(typedName("face"), { word: "face", add: true });
|
||||
assert.deepEqual(typedName("face", true), { word: "face", add: false });
|
||||
assert.deepEqual(typedName("-face"), { word: "face", add: false });
|
||||
assert.deepEqual(typedName(" - fur collar "), { word: "fur collar", add: false });
|
||||
assert.deepEqual(typedName("\u2212hair"), { word: "hair", add: false }, "the minus a phone types");
|
||||
assert.deepEqual(typedName("+hat", true), { word: "hat", add: true });
|
||||
assert.deepEqual(typedName("t-shirt", false), { word: "t-shirt", add: true }, "a dash inside a word is the word's");
|
||||
assert.deepEqual(typedName("-"), { word: "", add: false });
|
||||
assert.deepEqual(typedName(null), { word: "", add: true });
|
||||
});
|
||||
|
||||
test("looks at a picture the node no longer has are taken out of the queue before the next one is sent", () => {
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const stage = readFileSync(join(here, "..", "js", "shared", "touch_stage.mjs"), "utf8");
|
||||
const entry = readFileSync(join(here, "..", "js", "mask_by_touch", "index.js"), "utf8");
|
||||
const run = stage.slice(stage.indexOf("function requestRun"), stage.indexOf("function settle(face)"));
|
||||
// the stale ones are counted before this run joins them, and dropped before it is sent
|
||||
assert.ok(run.indexOf("const stale = face.flying.filter(") < run.indexOf("face.flying.push(run)"));
|
||||
assert.match(run, /other\.queued && other\.epoch !== run\.epoch/);
|
||||
assert.ok(run.indexOf("face.host.dropWaiting()") < run.indexOf("face.host.run("));
|
||||
// only this node's own runs (they carry an `ask`) that still wait: never the one that runs, never a workflow run
|
||||
const drop = entry.slice(entry.indexOf("async function dropWaiting"), entry.indexOf("// The picture goes above the settings"));
|
||||
assert.match(drop, /queue_pending/);
|
||||
assert.doesNotMatch(drop, /queue_running|interrupt/);
|
||||
assert.match(drop, /class_type === NODE_CLASS && String\(step\.inputs\?\.ask \?\? ""\) !== ""/);
|
||||
// a look still goes behind looks at an older picture, as before
|
||||
assert.equal(goesBehindAgain([{ ask: "scope", queued: true, epoch: 1 }], "scope", 2), true);
|
||||
});
|
||||
|
||||
test("a full graphics card is said in one plain line, anything else as it came", () => {
|
||||
const comfy = "Allocation on device 0 would exceed allowed memory. (out of memory)\nCurrently allocated : 2.07 GiB\nRequested : 40.50 MiB";
|
||||
assert.equal(stopMessage(comfy), OUT_OF_MEMORY);
|
||||
assert.equal(stopMessage("torch.OutOfMemoryError: CUDA"), OUT_OF_MEMORY);
|
||||
assert.equal(stopMessage("Nothing is picked on this picture."), "Nothing is picked on this picture.");
|
||||
assert.equal(stopMessage(null), "");
|
||||
});
|
||||
|
||||
@@ -39,7 +39,7 @@ test("a stopped run's message stays up until the next result, then goes", () =>
|
||||
const { face, setNote, shown } = noteFace();
|
||||
const stopped = SOURCE.match(/face\.stopped = \(message\) => \{([\s\S]*?)\n {2}\};/)?.[1];
|
||||
assert.ok(stopped, "touch_stage.mjs has no face.stopped");
|
||||
assert.match(stopped, /setNote\(face, String\(message \?\? ""\)\);\s*(?:\/\/[^\n]*\n\s*)*face\.stopNote = true;/,
|
||||
assert.match(stopped, /setNote\(face, stopMessage\(message\)\);\s*(?:\/\/[^\n]*\n\s*)*face\.stopNote = true;/,
|
||||
"a stop marks its message as a stop message, after it is set");
|
||||
// What face.stopped does to the note.
|
||||
setNote(face, "Mask by Touch: nothing is picked.");
|
||||
|
||||
Reference in New Issue
Block a user