* Support for attention merge at the wildcard/choice boundary.
Co-authored-by: Copilot <copilot@github.com>
This commit is contained in:
co-authored by
Copilot
parent
b03a522ed4
commit
db68347bcd
+2
-2
@@ -118,7 +118,7 @@ Options for cleanup processing, in case you want to change them from the default
|
||||
* **around_ands**: Removes consecutive ANDs and unnecessary commas and space around them.
|
||||
* **ands_with_eol**: Add a newline before ANDs.
|
||||
* **around_extranetwork_tags**: Removes spaces around extra network tags.
|
||||
* **merge_attention**: It merges attention modifiers when possible (merges into one, multiplying their values). Only merges individually nested modifiers.
|
||||
* **merge_attention**: It merges attention modifiers when possible (merges into one, multiplying their values). Only merges individually nested attention (even through choice/wildcard boundaries).
|
||||
* **remove_extranetwork_tags**: Removes all extra network tags.
|
||||
|
||||
Please note that *ComfyUI* does not natively support the `BREAK` and `AND` constructs, but the related settings are kept in that UI in case you use a node that supports them and the extension is configured to allow them (see the configuration file below).
|
||||
@@ -179,6 +179,6 @@ Options for extranetworks mapping, in case you want to change them from the defa
|
||||
* **Clean up around ANDs**: Removes consecutive ANDs and unnecessary commas and space around them.
|
||||
* **Use EOL instead of Space before ANDs**: Add a newline before ANDs.
|
||||
* **Clean up around extra network tags**: Removes spaces around extra network tags.
|
||||
* **Merge attention modifiers (weights) when possible**: It merges attention modifiers when possible (merges into one, multiplying their values). Only merges individually nested modifiers.
|
||||
* **Merge attention modifiers (weights) when possible**: It merges attention modifiers when possible (merges into one, multiplying their values). Only merges individually nested attention (even through choice/wildcard boundaries).
|
||||
* **Remove extra spaces**: Removes other unnecessary spaces.
|
||||
* **Remove extra network tags**: Removes all extra network tags.
|
||||
|
||||
+14
-4
@@ -411,11 +411,11 @@ Only choices labelled `fantasy` across all matched files are eligible.
|
||||
|
||||
Note: if no parameters are specified in the glob call, the parameters from the first matching file that defines them (sorted by key) are used. To avoid that, specify parameters explicitly in the call.
|
||||
|
||||
## Prefix/suffix on wildcard parameters
|
||||
## Prefix/suffix or container on wildcard parameters
|
||||
|
||||
Using the object format for wildcard parameters you can add a prefix and/or suffix that wrap every result. This is cleaner than repeating the wrapper in each choice.
|
||||
Using the object format for wildcard parameters you can set a prefix and/or suffix, or a container, that wraps the result. This is cleaner than repeating the wrapper in each choice.
|
||||
|
||||
Without prefix/suffix, every choice needs to repeat the attention modifier:
|
||||
In this example, without prefix/suffix, every choice needs to repeat the attention modifier:
|
||||
|
||||
```yaml
|
||||
qualities:
|
||||
@@ -434,7 +434,17 @@ qualities:
|
||||
- "intricate details"
|
||||
```
|
||||
|
||||
The prefix and suffix are added around the joined result (including the separator when multiple choices are selected). They can themselves contain constructs.
|
||||
The prefix and suffix are added around the wildcard's result. They can themselves contain constructs.
|
||||
|
||||
Another way is with the container property, which does the same in a slightly more flexible way:
|
||||
|
||||
```yaml
|
||||
qualities:
|
||||
- { container: "(${__choices[]}:1.3)" } # parameters line
|
||||
- "ultra detailed"
|
||||
- "highly detailed"
|
||||
- "intricate details"
|
||||
```
|
||||
|
||||
## `ifundefined` / `?=` for safe defaults
|
||||
|
||||
|
||||
+2
-2
@@ -155,14 +155,14 @@ Wildcard parameters in a json/yaml file can also be in object format, and suppor
|
||||
* `container`: includes the prefix, choices array variable, and suffix
|
||||
|
||||
```yaml
|
||||
{ sampler: "~", repeating: false, optional: false, from: 2, to: 3, description: "test wildcard", container: "prefix-${_choices[&'/']}-suffix" }
|
||||
{ sampler: "~", repeating: false, optional: false, from: 2, to: 3, description: "test wildcard", container: "prefix-${__choices[&'/']}-suffix" }
|
||||
{ sampler: "~", repeating: false, optional: false, count: 2, description: "test wildcard", prefix: "prefix-", suffix: "-suffix", separator: "/" }
|
||||
{ sampler: "~", repeating: false, optional: false, from: 2, to: 3, description: "test wildcard", prefix: "prefix-", suffix: "-suffix", separator: "/" }
|
||||
```
|
||||
|
||||
The prefix and suffix are added to the result along with the selected choices and separators. They can contain other constructs, but the separator can't.
|
||||
|
||||
The container is a new option that replaces prefix/suffix/separator, and makes use of the recent support for array variables. Its value would be the concatenation of any prefix and/or suffix with the echoing (with the chosen separator) of a temporary `_choices[]` variable that holds the chosen values. This property is preferred over prefix/suffix/separator unless you only need the separator.
|
||||
The container is a new option that replaces prefix/suffix/separator, and makes use of the recent support for array variables. Its value would be the concatenation of any prefix and/or suffix with the echoing (with the chosen separator) of a temporary `__choices[]` variable that holds the chosen values. This property is preferred over prefix/suffix/separator unless you only need the separator.
|
||||
|
||||
It is recommended to use the object format for the wildcard parameters and for choices with complex options.
|
||||
|
||||
|
||||
+112
@@ -933,6 +933,85 @@ class TreeProcessor(lark.visitors.Interpreter):
|
||||
t2 = time.monotonic_ns()
|
||||
self.__debug_end("alternate", start_result, t2 - t1)
|
||||
|
||||
@staticmethod
|
||||
def _try_extract_attention(s: str) -> tuple[str, float] | None:
|
||||
"""
|
||||
If s is entirely a single attention wrapper - (inner), (inner:W), or [inner] -
|
||||
return (inner_content, weight). Otherwise return None.
|
||||
|
||||
Used for post-visit merging when a wildcard or choices construct expands to a
|
||||
single attention that can be merged with an enclosing outer attention.
|
||||
"""
|
||||
if not s:
|
||||
return None
|
||||
open_char = s[0]
|
||||
if open_char == "(":
|
||||
close_char = ")"
|
||||
elif open_char == "[":
|
||||
close_char = "]"
|
||||
else:
|
||||
return None
|
||||
depth = 0
|
||||
for i, c in enumerate(s):
|
||||
if c == open_char:
|
||||
depth += 1
|
||||
elif c == close_char:
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
if i != len(s) - 1:
|
||||
return None # wrapper closes before end of string -> multiple items
|
||||
break
|
||||
else:
|
||||
return None # never fully closed
|
||||
inner = s[1:-1]
|
||||
if open_char == "[":
|
||||
# Disambiguate from alternation [a|b] and scheduling [before:after:N].
|
||||
# Both use [...] but are not attention constructs.
|
||||
paren_depth = 0
|
||||
bracket_depth = 0
|
||||
top_level_pipes = 0
|
||||
top_level_colons = 0
|
||||
last_colon_pos = -1
|
||||
for i, c in enumerate(inner):
|
||||
if c == "(":
|
||||
paren_depth += 1
|
||||
elif c == ")":
|
||||
paren_depth -= 1
|
||||
elif c == "[":
|
||||
bracket_depth += 1
|
||||
elif c == "]":
|
||||
bracket_depth -= 1
|
||||
elif paren_depth == 0 and bracket_depth == 0:
|
||||
if c == "|":
|
||||
top_level_pipes += 1
|
||||
elif c == ":":
|
||||
top_level_colons += 1
|
||||
last_colon_pos = i
|
||||
if top_level_pipes > 0:
|
||||
return None # alternation construct
|
||||
if top_level_colons >= 2 and last_colon_pos >= 0:
|
||||
try:
|
||||
float(inner[last_colon_pos + 1 :])
|
||||
return None # scheduling construct: [before:after:N]
|
||||
except ValueError:
|
||||
pass
|
||||
return (inner, 0.9)
|
||||
# Parenthesis form - scan backwards for a top-level :weight suffix
|
||||
depth = 0
|
||||
for i in range(len(inner) - 1, -1, -1):
|
||||
c = inner[i]
|
||||
if c in ")]":
|
||||
depth += 1
|
||||
elif c in "([":
|
||||
depth -= 1
|
||||
elif c == ":" and depth == 0:
|
||||
try:
|
||||
w = float(inner[i + 1 :])
|
||||
return (inner[:i], w)
|
||||
except ValueError:
|
||||
break
|
||||
return (inner, 1.1)
|
||||
|
||||
def attention(self, tree: lark.Tree):
|
||||
"""
|
||||
Process a attention change construct in the tree and add it to the accumulated shell.
|
||||
@@ -956,6 +1035,7 @@ class TreeProcessor(lark.visitors.Interpreter):
|
||||
self.log(logging.DEBUG, f"Shell attention with weight {weight}")
|
||||
current_tree = tree.children[0]
|
||||
if self.state.options.cup_merge_attention:
|
||||
# we check while the children are attentions, in which case we merge the weights
|
||||
while isinstance(current_tree, lark.Tree) and current_tree.data == "attention":
|
||||
# we merge the weights
|
||||
if len(current_tree.children) == 2:
|
||||
@@ -967,6 +1047,10 @@ class TreeProcessor(lark.visitors.Interpreter):
|
||||
else:
|
||||
inner_weight = 0.9
|
||||
weight *= inner_weight
|
||||
self.log(
|
||||
logging.DEBUG,
|
||||
f"Merging nested attention with weight {inner_weight}, cumulative weight now {weight}",
|
||||
)
|
||||
current_tree = current_tree.children[0]
|
||||
weight = math.floor(weight * 100) / 100 # we round to 2 decimals
|
||||
weight_str = f"{weight:.2f}".rstrip("0").rstrip(".")
|
||||
@@ -1014,6 +1098,34 @@ class TreeProcessor(lark.visitors.Interpreter):
|
||||
self.__result += starttag
|
||||
self.__visit(current_tree)
|
||||
endtag = f":{weight_str})"
|
||||
# Post-visit merge: if the entire visited content is a single attention wrapper
|
||||
# (e.g. from a wildcard or choices expansion), merge weights here.
|
||||
# The static tree-walk above only covers direct attention children; this
|
||||
# handles the case where the inner attention came from an expanded wildcard.
|
||||
if self.state.options.cup_merge_attention:
|
||||
visited_content = self.__result[len(start_result) + len(starttag) :]
|
||||
merge = TreeProcessor._try_extract_attention(visited_content)
|
||||
if merge is not None:
|
||||
inner_content, inner_weight = merge
|
||||
weight = math.floor(weight * inner_weight * 100) / 100
|
||||
self.log(
|
||||
logging.DEBUG,
|
||||
f"Merging nested attention with weight {inner_weight}, cumulative weight now {weight}",
|
||||
)
|
||||
weight_str = f"{weight:.2f}".rstrip("0").rstrip(".")
|
||||
if weight_str == "1.1":
|
||||
weight_kind = 2
|
||||
starttag = "("
|
||||
endtag = ")"
|
||||
elif weight_str == "0.9" and attention_processing != "parentheses":
|
||||
weight_kind = 1
|
||||
starttag = "["
|
||||
endtag = "]"
|
||||
else:
|
||||
weight_kind = 3
|
||||
starttag = "("
|
||||
endtag = f":{weight_str})"
|
||||
self.__result = start_result + starttag + inner_content
|
||||
if self.state.options.cup_empty_constructs and re.fullmatch(
|
||||
re.escape(start_result + starttag) + r"\s*", self.__result
|
||||
):
|
||||
|
||||
@@ -1064,3 +1064,9 @@ class TestVarCommands(TestPromptPostProcessorBase):
|
||||
self.extranetwork_maps_obj,
|
||||
),
|
||||
)
|
||||
|
||||
def test_var_attention_merge(self): # attention merge at variable boundary
|
||||
self.process(
|
||||
InputTuple("${v!=[content]}(${v}:1.5)", ""),
|
||||
OutputTuple("(content:1.35)", ""),
|
||||
)
|
||||
|
||||
@@ -386,19 +386,37 @@ class TestWildcards(TestPromptPostProcessorBase):
|
||||
def test_wc_wildcardPS2_yaml(self): # yaml wildcard with object formatted choices and options and prefix and suffix
|
||||
self.process(
|
||||
InputTuple("the choices are: [__yaml/wildcardPS2__]", ""),
|
||||
OutputTuple("the choices are: [(prefix2-choice2-suffix:1.5)]", ""),
|
||||
OutputTuple("the choices are: (prefix2-choice2-suffix:1.35)", ""),
|
||||
)
|
||||
|
||||
def test_wc_wildcardContainer_yaml(self): # yaml wildcard with object formatted choices and options and container
|
||||
self.process(
|
||||
InputTuple("the choices are: [__yaml/wildcardContainer__]", ""),
|
||||
OutputTuple("the choices are: [(prefix1-choice2/choice3-suffix:1.5)]", ""),
|
||||
OutputTuple("the choices are: (prefix1-choice2/choice3-suffix:1.35)", ""),
|
||||
)
|
||||
|
||||
def test_wc_wildcardAt_yaml(self): # yaml wildcard with attention in choices
|
||||
self.process(
|
||||
InputTuple("the choices are: [__yaml/wildcardAt__]", ""),
|
||||
OutputTuple("the choices are: [(choice2:1.5)]", ""),
|
||||
OutputTuple("the choices are: (choice2:1.35)", ""),
|
||||
)
|
||||
|
||||
def test_wc_merge_attention_bracket(self): # bracket attention from wildcard merges with outer attention
|
||||
self.process(
|
||||
InputTuple("(__yaml/wildcardAtBracket__:1.5)", ""),
|
||||
OutputTuple("(the content:1.35)", ""),
|
||||
)
|
||||
|
||||
def test_wc_no_merge_attention_alternation(self): # alternation from wildcard is not merged as attention
|
||||
self.process(
|
||||
InputTuple("(__yaml/wildcardAlt__:1.5)", ""),
|
||||
OutputTuple("([cat|dog]:1.5)", ""),
|
||||
)
|
||||
|
||||
def test_wc_no_merge_attention_scheduling(self): # scheduling from wildcard is not merged as attention
|
||||
self.process(
|
||||
InputTuple("(__yaml/wildcardSched__:1.5)", ""),
|
||||
OutputTuple("([cat:dog:0.5]:1.5)", ""),
|
||||
)
|
||||
|
||||
def test_wc_anonymouswildcard_yaml(self): # yaml anonymous wildcard
|
||||
|
||||
@@ -42,6 +42,15 @@ yaml:
|
||||
- (choice2:1.5)
|
||||
- (choice3:1)
|
||||
|
||||
wildcardAtBracket:
|
||||
- "[the content]"
|
||||
|
||||
wildcardAlt:
|
||||
- "[cat|dog]"
|
||||
|
||||
wildcardSched:
|
||||
- "[cat:dog:0.5]"
|
||||
|
||||
wildcardPS:
|
||||
- {
|
||||
sampler: "~",
|
||||
|
||||
Reference in New Issue
Block a user