nodes/acorderob-sd-webui-prompt-postprocessor
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

* Added new configuration options in ppp_config.yaml.defaults (and ppp_config.yaml) for host applications and models.

Browse Source 
* Implemented handling for attention, scheduling, alternation, AND, and BREAK operators in the new configuration file.
* Moved variant definitions to the new configuration file.
* Improved tests speed by creating the needed PPP object in the process method.
This commit is contained in:
Antonio Cordero Balcazar
2026-03-28 16:38:53 +01:00
parent 39d6008f6f
commit 87d12ca53f
7 changed files with 942 additions and 361 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -36,7 +36,7 @@ Notes:
Does not recognize tokenizer separators like `TE2:` and `TE3:`, so sending to negative prompt from those sections of the prompt will not add them in the corresponding section of the negative prompt.
*ComfyUI* only supports natively the attention using parenthesis, so the ones with the braces will be converted. The other constructs are not natively supported but some custom nodes implement them.
*ComfyUI* only supports natively the attention using parentheses, so the ones with the braces will be converted. The other constructs are not natively supported but some custom nodes implement them.
2. It recognizes wildcards in the `__wildcard__` and {choice|choice} formats (and almost everything that [Dynamic Prompts](https://github.com/adieyal/sd-dynamic-prompts) supports).
3. It does not create *AND/BREAK* constructs when moving content to the negative prompt.
docs/CONFIG.md
+93 -50
View File
@@ -1,84 +1,127 @@
# Prompt PostProcessor configuration
## ComfyUI specific (ACB Prompt Post Processor node)
## Configuration file
### Inputs
The extension supports a configuration file `ppp_config.yaml` with some settings that don't usually change.
By default this configuration is read from the file `ppp_config.yaml.defaults` in the extension folder (that file must not be modified). If you want to personalize any settings you should first copy this file as `ppp_config.yaml` in the same folder, or in the ComfyUI user folder (preferred, but only possible in ComfyUI). The options in this file will take precedence over those in the defaults file.
This file contains some options for how the host applications (WebUIs) should act in certain operations, and also options for the supported models, which currently include only the model variants definitions. Hosts and models names are fixed values (those supported by the extension). The defaults file contains comments to explain the available options.
The model variants now support regular expressions instead of a list of strings to detect the variant. If you used a non default value in previous versions you should re-do it in this file with the new format. As before, the default defines variants for *Pony* and *Illustrious* models.
## ComfyUI
### ACB Prompt Post Processor node inputs
* **model**: Connect here the MODEL or a string with the model class name used by *ComfyUI*. Needed for the model kind system variables.
* **modelname**: Name of the model. Needed for the model name system variables and detection of pony (this also requieres for the model to be SDXL).
* **seed**: Connect here the seed used. By default it is -1 (random).
* **pos_prompt**: Connect here the prompt text, or fill it as a widget.
* **neg_prompt**: Connect here the negative prompt text, or fill it as a widget.
* **wc_wildcards_input**: Wildcards definitions (in yaml or json format). Direct input added to the ones found in the wildcards folders. Allows wildcards to be included in the workflow.
* **en_mappings_input**: Extranetwork Mappings definitions (in yaml format). Direct input added to the ones found in the extranetwork mappings folders. Allows the mappings to be included in the workflow.
* **debug_level**: What to write to the console.
* **on_warnings**: Warn on the console or stop the generation.
* **process_wildcards**: Activates the wildcard processing.
* **do_cleanup**: Activates the cleanup processing.
* **wc_options**: Connection to a Wildcards options node.
* **stn_options**: Connection to a Send-To-Negative options node.
* **cup_options**: Connection to a Cleanup options node.
* **en_options**: Connection to a ExtraNetworkMapping options node.
Other common settings (see [below](#common-settings)) also appear as inputs or widgets.
The options nodes are optional. If you don't need to change any of the default values then you don't need to use them.
### Outputs
### ACB Prompt Post Processor node outputs
The outputs are the final positive and negative prompt and a variables dictionary.
You can use the "**ACB PPP Select Variable**" node to choose one and output its value. You can use this to send only part of the prompt to, for example, a detailer node. For example:
With this prompt: `__quality__, 1girl, ${head:__eyes__, __hair__, __expression__}, __body__, __clothes__, __background__, __style__` then you extract the `head` variable and send that as prompt for the head/face detailer.
With this prompt: `__quality__, 1girl, ${head=!__eyes__, __hair__, __expression__}${head}, __body__, __clothes__, __background__, __style__` then you extract the `head` variable and use `${head}` as prompt for the head/face detailer.
## A1111 (and compatible UIs) panel options
### ACB PPP Wildcard Options node
* **folders**: You can enter multiple folders separated by commas. You can leave it empty (the default) and add a `ppp_wildcards` or `wildcards` entry in the **extra_model_paths.yaml** file (recommended).
* **definitions**: Wildcards definitions (in yaml or json format). Direct input added to the ones found in the wildcards folders. Allows wildcards to be included in the workflow.
* **if_wildcards**: Select what do you want to do with any found wildcards/choices (when process wildcards is off or after the processing).
* **Ignore**: do not try to detect wildcards.
* **Remove**: detect wildcards and remove them.
* **Add visible warning**: detect wildcards and add a warning text to the prompt, that hopefully produces a noticeable generation.
* **Stop the generation**: detect wildcards and stop the generation.
* **choice_separator**: What do you want to use by default to separate multiple choices when the options allow it (by default it's ', ").
* **keep_choices_order**: If checked, a multiple choice construct will return them in the order they are in the construct.
### ACB PPP Send-To-Negative Options node
* **separator**: You can specify the separator used when adding to the negative prompt (by default it's ", ").
* **ignore_repeats**: It ignores repeated content to avoid repetitions in the negative prompt.
### ACB PPP Cleanup Options node
* **extra_spaces**: Removes other unnecessary spaces.
* **empty_constructs**: Removes attention/scheduling/alternation constructs when they are invalid.
* **extra_separators**: Removes unnecessary separators. This applies to the configured separator and regular commas.
* **extra_separators_additional**: Removes unnecessary separators at start or end of lines. This applies to the configured separator and regular commas.
* **extra_separators_include_eol**: In the previous two options it also removes EOLs attached to the separators.
* **around_breaks**: Removes consecutive BREAKs and unnecessary commas and space around them.
* **breaks_with_eol**: Add a newline before BREAKs.
* **around_ands**: Removes consecutive ANDs and unnecessary commas and space around them.
* **ands_with_eol**: Add a newline before ANDs.
* **around_extranetwork_tags**: Removes spaces around extra network tags.
* **merge_attention**: It merges attention modifiers when possible (merges into one, multiplying their values). Only merges individually nested modifiers.
* **remove_extranetwork_tags**: Removes all extra network tags.
Please note that *ComfyUI* does not natively support the `BREAK` and `AND` constructs, but the related settings are kept in that UI in case you use a node that supports them and the extension is configured to allow them (see the configuration file below).
### ACB PPP ExtraNetwork Mapping Options node
* **folders**: You can enter multiple folders separated by commas. You can leave it empty (the default) and add a `ppp_extranetworkmappings` entry in the **extra_model_paths.yaml** file (recommended).
* **definitions**: Extranetwork Mappings definitions (in yaml format). Direct input added to the ones found in the extranetwork mappings folders. Allows the mappings to be included in the workflow.
## A1111 (and compatible UIs)
### Panel options
* **Force equal seeds**: Changes the image seeds and variation seeds to be equal to the first of the batch. This allows using the same values for all the images in a batch.
* **Unlink seed**: Uses the specified seed for the prompt generation instead of the one from the image. This seed is only used for wildcards and choices.
* **Prompt seed**: The seed to use for the prompt generation. If -1 a random one will be used.
* **Incremental seed**: When using a batch you can use this to set the rest of the prompt seeds with consecutive values.
## Common settings
### General settings
* **Debug level**: what to write to the console. Note: in *SD.Next* debug messages only show if you launch it with the `--debug` argument.
* **What to do on invalid content warnings?**: warn on the console or stop the generation. This also affects integer comparisons with undefined or non-numeric variables: in *warn* mode the comparison evaluates to false, in *stop* mode the generation is stopped with an error.
* **Model variant definitions**: definitions for model variants to be recognized based on strings found in the full filename.
The format for each line is (with *kind* being one of the base model identifiers or not defined):
```name(kind)=comma separated list of substrings (case insensitive)```
The default value defines strings for *Pony* and *Illustrious* models.
* **Apply in img2img**: check if you want to do the processing in img2img processes (*does not apply to the ComfyUI node*).
* **Add original prompts to metadata**: adds original prompts to the metadata if they have changed (*does not apply to the ComfyUI node*).
* **Extranetwork Mappings folders**: you can enter multiple folders separated by commas. In *ComfyUI* you can leave it empty and add a `ppp_extranetworkmappings` entry in the **extra_model_paths.yaml** file.
* **Debug level**: What to write to the console. Note: in *SD.Next* debug messages only show if you launch it with the `--debug` argument.
* **What to do on invalid content warnings?**: Warn on the console or stop the generation. This also affects integer comparisons with undefined or non-numeric variables: in *warn* mode the comparison evaluates to false, in *stop* mode the generation is stopped with an error.
* **Apply in img2img**: Check if you want to do the processing in img2img processes.
* **Add original prompts to metadata**: Adds original prompts to the metadata if they have changed.
* **Extranetwork Mappings folders**: You can enter multiple folders separated by commas.
### Wildcard settings
* **Process wildcards**: you can choose to process wildcards and choices with this extension or use a different one.
* **Wildcards folders**: you can enter multiple folders separated by commas. In *ComfyUI* you can leave it empty and add a `ppp_wildcards` or `wildcards` entry in the **extra_model_paths.yaml** file.
* **What to do with remaining wildcards?**: select what do you want to do with any found wildcards/choices (when process wildcards is off or after the processing).
* **Ignore**: do not try to detect wildcards.
* **Remove**: detect wildcards and remove them.
* **Add visible warning**: detect wildcards and add a warning text to the prompt, that hopefully produces a noticeable generation.
* **Stop the generation**: detect wildcards and stop the generation.
* **Default separator used when adding multiple choices**: what do you want to use by default to separate multiple choices when the options allow it (by default it's ", ").
* **Keep the order of selected choices**: if checked, a multiple choice construct will return them in the order they are in the construct.
* **Process wildcards**: You can choose to process wildcards and choices with this extension or use a different one.
* **Wildcards folders**: You can enter multiple folders separated by commas.
* **What to do with remaining wildcards?**: Select what do you want to do with any found wildcards/choices (when process wildcards is off or after the processing).
* **Ignore**: Do not try to detect wildcards.
* **Remove**: Detect wildcards and remove them.
* **Add visible warning**: Detect wildcards and add a warning text to the prompt, that hopefully produces a noticeable generation.
* **Stop the generation**: Detect wildcards and stop the generation.
* **Default separator used when adding multiple choices**: What do you want to use by default to separate multiple choices when the options allow it (by default it's ", ").
* **Keep the order of selected choices**: If checked, a multiple choice construct will return them in the order they are in the construct.
### Send to negative prompt settings
* **Separator used when adding to the negative prompt**: you can specify the separator used when adding to the negative prompt (by default it's ", ").
* **Ignore repeated content**: it ignores repeated content to avoid repetitions in the negative prompt.
* **Separator used when adding to the negative prompt**: You can specify the separator used when adding to the negative prompt (by default it's ", ").
* **Ignore repeated content**: It ignores repeated content to avoid repetitions in the negative prompt.
### Clean up settings
* **Remove empty constructs**: removes attention/scheduling/alternation constructs when they are invalid.
* **Remove extra separators**: removes unnecessary separators. This applies to the configured separator and regular commas.
* **Remove additional extra separators**: removes unnecessary separators at start or end of lines. This applies to the configured separator and regular commas.
* **The extra separators options also remove EOLs**: in the previous two options it also removes EOLs attached to the separators.
* **Clean up around BREAKs**: removes consecutive BREAKs and unnecessary commas and space around them.
* **Use EOL instead of Space before BREAKs**: add a newline before BREAKs.
* **Clean up around ANDs**: removes consecutive ANDs and unnecessary commas and space around them.
* **Use EOL instead of Space before ANDs**: add a newline before ANDs.
* **Clean up around extra network tags**: removes spaces around them.
* **Merge attention modifiers (weights) when possible**: it merges attention modifiers when possible (merges into one, multiplying their values). Only merges individually nested modifiers.
* **Remove extra spaces**: removes other unnecessary spaces.
Please note that *ComfyUI* does not natively support the `BREAK` and `AND` constructs, but the related settings are kept in that UI.
### Content removal settings
* **Remove extra network tags**: removes all extra network tags.
* **Remove empty constructs**: Removes attention/scheduling/alternation constructs when they are invalid.
* **Remove extra separators**: Removes unnecessary separators. This applies to the configured separator and regular commas.
* **Remove additional extra separators**: Removes unnecessary separators at start or end of lines. This applies to the configured separator and regular commas.
* **The extra separators options also remove EOLs**: In the previous two options it also removes EOLs attached to the separators.
* **Clean up around BREAKs**: Removes consecutive BREAKs and unnecessary commas and space around them.
* **Use EOL instead of Space before BREAKs**: Add a newline before BREAKs.
* **Clean up around ANDs**: Removes consecutive ANDs and unnecessary commas and space around them.
* **Use EOL instead of Space before ANDs**: Add a newline before ANDs.
* **Clean up around extra network tags**: Removes spaces around extra network tags.
* **Merge attention modifiers (weights) when possible**: It merges attention modifiers when possible (merges into one, multiplying their values). Only merges individually nested modifiers.
* **Remove extra spaces**: Removes other unnecessary spaces.
* **Remove extra network tags**: Removes all extra network tags.
ppp.py
+246 -146
View File
@@ -10,6 +10,7 @@ from enum import Enum
from typing import Any, Callable, Optional
import lark
import numpy as np
import yaml
from ppp_hosts import SUPPORTED_APPS # pylint: disable=import-error
from ppp_logging import DEBUG_LEVEL # pylint: disable=import-error
@@ -72,7 +73,6 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
DEFAULT_ONWARNING = ONWARNING_CHOICES.warn.value
DEFAULT_STN_SEPARATOR = ", "
DEFAULT_STN_IGNORE_REPEATS = True
DEFAULT_VARIANTS_DEFINITIONS = "pony(sdxl)=pony,pny,pdxl\nillustrious(sdxl)=illustrious,illust,ilxl"
DEFAULT_WC_PROCESS = True
DEFAULT_IF_WILDCARDS = IFWILDCARDS_CHOICES.stop.value
DEFAULT_CHOICE_SEPARATOR = ", "
@@ -133,29 +133,50 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
self.env_info = env_info
self.wildcard_obj = wildcards_obj
self.extranetwork_mappings_obj = extranetwork_mappings_obj
default_config_file = os.path.join(os.path.dirname(os.path.realpath(__file__)), "ppp_config.yaml.defaults")
with open(default_config_file, "r", encoding="utf-8") as f:
self.config: dict[str, Any] = yaml.safe_load(f)
user_config_file = self.env_info.get("ppp_config", "")
if isinstance(user_config_file, dict):
user_config = user_config_file
else:
user_config = {}
if user_config_file == "":
if self.env_info.get("app", "") == SUPPORTED_APPS.comfyui.value:
try:
import folder_paths # pylint: disable=import-error # type: ignore
user_dir = folder_paths.get_user_directory()
if user_dir and os.path.isdir(user_dir):
user_config_file = os.path.join(user_dir, "default", "ppp_config.yaml")
except Exception: # pylint: disable=broad-exception-caught
pass
if not user_config_file or not os.path.exists(user_config_file):
user_config_file = os.path.join(os.path.dirname(os.path.realpath(__file__)), "ppp_config.yaml")
if user_config_file and os.path.exists(user_config_file):
with open(user_config_file, "r", encoding="utf-8") as f:
user_config = yaml.safe_load(f)
# Merge user config into default config
if len(user_config):
for k in ("hosts", "models"):
if k in user_config:
for key, value in user_config[k].items():
if key in self.config[k] and isinstance(self.config[k][key], list):
self.config[k][key].extend(value)
else:
self.config[k][key] = value
self.host_config: dict[str, Any] = (self.config.get("hosts") or {}).get(self.env_info.get("app", "")) or {}
# General options
self.debug_level = DEBUG_LEVEL(options.get("debug_level", self.DEFAULT_DEBUG_LEVEL))
self.gen_onwarning = self.ONWARNING_CHOICES(options.get("on_warning", self.DEFAULT_ONWARNING))
variants_definitions_option = str(options.get("variants_definitions", self.DEFAULT_VARIANTS_DEFINITIONS))
self.variants_definitions = {}
if variants_definitions_option:
lines = variants_definitions_option.splitlines()
for line in lines:
if "=" in line:
model_tag, elements = line.split("=", 1)
model_name, model_type = re.match(r"(\w+)(?:\((\w+)\))?", model_tag).groups()
if model_type is not None and model_type not in self.SUPPORTED_MODELS:
self.logger.warning(
f"Unsupported model type '{model_type}' in definition for variant '{model_name}'."
)
elif model_name in self.SUPPORTED_MODELS:
self.logger.warning(f"Invalid model name in definition for variant '{model_name}'.")
else:
self.variants_definitions[model_name.strip()] = (
model_type or "",
[element.strip() for element in elements.split(",")],
)
self.variants_definitions = {
v: (m, [vo["find_in_filename"]] if isinstance(vo["find_in_filename"], str) else vo["find_in_filename"])
for m in self.SUPPORTED_MODELS
for v, vo in (((self.config.get("models") or {}).get(m) or {}).get("variants") or {}).items()
}
if self.debug_level != DEBUG_LEVEL.none:
self.logger.debug(self.format_output(f"Host configuration: {self.host_config}"))
# Wildcards options
self.wil_process_wildcards = options.get("process_wildcards", self.DEFAULT_WC_PROCESS)
self.wil_keep_choices_order = options.get("keep_choices_order", self.DEFAULT_KEEP_CHOICES_ORDER)
@@ -430,15 +451,6 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
"""
return text.encode("unicode_escape").decode("utf-8")
def is_comfy_ui(self) -> bool:
"""
Checks if the current environment is ComfyUI.
Returns:
bool: True if the environment is ComfyUI, False otherwise.
"""
return self.env_info.get("app", "") == SUPPORTED_APPS.comfyui.value
def __init_sysvars(self):
"""
Initializes the system variables.
@@ -456,7 +468,7 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
self.system_variables["_modelclass"] = self.env_info.get("model_class", "")
is_models = {
model_name: (model_type_and_substrings[0] == "" or sdchecks.get(model_type_and_substrings[0], False))
and any(s in model_filename.lower() for s in model_type_and_substrings[1])
and any((re.search(s, model_filename, re.IGNORECASE) is not None) for s in model_type_and_substrings[1])
for model_name, model_type_and_substrings in self.variants_definitions.items()
if model_name not in self.SUPPORTED_MODELS
}
@@ -558,12 +570,13 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
negative_prompt = self.stn_separator.join(add_at_end)
return negative_prompt
def __cleanup(self, text: str) -> str:
def __cleanup(self, text: str, is_negative: bool) -> str:
"""
Trims the given text based on the specified cleanup options.
Args:
text (str): The text to be cleaned up.
is_negative (bool): Indicates if the text is a negative prompt.
Returns:
str: The resulting text.
@@ -575,6 +588,43 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
sep_options = [(optwhitespace_separator, self.stn_separator)] # sendtonegative separator
if optwhitespace_comma != optwhitespace_separator:
sep_options.append((optwhitespace_comma, ", ")) # regular comma separator
break_processing = self.host_config.get("break", "ok")
# break_processing == "ok" (and always)
if self.cup_breaks_eol:
# replace spaces before break with EOL
text = re.sub(r"[, ]+BREAK\b", "\nBREAK", text)
if self.cup_breaks:
# collapse separators and commas before BREAK
text = re.sub(r"[, ]+BREAK\b", " BREAK", text)
# collapse separators and commas after BREAK
text = re.sub(r"\bBREAK[, ]+", "BREAK ", text)
# collapse separators and commas around BREAK
text = re.sub(r"[, ]+BREAK[, ]+", " BREAK ", text)
# collapse BREAKs
text = re.sub(r"\bBREAK(?:\s+BREAK)+\b", " BREAK ", text)
# remove spaces between start of line and BREAK
text = re.sub(r"^[ ]+BREAK\b", "BREAK", text, flags=re.MULTILINE)
# remove spaces between BREAK and end of line
text = re.sub(r"\bBREAK[ ]+$", "BREAK", text, flags=re.MULTILINE)
# remove at start of prompt
text = re.sub(r"\A(?:\s*BREAK\b\s*)+", "", text)
# remove at end of prompt
text = re.sub(r"(?:\s*\bBREAK\s*)+\Z", "", text)
if break_processing == "eol":
text2 = re.sub(r"\b\s*BREAK\s*\b", "\n", text)
if text2 != text:
text = text2
if self.debug_level == DEBUG_LEVEL.full:
self.logger.debug("BREAK construct replaced with EOL")
elif break_processing == "remove":
text2 = re.sub(r"\b\s*BREAK\s*\b", " ", text)
if text2 != text:
text = text2
if self.debug_level == DEBUG_LEVEL.full:
self.logger.debug("BREAK construct removed")
elif break_processing == "error":
if re.search(r"\bBREAK\b", text):
self.warn_or_stop(is_negative, "BREAK constructs are not allowed!")
for sep, replacement in sep_options:
if self.cup_extraseparators:
# collapse separators
@@ -596,26 +646,6 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
text = re.sub(r"^(?:" + sep + r")+", "", text, flags=re.MULTILINE)
# remove at end of prompt or line
text = re.sub(r"(?:" + sep + r")+$", "", text, flags=re.MULTILINE)
if self.cup_breaks_eol:
# replace spaces before break with EOL
text = re.sub(r"[, ]+BREAK\b", "\nBREAK", text)
if self.cup_breaks:
# collapse separators and commas before BREAK
text = re.sub(r"[, ]+BREAK\b", " BREAK", text)
# collapse separators and commas after BREAK
text = re.sub(r"\bBREAK[, ]+", "BREAK ", text)
# collapse separators and commas around BREAK
text = re.sub(r"[, ]+BREAK[, ]+", " BREAK ", text)
# collapse BREAKs
text = re.sub(r"\bBREAK(?:\s+BREAK)+\b", " BREAK ", text)
# remove spaces between start of line and BREAK
text = re.sub(r"^[ ]+BREAK\b", "BREAK", text, flags=re.MULTILINE)
# remove spaces between BREAK and end of line
text = re.sub(r"\bBREAK[ ]+$", "BREAK", text, flags=re.MULTILINE)
# remove at start of prompt
text = re.sub(r"\A(?:\s*BREAK\b\s*)+", "", text)
# remove at end of prompt
text = re.sub(r"(?:\s*\bBREAK\s*)+\Z", "", text)
if self.cup_ands:
# collapse ANDs with space after
text = re.sub(r"\bAND(?:\s+AND)+\s+", "AND ", text)
@@ -774,8 +804,8 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
negative_prompt = self.__add_to_end(negative_prompt, p_processor.add_at["end"])
# Clean up
prompt = self.__cleanup(prompt)
negative_prompt = self.__cleanup(negative_prompt)
prompt = self.__cleanup(prompt, False)
negative_prompt = self.__cleanup(negative_prompt, True)
# Check for wildcards not processed
foundP = bool(p_processor.detectedWildcards)
@@ -919,6 +949,15 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
)
return parsed_prompt
def warn_or_stop(self, is_negative: bool, message: str, e: Exception = None):
if self.gen_onwarning == self.ONWARNING_CHOICES.stop:
raise PPPInterrupt(
message,
self.INVALID_CONTENT_STOP.format(message) if not is_negative else "",
self.INVALID_CONTENT_STOP.format(message) if is_negative else "",
) from e
self.logger.warning(message)
class TreeProcessor(lark.visitors.Interpreter):
"""
A class for interpreting and processing a tree generated by the prompt parser.
@@ -950,13 +989,7 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
self.result = ""
def warn_or_stop(self, message: str, e: Exception = None):
if self.__ppp.gen_onwarning == self.__ppp.ONWARNING_CHOICES.stop:
raise PPPInterrupt(
message,
self.__ppp.INVALID_CONTENT_STOP.format(message) if not self.__is_negative else "",
self.__ppp.INVALID_CONTENT_STOP.format(message) if self.__is_negative else "",
) from e
self.__ppp.logger.warning(message)
self.__ppp.warn_or_stop(self.__is_negative, message, e)
def start_visit(self, prompt_description: str, parsed_prompt: lark.Tree, is_negative: bool = False) -> str:
"""
@@ -1266,28 +1299,40 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
"""
Process a prompt composition construct in the tree.
"""
# if self.__ppp.is_comfy_ui():
# self.__ppp.logger.warning("Prompt composition is not supported in ComfyUI.")
start_result = self.result
t1 = time.monotonic_ns()
self.__visit(tree.children[0])
and_processing = self.__ppp.host_config.get("and", "ok")
if len(tree.children) > 1:
if tree.children[1] is not None:
self.result += f":{tree.children[1]}"
for i in range(2, len(tree.children), 3):
if self.__ppp.cup_ands:
self.result = re.sub(r"[, ]+$", "\n" if self.__ppp.cup_ands_eol else " ", self.result)
if self.result[-1:].isalnum(): # add space if needed
self.result += " "
self.result += "AND"
added_result = self.__visit(tree.children[i + 1], False, True)
if self.__ppp.cup_ands:
added_result = re.sub(r"^[, ]+", " ", added_result)
if added_result[0:1].isalnum(): # add space if needed
added_result = " " + added_result
self.result += added_result
if tree.children[i + 2] is not None:
self.result += f":{tree.children[i+2]}"
if and_processing == "eol":
self.result = (
self.result.rstrip() + "\n" + self.__visit(tree.children[i + 1], False, True).lstrip()
)
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug("AND construct replaced with EOL")
elif and_processing == "remove":
self.result += self.__visit(tree.children[i + 1], False, True)
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug("AND construct removed")
elif and_processing == "error":
self.warn_or_stop("AND constructs are not allowed!")
else: # and_processing == "ok":
if self.__ppp.cup_ands:
self.result = re.sub(r"[, ]+$", "\n" if self.__ppp.cup_ands_eol else " ", self.result)
if self.result[-1:].isalnum(): # add space if needed
self.result += " "
self.result += "AND"
added_result = self.__visit(tree.children[i + 1], False, True)
if self.__ppp.cup_ands:
added_result = re.sub(r"^[, ]+", " ", added_result)
if added_result[0:1].isalnum(): # add space if needed
added_result = " " + added_result
self.result += added_result
if tree.children[i + 2] is not None:
self.result += f":{tree.children[i+2]}"
t2 = time.monotonic_ns()
self.__debug_end("promptcomp", start_result, t2 - t1)
@@ -1295,8 +1340,6 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
"""
Process a scheduling construct in the tree and add it to the accumulated shell.
"""
# if self.__ppp.is_comfy_ui():
# self.__ppp.logger.warning("Prompt scheduling is not supported in ComfyUI.")
start_result = self.result
t1 = time.monotonic_ns()
before = tree.children[0]
@@ -1305,25 +1348,49 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
pos = float(pos_str)
if pos >= 1:
pos = int(pos)
# self.__shell.append(self.AccumulatedShell("sc", pos))
self.result += "["
if before is not None:
scheduling_processing = self.__ppp.host_config.get("scheduling", "ok")
if scheduling_processing == "before":
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug(f"Shell scheduled before with position {pos}")
self.__shell.append(self.AccumulatedShell("scb", pos))
self.__visit(before)
self.__ppp.logger.debug("Scheduling construct removed, taking before option")
if before is not None:
self.__visit(before)
elif scheduling_processing == "after":
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug("Scheduling construct removed, taking after option")
if after is not None:
self.__visit(after)
elif scheduling_processing == "first":
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug("Scheduling construct removed, taking first option")
if before is not None:
self.__visit(before)
elif after is not None:
self.__visit(after)
elif scheduling_processing == "remove":
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug("Scheduling construct removed")
elif scheduling_processing == "error":
self.warn_or_stop("Scheduling constructs are not allowed!")
else: # scheduling_processing == "ok"
# self.__shell.append(self.AccumulatedShell("sc", pos))
self.result += "["
if before is not None:
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug(f"Shell scheduled before with position {pos}")
self.__shell.append(self.AccumulatedShell("scb", pos))
self.__visit(before)
self.__shell.pop()
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug(f"Shell scheduled after with position {pos}")
self.__shell.append(self.AccumulatedShell("sca", pos))
self.result += ":"
self.__visit(after)
self.__shell.pop()
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug(f"Shell scheduled after with position {pos}")
self.__shell.append(self.AccumulatedShell("sca", pos))
self.result += ":"
self.__visit(after)
self.__shell.pop()
if self.__ppp.cup_emptyconstructs and re.fullmatch(re.escape(start_result) + r"\[:\s*", self.result):
self.result = start_result
else:
self.result += f":{pos_str}]"
# self.__shell.pop()
if self.__ppp.cup_emptyconstructs and re.fullmatch(re.escape(start_result) + r"\[:\s*", self.result):
self.result = start_result
else:
self.result += f":{pos_str}]"
# self.__shell.pop()
t2 = time.monotonic_ns()
self.__debug_end("scheduled", start_result, t2 - t1, pos_str)
@@ -1331,24 +1398,33 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
"""
Process an alternation construct in the tree and add it to the accumulated shell.
"""
# if self.__ppp.is_comfy_ui():
# self.__ppp.logger.warning("Prompt alternation is not supported in ComfyUI.")
start_result = self.result
t1 = time.monotonic_ns()
# self.__shell.append(self.AccumulatedShell("al", len(tree.children)))
self.result += "["
for i, opt in enumerate(tree.children):
alternation_processing = self.__ppp.host_config.get("alternation", "ok")
if alternation_processing == "first":
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug(f"Shell alternate option {i+1}")
self.__shell.append(self.AccumulatedShell("alo", {"pos": i + 1, "len": len(tree.children)}))
if i > 0:
self.result += "|"
self.__visit(opt)
self.__shell.pop()
self.result += "]"
if self.__ppp.cup_emptyconstructs and re.fullmatch(re.escape(start_result) + r"\[\s*\]", self.result):
self.result = start_result
# self.__shell.pop()
self.__ppp.logger.debug("Alternation construct removed, taking first option")
self.__visit(tree.children[0])
elif alternation_processing == "remove":
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug("Alternation construct removed")
elif alternation_processing == "error":
self.warn_or_stop("Alternation constructs are not allowed!")
else: # alternation_processing == "ok"
# self.__shell.append(self.AccumulatedShell("al", len(tree.children)))
self.result += "["
for i, opt in enumerate(tree.children):
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug(f"Shell alternate option {i+1}")
self.__shell.append(self.AccumulatedShell("alo", {"pos": i + 1, "len": len(tree.children)}))
if i > 0:
self.result += "|"
self.__visit(opt)
self.__shell.pop()
self.result += "]"
if self.__ppp.cup_emptyconstructs and re.fullmatch(re.escape(start_result) + r"\[\s*\]", self.result):
self.result = start_result
# self.__shell.pop()
t2 = time.monotonic_ns()
self.__debug_end("alternate", start_result, t2 - t1)
@@ -1358,17 +1434,18 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
"""
start_result = self.result
t1 = time.monotonic_ns()
# weight_kind: -1: remove, 0=none, 1=decrease, 2=increase, 3=specific
if len(tree.children) == 2:
weight_str = tree.children[-1]
if weight_str is not None:
weight_kind = 2 # specific weight
weight_kind = 3 # specific weight
weight = float(weight_str)
else:
weight_kind = 1 # increase attention
weight_kind = 2 # increase attention
weight = 1.1
weight_str = "1.1"
else:
weight_kind = 0 # decrease attention
weight_kind = 1 # decrease attention
weight = 0.9
weight_str = "0.9"
if self.__ppp.debug_level == DEBUG_LEVEL.full:
@@ -1390,37 +1467,59 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
weight = math.floor(weight * 100) / 100 # we round to 2 decimals
weight_str = f"{weight:.2f}".rstrip("0").rstrip(".")
if weight_str == "0.9":
weight_kind = 0
elif weight_str == "1.1":
weight_kind = 1
else:
elif weight_str == "1.1":
weight_kind = 2
if weight_kind == 0 and self.__ppp.is_comfy_ui():
weight_kind = 2
weight_str = "0.9"
self.__shell.append(self.AccumulatedShell("at", (weight_kind, weight_str)))
if weight_kind == 0:
starttag = "["
self.result += starttag
else:
weight_kind = 3
attention_processing = self.__ppp.host_config.get("attention", "ok")
if attention_processing == "parentheses":
if weight_kind == 1:
weight_kind = 3
weight_str = "0.9"
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug("Converted to parentheses format")
elif attention_processing == "disable":
weight_kind = 0
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug("Attention construct disabled")
elif attention_processing == "remove":
weight_kind = -1
if self.__ppp.debug_level == DEBUG_LEVEL.full:
self.__ppp.logger.debug("Attention construct removed")
elif attention_processing == "error":
self.warn_or_stop("Attention constructs are not allowed!")
# else: attention_processing == "ok":
if weight_kind == -1:
# we just ignore the attention construct
pass
elif weight_kind == 0:
# we just visit the content without adding any attention
self.__visit(current_tree)
endtag = "]"
elif weight_kind == 1:
starttag = "("
self.result += starttag
self.__visit(current_tree)
endtag = ")"
else:
starttag = "("
self.result += starttag
self.__visit(current_tree)
endtag = f":{weight_str})"
if self.__ppp.cup_emptyconstructs and re.fullmatch(
re.escape(start_result + starttag) + r"\s*", self.result
):
self.result = start_result
else:
self.result += endtag
self.__shell.pop()
self.__shell.append(self.AccumulatedShell("at", (weight_kind, weight_str)))
if weight_kind == 1:
starttag = "["
self.result += starttag
self.__visit(current_tree)
endtag = "]"
elif weight_kind == 2:
starttag = "("
self.result += starttag
self.__visit(current_tree)
endtag = ")"
else: # weight_kind == 3
starttag = "("
self.result += starttag
self.__visit(current_tree)
endtag = f":{weight_str})"
if self.__ppp.cup_emptyconstructs and re.fullmatch(
re.escape(start_result + starttag) + r"\s*", self.result
):
self.result = start_result
else:
self.result += endtag
self.__shell.pop()
t2 = time.monotonic_ns()
self.__debug_end("attention", start_result, t2 - t1, weight_str)
@@ -2285,6 +2384,7 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
self.result = ""
t1 = time.monotonic_ns()
self.__visit(tree.children)
attention_processing = self.__ppp.host_config.get("attention", "ok")
# process the found negative tags
for negtag in self.__negtags:
if self.__ppp.cup_mergeattention:
@@ -2296,12 +2396,12 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
/ 100
)
new_weight_str = f"{new_weight:.2f}".rstrip("0").rstrip(".")
if new_weight_str == "0.9" and not self.__ppp.is_comfy_ui():
new_kind = 0
elif new_weight_str == "1.1":
if new_weight_str == "0.9" and attention_processing != "parentheses":
new_kind = 1
else:
elif new_weight_str == "1.1":
new_kind = 2
else:
new_kind = 3
negtag.shell[i - 1] = self.AccumulatedShell(
"at",
(new_kind, new_weight_str),
@@ -2312,13 +2412,13 @@ class PromptPostProcessor: # pylint: disable=too-few-public-methods,too-many-in
for s in negtag.shell:
match s.type:
case "at":
if s.data[0] == 0:
if s.data[0] == 1:
start += "["
end = "]" + end
elif s.data[0] == 1:
elif s.data[0] == 2:
start += "("
end = ")" + end
else:
else: # 3
start += "("
end = f":{s.data[1]})" + end
# case "sc":
ppp_comfyui.py
+3 -38
View File
@@ -115,16 +115,6 @@ class PromptPostProcessorComfyUINode:
"label_off": "No",
},
),
"variants_definitions": (
"STRING",
{
"default": PromptPostProcessor.DEFAULT_VARIANTS_DEFINITIONS,
"multiline": True,
"placeholder": "",
"tooltip": "Definitions for variant models to be recognized based on strings found in the full filename. Format for each line is: 'name(kind)=comma separated list of substrings (case insensitive)' with kind being one of the base model types or not specified",
"dynamicPrompts": False,
},
),
"wc_options": (
"PPP_OPTIONS_WC",
{
@@ -192,37 +182,16 @@ class PromptPostProcessorComfyUINode:
pos_prompt,
neg_prompt,
seed,
debug_level, # pylint: disable=unused-argument
debug_level,
on_warnings,
process_wildcards,
do_cleanup,
variants_definitions,
wc_options,
stn_options,
cup_options,
en_options,
):
if process_wildcards:
return float(
"NaN"
) # since we can't detect changes in wildcards we assume they are always changed when enabled
new_run = { # everything except debug_level
"model": model,
"modelname": modelname,
"pos_prompt": pos_prompt,
"neg_prompt": neg_prompt,
"seed": seed,
"on_warnings": on_warnings,
"process_wildcards": process_wildcards,
"do_cleanup": do_cleanup,
"variants_definitions": variants_definitions,
"wc_options": wc_options,
"stn_options": stn_options,
"cup_options": cup_options,
"en_options": en_options,
}
return new_run.__hash__
# return float("NaN")
): # pylint: disable=unused-argument
return float("NaN") # always process because we don't control the content of wildcards and the config file
def process(
self,
@@ -235,7 +204,6 @@ class PromptPostProcessorComfyUINode:
on_warnings,
process_wildcards,
do_cleanup,
variants_definitions,
wc_options,
stn_options,
cup_options,
@@ -311,12 +279,9 @@ class PromptPostProcessorComfyUINode:
if f.strip() != ""
]
if variants_definitions != "" and not "=" in variants_definitions: # mainly to warn about the old format
raise ValueError("Invalid variants_definitions format")
options = {
"debug_level": debug_level,
"on_warnings": on_warnings,
"variants_definitions": variants_definitions,
"process_wildcards": process_wildcards,
"if_wildcards": (
wc_options["wc_if_wildcards"] if wc_options else PromptPostProcessor.IFWILDCARDS_CHOICES.stop.value
ppp_config.yaml.defaults
+82
View File
@@ -0,0 +1,82 @@
# If you want to change any of these options make first a copy of this file as "ppp_config.yaml" and edit that file.
# This file will be overwritten when updating the extension, but your custom "ppp_config.yaml" should not be affected.
hosts: # Options for supported host applications
# For each host define how to handle some operations:
# attention: how to handle attention operators
# - ok: leave as is
# - parentheses: convert to parentheses-based attention
# - disable: remove attention operators
# - remove: remove the whole block, including the text inside
# - error: raise an error
# scheduling: how to handle scheduling operators
# - ok: leave as is
# - before: replace with the first scheduling text
# - after: replace with the second scheduling text
# - first: replace with the first scheduling text if present, otherwise with the second scheduling text
# - remove: remove the whole block, including the text inside
# - error: raise an error
# alternation: how to handle alternation operators
# - ok: leave as is
# - first: replace with the first alternation text
# - remove: remove the whole block, including the text inside
# - error: raise an error
# and: how to handle "AND" operators
# - ok: leave as is
# - eol: replace with end-of-line (newline)
# - remove: remove the operator
# - error: raise an error
# break: how to handle "BREAK" operators
# - ok: leave as is
# - eol: replace with end-of-line (newline)
# - remove: remove the operator
# - error: raise an error
a1111:
attention: ok
scheduling: ok
alternation: ok
and: ok
break: ok
forge:
attention: ok
scheduling: ok
alternation: ok
and: ok
break: ok
reforge:
attention: ok
scheduling: ok
alternation: ok
and: ok
break: ok
sdnext:
attention: ok
scheduling: ok
alternation: ok
and: ok
break: ok
comfyui:
# comfyui only supports parentheses-based attention
attention: parentheses
# by default comfyui does not support scheduling, alternation, AND or BREAK operators, but you may be able to use them with a custom text encoder node
scheduling: error
alternation: error
and: remove
break: eol
models: # Options for supported base models and variants
sd1:
sd2:
sdxl:
variants:
# At this level goes the name of the defined variants
pony:
# find_in_filename: regex patterns as a string or list of strings to identify the variant in the model filename (ignores case)
find_in_filename: "(?:pony|pny|pdxl)"
illustrious:
find_in_filename: "(?:illustrious|illust|ilxl)"
sd3:
flux:
auraflow:
scripts/ppp_script.py
-16
View File
@@ -309,9 +309,6 @@ class PromptPostProcessorA1111Script(scripts.Script):
options = {
"debug_level": getattr(opts, "ppp_gen_debug_level", PromptPostProcessor.DEFAULT_DEBUG_LEVEL),
"on_warning": getattr(opts, "ppp_gen_onwarning", PromptPostProcessor.DEFAULT_ONWARNING),
"variants_definitions": getattr(
opts, "ppp_gen_variantsdefinitions", PromptPostProcessor.DEFAULT_VARIANTS_DEFINITIONS
),
"process_wildcards": getattr(opts, "ppp_wil_processwildcards", PromptPostProcessor.DEFAULT_WC_PROCESS),
"if_wildcards": getattr(opts, "ppp_wil_ifwildcards", PromptPostProcessor.DEFAULT_IF_WILDCARDS),
"choice_separator": getattr(opts, "ppp_wil_choice_separator", PromptPostProcessor.DEFAULT_CHOICE_SEPARATOR),
@@ -582,19 +579,6 @@ def on_ui_settings():
section=section,
),
)
shared.opts.add_option(
key="ppp_gen_variantsdefinitions",
info=shared.OptionInfo(
PromptPostProcessor.DEFAULT_VARIANTS_DEFINITIONS,
label="Definitions for variant models",
comment_after="Recognized based on strings found in the full filename. Format for each line is: 'name(kind)=comma separated list of substrings (case insensitive)' with kind being one of the base model types ("
+ ",".join(PromptPostProcessor.SUPPORTED_MODELS)
+ ") or not specified.",
component=gr.Textbox,
component_args={"lines": 7},
section=section,
),
)
shared.opts.add_option(
key="ppp_gen_doi2i",
info=shared.OptionInfo(
tests/tests.py
+517 -110
View File
File diff suppressed because it is too large Load Diff