Compare commits

...
21 Commits
Author SHA1 Message Date
Antonio Cordero Balcazar db43d409f8 * removed unnecessary processing of variables.
* error control when reading wildcard or mapping files, and second chance for files with incorrect encoding.
* fix processing inside extranetwork tag.
2025-09-27 13:22:19 +02:00
Antonio Cordero Balcazar 8769c21ee5 * new "optional" parameter for choices/wildcard.
* added "$$" to end of parameters as first choice in a wildcard, to avoid confusion with a choice that could be considered parameters.
* additional fixes in the parser.
2025-09-06 18:14:00 +02:00
Antonio Cordero Balcazar 54a20117ce * New extranetworks command to group the triggers with the extranetwork.
* Mapping of extranetworks through the new command. This allows you to use a "virtual" extranetwork that is converted to a real one depending on conditions, like the model kind/variant.
* Fixed a parsing bug with the if command.
* Some refactoring and additional checks.
2025-09-01 16:50:46 +02:00
Antonio Cordero Balcazar fe61e54f41 * Selection of prompt parser based on heuristics to improve parsing time on simpler cases.
* Improved timing logging.
* Refactor application support handling.
* Enhanced regex patterns in `ppp.py` for wildcard and choice detection to ensure proper escaping.
* Included icon source files.
2025-08-08 14:05:34 +02:00
Antonio Cordero Balcazar c03339a618 * Better processing of interruptions using an exception.
* New option on_warning to stop on invalid content.
* New option cleanup_extra_separators_include_eol to choose whether to also remove adjacent EOLs when removing extra separators or not. Default is not to do it. In previous versions it was forced.
* Refactoring of the cleanup method.
* Only simplify attention modifiers when the merge attention option is set.
* A1111: avoid logging about HR prompts and avoid saving them in extra metadata when not different.
* A1111: consider options and environment info in the caching of results.
* Some refactorings.
2025-05-31 12:08:57 +02:00
Antonio Cordero Balcazar bfc47d0d53 * A1111: add prompts to metadata only if they change 2025-05-03 12:07:18 +02:00
Antonio Cordero Balcazar b837f38422 * if_wildcards option defaults now to stop.
* ComfyUI: new wildcards_input.
* removed json import since yaml can read them.
* more models detected in ComfyUI.
* ComfyUI: example workflow.
2025-05-03 12:07:18 +02:00
Antonio Cordero Balcazar 5e3c8cbc44 * ComfyUI: updates to the ComfyUI registry workflow.
* A1111: option to add original prompts to metadata.
* fixed some bogus import error shenanigans.
* warn when a model name detects multiple variants.
2025-04-11 17:50:26 +02:00
Antonio Cordero Balcazar a1958db81f * Improved conditions with bool operations.
* Enhance PromptPostProcessorComfyUINode to return variables.
* New PromptPostProcessorSelectVariableComfyUINode for selecting variables.
* Fixed detection of Flux Schnell in Forge.
* Improved documentation.
2025-02-09 21:22:30 +01:00
Antonio Cordero Balcazar ce7b76f200 * refactoring
* additional tests
2025-01-25 16:47:43 +01:00
Antonio Cordero Balcazar e059a6d821 * add ifundefined modifier when setting variables.
* ComfyUI: better flux detection.
* Saving of UI settings and prompt/negative prompt to metadata.
2024-12-30 14:20:45 +01:00
Antonio Cordero Balcazar b06bcfa677 * Turned pony detection into detection of any kind of variant models based on filename. Added illustrious. Breaking change: the setting is different and changed pony substrings are not imported.
* Fixed and refactored some UI settings. Slight change in the generation of batched seeds.
* Detection of reforge.
* Small refactoring when getting choices.
2024-12-24 12:26:34 +01:00
Antonio Cordero Balcazar ee7ba94a30 * fix error in i2i detection.
* check for lingering special character sequences.
* small refactorings.
2024-11-23 14:00:43 +01:00
Antonio Cordero Balcazar 3810320f6e * Simpler versioning
* Cache now takes wildcard definitions into consideration.
* ComfyUI: node is not cached if wildcard processing is active.
* Add a space with prefix and suffix if needed.
* Checks requirements before installing them.
2024-10-27 16:05:55 +01:00
Antonio Cordero Balcazar c87c24924e * Support anonymous wildcards. 2024-10-19 16:05:21 +02:00
Antonio Cordero Balcazar e9092b877c * Improved wildcard definition support. 2024-10-19 12:45:48 +02:00
Antonio Cordero Balcazar a18e5e659c * Improved processing of wildcards and choices.
* Support for prefix and suffix in wildcard options and object format of the options in json/yaml files.
2024-10-13 12:57:25 +02:00
Antonio Cordero Balcazar 93f0102897 * Fix a bug with the variable echo command.
* Fix a bug obtaining random seed.
* Reading version number from pyproject file.
* A1111: Support for forcing the same seed/variation seed to all images in a batch.
* A1111: Support for unlinking the seed from the image seed and specifying one for the prompt.
* ComfyUI: Warning on use of invalid constructs. Related settings kept but default to False.
2024-10-11 18:13:57 +02:00
Antonio Cordero Balcazar 0b03e1773f * Added pyproject.toml to publish to ComfyUI registry.
* ComfyUI: optional modelname and model, and allowed sending a classname string instead of a model (#12)
2024-10-05 11:56:26 +02:00
Antonio Cordero Balcazar 830bd4f785 Add Github Action for Publishing to Comfy Registry (#14)
This PR adds a Github Action (publish-node-action) that will publish an
updated version of your custom node to the
[registry](https://comfyregistry.org/) whenever the `pyproject.toml`
file changes.
2024-10-05 11:03:28 +02:00
snomiao b019c5604b chore(publish): Add Github Action for Publishing to Comfy Registry 2024-10-01 16:01:06 +00:00
35 changed files with 5666 additions and 1225 deletions
+26
View File
@@ -0,0 +1,26 @@
name: Publish to Comfy registry
on:
workflow_dispatch:
push:
branches:
- main
- master
paths:
- "pyproject.toml"
permissions:
issues: write
jobs:
publish-node:
name: Publish Custom Node to registry
runs-on: ubuntu-latest
if: ${{ github.repository_owner == 'acorderob' }}
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Publish Custom Node
uses: Comfy-Org/publish-node-action@v1
with:
## Add your own personal access token to your Github Repository secrets and reference it here.
personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }}
+2
View File
@@ -3,3 +3,5 @@
.vscode/**/*
!.vscode/settings.json
!.vscode/launch.json
tests/tests_local.py
+2 -1
View File
@@ -1,5 +1,6 @@
{
"default": true,
"MD013": false,
"MD024": false
"MD024": false,
"MD033": false
}
+6
View File
@@ -4,6 +4,12 @@
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python Debugger: Attach using Process Id",
"type": "debugpy",
"request": "attach",
"processId": "${command:pickProcess}"
},
{
"name": "Tests",
"type": "debugpy",
+21 -332
View File
@@ -2,12 +2,11 @@
The Prompt PostProcessor (PPP), formerly known as "sd-webui-sendtonegative", is an extension designed to process the prompt, possibly after other extensions have modified it. This extension is compatible with:
* [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
* [AUTOMATIC1111 Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
* [SD.Next](https://github.com/vladmandic/automatic).
* [Forge](https://github.com/lllyasviel/stable-diffusion-webui-forge)
* [reForge](https://github.com/Panchovix/stable-diffusion-webui-reForge)
* ...and probably other forks
* [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
Currently this extension has these functions:
@@ -15,366 +14,56 @@ Currently this extension has these functions:
* Set and modify local variables.
* Filter content based on the loaded SD model or a variable.
* Process wildcards. Compatible with Dynamic Prompts formats. Can also detect invalid wildcards and act as you choose.
* Map extranetworks (LoRAs) depending on conditions (like the loaded model variant).
* Clean up the prompt and negative prompt.
Note: when used in an A1111 compatible webui, the extension must be loaded after any other extension that modifies the prompt (like another wildcards extension). Usually extensions load by their folder name in alphanumeric order, so if the extensions are not loading in the correct order just rename this extension's folder so the ordering works out. When in doubt, just rename this extension's folder with a "z" in front (for example) so that it is the last one to load, or manually set such folder name when installing it.
Note: when used in an *A1111* compatible webui, the extension must be loaded after any other extension that modifies the prompt (like another wildcards extension). Usually extensions load by their folder name in alphanumeric order, so if the extensions are not loading in the correct order just rename this extension's folder so the ordering works out. When in doubt, just rename this extension's folder with a "z" in front (for example) so that it is the last one to load, or manually set such folder name when installing it.
If the extension runs before others, like Dynamic Prompts, and the "Process wildcards" is enabled, the wildcards will be processed by PPP and those extensions will not get them. If you disable processing the wildcards, and intend another extension to process them, you should keep the "What to do with remaining wildcards?" option as "ignore".
Notes:
1. Other than its own commands, it only recognizes regular A1111 prompt formats. So:
1. Other than its own commands, it only recognizes regular *A1111* prompt formats. So:
* **Attention**: `\[prompt\] (prompt) (prompt:weight)`
* **Alternation**: `\[prompt1|prompt2|...\]`
* **Scheduling**: `\[prompt1:prompt2:step\]`
* **Extra networks**: `\<kind:model...\>`
* **Attention**: `[prompt] (prompt) (prompt:weight)`
* **Alternation**: `[prompt1|prompt2|...]`
* **Scheduling**: `[prompt1:prompt2:step]`
* **Extra networks**: `<kind:model...>`
* **BREAK**: `prompt1 BREAK prompt2`
* **Composable Diffusion**: `prompt1:weight1 AND prompt2:weight2`
In SD.Next that means only the *A1111* or *Full* parsers. It will warn you if you use the *Compel* parser.
In *SD.Next* that means only the *A1111* or *Full* parsers. It will warn you if you use the *Compel* parser.
Does not recognize tokenizer separators like "TE2:" and "TE3:", so sending to negative prompt from those sections of the prompt will not add them in the corresponding section of the negative prompt.
2. It recognizes wildcards in the *\_\_wildcard\_\_* and *{choice|choice}* formats (and almost everything that [Dynamic Prompts](https://github.com/adieyal/sd-dynamic-prompts) supports).
Does not recognize tokenizer separators like `TE2:` and `TE3:`, so sending to negative prompt from those sections of the prompt will not add them in the corresponding section of the negative prompt.
*ComfyUI* only supports natively the attention using parenthesis, so the ones with the braces will be converted. The other constructs are not natively supported but some custom nodes implement them.
2. It recognizes wildcards in the `__wildcard__` and {choice|choice} formats (and almost everything that [Dynamic Prompts](https://github.com/adieyal/sd-dynamic-prompts) supports).
3. It does not create *AND/BREAK* constructs when moving content to the negative prompt.
## Installation
On A1111 compatible webuis:
On *A1111* compatible webuis:
1. Go to Extensions > Install from URL
2. Paste <https://github.com/acorderob/sd-webui-prompt-postprocessor> in the URL for extension's git repository text field
3. Click the Install button
4. Restart the webui
On ComfyUI:
On *SD.Next* I recommend you to disable the native wildcard processing and use the old UI.
On *ComfyUI*:
1. Go to Manager > Custom Nodes Manager
2. Install through ComfyUI Manager
3. Click Install via Git URL and enter <https://github.com/acorderob/sd-webui-prompt-postprocessor>
4. Restart
2. Search for "Prompt PostProcessor" and install or click Install via Git URL and enter <https://github.com/acorderob/sd-webui-prompt-postprocessor>
3. Restart
## Usage
### Commands
The extension uses a format for its commands similar to an extranetwork, but it has a "ppp:" prefix followed by the command, and then a space and any parameters (if any).
```text
<ppp:command parameters>
```
When a command is associated with any content, it will be between an opening and a closing command:
```text
<ppp:command parameters>content<ppp:/command>
```
For wildcards and choices it uses the formats from the Dynamic Prompts extension, but sometimes with some additional options for more functionality.
### Choices
The generic format is:
```text
{parameters$$opt1::choice1|opt2::choice2|opt3::choice3}
```
Both the construct parameters (up to the '$$') and the individual choice options (up to the '::') are optional.
There is also a format where instead of "parameters$$" you just put the sampler, for compatibility with Dynamic Prompts.
The construct parameters can be written with the following options (all are optional):
* "**~**" or "**@**": sampler (for compatibility with Dynamic Prompts), but only "**~**" (random) is allowed.
* "**r**": means it allows repetition of the choices.
* "**n**" or "**n-m**" or "**n-**" or "**-m**": number or range of choices to select. Allows zero as the start of a range. Default is 1.
* "**$$sep**": separator when multiple choices are selected. Default is set in settings.
* "**$$**": end of the parameters.
The choice options are as follows:
* "**'identifiers'**": comma separated labels for the choice (optional, quotes can be single or double). Only makes sense inside a wildcard definition. Can be used when specifying the wildcard to select this specific choice. It's case insensitive.
* "**n**": weight of the choice (optional, default 1).
* "**if condition**": filters out the choice if the condition is false (optional; this is an extension to the Dynamic Prompts syntax). Same conditions as in the `if` command.
* "**::**": end of choice options
Whitespace is allowed between parameters.
These are examples of formats you can use to insert a choice construct:
```text
{opt1|5::opt2|3::opt3} # select 1 choice, two have weights
{3$$opt1|5 if _is_sd1::opt2|opt3} # select 3 choices, one has a weight and a condition
{2-3$$opt1|opt2|opt3} # select 2 to 3 choices
{r2-3$$opt1|opt2|opt3} # select 2 to 3 choices allowing repetition
{2-3$$ / $$opt1|opt2|opt3} # select 2 to 3 choices with separator " / "
```
Notes:
* The Dynamic Prompts format `{2$$__flavours__}` does not work as expected. It will only output one value. You can write is as `{r2$$__flavours__}` to get two values, but they may repeat since the evaluation of the wildcard is independent of the choices selection.
* Whitespace in the choices is not ignored like in Dynamic Prompts, but will be cleaned up if the appropriate settings are checked.
### Wildcards
The generic format is:
```text
__parameters$$wildcard'filter'(var=value)__
```
The parameters, the filter, and the setting of a variable are optional. The parameters follow the same format as for the choices.
The wildcard identifier can contain globbing formatting, to read multiple wildcards and merge their choices. Note that if there are no parameters specified, the globbing will use the ones from the first wildcard that matches and have parameters (sorted by keys), so if you don't want that you might want to specify them.
The filter can be used to filter specific choices from the wildcard. The filtering works before applying the choice conditions (if any). The surrounding quotes can be single or double. The filter is a comma separated list of an integer (positional choice index) or choice label. You can also compound them with "+". That is, the comma separated items act as an OR and the "+" inside them as an AND. Using labels can simplify the definitions of complex wildcards where you want to have direct access to specific choices on occasion (you don't need to create wildcards for each individual choice). There are some additional formats when using filters. You can specify "^wildcard" as a filter to use the filter of a previous wildcard in the chain. You can start the filter (regular or inherited) with "#" and it will not be applied to the current wildcard choices, but the filter will remain in memory to use by other descendant wildcards. You use "#" and "^" when you want to pass a filter to inner wildcards (see the test files).
The variable value only applies during the evaluation of the selected choices and is discarded afterward (the variable keeps its original value if there was one).
These are examples of formats you can use to insert a wildcard:
```text
__path/wildcard__ # select 1 choice
__path/wildcard'0'__ # select the first choice
__path/wildcard'label'__ # select the choices with label "label"
__path/wildcard'0,label1,label2'__ # select the first choice and those with labels "label1" or "label2"
__path/wildcard'0,label1+label2'__ # select the first choice and those with both labels "label1" and "label2"
__3$$path/wildcard__ # select 3 choices
__2-3$$path/wildcard__ # select 2 to 3 choices
__r2-3$$path/wildcard__ # select 2 to 3 choices allowing repetition
__2-3$$ / $$path/wildcard__ # select 2 to 3 choices with separator " / "
__path/wildcard(var=value)__ # select 1 choice using the specified variable value in the evaluation.
```
#### Wildcard definitions
A wildcard definition can be:
* A txt file. The wildcard name will be the relative path of the file, without the extension. Each line will be a choice. Lines starting with "#" or empty are ignored.
* An array or string inside a json or yaml file. The wildcard name includes the relative folder path of the file (without the name or extension) but also the path of the object inside the file.
In a choice, the content after a "#" is ignored.
If the first choice follows the format of wildcard parameters, it will be used as default parameters for that wildcard (see examples in the tests folder). The choices of the wildcard follow the same format as in the choices construct, or the object format of **Dynamic Prompts** (only in structured files). If using the object format for a choice you can use a new "if" property for the condition in addition to the standard "weight" and "text"/"content".
Wildcards can contain just one choice. In json and yaml formats this allows the use of a string value for the keys, rather than an array.
#### Detection of remaining wildcards
This extension should run after any other wildcard extensions, so if you don't use the internal wildcards processing, any remaining wildcards present in the prompt or negative_prompt at this point must be invalid. Usually you might not notice this problem until you check the image metadata, so this option gives you some ways to detect and treat the problem.
### Set command
This command sets the value of a variable that can be checked later.
The format is:
```text
<ppp:set varname>value<ppp:/set>
<ppp:set varname evaluate>value<ppp:/set>
<ppp:set varname add>value<ppp:/set>
<ppp:set varname evaluate add>value<ppp:/set>
```
The `evaluate` parameter makes it so the value of the variable is evaluated at this moment, instead of when it is used.
With the `add` parameter the value is added to the current value of the variable. It does not force an immediate evaluation of the old nor the added value.
The Dynamic Prompts format also works:
```text
${var=value}
${var=!value} # immediate evaluation
```
If also supports the addition as an extension of the Dynamic Prompts format:
```text
${var+=value}
${var+=!value}
```
### Echo command
This command prints the value of a variable.
The format is:
```text
<ppp:echo varname>
<ppp:echo varname>default<ppp:/echo>
```
The Dynamic Prompts format is:
```text
${var}
${var:default}
```
### If command
This command allows you to filter content based on conditions.
The full format is:
```text
<ppp:if condition1>content one<ppp:elif condition2>content two<ppp:else>other content<ppp:/if>
```
The *conditionN* compares a variable with a value or a list of values. The allowed formats are:
```text
[not] variable
[not] variable operation value
variable [not] operation value
[not] variable operation (value1,value2...)
variable [not] operation (value1,value2...)
```
When there is no value it will check if the variable is truthy.
For a simple value the allowed operations are `eq`, `ne`, `gt`, `lt`, `ge`, `le`, `contains` and the value can be a quoted string or an integer.
For a list of values the allowed operations are `contains`, `in` and the value of the variable is checked against all the elements of the list until one matches.
The variable can be one set with the `set` or `add` commands or you can use internal variables like these (names starting with an underscore are reserved):
* `_model` : the loaded model identifier (`"sd1"`, `"sd2"`, `"sdxl"`, `"sd3"`, `"flux"`, `"auraflow"`). `_sd` also works but is deprecated.
* `_modelname` : the loaded model filename (without path). `_sdname` also works but is deprecated.
* `_modelfullname`: the loaded model filename (with path). `_sdfullname` also works but is deprecated.
* `_modelclass`: the class used for the model. Note that this is dependent on the webui. In A1111 all SD versions use the same class. Can be used for new models that are not supported yet with the `_is_*` variables.
* `_is_sd`: true if the loaded model version is any version of SD
* `_is_sd1`: true if the loaded model version is SD 1.x
* `_is_sd2`: true if the loaded model version is SD 2.x
* `_is_sdxl`: true if the loaded model version is SDXL (includes Pony models)
* `_is_ssd`: true if the loaded model version is SSD (Segmind Stable Diffusion 1B). Note that for an SSD model `_is_sdxl` will also be true.
* `_is_sdxl_no_ssd`: true if the loaded model version is SDXL and not an SSD model.
* `_is_pony`: true if the loaded model version is SDXL and a Pony model (based on its filename). Note that for a pony model `_is_sdxl` will also be true.
* `_is_sdxl_no_pony`: true if the loaded model version is SDXL and not a Pony model.
* `_is_sd3`: true if the loaded model version is SD 3.x
* `_is_flux`: true if the loaded model is Flux
* `_is_auraflow`: true if the loaded model is AuraFlow
Any `elif`s (there can be multiple) and the `else` are optional.
#### Example
(multiline to be easier to read)
```text
<ppp:if _is_sd1><lora:test_sd1> test sd1x
<ppp:elif _sd_pony><lora:test_pony> test pony
<ppp:elif _sd_sdxl><lora:test_sdxl> test sdxl
<ppp:else>unknown model
<ppp:/if>
```
Only one of the options will end up in the prompt, depending on the loaded model.
### Sending content to the negative prompt
The new format for this command is like this:
```text
<ppp:stn position>content<ppp:/stn>
```
Where position is optional (defaults to the start) and can be:
* **s**: at the start of the negative prompt
* **e**: at the end of the negative prompt
* **pN**: at the position of the insertion point in the negative prompt with N being 0-9
The format of the insertion point to be used in the negative prompt is:
```text
<ppp:stn iN>
```
If the insertion point is not found it inserts at the start.
#### Example
You have a wildcard for hair colors (\_\_haircolors\_\_) with one being strawberry blonde, but you don't want strawberries. So in that option you add a command to add to the negative prompt, like so:
```text
blonde
strawberry blonde <ppp:stn>strawberry<ppp:/stn>
brunette
```
Then, if that option is chosen this extension will process it later and move that part to the negative prompt.
#### Old format
The old format (`<!...!>`) is not supported anymore.
### Notes on negative commands
Positional insertion commands have less priority that start/end commands, so even if they are at the start or end of the negative prompt, they will end up inside any start/end (and default position) commands.
The content of the negative commands is not processed and is copied as-is to the negative prompt. Other modifiers around the commands are processed in the following way.
#### Attention modifiers (weights)
They will be translated to the negative prompt. For example:
* `(red<ppp:stn>square<ppp:/stn>:1.5)` will end up as `(square:1.5)` in the negative prompt
* `(red[<ppp:stn>square<ppp:/stn>]:1.5)` will end up as `(square:1.35)` in the negative prompt (weight=1.5*0.9) if the merge attention option is enabled or `([square]:1.5)` otherwise.
* However `(red<ppp:stn>[square]<ppp:/stn>:1.5)` will end up as `([square]:1.5)` in the negative prompt. The content of the negative tag is copied as is, and is not merged with the surrounding modifier because the insertions happen after the attention merging.
#### Prompt editing constructs (alternation and scheduling)
Negative commands inside such constructs will copy the construct to the negative prompt, but separating its elements. For example:
* **Alternation**: `[red<ppp:stn>square<ppp:/stn>|blue<ppp:stn>circle<ppp:/stn>]` will end up as `[square|], [|circle]` in the negative prompt, instead of `[square|circle]`
* **Scheduling**: `[red<ppp:stn>square<ppp:/stn>:blue<ppp:stn>circle<ppp:/stn>:0.5]` will end up as `[square::0.5], [:circle:0.5]` instead of `[square:circle:0.5]`
This should still work as intended, and the only negative point i see is the unnecessary separators.
See the [syntax documentation](docs/SYNTAX.md).
## Configuration
### General settings
* **Debug level**: what to write to the console. Note: in SD.Next debug messages only show if you launch it with the --debug argument.
* **Pony substrings**: list of substrings to detect a Pony model.
* **Apply in img2img**: check if you want to do the processing in img2img processes (does not apply to ComfyUI node).
### Wildcard settings
* **Process wildcards**: you can choose to process them with this extension or use a different one.
* **Wildcards folders**: you can enter multiple folders separated by commas. In ComfyUI you can leave it empty and add a "wildcards" entry in the extra_model_paths.yaml file.
* **What to do with remaining wildcards?**: select what do you want to do with any found wildcards.
* **Ignore**: do not try to detect wildcards.
* **Remove**: detect wildcards and remove them.
* **Add visible warning**: detect wildcards and add a warning text to the prompt, that hopefully produces a noticeable generation.
* **Stop the generation**: detect wildcards and stop the generation.
* **Default separator used when adding multiple choices**: what do you want to use by default to separate multiple choices when the options allow it (by default it's ", ").
* **Keep the order of selected choices**: if checked, a multiple choice construct will return them in the order they are in the construct.
### Send to negative prompt settings
* **Separator used when adding to the negative prompt**: you can specify the separator used when adding to the negative prompt (by default it's ", ").
* **Ignore repeated content**: it ignores repeated content to avoid repetitions in the negative prompt.
### Clean up settings
* **Remove empty constructs**: removes attention/scheduling/alternation constructs when they are invalid.
* **Remove extra separators**: removes unnecessary separators. This applies to the configured separator and regular commas.
* **Remove additional extra separators**: removes unnecessary separators at start or end of lines. This applies to the configured separator and regular commas.
* **Clean up around BREAKs**: removes consecutive BREAKs and unnecessary commas and space around them.
* **Use EOL instead of Space before BREAKs**: add a newline before BREAKs.
* **Clean up around ANDs**: removes consecutive ANDs and unnecessary commas and space around them.
* **Use EOL instead of Space before ANDs**: add a newline before ANDs.
* **Clean up around extra network tags**: removes spaces around them.
* **Merge attention modifiers (weights) when possible**: it merges attention modifiers when possible (merges into one, multiplying their values). Only merges individually nested modifiers.
* **Remove extra spaces**: removes other unnecessary spaces.
### Content removal settings
* **Remove extra network tags**: removes all extra network tags.
See the [configuration documentation](docs/CONFIG.md).
## License
+8 -12
View File
@@ -10,19 +10,15 @@ import os
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
from .ppp_comfyui import PromptPostProcessorComfyUINode
from .ppp_comfyui import PromptPostProcessorComfyUINode, PromptPostProcessorSelectVariableComfyUINode
NODE_CLASS_MAPPINGS = {"ACBPromptPostProcessor": PromptPostProcessorComfyUINode}
NODE_DISPLAY_NAME_MAPPINGS = {"ACBPromptPostProcessor": "ACB Prompt Post Processor"}
MANIFEST = {
"name": "ACB Prompt Post Processor",
"version": PromptPostProcessorComfyUINode.VERSION,
"author": "ACB",
"project": "https://github.com/acorderob/sd-webui-prompt-postprocessor",
"description": "Node for processing prompts",
"license": "MIT",
NODE_CLASS_MAPPINGS = {
"ACBPromptPostProcessor": PromptPostProcessorComfyUINode,
"ACBPPPSelectVariable": PromptPostProcessorSelectVariableComfyUINode,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"ACBPromptPostProcessor": "ACB Prompt Post Processor",
"ACBPPPSelectVariable": "ACB PPP Select Variable",
}
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
+84
View File
@@ -0,0 +1,84 @@
# Prompt PostProcessor configuration
## ComfyUI specific (ACB Prompt Post Processor node)
### Inputs
* **model**: Connect here the MODEL or a string with the model class name used by *ComfyUI*. Needed for the model kind system variables.
* **modelname**: Name of the model. Needed for the model name system variables and detection of pony (this also requieres for the model to be SDXL).
* **seed**: Connect here the seed used. By default it is -1 (random).
* **pos_prompt**: Connect here the prompt text, or fill it as a widget.
* **neg_prompt**: Connect here the negative prompt text, or fill it as a widget.
* **wc_wildcards_input**: Wildcards definitions (in yaml or json format). Direct input added to the ones found in the wildcards folders. Allows wildcards to be included in the workflow.
* **en_mappings_input**: Extranetwork Mappings definitions (in yaml format). Direct input added to the ones found in the extranetwork mappings folders. Allows the mappings to be included in the workflow.
Other common settings (see [below](#common-settings)) also appear as inputs or widgets.
### Outputs
The outputs are the final positive and negative prompt and a variables dictionary.
You can use the "**ACB PPP Select Variable**" node to choose one and output its value. You can use this to send only part of the prompt to, for example, a detailer node. For example:
With this prompt: `__quality__, 1girl, ${head:__eyes__, __hair__, __expression__}, __body__, __clothes__, __background__, __style__` then you extract the `head` variable and send that as prompt for the head/face detailer.
## A1111 (and compatible UIs) panel options
* **Force equal seeds**: Changes the image seeds and variation seeds to be equal to the first of the batch. This allows using the same values for all the images in a batch.
* **Unlink seed**: Uses the specified seed for the prompt generation instead of the one from the image. This seed is only used for wildcards and choices.
* **Prompt seed**: The seed to use for the prompt generation. If -1 a random one will be used.
* **Incremental seed**: When using a batch you can use this to set the rest of the prompt seeds with consecutive values.
## Common settings
### General settings
* **Debug level**: what to write to the console. Note: in *SD.Next* debug messages only show if you launch it with the `--debug` argument.
* **What to do on invalid content warnings?**: warn on the console or stop the generation.
* **Model variant definitions**: definitions for model variants to be recognized based on strings found in the full filename.
The format for each line is (with *kind* being one of the base model identifiers or not defined):
```name(kind)=comma separated list of substrings (case insensitive)```
The default value defines strings for *Pony* and *Illustrious* models.
* **Apply in img2img**: check if you want to do the processing in img2img processes (*does not apply to the ComfyUI node*).
* **Add original prompts to metadata**: adds original prompts to the metadata if they have changed (*does not apply to the ComfyUI node*).
* **Extranetwork Mappings folders**: you can enter multiple folders separated by commas. In *ComfyUI* you can leave it empty and add a `ppp_extranetworkmappings` entry in the **extra_model_paths.yaml** file.
### Wildcard settings
* **Process wildcards**: you can choose to process wildcards and choices with this extension or use a different one.
* **Wildcards folders**: you can enter multiple folders separated by commas. In *ComfyUI* you can leave it empty and add a `ppp_wildcards` or `wildcards` entry in the **extra_model_paths.yaml** file.
* **What to do with remaining wildcards?**: select what do you want to do with any found wildcards/choices (when process wildcards is off or after the processing).
* **Ignore**: do not try to detect wildcards.
* **Remove**: detect wildcards and remove them.
* **Add visible warning**: detect wildcards and add a warning text to the prompt, that hopefully produces a noticeable generation.
* **Stop the generation**: detect wildcards and stop the generation.
* **Default separator used when adding multiple choices**: what do you want to use by default to separate multiple choices when the options allow it (by default it's ", ").
* **Keep the order of selected choices**: if checked, a multiple choice construct will return them in the order they are in the construct.
### Send to negative prompt settings
* **Separator used when adding to the negative prompt**: you can specify the separator used when adding to the negative prompt (by default it's ", ").
* **Ignore repeated content**: it ignores repeated content to avoid repetitions in the negative prompt.
### Clean up settings
* **Remove empty constructs**: removes attention/scheduling/alternation constructs when they are invalid.
* **Remove extra separators**: removes unnecessary separators. This applies to the configured separator and regular commas.
* **Remove additional extra separators**: removes unnecessary separators at start or end of lines. This applies to the configured separator and regular commas.
* **The extra separators options also remove EOLs**: in the previous two options it also removes EOLs attached to the separators.
* **Clean up around BREAKs**: removes consecutive BREAKs and unnecessary commas and space around them.
* **Use EOL instead of Space before BREAKs**: add a newline before BREAKs.
* **Clean up around ANDs**: removes consecutive ANDs and unnecessary commas and space around them.
* **Use EOL instead of Space before ANDs**: add a newline before ANDs.
* **Clean up around extra network tags**: removes spaces around them.
* **Merge attention modifiers (weights) when possible**: it merges attention modifiers when possible (merges into one, multiplying their values). Only merges individually nested modifiers.
* **Remove extra spaces**: removes other unnecessary spaces.
Please note that *ComfyUI* does not natively support the `BREAK` and `AND` constructs, but the related settings are kept in that UI.
### Content removal settings
* **Remove extra network tags**: removes all extra network tags.
+363
View File
@@ -0,0 +1,363 @@
# Prompt PostProcessor syntax
## Commands
The extension uses a format for its commands similar to an extranetwork, but it has a "ppp:" prefix followed by the command, and then a space and any parameters (if any).
`<ppp:command parameters>`
When a command is associated with any content, it will be between an opening and a closing command:
`<ppp:command parameters>content<ppp:/command>`
For wildcards and choices it uses the formats from the *Dynamic Prompts* extension, but sometimes with some additional options for extra functionality.
## Choices
The generic format is: `{parameters$$opt1::choice1|opt2::choice2|opt3::choice3}`
Both the construct parameters (up to the `$$`) and the individual choice options (up to the '::') are optional.
There is also a format where instead of `parameters$$` you just put the sampler, for compatibility with *Dynamic Prompts*.
The construct parameters can be written with the following options (all are optional):
* "**~**" or "**@**": sampler (for compatibility with *Dynamic Prompts*), but only "**~**" (random) is supported.
* "**r**": means it allows repetition of the choices.
* "**o**": means it is "optional", and no error will be raised if there are no choices to select from.
* "**n**" or "**n-m**" or "**n-**" or "**-m**": number or range of choices to select. Allows zero as the start of a range. Default is 1.
* "**$$sep**": separator when multiple choices are selected. Default is set in settings.
* "**$$**": end of the parameters (not optional if any parameters).
Regarding the "optional" flag, consider this scenario: due to their conditions no choice is available. It will raise an error. If you add the `o` then it will just return an empty string. This is only necessary if all choices have conditions and they could all be false. It is not the same as setting a range starting at 0, because that would be an allowed number of returned choices. If you do this and no choices are available, no error is raised.
The choice options are as follows:
* "**'identifiers'**": comma separated labels for the choice (optional, quotes can be single or double). Only makes sense inside a wildcard definition. Can be used when specifying the wildcard to select this specific choice. It's case insensitive.
* "**n**": weight of the choice (optional, default 1).
* "**if condition**": filters out the choice if the condition is false (optional; this is an extension to the *Dynamic Prompts* syntax). Same conditions as in the `if` command.
* "**::**": end of choice options (not optional if any options)
Whitespace is allowed between parameters/options.
These are examples of formats you can use to insert a choice construct:
| Construct | Result |
| --------- | ------ |
| `{choice1\|5::choice2\|3::choice3}` | select 1 choice, two of them have weights |
| `{3$$choice1\|5 if _is_sd1::choice2\|choice3}` | select 3 choices, one has a weight and a condition |
| `{2-3$$2::choice1\|choice2\|choice3}` | select 2 to 3 choices, one of them has a weight |
| `{r2-3$$choice1\|choice2\|choice3}` | select 2 to 3 choices allowing repetition |
| `{2-3$$ / $$choice1\|choice2\|choice3}` | select 2 to 3 choices with separator " / " |
| `{o$$if _is_sd1::choice1\|if _is_sd2::choice2}`| select 1 choice, both have conditions, if none matches it is allowed because we indicate that it is optional |
Notes:
* The *Dynamic Prompts* format `{2$$__flavours__}` does not work as expected. It will only output one value. You can write is as `{r2$$__flavours__}` to get two values, but they may repeat since the evaluation of the wildcard is independent of the choices selection.
* Whitespace around the choices is not ignored like in *Dynamic Prompts*, but will be cleaned up if the appropriate cleaning settings are selected.
## Wildcards
The generic format is: `__parameters$$wildcard'filter'(var=value)__`
The parameters, the filter, and the setting of a variable are optional. The parameters follow the same format as for the choices.
The wildcard identifier can have a relative path and contain globbing formatting, to read multiple wildcards and merge their choices. Note that if there are no parameters specified, the globbing will use the ones from the first wildcard that matches and have parameters (sorted by keys), so if you don't want that you might want to specify them. Also note that, unlike with *Dynamic Prompts*, the wildcard name has to be specified with its full path (unless you use globbing).
The filter can be used to filter specific choices from the wildcard. The filtering works before applying the choice conditions (if any). The surrounding quotes can be single or double. The filter is a comma separated list of an integer (positional choice index; zero-based) or choice label. You can also compound them with `+`. That is, the comma separated items act as an OR and the `+` inside them as an AND. Using labels can simplify the definitions of complex wildcards where you want to have direct access to specific choices on occasion (you don't need to create wildcards for each individual choice). There are some additional formats when using filters. You can specify `^wildcard` as a filter to use the filter of a previous wildcard in the chain. You can start the filter (regular or inherited) with `#` and it will not be applied to the current wildcard choices, but the filter will remain in memory to use by other descendant wildcards. You use `#` and `^` when you want to pass a filter to inner wildcards (see the test files).
The variable value only applies during the evaluation of the selected choices and is discarded afterward (the variable keeps its original value if there was one).
These are examples of formats you can use to insert a wildcard:
| Construct | Result |
| --------- | ------ |
| `__wildcard__` | select 1 choice |
| `__path/wildcard'0'__` | select the first choice |
| `__path/wildcard'label'__` | select the choices with label "label" |
| `__path/wildcard'0,label1,label2'__` | select the first choice and those with labels "label1" or "label2" |
| `__path/wildcard'0,label1+label2'__` | select the first choice and those with both labels "label1" and "label2" |
| `__3$$path/wildcard__` | select 3 choices |
| `__2-3$$path/wildcard__` | select 2 to 3 choices |
| `__r2-3$$path/wildcard__` | select 2 to 3 choices allowing repetition |
| `__2-3$$ / $$path/wildcard__` | select 2 to 3 choices with separator " / " |
| `__path/wildcard(var=value)__` | select 1 choice using the specified variable value in the evaluation. |
Wildcards cannot be used inside an extranetwork tag (because some lora names contain double underscores). If you need to choose from multiple loras put the whole extranetwork tag inside a wildcard, or use choices.
### Wildcard definitions
A wildcard definition can be:
* A txt file. The wildcard name will be the relative path of the file, without the extension. Each line will be a choice. Lines starting with `#` or empty are ignored. Doesn't support nesting.
* An array or scalar value inside a json or yaml file. The wildcard name includes the relative folder path of the file, without the extension, but also the path of the value inside the file (if there is one). If the file contains a dictionary, the filename part is not used for the wildcard name. Supports nesting by having dictionaries inside dictionaries.
The best format is a yaml file with a dictionary of wildcards inside. An editor supporting yaml syntax and linting is recommended (f.e. vscode).
In a choice, the content after a `#` is ignored.
If the first choice follows the format of wildcard parameters (*including the final `$$`*), it will be used as default parameters for that wildcard (see examples in the tests folder). The choices of the wildcard follow the same format as in the choices construct, or the object format of *Dynamic Prompts* (only in structured files). If using the object format for a choice you can use a new `if` property for the condition, and the `labels` property (an array of strings) in addition to the standard `weight` and `text`/`content`.
```yaml
{ labels: ["some_label"], weight: 2, if: "_is_pony", content: "the text" } # "text" property can be used instead of "content"
```
Wildcard parameters in a json/yaml file can also be in object format, and support two additional properties, prefix and suffix:
```yaml
{ sampler: "~", repeating: false, optional: false, count: 2, prefix: "prefix-", suffix: "-suffix", separator: "/" }
{ sampler: "~", repeating: false, optional: false, from: 2, to: 3, prefix: "prefix-", suffix: "-suffix", separator: "/" }
```
The prefix and suffix are added to the result along with the selected choices and separators. They can contain other constructs, but the separator can't.
It is recommended to use the object format for the wildcard parameters and for choices with complex options.
Wildcards can contain just one choice. In json and yaml formats this allows the use of a string value for the keys, rather than an array.
A choice inside a wildcard can also be a list or a dictionary of one element containing a list. These are considered anonymous wildcards. With a list it will be an anonymous wildcard with no choice options, and with a dictionary the key will be the options for the choice containing the anonymous wildcard and the value the choices of the anonymous wildcard. Anonymous wildcards can help formatting complex choice values that are used in only one place and thus creating a regular wildcard is not necessary. See test.yaml for examples.
Note: the files should have UTF-8 encoding. The extension will also try with windows-1252 if that fails.
Wildcard definitions are reloaded automatically on each generation if they change.
### Detection of remaining wildcards
This extension should run after any other wildcard extensions, so if you don't use the internal wildcards processing, any remaining wildcards present in the prompt or negative_prompt at this point must be invalid. Usually you might not notice this problem until you check the image metadata, so this option gives you some ways to detect and treat the problem.
## Set command
This command sets the value of a variable that can be checked later.
The format is: `<ppp:set varname [modifiers]>value<ppp:/set>`
These are the available optional modifiers:
* `evaluate`: the value of the variable is evaluated at this moment, instead of when it is used.
* `add`: the value is added to the current value of the variable. It does not force an immediate evaluation of the old nor the added value.
* `ifundefined`: the value will only be set if the variable is undefined.
The `add` and `ifundefined` modifiers are mutually exclusive and cannot be used together.
The *Dynamic Prompts* format also works:
| Construct | Meaning |
| --------- | ------- |
| `${var=value}` | regular evaluation |
| `${var=!value}` | immediate evaluation |
If also supports the addition and undefined check as an extension of the *Dynamic Prompts* format:
| Construct | Meaning |
| --------- | ------- |
| `${var+=value}` | equivalent to "add" |
| `${var+=!value}` | equivalent to "evaluate add" |
| `${var?=value}` | equivalent to "ifundefined" |
| `${var?=!value}` | equivalent to "evaluate ifundefined" |
## Echo command
This command prints the value of a variable, or the specified default if it doesn't exist.
The format is:
| Construct |
| --------- |
| `<ppp:echo varname>` |
| `<ppp:echo varname>default<ppp:/echo>` |
The *Dynamic Prompts* format is:
| Construct |
| --------- |
| `${varname}` |
| `${varname:default}` |
## If command
This command allows you to filter content based on conditions.
The full format is:
`<ppp:if condition1>content one<ppp:elif condition2>content two<ppp:else>other content<ppp:/if>`
Any `elif`s (there can be multiple) and the `else` are optional.
The `conditionN` can be:
| Construct | Meaning |
| --------- | ------- |
| `variable` | check truthyness of the variable |
| `variable [not] operation value` | check the variable against a value |
| `variable [not] operation (value1,value2,...)` | check the variable against a list of values |
For a simple value the allowed operations are `eq`, `ne`, `gt`, `lt`, `ge`, `le`, `contains` and the value can be a quoted string or an integer. For a list of values the allowed operations are `contains`, `in` and the value of the variable is checked against all the elements of the list until one matches. The operation can be preceded by `not` for readability, instead of using it in the front.
You can also build complex conditions joining them with boolean operators and/or/not and parentheses.
The variable can be one set with the `set` or `add` commands (user variables) or you can use system variables like these (names starting with an underscore are reserved for system variables):
| System variable | Value |
| --------------- | ----- |
| `_model` | the loaded model identifier (`"sd1"`, `"sd2"`, `"sdxl"`, `"sd3"`, `"flux"`, `"auraflow"`). `_sd` also works but is deprecated. |
| `_modelname` | the loaded model filename (without path). `_sdname` also works but is deprecated. |
| `_modelfullname` | the loaded model filename (with path). `_sdfullname` also works but is deprecated. |
| `_modelclass` | the class used for the model. Note that this is dependent on the webui. In A1111 all SD versions use the same class. Can be used for new models that are not supported yet with the `_is_*` variables. The debug setting will show all system variables when generating in case you need to see which one to use for a certain model. |
| `_is_sd` | true if the loaded model version is any version of SD |
| `_is_sd1` | true if the loaded model version is SD 1.x |
| `_is_sd2` | true if the loaded model version is SD 2.x |
| `_is_sdxl` | true if the loaded model version is SDXL (includes Pony models) |
| `_is_sd3` | true if the loaded model version is SD 3.x |
| `_is_flux` | true if the loaded model is Flux |
| `_is_auraflow` | true if the loaded model is AuraFlow |
| `_is_ssd` | true if the loaded model version is SSD (Segmind Stable Diffusion 1B). Note that for an SSD model `_is_sdxl` will also be true. |
| `_is_sdxl_no_ssd` | true if the loaded model version is SDXL and not an SSD model. |
| `_is_sdxl_no_pony` | true if the loaded model version is SDXL and not a Pony model (the "pony" variant must be defined in settings). Kept to maintain compatibility with previous versions. |
| `_is_vvvv` | true if the loaded model matches the *vvvv* model variant definition (based on its filename). Note that the corresponding variable for the model kind will also be true. |
| `_is_pure_kkkk` | true if the loaded model is of kind *kkkk* (f.e. sdxl) and not a variant. |
| `_is_variant_kkkk` | true if the loaded model version is any variant of model kind *kkkk* and not the pure version. Note that the corresponding variable for the model kind will also be true.|
### Example
(multiline to be easier to read)
```text
<ppp:if _is_sd1><lora:test_sd1> test sd1x
<ppp:elif _sd_pony><lora:test_pony> test pony
<ppp:elif _sd_pure_sdxl><lora:test_sdxl> test sdxl
<ppp:else>unknown model
<ppp:/if>
```
Only one of the options will end up in the prompt, depending on the loaded model.
## ExtraNetwork command
This command is a shortcut to add an extranetwork (usually a lora), and its triggers, with conditions. More legible and sometimes shorter than adding regular extranetworks inside if commands.
The full format is:
`<ppp:ext type name [parameters] [if condition]>[triggers]<ppp:/ext>`
`<ppp:ext type name [parameters] [if condition]>`
The `type` is the kind of extranetwork, like `lora` or `hypernet`.
The `name` is the extranetwork identifier. If it is not a regular identifier (i.e. starts with a number or contains spaces or symbols) it should be inside quotes.
The `parameters` is optional and its format depends on the extranetwork type. With loras or hypernets it is usually a single weight number, so if the type is one of those and there are no parameters it will default to `1`. If it is not a number it should go inside quotes.
The `condition` uses the same format as in the `if` command, and it is also optional.
The `triggers` are also optional, and can be any content. If there are no triggers the command ending can be omitted.
If the condition passes (or if there is no condition) the extranetwork tag will be built and added to the result along with any triggers.
### Examples
(multiline to be easier to read)
```text
<ppp:ext lora test_sd1 if _is_sd1>test sd1x<ppp:/ext>
<ppp:ext lora test_pony 0.5 if _is_pony>test pony<ppp:/ext>
<ppp:ext lora test_ilxl if _is_illustrious>
<ppp:ext lora 'test sdxl' '1:0.8' if _is_pure_sdxl>test sdxl<ppp:/ext>
```
Will turn into one of these (or none) depending on the model:
* `<lora:test_sd1:1>test sd1x`
* `<lora:test_pony:0.5>test pony`
* `<lora:test_illustrious:1>`
* `<lora:test sdxl:1:0.8>test sdxl`
### Extranetworks mappings
The extranetwork command supports specifying mappings of extranetworks (like LoRAs), so, for example, a different one can be used depending on the loaded model.
If the type of extranetwork is prefixed with a `$` the command will look for a mapping.
The mappings are configured in yaml files in any of the configured extranetwork mappings folders. The format is like this:
```yaml
extnettype:
mappingname:
- condition: "<a supported condition>"
name: "<name of the extranetwork>"
parameters: "<parameters of the extranetwork>"
triggers: [<list of triggers>]
weight: 1.0
...
```
Used like this:
```text
<ppp:ext $lora mappingname>
<ppp:ext $lora mappingname>inline triggers<ppp:/ext>
```
Each mapping can have any number of elements in its list of mappings. There are no mandatory properties for a mapping. The properties mean the following:
* `extnettype`: the kind of extranetwork, for example `lora`.
* `mappingname`: the name you want to give to the mapping, to be referenced in the command.
* `condition`: the condition to check for this mapping to be used (usually it should be one of the `_is_*` variables). If the conditions of multiple mappings evaluate to True, one will be chosen randomly. If the condition is missing it is considered True, to be used in the last mapping to catch as an "else" condition, and will be used if no other mapping applies.
* `name`: name of the real extranetwork. If it is missing no extranetwork tag will be added.
* `parameters`: parameters for the real extranetwork. If it is missing it is assumed "1" for loras and hypernets. If both this parameter and the parameter in the ext command are numbers they are multiplied for the result. In other case the parameter of the ext command, if it exists, is used.
* `triggers`: list of trigger strings. If it is missing, only the inline triggers in the ext command will be added.
* `weight`: weight for this variant, in case multiple of them apply, to choose one. Default is 1.
See the file in the tests folder as an example.
## Sending content to the negative prompt
The new format for this command is like this:
| Construct | Meaning |
| --------- | ------- |
| `<ppp:stn position>content<ppp:/stn>` | send to negative prompt |
| `<ppp:stn iN>` | insertion point to be used in the negative prompt as destination for the pN position |
Where position is optional (defaults to the start) and can be:
* **s**: at the start of the negative prompt
* **e**: at the end of the negative prompt
* **pN**: at the position of the insertion point in the negative prompt with N being 0-9. If the insertion point is not found it inserts at the start.
### Example
You have a wildcard for hair colors (`__haircolors__`) with one being strawberry blonde, but you don't want strawberries. So in that option you add a command to add to the negative prompt, like so:
```text
blonde
strawberry blonde <ppp:stn>strawberry<ppp:/stn>
brunette
```
Then, if that option is chosen this extension will process it later and move that part to the negative prompt.
### Old format
The old format (`<!...!>`) is not supported anymore.
### Notes
Positional insertion commands have less priority that start/end commands, so even if they are at the start or end of the negative prompt, they will end up inside any start/end (and default position) commands.
The content of the negative commands is not processed and is copied as-is to the negative prompt. Other modifiers around the commands are processed in the following way.
### Attention modifiers (weights)
They will be translated to the negative prompt. For example:
* `(red<ppp:stn>square<ppp:/stn>:1.5)` will end up as `(square:1.5)` in the negative prompt
* `(red[<ppp:stn>square<ppp:/stn>]:1.5)` will end up as `(square:1.35)` in the negative prompt (weight=1.5*0.9) if the merge attention option is enabled or `([square]:1.5)` otherwise.
* However `(red<ppp:stn>[square]<ppp:/stn>:1.5)` will end up as `([square]:1.5)` in the negative prompt. The content of the negative tag is copied as is, and is not merged with the surrounding modifier because the insertions happen after the attention merging.
### Prompt editing constructs (alternation and scheduling)
Negative commands inside such constructs will copy the construct to the negative prompt, but separating its elements. For example:
* **Alternation**: `[red<ppp:stn>square<ppp:/stn>|blue<ppp:stn>circle<ppp:/stn>]` will end up as `[square|], [|circle]` in the negative prompt, instead of `[square|circle]`
* **Scheduling**: `[red<ppp:stn>square<ppp:/stn>:blue<ppp:stn>circle<ppp:/stn>:0.5]` will end up as `[square::0.5], [:circle:0.5]` instead of `[square:circle:0.5]`
This should still work as intended, and the only negative point i see is the unnecessary separators.
+85 -24
View File
@@ -3,8 +3,10 @@
_WHITESPACE: /\s+/
STRING: /("(?!"").*?(?<!\\)(\\\\)*?"|'(?!'').*?(?<!\\)(\\\\)*?')/i
BOOLEAN: /true|false/i
INDEX: INT | IDENTIFIER
WILDCARD_NAME: /(?:(?!__|\$\$|[('"])\S)+/
INDEX: INT | IDENTIFIER
IDENTIFIER: CNAME
SIMPLEVALUE: STRING | SIGNED_NUMBER | BOOLEAN
// plain text and weights
?plain: /((?!__|\bAND\b|\${)[^\\()\[\]:<>${]|\\.)+/s // exclude only the starting ones
@@ -25,15 +27,56 @@ promptcomp.4: promptcomppart ( [ ":" numpar ] ( /\bAND\b/ promptcomppart [ ":" n
promptcomppart: content
// simple prompts
?content.2: ( old_content | new_content | plain | specialchars )*
?inside_content.2: ( new_content | plain | specialchars )*
?content_choice.2: ( old_content | new_content | plain_choice | specialchars_choice )*
?content_var.2: ( old_content | new_content | plain_var | specialchars_var )*
?content_negtag.2: ( old_content | new_content_negtag | plain | specialchars_negtag )*
?content_alternate.2: ( old_content | new_content | plain_alternate | specialchars_alternate )*
?old_content.2: ( attention | scheduled | alternate | extranetworktag )+
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | wildcard | choices )+
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | wildcard | choices )+
//#if ALLOW_NEW_CONTENT
?content.2: ( old_content | new_content | plain | specialchars )*
?content_choice.2: ( old_content | new_content | plain_choice | specialchars_choice )*
?content_var.2: ( old_content | new_content | plain_var | specialchars_var )*
?content_negtag.2: ( old_content | new_content_negtag | plain | specialchars_negtag )*
?content_alternate.2: ( old_content | new_content | plain_alternate | specialchars_alternate )*
?content_en.2: (new_content_en | plain | specialchars )*
//#if ALLOW_WILDCARDS ALLOW_CHOICES ALLOW_COMMVARS
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | commandext | wildcard | choices )+
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | commandext | wildcard | choices )+
?new_content_en.3: ( variableset | variableuse | commandset | commandecho | commandif | choices )+
//#elif ALLOW_WILDCARDS !ALLOW_CHOICES ALLOW_COMMVARS
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | commandext | wildcard )+
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | commandext | wildcard )+
?new_content_en.3: ( variableset | variableuse | commandset | commandecho | commandif )+
//#elif !ALLOW_WILDCARDS ALLOW_CHOICES ALLOW_COMMVARS
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | commandext | choices )+
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | commandext | choices )+
?new_content_en.3: ( variableset | variableuse | commandset | commandecho | commandif | choices )+
//#elif ALLOW_WILDCARDS ALLOW_CHOICES !ALLOW_COMMVARS
?new_content.3: ( wildcard | choices )+
?new_content_negtag.3: ( wildcard | choices )+
?new_content_en.3: ( choices )+
//#elif ALLOW_WILDCARDS !ALLOW_CHOICES !ALLOW_COMMVARS
?new_content.3: ( wildcard )+
?new_content_negtag.3: ( wildcard )+
?new_content_en.3: /(?!)./ // never matches
//#elif !ALLOW_WILDCARDS ALLOW_CHOICES !ALLOW_COMMVARS
?new_content.3: ( choices )+
?new_content_negtag.3: ( choices )+
?new_content_en.3: ( choices )+
//#elif !ALLOW_WILDCARDS !ALLOW_CHOICES ALLOW_COMMVARS
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | commandext )+
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | commandext )+
?new_content_en.3: ( variableset | variableuse | commandset | commandecho | commandif )+
//#else
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | commandext | wildcard | choices )+
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | wildcard | commandext | choices )+
?new_content_en.3: ( variableset | variableuse | commandset | commandecho | commandif | choices )+
//#endif
//#else
?content.2: ( old_content | plain | specialchars )*
?content_choice.2: ( old_content | plain_choice | specialchars_choice )*
?content_var.2: ( old_content | plain_var | specialchars_var )*
?content_negtag.2: ( old_content | plain | specialchars_negtag )*
?content_alternate.2: ( old_content | plain_alternate | specialchars_alternate )*
?content_en.2: (plain | specialchars )*
//#endif
// attention modifiers
attention: ( "(" content [ ":" numpar ] ")" ) | ( "[" content "]" )
@@ -44,7 +87,8 @@ alternateoption: content_alternate
scheduled: "[" [ content ":" ] content ":" numpar "]"
// extra network tags
extranetworktag: "<" /(?!ppp:)\w+:/ inside_content ">"
extranetworktag: "<" /(?!ppp:)\w+:/ encontent ">"
?encontent.3: content_en
// command: stn (send to negative)
commandstn: "<ppp:stn" [ _WHITESPACE /s|e|p\d/ ] _WHITESPACE? ">" content_negtag "<ppp:/stn>"
@@ -55,25 +99,42 @@ commandif.2: commandif_if commandif_elif* commandif_else? "<ppp:/if>"
commandif_if: "<ppp:if" _WHITESPACE condition _WHITESPACE? ">" ifvalue
commandif_elif: "<ppp:elif" _WHITESPACE condition _WHITESPACE? ">" ifvalue
commandif_else: "<ppp:else" _WHITESPACE? ">" ifvalue
ifvalue: content
ifvalue.3: content
// conditions
condition: conditionsimplevalue | conditionlistvalue | conditionnocomparison
conditionnocomparison: ( /not/ _WHITESPACE )? IDENTIFIER
conditionsimplevalue: ( /not/ _WHITESPACE )? IDENTIFIER _WHITESPACE ( /not/ _WHITESPACE )? /eq|ne|gt|lt|ge|le|contains/ _WHITESPACE SIMPLEVALUE
conditionlistvalue: ( /not/ _WHITESPACE )? IDENTIFIER _WHITESPACE ( /not/ _WHITESPACE )? /contains|in/ _WHITESPACE listvalue
IDENTIFIER: CNAME
SIMPLEVALUE: STRING | NUMBER | BOOLEAN
listvalue: "(" _WHITESPACE? SIMPLEVALUE ( _WHITESPACE? "," _WHITESPACE? SIMPLEVALUE )* _WHITESPACE? ")"
?condition: grouped_condition | ungrouped_condition
?ungrouped_condition: operation | basic_condition
?grouped_condition.8: "(" _WHITESPACE? condition _WHITESPACE? ")"
?basic_condition: comparison_simple_value | comparison_list_value | truthy_operand
?operation: operation_not | operation_and | operation_or
operation_and: condition _WHITESPACE "and" _WHITESPACE condition
operation_or: condition _WHITESPACE "or" _WHITESPACE condition
operation_not: "not" ( ( _WHITESPACE ungrouped_condition ) | ( _WHITESPACE? grouped_condition ) )
truthy_operand: IDENTIFIER
comparison_simple_value: IDENTIFIER _WHITESPACE ( /not/ _WHITESPACE )? /eq|ne|gt|lt|ge|le|contains/ _WHITESPACE SIMPLEVALUE
comparison_list_value: IDENTIFIER _WHITESPACE ( /not/ _WHITESPACE )? /contains|in/ _WHITESPACE listvalue
listvalue.9: "(" _WHITESPACE? SIMPLEVALUE ( _WHITESPACE? "," _WHITESPACE? SIMPLEVALUE )* _WHITESPACE? ")"
// command: set
commandset: "<ppp:set" _WHITESPACE IDENTIFIER [ _WHITESPACE? /evaluate/ ] [ _WHITESPACE? /add/ ] _WHITESPACE? ">" content "<ppp:/set>"
commandset: "<ppp:set" _WHITESPACE IDENTIFIER [ commandsetmodifiers ] _WHITESPACE? ">" commandsetcontent "<ppp:/set>"
commandsetmodifiers: (_WHITESPACE /evaluate|ifundefined|add/ )+
?commandsetcontent.3: content
// command: echo
commandecho: "<ppp:echo" _WHITESPACE IDENTIFIER _WHITESPACE? ">" [ content "<ppp:/echo>" ]
commandecho: "<ppp:echo" _WHITESPACE IDENTIFIER _WHITESPACE? ">" [ commandechodefault "<ppp:/echo>" ]
?commandechodefault.3: content
// command: ext
commandext: "<ppp:ext" _WHITESPACE commandexttype _WHITESPACE commandextid [ _WHITESPACE? commandextparams ] [ _WHITESPACE? commandextif ] _WHITESPACE? ">" [ commandexttriggers "<ppp:/ext>" ]
commandexttype: [/\$/] IDENTIFIER
?commandextid: STRING | CNAME
?commandextparams: STRING | SIGNED_NUMBER
?commandextif: "if" _WHITESPACE condition
?commandexttriggers.3: content
// variable set
variableset.2: "${" _WHITESPACE? IDENTIFIER _WHITESPACE? [ /\+/ ] "=" [ /!/ ] varvalue "}"
variableset.2: "${" _WHITESPACE? IDENTIFIER [ variablesetmodifiers ] _WHITESPACE? "=" [ /!/ ] varvalue "}"
variablesetmodifiers: /[+?!]+/
// variable use
variableuse.2: "${" _WHITESPACE? IDENTIFIER _WHITESPACE? [ ":" varvalue ] "}"
@@ -84,14 +145,14 @@ wildcard.2: "__" [ choicesoptions_sampler | ( choicesoptions _WHITESPACE? "$$" )
wc_filter: /["']/ ( [ /#/ ] wc_filter_or | ( /#?\^/ WILDCARD_NAME ) ) /["']/
wc_filter_or: wc_filter_and ( _WHITESPACE? "," _WHITESPACE? wc_filter_and )*
wc_filter_and: INDEX ( _WHITESPACE? "+" _WHITESPACE? INDEX )*
wildcardvar: "(" _WHITESPACE? IDENTIFIER _WHITESPACE? "=" varvalue ")"
wildcardvar.7: "(" _WHITESPACE? IDENTIFIER _WHITESPACE? "=" varvalue ")"
// choices
choices.2: "{" [ choicesoptions_sampler | ( choicesoptions _WHITESPACE? "$$" ) ] choice ( "|" choice )* "}"
choicesoptions: [ choicesoptions_sampler ] [ _WHITESPACE? choicesoptions_rep ] ( ( [ _WHITESPACE? choicesoptions_from ] "-" [ _WHITESPACE? choicesoptions_to ] ) | [ _WHITESPACE? choicesoptions_num ] ) [ _WHITESPACE? choicesoptions_sep ]
choicesoptions: [ choicesoptions_sampler ] [ _WHITESPACE? choicesoptions_flags ] ( ( [ _WHITESPACE? choicesoptions_from ] "-" [ _WHITESPACE? choicesoptions_to ] ) | [ _WHITESPACE? choicesoptions_num ] ) [ _WHITESPACE? choicesoptions_sep ]
choicesoptions_sampler: /[~@]/ // ~ for random, @ for cyclical
choicesoptions_rep: /r/
choicesoptions_flags: /[ro]{1,2}/ // r for repeating, o for optional
choicesoptions_num: INT
choicesoptions_from: INT
choicesoptions_to: INT
File diff suppressed because one or more lines are too long
Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 20 KiB

+10 -2
View File
@@ -1,5 +1,13 @@
import os
import launch
requirements_filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), "requirements.txt")
launch.run_pip(f'install -r "{requirements_filename}"', "requirements for Prompt Post-Processor")
try:
from modules.launch_utils import requirements_met, run_pip # A1111
if not requirements_met(requirements_filename):
run_pip(f'install -r "{requirements_filename}"', "requirements for Prompt Post-Processor")
except ImportError:
import launch
launch.run_pip(f'install -r "{requirements_filename}"', "requirements for Prompt Post-Processor")
+1199 -376
View File
File diff suppressed because it is too large Load Diff
+10 -3
View File
@@ -1,15 +1,20 @@
from collections import OrderedDict
from logging import Logger
from typing import Tuple
from ppp_logging import DEBUG_LEVEL # pylint: disable=import-error
class PPPLRUCache:
ProcessInput = Tuple[int, str, str]
ProcessResult = Tuple[str, str]
ProcessInput = Tuple[int, int, str, str] # (seed, wildcards_hash, positive_prompt, negative_prompt)
ProcessResult = Tuple[str, str] # (positive_prompt, negative_prompt)
def __init__(self, capacity: int):
def __init__(self, capacity: int, logger: Logger = None, debug_level: DEBUG_LEVEL = DEBUG_LEVEL.none):
self.cache = OrderedDict()
self.capacity = capacity
self._logger = logger
self._debug_level = debug_level
def get(self, key: ProcessInput) -> ProcessResult:
if key not in self.cache:
@@ -22,3 +27,5 @@ class PPPLRUCache:
self.cache.move_to_end(key)
if len(self.cache) > self.capacity:
self.cache.popitem(last=False)
# if self._logger is not None and self._debug_level != DEBUG_LEVEL.none:
# self._logger.debug(f"Cache size: {self.cache.__sizeof__()}")
+260 -58
View File
@@ -1,57 +1,54 @@
# pylint: disable=missing-module-docstring, missing-class-docstring, missing-function-docstring, invalid-name
import os
# pylint: disable=import-error
import folder_paths # type: ignore
import nodes # type: ignore
import nodes
from .ppp import PromptPostProcessor
from .ppp_hosts import SUPPORTED_APPS
from .ppp_logging import DEBUG_LEVEL, PromptPostProcessorLogFactory
from .ppp_wildcards import PPPWildcards
from .ppp_enmappings import PPPExtraNetworkMappings
if __name__ == "__main__":
raise SystemExit("This script must be run from ComfyUI")
class PromptPostProcessorComfyUINode:
VERSION = PromptPostProcessor.VERSION
"""
Node for processing prompts.
"""
logger = None
def __init__(self):
lf = PromptPostProcessorLogFactory()
lf = PromptPostProcessorLogFactory(SUPPORTED_APPS.comfyui)
self.logger = lf.log
grammar_filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), "grammar.lark")
with open(grammar_filename, "r", encoding="utf-8") as file:
self.grammar_content = file.read()
self.wildcards_obj = PPPWildcards(lf.log)
self.extranetwork_mappings_obj = PPPExtraNetworkMappings(lf.log)
self.logger.info(f"{PromptPostProcessor.NAME} {PromptPostProcessor.VERSION} initialized")
class SmartType(str):
def __ne__(self, other):
if self == "*" or other == "*":
return False
selfset = set(self.split(","))
otherset = set(other.split(","))
return not otherset.issubset(selfset)
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"model": (
"MODEL",
{
"forceInput": True,
},
),
"modelname": (
"STRING",
{
"default": "",
"forceInput": True,
},
),
"pos_prompt": (
"STRING",
{
"multiline": True,
"default": "",
"dynamicPrompts": False,
"forceInput": True,
},
),
"neg_prompt": (
@@ -60,16 +57,30 @@ class PromptPostProcessorComfyUINode:
"multiline": True,
"default": "",
"dynamicPrompts": False,
"forceInput": True,
},
),
},
"optional": {
"model": (
cls.SmartType("MODEL,STRING"),
{
"default": "",
"placeholder": "internal model class name",
"forceInput": True,
},
),
"modelname": (
"STRING",
{
"default": "",
"placeholder": "full path of the model",
"dynamicPrompts": False,
},
),
"seed": (
"INT",
{
"default": -1,
"forceInput": False,
},
),
"debug_level": (
@@ -79,12 +90,21 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Debug level",
},
),
"pony_substrings": (
"on_warnings": (
[e.value for e in PromptPostProcessor.ONWARNING_CHOICES],
{
"default": PromptPostProcessor.ONWARNING_CHOICES.warn.value,
"tooltip": "How to handle invalid content warnings",
},
),
"variants_definitions": (
"STRING",
{
"default": PromptPostProcessor.DEFAULT_PONY_SUBSTRINGS,
"placeholder": "comma separated list",
"tooltip": "Comma separated list of substrings to look for in the modelname to determine if the model is a pony model",
"default": PromptPostProcessor.DEFAULT_VARIANTS_DEFINITIONS,
"multiline": True,
"placeholder": "",
"tooltip": "Definitions for variant models to be recognized based on strings found in the full filename. Format for each line is: 'name(kind)=comma separated list of substrings (case insensitive)' with kind being one of the base model types or not specified",
"dynamicPrompts": False,
},
),
"wc_process_wildcards": (
@@ -101,12 +121,23 @@ class PromptPostProcessorComfyUINode:
{
"default": "",
"tooltip": "Comma separated list of wildcards folders",
"dynamicPrompts": False,
},
),
"wc_wildcards_input": (
"STRING",
{
"default": "",
"multiline": True,
"placeholder": "wildcards definitions",
"tooltip": "Wildcards definitions in yaml/json format",
"dynamicPrompts": False,
},
),
"wc_if_wildcards": (
[e.value for e in PromptPostProcessor.IFWILDCARDS_CHOICES],
{
"default": PromptPostProcessor.IFWILDCARDS_CHOICES.ignore.value,
"default": PromptPostProcessor.IFWILDCARDS_CHOICES.stop.value,
"tooltip": "How to handle invalid wildcards in the prompt",
},
),
@@ -115,6 +146,7 @@ class PromptPostProcessorComfyUINode:
{
"default": PromptPostProcessor.DEFAULT_CHOICE_SEPARATOR,
"tooltip": "Default separator for selected choices",
"dynamicPrompts": False,
},
),
"wc_keep_choices_order": (
@@ -131,6 +163,7 @@ class PromptPostProcessorComfyUINode:
{
"default": PromptPostProcessor.DEFAULT_STN_SEPARATOR,
"tooltip": "Separator for the content added to the negative prompt",
"dynamicPrompts": False,
},
),
"stn_ignore_repeats": (
@@ -178,10 +211,19 @@ class PromptPostProcessorComfyUINode:
"label_off": "No",
},
),
"cleanup_extra_separators_include_eol": (
"BOOLEAN",
{
"default": False,
"tooltip": "Extra separators options also remove EOLs",
"label_on": "Yes",
"label_off": "No",
},
),
"cleanup_breaks": (
"BOOLEAN",
{
"default": True,
"default": False,
"tooltip": "Cleanup around BREAKs",
"label_on": "Yes",
"label_off": "No",
@@ -199,7 +241,7 @@ class PromptPostProcessorComfyUINode:
"cleanup_ands": (
"BOOLEAN",
{
"default": True,
"default": False,
"tooltip": "Cleanup around ANDs",
"label_on": "Yes",
"label_off": "No",
@@ -241,16 +283,49 @@ class PromptPostProcessorComfyUINode:
"label_off": "No",
},
),
"en_mappings_folders": (
"STRING",
{
"default": "",
"tooltip": "Comma separated list of extranetwork mappings folders",
"dynamicPrompts": False,
},
),
"en_mappings_input": (
"STRING",
{
"default": "",
"multiline": True,
"placeholder": "extranetwork mappings definitions",
"tooltip": "Extranetwork mappings definitions in yaml format",
"dynamicPrompts": False,
},
),
},
}
@classmethod
def VALIDATE_INPUTS(cls, input_types: dict[str, str]):
it = cls.INPUT_TYPES()
expected = {
k: cls.SmartType("COMBO,STRING") if isinstance(v[0], list) else v[0] # we allow string for combos
for k, v in {**it["required"], **it["optional"]}.items()
}
for input_name, input_type in input_types.items():
t = expected[input_name]
if input_type != t:
return f"Invalid type for input '{input_name}': {input_type} (expected {t})"
return True
RETURN_TYPES = (
"STRING",
"STRING",
"PPP_DICT",
)
RETURN_NAMES = (
"pos_prompt",
"neg_prompt",
"variables",
)
FUNCTION = "process"
@@ -266,9 +341,11 @@ class PromptPostProcessorComfyUINode:
neg_prompt,
seed,
debug_level, # pylint: disable=unused-argument
pony_substrings,
on_warnings,
variants_definitions,
wc_process_wildcards,
wc_wildcards_folders,
wc_wildcards_input,
wc_if_wildcards,
wc_choice_separator,
wc_keep_choices_order,
@@ -278,6 +355,7 @@ class PromptPostProcessorComfyUINode:
cleanup_empty_constructs,
cleanup_extra_separators,
cleanup_extra_separators2,
cleanup_extra_separators_include_eol,
cleanup_breaks,
cleanup_breaks_eol,
cleanup_ands,
@@ -285,16 +363,24 @@ class PromptPostProcessorComfyUINode:
cleanup_extranetwork_tags,
cleanup_merge_attention,
remove_extranetwork_tags,
en_mappings_folders,
en_mappings_input,
):
new_run = {
if wc_process_wildcards:
return float(
"NaN"
) # since we can't detect changes in wildcards we assume they are always changed when enabled
new_run = { # everything except debug_level
"model": model,
"modelname": modelname,
"pos_prompt": pos_prompt,
"neg_prompt": neg_prompt,
"seed": seed,
"pony_substrings": pony_substrings,
"on_warnings": on_warnings,
"variants_definitions": variants_definitions,
"process_wildcards": wc_process_wildcards,
"wildcards_folders": wc_wildcards_folders,
"wildcards_input": wc_wildcards_input,
"if_wildcards": wc_if_wildcards,
"choice_separator": wc_choice_separator,
"keep_choices_order": wc_keep_choices_order,
@@ -304,6 +390,7 @@ class PromptPostProcessorComfyUINode:
"cleanup_empty_constructs": cleanup_empty_constructs,
"cleanup_extra_separators": cleanup_extra_separators,
"cleanup_extra_separators2": cleanup_extra_separators2,
"cleanup_extra_separators_include_eol": cleanup_extra_separators_include_eol,
"cleanup_breaks": cleanup_breaks,
"cleanup_breaks_eol": cleanup_breaks_eol,
"cleanup_ands": cleanup_ands,
@@ -311,6 +398,8 @@ class PromptPostProcessorComfyUINode:
"cleanup_extranetwork_tags": cleanup_extranetwork_tags,
"cleanup_merge_attention": cleanup_merge_attention,
"remove_extranetwork_tags": remove_extranetwork_tags,
"en_mappings_folders": en_mappings_folders,
"en_mappings_input": en_mappings_input,
}
return new_run.__hash__
# return float("NaN")
@@ -323,9 +412,11 @@ class PromptPostProcessorComfyUINode:
neg_prompt,
seed,
debug_level,
pony_substrings,
on_warnings,
variants_definitions,
wc_process_wildcards,
wc_wildcards_folders,
wc_wildcards_input,
wc_if_wildcards,
wc_choice_separator,
wc_keep_choices_order,
@@ -335,6 +426,7 @@ class PromptPostProcessorComfyUINode:
cleanup_empty_constructs,
cleanup_extra_separators,
cleanup_extra_separators2,
cleanup_extra_separators_include_eol,
cleanup_breaks,
cleanup_breaks_eol,
cleanup_ands,
@@ -342,33 +434,54 @@ class PromptPostProcessorComfyUINode:
cleanup_extranetwork_tags,
cleanup_merge_attention,
remove_extranetwork_tags,
en_mappings_folders,
en_mappings_input,
):
modelclass = (
model.model.model_config.__class__.__name__ if model is not None and not isinstance(model, str) else model
) or ""
if modelclass == "":
self.logger.warning("Model class is not provided. System variables might not be properly set.")
if modelname == "":
self.logger.warning("Modelname is not provided. System variables will not be properly set.")
# model class values in ComfyUI\comfy\supported_models.py
env_info = {
"app": "comfyui",
"app": SUPPORTED_APPS.comfyui.value,
"models_path": folder_paths.models_dir,
"model_filename": modelname, # path is relative to checkpoints folder
"model_class": model.model.model_config.__class__.__name__,
"is_sd1": model.model.model_config.__class__.__name__ in ("SD15", "SD15_instructpix2pix"),
"is_sd2": model.model.model_config.__class__.__name__ in ("SD20", "SD21UnclipL", "SD21UnclipH"),
"is_sdxl": model.model.model_config.__class__.__name__
in (
"SDXL",
"SDXLRefiner",
"SDXL_instructpix2pix",
"Segmind_Vega",
"KOALA_700M",
"KOALA_1B",
"model_filename": modelname or "", # path is relative to checkpoints folder
"model_class": modelclass,
"is_sd1": modelclass in ("SD15", "SD15_instructpix2pix"),
"is_sd2": modelclass in ("SD20", "SD21UnclipL", "SD21UnclipH", "LotusD"),
"is_sdxl": (
modelclass in ("SDXL", "SDXLRefiner", "SDXL_instructpix2pix", "Segmind_Vega", "KOALA_700M", "KOALA_1B")
),
"is_ssd": model.model.model_config.__class__.__name__ in ("SSD1B"),
"is_sd3": model.model.model_config.__class__.__name__ in ("SD3"),
"is_flux": model.model.model_config.__class__.__name__ in ("Flux"),
"is_auraflow": model.model.model_config.__class__.__name__ in ("AuraFlow"),
"is_ssd": modelclass in ("SSD1B",),
"is_sd3": modelclass in ("SD3",),
"is_flux": modelclass in ("Flux", "FluxInpaint", "FluxSchnell"),
"is_auraflow": modelclass in ("AuraFlow",),
"is_pixart": modelclass in ("PixArtAlpha", "PixArtSigma"),
"is_lumina2": modelclass in ("Lumina2",),
"is_ltxv": modelclass in ("LTXV",),
"is_cosmos": modelclass in ("CosmosT2V", "CosmosI2V"),
"is_genmomochi": modelclass in ("GenmoMochi",),
"is_hunyuan": modelclass in ("HunyuanDiT", "HunyuanDiT1"),
"is_hunyuanvideo": modelclass in ("HunyuanVideo", "HunyuanVideoI2V", "HunyuanVideoSkyreelsI2V"),
"is_hunyuan3d": modelclass in ("Hunyuan3Dv2", "Hunyuan3Dv2mini"),
"is_wanvideo": modelclass in ("WAN21_T2V", "WAN21_I2V", "WAN21_FunControl2V"),
"is_hidream": modelclass in ("HiDream",),
}
# SVD_img2vid, SVD3D_u, SVD3_p, Stable_Zero123, SD_X4Upscaler,
# Stable_Cascade_C, Stable_Cascade_B, StableAudio
# Also supported: SVD_img2vid, SVD3D_u, SVD3_p, Stable_Zero123, SD_X4Upscaler, Stable_Cascade_C, Stable_Cascade_B, StableAudio
if wc_wildcards_folders == "":
wc_wildcards_folders = ",".join(folder_paths.get_folder_paths("wildcards") or [])
try:
fp1 = folder_paths.get_folder_paths("ppp_wildcards")
except Exception: # pylint: disable=W0718
fp1 = None
try:
fp2 = folder_paths.get_folder_paths("wildcards")
except Exception: # pylint: disable=W0718
fp2 = None
wc_wildcards_folders = ",".join(fp1 or fp2 or [])
if wc_wildcards_folders == "":
wc_wildcards_folders = os.getenv("WILDCARD_DIR", PPPWildcards.DEFAULT_WILDCARDS_FOLDER)
wildcards_folders = [
@@ -376,9 +489,28 @@ class PromptPostProcessorComfyUINode:
for f in wc_wildcards_folders.split(",")
if f.strip() != ""
]
if en_mappings_folders == "":
try:
fp3 = folder_paths.get_folder_paths("ppp_extranetworkmappings")
except Exception: # pylint: disable=W0718
fp3 = None
en_mappings_folders = ",".join(fp3 or [])
if en_mappings_folders == "":
en_mappings_folders = os.getenv(
"EXTRANETWORKMAPPINGS_DIR", PPPExtraNetworkMappings.DEFAULT_ENMAPPINGS_FOLDER
)
enmappings_folders = [
(f if os.path.isabs(f) else os.path.abspath(os.path.join(folder_paths.models_dir, f)))
for f in en_mappings_folders.split(",")
if f.strip() != ""
]
if variants_definitions != "" and not "=" in variants_definitions: # mainly to warn about the old format
raise ValueError("Invalid variants_definitions format")
options = {
"debug_level": debug_level,
"pony_substrings": pony_substrings,
"on_warnings": on_warnings,
"variants_definitions": variants_definitions,
"process_wildcards": wc_process_wildcards,
"if_wildcards": wc_if_wildcards,
"choice_separator": wc_choice_separator,
@@ -389,6 +521,7 @@ class PromptPostProcessorComfyUINode:
"cleanup_empty_constructs": cleanup_empty_constructs,
"cleanup_extra_separators": cleanup_extra_separators,
"cleanup_extra_separators2": cleanup_extra_separators2,
"cleanup_extra_separators_include_eol": cleanup_extra_separators_include_eol,
"cleanup_breaks": cleanup_breaks,
"cleanup_breaks_eol": cleanup_breaks_eol,
"cleanup_ands": cleanup_ands,
@@ -397,15 +530,84 @@ class PromptPostProcessorComfyUINode:
"cleanup_merge_attention": cleanup_merge_attention,
"remove_extranetwork_tags": remove_extranetwork_tags,
}
self.wildcards_obj.refresh_wildcards(debug_level, wildcards_folders if options["process_wildcards"] else None)
ppp = PromptPostProcessor(
self.logger, self.interrupt, env_info, options, self.grammar_content, self.wildcards_obj
self.wildcards_obj.refresh_wildcards(
debug_level,
wildcards_folders if options["process_wildcards"] else None,
wc_wildcards_input,
)
pos_prompt, neg_prompt = ppp.process_prompt(pos_prompt, neg_prompt, seed if seed is not None else 1)
self.extranetwork_mappings_obj.refresh_extranetwork_mappings(
debug_level,
enmappings_folders,
en_mappings_input,
)
ppp = PromptPostProcessor(
self.logger,
self.interrupt,
env_info,
options,
self.grammar_content,
self.wildcards_obj,
self.extranetwork_mappings_obj,
)
pos_prompt, neg_prompt, variables = ppp.process_prompt(pos_prompt, neg_prompt, seed if seed is not None else 1)
return (
pos_prompt,
neg_prompt,
variables,
)
def interrupt(self):
nodes.interrupt_processing(True)
class PromptPostProcessorSelectVariableComfyUINode:
"""
Node for selecting a variable from a dictionary.
"""
def __init__(self):
pass
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"variables": (
"PPP_DICT",
{
"forceInput": True,
},
),
},
"optional": {
"name": (
"STRING",
{
"placeholder": "variable name",
"multiline": False,
"default": "",
"dynamicPrompts": False,
},
),
},
}
RETURN_TYPES = ("STRING",)
RETURN_NAMES = ("value",)
FUNCTION = "select"
CATEGORY = "ACB"
def select(
self,
variables: dict[str, str],
name: str,
):
value = ""
if variables:
if name == "":
value = "\n".join(f"{k}: {v}" for k, v in variables.items())
elif name in variables:
value = variables[name]
return (value,)
+269
View File
@@ -0,0 +1,269 @@
import os
from typing import Optional
import logging
import yaml
from ppp_logging import DEBUG_LEVEL # pylint: disable=import-error
from ppp_utils import deep_freeze # pylint: disable=import-error
class PPPENMappingVariant:
"""
A class to represent a variant of an extra network mapping.
Attributes:
condition (str): The condition for the variant.
name (str): The name of the variant.
parameters (float|str): The parameters for the variant.
triggers (list[str]): The triggers for the variant.
weight (float): The weight for the variant when multiple variants apply.
"""
def __init__(self, condition: str, name: str, parameters: float | str, triggers: list[str], weight: float):
self.condition: str = condition
self.name: str = name
self.parameters: float | str = parameters
self.triggers: list[str] = triggers
self.weight: float = weight
class PPPENMapping:
"""
A extra network mapping object.
Attributes:
kind (str): The kind of the extra network.
name (str): The name of the extra network mapping.
file (str): The path to the file where the extranetwork mapping is defined.
variants (list[PPPENMappingVariant]): The processed variants of the extranetwork mapping.
"""
def __init__(self, fullpath: str, kind: str, name: str, variants: list[dict]):
self.file: str = fullpath
self.kind: str = kind
self.name: str = name
self.variants: list[PPPENMappingVariant] = [
PPPENMappingVariant(
**{**{"condition": None, "name": None, "parameters": None, "triggers": None, "weight": 1.0}, **v}
)
for v in variants
]
def __hash__(self) -> int:
t = (self.kind, self.name, deep_freeze(self.variants))
return hash(t)
def __sizeof__(self):
return self.kind.__sizeof__() + self.name.__sizeof__() + self.file.__sizeof__() + self.variants.__sizeof__()
class PPPExtraNetworkMappings:
"""
A class to manage extra network mappings.
Attributes:
extranetwork_maps (dict[str, PPPENMapping]): The extra network mappings.
"""
DEFAULT_ENMAPPINGS_FOLDER = "extranetworkmappings"
LOCALINPUT_FILENAME = "#INPUT"
def __init__(self, logger):
self.__logger: logging.Logger = logger
self.__debug_level = DEBUG_LEVEL.none
self.__enmappings_folders = []
self.__enmappings_files = {}
self.extranetwork_mappings: dict[str, PPPENMapping] = {}
def __hash__(self) -> int:
return hash(deep_freeze(self.extranetwork_mappings))
def __sizeof__(self):
return (
self.extranetwork_mappings.__sizeof__()
+ self.__enmappings_folders.__sizeof__()
+ self.__enmappings_files.__sizeof__()
)
def refresh_extranetwork_mappings(
self, debug_level: DEBUG_LEVEL, enmappings_folders: Optional[list[str]], enmappings_input: str = None
):
"""
Initialize the extra network mappings.
"""
self.__debug_level = debug_level
self.__enmappings_folders = enmappings_folders or []
# if self.__debug_level != DEBUG_LEVEL.none:
# self.__logger.info("Refreshing extra network mappings...")
# t1 = time.monotonic_ns()
for fullpath in list(self.__enmappings_files.keys()):
if fullpath != self.LOCALINPUT_FILENAME:
path = os.path.dirname(fullpath)
if not os.path.exists(fullpath) or not any(
os.path.commonpath([path, folder]) == folder for folder in self.__enmappings_folders
):
self.__remove_extranetwork_mappings_from_path(fullpath)
elif enmappings_input is None:
self.__remove_extranetwork_mappings_from_path(fullpath)
if enmappings_folders is not None or enmappings_input is not None:
if enmappings_folders is not None:
for f in self.__enmappings_folders:
self.__get_extranetwork_mappings_in_directory(f)
if enmappings_input is not None:
self.__get_extranetwork_mappings_in_input(enmappings_input)
else:
self.extranetwork_mappings = {}
self.__enmappings_files = {}
# t2 = time.monotonic_ns()
# if self.__debug_level != DEBUG_LEVEL.none:
# self.__logger.info(f"Extra network mappings refresh time: {(t2 - t1) / 1_000_000_000:.3f} seconds")
# def get_extranetwork_mappings(self, key: str) -> list[PPPENMapping]:
# """
# Get all extra network mappings that match a key.
#
# Args:
# key (str): The key to match (kind:name).
#
# Returns:
# list: A list of all extra network mappings that match the key.
# """
# keys = sorted(fnmatch.filter(self.extranetwork_mappings.keys(), key))
# return [self.extranetwork_mappings[k] for k in keys]
def __remove_extranetwork_mappings_from_path(self, full_path: str, debug=True):
"""
Clear all extra network mappings in a file.
Args:
full_path (str): The path to the file.
debug (bool): Whether to print debug messages or not.
"""
last_modified_cached = self.__enmappings_files.get(full_path, None) # a time or a hash
if debug and last_modified_cached is not None and self.__debug_level != DEBUG_LEVEL.none:
if full_path == self.LOCALINPUT_FILENAME:
self.__logger.debug("Removing extra network mappings from input")
else:
self.__logger.debug(f"Removing extra network mappings from file: {full_path}")
if full_path in self.__enmappings_files.keys():
del self.__enmappings_files[full_path]
for key in list(self.extranetwork_mappings.keys()):
if self.extranetwork_mappings[key].file == full_path:
del self.extranetwork_mappings[key]
def __get_extranetwork_mappings_in_file(self, full_path: str):
"""
Get all extra network mappings in a file.
Args:
full_path (str): The path to the file.
"""
last_modified = os.path.getmtime(full_path)
last_modified_cached = self.__enmappings_files.get(full_path, None)
if last_modified_cached is not None and last_modified == self.__enmappings_files[full_path]:
return
filename = os.path.basename(full_path)
_, extension = os.path.splitext(filename)
if extension not in (".yaml", ".yml", ".json"):
return
self.__remove_extranetwork_mappings_from_path(full_path, False)
if last_modified_cached is not None and self.__debug_level != DEBUG_LEVEL.none:
self.__logger.debug(f"Updating extra network mappings from file: {full_path}")
self.__get_extranetwork_mappings_in_structured_file(full_path)
self.__enmappings_files[full_path] = last_modified
def __get_extranetwork_mappings_in_input(self, enmappings_input: str):
"""
Get all extra network mappings in the string.
Args:
enmappings_input (str): The input string containing extra network mappings in yaml format.
"""
new_h = hash(enmappings_input)
h = self.__enmappings_files.get(self.LOCALINPUT_FILENAME, None)
if h == new_h:
return
self.__remove_extranetwork_mappings_from_path(self.LOCALINPUT_FILENAME, False)
if h is not None and self.__debug_level != DEBUG_LEVEL.none:
self.__logger.debug("Updating extra network mappings from input")
enmappings_input = enmappings_input.strip()
if enmappings_input != "":
try:
content = yaml.safe_load(enmappings_input)
except yaml.YAMLError as e:
self.__logger.warning(f"Invalid format for input extra network mappings: {e}")
return
if content is not None:
self.__add_extranetwork_mapping(content, self.LOCALINPUT_FILENAME)
self.__enmappings_files[self.LOCALINPUT_FILENAME] = new_h
def __add_extranetwork_mapping(self, content: dict[str, dict[str, list[dict]]], full_path: str):
"""
Add an extra network mapping to the extra network mappings dictionary.
Args:
content (object): The content of the extra network mapping.
full_path (str): The path to the file that contains it.
"""
if not isinstance(content, dict):
self.__logger.warning(f"Invalid extra network mapping in file '{full_path}'!")
return
for kind, maps in content.items():
if not isinstance(maps, dict):
self.__logger.warning(f"Invalid extra network mapping definition for '{kind}:*' in file '{full_path}'!")
else:
for name, variants in maps.items():
key = f"{kind}:{name}"
if not isinstance(variants, list):
self.__logger.warning(
f"Invalid extra network mapping definition for '{key}' in file '{full_path}'!"
)
elif self.extranetwork_mappings.get(key, None) is not None:
self.__logger.warning(
f"Duplicate extra network mapping '{key}' in file '{full_path}' and '{self.extranetwork_mappings[key].file}'!"
)
elif not isinstance(variants, list) or not all(isinstance(v, dict) for v in variants):
self.__logger.warning(
f"Invalid extra network mapping definition for '{key}' in file '{full_path}'!"
)
else:
self.extranetwork_mappings[key] = PPPENMapping(full_path, kind, name, variants)
def __get_extranetwork_mappings_in_structured_file(self, full_path):
"""
Get all extra network mappings in a structured file.
Args:
full_path (str): The path to the file.
base (str): The base path for the extra network mappings.
"""
try:
try:
with open(full_path, "r", encoding="utf-8") as file:
content = yaml.safe_load(file)
except: # pylint: disable=bare-except
self.__logger.warning(f"Could not read file '{full_path}' with utf-8 encoding, trying windows-1252...")
with open(full_path, "r", encoding="windows-1252") as file:
content = yaml.safe_load(file)
self.__add_extranetwork_mapping(content, full_path)
except Exception as e: # pylint: disable=broad-except
self.__logger.error(f"Error reading extra network mappings from file '{full_path}': {e}")
def __get_extranetwork_mappings_in_directory(self, directory: str):
"""
Get all extra network mappings in a directory.
Args:
directory (str): The path to the directory.
"""
if not os.path.exists(directory):
self.__logger.warning(f"Extra network mappings directory '{directory}' does not exist!")
return
for filename in os.listdir(directory):
full_path = os.path.abspath(os.path.join(directory, filename))
if os.path.basename(full_path).startswith("."):
continue
if os.path.isdir(full_path):
self.__get_extranetwork_mappings_in_directory(full_path)
elif os.path.isfile(full_path):
self.__get_extranetwork_mappings_in_file(full_path)
+17
View File
@@ -0,0 +1,17 @@
from enum import Enum
class SUPPORTED_APPS(Enum):
comfyui = "comfyui"
a1111 = "a1111"
forge = "forge"
reforge = "reforge"
sdnext = "sdnext"
SUPPORTED_APPS_NAMES = {
SUPPORTED_APPS.comfyui: "ComfyUI",
SUPPORTED_APPS.sdnext: "SD.Next",
SUPPORTED_APPS.forge: "Forge",
SUPPORTED_APPS.reforge: "reForge",
SUPPORTED_APPS.a1111: "A1111 (or compatible)",
}
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

+4 -3
View File
@@ -2,6 +2,7 @@ from enum import Enum
import logging
import sys
import copy
from ppp_hosts import SUPPORTED_APPS # pylint: disable=import-error
class DEBUG_LEVEL(Enum):
@@ -52,14 +53,14 @@ class PromptPostProcessorLogFactory: # pylint: disable=too-few-public-methods
colored_record.levelname = f"{seq}{levelname:8s}{self.COLORS['RESET']}"
return super().format(colored_record)
def __init__(self):
def __init__(self, app: SUPPORTED_APPS = None): # pylint: disable=unused-argument
"""
Initializes the PromptPostProcessor class.
This method sets up the logger for the PromptPostProcessor class and configures its log level and handlers.
Args:
None
app (SUPPORTED_APPS): The application for which the logger is being created.
Returns:
None
@@ -68,7 +69,7 @@ class PromptPostProcessorLogFactory: # pylint: disable=too-few-public-methods
ppplog.propagate = False
if not ppplog.handlers:
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(self.ColoredFormatter("%(asctime)s %(levelname)s %(message)s")) # Used in A1111 / Forge / reForge / ComfyUI, but not in SD.Next
handler.setFormatter(self.ColoredFormatter("%(asctime)s %(levelname)s %(message)s"))
ppplog.addHandler(handler)
ppplog.setLevel(logging.DEBUG)
self.log = PromptPostProcessorLogCustomAdapter(ppplog)
+17
View File
@@ -0,0 +1,17 @@
def deep_freeze(obj):
"""
Deep freeze an object.
Args:
obj (object): The object to freeze.
Returns:
object: The frozen object.
"""
if isinstance(obj, dict):
return tuple((k, deep_freeze(v)) for k, v in sorted(obj.items()))
if isinstance(obj, list):
return tuple(deep_freeze(i) for i in obj)
if isinstance(obj, set):
return tuple(deep_freeze(i) for i in sorted(obj))
return obj
+329 -106
View File
@@ -1,60 +1,113 @@
import fnmatch
import os
import json
from typing import Optional
import logging
import yaml
from ppp_logging import DEBUG_LEVEL
from ppp_logging import DEBUG_LEVEL # pylint: disable=import-error
from ppp_utils import deep_freeze # pylint: disable=import-error
class PPPWildcard:
"""
A wildcard object.
Attributes:
key (str): The key of the wildcard.
file (str): The path to the file where the wildcard is defined.
unprocessed_choices (list[str]): The unprocessed choices of the wildcard.
choices (list[dict]): The processed choices of the wildcard.
options (dict): The options of the wildcard.
"""
def __init__(self, fullpath: str, key: str, choices: list[str]):
self.key: str = key
self.file: str = fullpath
self.choices: list[str] = choices
self.choices_obj: list[object] = None
self.options_obj: object = None
self.unprocessed_choices: list[str] = choices
self.choices: list[dict] = None
self.options: dict = None
def __hash__(self) -> int:
t = (self.key, deep_freeze(self.unprocessed_choices))
return hash(t)
def __sizeof__(self):
return (
self.key.__sizeof__()
+ self.file.__sizeof__()
+ self.unprocessed_choices.__sizeof__()
+ self.choices.__sizeof__()
+ self.options.__sizeof__()
)
class PPPWildcards:
"""
A class to manage wildcards.
Attributes:
wildcards (dict[str, PPPWildcard]): The wildcards.
"""
DEFAULT_WILDCARDS_FOLDER = "wildcards"
LOCALINPUT_FILENAME = "#INPUT"
def __init__(self, logger):
self.logger = logger
self.debug_level = DEBUG_LEVEL.none
self.wildcards_folders = []
self.__logger: logging.Logger = logger
self.__debug_level = DEBUG_LEVEL.none
self.__wildcards_folders = []
self.__wildcard_files = {}
self.wildcards: dict[str, PPPWildcard] = {}
self.wildcard_files = {}
def refresh_wildcards(self, debug_level: DEBUG_LEVEL, wildcards_folders: Optional[list[str]]):
def __hash__(self) -> int:
return hash(deep_freeze(self.wildcards))
def __sizeof__(self):
return self.wildcards.__sizeof__() + self.__wildcards_folders.__sizeof__() + self.__wildcard_files.__sizeof__()
def refresh_wildcards(
self, debug_level: DEBUG_LEVEL, wildcards_folders: Optional[list[str]], wildcards_input: str = None
):
"""
Initialize the wildcards.
"""
self.debug_level = debug_level
self.wildcards_folders = wildcards_folders
if wildcards_folders is not None:
# if self.debug_level != DEBUG_LEVEL.none:
# self.logger.info("Initializing wildcards...")
# t1 = time.time()
for fullpath in list(self.wildcard_files.keys()):
self.__debug_level = debug_level
self.__wildcards_folders = wildcards_folders or []
# if self.__debug_level != DEBUG_LEVEL.none:
# self.__logger.info("Refreshing wildcards...")
# t1 = time.monotonic_ns()
for fullpath in list(self.__wildcard_files.keys()):
if fullpath != self.LOCALINPUT_FILENAME:
path = os.path.dirname(fullpath)
if not os.path.exists(fullpath) or not any(
os.path.commonpath([path, folder]) == folder for folder in self.wildcards_folders
os.path.commonpath([path, folder]) == folder for folder in self.__wildcards_folders
):
self.__remove_wildcards_from_file(fullpath)
for f in self.wildcards_folders:
self.__get_wildcards_in_directory(f, f)
# t2 = time.time()
# if self.debug_level != DEBUG_LEVEL.none:
# self.logger.info(f"Wildcards init time: {t2 - t1:.3f} seconds")
self.__remove_wildcards_from_path(fullpath)
elif wildcards_input is None:
self.__remove_wildcards_from_path(fullpath)
if wildcards_folders is not None or wildcards_input is not None:
if wildcards_folders is not None:
for f in self.__wildcards_folders:
self.__get_wildcards_in_directory(f, f)
if wildcards_input is not None:
self.__get_wildcards_in_input(wildcards_input)
else:
self.wildcards_folders = []
self.wildcards = {}
self.wildcard_files = {}
self.__wildcard_files = {}
# t2 = time.monotonic_ns()
# if self.__debug_level != DEBUG_LEVEL.none:
# self.__logger.info(f"Wildcards refresh time: {(t2 - t1) / 1_000_000_000:.3f} seconds")
def get_wildcards(self, key: str) -> list[PPPWildcard]:
"""
Get all wildcards that match a key.
Args:
key (str): The key to match.
Returns:
list: A list of all wildcards that match the key.
"""
keys = sorted(fnmatch.filter(self.wildcards.keys(), key))
return [self.wildcards[k] for k in keys]
@@ -96,7 +149,7 @@ class PPPWildcards:
return None
return current_dict
def __remove_wildcards_from_file(self, full_path: str, debug=True):
def __remove_wildcards_from_path(self, full_path: str, debug=True):
"""
Clear all wildcards in a file.
@@ -104,11 +157,14 @@ class PPPWildcards:
full_path (str): The path to the file.
debug (bool): Whether to print debug messages or not.
"""
last_modified_cached = self.wildcard_files.get(full_path, None)
if debug and last_modified_cached is not None and self.debug_level != DEBUG_LEVEL.none:
self.logger.debug(f"Removing wildcards from file: {full_path}")
if full_path in self.wildcard_files.keys():
del self.wildcard_files[full_path]
last_modified_cached = self.__wildcard_files.get(full_path, None) # a time or a hash
if debug and last_modified_cached is not None and self.__debug_level != DEBUG_LEVEL.none:
if full_path == self.LOCALINPUT_FILENAME:
self.__logger.debug("Removing from memory wildcards from input")
else:
self.__logger.debug(f"Removing from memory wildcards from file: {full_path}")
if full_path in self.__wildcard_files.keys():
del self.__wildcard_files[full_path]
for key in list(self.wildcards.keys()):
if self.wildcards[key].file == full_path:
del self.wildcards[key]
@@ -121,88 +177,255 @@ class PPPWildcards:
base (str): The base path for the wildcards.
full_path (str): The path to the file.
"""
last_modified = os.path.getmtime(full_path)
last_modified_cached = self.wildcard_files.get(full_path, None)
if last_modified_cached is not None and last_modified == self.wildcard_files[full_path]:
return
filename = os.path.basename(full_path)
name, extension = os.path.splitext(filename)
if extension not in (".txt", ".json", ".yaml", ".yml"):
return
self.__remove_wildcards_from_file(full_path, False)
if last_modified_cached is not None and self.debug_level != DEBUG_LEVEL.none:
self.logger.debug(f"Updating wildcards from file: {full_path}")
relfolders = os.path.relpath(os.path.dirname(full_path), base)
if relfolders == ".":
relfolders = ""
elif relfolders != "":
relfolders += "/"
if extension == ".txt":
self.__get_wildcards_in_text_file(full_path, relfolders, name)
elif extension in (".json", ".yaml", ".yml"):
self.__get_wildcards_in_structured_file(full_path, relfolders, extension)
self.wildcard_files[full_path] = last_modified
try:
last_modified = os.path.getmtime(full_path)
last_modified_cached = self.__wildcard_files.get(full_path, None)
if last_modified_cached is not None and last_modified == self.__wildcard_files[full_path]:
return
filename = os.path.basename(full_path)
_, extension = os.path.splitext(filename)
if extension not in (".txt", ".json", ".yaml", ".yml"):
return
self.__remove_wildcards_from_path(full_path, False)
if last_modified_cached is not None and self.__debug_level != DEBUG_LEVEL.none:
self.__logger.debug(f"Updating wildcards from file: {full_path}")
if extension == ".txt":
self.__get_wildcards_in_text_file(full_path, base)
elif extension in (".json", ".yaml", ".yml"):
self.__get_wildcards_in_structured_file(full_path, base)
self.__wildcard_files[full_path] = last_modified
except Exception as e: # pylint: disable=broad-except
self.__logger.error(f"Error reading wildcard file '{full_path}': {e}")
def __get_wildcards_in_structured_file(self, full_path, relfolders, extension):
with open(full_path, "r", encoding="utf-8") as file:
if extension == ".json":
content = json.loads(file.read())
def __get_wildcards_in_input(self, wildcards_input: str):
"""
Get all wildcards in the string.
Args:
wildcards_input (str): The input string containing wildcards in json or yaml format.
"""
try:
new_h = hash(wildcards_input)
h = self.__wildcard_files.get(self.LOCALINPUT_FILENAME, None)
if h == new_h:
return
self.__remove_wildcards_from_path(self.LOCALINPUT_FILENAME, False)
if h is not None and self.__debug_level != DEBUG_LEVEL.none:
self.__logger.debug("Updating wildcards from input")
wildcards_input = wildcards_input.strip()
if wildcards_input != "":
try:
content = yaml.safe_load(wildcards_input)
except yaml.YAMLError as e:
self.__logger.warning(f"Invalid format for input wildcards: {e}")
return
if content is not None:
self.__add_wildcard(content, self.LOCALINPUT_FILENAME, [self.LOCALINPUT_FILENAME])
self.__wildcard_files[self.LOCALINPUT_FILENAME] = new_h
except Exception as e: # pylint: disable=broad-except
self.__logger.error(f"Error reading wildcards input: {e}")
def is_dict_choices_options(self, d: dict) -> bool:
"""
Check if a dictionary is a valid choices options dictionary.
Args:
d (dict): The dictionary to check.
Returns:
bool: Whether the dictionary is a valid choices options dictionary or not.
"""
return all(
k in ["sampler", "repeating", "optional", "count", "from", "to", "prefix", "suffix", "separator"]
for k in d.keys()
)
def is_dict_choice_options(self, d: dict) -> bool:
"""
Check if a dictionary is a valid choice options dictionary.
Args:
d (dict): The dictionary to check.
Returns:
bool: Whether the dictionary is a valid choice options dictionary or not.
"""
return all(k in ["labels", "weight", "if", "content", "text"] for k in d.keys())
def __get_choices(self, obj: object, full_path: str, key_parts: list[str]) -> list:
"""
We process the choices in the object and return them as a list.
Args:
obj (object): the value of a wildcard
full_path (str): path to the file where the wildcard is defined
key_parts (list[str]): parts of the key for the wildcard
Returns:
list: list of choices
"""
if obj is None:
return None
if isinstance(obj, (str, dict)):
return [obj]
if isinstance(obj, (int, float, bool)):
return [str(obj)]
if not isinstance(obj, list) or len(obj) == 0:
self.__logger.warning(f"Invalid format in wildcard '{'/'.join(key_parts)}' in file '{full_path}'!")
return None
choices = []
for i, c in enumerate(obj):
if isinstance(c, (str, int, float, bool)):
choices.append(str(c))
elif isinstance(c, list):
# we create an anonymous wildcard
choices.append(self.__create_anonymous_wildcard(full_path, key_parts, i, c))
elif isinstance(c, dict):
choices.append(self.__process_dict_choice(c, full_path, key_parts, i))
else:
content = yaml.safe_load(file)
keys = self.__get_keys_in_dict(content)
for key in keys:
fullkey = f"{relfolders}{key}"
if self.wildcards.get(fullkey, None) is not None:
self.logger.warning(
f"Duplicate wildcard '{fullkey}' in file '{full_path}' and '{self.wildcards[fullkey].file}'!"
self.__logger.warning(
f"Invalid choice {i+1} in wildcard '{'/'.join(key_parts)}' in file '{full_path}'!"
)
else:
obj = self.__get_nested(content, key)
if obj is not None:
if isinstance(obj, str):
choices = [obj]
elif isinstance(obj, (int, float, bool)):
choices = [str(obj)]
elif isinstance(obj, list) and len(obj) > 0:
choices = []
for c in obj:
if isinstance(c, str):
choices.append(c)
elif isinstance(c, dict): # we convert the dict to a string
d = ""
if "weight" in c.keys():
d += str(c["weight"])
if "if" in c.keys():
d += f" if {c['if']}"
if d != "":
d += "::"
if "text" in c.keys():
d += c["text"]
elif "content" in c.keys():
d += c["content"]
choices.append(d)
else:
obj = None
if obj is None:
self.logger.warning(f"Invalid wildcard '{fullkey}' in file '{full_path}'!")
else:
self.wildcards[fullkey] = PPPWildcard(full_path, fullkey, choices)
return choices
def __get_wildcards_in_text_file(self, full_path, relfolders, name):
with open(full_path, "r", encoding="utf-8") as file:
text_content = map(lambda x: x.strip("\n\r"), file.readlines())
text_content = list(filter(lambda x: x.strip() != "" and not x.strip().startswith("#"), text_content))
text_content = [x.split("#")[0].rstrip() if len(x.split("#")) > 1 else x for x in text_content]
fullkey = f"{relfolders}{name}"
def __process_dict_choice(self, c: dict, full_path: str, key_parts: list[str], i: int) -> dict:
"""
Process a dictionary choice.
Args:
c (dict): The dictionary choice.
full_path (str): The path to the file.
key_parts (list[str]): The parts of the key.
i (int): The index of the choice.
Returns:
dict: The processed choice.
"""
if self.is_dict_choices_options(c) or self.is_dict_choice_options(c):
# we assume it is a choice or wildcard parameters in object format
choice = c
choice_content = choice.get("content", choice.get("text", None))
if choice_content is not None and isinstance(choice_content, list):
# we create an anonymous wildcard
choice["content"] = self.__create_anonymous_wildcard(full_path, key_parts, i, choice_content)
if "text" in choice:
del choice["text"]
return choice
if len(c) == 1:
# we assume it is an anonymous wildcard with options
firstkey = list(c.keys())[0]
return self.__create_anonymous_wildcard(full_path, key_parts, i, c[firstkey], firstkey)
self.__logger.warning(f"Invalid choice {i+1} in wildcard '{'/'.join(key_parts)}' in file '{full_path}'!")
return None
def __create_anonymous_wildcard(self, full_path, key_parts, i, content, options=None):
"""
Create an anonymous wildcard.
Args:
full_path (str): The path to the file that contains it.
key_parts (list[str]): The parts of the key.
i (int): The index of the wildcard.
content (object): The content of the wildcard.
options (str): The options for the choice where the wildcard is defined.
Returns:
str: The resulting value for the choice.
"""
new_parts = key_parts + [f"#ANON_{i}"]
self.__add_wildcard(content, full_path, new_parts)
value = f"__{'/'.join(new_parts)}__"
if options is not None:
value = f"{options}::{value}"
return value
def __add_wildcard(self, content: object, full_path: str, external_key_parts: list[str]):
"""
Add a wildcard to the wildcards dictionary.
Args:
content (object): The content of the wildcard.
full_path (str): The path to the file that contains it.
external_key_parts (list[str]): The parts of the key.
"""
key_parts = external_key_parts.copy()
if isinstance(content, dict):
key_parts.pop()
keys = self.__get_keys_in_dict(content)
for key in keys:
tmp_key_parts = key_parts.copy()
tmp_key_parts.extend(key.split("/"))
fullkey = "/".join(tmp_key_parts)
if self.wildcards.get(fullkey, None) is not None:
self.__logger.warning(
f"Duplicate wildcard '{fullkey}' in file '{full_path}' and '{self.wildcards[fullkey].file}'!"
)
else:
obj = self.__get_nested(content, key)
choices = self.__get_choices(obj, full_path, tmp_key_parts)
if choices is None:
self.__logger.warning(f"Invalid wildcard '{fullkey}' in file '{full_path}'!")
else:
self.wildcards[fullkey] = PPPWildcard(full_path, fullkey, choices)
return
if isinstance(content, str):
content = [content]
elif isinstance(content, (int, float, bool)):
content = [str(content)]
if not isinstance(content, list):
self.__logger.warning(f"Invalid wildcard in file '{full_path}'!")
return
fullkey = "/".join(key_parts)
if self.wildcards.get(fullkey, None) is not None:
self.logger.warning(
self.__logger.warning(
f"Duplicate wildcard '{fullkey}' in file '{full_path}' and '{self.wildcards[fullkey].file}'!"
)
else:
if len(text_content) == 0:
self.logger.warning(f"Invalid wildcard in file '{full_path}'!")
choices = self.__get_choices(content, full_path, key_parts)
if choices is None:
self.__logger.warning(f"Invalid wildcard '{fullkey}' in file '{full_path}'!")
else:
self.wildcards[fullkey] = PPPWildcard(full_path, fullkey, text_content)
self.wildcards[fullkey] = PPPWildcard(full_path, fullkey, choices)
def __get_wildcards_in_structured_file(self, full_path, base):
"""
Get all wildcards in a structured file.
Args:
full_path (str): The path to the file.
base (str): The base path for the wildcards.
"""
external_key: str = os.path.relpath(os.path.splitext(full_path)[0], base)
external_key_parts = external_key.split(os.sep)
try:
with open(full_path, "r", encoding="utf-8") as file:
content = yaml.safe_load(file)
except: # pylint: disable=bare-except
self.__logger.warning(f"Could not read file '{full_path}' with utf-8 encoding, trying windows-1252...")
with open(full_path, "r", encoding="windows-1252") as file:
content = yaml.safe_load(file)
self.__add_wildcard(content, full_path, external_key_parts)
def __get_wildcards_in_text_file(self, full_path, base):
"""
Get all wildcards in a text file.
Args:
full_path (str): The path to the file.
base (str): The base path for the wildcards.
"""
external_key: str = os.path.relpath(os.path.splitext(full_path)[0], base)
external_key_parts = external_key.split(os.sep)
try:
with open(full_path, "r", encoding="utf-8") as file:
text_content = map(lambda x: x.strip("\n\r"), file.readlines())
except: # pylint: disable=bare-except
self.__logger.warning(f"Could not read file '{full_path}' with utf-8 encoding, trying windows-1252...")
with open(full_path, "r", encoding="windows-1252") as file:
text_content = map(lambda x: x.strip("\n\r"), file.readlines())
text_content = list(filter(lambda x: x.strip() != "" and not x.strip().startswith("#"), text_content))
text_content = [x.split("#")[0].rstrip() if len(x.split("#")) > 1 else x for x in text_content]
self.__add_wildcard(text_content, full_path, external_key_parts)
def __get_wildcards_in_directory(self, base: str, directory: str):
"""
@@ -213,7 +436,7 @@ class PPPWildcards:
directory (str): The path to the directory.
"""
if not os.path.exists(directory):
self.logger.warning(f"Wildcard directory '{directory}' does not exist!")
self.__logger.warning(f"Wildcard directory '{directory}' does not exist!")
return
for filename in os.listdir(directory):
full_path = os.path.abspath(os.path.join(directory, filename))
+15
View File
@@ -0,0 +1,15 @@
[project]
name = "sd-webui-prompt-postprocessor"
description = "Stable Diffusion WebUI & ComfyUI extension to post-process the prompt, including sending content from the prompt to the negative prompt and wildcards."
version = "2.12.1"
license = { file = "LICENSE.txt" }
dependencies = ["lark", "numpy", "pyyaml"]
[project.urls]
Repository = "https://github.com/acorderob/sd-webui-prompt-postprocessor"
# Used by Comfy Registry https://comfyregistry.org
[tool.comfy]
PublisherId = "acorderob"
DisplayName = "sd-webui-prompt-postprocessor"
Icon = "ppp_icon.png"
+2
View File
@@ -1 +1,3 @@
lark
numpy
pyyaml
+344 -67
View File
@@ -4,19 +4,22 @@ if __name__ == "__main__":
import sys
import os
import time
from pathlib import Path
import numpy as np
sys.path.append(os.path.join(sys.path[0], ".."))
sys.path.append(str(Path(__file__).parent)) # base path for the extension
from modules import scripts, shared, script_callbacks
from modules.processing import StableDiffusionProcessing
from modules.shared import opts
from modules.paths import models_path
import gradio as gr
from ppp import PromptPostProcessor
from ppp_logging import DEBUG_LEVEL, PromptPostProcessorLogFactory
from ppp_cache import PPPLRUCache
from ppp_wildcards import PPPWildcards
from modules import scripts, shared, script_callbacks # pylint: disable=import-error
from modules.processing import StableDiffusionProcessing # pylint: disable=import-error
from modules.shared import opts # pylint: disable=import-error
from modules.paths import models_path # pylint: disable=import-error
import gradio as gr # pylint: disable=import-error
from ppp import PromptPostProcessor # pylint: disable=import-error
from ppp_hosts import SUPPORTED_APPS, SUPPORTED_APPS_NAMES # pylint: disable=import-error
from ppp_logging import DEBUG_LEVEL, PromptPostProcessorLogFactory # pylint: disable=import-error
from ppp_cache import PPPLRUCache # pylint: disable=import-error
from ppp_wildcards import PPPWildcards # pylint: disable=import-error
from ppp_enmappings import PPPExtraNetworkMappings # pylint: disable=import-error
class PromptPostProcessorA1111Script(scripts.Script):
@@ -36,27 +39,37 @@ class PromptPostProcessorA1111Script(scripts.Script):
__on_ui_settings(): Callback function for UI settings.
"""
instance_count = 0
@classmethod
def increment_instance_count(cls):
cls.instance_count += 1
return cls.instance_count
@classmethod
def get_instance_count(cls):
return cls.instance_count
def __init__(self):
"""
Initializes the PromptPostProcessor object.
This method adds callbacks for UI settings and initializes the logger.
Parameters:
None
Returns:
None
"""
lf = PromptPostProcessorLogFactory()
self.instance_index = self.increment_instance_count()
self.name = PromptPostProcessor.NAME
self.ppp_logger = lf.log
self.ppp_debug_level = DEBUG_LEVEL(getattr(opts, "ppp_gen_debug_level", DEBUG_LEVEL.none.value))
self.lru_cache = PPPLRUCache(1000)
grammar_filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), "../grammar.lark")
with open(grammar_filename, "r", encoding="utf-8") as file:
self.grammar_content = file.read()
self.wildcards_obj = PPPWildcards(lf.log)
self.ppp_logger = None
self.ppp_debug_level = DEBUG_LEVEL.none.value
self.lru_cache = None
self.wildcards_obj = None
self.extranetwork_mappings_obj = None
def title(self):
"""
@@ -67,7 +80,7 @@ class PromptPostProcessorA1111Script(scripts.Script):
"""
return PromptPostProcessor.NAME
def show(self, is_img2img):
def show(self, is_img2img): # pylint: disable=unused-argument
"""
Determines whether the script should be shown based on the kind of processing.
@@ -79,35 +92,137 @@ class PromptPostProcessorA1111Script(scripts.Script):
"""
return scripts.AlwaysVisible
def process(self, p: StableDiffusionProcessing, *args, **kwargs): # pylint: disable=unused-argument
def ui(self, is_img2img): # pylint: disable=unused-argument
with gr.Accordion(PromptPostProcessor.NAME, open=False):
force_equal_seeds = gr.Checkbox(
label="Force equal seeds",
info="Force all image seeds and variation seeds to be equal to the first one, disabling the default autoincrease.",
value=False,
# show_label=True,
elem_id="ppp_force_equal_seeds",
)
gr.HTML("<br>")
gr.Markdown(
"""
Unlink the seed to use the specified one for the prompts instead of the image seed.
* A seed of -1 and "Incremental seed" checked will use a random seed for the first prompt and consecutive values for the rest. This is the same as when you use -1 for the image seed.
* A seed of -1 and "Incremental seed" unchecked will use a random seed for each prompt.
* Any other seed value and "Incremental seed" checked will use the specified seed for the first prompt and consecutive values for the rest.
* Any other seed value and "Incremental seed" unchecked will use the specified seed for all the prompts.
Seeds are only used for the wildcards and choice constructs.
"""
)
gr.HTML("<br>")
with gr.Row(equal_height=True):
unlink_seed = gr.Checkbox(
label="Unlink seed",
value=False,
# show_label=True,
elem_id="ppp_unlink_seed",
)
seed = gr.Number(
label="Prompt seed",
value=-1,
precision=0,
# minimum=-1,
# maximum=2**32 - 1,
# step=1,
# show_label=True,
min_width=100,
elem_id="ppp_seed",
)
incremental_seed = gr.Checkbox(
label="Incremental seed (only applies to batches)",
value=False,
# show_label=True,
elem_id="ppp_incremental_seed",
)
return [force_equal_seeds, unlink_seed, seed, incremental_seed]
def process(
self,
p: StableDiffusionProcessing,
input_force_equal_seeds,
input_unlink_seed,
input_seed,
input_incremental_seed,
): # pylint: disable=arguments-differ
"""
Processes the prompts and applies post-processing operations.
Args:
p (StableDiffusionProcessing): The StableDiffusionProcessing object containing the prompts.
input_force_equal_seeds (bool): Flag indicating whether to force equal seeds.
input_unlink_seed (bool): Flag indicating whether to unlink the seed.
input_seed (int): The seed value.
input_incremental_seed (bool): Flag indicating whether to use incremental seed.
Returns:
None
"""
t1 = time.time()
app = (
SUPPORTED_APPS.forge
if hasattr(p.sd_model, "model_config")
else (
SUPPORTED_APPS.reforge
if hasattr(p.sd_model, "forge_objects")
else (
SUPPORTED_APPS.sdnext
if hasattr(p.sd_model, "is_sdxl") and not hasattr(p.sd_model, "is_ssd")
else SUPPORTED_APPS.a1111
)
)
)
if self.ppp_logger is None:
lf = PromptPostProcessorLogFactory(app)
self.ppp_logger = lf.log
self.ppp_debug_level = DEBUG_LEVEL(getattr(opts, "ppp_gen_debug_level", DEBUG_LEVEL.none.value))
self.lru_cache = PPPLRUCache(1000, logger=self.ppp_logger, debug_level=self.ppp_debug_level)
self.wildcards_obj = PPPWildcards(self.ppp_logger)
self.extranetwork_mappings_obj = PPPExtraNetworkMappings(self.ppp_logger)
self.ppp_logger.info(
f"{PromptPostProcessor.NAME} {PromptPostProcessor.VERSION} initialized, running on {SUPPORTED_APPS_NAMES[app]}"
)
t1 = time.monotonic_ns()
if getattr(opts, "prompt_attention", "") == "Compel parser":
self.ppp_logger.warning("Compel parser is not supported!")
is_i2i = getattr(p, "init_images", [None])[0] is not None
init_images = getattr(p, "init_images", [None]) or [None]
is_i2i = bool(init_images[0])
self.ppp_debug_level = DEBUG_LEVEL(getattr(opts, "ppp_gen_debug_level", DEBUG_LEVEL.none.value))
do_i2i = getattr(opts, "ppp_gen_doi2i", False)
add_prompts = getattr(opts, "ppp_gen_addpromptstometadata", True)
if is_i2i and not do_i2i:
if self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info("Not processing the prompt for i2i")
return
p.extra_generation_params.update(
{
"PPP force equal seeds": input_force_equal_seeds,
"PPP unlink seed": input_unlink_seed,
"PPP prompt seed": input_seed,
"PPP incremental seed": input_incremental_seed,
}
)
if self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info(f"Post-processing prompts ({'i2i' if is_i2i else 't2i'})")
app = (
"forge"
if hasattr(p.sd_model, "model_config")
else "sdnext" if hasattr(p.sd_model, "is_sdxl") and not hasattr(p.sd_model, "is_ssd") else "a1111"
)
models_supported = {x: True for x in PromptPostProcessor.SUPPORTED_MODELS}
if app == SUPPORTED_APPS.sdnext:
models_supported["ssd"] = False
elif app == SUPPORTED_APPS.forge:
models_supported["ssd"] = False
models_supported["auraflow"] = False
elif app == SUPPORTED_APPS.reforge:
models_supported["flux"] = False
models_supported["auraflow"] = False
else: # assume A1111 compatible
models_supported["flux"] = False
models_supported["auraflow"] = False
env_info = {
"app": app,
"app": app.value,
"models_path": models_path,
"model_filename": getattr(p.sd_model.sd_checkpoint_info, "filename", ""),
"model_class": "",
@@ -118,8 +233,18 @@ class PromptPostProcessorA1111Script(scripts.Script):
"is_sd3": False, # Stable Diffusion 3
"is_flux": False, # Flux
"is_auraflow": False, # AuraFlow
"is_pixart": False, # PixArt
"is_lumina2": False, # Lumina2
"is_ltxv": False, # LTXV
"is_cosmos": False, # Cosmos
"is_genmomochi": False, # GenmoMochi
"is_hunyuan": False, # Hunyuan
"is_hunyuanvideo": False, # HunyuanVideo
"is_hunyuan3d": False, # Hunyuan3D
"is_wanvideo": False, # WanVideo
"is_hidream": False, # HiDream
}
if app == "sdnext":
if app == SUPPORTED_APPS.sdnext:
# cannot differentiate SD1 and SD2, we set True to both
# LatentDiffusion is for the original backend, StableDiffusionPipeline is for the diffusers backend
env_info["model_class"] = p.sd_model.__class__.__name__
@@ -130,15 +255,28 @@ class PromptPostProcessorA1111Script(scripts.Script):
env_info["is_sd3"] = p.sd_model.__class__.__name__ == "StableDiffusion3Pipeline"
env_info["is_flux"] = p.sd_model.__class__.__name__ == "FluxPipeline"
env_info["is_auraflow"] = p.sd_model.__class__.__name__ == "AuraFlowPipeline"
elif app == "forge":
# also supports 'Latent Consistency Model': LatentConsistencyModelPipeline', 'PixArt-Alpha': 'PixArtAlphaPipeline', 'UniDiffuser': 'UniDiffuserPipeline', 'Wuerstchen': 'WuerstchenCombinedPipeline', 'Kandinsky 2.1': 'KandinskyPipeline', 'Kandinsky 2.2': 'KandinskyV22Pipeline', 'Kandinsky 3': 'Kandinsky3Pipeline', 'DeepFloyd IF': 'IFPipeline', 'Custom Diffusers Pipeline': 'DiffusionPipeline', 'InstaFlow': 'StableDiffusionPipeline', 'SegMoE': 'StableDiffusionPipeline', 'Kolors': 'KolorsPipeline', 'AuraFlow': 'AuraFlowPipeline', 'CogView': 'CogView3PlusPipeline'
elif app == SUPPORTED_APPS.forge:
# from repositories\huggingface_guess\huggingface_guess\model_list.py
env_info["model_class"] = p.sd_model.model_config.__class__.__name__
env_info["is_sd1"] = getattr(p.sd_model, "is_sd1", False)
env_info["is_sd2"] = getattr(p.sd_model, "is_sd2", False)
env_info["is_sdxl"] = getattr(p.sd_model, "is_sdxl", False)
env_info["is_ssd"] = False # ?
env_info["is_sd3"] = getattr(
p.sd_model, "is_sd3", False
) # p.sd_model.model_config.__class__.__name__ == "SD3" # not actually supported?
env_info["is_flux"] = p.sd_model.model_config.__class__.__name__ in ("Flux", "FluxSchnell")
env_info["is_auraflow"] = False # p.sd_model.model_config.__class__.__name__ == "AuraFlow" # not supported
elif app == SUPPORTED_APPS.reforge:
env_info["model_class"] = p.sd_model.__class__.__name__
env_info["is_sd1"] = getattr(p.sd_model, "is_sd1", False)
env_info["is_sd2"] = getattr(p.sd_model, "is_sd2", False)
env_info["is_sdxl"] = getattr(p.sd_model, "is_sdxl", False)
env_info["is_ssd"] = getattr(p.sd_model, "is_ssd", False)
env_info["is_sd3"] = getattr(p.sd_model, "is_sd3", False)
env_info["is_flux"] = p.sd_model.model_config.__class__.__name__ == "Flux"
env_info["is_auraflow"] = False # p.sd_model.model_config.__class__.__name__ == "AuraFlow"
env_info["is_flux"] = False
env_info["is_auraflow"] = False
else: # assume A1111 compatible (p.sd_model.__class__.__name__=="DiffusionEngine")
env_info["model_class"] = p.sd_model.__class__.__name__
env_info["is_sd1"] = getattr(p.sd_model, "is_sd1", False)
@@ -148,6 +286,7 @@ class PromptPostProcessorA1111Script(scripts.Script):
env_info["is_sd3"] = getattr(p.sd_model, "is_sd3", False)
env_info["is_flux"] = False
env_info["is_auraflow"] = False
hash_envinfo = hash(tuple(sorted(env_info.items())))
wc_wildcards_folders = getattr(opts, "ppp_wil_wildcardsfolders", "")
if wc_wildcards_folders == "":
wc_wildcards_folders = os.getenv("WILDCARD_DIR", PPPWildcards.DEFAULT_WILDCARDS_FOLDER)
@@ -156,11 +295,25 @@ class PromptPostProcessorA1111Script(scripts.Script):
for f in wc_wildcards_folders.split(",")
if f.strip() != ""
]
en_mappings_folders = getattr(opts, "ppp_en_mappingsfolders", "")
if en_mappings_folders == "":
en_mappings_folders = os.getenv(
"EXTRANETWORKMAPPINGS_DIR",
PPPExtraNetworkMappings.DEFAULT_ENMAPPINGS_FOLDER,
)
enmappings_folders = [
(f if os.path.isabs(f) else os.path.abspath(os.path.join(models_path, f)))
for f in en_mappings_folders.split(",")
if f.strip() != ""
]
options = {
"debug_level": getattr(opts, "ppp_gen_debug_level", DEBUG_LEVEL.none.value),
"pony_substrings": getattr(opts, "ppp_gen_ponysubstrings", PromptPostProcessor.DEFAULT_PONY_SUBSTRINGS),
"on_warning": getattr(opts, "ppp_gen_onwarning", PromptPostProcessor.ONWARNING_CHOICES.warn.value),
"variants_definitions": getattr(
opts, "ppp_gen_variantsdefinitions", PromptPostProcessor.DEFAULT_VARIANTS_DEFINITIONS
),
"process_wildcards": getattr(opts, "ppp_wil_processwildcards", True),
"if_wildcards": getattr(opts, "ppp_wil_ifwildcards", PromptPostProcessor.IFWILDCARDS_CHOICES.ignore.value),
"if_wildcards": getattr(opts, "ppp_wil_ifwildcards", PromptPostProcessor.IFWILDCARDS_CHOICES.stop.value),
"choice_separator": getattr(opts, "ppp_wil_choice_separator", PromptPostProcessor.DEFAULT_CHOICE_SEPARATOR),
"keep_choices_order": getattr(opts, "ppp_wil_keep_choices_order", False),
"stn_separator": getattr(opts, "ppp_stn_separator", PromptPostProcessor.DEFAULT_STN_SEPARATOR),
@@ -169,6 +322,7 @@ class PromptPostProcessorA1111Script(scripts.Script):
"cleanup_empty_constructs": getattr(opts, "ppp_cup_emptyconstructs", True),
"cleanup_extra_separators": getattr(opts, "ppp_cup_extraseparators", True),
"cleanup_extra_separators2": getattr(opts, "ppp_cup_extraseparators2", True),
"cleanup_extra_separators_include_eol": getattr(opts, "ppp_cup_extraseparators_include_eol", True),
"cleanup_breaks": getattr(opts, "ppp_cup_breaks", True),
"cleanup_breaks_eol": getattr(opts, "ppp_cup_breaks_eol", False),
"cleanup_ands": getattr(opts, "ppp_cup_ands", True),
@@ -177,31 +331,62 @@ class PromptPostProcessorA1111Script(scripts.Script):
"cleanup_merge_attention": getattr(opts, "ppp_cup_mergeattention", True),
"remove_extranetwork_tags": getattr(opts, "ppp_rem_removeextranetworktags", False),
}
hash_options = hash(tuple(sorted(options.items())))
self.wildcards_obj.refresh_wildcards(
self.ppp_debug_level, wildcards_folders if options["process_wildcards"] else None
)
self.extranetwork_mappings_obj.refresh_extranetwork_mappings(self.ppp_debug_level, enmappings_folders)
ppp = PromptPostProcessor(
self.ppp_logger, self.ppp_interrupt, env_info, options, self.grammar_content, self.wildcards_obj
self.ppp_logger,
self.ppp_interrupt,
env_info,
options,
self.grammar_content,
self.wildcards_obj,
self.extranetwork_mappings_obj,
)
prompts_list = []
seeds = getattr(p, "all_seeds", [])
subseeds = getattr(p, "all_subseeds", [])
subseed_strength = getattr(p, "subseed_strength", 0.0)
if subseed_strength > 0:
calculated_seeds = [
int(subseed * subseed_strength + seed * (1 - subseed_strength))
for seed, subseed in zip(seeds, subseeds)
]
if input_force_equal_seeds:
if self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info("Forcing equal seeds")
seeds = getattr(p, "all_seeds", [])
subseeds = getattr(p, "all_subseeds", [])
p.all_seeds = [seeds[0] for _ in seeds]
p.all_subseeds = [subseeds[0] for _ in subseeds]
if input_unlink_seed:
if self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info("Using unlinked seed")
num_seeds = len(getattr(p, "all_seeds", []))
if input_incremental_seed:
first_seed = np.random.randint(0, 2**32, dtype=np.int64) if input_seed == -1 else input_seed
calculated_seeds = [first_seed + i for i in range(num_seeds)]
elif input_seed == -1:
calculated_seeds = np.random.randint(0, 2**32, size=num_seeds, dtype=np.int64)
else:
calculated_seeds = [input_seed for _ in range(num_seeds)]
else:
calculated_seeds = seeds
if len(set(calculated_seeds)) < len(calculated_seeds):
self.ppp_logger.info("Adjusting seeds because some are equal.")
calculated_seeds = [seed + i for i, seed in enumerate(calculated_seeds)]
seeds = getattr(p, "all_seeds", [])
subseeds = getattr(p, "all_subseeds", [])
subseed_strength = getattr(p, "subseed_strength", 0.0)
if subseed_strength > 0:
calculated_seeds = [
int(subseed * subseed_strength + seed * (1 - subseed_strength))
for seed, subseed in zip(seeds, subseeds)
]
# if len(set(calculated_seeds)) < len(calculated_seeds):
# self.ppp_logger.info("Adjusting seeds because some are equal.")
# calculated_seeds = [seed + i for i, seed in enumerate(calculated_seeds)]
else:
calculated_seeds = seeds
# initialize extra generation parameters
extra_params = {}
# adds regular prompts
rpr = getattr(p, "all_prompts", None)
rnr = getattr(p, "all_negative_prompts", None)
rpr: list[str] = getattr(p, "all_prompts", None)
rnr: list[str] = getattr(p, "all_negative_prompts", None)
if rpr is not None and rnr is not None:
prompts_list += [
("regular", seed, prompt, negative_prompt)
@@ -209,9 +394,9 @@ class PromptPostProcessorA1111Script(scripts.Script):
if (seed, prompt, negative_prompt) not in prompts_list
]
# make it compatible with A1111 hires fix
rph = getattr(p, "all_hr_prompts", None)
rnh = getattr(p, "all_hr_negative_prompts", None)
if rph is not None and rnh is not None:
rph: list[str] = getattr(p, "all_hr_prompts", None)
rnh: list[str] = getattr(p, "all_hr_negative_prompts", None)
if rph is not None and rnh is not None and (rph != rpr or rnh != rnr):
prompts_list += [
("hiresfix", seed, prompt, negative_prompt)
for seed, prompt, negative_prompt in zip(calculated_seeds, rph, rnh)
@@ -222,31 +407,77 @@ class PromptPostProcessorA1111Script(scripts.Script):
for i, (prompttype, seed, prompt, negative_prompt) in enumerate(prompts_list):
if self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info(f"processing prompts[{i+1}] ({prompttype})")
if self.lru_cache.get((seed, prompt, negative_prompt)) is None:
pp, np = ppp.process_prompt(prompt, negative_prompt, seed)
self.lru_cache.put((seed, prompt, negative_prompt), (pp, np))
if (
self.lru_cache.get(
(hash_envinfo, hash_options, seed, hash(self.wildcards_obj), prompt, negative_prompt)
)
is None
):
posp, negp, _ = ppp.process_prompt(prompt, negative_prompt, seed)
self.lru_cache.put(
(hash_envinfo, hash_options, seed, hash(self.wildcards_obj), prompt, negative_prompt), (posp, negp)
)
# adds also the result so i2i doesn't process it unnecessarily
self.lru_cache.put((seed, pp, np), (pp, np))
self.lru_cache.put(
(hash_envinfo, hash_options, seed, hash(self.wildcards_obj), posp, negp), (posp, negp)
)
elif self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info("result already in cache")
# updates the prompts
rpr_copy = None
rnr_copy = None
if rpr is not None and rnr is not None:
rpr_changes = False
rnr_changes = False
rpr_copy = rpr.copy()
rnr_copy = rnr.copy()
for i, (seed, prompt, negative_prompt) in enumerate(zip(calculated_seeds, rpr, rnr)):
found = self.lru_cache.get((seed, prompt, negative_prompt))
found = self.lru_cache.get(
(hash_envinfo, hash_options, seed, hash(self.wildcards_obj), prompt, negative_prompt)
)
if found is not None:
if rpr[i].strip() != found[0].strip():
rpr_changes = True
if rnr[i].strip() != found[1].strip():
rnr_changes = True
rpr[i] = found[0]
rnr[i] = found[1]
if add_prompts:
if rpr_changes:
extra_params["PPP original prompts"] = rpr_copy
if rnr_changes:
extra_params["PPP original negative prompts"] = rnr_copy
if rph is not None and rnh is not None:
rph_changes = False
rnh_changes = False
rph_copy = rph.copy()
rnh_copy = rnh.copy()
for i, (seed, prompt, negative_prompt) in enumerate(zip(calculated_seeds, rph, rnh)):
found = self.lru_cache.get((seed, prompt, negative_prompt))
found = self.lru_cache.get(
(hash_envinfo, hash_options, seed, hash(self.wildcards_obj), prompt, negative_prompt)
)
if found is not None:
if rph[i].strip() != found[0].strip() and (not rpr_copy or rph[i].strip() != rpr_copy[i].strip()):
rph_changes = True
if rnh[i].strip() != found[1].strip() and (not rnr_copy or rnh[i].strip() != rnr_copy[i].strip()):
rnh_changes = True
rph[i] = found[0]
rnh[i] = found[1]
if add_prompts:
if rph_changes:
extra_params["PPP original HR prompts"] = rph_copy
if rnh_changes:
extra_params["PPP original HR negative prompts"] = rnh_copy
t2 = time.time()
# fill extra generation parameters only if not already present
for k, v in extra_params.items():
if p.extra_generation_params.get(k) is None:
p.extra_generation_params[k] = v
t2 = time.monotonic_ns()
if self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info(f"process time: {t2 - t1:.3f} seconds")
self.ppp_logger.info(f"process time: {(t2 - t1) / 1_000_000_000:.3f} seconds")
def ppp_interrupt(self):
"""
@@ -316,10 +547,30 @@ def on_ui_settings():
),
)
shared.opts.add_option(
key="ppp_gen_ponysubstrings",
key="ppp_gen_onwarning",
info=shared.OptionInfo(
PromptPostProcessor.DEFAULT_PONY_SUBSTRINGS,
label="Comma separated list of substrings to look for in the model full filename to flag it as Pony (case insensitive)",
default=PromptPostProcessor.ONWARNING_CHOICES.warn.value,
label="What to do on invalid content warnings?",
component=gr.Radio,
component_args={
"choices": (
("Show warning in console", PromptPostProcessor.ONWARNING_CHOICES.warn.value),
("Stop the generation", PromptPostProcessor.ONWARNING_CHOICES.stop.value),
)
},
section=section,
),
)
shared.opts.add_option(
key="ppp_gen_variantsdefinitions",
info=shared.OptionInfo(
PromptPostProcessor.DEFAULT_VARIANTS_DEFINITIONS,
label="Definitions for variant models",
comment_after="Recognized based on strings found in the full filename. Format for each line is: 'name(kind)=comma separated list of substrings (case insensitive)' with kind being one of the base model types ("
+ ",".join(PromptPostProcessor.SUPPORTED_MODELS)
+ ") or not specified.",
component=gr.Textbox,
component_args={"lines": 7},
section=section,
),
)
@@ -332,11 +583,29 @@ def on_ui_settings():
section=section,
),
)
shared.opts.add_option(
key="ppp_gen_addpromptstometadata",
info=shared.OptionInfo(
True,
label="Add original prompts to metadata (if they change)",
section=section,
),
)
shared.opts.add_option(
key="ppp_en_mappingsfolders",
info=shared.OptionInfo(
PPPExtraNetworkMappings.DEFAULT_ENMAPPINGS_FOLDER,
label="Extranetwork Mappings folders",
comment_after='<span class="info">(absolute or relative to the models folder)</span>',
section=section,
),
)
# wildcard settings
shared.opts.add_option(
key="ppp_wil_sep",
info=new_html_title('<br><h2>Wildcard settings</h2>'),
info=new_html_title("<br><h2>Wildcard settings</h2>"),
)
shared.opts.add_option(
key="ppp_wil_processwildcards",
@@ -395,7 +664,7 @@ def on_ui_settings():
# content removal settings
shared.opts.add_option(
key="ppp_rem_sep",
info=new_html_title('<br><h2>Content removal settings</h2>'),
info=new_html_title("<br><h2>Content removal settings</h2>"),
)
shared.opts.add_option(
key="ppp_rem_removeextranetworktags",
@@ -409,7 +678,7 @@ def on_ui_settings():
# send to negative settings
shared.opts.add_option(
key="ppp_stn_sep",
info=new_html_title('<br><h2>Send to Negative settings</h2>'),
info=new_html_title("<br><h2>Send to Negative settings</h2>"),
)
shared.opts.add_option(
key="ppp_stn_separator",
@@ -430,7 +699,7 @@ def on_ui_settings():
# clean-up settings
shared.opts.add_option(
key="ppp_cup_sep",
info=new_html_title('<br><h2>Clean-up settings</h2>'),
info=new_html_title("<br><h2>Clean-up settings</h2>"),
)
shared.opts.add_option(
key="ppp_cup_emptyconstructs",
@@ -456,6 +725,14 @@ def on_ui_settings():
section=section,
),
)
shared.opts.add_option(
key="ppp_cup_extraseparators_include_eol",
info=shared.OptionInfo(
False,
label="The extra separators options also remove EOLs",
section=section,
),
)
shared.opts.add_option(
key="ppp_cup_breaks",
info=shared.OptionInfo(
View File
+11
View File
@@ -0,0 +1,11 @@
lora:
lora1:
- condition: _is_pony
name: lorapony
parameters: 0.8
triggers: ["triggerpony1", "triggerpony2"]
- condition: _is_illustrious
name: loraillustrious
parameters: "0.9:0.8"
triggers: ["triggerillustrious1", "triggerillustrious2"]
- triggers: ["triggergeneric1", "triggergeneric2", "{one|two}"]
+684 -234
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -6,7 +6,7 @@
"choice3"
],
"wildcard2": [
"r2-3$$-",
"r2-3$$-$$",
"4::choice1",
"3:: choice2 ",
"2::choice3",
+46 -4
View File
@@ -5,10 +5,10 @@ yaml:
- choice3
wildcard2:
- ~r2-3$$-
- ~r2-3$$-$$
- "'label1,label2'4::choice1"
- "3:: choice2 "
- "'label1,label3'2::choice3"
- { labels: ["label1", "label3"], weight: 2, content: choice3 }
- 5 if _is_sd1::choice4
wildcard2bis:
@@ -28,9 +28,51 @@ yaml:
- { weight: 2, text: choice1 }
- { weight: 3, content: choice2 }
- { text: choice3 }
- { weight: 4, if: "_is_ssd", text: choice4 }
- { weight: 4, if: _is_ssd, text: choice4 }
wildcard7:
- <lora:test1:1>
- <lora:test2:1>
- <lora:test3:1>
- <lora:test3:1>
wildcardPS:
- {
sampler: "~",
repeating: false,
optional: false,
count: 2,
prefix: "prefix-",
suffix: "-suffix",
separator: "/",
}
- { weight: 3, text: choice1 }
- { weight: 2, text: choice2 }
- { weight: 1, text: choice3 }
more_nested:
even_more_nested: # this would be __yaml/more_nested/even_more_nested__
- one
- two
anonwildcards:
- one
- two
- # choice without options and anonymous wildcard
- three
- four
- 3 if _is_sdxl: # choice with options and anonymous wildcard
- five
- six
- { weight: 1, text: [seven, eight] } # anonymous wildcard used in a choice in object format
- # choice without options and anonymous wildcard with parameters
- { count: 2, prefix: "#" }
- nine
- ten
empty_wildcard:
- o$$ # parameters: optional
- if false::1
- if false::2
- if false::3
- if _sd in ("test1", "test2")::4
- if (false or false)::5
+3
View File
@@ -0,0 +1,3 @@
- one
- 2
- three
+1
View File
@@ -0,0 +1 @@
one choice
+1 -1
View File
@@ -1,5 +1,5 @@
# wildcard2
r2-3$$-
r2-3$$-$$
4::choice1
3:: choice2
2::choice3
+2 -1
View File
@@ -1,2 +1,3 @@
# wildcard3
__2$$,$$text/wildcard2__
__2$$,$$text/wildcard2__
0::áüç
File diff suppressed because one or more lines are too long