Compare commits

...
8 Commits
Author SHA1 Message Date
Antonio Cordero Balcazar db43d409f8 * removed unnecessary processing of variables.
* error control when reading wildcard or mapping files, and second chance for files with incorrect encoding.
* fix processing inside extranetwork tag.
2025-09-27 13:22:19 +02:00
Antonio Cordero Balcazar 8769c21ee5 * new "optional" parameter for choices/wildcard.
* added "$$" to end of parameters as first choice in a wildcard, to avoid confusion with a choice that could be considered parameters.
* additional fixes in the parser.
2025-09-06 18:14:00 +02:00
Antonio Cordero Balcazar 54a20117ce * New extranetworks command to group the triggers with the extranetwork.
* Mapping of extranetworks through the new command. This allows you to use a "virtual" extranetwork that is converted to a real one depending on conditions, like the model kind/variant.
* Fixed a parsing bug with the if command.
* Some refactoring and additional checks.
2025-09-01 16:50:46 +02:00
Antonio Cordero Balcazar fe61e54f41 * Selection of prompt parser based on heuristics to improve parsing time on simpler cases.
* Improved timing logging.
* Refactor application support handling.
* Enhanced regex patterns in `ppp.py` for wildcard and choice detection to ensure proper escaping.
* Included icon source files.
2025-08-08 14:05:34 +02:00
Antonio Cordero Balcazar c03339a618 * Better processing of interruptions using an exception.
* New option on_warning to stop on invalid content.
* New option cleanup_extra_separators_include_eol to choose whether to also remove adjacent EOLs when removing extra separators or not. Default is not to do it. In previous versions it was forced.
* Refactoring of the cleanup method.
* Only simplify attention modifiers when the merge attention option is set.
* A1111: avoid logging about HR prompts and avoid saving them in extra metadata when not different.
* A1111: consider options and environment info in the caching of results.
* Some refactorings.
2025-05-31 12:08:57 +02:00
Antonio Cordero Balcazar bfc47d0d53 * A1111: add prompts to metadata only if they change 2025-05-03 12:07:18 +02:00
Antonio Cordero Balcazar b837f38422 * if_wildcards option defaults now to stop.
* ComfyUI: new wildcards_input.
* removed json import since yaml can read them.
* more models detected in ComfyUI.
* ComfyUI: example workflow.
2025-05-03 12:07:18 +02:00
Antonio Cordero Balcazar 5e3c8cbc44 * ComfyUI: updates to the ComfyUI registry workflow.
* A1111: option to add original prompts to metadata.
* fixed some bogus import error shenanigans.
* warn when a model name detects multiple variants.
2025-04-11 17:50:26 +02:00
27 changed files with 4021 additions and 731 deletions
+5 -3
View File
@@ -8,17 +8,19 @@ on:
paths: paths:
- "pyproject.toml" - "pyproject.toml"
permissions:
issues: write
jobs: jobs:
publish-node: publish-node:
name: Publish Custom Node to registry name: Publish Custom Node to registry
runs-on: ubuntu-latest runs-on: ubuntu-latest
# if this is a forked repository. Skipping the workflow. if: ${{ github.repository_owner == 'acorderob' }}
if: github.event.repository.fork == false
steps: steps:
- name: Check out code - name: Check out code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Publish Custom Node - name: Publish Custom Node
uses: Comfy-Org/publish-node-action@main uses: Comfy-Org/publish-node-action@v1
with: with:
## Add your own personal access token to your Github Repository secrets and reference it here. ## Add your own personal access token to your Github Repository secrets and reference it here.
personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }} personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }}
+2
View File
@@ -3,3 +3,5 @@
.vscode/**/* .vscode/**/*
!.vscode/settings.json !.vscode/settings.json
!.vscode/launch.json !.vscode/launch.json
tests/tests_local.py
+3 -3
View File
@@ -2,12 +2,11 @@
The Prompt PostProcessor (PPP), formerly known as "sd-webui-sendtonegative", is an extension designed to process the prompt, possibly after other extensions have modified it. This extension is compatible with: The Prompt PostProcessor (PPP), formerly known as "sd-webui-sendtonegative", is an extension designed to process the prompt, possibly after other extensions have modified it. This extension is compatible with:
* [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
* [AUTOMATIC1111 Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui) * [AUTOMATIC1111 Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
* [Forge](https://github.com/lllyasviel/stable-diffusion-webui-forge) * [Forge](https://github.com/lllyasviel/stable-diffusion-webui-forge)
* [reForge](https://github.com/Panchovix/stable-diffusion-webui-reForge) * [reForge](https://github.com/Panchovix/stable-diffusion-webui-reForge)
* [SD.Next](https://github.com/vladmandic/automatic)
* ...and probably other forks * ...and probably other forks
* [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
Currently this extension has these functions: Currently this extension has these functions:
@@ -15,6 +14,7 @@ Currently this extension has these functions:
* Set and modify local variables. * Set and modify local variables.
* Filter content based on the loaded SD model or a variable. * Filter content based on the loaded SD model or a variable.
* Process wildcards. Compatible with Dynamic Prompts formats. Can also detect invalid wildcards and act as you choose. * Process wildcards. Compatible with Dynamic Prompts formats. Can also detect invalid wildcards and act as you choose.
* Map extranetworks (LoRAs) depending on conditions (like the loaded model variant).
* Clean up the prompt and negative prompt. * Clean up the prompt and negative prompt.
Note: when used in an *A1111* compatible webui, the extension must be loaded after any other extension that modifies the prompt (like another wildcards extension). Usually extensions load by their folder name in alphanumeric order, so if the extensions are not loading in the correct order just rename this extension's folder so the ordering works out. When in doubt, just rename this extension's folder with a "z" in front (for example) so that it is the last one to load, or manually set such folder name when installing it. Note: when used in an *A1111* compatible webui, the extension must be loaded after any other extension that modifies the prompt (like another wildcards extension). Usually extensions load by their folder name in alphanumeric order, so if the extensions are not loading in the correct order just rename this extension's folder so the ordering works out. When in doubt, just rename this extension's folder with a "z" in front (for example) so that it is the last one to load, or manually set such folder name when installing it.
@@ -49,7 +49,7 @@ On *A1111* compatible webuis:
3. Click the Install button 3. Click the Install button
4. Restart the webui 4. Restart the webui
On *SD.Next* I recommend you disable the native wildcard processing. On *SD.Next* I recommend you to disable the native wildcard processing and use the old UI.
On *ComfyUI*: On *ComfyUI*:
+9 -3
View File
@@ -9,6 +9,8 @@
* **seed**: Connect here the seed used. By default it is -1 (random). * **seed**: Connect here the seed used. By default it is -1 (random).
* **pos_prompt**: Connect here the prompt text, or fill it as a widget. * **pos_prompt**: Connect here the prompt text, or fill it as a widget.
* **neg_prompt**: Connect here the negative prompt text, or fill it as a widget. * **neg_prompt**: Connect here the negative prompt text, or fill it as a widget.
* **wc_wildcards_input**: Wildcards definitions (in yaml or json format). Direct input added to the ones found in the wildcards folders. Allows wildcards to be included in the workflow.
* **en_mappings_input**: Extranetwork Mappings definitions (in yaml format). Direct input added to the ones found in the extranetwork mappings folders. Allows the mappings to be included in the workflow.
Other common settings (see [below](#common-settings)) also appear as inputs or widgets. Other common settings (see [below](#common-settings)) also appear as inputs or widgets.
@@ -32,6 +34,7 @@ With this prompt: `__quality__, 1girl, ${head:__eyes__, __hair__, __expression__
### General settings ### General settings
* **Debug level**: what to write to the console. Note: in *SD.Next* debug messages only show if you launch it with the `--debug` argument. * **Debug level**: what to write to the console. Note: in *SD.Next* debug messages only show if you launch it with the `--debug` argument.
* **What to do on invalid content warnings?**: warn on the console or stop the generation.
* **Model variant definitions**: definitions for model variants to be recognized based on strings found in the full filename. * **Model variant definitions**: definitions for model variants to be recognized based on strings found in the full filename.
The format for each line is (with *kind* being one of the base model identifiers or not defined): The format for each line is (with *kind* being one of the base model identifiers or not defined):
@@ -40,12 +43,14 @@ With this prompt: `__quality__, 1girl, ${head:__eyes__, __hair__, __expression__
The default value defines strings for *Pony* and *Illustrious* models. The default value defines strings for *Pony* and *Illustrious* models.
* **Apply in img2img**: check if you want to do the processing in img2img processes (*does not apply to the ComfyUI node*). * **Apply in img2img**: check if you want to do the processing in img2img processes (*does not apply to the ComfyUI node*).
* **Add original prompts to metadata**: adds original prompts to the metadata if they have changed (*does not apply to the ComfyUI node*).
* **Extranetwork Mappings folders**: you can enter multiple folders separated by commas. In *ComfyUI* you can leave it empty and add a `ppp_extranetworkmappings` entry in the **extra_model_paths.yaml** file.
### Wildcard settings ### Wildcard settings
* **Process wildcards**: you can choose to process them with this extension or use a different one. * **Process wildcards**: you can choose to process wildcards and choices with this extension or use a different one.
* **Wildcards folders**: you can enter multiple folders separated by commas. In *ComfyUI* you can leave it empty and add a `wildcards` entry in the **extra_model_paths.yaml** file. * **Wildcards folders**: you can enter multiple folders separated by commas. In *ComfyUI* you can leave it empty and add a `ppp_wildcards` or `wildcards` entry in the **extra_model_paths.yaml** file.
* **What to do with remaining wildcards?**: select what do you want to do with any found wildcards. * **What to do with remaining wildcards?**: select what do you want to do with any found wildcards/choices (when process wildcards is off or after the processing).
* **Ignore**: do not try to detect wildcards. * **Ignore**: do not try to detect wildcards.
* **Remove**: detect wildcards and remove them. * **Remove**: detect wildcards and remove them.
* **Add visible warning**: detect wildcards and add a warning text to the prompt, that hopefully produces a noticeable generation. * **Add visible warning**: detect wildcards and add a warning text to the prompt, that hopefully produces a noticeable generation.
@@ -63,6 +68,7 @@ With this prompt: `__quality__, 1girl, ${head:__eyes__, __hair__, __expression__
* **Remove empty constructs**: removes attention/scheduling/alternation constructs when they are invalid. * **Remove empty constructs**: removes attention/scheduling/alternation constructs when they are invalid.
* **Remove extra separators**: removes unnecessary separators. This applies to the configured separator and regular commas. * **Remove extra separators**: removes unnecessary separators. This applies to the configured separator and regular commas.
* **Remove additional extra separators**: removes unnecessary separators at start or end of lines. This applies to the configured separator and regular commas. * **Remove additional extra separators**: removes unnecessary separators at start or end of lines. This applies to the configured separator and regular commas.
* **The extra separators options also remove EOLs**: in the previous two options it also removes EOLs attached to the separators.
* **Clean up around BREAKs**: removes consecutive BREAKs and unnecessary commas and space around them. * **Clean up around BREAKs**: removes consecutive BREAKs and unnecessary commas and space around them.
* **Use EOL instead of Space before BREAKs**: add a newline before BREAKs. * **Use EOL instead of Space before BREAKs**: add a newline before BREAKs.
* **Clean up around ANDs**: removes consecutive ANDs and unnecessary commas and space around them. * **Clean up around ANDs**: removes consecutive ANDs and unnecessary commas and space around them.
+92 -5
View File
@@ -24,16 +24,19 @@ The construct parameters can be written with the following options (all are opti
* "**~**" or "**@**": sampler (for compatibility with *Dynamic Prompts*), but only "**~**" (random) is supported. * "**~**" or "**@**": sampler (for compatibility with *Dynamic Prompts*), but only "**~**" (random) is supported.
* "**r**": means it allows repetition of the choices. * "**r**": means it allows repetition of the choices.
* "**o**": means it is "optional", and no error will be raised if there are no choices to select from.
* "**n**" or "**n-m**" or "**n-**" or "**-m**": number or range of choices to select. Allows zero as the start of a range. Default is 1. * "**n**" or "**n-m**" or "**n-**" or "**-m**": number or range of choices to select. Allows zero as the start of a range. Default is 1.
* "**$$sep**": separator when multiple choices are selected. Default is set in settings. * "**$$sep**": separator when multiple choices are selected. Default is set in settings.
* "**$$**": end of the parameters. * "**$$**": end of the parameters (not optional if any parameters).
Regarding the "optional" flag, consider this scenario: due to their conditions no choice is available. It will raise an error. If you add the `o` then it will just return an empty string. This is only necessary if all choices have conditions and they could all be false. It is not the same as setting a range starting at 0, because that would be an allowed number of returned choices. If you do this and no choices are available, no error is raised.
The choice options are as follows: The choice options are as follows:
* "**'identifiers'**": comma separated labels for the choice (optional, quotes can be single or double). Only makes sense inside a wildcard definition. Can be used when specifying the wildcard to select this specific choice. It's case insensitive. * "**'identifiers'**": comma separated labels for the choice (optional, quotes can be single or double). Only makes sense inside a wildcard definition. Can be used when specifying the wildcard to select this specific choice. It's case insensitive.
* "**n**": weight of the choice (optional, default 1). * "**n**": weight of the choice (optional, default 1).
* "**if condition**": filters out the choice if the condition is false (optional; this is an extension to the *Dynamic Prompts* syntax). Same conditions as in the `if` command. * "**if condition**": filters out the choice if the condition is false (optional; this is an extension to the *Dynamic Prompts* syntax). Same conditions as in the `if` command.
* "**::**": end of choice options * "**::**": end of choice options (not optional if any options)
Whitespace is allowed between parameters/options. Whitespace is allowed between parameters/options.
@@ -46,6 +49,7 @@ These are examples of formats you can use to insert a choice construct:
| `{2-3$$2::choice1\|choice2\|choice3}` | select 2 to 3 choices, one of them has a weight | | `{2-3$$2::choice1\|choice2\|choice3}` | select 2 to 3 choices, one of them has a weight |
| `{r2-3$$choice1\|choice2\|choice3}` | select 2 to 3 choices allowing repetition | | `{r2-3$$choice1\|choice2\|choice3}` | select 2 to 3 choices allowing repetition |
| `{2-3$$ / $$choice1\|choice2\|choice3}` | select 2 to 3 choices with separator " / " | | `{2-3$$ / $$choice1\|choice2\|choice3}` | select 2 to 3 choices with separator " / " |
| `{o$$if _is_sd1::choice1\|if _is_sd2::choice2}`| select 1 choice, both have conditions, if none matches it is allowed because we indicate that it is optional |
Notes: Notes:
@@ -79,6 +83,8 @@ These are examples of formats you can use to insert a wildcard:
| `__2-3$$ / $$path/wildcard__` | select 2 to 3 choices with separator " / " | | `__2-3$$ / $$path/wildcard__` | select 2 to 3 choices with separator " / " |
| `__path/wildcard(var=value)__` | select 1 choice using the specified variable value in the evaluation. | | `__path/wildcard(var=value)__` | select 1 choice using the specified variable value in the evaluation. |
Wildcards cannot be used inside an extranetwork tag (because some lora names contain double underscores). If you need to choose from multiple loras put the whole extranetwork tag inside a wildcard, or use choices.
### Wildcard definitions ### Wildcard definitions
A wildcard definition can be: A wildcard definition can be:
@@ -90,7 +96,7 @@ The best format is a yaml file with a dictionary of wildcards inside. An editor
In a choice, the content after a `#` is ignored. In a choice, the content after a `#` is ignored.
If the first choice follows the format of wildcard parameters, it will be used as default parameters for that wildcard (see examples in the tests folder). The choices of the wildcard follow the same format as in the choices construct, or the object format of *Dynamic Prompts* (only in structured files). If using the object format for a choice you can use a new `if` property for the condition, and the `labels` property (an array of strings) in addition to the standard `weight` and `text`/`content`. If the first choice follows the format of wildcard parameters (*including the final `$$`*), it will be used as default parameters for that wildcard (see examples in the tests folder). The choices of the wildcard follow the same format as in the choices construct, or the object format of *Dynamic Prompts* (only in structured files). If using the object format for a choice you can use a new `if` property for the condition, and the `labels` property (an array of strings) in addition to the standard `weight` and `text`/`content`.
```yaml ```yaml
{ labels: ["some_label"], weight: 2, if: "_is_pony", content: "the text" } # "text" property can be used instead of "content" { labels: ["some_label"], weight: 2, if: "_is_pony", content: "the text" } # "text" property can be used instead of "content"
@@ -99,8 +105,8 @@ If the first choice follows the format of wildcard parameters, it will be used a
Wildcard parameters in a json/yaml file can also be in object format, and support two additional properties, prefix and suffix: Wildcard parameters in a json/yaml file can also be in object format, and support two additional properties, prefix and suffix:
```yaml ```yaml
{ sampler: "~", repeating: false, count: 2, prefix: "prefix-", suffix: "-suffix", separator: "/" } { sampler: "~", repeating: false, optional: false, count: 2, prefix: "prefix-", suffix: "-suffix", separator: "/" }
{ sampler: "~", repeating: false, from: 2, to: 3, prefix: "prefix-", suffix: "-suffix", separator: "/" } { sampler: "~", repeating: false, optional: false, from: 2, to: 3, prefix: "prefix-", suffix: "-suffix", separator: "/" }
``` ```
The prefix and suffix are added to the result along with the selected choices and separators. They can contain other constructs, but the separator can't. The prefix and suffix are added to the result along with the selected choices and separators. They can contain other constructs, but the separator can't.
@@ -111,6 +117,10 @@ Wildcards can contain just one choice. In json and yaml formats this allows the
A choice inside a wildcard can also be a list or a dictionary of one element containing a list. These are considered anonymous wildcards. With a list it will be an anonymous wildcard with no choice options, and with a dictionary the key will be the options for the choice containing the anonymous wildcard and the value the choices of the anonymous wildcard. Anonymous wildcards can help formatting complex choice values that are used in only one place and thus creating a regular wildcard is not necessary. See test.yaml for examples. A choice inside a wildcard can also be a list or a dictionary of one element containing a list. These are considered anonymous wildcards. With a list it will be an anonymous wildcard with no choice options, and with a dictionary the key will be the options for the choice containing the anonymous wildcard and the value the choices of the anonymous wildcard. Anonymous wildcards can help formatting complex choice values that are used in only one place and thus creating a regular wildcard is not necessary. See test.yaml for examples.
Note: the files should have UTF-8 encoding. The extension will also try with windows-1252 if that fails.
Wildcard definitions are reloaded automatically on each generation if they change.
### Detection of remaining wildcards ### Detection of remaining wildcards
This extension should run after any other wildcard extensions, so if you don't use the internal wildcards processing, any remaining wildcards present in the prompt or negative_prompt at this point must be invalid. Usually you might not notice this problem until you check the image metadata, so this option gives you some ways to detect and treat the problem. This extension should run after any other wildcard extensions, so if you don't use the internal wildcards processing, any remaining wildcards present in the prompt or negative_prompt at this point must be invalid. Usually you might not notice this problem until you check the image metadata, so this option gives you some ways to detect and treat the problem.
@@ -221,6 +231,83 @@ The variable can be one set with the `set` or `add` commands (user variables) or
Only one of the options will end up in the prompt, depending on the loaded model. Only one of the options will end up in the prompt, depending on the loaded model.
## ExtraNetwork command
This command is a shortcut to add an extranetwork (usually a lora), and its triggers, with conditions. More legible and sometimes shorter than adding regular extranetworks inside if commands.
The full format is:
`<ppp:ext type name [parameters] [if condition]>[triggers]<ppp:/ext>`
`<ppp:ext type name [parameters] [if condition]>`
The `type` is the kind of extranetwork, like `lora` or `hypernet`.
The `name` is the extranetwork identifier. If it is not a regular identifier (i.e. starts with a number or contains spaces or symbols) it should be inside quotes.
The `parameters` is optional and its format depends on the extranetwork type. With loras or hypernets it is usually a single weight number, so if the type is one of those and there are no parameters it will default to `1`. If it is not a number it should go inside quotes.
The `condition` uses the same format as in the `if` command, and it is also optional.
The `triggers` are also optional, and can be any content. If there are no triggers the command ending can be omitted.
If the condition passes (or if there is no condition) the extranetwork tag will be built and added to the result along with any triggers.
### Examples
(multiline to be easier to read)
```text
<ppp:ext lora test_sd1 if _is_sd1>test sd1x<ppp:/ext>
<ppp:ext lora test_pony 0.5 if _is_pony>test pony<ppp:/ext>
<ppp:ext lora test_ilxl if _is_illustrious>
<ppp:ext lora 'test sdxl' '1:0.8' if _is_pure_sdxl>test sdxl<ppp:/ext>
```
Will turn into one of these (or none) depending on the model:
* `<lora:test_sd1:1>test sd1x`
* `<lora:test_pony:0.5>test pony`
* `<lora:test_illustrious:1>`
* `<lora:test sdxl:1:0.8>test sdxl`
### Extranetworks mappings
The extranetwork command supports specifying mappings of extranetworks (like LoRAs), so, for example, a different one can be used depending on the loaded model.
If the type of extranetwork is prefixed with a `$` the command will look for a mapping.
The mappings are configured in yaml files in any of the configured extranetwork mappings folders. The format is like this:
```yaml
extnettype:
mappingname:
- condition: "<a supported condition>"
name: "<name of the extranetwork>"
parameters: "<parameters of the extranetwork>"
triggers: [<list of triggers>]
weight: 1.0
...
```
Used like this:
```text
<ppp:ext $lora mappingname>
<ppp:ext $lora mappingname>inline triggers<ppp:/ext>
```
Each mapping can have any number of elements in its list of mappings. There are no mandatory properties for a mapping. The properties mean the following:
* `extnettype`: the kind of extranetwork, for example `lora`.
* `mappingname`: the name you want to give to the mapping, to be referenced in the command.
* `condition`: the condition to check for this mapping to be used (usually it should be one of the `_is_*` variables). If the conditions of multiple mappings evaluate to True, one will be chosen randomly. If the condition is missing it is considered True, to be used in the last mapping to catch as an "else" condition, and will be used if no other mapping applies.
* `name`: name of the real extranetwork. If it is missing no extranetwork tag will be added.
* `parameters`: parameters for the real extranetwork. If it is missing it is assumed "1" for loras and hypernets. If both this parameter and the parameter in the ext command are numbers they are multiplied for the result. In other case the parameter of the ext command, if it exists, is used.
* `triggers`: list of trigger strings. If it is missing, only the inline triggers in the ext command will be added.
* `weight`: weight for this variant, in case multiple of them apply, to choose one. Default is 1.
See the file in the tests folder as an example.
## Sending content to the negative prompt ## Sending content to the negative prompt
The new format for this command is like this: The new format for this command is like this:
+70 -18
View File
@@ -6,7 +6,7 @@ BOOLEAN: /true|false/i
WILDCARD_NAME: /(?:(?!__|\$\$|[('"])\S)+/ WILDCARD_NAME: /(?:(?!__|\$\$|[('"])\S)+/
INDEX: INT | IDENTIFIER INDEX: INT | IDENTIFIER
IDENTIFIER: CNAME IDENTIFIER: CNAME
SIMPLEVALUE: STRING | NUMBER | BOOLEAN SIMPLEVALUE: STRING | SIGNED_NUMBER | BOOLEAN
// plain text and weights // plain text and weights
?plain: /((?!__|\bAND\b|\${)[^\\()\[\]:<>${]|\\.)+/s // exclude only the starting ones ?plain: /((?!__|\bAND\b|\${)[^\\()\[\]:<>${]|\\.)+/s // exclude only the starting ones
@@ -27,15 +27,56 @@ promptcomp.4: promptcomppart ( [ ":" numpar ] ( /\bAND\b/ promptcomppart [ ":" n
promptcomppart: content promptcomppart: content
// simple prompts // simple prompts
?content.2: ( old_content | new_content | plain | specialchars )*
?inside_content.2: ( new_content | plain | specialchars )*
?content_choice.2: ( old_content | new_content | plain_choice | specialchars_choice )*
?content_var.2: ( old_content | new_content | plain_var | specialchars_var )*
?content_negtag.2: ( old_content | new_content_negtag | plain | specialchars_negtag )*
?content_alternate.2: ( old_content | new_content | plain_alternate | specialchars_alternate )*
?old_content.2: ( attention | scheduled | alternate | extranetworktag )+ ?old_content.2: ( attention | scheduled | alternate | extranetworktag )+
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | wildcard | choices )+ //#if ALLOW_NEW_CONTENT
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | wildcard | choices )+ ?content.2: ( old_content | new_content | plain | specialchars )*
?content_choice.2: ( old_content | new_content | plain_choice | specialchars_choice )*
?content_var.2: ( old_content | new_content | plain_var | specialchars_var )*
?content_negtag.2: ( old_content | new_content_negtag | plain | specialchars_negtag )*
?content_alternate.2: ( old_content | new_content | plain_alternate | specialchars_alternate )*
?content_en.2: (new_content_en | plain | specialchars )*
//#if ALLOW_WILDCARDS ALLOW_CHOICES ALLOW_COMMVARS
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | commandext | wildcard | choices )+
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | commandext | wildcard | choices )+
?new_content_en.3: ( variableset | variableuse | commandset | commandecho | commandif | choices )+
//#elif ALLOW_WILDCARDS !ALLOW_CHOICES ALLOW_COMMVARS
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | commandext | wildcard )+
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | commandext | wildcard )+
?new_content_en.3: ( variableset | variableuse | commandset | commandecho | commandif )+
//#elif !ALLOW_WILDCARDS ALLOW_CHOICES ALLOW_COMMVARS
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | commandext | choices )+
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | commandext | choices )+
?new_content_en.3: ( variableset | variableuse | commandset | commandecho | commandif | choices )+
//#elif ALLOW_WILDCARDS ALLOW_CHOICES !ALLOW_COMMVARS
?new_content.3: ( wildcard | choices )+
?new_content_negtag.3: ( wildcard | choices )+
?new_content_en.3: ( choices )+
//#elif ALLOW_WILDCARDS !ALLOW_CHOICES !ALLOW_COMMVARS
?new_content.3: ( wildcard )+
?new_content_negtag.3: ( wildcard )+
?new_content_en.3: /(?!)./ // never matches
//#elif !ALLOW_WILDCARDS ALLOW_CHOICES !ALLOW_COMMVARS
?new_content.3: ( choices )+
?new_content_negtag.3: ( choices )+
?new_content_en.3: ( choices )+
//#elif !ALLOW_WILDCARDS !ALLOW_CHOICES ALLOW_COMMVARS
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | commandext )+
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | commandext )+
?new_content_en.3: ( variableset | variableuse | commandset | commandecho | commandif )+
//#else
?new_content.3: ( variableset | variableuse | commandstn | commandstni | commandset | commandecho | commandif | commandext | wildcard | choices )+
?new_content_negtag.3: ( variableset | variableuse | commandset | commandecho | commandif | wildcard | commandext | choices )+
?new_content_en.3: ( variableset | variableuse | commandset | commandecho | commandif | choices )+
//#endif
//#else
?content.2: ( old_content | plain | specialchars )*
?content_choice.2: ( old_content | plain_choice | specialchars_choice )*
?content_var.2: ( old_content | plain_var | specialchars_var )*
?content_negtag.2: ( old_content | plain | specialchars_negtag )*
?content_alternate.2: ( old_content | plain_alternate | specialchars_alternate )*
?content_en.2: (plain | specialchars )*
//#endif
// attention modifiers // attention modifiers
attention: ( "(" content [ ":" numpar ] ")" ) | ( "[" content "]" ) attention: ( "(" content [ ":" numpar ] ")" ) | ( "[" content "]" )
@@ -46,7 +87,8 @@ alternateoption: content_alternate
scheduled: "[" [ content ":" ] content ":" numpar "]" scheduled: "[" [ content ":" ] content ":" numpar "]"
// extra network tags // extra network tags
extranetworktag: "<" /(?!ppp:)\w+:/ inside_content ">" extranetworktag: "<" /(?!ppp:)\w+:/ encontent ">"
?encontent.3: content_en
// command: stn (send to negative) // command: stn (send to negative)
commandstn: "<ppp:stn" [ _WHITESPACE /s|e|p\d/ ] _WHITESPACE? ">" content_negtag "<ppp:/stn>" commandstn: "<ppp:stn" [ _WHITESPACE /s|e|p\d/ ] _WHITESPACE? ">" content_negtag "<ppp:/stn>"
@@ -57,12 +99,12 @@ commandif.2: commandif_if commandif_elif* commandif_else? "<ppp:/if>"
commandif_if: "<ppp:if" _WHITESPACE condition _WHITESPACE? ">" ifvalue commandif_if: "<ppp:if" _WHITESPACE condition _WHITESPACE? ">" ifvalue
commandif_elif: "<ppp:elif" _WHITESPACE condition _WHITESPACE? ">" ifvalue commandif_elif: "<ppp:elif" _WHITESPACE condition _WHITESPACE? ">" ifvalue
commandif_else: "<ppp:else" _WHITESPACE? ">" ifvalue commandif_else: "<ppp:else" _WHITESPACE? ">" ifvalue
ifvalue: content ifvalue.3: content
// conditions // conditions
?condition: grouped_condition | ungrouped_condition ?condition: grouped_condition | ungrouped_condition
?ungrouped_condition: operation | basic_condition ?ungrouped_condition: operation | basic_condition
?grouped_condition: "(" _WHITESPACE? condition _WHITESPACE? ")" ?grouped_condition.8: "(" _WHITESPACE? condition _WHITESPACE? ")"
?basic_condition: comparison_simple_value | comparison_list_value | truthy_operand ?basic_condition: comparison_simple_value | comparison_list_value | truthy_operand
?operation: operation_not | operation_and | operation_or ?operation: operation_not | operation_and | operation_or
operation_and: condition _WHITESPACE "and" _WHITESPACE condition operation_and: condition _WHITESPACE "and" _WHITESPACE condition
@@ -71,14 +113,24 @@ operation_not: "not" ( ( _WHITESPACE ungrouped_condition ) | ( _WHITESPACE? grou
truthy_operand: IDENTIFIER truthy_operand: IDENTIFIER
comparison_simple_value: IDENTIFIER _WHITESPACE ( /not/ _WHITESPACE )? /eq|ne|gt|lt|ge|le|contains/ _WHITESPACE SIMPLEVALUE comparison_simple_value: IDENTIFIER _WHITESPACE ( /not/ _WHITESPACE )? /eq|ne|gt|lt|ge|le|contains/ _WHITESPACE SIMPLEVALUE
comparison_list_value: IDENTIFIER _WHITESPACE ( /not/ _WHITESPACE )? /contains|in/ _WHITESPACE listvalue comparison_list_value: IDENTIFIER _WHITESPACE ( /not/ _WHITESPACE )? /contains|in/ _WHITESPACE listvalue
listvalue: "(" _WHITESPACE? SIMPLEVALUE ( _WHITESPACE? "," _WHITESPACE? SIMPLEVALUE )* _WHITESPACE? ")" listvalue.9: "(" _WHITESPACE? SIMPLEVALUE ( _WHITESPACE? "," _WHITESPACE? SIMPLEVALUE )* _WHITESPACE? ")"
// command: set // command: set
commandset: "<ppp:set" _WHITESPACE IDENTIFIER [ commandsetmodifiers ] _WHITESPACE? ">" content "<ppp:/set>" commandset: "<ppp:set" _WHITESPACE IDENTIFIER [ commandsetmodifiers ] _WHITESPACE? ">" commandsetcontent "<ppp:/set>"
commandsetmodifiers: (_WHITESPACE /evaluate|ifundefined|add/ )+ commandsetmodifiers: (_WHITESPACE /evaluate|ifundefined|add/ )+
?commandsetcontent.3: content
// command: echo // command: echo
commandecho: "<ppp:echo" _WHITESPACE IDENTIFIER _WHITESPACE? ">" [ content "<ppp:/echo>" ] commandecho: "<ppp:echo" _WHITESPACE IDENTIFIER _WHITESPACE? ">" [ commandechodefault "<ppp:/echo>" ]
?commandechodefault.3: content
// command: ext
commandext: "<ppp:ext" _WHITESPACE commandexttype _WHITESPACE commandextid [ _WHITESPACE? commandextparams ] [ _WHITESPACE? commandextif ] _WHITESPACE? ">" [ commandexttriggers "<ppp:/ext>" ]
commandexttype: [/\$/] IDENTIFIER
?commandextid: STRING | CNAME
?commandextparams: STRING | SIGNED_NUMBER
?commandextif: "if" _WHITESPACE condition
?commandexttriggers.3: content
// variable set // variable set
variableset.2: "${" _WHITESPACE? IDENTIFIER [ variablesetmodifiers ] _WHITESPACE? "=" [ /!/ ] varvalue "}" variableset.2: "${" _WHITESPACE? IDENTIFIER [ variablesetmodifiers ] _WHITESPACE? "=" [ /!/ ] varvalue "}"
@@ -93,14 +145,14 @@ wildcard.2: "__" [ choicesoptions_sampler | ( choicesoptions _WHITESPACE? "$$" )
wc_filter: /["']/ ( [ /#/ ] wc_filter_or | ( /#?\^/ WILDCARD_NAME ) ) /["']/ wc_filter: /["']/ ( [ /#/ ] wc_filter_or | ( /#?\^/ WILDCARD_NAME ) ) /["']/
wc_filter_or: wc_filter_and ( _WHITESPACE? "," _WHITESPACE? wc_filter_and )* wc_filter_or: wc_filter_and ( _WHITESPACE? "," _WHITESPACE? wc_filter_and )*
wc_filter_and: INDEX ( _WHITESPACE? "+" _WHITESPACE? INDEX )* wc_filter_and: INDEX ( _WHITESPACE? "+" _WHITESPACE? INDEX )*
wildcardvar: "(" _WHITESPACE? IDENTIFIER _WHITESPACE? "=" varvalue ")" wildcardvar.7: "(" _WHITESPACE? IDENTIFIER _WHITESPACE? "=" varvalue ")"
// choices // choices
choices.2: "{" [ choicesoptions_sampler | ( choicesoptions _WHITESPACE? "$$" ) ] choice ( "|" choice )* "}" choices.2: "{" [ choicesoptions_sampler | ( choicesoptions _WHITESPACE? "$$" ) ] choice ( "|" choice )* "}"
choicesoptions: [ choicesoptions_sampler ] [ _WHITESPACE? choicesoptions_rep ] ( ( [ _WHITESPACE? choicesoptions_from ] "-" [ _WHITESPACE? choicesoptions_to ] ) | [ _WHITESPACE? choicesoptions_num ] ) [ _WHITESPACE? choicesoptions_sep ] choicesoptions: [ choicesoptions_sampler ] [ _WHITESPACE? choicesoptions_flags ] ( ( [ _WHITESPACE? choicesoptions_from ] "-" [ _WHITESPACE? choicesoptions_to ] ) | [ _WHITESPACE? choicesoptions_num ] ) [ _WHITESPACE? choicesoptions_sep ]
choicesoptions_sampler: /[~@]/ // ~ for random, @ for cyclical choicesoptions_sampler: /[~@]/ // ~ for random, @ for cyclical
choicesoptions_rep: /r/ choicesoptions_flags: /[ro]{1,2}/ // r for repeating, o for optional
choicesoptions_num: INT choicesoptions_num: INT
choicesoptions_from: INT choicesoptions_from: INT
choicesoptions_to: INT choicesoptions_to: INT
File diff suppressed because one or more lines are too long
Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 20 KiB

+682 -196
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -2,7 +2,7 @@ from collections import OrderedDict
from logging import Logger from logging import Logger
from typing import Tuple from typing import Tuple
from ppp_logging import DEBUG_LEVEL from ppp_logging import DEBUG_LEVEL # pylint: disable=import-error
class PPPLRUCache: class PPPLRUCache:
+128 -62
View File
@@ -1,14 +1,14 @@
# pylint: disable=missing-module-docstring, missing-class-docstring, missing-function-docstring, invalid-name
import os import os
# pylint: disable=import-error # pylint: disable=import-error
import folder_paths # type: ignore import folder_paths # type: ignore
import nodes # type: ignore import nodes
from .ppp import PromptPostProcessor from .ppp import PromptPostProcessor
from .ppp_hosts import SUPPORTED_APPS
from .ppp_logging import DEBUG_LEVEL, PromptPostProcessorLogFactory from .ppp_logging import DEBUG_LEVEL, PromptPostProcessorLogFactory
from .ppp_wildcards import PPPWildcards from .ppp_wildcards import PPPWildcards
from .ppp_enmappings import PPPExtraNetworkMappings
if __name__ == "__main__": if __name__ == "__main__":
raise SystemExit("This script must be run from ComfyUI") raise SystemExit("This script must be run from ComfyUI")
@@ -22,12 +22,13 @@ class PromptPostProcessorComfyUINode:
logger = None logger = None
def __init__(self): def __init__(self):
lf = PromptPostProcessorLogFactory() lf = PromptPostProcessorLogFactory(SUPPORTED_APPS.comfyui)
self.logger = lf.log self.logger = lf.log
grammar_filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), "grammar.lark") grammar_filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), "grammar.lark")
with open(grammar_filename, "r", encoding="utf-8") as file: with open(grammar_filename, "r", encoding="utf-8") as file:
self.grammar_content = file.read() self.grammar_content = file.read()
self.wildcards_obj = PPPWildcards(lf.log) self.wildcards_obj = PPPWildcards(lf.log)
self.extranetwork_mappings_obj = PPPExtraNetworkMappings(lf.log)
self.logger.info(f"{PromptPostProcessor.NAME} {PromptPostProcessor.VERSION} initialized") self.logger.info(f"{PromptPostProcessor.NAME} {PromptPostProcessor.VERSION} initialized")
class SmartType(str): class SmartType(str):
@@ -48,8 +49,6 @@ class PromptPostProcessorComfyUINode:
"multiline": True, "multiline": True,
"default": "", "default": "",
"dynamicPrompts": False, "dynamicPrompts": False,
"defaultInput": True,
"forceInput": False,
}, },
), ),
"neg_prompt": ( "neg_prompt": (
@@ -58,8 +57,6 @@ class PromptPostProcessorComfyUINode:
"multiline": True, "multiline": True,
"default": "", "default": "",
"dynamicPrompts": False, "dynamicPrompts": False,
"defaultInput": True,
"forceInput": False,
}, },
), ),
}, },
@@ -77,16 +74,13 @@ class PromptPostProcessorComfyUINode:
{ {
"default": "", "default": "",
"placeholder": "full path of the model", "placeholder": "full path of the model",
"defaultInput": True, "dynamicPrompts": False,
"forceInput": False,
}, },
), ),
"seed": ( "seed": (
"INT", "INT",
{ {
"default": -1, "default": -1,
"defaultInput": True,
"forceInput": False,
}, },
), ),
"debug_level": ( "debug_level": (
@@ -94,8 +88,13 @@ class PromptPostProcessorComfyUINode:
{ {
"default": DEBUG_LEVEL.minimal.value, "default": DEBUG_LEVEL.minimal.value,
"tooltip": "Debug level", "tooltip": "Debug level",
"defaultInput": False, },
"forceInput": False, ),
"on_warnings": (
[e.value for e in PromptPostProcessor.ONWARNING_CHOICES],
{
"default": PromptPostProcessor.ONWARNING_CHOICES.warn.value,
"tooltip": "How to handle invalid content warnings",
}, },
), ),
"variants_definitions": ( "variants_definitions": (
@@ -105,8 +104,7 @@ class PromptPostProcessorComfyUINode:
"multiline": True, "multiline": True,
"placeholder": "", "placeholder": "",
"tooltip": "Definitions for variant models to be recognized based on strings found in the full filename. Format for each line is: 'name(kind)=comma separated list of substrings (case insensitive)' with kind being one of the base model types or not specified", "tooltip": "Definitions for variant models to be recognized based on strings found in the full filename. Format for each line is: 'name(kind)=comma separated list of substrings (case insensitive)' with kind being one of the base model types or not specified",
"defaultInput": False, "dynamicPrompts": False,
"forceInput": False,
}, },
), ),
"wc_process_wildcards": ( "wc_process_wildcards": (
@@ -116,8 +114,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Process wildcards in the prompt", "tooltip": "Process wildcards in the prompt",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"wc_wildcards_folders": ( "wc_wildcards_folders": (
@@ -125,17 +121,24 @@ class PromptPostProcessorComfyUINode:
{ {
"default": "", "default": "",
"tooltip": "Comma separated list of wildcards folders", "tooltip": "Comma separated list of wildcards folders",
"defaultInput": False, "dynamicPrompts": False,
"forceInput": False, },
),
"wc_wildcards_input": (
"STRING",
{
"default": "",
"multiline": True,
"placeholder": "wildcards definitions",
"tooltip": "Wildcards definitions in yaml/json format",
"dynamicPrompts": False,
}, },
), ),
"wc_if_wildcards": ( "wc_if_wildcards": (
[e.value for e in PromptPostProcessor.IFWILDCARDS_CHOICES], [e.value for e in PromptPostProcessor.IFWILDCARDS_CHOICES],
{ {
"default": PromptPostProcessor.IFWILDCARDS_CHOICES.ignore.value, "default": PromptPostProcessor.IFWILDCARDS_CHOICES.stop.value,
"tooltip": "How to handle invalid wildcards in the prompt", "tooltip": "How to handle invalid wildcards in the prompt",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"wc_choice_separator": ( "wc_choice_separator": (
@@ -143,8 +146,7 @@ class PromptPostProcessorComfyUINode:
{ {
"default": PromptPostProcessor.DEFAULT_CHOICE_SEPARATOR, "default": PromptPostProcessor.DEFAULT_CHOICE_SEPARATOR,
"tooltip": "Default separator for selected choices", "tooltip": "Default separator for selected choices",
"defaultInput": False, "dynamicPrompts": False,
"forceInput": False,
}, },
), ),
"wc_keep_choices_order": ( "wc_keep_choices_order": (
@@ -154,8 +156,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Keep the order of the choices in the prompt", "tooltip": "Keep the order of the choices in the prompt",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"stn_separator": ( "stn_separator": (
@@ -163,8 +163,7 @@ class PromptPostProcessorComfyUINode:
{ {
"default": PromptPostProcessor.DEFAULT_STN_SEPARATOR, "default": PromptPostProcessor.DEFAULT_STN_SEPARATOR,
"tooltip": "Separator for the content added to the negative prompt", "tooltip": "Separator for the content added to the negative prompt",
"defaultInput": False, "dynamicPrompts": False,
"forceInput": False,
}, },
), ),
"stn_ignore_repeats": ( "stn_ignore_repeats": (
@@ -174,8 +173,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Ignore repeated content added to the negative prompt", "tooltip": "Ignore repeated content added to the negative prompt",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"cleanup_extra_spaces": ( "cleanup_extra_spaces": (
@@ -185,8 +182,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Remove extra spaces", "tooltip": "Remove extra spaces",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"cleanup_empty_constructs": ( "cleanup_empty_constructs": (
@@ -196,8 +191,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Remove empty constructs", "tooltip": "Remove empty constructs",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"cleanup_extra_separators": ( "cleanup_extra_separators": (
@@ -207,8 +200,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Remove extra separators", "tooltip": "Remove extra separators",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"cleanup_extra_separators2": ( "cleanup_extra_separators2": (
@@ -218,8 +209,15 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Remove extra separators (additional cases)", "tooltip": "Remove extra separators (additional cases)",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False, },
"forceInput": False, ),
"cleanup_extra_separators_include_eol": (
"BOOLEAN",
{
"default": False,
"tooltip": "Extra separators options also remove EOLs",
"label_on": "Yes",
"label_off": "No",
}, },
), ),
"cleanup_breaks": ( "cleanup_breaks": (
@@ -229,8 +227,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Cleanup around BREAKs", "tooltip": "Cleanup around BREAKs",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"cleanup_breaks_eol": ( "cleanup_breaks_eol": (
@@ -240,8 +236,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Set BREAKs in their own line", "tooltip": "Set BREAKs in their own line",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"cleanup_ands": ( "cleanup_ands": (
@@ -251,8 +245,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Cleanup around ANDs", "tooltip": "Cleanup around ANDs",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"cleanup_ands_eol": ( "cleanup_ands_eol": (
@@ -262,8 +254,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Set ANDs in their own line", "tooltip": "Set ANDs in their own line",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"cleanup_extranetwork_tags": ( "cleanup_extranetwork_tags": (
@@ -273,8 +263,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Clean up around extra network tags", "tooltip": "Clean up around extra network tags",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"cleanup_merge_attention": ( "cleanup_merge_attention": (
@@ -284,8 +272,6 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Merge nested attention constructs", "tooltip": "Merge nested attention constructs",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False,
"forceInput": False,
}, },
), ),
"remove_extranetwork_tags": ( "remove_extranetwork_tags": (
@@ -295,8 +281,24 @@ class PromptPostProcessorComfyUINode:
"tooltip": "Remove extra network tags", "tooltip": "Remove extra network tags",
"label_on": "Yes", "label_on": "Yes",
"label_off": "No", "label_off": "No",
"defaultInput": False, },
"forceInput": False, ),
"en_mappings_folders": (
"STRING",
{
"default": "",
"tooltip": "Comma separated list of extranetwork mappings folders",
"dynamicPrompts": False,
},
),
"en_mappings_input": (
"STRING",
{
"default": "",
"multiline": True,
"placeholder": "extranetwork mappings definitions",
"tooltip": "Extranetwork mappings definitions in yaml format",
"dynamicPrompts": False,
}, },
), ),
}, },
@@ -339,9 +341,11 @@ class PromptPostProcessorComfyUINode:
neg_prompt, neg_prompt,
seed, seed,
debug_level, # pylint: disable=unused-argument debug_level, # pylint: disable=unused-argument
on_warnings,
variants_definitions, variants_definitions,
wc_process_wildcards, wc_process_wildcards,
wc_wildcards_folders, wc_wildcards_folders,
wc_wildcards_input,
wc_if_wildcards, wc_if_wildcards,
wc_choice_separator, wc_choice_separator,
wc_keep_choices_order, wc_keep_choices_order,
@@ -351,6 +355,7 @@ class PromptPostProcessorComfyUINode:
cleanup_empty_constructs, cleanup_empty_constructs,
cleanup_extra_separators, cleanup_extra_separators,
cleanup_extra_separators2, cleanup_extra_separators2,
cleanup_extra_separators_include_eol,
cleanup_breaks, cleanup_breaks,
cleanup_breaks_eol, cleanup_breaks_eol,
cleanup_ands, cleanup_ands,
@@ -358,6 +363,8 @@ class PromptPostProcessorComfyUINode:
cleanup_extranetwork_tags, cleanup_extranetwork_tags,
cleanup_merge_attention, cleanup_merge_attention,
remove_extranetwork_tags, remove_extranetwork_tags,
en_mappings_folders,
en_mappings_input,
): ):
if wc_process_wildcards: if wc_process_wildcards:
return float( return float(
@@ -369,9 +376,11 @@ class PromptPostProcessorComfyUINode:
"pos_prompt": pos_prompt, "pos_prompt": pos_prompt,
"neg_prompt": neg_prompt, "neg_prompt": neg_prompt,
"seed": seed, "seed": seed,
"on_warnings": on_warnings,
"variants_definitions": variants_definitions, "variants_definitions": variants_definitions,
"process_wildcards": wc_process_wildcards, "process_wildcards": wc_process_wildcards,
"wildcards_folders": wc_wildcards_folders, "wildcards_folders": wc_wildcards_folders,
"wildcards_input": wc_wildcards_input,
"if_wildcards": wc_if_wildcards, "if_wildcards": wc_if_wildcards,
"choice_separator": wc_choice_separator, "choice_separator": wc_choice_separator,
"keep_choices_order": wc_keep_choices_order, "keep_choices_order": wc_keep_choices_order,
@@ -381,6 +390,7 @@ class PromptPostProcessorComfyUINode:
"cleanup_empty_constructs": cleanup_empty_constructs, "cleanup_empty_constructs": cleanup_empty_constructs,
"cleanup_extra_separators": cleanup_extra_separators, "cleanup_extra_separators": cleanup_extra_separators,
"cleanup_extra_separators2": cleanup_extra_separators2, "cleanup_extra_separators2": cleanup_extra_separators2,
"cleanup_extra_separators_include_eol": cleanup_extra_separators_include_eol,
"cleanup_breaks": cleanup_breaks, "cleanup_breaks": cleanup_breaks,
"cleanup_breaks_eol": cleanup_breaks_eol, "cleanup_breaks_eol": cleanup_breaks_eol,
"cleanup_ands": cleanup_ands, "cleanup_ands": cleanup_ands,
@@ -388,6 +398,8 @@ class PromptPostProcessorComfyUINode:
"cleanup_extranetwork_tags": cleanup_extranetwork_tags, "cleanup_extranetwork_tags": cleanup_extranetwork_tags,
"cleanup_merge_attention": cleanup_merge_attention, "cleanup_merge_attention": cleanup_merge_attention,
"remove_extranetwork_tags": remove_extranetwork_tags, "remove_extranetwork_tags": remove_extranetwork_tags,
"en_mappings_folders": en_mappings_folders,
"en_mappings_input": en_mappings_input,
} }
return new_run.__hash__ return new_run.__hash__
# return float("NaN") # return float("NaN")
@@ -400,9 +412,11 @@ class PromptPostProcessorComfyUINode:
neg_prompt, neg_prompt,
seed, seed,
debug_level, debug_level,
on_warnings,
variants_definitions, variants_definitions,
wc_process_wildcards, wc_process_wildcards,
wc_wildcards_folders, wc_wildcards_folders,
wc_wildcards_input,
wc_if_wildcards, wc_if_wildcards,
wc_choice_separator, wc_choice_separator,
wc_keep_choices_order, wc_keep_choices_order,
@@ -412,6 +426,7 @@ class PromptPostProcessorComfyUINode:
cleanup_empty_constructs, cleanup_empty_constructs,
cleanup_extra_separators, cleanup_extra_separators,
cleanup_extra_separators2, cleanup_extra_separators2,
cleanup_extra_separators_include_eol,
cleanup_breaks, cleanup_breaks,
cleanup_breaks_eol, cleanup_breaks_eol,
cleanup_ands, cleanup_ands,
@@ -419,6 +434,8 @@ class PromptPostProcessorComfyUINode:
cleanup_extranetwork_tags, cleanup_extranetwork_tags,
cleanup_merge_attention, cleanup_merge_attention,
remove_extranetwork_tags, remove_extranetwork_tags,
en_mappings_folders,
en_mappings_input,
): ):
modelclass = ( modelclass = (
model.model.model_config.__class__.__name__ if model is not None and not isinstance(model, str) else model model.model.model_config.__class__.__name__ if model is not None and not isinstance(model, str) else model
@@ -427,13 +444,14 @@ class PromptPostProcessorComfyUINode:
self.logger.warning("Model class is not provided. System variables might not be properly set.") self.logger.warning("Model class is not provided. System variables might not be properly set.")
if modelname == "": if modelname == "":
self.logger.warning("Modelname is not provided. System variables will not be properly set.") self.logger.warning("Modelname is not provided. System variables will not be properly set.")
# model class values in ComfyUI\comfy\supported_models.py
env_info = { env_info = {
"app": "comfyui", "app": SUPPORTED_APPS.comfyui.value,
"models_path": folder_paths.models_dir, "models_path": folder_paths.models_dir,
"model_filename": modelname or "", # path is relative to checkpoints folder "model_filename": modelname or "", # path is relative to checkpoints folder
"model_class": modelclass, "model_class": modelclass,
"is_sd1": modelclass in ("SD15", "SD15_instructpix2pix"), "is_sd1": modelclass in ("SD15", "SD15_instructpix2pix"),
"is_sd2": modelclass in ("SD20", "SD21UnclipL", "SD21UnclipH"), "is_sd2": modelclass in ("SD20", "SD21UnclipL", "SD21UnclipH", "LotusD"),
"is_sdxl": ( "is_sdxl": (
modelclass in ("SDXL", "SDXLRefiner", "SDXL_instructpix2pix", "Segmind_Vega", "KOALA_700M", "KOALA_1B") modelclass in ("SDXL", "SDXLRefiner", "SDXL_instructpix2pix", "Segmind_Vega", "KOALA_700M", "KOALA_1B")
), ),
@@ -441,12 +459,29 @@ class PromptPostProcessorComfyUINode:
"is_sd3": modelclass in ("SD3",), "is_sd3": modelclass in ("SD3",),
"is_flux": modelclass in ("Flux", "FluxInpaint", "FluxSchnell"), "is_flux": modelclass in ("Flux", "FluxInpaint", "FluxSchnell"),
"is_auraflow": modelclass in ("AuraFlow",), "is_auraflow": modelclass in ("AuraFlow",),
"is_pixart": modelclass in ("PixArtAlpha", "PixArtSigma"),
"is_lumina2": modelclass in ("Lumina2",),
"is_ltxv": modelclass in ("LTXV",),
"is_cosmos": modelclass in ("CosmosT2V", "CosmosI2V"),
"is_genmomochi": modelclass in ("GenmoMochi",),
"is_hunyuan": modelclass in ("HunyuanDiT", "HunyuanDiT1"),
"is_hunyuanvideo": modelclass in ("HunyuanVideo", "HunyuanVideoI2V", "HunyuanVideoSkyreelsI2V"),
"is_hunyuan3d": modelclass in ("Hunyuan3Dv2", "Hunyuan3Dv2mini"),
"is_wanvideo": modelclass in ("WAN21_T2V", "WAN21_I2V", "WAN21_FunControl2V"),
"is_hidream": modelclass in ("HiDream",),
} }
# Also supported: SVD_img2vid, SVD3D_u, SVD3_p, Stable_Zero123, SD_X4Upscaler, # Also supported: SVD_img2vid, SVD3D_u, SVD3_p, Stable_Zero123, SD_X4Upscaler, Stable_Cascade_C, Stable_Cascade_B, StableAudio
# Stable_Cascade_C, Stable_Cascade_B, StableAudio, HunyuanDiT, HunyuanDiT1, GenmoMochi, LTXV
if wc_wildcards_folders == "": if wc_wildcards_folders == "":
wc_wildcards_folders = ",".join(folder_paths.get_folder_paths("wildcards") or []) try:
fp1 = folder_paths.get_folder_paths("ppp_wildcards")
except Exception: # pylint: disable=W0718
fp1 = None
try:
fp2 = folder_paths.get_folder_paths("wildcards")
except Exception: # pylint: disable=W0718
fp2 = None
wc_wildcards_folders = ",".join(fp1 or fp2 or [])
if wc_wildcards_folders == "": if wc_wildcards_folders == "":
wc_wildcards_folders = os.getenv("WILDCARD_DIR", PPPWildcards.DEFAULT_WILDCARDS_FOLDER) wc_wildcards_folders = os.getenv("WILDCARD_DIR", PPPWildcards.DEFAULT_WILDCARDS_FOLDER)
wildcards_folders = [ wildcards_folders = [
@@ -454,10 +489,27 @@ class PromptPostProcessorComfyUINode:
for f in wc_wildcards_folders.split(",") for f in wc_wildcards_folders.split(",")
if f.strip() != "" if f.strip() != ""
] ]
if en_mappings_folders == "":
try:
fp3 = folder_paths.get_folder_paths("ppp_extranetworkmappings")
except Exception: # pylint: disable=W0718
fp3 = None
en_mappings_folders = ",".join(fp3 or [])
if en_mappings_folders == "":
en_mappings_folders = os.getenv(
"EXTRANETWORKMAPPINGS_DIR", PPPExtraNetworkMappings.DEFAULT_ENMAPPINGS_FOLDER
)
enmappings_folders = [
(f if os.path.isabs(f) else os.path.abspath(os.path.join(folder_paths.models_dir, f)))
for f in en_mappings_folders.split(",")
if f.strip() != ""
]
if variants_definitions != "" and not "=" in variants_definitions: # mainly to warn about the old format if variants_definitions != "" and not "=" in variants_definitions: # mainly to warn about the old format
raise ValueError("Invalid variants_definitions format") raise ValueError("Invalid variants_definitions format")
options = { options = {
"debug_level": debug_level, "debug_level": debug_level,
"on_warnings": on_warnings,
"variants_definitions": variants_definitions, "variants_definitions": variants_definitions,
"process_wildcards": wc_process_wildcards, "process_wildcards": wc_process_wildcards,
"if_wildcards": wc_if_wildcards, "if_wildcards": wc_if_wildcards,
@@ -469,6 +521,7 @@ class PromptPostProcessorComfyUINode:
"cleanup_empty_constructs": cleanup_empty_constructs, "cleanup_empty_constructs": cleanup_empty_constructs,
"cleanup_extra_separators": cleanup_extra_separators, "cleanup_extra_separators": cleanup_extra_separators,
"cleanup_extra_separators2": cleanup_extra_separators2, "cleanup_extra_separators2": cleanup_extra_separators2,
"cleanup_extra_separators_include_eol": cleanup_extra_separators_include_eol,
"cleanup_breaks": cleanup_breaks, "cleanup_breaks": cleanup_breaks,
"cleanup_breaks_eol": cleanup_breaks_eol, "cleanup_breaks_eol": cleanup_breaks_eol,
"cleanup_ands": cleanup_ands, "cleanup_ands": cleanup_ands,
@@ -477,9 +530,24 @@ class PromptPostProcessorComfyUINode:
"cleanup_merge_attention": cleanup_merge_attention, "cleanup_merge_attention": cleanup_merge_attention,
"remove_extranetwork_tags": remove_extranetwork_tags, "remove_extranetwork_tags": remove_extranetwork_tags,
} }
self.wildcards_obj.refresh_wildcards(debug_level, wildcards_folders if options["process_wildcards"] else None) self.wildcards_obj.refresh_wildcards(
debug_level,
wildcards_folders if options["process_wildcards"] else None,
wc_wildcards_input,
)
self.extranetwork_mappings_obj.refresh_extranetwork_mappings(
debug_level,
enmappings_folders,
en_mappings_input,
)
ppp = PromptPostProcessor( ppp = PromptPostProcessor(
self.logger, self.interrupt, env_info, options, self.grammar_content, self.wildcards_obj self.logger,
self.interrupt,
env_info,
options,
self.grammar_content,
self.wildcards_obj,
self.extranetwork_mappings_obj,
) )
pos_prompt, neg_prompt, variables = ppp.process_prompt(pos_prompt, neg_prompt, seed if seed is not None else 1) pos_prompt, neg_prompt, variables = ppp.process_prompt(pos_prompt, neg_prompt, seed if seed is not None else 1)
return ( return (
@@ -519,8 +587,6 @@ class PromptPostProcessorSelectVariableComfyUINode:
"multiline": False, "multiline": False,
"default": "", "default": "",
"dynamicPrompts": False, "dynamicPrompts": False,
"defaultInput": False,
"forceInput": False,
}, },
), ),
}, },
+269
View File
@@ -0,0 +1,269 @@
import os
from typing import Optional
import logging
import yaml
from ppp_logging import DEBUG_LEVEL # pylint: disable=import-error
from ppp_utils import deep_freeze # pylint: disable=import-error
class PPPENMappingVariant:
"""
A class to represent a variant of an extra network mapping.
Attributes:
condition (str): The condition for the variant.
name (str): The name of the variant.
parameters (float|str): The parameters for the variant.
triggers (list[str]): The triggers for the variant.
weight (float): The weight for the variant when multiple variants apply.
"""
def __init__(self, condition: str, name: str, parameters: float | str, triggers: list[str], weight: float):
self.condition: str = condition
self.name: str = name
self.parameters: float | str = parameters
self.triggers: list[str] = triggers
self.weight: float = weight
class PPPENMapping:
"""
A extra network mapping object.
Attributes:
kind (str): The kind of the extra network.
name (str): The name of the extra network mapping.
file (str): The path to the file where the extranetwork mapping is defined.
variants (list[PPPENMappingVariant]): The processed variants of the extranetwork mapping.
"""
def __init__(self, fullpath: str, kind: str, name: str, variants: list[dict]):
self.file: str = fullpath
self.kind: str = kind
self.name: str = name
self.variants: list[PPPENMappingVariant] = [
PPPENMappingVariant(
**{**{"condition": None, "name": None, "parameters": None, "triggers": None, "weight": 1.0}, **v}
)
for v in variants
]
def __hash__(self) -> int:
t = (self.kind, self.name, deep_freeze(self.variants))
return hash(t)
def __sizeof__(self):
return self.kind.__sizeof__() + self.name.__sizeof__() + self.file.__sizeof__() + self.variants.__sizeof__()
class PPPExtraNetworkMappings:
"""
A class to manage extra network mappings.
Attributes:
extranetwork_maps (dict[str, PPPENMapping]): The extra network mappings.
"""
DEFAULT_ENMAPPINGS_FOLDER = "extranetworkmappings"
LOCALINPUT_FILENAME = "#INPUT"
def __init__(self, logger):
self.__logger: logging.Logger = logger
self.__debug_level = DEBUG_LEVEL.none
self.__enmappings_folders = []
self.__enmappings_files = {}
self.extranetwork_mappings: dict[str, PPPENMapping] = {}
def __hash__(self) -> int:
return hash(deep_freeze(self.extranetwork_mappings))
def __sizeof__(self):
return (
self.extranetwork_mappings.__sizeof__()
+ self.__enmappings_folders.__sizeof__()
+ self.__enmappings_files.__sizeof__()
)
def refresh_extranetwork_mappings(
self, debug_level: DEBUG_LEVEL, enmappings_folders: Optional[list[str]], enmappings_input: str = None
):
"""
Initialize the extra network mappings.
"""
self.__debug_level = debug_level
self.__enmappings_folders = enmappings_folders or []
# if self.__debug_level != DEBUG_LEVEL.none:
# self.__logger.info("Refreshing extra network mappings...")
# t1 = time.monotonic_ns()
for fullpath in list(self.__enmappings_files.keys()):
if fullpath != self.LOCALINPUT_FILENAME:
path = os.path.dirname(fullpath)
if not os.path.exists(fullpath) or not any(
os.path.commonpath([path, folder]) == folder for folder in self.__enmappings_folders
):
self.__remove_extranetwork_mappings_from_path(fullpath)
elif enmappings_input is None:
self.__remove_extranetwork_mappings_from_path(fullpath)
if enmappings_folders is not None or enmappings_input is not None:
if enmappings_folders is not None:
for f in self.__enmappings_folders:
self.__get_extranetwork_mappings_in_directory(f)
if enmappings_input is not None:
self.__get_extranetwork_mappings_in_input(enmappings_input)
else:
self.extranetwork_mappings = {}
self.__enmappings_files = {}
# t2 = time.monotonic_ns()
# if self.__debug_level != DEBUG_LEVEL.none:
# self.__logger.info(f"Extra network mappings refresh time: {(t2 - t1) / 1_000_000_000:.3f} seconds")
# def get_extranetwork_mappings(self, key: str) -> list[PPPENMapping]:
# """
# Get all extra network mappings that match a key.
#
# Args:
# key (str): The key to match (kind:name).
#
# Returns:
# list: A list of all extra network mappings that match the key.
# """
# keys = sorted(fnmatch.filter(self.extranetwork_mappings.keys(), key))
# return [self.extranetwork_mappings[k] for k in keys]
def __remove_extranetwork_mappings_from_path(self, full_path: str, debug=True):
"""
Clear all extra network mappings in a file.
Args:
full_path (str): The path to the file.
debug (bool): Whether to print debug messages or not.
"""
last_modified_cached = self.__enmappings_files.get(full_path, None) # a time or a hash
if debug and last_modified_cached is not None and self.__debug_level != DEBUG_LEVEL.none:
if full_path == self.LOCALINPUT_FILENAME:
self.__logger.debug("Removing extra network mappings from input")
else:
self.__logger.debug(f"Removing extra network mappings from file: {full_path}")
if full_path in self.__enmappings_files.keys():
del self.__enmappings_files[full_path]
for key in list(self.extranetwork_mappings.keys()):
if self.extranetwork_mappings[key].file == full_path:
del self.extranetwork_mappings[key]
def __get_extranetwork_mappings_in_file(self, full_path: str):
"""
Get all extra network mappings in a file.
Args:
full_path (str): The path to the file.
"""
last_modified = os.path.getmtime(full_path)
last_modified_cached = self.__enmappings_files.get(full_path, None)
if last_modified_cached is not None and last_modified == self.__enmappings_files[full_path]:
return
filename = os.path.basename(full_path)
_, extension = os.path.splitext(filename)
if extension not in (".yaml", ".yml", ".json"):
return
self.__remove_extranetwork_mappings_from_path(full_path, False)
if last_modified_cached is not None and self.__debug_level != DEBUG_LEVEL.none:
self.__logger.debug(f"Updating extra network mappings from file: {full_path}")
self.__get_extranetwork_mappings_in_structured_file(full_path)
self.__enmappings_files[full_path] = last_modified
def __get_extranetwork_mappings_in_input(self, enmappings_input: str):
"""
Get all extra network mappings in the string.
Args:
enmappings_input (str): The input string containing extra network mappings in yaml format.
"""
new_h = hash(enmappings_input)
h = self.__enmappings_files.get(self.LOCALINPUT_FILENAME, None)
if h == new_h:
return
self.__remove_extranetwork_mappings_from_path(self.LOCALINPUT_FILENAME, False)
if h is not None and self.__debug_level != DEBUG_LEVEL.none:
self.__logger.debug("Updating extra network mappings from input")
enmappings_input = enmappings_input.strip()
if enmappings_input != "":
try:
content = yaml.safe_load(enmappings_input)
except yaml.YAMLError as e:
self.__logger.warning(f"Invalid format for input extra network mappings: {e}")
return
if content is not None:
self.__add_extranetwork_mapping(content, self.LOCALINPUT_FILENAME)
self.__enmappings_files[self.LOCALINPUT_FILENAME] = new_h
def __add_extranetwork_mapping(self, content: dict[str, dict[str, list[dict]]], full_path: str):
"""
Add an extra network mapping to the extra network mappings dictionary.
Args:
content (object): The content of the extra network mapping.
full_path (str): The path to the file that contains it.
"""
if not isinstance(content, dict):
self.__logger.warning(f"Invalid extra network mapping in file '{full_path}'!")
return
for kind, maps in content.items():
if not isinstance(maps, dict):
self.__logger.warning(f"Invalid extra network mapping definition for '{kind}:*' in file '{full_path}'!")
else:
for name, variants in maps.items():
key = f"{kind}:{name}"
if not isinstance(variants, list):
self.__logger.warning(
f"Invalid extra network mapping definition for '{key}' in file '{full_path}'!"
)
elif self.extranetwork_mappings.get(key, None) is not None:
self.__logger.warning(
f"Duplicate extra network mapping '{key}' in file '{full_path}' and '{self.extranetwork_mappings[key].file}'!"
)
elif not isinstance(variants, list) or not all(isinstance(v, dict) for v in variants):
self.__logger.warning(
f"Invalid extra network mapping definition for '{key}' in file '{full_path}'!"
)
else:
self.extranetwork_mappings[key] = PPPENMapping(full_path, kind, name, variants)
def __get_extranetwork_mappings_in_structured_file(self, full_path):
"""
Get all extra network mappings in a structured file.
Args:
full_path (str): The path to the file.
base (str): The base path for the extra network mappings.
"""
try:
try:
with open(full_path, "r", encoding="utf-8") as file:
content = yaml.safe_load(file)
except: # pylint: disable=bare-except
self.__logger.warning(f"Could not read file '{full_path}' with utf-8 encoding, trying windows-1252...")
with open(full_path, "r", encoding="windows-1252") as file:
content = yaml.safe_load(file)
self.__add_extranetwork_mapping(content, full_path)
except Exception as e: # pylint: disable=broad-except
self.__logger.error(f"Error reading extra network mappings from file '{full_path}': {e}")
def __get_extranetwork_mappings_in_directory(self, directory: str):
"""
Get all extra network mappings in a directory.
Args:
directory (str): The path to the directory.
"""
if not os.path.exists(directory):
self.__logger.warning(f"Extra network mappings directory '{directory}' does not exist!")
return
for filename in os.listdir(directory):
full_path = os.path.abspath(os.path.join(directory, filename))
if os.path.basename(full_path).startswith("."):
continue
if os.path.isdir(full_path):
self.__get_extranetwork_mappings_in_directory(full_path)
elif os.path.isfile(full_path):
self.__get_extranetwork_mappings_in_file(full_path)
+17
View File
@@ -0,0 +1,17 @@
from enum import Enum
class SUPPORTED_APPS(Enum):
comfyui = "comfyui"
a1111 = "a1111"
forge = "forge"
reforge = "reforge"
sdnext = "sdnext"
SUPPORTED_APPS_NAMES = {
SUPPORTED_APPS.comfyui: "ComfyUI",
SUPPORTED_APPS.sdnext: "SD.Next",
SUPPORTED_APPS.forge: "Forge",
SUPPORTED_APPS.reforge: "reForge",
SUPPORTED_APPS.a1111: "A1111 (or compatible)",
}
+4 -3
View File
@@ -2,6 +2,7 @@ from enum import Enum
import logging import logging
import sys import sys
import copy import copy
from ppp_hosts import SUPPORTED_APPS # pylint: disable=import-error
class DEBUG_LEVEL(Enum): class DEBUG_LEVEL(Enum):
@@ -52,14 +53,14 @@ class PromptPostProcessorLogFactory: # pylint: disable=too-few-public-methods
colored_record.levelname = f"{seq}{levelname:8s}{self.COLORS['RESET']}" colored_record.levelname = f"{seq}{levelname:8s}{self.COLORS['RESET']}"
return super().format(colored_record) return super().format(colored_record)
def __init__(self): def __init__(self, app: SUPPORTED_APPS = None): # pylint: disable=unused-argument
""" """
Initializes the PromptPostProcessor class. Initializes the PromptPostProcessor class.
This method sets up the logger for the PromptPostProcessor class and configures its log level and handlers. This method sets up the logger for the PromptPostProcessor class and configures its log level and handlers.
Args: Args:
None app (SUPPORTED_APPS): The application for which the logger is being created.
Returns: Returns:
None None
@@ -68,7 +69,7 @@ class PromptPostProcessorLogFactory: # pylint: disable=too-few-public-methods
ppplog.propagate = False ppplog.propagate = False
if not ppplog.handlers: if not ppplog.handlers:
handler = logging.StreamHandler(sys.stdout) handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(self.ColoredFormatter("%(asctime)s %(levelname)s %(message)s")) # Used in A1111 / Forge / reForge / ComfyUI, but not in SD.Next handler.setFormatter(self.ColoredFormatter("%(asctime)s %(levelname)s %(message)s"))
ppplog.addHandler(handler) ppplog.addHandler(handler)
ppplog.setLevel(logging.DEBUG) ppplog.setLevel(logging.DEBUG)
self.log = PromptPostProcessorLogCustomAdapter(ppplog) self.log = PromptPostProcessorLogCustomAdapter(ppplog)
+17
View File
@@ -0,0 +1,17 @@
def deep_freeze(obj):
"""
Deep freeze an object.
Args:
obj (object): The object to freeze.
Returns:
object: The frozen object.
"""
if isinstance(obj, dict):
return tuple((k, deep_freeze(v)) for k, v in sorted(obj.items()))
if isinstance(obj, list):
return tuple(deep_freeze(i) for i in obj)
if isinstance(obj, set):
return tuple(deep_freeze(i) for i in sorted(obj))
return obj
+92 -63
View File
@@ -1,31 +1,11 @@
import fnmatch import fnmatch
import os import os
import json
from typing import Optional from typing import Optional
import logging import logging
import yaml import yaml
from ppp_logging import DEBUG_LEVEL from ppp_logging import DEBUG_LEVEL # pylint: disable=import-error
from ppp_utils import deep_freeze # pylint: disable=import-error
def deep_freeze(obj):
"""
Deep freeze an object.
Args:
obj (object): The object to freeze.
Returns:
object: The frozen object.
"""
if isinstance(obj, dict):
return tuple((k, deep_freeze(v)) for k, v in sorted(obj.items()))
elif isinstance(obj, list):
return tuple(deep_freeze(i) for i in obj)
elif isinstance(obj, set):
return tuple(deep_freeze(i) for i in sorted(obj))
else:
return obj
class PPPWildcard: class PPPWildcard:
@@ -70,6 +50,7 @@ class PPPWildcards:
""" """
DEFAULT_WILDCARDS_FOLDER = "wildcards" DEFAULT_WILDCARDS_FOLDER = "wildcards"
LOCALINPUT_FILENAME = "#INPUT"
def __init__(self, logger): def __init__(self, logger):
self.__logger: logging.Logger = logger self.__logger: logging.Logger = logger
@@ -84,31 +65,38 @@ class PPPWildcards:
def __sizeof__(self): def __sizeof__(self):
return self.wildcards.__sizeof__() + self.__wildcards_folders.__sizeof__() + self.__wildcard_files.__sizeof__() return self.wildcards.__sizeof__() + self.__wildcards_folders.__sizeof__() + self.__wildcard_files.__sizeof__()
def refresh_wildcards(self, debug_level: DEBUG_LEVEL, wildcards_folders: Optional[list[str]]): def refresh_wildcards(
self, debug_level: DEBUG_LEVEL, wildcards_folders: Optional[list[str]], wildcards_input: str = None
):
""" """
Initialize the wildcards. Initialize the wildcards.
""" """
self.__debug_level = debug_level self.__debug_level = debug_level
self.__wildcards_folders = wildcards_folders self.__wildcards_folders = wildcards_folders or []
if wildcards_folders is not None: # if self.__debug_level != DEBUG_LEVEL.none:
# if self.debug_level != DEBUG_LEVEL.none: # self.__logger.info("Refreshing wildcards...")
# self.logger.info("Initializing wildcards...") # t1 = time.monotonic_ns()
# t1 = time.time() for fullpath in list(self.__wildcard_files.keys()):
for fullpath in list(self.__wildcard_files.keys()): if fullpath != self.LOCALINPUT_FILENAME:
path = os.path.dirname(fullpath) path = os.path.dirname(fullpath)
if not os.path.exists(fullpath) or not any( if not os.path.exists(fullpath) or not any(
os.path.commonpath([path, folder]) == folder for folder in self.__wildcards_folders os.path.commonpath([path, folder]) == folder for folder in self.__wildcards_folders
): ):
self.__remove_wildcards_from_file(fullpath) self.__remove_wildcards_from_path(fullpath)
for f in self.__wildcards_folders: elif wildcards_input is None:
self.__get_wildcards_in_directory(f, f) self.__remove_wildcards_from_path(fullpath)
# t2 = time.time() if wildcards_folders is not None or wildcards_input is not None:
# if self.debug_level != DEBUG_LEVEL.none: if wildcards_folders is not None:
# self.logger.info(f"Wildcards init time: {t2 - t1:.3f} seconds") for f in self.__wildcards_folders:
self.__get_wildcards_in_directory(f, f)
if wildcards_input is not None:
self.__get_wildcards_in_input(wildcards_input)
else: else:
self.__wildcards_folders = []
self.wildcards = {} self.wildcards = {}
self.__wildcard_files = {} self.__wildcard_files = {}
# t2 = time.monotonic_ns()
# if self.__debug_level != DEBUG_LEVEL.none:
# self.__logger.info(f"Wildcards refresh time: {(t2 - t1) / 1_000_000_000:.3f} seconds")
def get_wildcards(self, key: str) -> list[PPPWildcard]: def get_wildcards(self, key: str) -> list[PPPWildcard]:
""" """
@@ -161,7 +149,7 @@ class PPPWildcards:
return None return None
return current_dict return current_dict
def __remove_wildcards_from_file(self, full_path: str, debug=True): def __remove_wildcards_from_path(self, full_path: str, debug=True):
""" """
Clear all wildcards in a file. Clear all wildcards in a file.
@@ -169,9 +157,12 @@ class PPPWildcards:
full_path (str): The path to the file. full_path (str): The path to the file.
debug (bool): Whether to print debug messages or not. debug (bool): Whether to print debug messages or not.
""" """
last_modified_cached = self.__wildcard_files.get(full_path, None) last_modified_cached = self.__wildcard_files.get(full_path, None) # a time or a hash
if debug and last_modified_cached is not None and self.__debug_level != DEBUG_LEVEL.none: if debug and last_modified_cached is not None and self.__debug_level != DEBUG_LEVEL.none:
self.__logger.debug(f"Removing wildcards from file: {full_path}") if full_path == self.LOCALINPUT_FILENAME:
self.__logger.debug("Removing from memory wildcards from input")
else:
self.__logger.debug(f"Removing from memory wildcards from file: {full_path}")
if full_path in self.__wildcard_files.keys(): if full_path in self.__wildcard_files.keys():
del self.__wildcard_files[full_path] del self.__wildcard_files[full_path]
for key in list(self.wildcards.keys()): for key in list(self.wildcards.keys()):
@@ -186,22 +177,53 @@ class PPPWildcards:
base (str): The base path for the wildcards. base (str): The base path for the wildcards.
full_path (str): The path to the file. full_path (str): The path to the file.
""" """
last_modified = os.path.getmtime(full_path) try:
last_modified_cached = self.__wildcard_files.get(full_path, None) last_modified = os.path.getmtime(full_path)
if last_modified_cached is not None and last_modified == self.__wildcard_files[full_path]: last_modified_cached = self.__wildcard_files.get(full_path, None)
return if last_modified_cached is not None and last_modified == self.__wildcard_files[full_path]:
filename = os.path.basename(full_path) return
_, extension = os.path.splitext(filename) filename = os.path.basename(full_path)
if extension not in (".txt", ".json", ".yaml", ".yml"): _, extension = os.path.splitext(filename)
return if extension not in (".txt", ".json", ".yaml", ".yml"):
self.__remove_wildcards_from_file(full_path, False) return
if last_modified_cached is not None and self.__debug_level != DEBUG_LEVEL.none: self.__remove_wildcards_from_path(full_path, False)
self.__logger.debug(f"Updating wildcards from file: {full_path}") if last_modified_cached is not None and self.__debug_level != DEBUG_LEVEL.none:
if extension == ".txt": self.__logger.debug(f"Updating wildcards from file: {full_path}")
self.__get_wildcards_in_text_file(full_path, base) if extension == ".txt":
elif extension in (".json", ".yaml", ".yml"): self.__get_wildcards_in_text_file(full_path, base)
self.__get_wildcards_in_structured_file(full_path, base) elif extension in (".json", ".yaml", ".yml"):
self.__wildcard_files[full_path] = last_modified self.__get_wildcards_in_structured_file(full_path, base)
self.__wildcard_files[full_path] = last_modified
except Exception as e: # pylint: disable=broad-except
self.__logger.error(f"Error reading wildcard file '{full_path}': {e}")
def __get_wildcards_in_input(self, wildcards_input: str):
"""
Get all wildcards in the string.
Args:
wildcards_input (str): The input string containing wildcards in json or yaml format.
"""
try:
new_h = hash(wildcards_input)
h = self.__wildcard_files.get(self.LOCALINPUT_FILENAME, None)
if h == new_h:
return
self.__remove_wildcards_from_path(self.LOCALINPUT_FILENAME, False)
if h is not None and self.__debug_level != DEBUG_LEVEL.none:
self.__logger.debug("Updating wildcards from input")
wildcards_input = wildcards_input.strip()
if wildcards_input != "":
try:
content = yaml.safe_load(wildcards_input)
except yaml.YAMLError as e:
self.__logger.warning(f"Invalid format for input wildcards: {e}")
return
if content is not None:
self.__add_wildcard(content, self.LOCALINPUT_FILENAME, [self.LOCALINPUT_FILENAME])
self.__wildcard_files[self.LOCALINPUT_FILENAME] = new_h
except Exception as e: # pylint: disable=broad-except
self.__logger.error(f"Error reading wildcards input: {e}")
def is_dict_choices_options(self, d: dict) -> bool: def is_dict_choices_options(self, d: dict) -> bool:
""" """
@@ -214,7 +236,8 @@ class PPPWildcards:
bool: Whether the dictionary is a valid choices options dictionary or not. bool: Whether the dictionary is a valid choices options dictionary or not.
""" """
return all( return all(
k in ["sampler", "repeating", "count", "from", "to", "prefix", "suffix", "separator"] for k in d.keys() k in ["sampler", "repeating", "optional", "count", "from", "to", "prefix", "suffix", "separator"]
for k in d.keys()
) )
def is_dict_choice_options(self, d: dict) -> bool: def is_dict_choice_options(self, d: dict) -> bool:
@@ -374,11 +397,12 @@ class PPPWildcards:
""" """
external_key: str = os.path.relpath(os.path.splitext(full_path)[0], base) external_key: str = os.path.relpath(os.path.splitext(full_path)[0], base)
external_key_parts = external_key.split(os.sep) external_key_parts = external_key.split(os.sep)
_, extension = os.path.splitext(full_path) try:
with open(full_path, "r", encoding="utf-8") as file: with open(full_path, "r", encoding="utf-8") as file:
if extension == ".json": content = yaml.safe_load(file)
content = json.loads(file.read()) except: # pylint: disable=bare-except
else: self.__logger.warning(f"Could not read file '{full_path}' with utf-8 encoding, trying windows-1252...")
with open(full_path, "r", encoding="windows-1252") as file:
content = yaml.safe_load(file) content = yaml.safe_load(file)
self.__add_wildcard(content, full_path, external_key_parts) self.__add_wildcard(content, full_path, external_key_parts)
@@ -392,8 +416,13 @@ class PPPWildcards:
""" """
external_key: str = os.path.relpath(os.path.splitext(full_path)[0], base) external_key: str = os.path.relpath(os.path.splitext(full_path)[0], base)
external_key_parts = external_key.split(os.sep) external_key_parts = external_key.split(os.sep)
with open(full_path, "r", encoding="utf-8") as file: try:
text_content = map(lambda x: x.strip("\n\r"), file.readlines()) with open(full_path, "r", encoding="utf-8") as file:
text_content = map(lambda x: x.strip("\n\r"), file.readlines())
except: # pylint: disable=bare-except
self.__logger.warning(f"Could not read file '{full_path}' with utf-8 encoding, trying windows-1252...")
with open(full_path, "r", encoding="windows-1252") as file:
text_content = map(lambda x: x.strip("\n\r"), file.readlines())
text_content = list(filter(lambda x: x.strip() != "" and not x.strip().startswith("#"), text_content)) text_content = list(filter(lambda x: x.strip() != "" and not x.strip().startswith("#"), text_content))
text_content = [x.split("#")[0].rstrip() if len(x.split("#")) > 1 else x for x in text_content] text_content = [x.split("#")[0].rstrip() if len(x.split("#")) > 1 else x for x in text_content]
self.__add_wildcard(text_content, full_path, external_key_parts) self.__add_wildcard(text_content, full_path, external_key_parts)
+1 -1
View File
@@ -1,7 +1,7 @@
[project] [project]
name = "sd-webui-prompt-postprocessor" name = "sd-webui-prompt-postprocessor"
description = "Stable Diffusion WebUI & ComfyUI extension to post-process the prompt, including sending content from the prompt to the negative prompt and wildcards." description = "Stable Diffusion WebUI & ComfyUI extension to post-process the prompt, including sending content from the prompt to the negative prompt and wildcards."
version = "2.9.0" version = "2.12.1"
license = { file = "LICENSE.txt" } license = { file = "LICENSE.txt" }
dependencies = ["lark", "numpy", "pyyaml"] dependencies = ["lark", "numpy", "pyyaml"]
+200 -91
View File
@@ -4,20 +4,22 @@ if __name__ == "__main__":
import sys import sys
import os import os
import time import time
from pathlib import Path
import numpy as np import numpy as np
sys.path.append(os.path.join(sys.path[0], "..")) sys.path.append(str(Path(__file__).parent)) # base path for the extension
from modules import scripts, shared, script_callbacks # pylint: disable=import-error
from modules import scripts, shared, script_callbacks from modules.processing import StableDiffusionProcessing # pylint: disable=import-error
from modules.processing import StableDiffusionProcessing from modules.shared import opts # pylint: disable=import-error
from modules.shared import opts from modules.paths import models_path # pylint: disable=import-error
from modules.paths import models_path import gradio as gr # pylint: disable=import-error
import gradio as gr from ppp import PromptPostProcessor # pylint: disable=import-error
from ppp import PromptPostProcessor from ppp_hosts import SUPPORTED_APPS, SUPPORTED_APPS_NAMES # pylint: disable=import-error
from ppp_logging import DEBUG_LEVEL, PromptPostProcessorLogFactory from ppp_logging import DEBUG_LEVEL, PromptPostProcessorLogFactory # pylint: disable=import-error
from ppp_cache import PPPLRUCache from ppp_cache import PPPLRUCache # pylint: disable=import-error
from ppp_wildcards import PPPWildcards from ppp_wildcards import PPPWildcards # pylint: disable=import-error
from ppp_enmappings import PPPExtraNetworkMappings # pylint: disable=import-error
class PromptPostProcessorA1111Script(scripts.Script): class PromptPostProcessorA1111Script(scripts.Script):
@@ -42,6 +44,7 @@ class PromptPostProcessorA1111Script(scripts.Script):
@classmethod @classmethod
def increment_instance_count(cls): def increment_instance_count(cls):
cls.instance_count += 1 cls.instance_count += 1
return cls.instance_count
@classmethod @classmethod
def get_instance_count(cls): def get_instance_count(cls):
@@ -51,27 +54,22 @@ class PromptPostProcessorA1111Script(scripts.Script):
""" """
Initializes the PromptPostProcessor object. Initializes the PromptPostProcessor object.
This method adds callbacks for UI settings and initializes the logger.
Parameters: Parameters:
None None
Returns: Returns:
None None
""" """
self.increment_instance_count() self.instance_index = self.increment_instance_count()
lf = PromptPostProcessorLogFactory()
self.name = PromptPostProcessor.NAME self.name = PromptPostProcessor.NAME
self.ppp_logger = lf.log
self.ppp_debug_level = DEBUG_LEVEL(getattr(opts, "ppp_gen_debug_level", DEBUG_LEVEL.none.value))
self.lru_cache = PPPLRUCache(1000, logger=self.ppp_logger, debug_level=self.ppp_debug_level)
grammar_filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), "../grammar.lark") grammar_filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), "../grammar.lark")
with open(grammar_filename, "r", encoding="utf-8") as file: with open(grammar_filename, "r", encoding="utf-8") as file:
self.grammar_content = file.read() self.grammar_content = file.read()
self.wildcards_obj = PPPWildcards(lf.log) self.ppp_logger = None
i = self.get_instance_count() self.ppp_debug_level = DEBUG_LEVEL.none.value
if i == 1: # some UIs create multiple instances self.lru_cache = None
self.ppp_logger.info(f"{PromptPostProcessor.NAME} {PromptPostProcessor.VERSION} initialized") self.wildcards_obj = None
self.extranetwork_mappings_obj = None
def title(self): def title(self):
""" """
@@ -82,7 +80,7 @@ class PromptPostProcessorA1111Script(scripts.Script):
""" """
return PromptPostProcessor.NAME return PromptPostProcessor.NAME
def show(self, is_img2img): def show(self, is_img2img): # pylint: disable=unused-argument
""" """
Determines whether the script should be shown based on the kind of processing. Determines whether the script should be shown based on the kind of processing.
@@ -94,7 +92,7 @@ class PromptPostProcessorA1111Script(scripts.Script):
""" """
return scripts.AlwaysVisible return scripts.AlwaysVisible
def ui(self, is_img2img): def ui(self, is_img2img): # pylint: disable=unused-argument
with gr.Accordion(PromptPostProcessor.NAME, open=False): with gr.Accordion(PromptPostProcessor.NAME, open=False):
force_equal_seeds = gr.Checkbox( force_equal_seeds = gr.Checkbox(
label="Force equal seeds", label="Force equal seeds",
@@ -164,13 +162,37 @@ class PromptPostProcessorA1111Script(scripts.Script):
Returns: Returns:
None None
""" """
t1 = time.time() app = (
SUPPORTED_APPS.forge
if hasattr(p.sd_model, "model_config")
else (
SUPPORTED_APPS.reforge
if hasattr(p.sd_model, "forge_objects")
else (
SUPPORTED_APPS.sdnext
if hasattr(p.sd_model, "is_sdxl") and not hasattr(p.sd_model, "is_ssd")
else SUPPORTED_APPS.a1111
)
)
)
if self.ppp_logger is None:
lf = PromptPostProcessorLogFactory(app)
self.ppp_logger = lf.log
self.ppp_debug_level = DEBUG_LEVEL(getattr(opts, "ppp_gen_debug_level", DEBUG_LEVEL.none.value))
self.lru_cache = PPPLRUCache(1000, logger=self.ppp_logger, debug_level=self.ppp_debug_level)
self.wildcards_obj = PPPWildcards(self.ppp_logger)
self.extranetwork_mappings_obj = PPPExtraNetworkMappings(self.ppp_logger)
self.ppp_logger.info(
f"{PromptPostProcessor.NAME} {PromptPostProcessor.VERSION} initialized, running on {SUPPORTED_APPS_NAMES[app]}"
)
t1 = time.monotonic_ns()
if getattr(opts, "prompt_attention", "") == "Compel parser": if getattr(opts, "prompt_attention", "") == "Compel parser":
self.ppp_logger.warning("Compel parser is not supported!") self.ppp_logger.warning("Compel parser is not supported!")
init_images = getattr(p, "init_images", [None]) or [None] init_images = getattr(p, "init_images", [None]) or [None]
is_i2i = bool(init_images[0]) is_i2i = bool(init_images[0])
self.ppp_debug_level = DEBUG_LEVEL(getattr(opts, "ppp_gen_debug_level", DEBUG_LEVEL.none.value)) self.ppp_debug_level = DEBUG_LEVEL(getattr(opts, "ppp_gen_debug_level", DEBUG_LEVEL.none.value))
do_i2i = getattr(opts, "ppp_gen_doi2i", False) do_i2i = getattr(opts, "ppp_gen_doi2i", False)
add_prompts = getattr(opts, "ppp_gen_addpromptstometadata", True)
if is_i2i and not do_i2i: if is_i2i and not do_i2i:
if self.ppp_debug_level != DEBUG_LEVEL.none: if self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info("Not processing the prompt for i2i") self.ppp_logger.info("Not processing the prompt for i2i")
@@ -185,37 +207,22 @@ class PromptPostProcessorA1111Script(scripts.Script):
} }
) )
app_names = {
"sdnext": "SD.Next",
"forge": "Forge",
"reforge": "reForge",
"a1111": "A1111 (or compatible)",
}
app = (
"forge"
if hasattr(p.sd_model, "model_config")
else (
"reforge"
if hasattr(p.sd_model, "forge_objects")
else ("sdnext" if hasattr(p.sd_model, "is_sdxl") and not hasattr(p.sd_model, "is_ssd") else "a1111")
)
)
if self.ppp_debug_level != DEBUG_LEVEL.none: if self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info(f"Post-processing prompts ({'i2i' if is_i2i else 't2i'}) running on {app_names[app]}") self.ppp_logger.info(f"Post-processing prompts ({'i2i' if is_i2i else 't2i'})")
models_supported = {x: True for x in PromptPostProcessor.SUPPORTED_MODELS} models_supported = {x: True for x in PromptPostProcessor.SUPPORTED_MODELS}
if app == "sdnext": if app == SUPPORTED_APPS.sdnext:
models_supported["ssd"] = False models_supported["ssd"] = False
elif app == "forge": elif app == SUPPORTED_APPS.forge:
models_supported["ssd"] = False models_supported["ssd"] = False
models_supported["auraflow"] = False models_supported["auraflow"] = False
elif app == "reforge": elif app == SUPPORTED_APPS.reforge:
models_supported["flux"] = False models_supported["flux"] = False
models_supported["auraflow"] = False models_supported["auraflow"] = False
else: # assume A1111 compatible else: # assume A1111 compatible
models_supported["flux"] = False models_supported["flux"] = False
models_supported["auraflow"] = False models_supported["auraflow"] = False
env_info = { env_info = {
"app": app, "app": app.value,
"models_path": models_path, "models_path": models_path,
"model_filename": getattr(p.sd_model.sd_checkpoint_info, "filename", ""), "model_filename": getattr(p.sd_model.sd_checkpoint_info, "filename", ""),
"model_class": "", "model_class": "",
@@ -226,8 +233,18 @@ class PromptPostProcessorA1111Script(scripts.Script):
"is_sd3": False, # Stable Diffusion 3 "is_sd3": False, # Stable Diffusion 3
"is_flux": False, # Flux "is_flux": False, # Flux
"is_auraflow": False, # AuraFlow "is_auraflow": False, # AuraFlow
"is_pixart": False, # PixArt
"is_lumina2": False, # Lumina2
"is_ltxv": False, # LTXV
"is_cosmos": False, # Cosmos
"is_genmomochi": False, # GenmoMochi
"is_hunyuan": False, # Hunyuan
"is_hunyuanvideo": False, # HunyuanVideo
"is_hunyuan3d": False, # Hunyuan3D
"is_wanvideo": False, # WanVideo
"is_hidream": False, # HiDream
} }
if app == "sdnext": if app == SUPPORTED_APPS.sdnext:
# cannot differentiate SD1 and SD2, we set True to both # cannot differentiate SD1 and SD2, we set True to both
# LatentDiffusion is for the original backend, StableDiffusionPipeline is for the diffusers backend # LatentDiffusion is for the original backend, StableDiffusionPipeline is for the diffusers backend
env_info["model_class"] = p.sd_model.__class__.__name__ env_info["model_class"] = p.sd_model.__class__.__name__
@@ -239,7 +256,7 @@ class PromptPostProcessorA1111Script(scripts.Script):
env_info["is_flux"] = p.sd_model.__class__.__name__ == "FluxPipeline" env_info["is_flux"] = p.sd_model.__class__.__name__ == "FluxPipeline"
env_info["is_auraflow"] = p.sd_model.__class__.__name__ == "AuraFlowPipeline" env_info["is_auraflow"] = p.sd_model.__class__.__name__ == "AuraFlowPipeline"
# also supports 'Latent Consistency Model': LatentConsistencyModelPipeline', 'PixArt-Alpha': 'PixArtAlphaPipeline', 'UniDiffuser': 'UniDiffuserPipeline', 'Wuerstchen': 'WuerstchenCombinedPipeline', 'Kandinsky 2.1': 'KandinskyPipeline', 'Kandinsky 2.2': 'KandinskyV22Pipeline', 'Kandinsky 3': 'Kandinsky3Pipeline', 'DeepFloyd IF': 'IFPipeline', 'Custom Diffusers Pipeline': 'DiffusionPipeline', 'InstaFlow': 'StableDiffusionPipeline', 'SegMoE': 'StableDiffusionPipeline', 'Kolors': 'KolorsPipeline', 'AuraFlow': 'AuraFlowPipeline', 'CogView': 'CogView3PlusPipeline' # also supports 'Latent Consistency Model': LatentConsistencyModelPipeline', 'PixArt-Alpha': 'PixArtAlphaPipeline', 'UniDiffuser': 'UniDiffuserPipeline', 'Wuerstchen': 'WuerstchenCombinedPipeline', 'Kandinsky 2.1': 'KandinskyPipeline', 'Kandinsky 2.2': 'KandinskyV22Pipeline', 'Kandinsky 3': 'Kandinsky3Pipeline', 'DeepFloyd IF': 'IFPipeline', 'Custom Diffusers Pipeline': 'DiffusionPipeline', 'InstaFlow': 'StableDiffusionPipeline', 'SegMoE': 'StableDiffusionPipeline', 'Kolors': 'KolorsPipeline', 'AuraFlow': 'AuraFlowPipeline', 'CogView': 'CogView3PlusPipeline'
elif app == "forge": elif app == SUPPORTED_APPS.forge:
# from repositories\huggingface_guess\huggingface_guess\model_list.py # from repositories\huggingface_guess\huggingface_guess\model_list.py
env_info["model_class"] = p.sd_model.model_config.__class__.__name__ env_info["model_class"] = p.sd_model.model_config.__class__.__name__
env_info["is_sd1"] = getattr(p.sd_model, "is_sd1", False) env_info["is_sd1"] = getattr(p.sd_model, "is_sd1", False)
@@ -251,7 +268,7 @@ class PromptPostProcessorA1111Script(scripts.Script):
) # p.sd_model.model_config.__class__.__name__ == "SD3" # not actually supported? ) # p.sd_model.model_config.__class__.__name__ == "SD3" # not actually supported?
env_info["is_flux"] = p.sd_model.model_config.__class__.__name__ in ("Flux", "FluxSchnell") env_info["is_flux"] = p.sd_model.model_config.__class__.__name__ in ("Flux", "FluxSchnell")
env_info["is_auraflow"] = False # p.sd_model.model_config.__class__.__name__ == "AuraFlow" # not supported env_info["is_auraflow"] = False # p.sd_model.model_config.__class__.__name__ == "AuraFlow" # not supported
elif app == "reforge": elif app == SUPPORTED_APPS.reforge:
env_info["model_class"] = p.sd_model.__class__.__name__ env_info["model_class"] = p.sd_model.__class__.__name__
env_info["is_sd1"] = getattr(p.sd_model, "is_sd1", False) env_info["is_sd1"] = getattr(p.sd_model, "is_sd1", False)
env_info["is_sd2"] = getattr(p.sd_model, "is_sd2", False) env_info["is_sd2"] = getattr(p.sd_model, "is_sd2", False)
@@ -269,6 +286,7 @@ class PromptPostProcessorA1111Script(scripts.Script):
env_info["is_sd3"] = getattr(p.sd_model, "is_sd3", False) env_info["is_sd3"] = getattr(p.sd_model, "is_sd3", False)
env_info["is_flux"] = False env_info["is_flux"] = False
env_info["is_auraflow"] = False env_info["is_auraflow"] = False
hash_envinfo = hash(tuple(sorted(env_info.items())))
wc_wildcards_folders = getattr(opts, "ppp_wil_wildcardsfolders", "") wc_wildcards_folders = getattr(opts, "ppp_wil_wildcardsfolders", "")
if wc_wildcards_folders == "": if wc_wildcards_folders == "":
wc_wildcards_folders = os.getenv("WILDCARD_DIR", PPPWildcards.DEFAULT_WILDCARDS_FOLDER) wc_wildcards_folders = os.getenv("WILDCARD_DIR", PPPWildcards.DEFAULT_WILDCARDS_FOLDER)
@@ -277,13 +295,25 @@ class PromptPostProcessorA1111Script(scripts.Script):
for f in wc_wildcards_folders.split(",") for f in wc_wildcards_folders.split(",")
if f.strip() != "" if f.strip() != ""
] ]
en_mappings_folders = getattr(opts, "ppp_en_mappingsfolders", "")
if en_mappings_folders == "":
en_mappings_folders = os.getenv(
"EXTRANETWORKMAPPINGS_DIR",
PPPExtraNetworkMappings.DEFAULT_ENMAPPINGS_FOLDER,
)
enmappings_folders = [
(f if os.path.isabs(f) else os.path.abspath(os.path.join(models_path, f)))
for f in en_mappings_folders.split(",")
if f.strip() != ""
]
options = { options = {
"debug_level": getattr(opts, "ppp_gen_debug_level", DEBUG_LEVEL.none.value), "debug_level": getattr(opts, "ppp_gen_debug_level", DEBUG_LEVEL.none.value),
"on_warning": getattr(opts, "ppp_gen_onwarning", PromptPostProcessor.ONWARNING_CHOICES.warn.value),
"variants_definitions": getattr( "variants_definitions": getattr(
opts, "ppp_gen_variantsdefinitions", PromptPostProcessor.DEFAULT_VARIANTS_DEFINITIONS opts, "ppp_gen_variantsdefinitions", PromptPostProcessor.DEFAULT_VARIANTS_DEFINITIONS
), ),
"process_wildcards": getattr(opts, "ppp_wil_processwildcards", True), "process_wildcards": getattr(opts, "ppp_wil_processwildcards", True),
"if_wildcards": getattr(opts, "ppp_wil_ifwildcards", PromptPostProcessor.IFWILDCARDS_CHOICES.ignore.value), "if_wildcards": getattr(opts, "ppp_wil_ifwildcards", PromptPostProcessor.IFWILDCARDS_CHOICES.stop.value),
"choice_separator": getattr(opts, "ppp_wil_choice_separator", PromptPostProcessor.DEFAULT_CHOICE_SEPARATOR), "choice_separator": getattr(opts, "ppp_wil_choice_separator", PromptPostProcessor.DEFAULT_CHOICE_SEPARATOR),
"keep_choices_order": getattr(opts, "ppp_wil_keep_choices_order", False), "keep_choices_order": getattr(opts, "ppp_wil_keep_choices_order", False),
"stn_separator": getattr(opts, "ppp_stn_separator", PromptPostProcessor.DEFAULT_STN_SEPARATOR), "stn_separator": getattr(opts, "ppp_stn_separator", PromptPostProcessor.DEFAULT_STN_SEPARATOR),
@@ -292,6 +322,7 @@ class PromptPostProcessorA1111Script(scripts.Script):
"cleanup_empty_constructs": getattr(opts, "ppp_cup_emptyconstructs", True), "cleanup_empty_constructs": getattr(opts, "ppp_cup_emptyconstructs", True),
"cleanup_extra_separators": getattr(opts, "ppp_cup_extraseparators", True), "cleanup_extra_separators": getattr(opts, "ppp_cup_extraseparators", True),
"cleanup_extra_separators2": getattr(opts, "ppp_cup_extraseparators2", True), "cleanup_extra_separators2": getattr(opts, "ppp_cup_extraseparators2", True),
"cleanup_extra_separators_include_eol": getattr(opts, "ppp_cup_extraseparators_include_eol", True),
"cleanup_breaks": getattr(opts, "ppp_cup_breaks", True), "cleanup_breaks": getattr(opts, "ppp_cup_breaks", True),
"cleanup_breaks_eol": getattr(opts, "ppp_cup_breaks_eol", False), "cleanup_breaks_eol": getattr(opts, "ppp_cup_breaks_eol", False),
"cleanup_ands": getattr(opts, "ppp_cup_ands", True), "cleanup_ands": getattr(opts, "ppp_cup_ands", True),
@@ -300,11 +331,19 @@ class PromptPostProcessorA1111Script(scripts.Script):
"cleanup_merge_attention": getattr(opts, "ppp_cup_mergeattention", True), "cleanup_merge_attention": getattr(opts, "ppp_cup_mergeattention", True),
"remove_extranetwork_tags": getattr(opts, "ppp_rem_removeextranetworktags", False), "remove_extranetwork_tags": getattr(opts, "ppp_rem_removeextranetworktags", False),
} }
hash_options = hash(tuple(sorted(options.items())))
self.wildcards_obj.refresh_wildcards( self.wildcards_obj.refresh_wildcards(
self.ppp_debug_level, wildcards_folders if options["process_wildcards"] else None self.ppp_debug_level, wildcards_folders if options["process_wildcards"] else None
) )
self.extranetwork_mappings_obj.refresh_extranetwork_mappings(self.ppp_debug_level, enmappings_folders)
ppp = PromptPostProcessor( ppp = PromptPostProcessor(
self.ppp_logger, self.ppp_interrupt, env_info, options, self.grammar_content, self.wildcards_obj self.ppp_logger,
self.ppp_interrupt,
env_info,
options,
self.grammar_content,
self.wildcards_obj,
self.extranetwork_mappings_obj,
) )
prompts_list = [] prompts_list = []
@@ -349,12 +388,6 @@ class PromptPostProcessorA1111Script(scripts.Script):
rpr: list[str] = getattr(p, "all_prompts", None) rpr: list[str] = getattr(p, "all_prompts", None)
rnr: list[str] = getattr(p, "all_negative_prompts", None) rnr: list[str] = getattr(p, "all_negative_prompts", None)
if rpr is not None and rnr is not None: if rpr is not None and rnr is not None:
extra_params.update(
{
"PPP original prompts": rpr.copy(),
"PPP original negative prompts": rnr.copy(),
}
)
prompts_list += [ prompts_list += [
("regular", seed, prompt, negative_prompt) ("regular", seed, prompt, negative_prompt)
for seed, prompt, negative_prompt in zip(calculated_seeds, rpr, rnr) for seed, prompt, negative_prompt in zip(calculated_seeds, rpr, rnr)
@@ -363,53 +396,88 @@ class PromptPostProcessorA1111Script(scripts.Script):
# make it compatible with A1111 hires fix # make it compatible with A1111 hires fix
rph: list[str] = getattr(p, "all_hr_prompts", None) rph: list[str] = getattr(p, "all_hr_prompts", None)
rnh: list[str] = getattr(p, "all_hr_negative_prompts", None) rnh: list[str] = getattr(p, "all_hr_negative_prompts", None)
if rph is not None and rnh is not None: if rph is not None and rnh is not None and (rph != rpr or rnh != rnr):
extra_params.update(
{
"PPP original HR prompts": rph.copy(),
"PPP original HR negative prompts": rnh.copy(),
}
)
prompts_list += [ prompts_list += [
("hiresfix", seed, prompt, negative_prompt) ("hiresfix", seed, prompt, negative_prompt)
for seed, prompt, negative_prompt in zip(calculated_seeds, rph, rnh) for seed, prompt, negative_prompt in zip(calculated_seeds, rph, rnh)
if (seed, prompt, negative_prompt) not in prompts_list if (seed, prompt, negative_prompt) not in prompts_list
] ]
# processes prompts
for i, (prompttype, seed, prompt, negative_prompt) in enumerate(prompts_list):
if self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info(f"processing prompts[{i+1}] ({prompttype})")
if (
self.lru_cache.get(
(hash_envinfo, hash_options, seed, hash(self.wildcards_obj), prompt, negative_prompt)
)
is None
):
posp, negp, _ = ppp.process_prompt(prompt, negative_prompt, seed)
self.lru_cache.put(
(hash_envinfo, hash_options, seed, hash(self.wildcards_obj), prompt, negative_prompt), (posp, negp)
)
# adds also the result so i2i doesn't process it unnecessarily
self.lru_cache.put(
(hash_envinfo, hash_options, seed, hash(self.wildcards_obj), posp, negp), (posp, negp)
)
elif self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info("result already in cache")
# updates the prompts
rpr_copy = None
rnr_copy = None
if rpr is not None and rnr is not None:
rpr_changes = False
rnr_changes = False
rpr_copy = rpr.copy()
rnr_copy = rnr.copy()
for i, (seed, prompt, negative_prompt) in enumerate(zip(calculated_seeds, rpr, rnr)):
found = self.lru_cache.get(
(hash_envinfo, hash_options, seed, hash(self.wildcards_obj), prompt, negative_prompt)
)
if found is not None:
if rpr[i].strip() != found[0].strip():
rpr_changes = True
if rnr[i].strip() != found[1].strip():
rnr_changes = True
rpr[i] = found[0]
rnr[i] = found[1]
if add_prompts:
if rpr_changes:
extra_params["PPP original prompts"] = rpr_copy
if rnr_changes:
extra_params["PPP original negative prompts"] = rnr_copy
if rph is not None and rnh is not None:
rph_changes = False
rnh_changes = False
rph_copy = rph.copy()
rnh_copy = rnh.copy()
for i, (seed, prompt, negative_prompt) in enumerate(zip(calculated_seeds, rph, rnh)):
found = self.lru_cache.get(
(hash_envinfo, hash_options, seed, hash(self.wildcards_obj), prompt, negative_prompt)
)
if found is not None:
if rph[i].strip() != found[0].strip() and (not rpr_copy or rph[i].strip() != rpr_copy[i].strip()):
rph_changes = True
if rnh[i].strip() != found[1].strip() and (not rnr_copy or rnh[i].strip() != rnr_copy[i].strip()):
rnh_changes = True
rph[i] = found[0]
rnh[i] = found[1]
if add_prompts:
if rph_changes:
extra_params["PPP original HR prompts"] = rph_copy
if rnh_changes:
extra_params["PPP original HR negative prompts"] = rnh_copy
# fill extra generation parameters only if not already present # fill extra generation parameters only if not already present
for k, v in extra_params.items(): for k, v in extra_params.items():
if p.extra_generation_params.get(k) is None: if p.extra_generation_params.get(k) is None:
p.extra_generation_params[k] = v p.extra_generation_params[k] = v
# processes prompts t2 = time.monotonic_ns()
for i, (prompttype, seed, prompt, negative_prompt) in enumerate(prompts_list):
if self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info(f"processing prompts[{i+1}] ({prompttype})")
if self.lru_cache.get((seed, hash(self.wildcards_obj), prompt, negative_prompt)) is None:
posp, negp, _ = ppp.process_prompt(prompt, negative_prompt, seed)
self.lru_cache.put((seed, hash(self.wildcards_obj), prompt, negative_prompt), (posp, negp))
# adds also the result so i2i doesn't process it unnecessarily
self.lru_cache.put((seed, hash(self.wildcards_obj), posp, negp), (posp, negp))
elif self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info("result already in cache")
# updates the prompts
if rpr is not None and rnr is not None:
for i, (seed, prompt, negative_prompt) in enumerate(zip(calculated_seeds, rpr, rnr)):
found = self.lru_cache.get((seed, hash(self.wildcards_obj), prompt, negative_prompt))
if found is not None:
rpr[i] = found[0]
rnr[i] = found[1]
if rph is not None and rnh is not None:
for i, (seed, prompt, negative_prompt) in enumerate(zip(calculated_seeds, rph, rnh)):
found = self.lru_cache.get((seed, hash(self.wildcards_obj), prompt, negative_prompt))
if found is not None:
rph[i] = found[0]
rnh[i] = found[1]
t2 = time.time()
if self.ppp_debug_level != DEBUG_LEVEL.none: if self.ppp_debug_level != DEBUG_LEVEL.none:
self.ppp_logger.info(f"process time: {t2 - t1:.3f} seconds") self.ppp_logger.info(f"process time: {(t2 - t1) / 1_000_000_000:.3f} seconds")
def ppp_interrupt(self): def ppp_interrupt(self):
""" """
@@ -478,6 +546,21 @@ def on_ui_settings():
section=section, section=section,
), ),
) )
shared.opts.add_option(
key="ppp_gen_onwarning",
info=shared.OptionInfo(
default=PromptPostProcessor.ONWARNING_CHOICES.warn.value,
label="What to do on invalid content warnings?",
component=gr.Radio,
component_args={
"choices": (
("Show warning in console", PromptPostProcessor.ONWARNING_CHOICES.warn.value),
("Stop the generation", PromptPostProcessor.ONWARNING_CHOICES.stop.value),
)
},
section=section,
),
)
shared.opts.add_option( shared.opts.add_option(
key="ppp_gen_variantsdefinitions", key="ppp_gen_variantsdefinitions",
info=shared.OptionInfo( info=shared.OptionInfo(
@@ -500,6 +583,24 @@ def on_ui_settings():
section=section, section=section,
), ),
) )
shared.opts.add_option(
key="ppp_gen_addpromptstometadata",
info=shared.OptionInfo(
True,
label="Add original prompts to metadata (if they change)",
section=section,
),
)
shared.opts.add_option(
key="ppp_en_mappingsfolders",
info=shared.OptionInfo(
PPPExtraNetworkMappings.DEFAULT_ENMAPPINGS_FOLDER,
label="Extranetwork Mappings folders",
comment_after='<span class="info">(absolute or relative to the models folder)</span>',
section=section,
),
)
# wildcard settings # wildcard settings
shared.opts.add_option( shared.opts.add_option(
@@ -624,6 +725,14 @@ def on_ui_settings():
section=section, section=section,
), ),
) )
shared.opts.add_option(
key="ppp_cup_extraseparators_include_eol",
info=shared.OptionInfo(
False,
label="The extra separators options also remove EOLs",
section=section,
),
)
shared.opts.add_option( shared.opts.add_option(
key="ppp_cup_breaks", key="ppp_cup_breaks",
info=shared.OptionInfo( info=shared.OptionInfo(
View File
+11
View File
@@ -0,0 +1,11 @@
lora:
lora1:
- condition: _is_pony
name: lorapony
parameters: 0.8
triggers: ["triggerpony1", "triggerpony2"]
- condition: _is_illustrious
name: loraillustrious
parameters: "0.9:0.8"
triggers: ["triggerillustrious1", "triggerillustrious2"]
- triggers: ["triggergeneric1", "triggergeneric2", "{one|two}"]
+560 -278
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -6,7 +6,7 @@
"choice3" "choice3"
], ],
"wildcard2": [ "wildcard2": [
"r2-3$$-", "r2-3$$-$$",
"4::choice1", "4::choice1",
"3:: choice2 ", "3:: choice2 ",
"2::choice3", "2::choice3",
+10 -1
View File
@@ -5,7 +5,7 @@ yaml:
- choice3 - choice3
wildcard2: wildcard2:
- ~r2-3$$- - ~r2-3$$-$$
- "'label1,label2'4::choice1" - "'label1,label2'4::choice1"
- "3:: choice2 " - "3:: choice2 "
- { labels: ["label1", "label3"], weight: 2, content: choice3 } - { labels: ["label1", "label3"], weight: 2, content: choice3 }
@@ -39,6 +39,7 @@ yaml:
- { - {
sampler: "~", sampler: "~",
repeating: false, repeating: false,
optional: false,
count: 2, count: 2,
prefix: "prefix-", prefix: "prefix-",
suffix: "-suffix", suffix: "-suffix",
@@ -67,3 +68,11 @@ yaml:
- { count: 2, prefix: "#" } - { count: 2, prefix: "#" }
- nine - nine
- ten - ten
empty_wildcard:
- o$$ # parameters: optional
- if false::1
- if false::2
- if false::3
- if _sd in ("test1", "test2")::4
- if (false or false)::5
+1 -1
View File
@@ -1,5 +1,5 @@
# wildcard2 # wildcard2
r2-3$$- r2-3$$-$$
4::choice1 4::choice1
3:: choice2 3:: choice2
2::choice3 2::choice3
+2 -1
View File
@@ -1,2 +1,3 @@
# wildcard3 # wildcard3
__2$$,$$text/wildcard2__ __2$$,$$text/wildcard2__
0::áüç
File diff suppressed because one or more lines are too long