v1.1.0: Safe mode (#3)
This commit is contained in:
Lex Darlog
+8
-1
@@ -1,7 +1,14 @@
|
||||
## `TODO`
|
||||
|
||||
The pack is considered feature-complete.
|
||||
No other features planned. To do some fancy stuff with dictionaries, specialized node packs are recommended instead.
|
||||
No other features planned (but I have nothing against improvements: feel free to [Pull Request](../../pulls)).
|
||||
|
||||
To do some fancy stuff with dictionaries, specialized node packs are recommended instead.
|
||||
|
||||
# v1.1.0
|
||||
|
||||
- `✨New feature` Safe-formatting mode:
|
||||
- When an invalid pattern included in the formatted template - instead of throwing an error, leaves this part of the template as-is. Useful for templates with curly braces in them intended to stay: JSON/CSS.
|
||||
|
||||
# v1.0.4
|
||||
|
||||
|
||||
@@ -159,16 +159,33 @@ neg_common
|
||||
|
||||
### When you need curly braces themselves
|
||||
|
||||
To have the literal curly-brace characters inside the **formatted** prompt, you need to "escape" them: whenever you need one, you type it twice (`'{{'` or `'}}'`). Then, after formatting, it will turn to `'{'` or `'}'`, respectively.
|
||||
#### Safe mode `✨New in v1.1.0`
|
||||
|
||||
Keep in mind though, that with [Recursive formatting](#recursive-formatting-), any `{{text}}` will become `{text}` after very first iteration, and thus on the next one, it still will be treated as a placeholder to put a string with a `text` key into.
|
||||
The simplest solution - just enable `safe_format` toggle. It changes the node behavior: whenever formatter encounters any `{text inside curly braces}` that cannot be formatted _(for any reason: the dict misses this key, or it's an outright invalid pattern)_, it doesn't throw an error and instead simply leaves this part of template as-is.
|
||||
|
||||
It's extremely useful for code-like templates (e.g., containing JSON or CSS).
|
||||
|
||||
However, even for proper `{patterns}` referencing proper dict keys, the manual escaping (see below) also applies.
|
||||
|
||||
> [!NOTE]
|
||||
> Keep in mind that any spaces inside braces immediately make this pattern invalid.
|
||||
>
|
||||
> Thus, if you want to have something like: `{{my_key}}` _(so, `{my_key}` to be replaced + it's wrapped into another set of braces which you want to have in the output)_, the easiest way to do it without escaping is simply enabling safe mode and adding a couple of spaces inside the braces you want to stay. So: `{ {my_key} }` - the innermost part will be replaced with the value of `my_key` item from the dict, while the outer braces will stay.
|
||||
|
||||
#### Manual escaping - double braces
|
||||
|
||||
If you want to explicitly control which braces perform formatting and which don't - to have the literal curly-brace characters inside the prompt **after** formatting, you need to [escape](https://docs.python.org/3/library/string.html#format-string-syntax) them: whenever you need one, you type it twice (`'{{'` or `'}}'`). Then, after formatting, it will turn to `'{'` or `'}'`, respectively.
|
||||
|
||||
Keep in mind though, that with escaping approach and [Recursive formatting](#recursive-formatting-), any `{{text}}` will become `{text}` after very first iteration, and thus on the next one, it still will be treated as a placeholder to put a value of the `text` key into.
|
||||
|
||||
However, this might be exactly what you want for...
|
||||
|
||||
### Dynamic pattern aka conditional formatting
|
||||
|
||||
In other words, you build a prompt, where **keys themselves** are compiled from pieces. For example:
|
||||
- Your main text template has {{character_`{active_char}`}} pattern somewhere inside it.
|
||||
- Your main text template has one of the following patterns somewhere inside it:
|
||||
- {{character_`{active_char}`}} - if safe mode is off;
|
||||
- {character_`{active_char}`} - if safe mode is on.
|
||||
- You also have a string named `active_char` in your dict, which you simply set to a number.
|
||||
- Also-also, you have strings named `character_1`, `character_2`, etc.
|
||||
- Then, with recursive formatting, depending on just a **single** value you set the `active_char` element to, the following happens:
|
||||
|
||||
@@ -8,16 +8,16 @@ from inspect import getdoc as _getdoc
|
||||
import re as _re
|
||||
|
||||
|
||||
_re_indent_match = _re.compile("(\t*)( +)(\t*)(.*?)$").match
|
||||
_re_tab_indent_match = _re.compile("(\t+)(.*?)$").match
|
||||
_re_indent_match = _re.compile(r"(\t*)( +)(\t*)(.*?)$").match
|
||||
_re_tab_indent_match = _re.compile(r"(\t+)(.*?)$").match
|
||||
_re_list_line_match = _re.compile(
|
||||
"(\s*)("
|
||||
"[-*•]+"
|
||||
"|"
|
||||
"[a-zA-Z]\s*[.)]"
|
||||
"|"
|
||||
"[0-9+]\s*[.)]"
|
||||
")\s+"
|
||||
r"(\s*)("
|
||||
r"[-*•]+"
|
||||
r"|"
|
||||
r"[a-zA-Z]\s*[.)]"
|
||||
r"|"
|
||||
r"[0-9+]\s*[.)]"
|
||||
r")\s+"
|
||||
).match
|
||||
|
||||
|
||||
|
||||
+178
-47
@@ -1,10 +1,13 @@
|
||||
# encoding: utf-8
|
||||
"""
|
||||
Code for ``StringConstructorFormatter`` node.
|
||||
"""
|
||||
|
||||
import typing as _t
|
||||
|
||||
from dataclasses import dataclass as _dataclass, field as _field
|
||||
from inspect import cleandoc as _cleandoc
|
||||
import re as _re
|
||||
import sys as _sys
|
||||
|
||||
from frozendict import deepfreeze as _deepfreeze
|
||||
@@ -15,43 +18,172 @@ from . import _meta
|
||||
from .docstring_formatter import format_docstring as _format_docstring
|
||||
from .enums import DataTypes as _DataTypes
|
||||
from .funcs_common import _show_text_on_node, _verify_input_dict
|
||||
# from .funcs_formatter import formatter as _formatter
|
||||
|
||||
|
||||
_RECURSION_LIMIT = max(int(_sys.getrecursionlimit()), 1) # You can externally monkey-patch it... but if it blows up, your fault 🤷🏻♂️
|
||||
_RECURSION_LIMIT = max(int(_sys.getrecursionlimit()), 1) # You can externally monkey-patch it... but if it blows up, your fault 🤷🏻♂️single
|
||||
|
||||
__dataclass_slots_args = dict() if _sys.version_info < (3, 10) else dict(slots=True)
|
||||
|
||||
_re_formatting_keyword_match = _re.compile( # Pre-compiled regex match to extract ``{keyword}`` patterns
|
||||
r'(?P<prefix>.*?)'
|
||||
r'(?P<open_brackets>\{+)'
|
||||
r'(?P<inside_brackets>[^{}]+)'
|
||||
r'(?P<closed_brackets>\}+)'
|
||||
r'(?P<suffix>[^{}].*)?$',
|
||||
# flags=_re.DOTALL | _re.IGNORECASE,
|
||||
flags = _re.DOTALL, # We need dot to match new lines, too
|
||||
).match
|
||||
|
||||
|
||||
def _recursive_format(template: str, format_dict: _t.Dict[str, _t.Any], show: bool = True, unique_id: str = None) -> str:
|
||||
@_dataclass(**__dataclass_slots_args)
|
||||
class _Formatter:
|
||||
"""
|
||||
It's not actually recursive - because, you know, any recursion could be turned into iteration,
|
||||
and good boys do that. 😊
|
||||
A callable (function-like) class, which does the actual formatting, while respecting all the options.
|
||||
|
||||
It's made as a class to split the formatting into two stages:
|
||||
- First, the instance is properly initialized with the shared arguments (the methods to call are conditionally assigned depending on options);
|
||||
- Then, the actual instance is treated as a function - it needs to be called with the formatted string as the only argument.
|
||||
|
||||
It's done this way to avoid extra conditions in the loop + to organize the convoluted mess of intertwined functions
|
||||
into a more readable code.
|
||||
"""
|
||||
assert isinstance(_RECURSION_LIMIT, int) and _RECURSION_LIMIT > 0
|
||||
format_dict: _t.Optional[_t.Dict[str, _t.Any]] = None
|
||||
|
||||
prev: str = ''
|
||||
new: str = template
|
||||
for i in range(_RECURSION_LIMIT):
|
||||
if prev == new:
|
||||
break
|
||||
prev = new
|
||||
new = new.format_map(format_dict)
|
||||
if prev == new:
|
||||
return new
|
||||
recursive: bool = False
|
||||
safe: bool = True
|
||||
|
||||
msg = (
|
||||
f"Recursion limit ({_RECURSION_LIMIT}) reached on attempt to format a string: {template!r}\n"
|
||||
f"Last two formatting attempts:\n{prev!r}\n{new!r}"
|
||||
)
|
||||
if show and unique_id:
|
||||
_show_text_on_node(msg, unique_id)
|
||||
raise RecursionError(msg)
|
||||
# noinspection PyUnreachableCode
|
||||
return '' # just to be safe
|
||||
show_status: bool = True
|
||||
unique_node_id: str = None
|
||||
|
||||
__format_single: _t.Callable[[str], str] = _field(init=False, repr=False, compare=False, default=lambda x: x)
|
||||
_format: _t.Callable[[str], str] = _field(init=False, repr=False, compare=False, default=lambda x: x)
|
||||
|
||||
def __post_init__(self): # called by dataclass init
|
||||
if self.format_dict is None:
|
||||
self.format_dict = dict()
|
||||
|
||||
format_dict = self.format_dict
|
||||
_verify_input_dict(format_dict)
|
||||
self.__format_single = self.__format_single_safe if self.safe else self.__format_single_unsafe
|
||||
|
||||
if format_dict:
|
||||
self._format = self.__format_recursive if self.recursive else self.__format_single
|
||||
else:
|
||||
self._format = self.__dummy_return_intact
|
||||
|
||||
@staticmethod
|
||||
def __dummy_return_intact(template: str) -> str:
|
||||
return template
|
||||
|
||||
def __format_single_unsafe(self, template: str):
|
||||
return template.format_map(self.format_dict)
|
||||
|
||||
def __format_single_safe_parts_gen(self, template: str) -> _t.Generator[str, None, None]:
|
||||
"""
|
||||
EAFP: https://docs.python.org/3/glossary.html#term-EAFP
|
||||
|
||||
Instead of pre-escaping the whole template
|
||||
(which would require basically re-implementing the entire format-parsing logic),
|
||||
let's extract individual formatted pieces, actually try formatting them one by one,
|
||||
and return anything that cannot be formatted as-is - without any processing at all.
|
||||
|
||||
This generator returns such pieces - formatted or intact.
|
||||
"""
|
||||
format_dict = self.format_dict
|
||||
to_piece_template = '{{{}}}'.format
|
||||
|
||||
suffix: str = template
|
||||
while suffix:
|
||||
match = _re_formatting_keyword_match(suffix)
|
||||
if not match:
|
||||
break
|
||||
|
||||
prefix = match.group('prefix')
|
||||
open_brackets = match.group('open_brackets')
|
||||
inside_brackets = match.group('inside_brackets')
|
||||
closed_brackets = match.group('closed_brackets')
|
||||
suffix = match.group('suffix')
|
||||
|
||||
if prefix:
|
||||
yield prefix
|
||||
|
||||
piece_template = to_piece_template(inside_brackets)
|
||||
# noinspection PyBroadException
|
||||
try:
|
||||
formatted_piece = piece_template.format_map(format_dict)
|
||||
except Exception:
|
||||
# If, for ANY reason, we're unable to format the piece, return the template piece intact:
|
||||
yield open_brackets
|
||||
yield inside_brackets
|
||||
yield closed_brackets
|
||||
continue
|
||||
|
||||
# The key is found. Treat the piece as the actual formatting pattern.
|
||||
# Formatting "eats" one set of brackets either way:
|
||||
open_brackets = open_brackets[:-1]
|
||||
closed_brackets = closed_brackets[:-1]
|
||||
|
||||
# Now, even though we've succeeded, the template might've been pre-escaped.
|
||||
if open_brackets and closed_brackets:
|
||||
# The keyword is already pre-escaped. Return it intact:
|
||||
formatted_piece = inside_brackets
|
||||
|
||||
yield open_brackets
|
||||
yield formatted_piece
|
||||
yield closed_brackets
|
||||
|
||||
if suffix:
|
||||
yield suffix
|
||||
|
||||
def __format_single_safe(self, template: str) -> str:
|
||||
"""
|
||||
Format the pattern (single iteration of formatting) in the safe mode.
|
||||
Safe mode preserves any unknown ``{text patterns}`` inside curly brackets if they cannot be formatted.
|
||||
Correctly handles any formatting patterns natively supported by python
|
||||
(even the most fancy ones, involving ':', '!', attribute or index access, etc.).
|
||||
|
||||
Useful when JSON/CSS-like code is in the formatted template.
|
||||
"""
|
||||
return ''.join(self.__format_single_safe_parts_gen(template))
|
||||
|
||||
def __format_recursive(self, template: str) -> str:
|
||||
"""
|
||||
It's not actually recursive - because, you know, any recursion could be turned into iteration,
|
||||
and good boys do that. 😊
|
||||
"""
|
||||
assert isinstance(_RECURSION_LIMIT, int) and _RECURSION_LIMIT > 0
|
||||
|
||||
format_single_func = self.__format_single
|
||||
|
||||
prev: str = ''
|
||||
new: str = template
|
||||
for i in range(_RECURSION_LIMIT):
|
||||
if prev == new:
|
||||
return new
|
||||
prev = new
|
||||
new = format_single_func(new)
|
||||
|
||||
msg = (
|
||||
f"Recursion limit ({_RECURSION_LIMIT}) reached on attempt to format a string: {template!r}\n"
|
||||
f"Last two formatting attempts:\n{prev!r}\n{new!r}"
|
||||
)
|
||||
if self.show_status and self.unique_node_id:
|
||||
_show_text_on_node(msg, self.unique_node_id)
|
||||
raise RecursionError(msg)
|
||||
# noinspection PyUnreachableCode
|
||||
return '' # just to be extra-safe, if RecursionError is treated as warning
|
||||
|
||||
def __call__(self, template: str) -> str:
|
||||
if not isinstance(template, str):
|
||||
raise TypeError(f"Not a string: {template!r}")
|
||||
|
||||
out_text = self._format(template) if template else ''
|
||||
if self.show_status and self.unique_node_id:
|
||||
_show_text_on_node(out_text, self.unique_node_id)
|
||||
return out_text
|
||||
|
||||
# --------------------------------------
|
||||
|
||||
_dict = dict
|
||||
|
||||
_input_types = _deepfreeze({
|
||||
'required': {
|
||||
'template': (_IO.STRING, {'multiline': True, 'tooltip': (
|
||||
@@ -61,20 +193,22 @@ _input_types = _deepfreeze({
|
||||
"{char1_long}\n{char2_long}"
|
||||
)}),
|
||||
'recursive_format': (_IO.BOOLEAN, {'default': False, 'label_on': '❗ yes', 'label_off': 'no', 'tooltip': (
|
||||
"Do recursive format - i.e., allow the chunks from the dictionary to reference other chunks - which in turn "
|
||||
"lets you do really crazy stuff like building entire HIERARCHIES of sub-prompts, all linked to the same "
|
||||
"wording typed in one place.\n\n"
|
||||
"The exclamation mark reminds you that with great power comes great responsibility!\n"
|
||||
"You can end up with chunks cross-referencing each other in a loop. In such case, the node will just error out "
|
||||
"and won't let you crash the entire ComfyUI - but still, it's your responsibility to prevent "
|
||||
"such looping dictionaries."
|
||||
"Do recursive format - i.e., allow the chunks from the dictionary to reference other chunks."
|
||||
)}),
|
||||
'safe_format': (_IO.BOOLEAN, {'default': True, 'label_on': 'yes', 'label_off': 'no', 'tooltip': (
|
||||
"Safe mode: If a specific {text pattern} can't be formatted "
|
||||
"(it doesn't exist in the dict or isn't a valid formatting pattern at all), "
|
||||
"leave it as-is.\n"
|
||||
"On: invalid {patterns} preserved intact.\n"
|
||||
"Off: invalid {patterns} raise an error.\n\n"
|
||||
"Safe mode is recommended for templates with JSON, CSS, or other literal curly brackets."
|
||||
)}),
|
||||
'show_status': (_IO.BOOLEAN, {'default': True, 'label_on': 'formatted string', 'label_off': 'no', 'tooltip': (
|
||||
"Show the final string constructed from the text-template and format-dictionary?"
|
||||
)}),
|
||||
},
|
||||
'optional': {
|
||||
'dict': _DataTypes.input_dict(tooltip=(
|
||||
'dict': _DataTypes.input_dict(tooltip=( # It's not actually optional, but is here since there's no dict-widget
|
||||
"The dictionary to take named sub-strings from. It could be left unconnected, if the pattern doesn't reference "
|
||||
"any sub-strings - then, this node acts exactly the same as a regular string-primitive node."
|
||||
)),
|
||||
@@ -106,20 +240,17 @@ class StringConstructorFormatter:
|
||||
|
||||
@staticmethod
|
||||
def main(
|
||||
template: str, recursive_format: bool = False, show_status: bool = False, dict: _t.Dict[str, _t.Any] = None,
|
||||
template: str,
|
||||
recursive_format: bool = False,
|
||||
safe_format: bool = True,
|
||||
show_status: bool = False,
|
||||
dict: _t.Dict[str, _t.Any] = None, #actually, required - but it's here to keep the declared params order
|
||||
unique_id: str = None
|
||||
) -> _t.Tuple[str]:
|
||||
if dict is None:
|
||||
dict = _dict()
|
||||
_verify_input_dict(dict)
|
||||
if not isinstance(template, str):
|
||||
raise TypeError(f"Not a string: {template!r}")
|
||||
|
||||
out_text = (
|
||||
_recursive_format(template, dict, show_status, unique_id)
|
||||
if recursive_format
|
||||
else template.format_map(dict)
|
||||
formatter = _Formatter(
|
||||
format_dict=dict,
|
||||
recursive=recursive_format, safe=safe_format,
|
||||
show_status=show_status, unique_node_id=unique_id,
|
||||
)
|
||||
if show_status and unique_id:
|
||||
_show_text_on_node(out_text, unique_id)
|
||||
out_text = formatter(template)
|
||||
return (out_text, )
|
||||
|
||||
+12
-4
@@ -1,19 +1,27 @@
|
||||
[project]
|
||||
name = "string-constructor"
|
||||
version = "1.1.0"
|
||||
description = "String formatting (compiling text from pieces) made for humans:\n• Turn parts of prompt into a dictionary - to share them all across the entire workflow as a single connection.\n• Compose your prompt in-place from these snippets and regular text - with just one text field.\n• Easily update the dictionary down the line - get different prompts from the same template.\n• Reference text chunks within each other to build complex hierarchies of self-updating prompts.\n• A true time-saver for regional prompting (aka area composition)."
|
||||
version = "1.0.4"
|
||||
license = {file = "LICENSE.md"}
|
||||
readme = "README.md"
|
||||
authors = [
|
||||
{name = "Lex Darlog"}
|
||||
]
|
||||
license = {file = "LICENSE.md"}
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.7"
|
||||
dependencies = ["frozendict"]
|
||||
classifiers = [
|
||||
# https://docs.comfy.org/registry/specifications#classifiers-recommended
|
||||
"Operating System :: OS Independent",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
Repository = "https://github.com/Lex-DRL/ComfyUI-StringConstructor"
|
||||
"Feedback/Bugs/Feature Requests" = "https://github.com/Lex-DRL/ComfyUI-StringConstructor/issues"
|
||||
# Used by Comfy Registry https://comfyregistry.org
|
||||
|
||||
[tool.comfy]
|
||||
PublisherId = "lex-drl"
|
||||
DisplayName = "String Constructor (Text-Formatting)"
|
||||
Icon = ""
|
||||
# Icon = "https://raw.githubusercontent.com/username/repo/main/icon.png"
|
||||
# Banner = "https://raw.githubusercontent.com/username/repo/main/banner.png"
|
||||
# requires-comfyui = ">=0.3.51" # Node schema v3 released # TODO
|
||||
|
||||
Reference in New Issue
Block a user