Compare commits
12
Commits
prep_v2
...
v2.0.0-beta.2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2f5d62b46b | ||
|
|
48c0286f09 | ||
|
|
e64a71fc6a | ||
|
|
dbd5a0e6d6 | ||
|
|
f3a4b12bc0 | ||
|
|
be921a9eea | ||
|
|
5aa6110206 | ||
|
|
48b727f4f6 | ||
|
|
5ff6d21e43 | ||
|
|
d8b850cafd | ||
|
|
24a9dd32f7 | ||
|
|
7b358e5127 |
@@ -1,6 +1,6 @@
|
||||
# ComfyUI prompt control
|
||||
|
||||
Nodes for LoRA and prompt scheduling that make basic operations in ComfyUI completely prompt-controllable.
|
||||
Control LoRA and prompt scheduling, advanced text encoding, regional prompting, and much more, through your text prompt. Generates dynamic graphs that are literally identical to handcrafted noodle soup.
|
||||
|
||||
## Prompt Control v2
|
||||
|
||||
@@ -19,11 +19,11 @@ Prompt Control also comes with `PCTextEncode`, which provides advanced text enco
|
||||
|
||||
### Is it stable now?
|
||||
|
||||
Unless I run into bugs or significant annoyances that require changing the interface, probably.
|
||||
Unless I run into bugs or significant annoyances that require changing the interface, it probably won't change too much, but until I tag 2.0, everything can change.
|
||||
|
||||
### Everything broke, where are the old nodes?
|
||||
|
||||
If you really need them, check out the [legacy branch](https://github.com/asagi4/comfyui-prompt-control/tree/legacy). However, I will not fix bugs in that branch, and I strongly recommend just migrating your workflows to the new nodes.
|
||||
If you really need them, you can install the [legacy nodes](https://github.com/asagi4/comfyui-prompt-control-legacy). However, I will not fix bugs in those nodes, and I strongly recommend just migrating your workflows to the new nodes.
|
||||
|
||||
You can have both installed at the same time; none of the nodes conflict.
|
||||
|
||||
@@ -43,9 +43,9 @@ See the [syntax documentation](doc/syntax.md)
|
||||
If you find prompt scheduling inconvenient for some reason, `PCTextEncode` can be used as a drop-in replacement for `CLIPTextEncode` to get everything else.
|
||||
|
||||
|
||||
[This example workflow](workflows/example-lazy.json?raw=1) shows LoRA scheduling and prompt editing and compares it with the same prompt implemented with built-in ComfyUI nodes.
|
||||
[This workflow](workflows/example-lazy.json?raw=1) shows LoRA scheduling and prompt editing and compares it with the same prompt implemented with built-in ComfyUI nodes.
|
||||
|
||||
[This example workflow](workflows/example.json?raw=1) implements a two-pass workflow illustrating more features, including custom masks and filtering.
|
||||
[Here](workflows/example-2pass.json?raw=1) is a two-pass workflow illustrating more features, including custom masks and filtering.
|
||||
|
||||
The tools in this repository combine well with the macro and wildcard functionality in [comfyui-utility-nodes](https://github.com/asagi4/comfyui-utility-nodes)
|
||||
|
||||
@@ -65,23 +65,23 @@ Then restart ComfyUI afterwards.
|
||||
|
||||
# Core nodes
|
||||
|
||||
## PCLazyTextEncode and PCLazyTextEncodeAvanced
|
||||
## PCLazyTextEncode and PCLazyTextEncodeAdvanced
|
||||
|
||||
`PCLazyTextEncode` uses ComfyUI's lazy graph execution mechanism to generate a graph of `PCTextEncode` and `SetConditioningTimestepRange` nodes from a prompt with schedules. This has the advantage that if a part of the schedule doesn't change, ComfyUI's caching mechanism allows you to avoid re-encoding the non-changed part.
|
||||
|
||||
for example, if you first encode `[cat:dog:0.1]` and later change that to `[cat:dog:0.5]`, no re-encoding takes place.
|
||||
|
||||
for added fun, put `NODE(NodeClassName, paramname)` in a prompt to generate a graph using **any other node** that's compatible. The node can't have required parameters besides a single CLIP parameter (which must be named `clip`) and the text prompt, and it must return a `CONDITIONING` as its first return value.
|
||||
for added fun, put `NODE(NodeClassName, textinputname)` in a prompt to generate a graph using **any other node** that's compatible. The node can't have required parameters besides a single CLIP parameter (which must be named `clip`) and the text prompt, and it must return a `CONDITIONING` as its first return value. The "default" values are `PCTextEncode` and `text`.
|
||||
|
||||
For example, if you for some reason do not want the advanced features of `PCTextEncode`, use `NODE(CLIPTextEncode)` in the prompt and you'll still get scheduling with ComfyUI's regular TE node.
|
||||
|
||||
The advanced node enables filtering the prompt for multi-pass workflows.
|
||||
|
||||
## PCLazyLoraLoader and PCLazyLoraLoaderAdvanced
|
||||
|
||||
This node reads LoRA expressions from the scheduled prompt and constructs a graph of `LoraLoader`s and `CreateHookLora`s as necessary to provide the necessary LoRA scheduling.
|
||||
This node reads LoRA expressions from the scheduled prompt and constructs a graph of `LoraLoader`s and `CreateHookLora`s as necessary to provide the necessary LoRA scheduling. Just use it in place of a `LoRALoader` and use the output normally.
|
||||
|
||||
If you have `apply_hooks` set to true, you **do not** need to apply the `HOOKS` output to a CLIP model separately; it's provided in case you want to use it elsewhere.
|
||||
|
||||
The advanced node enables filtering the prompt for multi-pass workflows.
|
||||
The Advanced node gives you access to the generated hooks. If you have `apply_hooks` set to true, you **do not** need to apply the `HOOKS` output to a CLIP model separately; it's provided in case you want to use it elsewhere. The advanced node also enables filtering the prompt for multi-pass workflows.
|
||||
|
||||
## PCTextEncode
|
||||
|
||||
|
||||
@@ -1,3 +1,10 @@
|
||||
"""
|
||||
@author: asagi4
|
||||
@title: ComfyUI Prompt Control
|
||||
@nickname: ComfyUI Prompt Control
|
||||
@description: Control LoRA and prompt scheduling, advanced text encoding, regional prompting, and much more, through your text prompt. Generates dynamic graphs that are literally identical to handcrafted noodle soup.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import logging
|
||||
|
||||
@@ -67,7 +67,7 @@ def create_hook_nodes_for_lora(graph, path, info, existing_node, start_pct, end_
|
||||
return hook_node, next_keyframe
|
||||
|
||||
|
||||
def build_lora_schedule(graph, schedule, model, clip, apply_hooks):
|
||||
def build_lora_schedule(graph, schedule, model, clip, apply_hooks=True, return_hooks=True):
|
||||
# This gets rid of non-existent LoRAs
|
||||
consolidated = consolidate_schedule(schedule)
|
||||
non_scheduled = find_nonscheduled_loras(consolidated)
|
||||
@@ -116,7 +116,12 @@ def build_lora_schedule(graph, schedule, model, clip, apply_hooks):
|
||||
|
||||
r = graph.finalize()
|
||||
|
||||
return {"result": (model, clip, res), "expand": r}
|
||||
if return_hooks:
|
||||
ret = (model, clip, res)
|
||||
else:
|
||||
ret = (model, clip)
|
||||
|
||||
return {"result": ret, "expand": r}
|
||||
|
||||
|
||||
class PCLazyLoraLoaderAdvanced:
|
||||
@@ -145,7 +150,7 @@ class PCLazyLoraLoaderAdvanced:
|
||||
def apply(self, model, clip, text, apply_hooks, unique_id, tags="", start=0.0, end=1.0):
|
||||
schedule = parse_prompt_schedules(text).with_filters(filters=tags, start=start, end=end)
|
||||
graph = GraphBuilder(f"PCLazyLoraLoaderAdvanced-{unique_id}")
|
||||
return build_lora_schedule(graph, schedule, model, clip, apply_hooks)
|
||||
return build_lora_schedule(graph, schedule, model, clip, apply_hooks=apply_hooks, return_hooks=True)
|
||||
|
||||
|
||||
class PCLazyLoraLoader:
|
||||
@@ -153,23 +158,25 @@ class PCLazyLoraLoader:
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"text": ("STRING", {"multiline": True}),
|
||||
"model": ("MODEL", {"rawLink": True}),
|
||||
"clip": ("CLIP", {"rawLink": True}),
|
||||
"apply_hooks": ("BOOLEAN", {"default": True}),
|
||||
"text": ("STRING", {"multiline": True}),
|
||||
},
|
||||
"hidden": {"unique_id": "UNIQUE_ID"},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("MODEL", "CLIP", "HOOKS")
|
||||
RETURN_TYPES = (
|
||||
"MODEL",
|
||||
"CLIP",
|
||||
)
|
||||
OUTPUT_TOOLTIPS = ("Returns a model and clip with LoRAs scheduled",)
|
||||
CATEGORY = "promptcontrol"
|
||||
FUNCTION = "apply"
|
||||
|
||||
def apply(self, model, clip, text, apply_hooks, unique_id):
|
||||
def apply(self, model, clip, text, unique_id):
|
||||
graph = GraphBuilder(f"PCLazyLoraLoader-{unique_id}")
|
||||
schedule = parse_prompt_schedules(text)
|
||||
return build_lora_schedule(graph, schedule, model, clip, apply_hooks)
|
||||
return build_lora_schedule(graph, schedule, model, clip, apply_hooks=True, return_hooks=False)
|
||||
|
||||
|
||||
def build_scheduled_prompts(graph, schedules, clip):
|
||||
|
||||
@@ -14,16 +14,13 @@ if lark.__version__ == "0.12.0":
|
||||
prompt_parser = lark.Lark(
|
||||
r"""
|
||||
!start: (prompt | /[][():|]/+)*
|
||||
prompt: (emphasized | embedding | scheduled | alternate | sequence | interpolate | loraspec | PLAIN | /</ | />/ | WHITESPACE)+
|
||||
prompt: (emphasized | embedding | scheduled | alternate | sequence | loraspec | PLAIN | /</ | />/ | WHITESPACE)+
|
||||
!emphasized: "(" prompt? ")"
|
||||
| "(" prompt ":" prompt ")"
|
||||
| "[" prompt "]"
|
||||
scheduled: "[" [prompt ":"] [prompt] ":" _WS? NUMBER ["," NUMBER] "]"
|
||||
| "[" [prompt ":"] [prompt] ":" _WS? TAG "]"
|
||||
sequence: "[SEQ" ":" [prompt] ":" NUMBER (":" [prompt] ":" NUMBER)+ "]"
|
||||
interpolate.100: "[INT" ":" interp_prompts ":" interp_steps "]"
|
||||
interp_prompts: prompt (":" [prompt])+
|
||||
interp_steps: NUMBER ("," NUMBER)+ [":" NUMBER]
|
||||
alternate: "[" [prompt] ("|" [prompt])+ [":" NUMBER] "]"
|
||||
loraspec.99: "<lora:" FILENAME lora_weights [lora_block_weights] ">"
|
||||
lora_weights.1: (":" _WS? NUMBER)~1..2
|
||||
@@ -94,7 +91,6 @@ def clamp(a, b, c):
|
||||
|
||||
def get_steps(tree):
|
||||
res = [100]
|
||||
interpolation_steps = []
|
||||
|
||||
def tostep(s):
|
||||
w = float(s) * 100
|
||||
@@ -116,7 +112,6 @@ def get_steps(tree):
|
||||
for i, _ in enumerate(tree.children[:-1]):
|
||||
tree.children[i] = tostep(tree.children[i])
|
||||
|
||||
interpolation_steps.append((tuple(tree.children[:-1]), tree.children[-1]))
|
||||
res.extend(tree.children[:-1])
|
||||
|
||||
def sequence(self, tree):
|
||||
@@ -134,7 +129,7 @@ def get_steps(tree):
|
||||
|
||||
CollectSteps().visit(tree)
|
||||
|
||||
return sorted(set(interpolation_steps)), sorted(set(res))
|
||||
return sorted(set(res))
|
||||
|
||||
|
||||
def at_step(step, filters, tree):
|
||||
@@ -180,24 +175,6 @@ def at_step(step, filters, tree):
|
||||
previous_step = s
|
||||
return ""
|
||||
|
||||
def interpolate(self, args):
|
||||
prompts, starts = args
|
||||
starts = starts[:-1]
|
||||
prev_prompt = None
|
||||
if step < starts[0]:
|
||||
return prompts[0]
|
||||
for i, x in enumerate(starts):
|
||||
prev_prompt = prompts[i]
|
||||
if x >= step:
|
||||
break
|
||||
return prev_prompt
|
||||
|
||||
def interp_steps(self, args):
|
||||
return list(args)
|
||||
|
||||
def interp_prompts(self, args):
|
||||
return ["".join(flatten(a or [])) for a in args]
|
||||
|
||||
def alternate(self, args):
|
||||
step_size = args[-1]
|
||||
idx = ceil(step / step_size)
|
||||
@@ -268,22 +245,15 @@ def at_step(step, filters, tree):
|
||||
|
||||
|
||||
class PromptSchedule(object):
|
||||
def __init__(self, prompt, filters="", start=0.0, end=1.0, defaults=None, masks=None):
|
||||
def __init__(self, prompt, filters="", start=0.0, end=1.0):
|
||||
self.filters = filters
|
||||
self.start = start
|
||||
self.end = end
|
||||
self.prompt = prompt.strip()
|
||||
self.defaults = {}
|
||||
if defaults:
|
||||
self.defaults = defaults
|
||||
self.loaded_loras = {}
|
||||
|
||||
self.interpolations = None
|
||||
self.parsed_prompt = None
|
||||
self.interpolations, self.parsed_prompt = self._parse()
|
||||
self.masks = masks
|
||||
if masks is None:
|
||||
self.masks = []
|
||||
self.parsed_prompt = self._parse()
|
||||
|
||||
def __iter__(self):
|
||||
# Filter out zero, it's only useful for interpolation
|
||||
@@ -293,27 +263,14 @@ class PromptSchedule(object):
|
||||
filters = [x.strip() for x in self.filters.upper().split(",")]
|
||||
try:
|
||||
parsed = []
|
||||
interpolations = set()
|
||||
tree = prompt_parser.parse(self.prompt)
|
||||
interpolation_steps, steps = get_steps(tree)
|
||||
log.debug("Interpolation steps: %s", interpolation_steps)
|
||||
steps = get_steps(tree)
|
||||
|
||||
def f(x):
|
||||
return round(x / 100, 2)
|
||||
|
||||
for t in steps:
|
||||
p = at_step(t, filters, tree)
|
||||
for control_points, step in interpolation_steps:
|
||||
interp_start = None
|
||||
interp_end = None
|
||||
if t == control_points[-1]:
|
||||
interp_start = max(control_points[0], int(self.start * 100))
|
||||
interp_end = min(control_points[-1], int(self.end * 100))
|
||||
control_points = tuple(
|
||||
sorted(set(f(c) for c in control_points if c >= interp_start or c <= interp_end))
|
||||
)
|
||||
if interp_start is not None and interp_end is not None and interp_end > interp_start:
|
||||
interpolations.add((control_points, f(step)))
|
||||
parsed.append([f(t), p])
|
||||
|
||||
except lark.exceptions.LarkError as e:
|
||||
@@ -322,14 +279,9 @@ class PromptSchedule(object):
|
||||
|
||||
# Tag filtering may return redundant prompts, so filter them out here
|
||||
res = []
|
||||
prev_p = None
|
||||
prev_end = -1
|
||||
|
||||
for end_at, p in parsed:
|
||||
# Preserve prompt if it ends at the start of an interpolation, otherwise bump its end time
|
||||
if p == prev_p and res[-1][0] not in [x[0][0] for x in interpolations]:
|
||||
res[-1][0] = end_at
|
||||
continue
|
||||
if end_at < self.start:
|
||||
continue
|
||||
elif end_at <= self.end:
|
||||
@@ -338,18 +290,12 @@ class PromptSchedule(object):
|
||||
elif end_at > self.end and prev_end < self.end:
|
||||
res.append([end_at, p])
|
||||
break
|
||||
prev_p = p
|
||||
|
||||
# Always use the last prompt if everything was filtered
|
||||
if len(res) == 0:
|
||||
res = [[1.0, parsed[-1][1]]]
|
||||
|
||||
return interpolations, res
|
||||
|
||||
def add_masks(self, *masks):
|
||||
for mask in masks:
|
||||
if mask is not None:
|
||||
self.masks.append(mask)
|
||||
return res
|
||||
|
||||
def clone(self):
|
||||
return self.with_filters()
|
||||
@@ -363,8 +309,6 @@ class PromptSchedule(object):
|
||||
filters=ifspecified(filters, self.filters),
|
||||
start=ifspecified(start, self.start),
|
||||
end=ifspecified(end, self.end),
|
||||
defaults=ifspecified(defaults, self.defaults),
|
||||
masks=self.masks[:],
|
||||
)
|
||||
return p
|
||||
|
||||
@@ -378,23 +322,6 @@ class PromptSchedule(object):
|
||||
return i, x
|
||||
return len(self.parsed_prompt) - 1, self.parsed_prompt[-1]
|
||||
|
||||
def interpolation_at(self, step, total_steps=1):
|
||||
i, x = self.at_step_idx(step, total_steps)
|
||||
for y in self.parsed_prompt[i:]:
|
||||
step = min(y[0], 1.0)
|
||||
if x[1]["prompt"] != y[1]["prompt"]:
|
||||
return step, y
|
||||
return 1.0, self.parsed_prompt[-1]
|
||||
|
||||
def load_loras(self, lora_cache=None):
|
||||
from .utils import Timer, load_loras_from_schedule
|
||||
|
||||
if lora_cache is not None:
|
||||
self.loaded_loras = lora_cache
|
||||
with Timer("PromptSchedule.load_loras()"):
|
||||
self.loaded_loras = load_loras_from_schedule(self.parsed_prompt, self.loaded_loras)
|
||||
return self.loaded_loras
|
||||
|
||||
|
||||
def parse_prompt_schedules(prompt):
|
||||
return PromptSchedule(prompt)
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "comfyui-prompt-control"
|
||||
description = "Nodes for convenient prompt editing, making many common operations prompt-controllable"
|
||||
version = "1.2.1"
|
||||
version = "2.0.0-beta.2"
|
||||
license = { file = "LICENSE" }
|
||||
# some lark versions older than 1.1.9 apparently have a bug that breaks things, see https://github.com/asagi4/comfyui-prompt-control/issues/35
|
||||
dependencies = ["lark >= 1.1.9"]
|
||||
|
||||
Reference in New Issue
Block a user