LoRA and prompt scheduling should produce identical output to the equivalent ComfyUI workflow using multiple samplers or the various conditioning manipulation nodes. If you find situations where this is not the case, please report a bug.

Note about stability

The nodes are currently undergoing some refactoring and rewriting.

Interfaces for nodes marked "v2" are subject to breaking change at any time, though at worst you'll likely just have to recreate a few nodes.

Nodes marked "experimental" are very likely to change or disappear completely, or might not even work completely.

What can it do?

See features below. Things you can control via the prompt:

Weight interpretation types (comfy, A1111, etc.)
Prompt editing and filtering without multiple samplers
LoRA loading and scheduling via ComfyUI's hook system
Masking, compositions and area control, combining prompts (AND), BREAK
Prompt masking with cutoff
SDXL parameters

If you find prompt scheduling inconvenient, PCEncodeSingle can be used as a drop-in replacement for CLIPTextEncode to get everything else.

This example workflow implements a two-pass workflow illustrating most scheduling features.

The tools in this repository combine well with the macro and wildcard functionality in comfyui-utility-nodes

Requirements

For PCEncodeSchedule and PCLoraHooksFromSchedule, you'll need at least version 0.3.7 of ComfyUI (0.3.36 of ComfyUI desktop).

You need to have lark installed in your Python environment for parsing to work (If you reuse A1111's venv, it'll already be there).

If you use the portable version of ComfyUI on Windows with its embedded Python, you must open a terminal in the ComfyUI installation directory and run the command:

.\python_embeded\python.exe -m pip install lark

Then restart ComfyUI afterwards.

Notable changes

I try to avoid behavioural changes that break old prompts, but they may happen occasionally.

2024-12-03 ComfyUI merged support for model/conditioning hooks. There are two new nodes, PCEncodeSchedule and PCLoraHooksFromSchedule that can be used in combination with the hook nodes. Some functionality is still missing from them, but going forward, these nodes will be the only nodes supported; I will not spend significant time fixing bugs in the old monkeypatched nodes anymore.

Note on how schedules work

ComfyUI does not use the step number to determine whether to apply conds; instead, it uses the sampler's timestep value which is affected by the scheduler you're using. This means that when the sampler scheduler isn't linear, the schedules generated by prompt control will not be either.

Features

Scheduling and LoRA loading

See the syntax documentation

Advanced CLIP encoding

If you use PCEncodeSchedule or PCEncodeSingle. advanced encodings are available automatically. Thanks to BlenderNeko for the original code.

You can use the syntax STYLE(weight_interpretation, normalization) in a prompt to affect how prompts are interpreted.

The weight interpretations available are:

comfy (default)
comfy++
compel
down_weight
A1111
perp

Normalizations are:

none (default)
length
mean

The normalization calculations are independent operations and you can combine them with +, eg STYLE(A1111, length+mean) or STYLE(comfy, mean+length), or even something silly like STYLE(perp, mean+length+mean+length)

For the legacy nodes, you need to have Advanced CLIP Encoding nodes cloned into your custom_nodes for more options will be available.

The style can be specified separately for each AND:ed prompt, but the first prompt is special; later prompts will "inherit" it as default. For example:

STYLE(A1111) a (red:1.1) cat with (brown:0.9) spots and a long tail AND an (old:0.5) dog AND a (green:1.4) (balloon:1.1)

will interpret everything as A1111, but

a (red:1.1) cat with (brown:0.9) spots and a long tail AND STYLE(A1111) an (old:0.5) dog AND a (green:1.4) (balloon:1.1)

Will interpret the first one using the default ComfyUI behaviour, the second prompt with A1111 and the last prompt with the default again

For things (ie. the code imports) to work, the nodes must be cloned in a directory named exactly ComfyUI_ADV_CLIP_emb.

Cutoff

PCEncodeSchedule reimplements cutoff from ComfyUI Cutoff. You do not need to have the nodes installed.

The syntax is

a group of animals, [CUT:white cat:white], [CUT:brown dog:brown:0.5:1.0:1.0:_]

You should read the prompt as a group of animals, white cat, brown dog, but CUT causes the tokens in target_tokens to be masked off from the base prompt in region_text, so that their effect can be isolated, and you're less likely to get brown cats or white dogs.

Target tokens are treated individually, separated by space, for example, [CUT:green apple, red apple, green leaf:green apple] will mask both greens and the apple, giving you + +, red +, + leaf. To mask out just green apple, use [CUT:green apple, red apple:green_apple] which will result in a masked prompt of + +, red apple. Escape _ with a \.

the parameters in the CUT section are region_text:target_tokens:weight;strict_mask:start_from_masked:padding_token of which only the first two are required. The default values are weight=1.0, strict_mask=1.0 start_from_masked=1.0, padding_token=+

If strict_mask, start_from_masked or padding_token are specified in more than one CUT, the last one becomes the default for any CUTs afterwards that do not explicitly set the parameters. For example, in:

[CUT:white cat:white:0.5] and [CUT:black parrot, flying:black:1.0:0.5] and [CUT:green apple:green]

white cat will a weight of 0.5, and 1.0 for all parameters, and black parrot and green apple will both have a strict_mask parameter of 0.5.

The parameters affect how the masked and unmasked prompts are combined to produce the final embedding. Just play around with them.

Nodes

PCLoraHooksFromSchedule

Creates a ComfyUI HOOKS object from a prompt schedule. Can be attached to a CLIP model to perform encoding and LoRA switching

The hooks can be a bit slow sometimes, especially in cases where the LoRA spec doesn't actually require reloading the patches every time you sample. Try PCHooksFromScheduleWithOptimizationTest. (Though that node is likely to go away eventually).

PCEncodeSchedule

Encodes all prompts in a schedule. Pass in a CLIP object with hooks attached for LoRA scheduling, then use the resulting CONDITIONING normally

PCEncodeSingle

Encodes a single prompt without scheduling or LoRA loading features (but including everything else).

Note: This does not currently ignore <lora:...:1> and will treat it as part of the prompt.

PromptToSchedule

Parses a schedule from a text prompt. A schedule is essentially an array of (valid_until, prompt) pairs that the other nodes can use.

FilterSchedule

Filters a schedule according to its parameters, removing any changes that do not occur within [start, end).

The node also does tag filtering if any tags are specified.

Always returns at least the last prompt in the schedule if everything would otherwise be filtered.

start=0, end=0 returns the prompt at the start and start=1.0, end=1.0 returns the prompt at the end.

PCScheduleSettings

Returns an object representing default values for the SDXL function and allows configuring MASK_SIZE outside the prompt. You need to apply them to a schedule with PCApplySettings. Note that for the SDXL settings to apply, you still need to have SDXL() in the prompt.

The "steps" parameter currently does nothing; it's for future features.

PCApplySettings

Applies the give default values from PCScheduleSettings to a schedule

PCPromptFromSchedule

Extracts a text prompt from a schedule; also logs it to the console. LoRAs are not included in the text prompt, though they are logged.

PCScheduleAddMasks

Add masks to a schedule object, for use with IMASK (see syntax documentation)

Experimental nodes

PCEncodeLazy is an experiment that uses ComfyUI's lazy graph execution mechanism to dynamically generate a graph of PCEncodeSingle and SetConditioningTimestepRange nodes from a prompt with schedules. This has the advantage that if a part of the schedule doesn't change, ComfyUI's caching mechanism allows you to avoid re-encoding the non-changed part.

for example, if you first encode [cat:dog:0.1] and later change that to [cat:dog:0.5], no re-encoding takes place.

for added fun, put NODE(NodeClassName, paramname) in a prompt to encode the text using any other node (the default is to use PCEncodeSingle and text)

Note that the node can't have required parameters besides a single CLIP parameter (which must be named clip) and the text prompt, and it must return a CONDITIONING as its first return value.

Legacy nodes

See here

Known issues

ComfyUI's LoRA hooks are a bit slower than LoRALoader currently when the LoRA doesn't actually require scheduling. Hopefully this will improve upstream.