Merge pull request #9 from dmarx/dev

Documentation sprint
2023-12-09 19:45:37 -08:00
parent 7a298cebd9 e3fbee045c
commit a99e1d9540
2 changed files with 81 additions and 6 deletions
@@ -0,0 +1,20 @@
+# https://github.com/ekalinin/github-markdown-toc/blob/master/README.md#toc-generation-with-github-actions
+
+on:
+  push:
+    branches:
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    steps:
+      - uses: actions/checkout@v2
+      - run: |
+          curl https://raw.githubusercontent.com/ekalinin/github-markdown-toc/0.8.0/gh-md-toc -o gh-md-toc
+          chmod a+x gh-md-toc
+          ./gh-md-toc --insert --no-backup --hide-footer README.md
+          rm gh-md-toc
+      - uses: stefanzweifel/git-auto-commit-action@v4
+        with:
+          commit_message: Auto update markdown TOC
@@ -2,23 +2,54 @@

 🚧 Work In Progress 🚧 - ComfyUI nodes to facilitate value keyframing by providing an interface for using [keyframed](https://github.com/dmarx/keyframed) in ComfyUI workflows.

+Similar project you might find more convenient for certain use cases https://github.com/FizzleDorf/ComfyUI_FizzNodes

-...Open question: if I make this, what will differentiate it from https://github.com/FizzleDorf/ComfyUI_FizzNodes ?
+<!--ts-->
+* [ComfyUI-Keyframed](#comfyui-keyframed)
+* [Overview](#overview)
+* [Starter Workflows](#starter-workflows)
+   * [Prompt Scheduling](#prompt-scheduling)
+   * [Prompt Interleaving (aka Prompt Entanglement, aka Prompt Superposition)](#prompt-interleaving-aka-prompt-entanglement-aka-prompt-superposition)
+   * [Simple Curved Parameter](#simple-curved-parameter)
+   * [Multi-Prompt Transition With Manually Specified Curves](#multi-prompt-transition-with-manually-specified-curves)
+* [Nodes](#nodes)
+   * [Curve Constructors](#curve-constructors)
+      * [Curve From String](#curve-from-string)
+      * [Curve From YAML](#curve-from-yaml)
+      * [Constant-Valued Curve](#constant-valued-curve)
+      * [Entangled Curves](#entangled-curves)
+   * [Curve Operators](#curve-operators)
+      * [Evaluate Curve At T](#evaluate-curve-at-t)
+      * [Apply Curve To Conditioning](#apply-curve-to-conditioning)
+      * [Add Conditions](#add-conditions)
+      * [Curve Arithmetic Operators](#curve-arithmetic-operators)
+   * [Scheduling](#scheduling)
+      * [Keyframed Condition](#keyframed-condition)
+         * [Interpolation Methods](#interpolation-methods)
+      * [Set Keyframe](#set-keyframe)
+      * [Evaluate Schedule](#evaluate-schedule)
+<!--te-->

-* easier curve composition
-* easier to change interpolators/easing functions

-# Philosophy
+# Overview
+
+**Philosophy**
+
+* Treat curves/schedules and keyframes as objects that can be passed around, plugged and unplugged, interchanged, and manipulated atomically.
+* Leverage nodes to facilitate modularity and flexibility.
+* Facilitate fast iteration
+* Provide convenience functions for most common use cases, and also low-leverl operators capable of reproducing the behavior of those convenience functions to permit user customization in "node space".

-Treat curves/schedules and keyframes as objects that can be passed around, plugged and unplugged, interchanged, and manipulated atomically.

 # Starter Workflows

-
 ## Prompt Scheduling

 ![Prompt Scheduling](examples/prompt-scheduling.png)

+ This one is probably why you are here. This workflow demonstrates how to use the `keyframed/schedule` nodes to achieve similar behavior as [FizzNodes'](https://github.com/FizzleDorf/ComfyUI_FizzNodes) **PromptSchedule** node, but implemented differently.
+
+This schedule is essentailly a normal AnimateDiff workflow where several nodes have replaced the normal conditioning setup. Rather than a single `CLIP Text Encode` node, we can have multiple prompts which transition sequentially over time. For documentation detailing how this workflow works, see the [`Nodes > Scheduling`](https://github.com/dmarx/ComfyUI-Keyframed/blob/dev/README.md#scheduling) section below.

 ## Prompt Interleaving (aka Prompt Entanglement, aka Prompt Superposition)

@@ -62,6 +93,9 @@ Which is the node equivalent for achieving this type of thing

 ![Entangled Curves](assets/nodes_entangled.png)

+Each output curve of the node is a sine wave that oscillates from `0` to `1` at the given frequency or wavelength. The outputs of a given node are phase-offset such that at any given time, the sum of the generated curves is `1`. 
+
+Reference the [Prompt Interleaving Workflow](https://github.com/dmarx/ComfyUI-Keyframed/blob/dev/README.md#prompt-interleaving-aka-prompt-entanglement-aka-prompt-superposition) for a demonstrative example.

 ## Curve Operators

@@ -89,16 +123,37 @@ NB: the division operator is unreliable at the time of this writing (2023-12-09)

 ## Scheduling

+These nodes work together to facilitate transitioning through a sequence of conditionings (i.e. prompts). We'll call this sequence the "schedule" of the conditionings. The primary use case here is for manipulating the positive prompt, i.e. for building a "prompt schedule". Given a particular time (e.g. frame id) in an animation seuquence, we can query the prompt schedule at that time to get the appropriate conditioning to pass to the KSampler. 
+
+Reference the [Prompt Scheduling Workflow](https://github.com/dmarx/ComfyUI-Keyframed/tree/dev?tab=readme-ov-file#prompt-scheduling) for a demonstrative example
+
 ### Keyframed Condition

 ![Keyframed Condition](assets/node_keyframed-condition.png)

+This node attaches a `conditioning` to a `keyframe`. This let's us assign a time to the conditioning and set what interpolation method to use when we're between keyframes. 
+
+#### Interpolation Methods
+
+Consider three time points `a,b,c` such that `a<b<c`, and two keyframes `X,Y` such that `X.time = a` and `Y.time = c`. To interpolate a value at time `b`, we would use `X.interpolation_method` to "tween" the value between `X.value` and `Y.value`. 
+
+* **`null`** - If the interpolation value is not set, the default interpolation is "previous".
+* **`previous`** - `X.time`
+* **`next`** - `Y.time`
+* **`linear`** - normal linear lerp. Matches the behavior of Deforum and FizzNodes.
+* **`sine_wave`** - sine function easing. slower close to terminal values, fastest at the middle of the transition.
+* **`eased_lerp** - sin2 easing. similar to sine but starts and ends slower and the fastest part is faster.
+* **`exp_decay`** - starts fast ends slow.

 ### Set Keyframe

 ![Set Keyframe](assets/node_set-keyframe.png)

+Attaches a keyframe to a schedule. If you haven't created a schedule yet, pass your keyframe into this node to create one, then pass the output schedule to subsequent `Set Keyframe` nodes to attach additional keyframes to the schedule. The first keyframe in the schedule should always be at `time=0`.
+
 ### Evaluate Schedule

 ![Evaluate Schedule](assets/nodes_evaluate-schedule.png)

+Evaluate the schedule at a time or time slice to extract a single conditioning or conditioning batch (respectively) to provide to e.g. a KSampler. The non-batch version gives the same kind of conditioning you'd get from a `CLIPTextConditioning` node. The intention of the batch version is to use with AnimateDiff to apply different conditionings per frame.
+