arcum42-ComfyUI_SageUtils/ref_docs/frontend/javascript_objects_and_hijacking.md

Comfy Objects - UI Framework Reference

Source: https://docs.comfy.org/custom-nodes/js/javascript_objects_and_hijacking

LiteGraph

The Comfy UI is built on top of LiteGraph. Much of the Comfy functionality is provided by LiteGraph, so if developing more complex nodes you will probably find it helpful to clone that repository and browse the documentation, which can be found at doc/index.html.

ComfyApp

The app object (always accessible by import { app } from "../../scripts/app.js";) represents the Comfy application running in the browser, and contains a number of useful properties and functions, some of which are listed below.

Warning

: Hijacking functions on app is not recommended, as Comfy is under constant development, and core behavior may change.

Properties

Important properties of app include (this is not an exhaustive list):

Property	Description
canvas	An LGraphCanvas object, representing the current user interface. It contains some potentially interesting properties, such as node_over and selected_nodes.
canvasEl	The DOM <canvas> element
graph	A reference to the LGraph object describing the current graph
runningNodeId	During execution, the node currently being executed
ui	Provides access to some UI elements, such as the queue, menu, and dialogs

canvas (for graphical elements) and graph (for logical connections) are probably the ones you are most likely to want to access.

Functions

Again, there are many. A few significant ones are:

Function	Description
graphToPrompt	Convert the graph into a prompt that can be sent to the Python server
loadGraphData	Load a graph
queuePrompt	Submit a prompt to the queue
registerExtension	You've seen this one - used to add an extension

LGraph

The LGraph object is part of the LiteGraph framework, and represents the current logical state of the graph (nodes and links). If you want to manipulate the graph, the LiteGraph documentation (at doc/index.html if you clone the LiteGraph repository) describes the functions you will need.

You can use graph to obtain details of nodes and links, for example:

const ComfyNode_object_for_my_node = app.graph._nodes_by_id(my_node_id) 
ComfyNode_object_for_my_node.inputs.forEach(input => {
    const link_id = input.link;
    if (link_id) {
        const LLink_object = app.graph.links[link_id]
        const id_of_upstream_node = LLink_object.origin_id
        // etc
    }
});

LLink

The LLink object, accessible through graph.links, represents a single link in the graph, from node link.origin_id output slot link.origin_slot to node link.target_id slot link.target_slot. It also has a string representing the data type, in link.type, and link.id.

LLinks are created in the connect method of a LGraphNode (of which ComfyNode is a subclass).

Tip

: Avoid creating your own LLink objects - use the LiteGraph functions instead.

ComfyNode

ComfyNode is a subclass of LGraphNode, and the LiteGraph documentation is therefore helpful for more generic operations. However, Comfy has significantly extended the LiteGraph core behavior, and also does not make use of all LiteGraph functionality.

Tip

: The description that follows applies to a normal node. Group nodes, primitive nodes, notes, and redirect nodes have different properties.

A ComfyNode object represents a node in the current workflow. It has a number of important properties that you may wish to make use of, a very large number of functions that you may wish to use, or hijack to modify behavior.

To get a more complete sense of the node object, you may find it helpful to insert the following code into your extension and place a breakpoint on the console.log command. When you then create a new node you can use your favorite debugger to interrogate the node.

async nodeCreated(node) {
    console.log("nodeCreated")
}

Properties

Property	Description
bgcolor	The background color of the node, or undefined for the default
comfyClass	The Python class representing the node
flags	A dictionary that may contain flags related to the state of the node. In particular, flags.collapsed is true for collapsed nodes.
graph	A reference to the LGraph object
id	A unique id
input_type	A list of the input types (eg "STRING", "MODEL", "CLIP" etc). Generally matches the Python INPUT_TYPES
inputs	A list of inputs (discussed below)
mode	Normally 0, set to 2 if the node is muted and 4 if the node is bypassed. Values of 1 and 3 are not used by Comfy
order	The node's position in the execution order. Set by LGraph.computeExecutionOrder() when the prompt is submitted
pos	The [x,y] position of the node on the canvas
properties	A dictionary containing "Node name for S&R", used by LiteGraph
properties_info	The type and default value of entries in properties
size	The width and height of the node on the canvas
title	Display Title
type	The unique name (from Python) of the node class
widgets	A list of widgets (discussed below)
widgets_values	A list of the current values of widgets

Functions

There are a very large number of functions (85, last time I counted). A selection are listed below. Most of these functions are unmodified from the LiteGraph core code.

Inputs, Outputs, Widgets

Function	Description
addInput	Create a new input, defined by name and type
addInputs	Array version of addInput
findInputSlot	Find the slot index from the input name
findInputSlotByType	Find an input matching the type. Options to prefer, or only use, free slots
removeInput	By slot index
getInputNode	Get the node connected to this input. The output equivalent is getOutputNodes and returns a list
getInputLink	Get the LLink connected to this input. No output equivalent
addWidget	Add a standard Comfy widget
addCustomWidget	Add a custom widget (defined in the getComfyWidgets hook)
addDOMWidget	Add a widget defined by a DOM element
convertWidgetToInput	Convert a widget to an input if allowed by isConvertableWidget (in widgetInputs.js)

Connections

Function	Description
connect	Connect this node's output to another node's input
connectByType	Connect output to another node by specifying the type - connects to first available matching slot
connectByTypeOutput	Connect input to another node output by type
disconnectInput	Remove any link into the input (specified by name or index)
disconnectOutput	Disconnect an output from a specified node's input
onConnectionChange	Called on each node. side==1 if it's an input on this node
onConnectInput	Called before a connection is made. If this returns false, the connection is refused

Display

Function	Description
setDirtyCanvas	Specify that the foreground (nodes) and/or background (links and images) need to be redrawn
onDrawBackground	Called with a CanvasRenderingContext2D object to draw the background. Used by Comfy to render images
onDrawForeground	Called with a CanvasRenderingContext2D object to draw the node.
getTitle	The title to be displayed.
collapse	Toggles the collapsed state of the node.

Warning

: collapse is badly named; it toggles the collapsed state. It takes a boolean parameter, which can be used to override node.collapsable === false.

Other

Function	Description
changeMode	Use to set the node to bypassed (mode == 4) or not (mode == 0)

Inputs and Widgets

Inputs and Widgets represent the two ways that data can be fed into a node. In general a widget can be converted to an input, but not all inputs can be converted to a widget (as many datatypes can't be entered through a UI element).

node.inputs is a list of the current inputs (colored dots on the left hand side of the node), specifying their .name, .type, and .link (a reference to the connected LLink in app.graph.links).

If an input is a widget which has been converted, it also holds a reference to the, now inactive, widget in .widget.

node.widgets is a list of all widgets, whether or not they have been converted to an input. A widget has:

Property	Description
callback	A function called when the widget value is changed
last_y	The vertical position of the widget in the node
name	The (unique within a node) widget name
options	As specified in the Python code (such as default, min, and max)
type	The name of the widget type (see below) in lowercase
value	The current widget value. This is a property with get and set methods

Widget Types

app.widgets is a dictionary of currently registered widget types, keyed in the UPPER CASE version of the name of the type. Build in Comfy widgets types include the self explanatory BOOLEAN, INT, and FLOAT, as well as STRING (which comes in two flavours, single line and multiline), COMBO for dropdown selection from a list, and IMAGEUPLOAD, used in Load Image nodes.

Custom widget types can be added by providing a getCustomWidgets method in your extension.

Linked widgets

Widgets can also be linked - the built in behavior of seed and control_after_generate, for example. A linked widget has .type = 'base_widget_type:base_widget_name'; so control_after_generate may have type int:seed.

Prompt

When you press the Queue Prompt button in Comfy, the app.graphToPrompt() method is called to convert the current graph into a prompt that can be sent to the server.

app.graphToPrompt returns an object (referred to herein as prompt) with two properties, output and workflow.

output

prompt.output maps from the node_id of each node in the graph to an object with two properties.

prompt.output[node_id].class_type, the unique name of the custom node class, as defined in the Python code prompt.output[node_id].inputs, which contains the value of each input (or widget) as a map from the input name to:

• the selected value, if it is a widget, or
• an array containing (upstream_node_id, upstream_node_output_slot) if there is a link connected to the input, or
• undefined, if it is a widget that has been converted to an input and is not connected
• other unconnected inputs are not included in .inputs

Tip

: Note that the upstream_node_id in the array describing a connected input is represented as a string, not an integer.

workflow

prompt.workflow contains the following properties:

• config - a dictionary of additional configuration options (empty by default)
• extra - a dictionary containing extra information about the workflow. By default it contains:
  • extra.ds - describes the current view of the graph (scale and offset)
• groups - all groups in the workflow
• last_link_id - the id of the last link added
• last_node_id - the id of the last node added
• links - a list of all links in the graph. Each entry is an array of five integers and one string:
  • (link_id, upstream_node_id, upstream_node_output_slot, downstream_node_id, downstream_node_input_slot, data type)
• nodes - a list of all nodes in the graph. Each entry is a map of a subset of the properties of the node as described above
  • The following properties are included: flags, id, inputs, mode, order, pos, properties, size, type, widgets_values
  • In addition, unless a node has no outputs, there is an outputs property, which is a list of the outputs of the node, each of which contains:
    • name - the name of the output
    • type - the data type of the output
    • links - a list of the link_id of all links from this output (if there are no connections, may be an empty list, or null),
    • shape - the shape used to draw the output (default 3 for a dot)
    • slot_index - the slot number of the output
• version - the LiteGraph version number (at time of writing, 0.4)

Tip

: nodes.output is absent for nodes with no outputs, not an empty list.