Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 kantan-kanto
|
9cdfb68583 | ||
|
kantan-kanto
|
1f3bd2f7d4 | ||
|
kantan-kanto
|
5d4e5ac0af | ||
|
kantan-kanto
|
5284e69f90 | ||
|
kantan-kanto
|
2bb6cb8717 | ||
|
kantan-kanto
|
ce81103c06 | ||
|
kantan-kanto
|
1d7635ba7e |
@@ -6,6 +6,28 @@ All notable changes to ComfyUI-ScriptFlow will be documented in this file.
|
||||
## [Unreleased]
|
||||
|
||||
|
||||
## [1.3.2] - 2026-10-03
|
||||
|
||||
- `MultiOutputScript (Jev)` batches questions without script changes
|
||||
- Questions are queued while the script runs; the run stops when the script first uses an answer's value, the queued questions are sent, and the script reruns from the start with the answers
|
||||
- Questions about the same `state` are sent in one request (one TypeSafe `system_one` call with several questions); requests for different states are sent in parallel with the TypeSafe API and one after another with a local model
|
||||
- Storing an answer, passing it to a user-defined function, or calling `float()` on it does not stop the run
|
||||
- Scripts ask the same questions and produce the same outputs as before; each rerun starts from the same `random` state and time
|
||||
- Changed `max_jev_calls` to limit requests instead of questions; a run stops with an error before sending requests that would exceed it
|
||||
- `MultiOutputScript (Jev)` prints `[ComfyUI-ScriptFlow] Jev requests: N, states: S, questions: Q, responses: M` to the console after each run, including runs that end with an error
|
||||
- Questions answered from the run's cache are not counted
|
||||
- Set the `httpx2` and `typesafe_sdk` loggers to WARNING when `typesafe-sdk` is installed, removing two INFO lines per TypeSafe API request
|
||||
- `datetime.datetime.now()` and `datetime.date.today()` return the time the run started in both script nodes, so repeated calls in one script return the same value
|
||||
|
||||
|
||||
## [1.3.1] - 2026-09-29
|
||||
|
||||
- Fixed ComfyUI Registry publishing of 1.3.0
|
||||
- The Registry's automated dynamic-execution scan flagged the local Jev backend because it called the llama-cpp-python evaluation helper, whose method name matches Python's built-in code evaluation
|
||||
- The local backend now decodes the prompt with `llama_batch_init` / `llama_decode` / `llama_batch_free` directly; answers and probabilities are unchanged
|
||||
- Local models are loaded with `n_batch` equal to the 8192-token context so a whole prompt is decoded in one call
|
||||
|
||||
|
||||
## [1.3.0] - 2026-09-29
|
||||
|
||||
- Added `MultiOutputScript (Jev)` node (`utils`)
|
||||
|
||||
+492
@@ -0,0 +1,492 @@
|
||||
# ComfyUI-ScriptFlow（日本語版）
|
||||
[[en](README.md) | ja]
|
||||
|
||||
---
|
||||
|
||||
**Version:** 1.3.2
|
||||
**License:** GPL-3.0
|
||||
|
||||
System One の判断モデルを使える、安全な Python 風スクリプトノードです。
|
||||
TypeSafe AI の Jev API（フォールバックとしてローカルの GGUF LLM）に Yes/No・選択・スコアの質問をして、その答えを `if` 文の中でそのまま使えます。
|
||||
|
||||
|
||||
|
||||
デモ：`MultiOutputScript (Jev)` が TypeSafe API を使ってテキストプロンプトに合うアスペクト比を選び、各比率の確率を一覧にして、幅と高さを画像ノードに渡しています。
|
||||
|
||||
デモを簡略化した例です。テキストプロンプトを `in_text_1` につなぐと、その答えで幅と高さが決まります。
|
||||
|
||||
```python
|
||||
if jev.yes(it1, "Does a portrait orientation suit this prompt?"):
|
||||
ov1, ov2 = 832, 1280
|
||||
else:
|
||||
ov1, ov2 = 1280, 832
|
||||
```
|
||||
|
||||
System One モデルはテキストを生成せず、1 回の高速な処理で確率として答えを返します。そのため、答えをそのままワークフローの判断に使えます。
|
||||
|
||||
複数の入出力により、計算・論理判断・テキスト編集・条件分岐・解像度の選択・ファイル名の生成などを 1 つのノードで扱えます。これまで複数の基本ノードを組み合わせていた処理をまとめられます。スクリプトは制限付きのインタープリタで実行され、ファイルや OS にはアクセスしません。
|
||||
|
||||
## インストール
|
||||
|
||||
1. `ComfyUI/custom_nodes/` ディレクトリに移動します。
|
||||
2. このリポジトリをクローンします。
|
||||
```bash
|
||||
git clone https://github.com/kantan-kanto/ComfyUI-ScriptFlow
|
||||
```
|
||||
3. ComfyUI を再起動します。
|
||||
|
||||
`MultiOutputScript` と `centi` ノードに追加のパッケージは不要です。`MultiOutputScript (Jev)` は、使うバックエンドに応じて以下のセットアップが必要です。
|
||||
|
||||
### `MultiOutputScript (Jev)` のセットアップ
|
||||
|
||||
#### TypeSafe APIを使う場合
|
||||
|
||||
1. ComfyUI を実行している Python 環境に SDK をインストールします。
|
||||
```bash
|
||||
pip install typesafe-sdk
|
||||
```
|
||||
2. このカスタムノードの実際のインストール先ディレクトリに `api_key.txt` を作成し、TypeSafe の API キーを記入します（1 行、引用符なし）。
|
||||
```
|
||||
ComfyUI/custom_nodes/<your-installation-folder>/api_key.txt
|
||||
```
|
||||
キーは実行のたびにこのファイルから読み込まれます。`api_key.txt` は `.gitignore` に登録されています。コミットしないでください。
|
||||
|
||||
#### ローカルモデル（フォールバック）を使う場合
|
||||
|
||||
- ComfyUI を実行している Python 環境に [llama-cpp-python](https://github.com/JamePeng/llama-cpp-python) をインストールします。JamePeng 版と、本家の [abetlen/llama-cpp-python](https://github.com/abetlen/llama-cpp-python) 0.3.35 で動作を確認しています。
|
||||
- 指示追従型（instruction-tuned）の GGUF モデルを `ComfyUI/models/LLM`（または `models/text_encoders`）に置きます。GGUF にはチャットテンプレート（`tokenizer.chat_template` メタデータ）が含まれている必要があります。ほとんどの指示追従型 GGUF には含まれています。Qwen3.5-9B Q8_0 と Gemma-4-E4B Q8 で動作を確認しています。
|
||||
|
||||
## 主な機能
|
||||
- 複数の入力：数値とテキストの入力を自由に組み合わせられます。
|
||||
- 計算と論理：四則演算、比較、簡単な条件分岐に対応します。
|
||||
- 複数の出力：結果を別々の出力ポートから下流のノードに渡せます。
|
||||
- AI による判断：`MultiOutputScript (Jev)` で `jev.yes` / `jev.choice` / `jev.score` を確率付きで使えます。TypeSafe AI の Jev API、またはフォールバックとしてローカルの GGUF モデルを使います。
|
||||
- 安全な実行：スクリプトは構文を制限した AST インタープリタで実行され、ファイルや OS にはアクセスしません。モデルや TypeSafe API にアクセスするのは、Jev ノードの `jev` 関数だけです。
|
||||
|
||||
## 使用例
|
||||
- Wan2.2 の I2V ワークフローで、入力画像が縦長か横長かに応じて推奨解像度
|
||||
（512×384 または 384×512）を自動で選ぶ。
|
||||
- ファイル名をまとめて生成する（例：ローカル LLM のプロンプト履歴、出力ログ）。
|
||||
- 条件分岐でワークフローの動作を制御する（例：入力に応じてパラメータや経路を切り替える）。
|
||||
- LLM の出力を解析・変換し、下流のノード向けの構造化された値にする。
|
||||
- シード、しきい値、倍率などの数値パラメータを、独自の計算で動的に生成する。
|
||||
- 生成前に、テキストプロンプトに合うアスペクト比とサイズを、各比率の確率付きで選ぶ（Jev ノードの初期スクリプト）。
|
||||
- テキストの意味でワークフローを振り分ける（例：プロンプトが夜の情景かを判定する、主題を分類する）。
|
||||
|
||||
## ノード
|
||||
|
||||
### `MultiOutputScript`
|
||||
**Category:** `utils`
|
||||
|
||||
入力（任意、接続可能）：
|
||||
- `in_text_1`（任意の入力を受け付け、実行時に `str` か検証）
|
||||
- `in_text_2`（任意の入力を受け付け、実行時に `str` か検証）
|
||||
- `in_text_3`（任意の入力を受け付け、実行時に `str` か検証）
|
||||
- `in_value_1`（任意の入力を受け付け、実行時に `int` または `float` か検証）
|
||||
- `in_value_2`（任意の入力を受け付け、実行時に `int` または `float` か検証）
|
||||
- `in_value_3`（任意の入力を受け付け、実行時に `int` または `float` か検証）
|
||||
- `code`（複数行の文字列）
|
||||
|
||||
出力：
|
||||
- `out_text_1`（STRING）
|
||||
- `out_text_2`（STRING）
|
||||
- `out_text_3`（STRING）
|
||||
- `out_value_1`（INT）
|
||||
- `out_value_2`（INT）
|
||||
- `out_value_3`（INT）
|
||||
|
||||
数値出力についての注意：
|
||||
- 出力は `INT` です。
|
||||
- 整数入力から `FLOAT` の値が必要な場合は、`centi` ノードを使ってください。
|
||||
- 値が小数の場合、出力時に小数部分は切り捨てられます。
|
||||
- 四捨五入や切り上げをしたい場合は、スクリプト内で処理してください（例：`round(...)`、`math.ceil(...)`）。
|
||||
|
||||
### スクリプト変数
|
||||
`code` の中では次の変数を使います。
|
||||
- 入力：`it1`、`it2`、`it3`、`iv1`、`iv2`、`iv3`
|
||||
- 出力：`ot1`、`ot2`、`ot3`、`ov1`、`ov2`、`ov3`
|
||||
|
||||
代入しなかった出力は `None` になります。
|
||||
|
||||
### スクリプト例
|
||||
```python
|
||||
# Example: keep landscape at 512x384, portrait at 384x512
|
||||
# Connect GetImageSize width/height to in_value_1/in_value_2
|
||||
# iv1: image width, iv2: image height
|
||||
w, h = iv1, iv2
|
||||
ov1, ov2 = (512, 384) if w >= h else (384, 512)
|
||||
```
|
||||
|
||||
### `MultiOutputScript (Jev)`
|
||||
**Category:** `utils`
|
||||
|
||||
`MultiOutputScript` と同じ入力・出力・スクリプトの規則に加えて、`jev` 名前空間を使えます。TypeSafe AI の判断モデルである [Jev](https://typesafe.ai) に、テキストについての型付きの質問をし、その答えでスクリプトを分岐できます。
|
||||
|
||||
- **TypeSafe API**（基本）：質問をインターネット経由で Jev に送ります。
|
||||
- **ローカル GGUF モデル**（フォールバック）：Jev API を使えない場合や、テキストを手元のマシンから出したくない場合に使います。汎用の指示追従型モデルで Jev 風の判断を近似するもので、[SemIf (OpenJev)](https://github.com/TheoLeeCJ/SemIf-OpenJev) の方式に従い、文字を割り当てた選択肢の次トークン確率を 1 回の順伝播で読み取ります。テキストは生成せず、ネットワークにも何も送りません。答えや確率は Jev とは異なります。
|
||||
|
||||
スクリプトはどちらでも同じです。各バックエンドのセットアップは[インストール](#インストール)を参照してください。
|
||||
|
||||
追加の入力：
|
||||
- `model`：`TypeSafe API`、またはローカルの GGUF モデル（`mmproj-*` ファイルは表示されません）。
|
||||
- `max_jev_calls`（INT、初期値 8）：1 回の実行での Jev リクエストの上限です。同じ `state` についての質問は 1 回のリクエストにまとめて送るため（[質問のまとめ送り](#質問のまとめ送り)を参照）、質問数ではなくリクエスト数を制限します。上限を超えるリクエストは送る前にエラーで止まります。同じ実行の中で同じ質問は 1 回しか問い合わせません。
|
||||
|
||||
関数（`state` は文字列・辞書・リスト。テキストのみ）：
|
||||
- `jev.yes(state, question[, threshold])` → `bool`（Yes の確率が `threshold` 以上か。初期値 `0.5`）
|
||||
- `jev.noul(state, question)` → Yes の確率（`0.0`〜`1.0`）。しきい値と比較して使ってください。そのまま条件に使うとエラーになります。
|
||||
- `jev.choice(state, question, options)` → 最も確からしい選択肢（`str`）。`options` はラベルのリスト、またはラベルから説明への辞書です（ローカルモデルでは 2〜16 個）。
|
||||
- `jev.probabilities(state, question, options)` → 各選択肢とその確率の `dict`（合計 `1.0`）。`options` は `jev.choice` と同じです。同じ引数で両方を呼んでも、問い合わせは 1 回です。
|
||||
- `jev.score(state, question, levels)` → 順序付きの段階の説明に対する期待値（`float`、`0`〜`len(levels) - 1`）（ローカルモデルでは 2〜16 段階）
|
||||
|
||||
```python
|
||||
# it1: a prompt text
|
||||
if jev.yes(it1, "Does this prompt describe a night scene?", 0.7):
|
||||
ov1 = 30
|
||||
else:
|
||||
ov1 = 20
|
||||
|
||||
kind = jev.choice(it1, "Which orientation suits this prompt?", ["portrait", "landscape"])
|
||||
ov2, ov3 = (384, 512) if kind == "portrait" else (512, 384)
|
||||
|
||||
probs = jev.probabilities(it1, "Which orientation suits this prompt?", ["portrait", "landscape"])
|
||||
ot1 = f"{kind} (portrait: {probs['portrait']:.3f}, landscape: {probs['landscape']:.3f})"
|
||||
```
|
||||
|
||||
ノードの初期スクリプトは、`it1` のプロンプトに合う 8 種類のアスペクト比から 1 つを選び、`in_value_1` のメガピクセル数（未接続時は `1.0`。幅と高さは 64 の倍数に丸め）に合わせてサイズを決めます。幅と高さを `out_value_1` / `out_value_2` に、各比率の確率を `out_text_1` に出力します。
|
||||
|
||||
各関数は 1 つの質問をします。[TypeSafe Python SDK](https://pypi.org/project/typesafe-sdk/) で書いた場合のコードとの対応を示します。
|
||||
|
||||
| ScriptFlow | TypeSafe SDK |
|
||||
| --- | --- |
|
||||
| `jev.noul(state, q)` | `client.system_one(state=state, questions={"q": Noul(instructions=q)}).answers["q"].noul` |
|
||||
| `jev.yes(state, q, t)` | `... .answers["q"].noul >= t` |
|
||||
| `jev.choice(state, q, options)` | `client.system_one(state=state, questions={"q": Choice(instructions=q, criteria=options)}).answers["q"].choice` |
|
||||
| `jev.probabilities(state, q, options)` | `... .answers["q"].probabilities`（`jev.choice` と同じリクエスト） |
|
||||
| `jev.score(state, q, levels)` | `client.system_one(state=state, questions={"q": Score(instructions=q, criteria=levels)}).answers["q"].score` |
|
||||
|
||||
リストで渡した `options` は `criteria={label: None, ...}` として送られます。`confidence`、Score の段階ごとの確率、Noul の `criteria` は利用できません。
|
||||
|
||||
#### 質問のまとめ送り
|
||||
|
||||
質問は可能な範囲でまとめて送られます。
|
||||
|
||||
1. スクリプトの実行中、`jev` の質問はその場では送らず、順に溜めておきます。
|
||||
2. 答えを変数やリストに入れる、自分で定義した関数に渡す、`float()` を通す、といった使い方なら実行を続けます。`if` の条件、比較、計算、f 文字列、`sorted`、`join`、出力などで答えの値を初めて使ったところで、実行を止めます。
|
||||
3. 溜めた質問を `state` ごとに 1 回のリクエストで送り、スクリプトを最初から実行し直します。答えが分かった質問はその値を返し、新しい質問は同じように溜めます。
|
||||
|
||||
`state` が違うリクエストは、TypeSafe API では同時に送り、ローカルモデルでは順に処理します。質問を 1 つずつ問い合わせる場合と同じ質問をし、同じ結果を出力します。1 回の実行の中では、実行し直すたびに `random` の状態と `datetime.datetime.now()` の時刻が同じところから始まるので、実行し直しても質問は変わりません。
|
||||
|
||||
次の例では、`a` は `if` まで使われないので、`q1` と `q2` は 1 回のリクエストで送られます。`q3` は `a` が分かってから 2 回目のリクエストで送られます。
|
||||
|
||||
```python
|
||||
a = jev.noul(it1, "q1")
|
||||
b = jev.noul(it1, "q2")
|
||||
if a > 0.5:
|
||||
c = jev.noul(it1, "q3")
|
||||
```
|
||||
|
||||
多くの質問を 1 回のリクエストで送るには、`state` を同じにして、質問ごとに違う内容は質問文に入れてください。
|
||||
|
||||
TypeSafe API では、1 回のリクエストが [TypeSafe Python SDK](https://pypi.org/project/typesafe-sdk/) の `system_one` 呼び出し 1 回にあたります。上の例の 1 回目のリクエストは次のとおりです。
|
||||
|
||||
```python
|
||||
client.system_one(state=it1, questions={"q0": Noul(instructions="q1"), "q1": Noul(instructions="q2")})
|
||||
```
|
||||
|
||||
注意：
|
||||
- TypeSafe API は `state` と質問をインターネット経由で送信します。外部に出せないテキストには使わないでください。
|
||||
- ローカルモデルの質問は 1 回の順伝播で処理されます（GPU 上の 4B〜9B モデルで約 1〜1.5 秒）。モデルは実行ごとに最初の質問で読み込まれ（数秒）、実行が終わるとアンロードされて、ワークフローの残りのために VRAM を解放します。
|
||||
- ローカルモデルの確率は較正されておらず、モデルによって変わります。しきい値は実際の入力で試して決めてください。
|
||||
- 実行ごとにコンソールへ `[ComfyUI-ScriptFlow] Jev requests: N, states: S, questions: Q, responses: M` を出力します。送ったリクエスト数、異なる `state` の数、質問数、返信があったリクエスト数です。返信数がリクエスト数より少なければ、途中のリクエストが失敗しています。同じ実行のキャッシュから答えた質問は数えません。
|
||||
- `typesafe-sdk` がインストールされていると、`httpx2`（SDK が使う HTTP クライアント）と `typesafe_sdk` のロガーを WARNING に設定するため、リクエストごとの INFO ログは表示されません。
|
||||
- `MultiOutputScript` はモデルを読み込まず、ネットワークにもアクセスしません。`jev` は使えません。
|
||||
- このプロジェクトは独立したもので、TypeSafe AI とは関係ありません。
|
||||
|
||||
### `centi`
|
||||
**Category:** `utils`
|
||||
|
||||
入力（任意、接続可能）：
|
||||
- `int_1`（INT）
|
||||
- `int_2`（INT）
|
||||
- `int_3`（INT）
|
||||
|
||||
出力：
|
||||
- `float_1`（FLOAT）= `int_1 / 100`
|
||||
- `float_2`（FLOAT）= `int_2 / 100`
|
||||
- `float_3`（FLOAT）= `int_3 / 100`
|
||||
|
||||
注意：
|
||||
- `int_n` が接続されていない場合、対応する `float_n` の出力は `None` です。
|
||||
- 意図的に最小限の、接続のみのノードにしています。
|
||||
|
||||
---
|
||||
|
||||
## 実行環境
|
||||
スクリプトは Python の AST で解析され、ScriptFlow の安全なインタープリタで評価されます。代入、計算、分岐、ループ、ユーザー定義のヘルパー関数、辞書、リスト、文字列、一部のメソッドなど、ワークフローでよく使う処理に対応しています。
|
||||
|
||||
### 使える関数
|
||||
- `len`、`min`、`max`、`sum`、`abs`、`round`
|
||||
- `int`、`float`、`str`、`bool`
|
||||
- `sorted`、`reversed`
|
||||
- `enumerate`、`range`、`zip`
|
||||
- `any`、`all`、`pow`、`divmod`
|
||||
- `list`、`dict`、`tuple`
|
||||
- `jev.yes`、`jev.noul`、`jev.choice`、`jev.probabilities`、`jev.score`（`MultiOutputScript (Jev)` のみ。[該当の節](#multioutputscript-jev)を参照）
|
||||
|
||||
### 使える名前空間
|
||||
- `random`：`random`、`randint`、`uniform`、`choice`
|
||||
- `datetime`：`datetime.now`、`date.today`、`year`・`month`・`day`・`hour`・`minute`・`second` などの日時の項目、`isoformat` と `strftime`（`strftime` は v1.2.0 以降）
|
||||
- `math`：`ceil`、`floor`、`sqrt`、`sin`、`cos`、`tan`、`asin`、`acos`、`atan`、`atan2`、`log`、`log10`、`exp`、`pow`、`fabs`、`isfinite`、`pi`、`e`、`tau`
|
||||
|
||||
### 使えるメソッド
|
||||
- `str`：`replace`、`find`、`strip`、`split`、`splitlines`、`startswith`、`endswith`、`join`、`lower`、`upper`
|
||||
- `list`：`append`
|
||||
|
||||
### 使えないもの
|
||||
- `import` 文
|
||||
- ファイルへのアクセス（`open` など）
|
||||
- OS の操作（`os`、`sys`）
|
||||
- Python の動的な実行
|
||||
- セーフモードで禁止している構文：`Import`、`ImportFrom`、`Global`、`Nonlocal`、`ClassDef`、`Try`、`With`、`AsyncWith`、`Lambda`、`Delete`、`Yield`、`Await`
|
||||
- セーフモードで禁止している呼び出し：動的な実行、ファイルや入力へのアクセス、実行時の名前空間の参照、属性の動的な変更
|
||||
|
||||
### 実行時の注意
|
||||
- `random` や `datetime` を使うと、出力は毎回変わる可能性があります。
|
||||
- v1.3.2 以降では、`datetime.datetime.now()` と `datetime.date.today()` は実行を開始した時刻を返すため、1 つのスクリプトの中で何度呼んでも同じ値になります。
|
||||
- v1.2.0 以降では、`datetime.strftime(...)` でタイムスタンプを簡潔に整形できます。
|
||||
```python
|
||||
now = datetime.datetime.now()
|
||||
ot1 = now.strftime("%Y%m%d%H%M%S")
|
||||
```
|
||||
- v1.2.0 より前のバージョンでは、同じタイムスタンプを手作業で整形します。
|
||||
```python
|
||||
now = datetime.datetime.now()
|
||||
ot1 = f'{now.year:04d}{now.month:02d}{now.day:02d}{now.hour:02d}{now.minute:02d}{now.second:02d}'
|
||||
```
|
||||
- 型が合わない場合はエラーになり、実行が止まります。
|
||||
- セーフモードでは評価の前に AST を検証し、ステップ数やループ回数の上限を超えたスクリプトを止めます。
|
||||
- 信頼できるスクリプトやワークフローだけを使ってください。出所の分からないコードは実行しないでください。
|
||||
|
||||
## 例
|
||||
|
||||
### ワークフローの例
|
||||
|
||||
```
|
||||
examples/
|
||||
├─ example_workflow.json
|
||||
```
|
||||
|
||||
## 応用レシピ
|
||||
|
||||
### `LLM Dialogue Cycle` の会話ログを `dialogue_segments_json` に変換する
|
||||
|
||||
<details>
|
||||
<summary>ComfyUI-LLM-Session の会話ログ変換レシピを表示</summary>
|
||||
|
||||
`ComfyUI-LLM-Session` は、`LLM Dialogue Cycle` の結果全体を `transcript_text` として出力します。
|
||||
そのテキストを `MultiOutputScript.in_text_1` につなぎ、下のスクリプトを `code` 欄に貼り付けると、
|
||||
後段の会話用 TTS ノード向けに、話者タグ付きの JSON テキストを生成できます。
|
||||
|
||||
推奨するワークフロー：
|
||||
|
||||
```text
|
||||
LLM Dialogue Cycle
|
||||
-> transcript_text
|
||||
-> MultiOutputScript.in_text_1
|
||||
-> MultiOutputScript.out_text_1
|
||||
-> dialogue_segments_json input of a TTS node
|
||||
```
|
||||
|
||||
次のコードを `MultiOutputScript.code` に貼り付けます。
|
||||
|
||||
```python
|
||||
# Convert ComfyUI-LLM-Session "LLM Dialogue Cycle" transcript_text
|
||||
# into dialogue_segments_json for TTS.
|
||||
# Only text inside Japanese corner quotes 「...」 is used as spoken dialogue.
|
||||
#
|
||||
# Input:
|
||||
# it1: transcript_text
|
||||
#
|
||||
# Output:
|
||||
# ot1: JSON text with schema "kantan.dialogue.v1"
|
||||
#
|
||||
# Expected transcript lines:
|
||||
# [2026-05-16T12:00:00] USER -> A: initial prompt
|
||||
# [2026-05-16T12:00:01] A: 彼女は少し考えた。「こんにちは。」
|
||||
# [2026-05-16T12:00:02] B: 彼は笑った。「やあ。」「調子はどう？」
|
||||
#
|
||||
# The parser also accepts the unicode arrow form:
|
||||
# USER → A:
|
||||
|
||||
def json_escape(value):
|
||||
s = str(value)
|
||||
s = s.replace("\\", "\\\\")
|
||||
s = s.replace('"', '\\"')
|
||||
s = s.replace("\n", "\\n")
|
||||
s = s.replace("\r", "\\r")
|
||||
s = s.replace("\t", "\\t")
|
||||
return s
|
||||
|
||||
def extract_spoken_text(value):
|
||||
source = str(value)
|
||||
parts = []
|
||||
position = 0
|
||||
|
||||
while position < len(source):
|
||||
start = source.find("「", position)
|
||||
if start < 0:
|
||||
position = len(source)
|
||||
else:
|
||||
end = source.find("」", start + 1)
|
||||
if end < 0:
|
||||
position = len(source)
|
||||
else:
|
||||
spoken = source[start + 1:end].strip()
|
||||
if spoken:
|
||||
parts.append(spoken)
|
||||
position = end + 1
|
||||
|
||||
return "\n".join(parts)
|
||||
|
||||
transcript = str(it1 or "")
|
||||
lines = transcript.splitlines()
|
||||
utterances = []
|
||||
current = None
|
||||
|
||||
for raw_line in lines:
|
||||
line = str(raw_line)
|
||||
speaker = None
|
||||
text = None
|
||||
timestamp = ""
|
||||
|
||||
if line.startswith("[") and "] " in line:
|
||||
close_index = line.find("] ")
|
||||
timestamp = line[1:close_index]
|
||||
rest = line[close_index + 2:]
|
||||
|
||||
if rest.startswith("A:"):
|
||||
speaker = "A"
|
||||
text = rest[2:].strip()
|
||||
elif rest.startswith("B:"):
|
||||
speaker = "B"
|
||||
text = rest[2:].strip()
|
||||
elif rest.startswith("USER -> A:"):
|
||||
speaker = "USER"
|
||||
text = rest[len("USER -> A:"):].strip()
|
||||
elif rest.startswith("USER → A:"):
|
||||
speaker = "USER"
|
||||
text = rest[len("USER → A:"):].strip()
|
||||
|
||||
if speaker == "A" or speaker == "B":
|
||||
if current is not None:
|
||||
utterances.append(current)
|
||||
current = {
|
||||
"speaker": speaker,
|
||||
"text": text,
|
||||
"timestamp": timestamp,
|
||||
}
|
||||
else:
|
||||
if current is not None:
|
||||
if current["text"]:
|
||||
current["text"] = current["text"] + "\n" + line
|
||||
else:
|
||||
current["text"] = line
|
||||
|
||||
if current is not None:
|
||||
utterances.append(current)
|
||||
|
||||
items = []
|
||||
index = 1
|
||||
for u in utterances:
|
||||
spoken_text = extract_spoken_text(u["text"])
|
||||
if not spoken_text:
|
||||
continue
|
||||
|
||||
item = (
|
||||
'{"index":'
|
||||
+ str(index)
|
||||
+ ',"speaker":"'
|
||||
+ json_escape(u["speaker"])
|
||||
+ '","text":"'
|
||||
+ json_escape(spoken_text)
|
||||
+ '","timestamp":"'
|
||||
+ json_escape(u["timestamp"])
|
||||
+ '"}'
|
||||
)
|
||||
items.append(item)
|
||||
index = index + 1
|
||||
|
||||
ot1 = (
|
||||
'{"schema":"kantan.dialogue.v1",'
|
||||
+ '"source":"ComfyUI-LLM-Session LLM Dialogue Cycle",'
|
||||
+ '"utterances":['
|
||||
+ ",".join(items)
|
||||
+ "]}"
|
||||
)
|
||||
```
|
||||
|
||||
出力される `out_text_1` は、次のような JSON 文字列です。
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "kantan.dialogue.v1",
|
||||
"source": "ComfyUI-LLM-Session LLM Dialogue Cycle",
|
||||
"utterances": [
|
||||
{
|
||||
"index": 1,
|
||||
"speaker": "A",
|
||||
"text": "こんにちは。",
|
||||
"timestamp": "2026-05-16T12:00:01"
|
||||
},
|
||||
{
|
||||
"index": 2,
|
||||
"speaker": "B",
|
||||
"text": "やあ。\n調子はどう？",
|
||||
"timestamp": "2026-05-16T12:00:02"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
注意：
|
||||
|
||||
- TTS がキャラクターのセリフだけを読むよう、`USER -> A` / `USER → A` は意図的に除外しています。
|
||||
- モデルの出力が複数行にわたる場合は、直前の `A` または `B` の発話に含めます。
|
||||
- `utterances[].text` に出力するのは `「...」` の中のテキストだけです。かぎかっこの外の地の文や心の声は除外します。
|
||||
- 1 つの発話に複数のかぎかっこがある場合は、改行でつなぎます。
|
||||
- `「...」` を含まない発話は、TTS 用のスクリプトから除外します。
|
||||
- このレシピでは、`out_text_2`、`out_text_3`、数値出力は使いません。
|
||||
|
||||
</details>
|
||||
|
||||
## ライセンス
|
||||
|
||||
このプロジェクトは **GNU General Public License v3.0** の下でライセンスされています。
|
||||
|
||||
**Copyright (C) 2026 kantan-kanto**
|
||||
GitHub: https://github.com/kantan-kanto
|
||||
|
||||
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
|
||||
|
||||
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.
|
||||
|
||||
## サポート
|
||||
|
||||
- **Issues**：不具合の報告や機能の要望は GitHub Issues へお願いします
|
||||
- **Documentation**：バージョン履歴は [CHANGELOG.md](CHANGELOG.md) を参照してください
|
||||
- **Examples**：ワークフローのテンプレートは [examples/](examples/) を参照してください
|
||||
|
||||
---
|
||||
|
||||
## 変更履歴
|
||||
|
||||
詳しいバージョン履歴は [CHANGELOG.md](CHANGELOG.md) を参照してください。
|
||||
|
||||
### 現在のバージョン：1.3.2
|
||||
- `MultiOutputScript (Jev)` がスクリプトを変えずに質問をまとめて送るように：同じ `state` の質問は 1 回のリクエストで送り、`state` が違うリクエストは同時に送る
|
||||
- `max_jev_calls` が質問数ではなくリクエスト数を制限するように
|
||||
- 実行ごとに Jev のリクエスト数、`state` 数、質問数、返信数をコンソールに出力
|
||||
- TypeSafe SDK と HTTP クライアントのリクエストごとの INFO ログを非表示に
|
||||
- `datetime.datetime.now()` と `datetime.date.today()` が実行を開始した時刻を返すように
|
||||
@@ -1,6 +1,7 @@
|
||||
# ComfyUI-ScriptFlow
|
||||
[en | [ja](README.ja.md)]
|
||||
|
||||
**Version:** 1.3.0
|
||||
**Version:** 1.3.2
|
||||
**License:** GPL-3.0
|
||||
|
||||
Safe Python-like script node for ComfyUI with a System One decision model.
|
||||
@@ -46,12 +47,12 @@ The `MultiOutputScript` and `centi` nodes need no extra packages. `MultiOutputSc
|
||||
```
|
||||
ComfyUI/custom_nodes/<your-installation-folder>/api_key.txt
|
||||
```
|
||||
The key is read from this file on every run; the `TYPESAFE_API_KEY` environment variable is not used. `api_key.txt` is listed in `.gitignore`; never commit it.
|
||||
The key is read from this file on every run. `api_key.txt` is listed in `.gitignore`; never commit it.
|
||||
|
||||
#### Local models (fallback)
|
||||
|
||||
- Install [llama-cpp-python](https://github.com/JamePeng/llama-cpp-python) with GPU support for your platform in the Python environment that runs ComfyUI. Tested with the JamePeng fork and the upstream [abetlen/llama-cpp-python](https://github.com/abetlen/llama-cpp-python) 0.3.35. A CPU-only build works but is much slower.
|
||||
- Put an instruction-tuned GGUF model with a chat template in `ComfyUI/models/LLM` (or `models/text_encoders`). Tested with Qwen3.5-9B Q8_0 and Gemma-4-E4B Q8.
|
||||
- Install [llama-cpp-python](https://github.com/JamePeng/llama-cpp-python) in the Python environment that runs ComfyUI. Tested with the JamePeng fork and the upstream [abetlen/llama-cpp-python](https://github.com/abetlen/llama-cpp-python) 0.3.35.
|
||||
- Put an instruction-tuned GGUF model in `ComfyUI/models/LLM` (or `models/text_encoders`). The GGUF must include a chat template (`tokenizer.chat_template` metadata), as most instruction-tuned GGUFs do. Tested with Qwen3.5-9B Q8_0 and Gemma-4-E4B Q8.
|
||||
|
||||
## Key Features
|
||||
- Multiple inputs: freely combine numeric and text inputs.
|
||||
@@ -126,7 +127,7 @@ Scripts are the same for both. See [Installation](#installation) for the setup o
|
||||
|
||||
Extra inputs:
|
||||
- `model`: `TypeSafe API`, or a local GGUF model (`mmproj-*` files are hidden).
|
||||
- `max_jev_calls` (INT, default 8): maximum API requests or model evaluations per run. Identical questions within a run are asked only once.
|
||||
- `max_jev_calls` (INT, default 8): maximum Jev requests per run. Questions about the same `state` share one request (see [Batched questions](#batched-questions)), so this limits requests, not questions. A run stops with an error before sending requests that would exceed it. Identical questions within a run are asked only once.
|
||||
|
||||
Functions (`state` is a string, dict, or list; text only):
|
||||
- `jev.yes(state, question[, threshold])` → `bool` (yes-probability ≥ `threshold`, default `0.5`)
|
||||
@@ -151,7 +152,7 @@ ot1 = f"{kind} (portrait: {probs['portrait']:.3f}, landscape: {probs['landscape'
|
||||
|
||||
The node's default script picks one of eight aspect ratios for the prompt in `it1`, sizes it to the megapixels in `in_value_1` (default `1.0` when unconnected; width and height rounded to multiples of 64), outputs the width and height to `out_value_1` / `out_value_2`, and lists every ratio's probability in `out_text_1`.
|
||||
|
||||
Each function asks one question. With the TypeSafe API it maps to one `system_one` request of the [TypeSafe Python SDK](https://pypi.org/project/typesafe-sdk/):
|
||||
Each function asks one question. The table shows the equivalent code written with the [TypeSafe Python SDK](https://pypi.org/project/typesafe-sdk/):
|
||||
|
||||
| ScriptFlow | TypeSafe SDK |
|
||||
| --- | --- |
|
||||
@@ -163,11 +164,40 @@ Each function asks one question. With the TypeSafe API it maps to one `system_on
|
||||
|
||||
A list of `options` is sent as `criteria={label: None, ...}`. `confidence`, the Score level probabilities, and Noul `criteria` are not exposed.
|
||||
|
||||
#### Batched questions
|
||||
|
||||
Questions are sent in batches where possible:
|
||||
|
||||
1. While the script runs, each `jev` question is queued instead of being sent.
|
||||
2. Storing an answer in a variable or list, passing it to a function you defined, or calling `float()` on it keeps the run going. The run stops when the script first uses an answer's value: in an `if` condition, a comparison, arithmetic, an f-string, `sorted`, `join`, an output, and so on.
|
||||
3. The queued questions are sent, one request per `state`, and the script runs again from the start. Answered questions now return their values, and new questions are queued the same way.
|
||||
|
||||
Requests for different states are sent in parallel with the TypeSafe API, and one after another with a local model. A script asks the same questions and produces the same outputs as when each question is asked on its own. Within one run, every rerun starts from the same `random` state and `datetime.datetime.now()` time, so the questions do not change between reruns.
|
||||
|
||||
In this example, `q1` and `q2` go in one request because `a` is not used until the `if`. `q3` is sent in a second request after `a` is known:
|
||||
|
||||
```python
|
||||
a = jev.noul(it1, "q1")
|
||||
b = jev.noul(it1, "q2")
|
||||
if a > 0.5:
|
||||
c = jev.noul(it1, "q3")
|
||||
```
|
||||
|
||||
To ask many questions in one request, keep the `state` the same and put what differs between questions in the question text.
|
||||
|
||||
With the TypeSafe API, each request is one `system_one` call of the [TypeSafe Python SDK](https://pypi.org/project/typesafe-sdk/). The first request of the example above is:
|
||||
|
||||
```python
|
||||
client.system_one(state=it1, questions={"q0": Noul(instructions="q1"), "q1": Noul(instructions="q2")})
|
||||
```
|
||||
|
||||
Notes:
|
||||
- The TypeSafe API sends `state` and questions over the internet. Do not use it with text you cannot share externally.
|
||||
- A local question takes one forward pass (about 1–1.5 s with a 4B–9B model on GPU). The model is loaded on the first question of each run (several seconds) and unloaded when the run ends to free VRAM for the rest of the workflow.
|
||||
- Local probabilities are uncalibrated and depend on the model; test thresholds on your own inputs.
|
||||
- `MultiOutputScript` itself never loads a model or makes network requests; `jev` is not available there.
|
||||
- Each run prints `[ComfyUI-ScriptFlow] Jev requests: N, states: S, questions: Q, responses: M` to the console: requests sent, distinct states, questions asked, and requests answered. Fewer responses than requests means a request failed. Questions answered from the run's cache are not counted.
|
||||
- When `typesafe-sdk` is installed, the `httpx2` (the SDK's HTTP client) and `typesafe_sdk` loggers are set to WARNING, so per-request INFO lines are not shown.
|
||||
- `MultiOutputScript` never loads a model or makes network requests; `jev` is not available there.
|
||||
- This project is independent and not affiliated with TypeSafe AI.
|
||||
|
||||
### `centi`
|
||||
@@ -199,6 +229,7 @@ The script is parsed with Python AST and evaluated by ScriptFlow's safe interpre
|
||||
- `enumerate`, `range`, `zip`
|
||||
- `any`, `all`, `pow`, `divmod`
|
||||
- `list`, `dict`, `tuple`
|
||||
- `jev.yes`, `jev.noul`, `jev.choice`, `jev.probabilities`, `jev.score` (`MultiOutputScript (Jev)` only; see [its section](#multioutputscript-jev))
|
||||
|
||||
### Allowed namespaces
|
||||
- `random`: `random`, `randint`, `uniform`, `choice`
|
||||
@@ -219,6 +250,7 @@ The script is parsed with Python AST and evaluated by ScriptFlow's safe interpre
|
||||
|
||||
### Execution Notes
|
||||
- Using `random` or `datetime` makes outputs non-deterministic.
|
||||
- In v1.3.2 or later, `datetime.datetime.now()` and `datetime.date.today()` return the time the run started, so repeated calls in one script return the same value.
|
||||
- `datetime.strftime(...)` is supported in v1.2.0 or later for concise timestamp formatting:
|
||||
```python
|
||||
now = datetime.datetime.now()
|
||||
@@ -450,10 +482,9 @@ You should have received a copy of the GNU General Public License along with thi
|
||||
|
||||
See [CHANGELOG.md](CHANGELOG.md) for detailed version history.
|
||||
|
||||
### Current Version: 1.3.0
|
||||
- Added `MultiOutputScript (Jev)` node with a `jev` namespace for typed decisions in scripts
|
||||
- Added `jev.yes`, `jev.noul`, `jev.choice`, `jev.probabilities`, and `jev.score`
|
||||
- Added TypeSafe API backend with the API key read from `api_key.txt`
|
||||
- Added local GGUF backend through llama-cpp-python, adapted from SemIf (OpenJev)
|
||||
- Added a default script that picks one of eight aspect ratios and sizes it from megapixels
|
||||
- Kept `MultiOutputScript` and `centi` unchanged
|
||||
### Current Version: 1.3.2
|
||||
- `MultiOutputScript (Jev)` batches questions without script changes: questions about the same `state` are sent in one request, and requests for different states are sent in parallel
|
||||
- `max_jev_calls` now limits requests instead of questions
|
||||
- Added a console summary of Jev requests, states, questions, and responses after each run
|
||||
- Hid the per-request INFO log lines of the TypeSafe SDK and its HTTP client
|
||||
- `datetime.datetime.now()` and `datetime.date.today()` return the time the run started
|
||||
|
||||
+44
-8
@@ -16,8 +16,10 @@ from pathlib import Path
|
||||
from typing import Any, Dict
|
||||
|
||||
from .jev_backend import (
|
||||
UNANSWERED,
|
||||
JevBackend,
|
||||
LocalJev,
|
||||
NeedAnswers,
|
||||
TypeSafeClient,
|
||||
TypeSafeJev,
|
||||
list_local_models,
|
||||
@@ -194,11 +196,13 @@ class _SafeScriptInterpreter:
|
||||
def __init__(
|
||||
self,
|
||||
variables: Dict[str, Any],
|
||||
now: datetime.datetime,
|
||||
max_steps: int = _DEFAULT_MAX_STEPS,
|
||||
max_call_depth: int = _DEFAULT_MAX_CALL_DEPTH,
|
||||
jev: JevBackend | None = None,
|
||||
):
|
||||
self.frames = [variables]
|
||||
self.now = now
|
||||
self.jev = jev
|
||||
self.max_steps = max_steps
|
||||
self.max_call_depth = max_call_depth
|
||||
@@ -392,6 +396,9 @@ class _SafeScriptInterpreter:
|
||||
if name == "int":
|
||||
return int(*args)
|
||||
if name == "float":
|
||||
# float() of a queued probability stays queued so storing it does not end the batch.
|
||||
if len(args) == 1 and args[0] is UNANSWERED:
|
||||
return UNANSWERED
|
||||
return float(*args)
|
||||
if name == "str":
|
||||
return str(*args)
|
||||
@@ -500,11 +507,11 @@ class _SafeScriptInterpreter:
|
||||
if namespace == "datetime.datetime" and name == "now":
|
||||
if args:
|
||||
raise ValueError("datetime.datetime.now() expects no arguments")
|
||||
return datetime.datetime.now()
|
||||
return self.now
|
||||
if namespace == "datetime.date" and name == "today":
|
||||
if args:
|
||||
raise ValueError("datetime.date.today() expects no arguments")
|
||||
return datetime.date.today()
|
||||
return self.now.date()
|
||||
if namespace == "math" and name in _SAFE_MATH_FUNCTIONS:
|
||||
return self._call_math(name, args)
|
||||
if namespace == "jev":
|
||||
@@ -516,11 +523,13 @@ class _SafeScriptInterpreter:
|
||||
if len(args) not in (2, 3):
|
||||
raise ValueError("jev.yes() expects 2 or 3 arguments")
|
||||
threshold = args[2] if len(args) == 3 else 0.5
|
||||
return self.jev.noul(args[0], args[1]) >= threshold
|
||||
p = self.jev.noul(args[0], args[1])
|
||||
return p if p is UNANSWERED else p >= threshold
|
||||
if name == "noul":
|
||||
if len(args) != 2:
|
||||
raise ValueError("jev.noul() expects 2 arguments")
|
||||
return _JevProbability(self.jev.noul(*args))
|
||||
p = self.jev.noul(*args)
|
||||
return p if p is UNANSWERED else _JevProbability(p)
|
||||
if name == "choice":
|
||||
if len(args) != 3:
|
||||
raise ValueError("jev.choice() expects 3 arguments")
|
||||
@@ -680,6 +689,8 @@ class _SafeScriptInterpreter:
|
||||
return left > right
|
||||
if isinstance(op, ast.GtE):
|
||||
return left >= right
|
||||
if isinstance(op, (ast.Is, ast.IsNot)) and (left is UNANSWERED or right is UNANSWERED):
|
||||
raise NeedAnswers()
|
||||
if isinstance(op, ast.Is):
|
||||
return left is right
|
||||
if isinstance(op, ast.IsNot):
|
||||
@@ -868,7 +879,7 @@ class MultiOutputScript:
|
||||
in_value_3: Any = None,
|
||||
):
|
||||
return _run_script(
|
||||
code, None, in_text_1, in_text_2, in_text_3, in_value_1, in_value_2, in_value_3
|
||||
code, None, datetime.datetime.now(), in_text_1, in_text_2, in_text_3, in_value_1, in_value_2, in_value_3
|
||||
)
|
||||
|
||||
|
||||
@@ -926,16 +937,40 @@ class MultiOutputScriptJev(MultiOutputScript):
|
||||
inputs = (in_text_1, in_text_2, in_text_3, in_value_1, in_value_2, in_value_3)
|
||||
if model == _TYPESAFE_API:
|
||||
with _typesafe_client() as client:
|
||||
return _run_script(code, JevBackend(TypeSafeJev(client), max_jev_calls), *inputs)
|
||||
return _run_jev_script(code, JevBackend(TypeSafeJev(client), max_jev_calls, True), inputs)
|
||||
model_path = list_local_models().get(model)
|
||||
if model_path is None:
|
||||
raise ValueError(f"GGUF model not found under models/LLM or models/text_encoders: {model}")
|
||||
try:
|
||||
return _run_script(code, JevBackend(LocalJev(model_path), max_jev_calls), *inputs)
|
||||
return _run_jev_script(code, JevBackend(LocalJev(model_path), max_jev_calls, False), inputs)
|
||||
finally:
|
||||
unload_local_model()
|
||||
|
||||
|
||||
def _run_jev_script(code: str, jev: JevBackend, inputs: tuple):
|
||||
"""Reruns the script until it finishes, asking the questions queued before each stop in one batch.
|
||||
|
||||
Every rerun starts from the same random state and clock, so it asks the same questions as the run before.
|
||||
A script error while questions are queued may come from a queued answer, so those are asked and the script reruns.
|
||||
"""
|
||||
random_state = random.getstate()
|
||||
now = datetime.datetime.now()
|
||||
try:
|
||||
while True:
|
||||
random.setstate(random_state)
|
||||
try:
|
||||
return _run_script(code, jev, now, *inputs)
|
||||
except Exception:
|
||||
if not jev.pending:
|
||||
raise
|
||||
jev.flush()
|
||||
finally:
|
||||
print(
|
||||
f"[ComfyUI-ScriptFlow] Jev requests: {jev.requests}, states: {len(jev.states)}, "
|
||||
f"questions: {jev.questions}, responses: {jev.responses}"
|
||||
)
|
||||
|
||||
|
||||
def _typesafe_client() -> Any:
|
||||
if TypeSafeClient is None:
|
||||
raise RuntimeError("typesafe-sdk is not installed. Run: pip install typesafe-sdk")
|
||||
@@ -954,6 +989,7 @@ def _typesafe_client() -> Any:
|
||||
def _run_script(
|
||||
code: str,
|
||||
jev: JevBackend | None,
|
||||
now: datetime.datetime,
|
||||
in_text_1: Any,
|
||||
in_text_2: Any,
|
||||
in_text_3: Any,
|
||||
@@ -983,7 +1019,7 @@ def _run_script(
|
||||
"ov3": None,
|
||||
}
|
||||
|
||||
locals_dict = _SafeScriptInterpreter(locals_dict, jev=jev).run(code)
|
||||
locals_dict = _SafeScriptInterpreter(locals_dict, now, jev=jev).run(code)
|
||||
|
||||
_validate_outputs(locals_dict)
|
||||
|
||||
|
||||
+153
-49
@@ -13,7 +13,9 @@ from __future__ import annotations
|
||||
|
||||
import gc
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from typing import Any
|
||||
|
||||
import comfy.model_management
|
||||
@@ -21,13 +23,24 @@ import folder_paths
|
||||
import numpy as np
|
||||
|
||||
try:
|
||||
from llama_cpp import Llama, llama_get_logits_ith, llama_get_memory, llama_memory_clear
|
||||
from llama_cpp import (
|
||||
Llama,
|
||||
llama_batch_free,
|
||||
llama_batch_init,
|
||||
llama_decode,
|
||||
llama_get_logits_ith,
|
||||
llama_get_memory,
|
||||
llama_memory_clear,
|
||||
)
|
||||
from llama_cpp.llama_chat_format import Jinja2ChatFormatter
|
||||
except ImportError:
|
||||
Llama = None
|
||||
|
||||
try:
|
||||
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient
|
||||
# One INFO line pair per request floods the console; the run summary is printed instead.
|
||||
logging.getLogger("httpx2").setLevel(logging.WARNING)
|
||||
logging.getLogger("typesafe_sdk").setLevel(logging.WARNING)
|
||||
except ImportError:
|
||||
TypeSafeClient = None
|
||||
|
||||
@@ -42,77 +55,155 @@ _N_CTX = 8192
|
||||
_loaded_model: tuple[str, Any, Any] | None = None
|
||||
|
||||
|
||||
class JevBackend:
|
||||
"""Caches answers and limits model calls for a single script run."""
|
||||
class NeedAnswers(Exception):
|
||||
"""Raised when a script uses an answer that is still queued."""
|
||||
|
||||
def __init__(self, engine: Any, max_calls: int):
|
||||
|
||||
class _Unanswered:
|
||||
"""Stands in for a queued answer. Storing or passing it is fine; using its value raises NeedAnswers."""
|
||||
|
||||
def _need(self, *args: Any) -> Any:
|
||||
raise NeedAnswers()
|
||||
|
||||
__bool__ = __eq__ = __ne__ = __lt__ = __le__ = __gt__ = __ge__ = __hash__ = _need
|
||||
__str__ = __repr__ = __format__ = __float__ = __int__ = __index__ = _need
|
||||
__len__ = __iter__ = __contains__ = __getitem__ = _need
|
||||
|
||||
|
||||
UNANSWERED = _Unanswered()
|
||||
|
||||
|
||||
def _json_default(value: Any) -> Any:
|
||||
if value is UNANSWERED:
|
||||
raise NeedAnswers()
|
||||
raise TypeError(f"Object of type {type(value).__name__} is not JSON serializable")
|
||||
|
||||
|
||||
class JevBackend:
|
||||
"""Queues questions during a script run and asks them in batches, one request per state.
|
||||
|
||||
The caller reruns the script after each flush(); answered questions then return their values from the cache.
|
||||
max_calls limits the number of requests.
|
||||
"""
|
||||
|
||||
def __init__(self, engine: Any, max_calls: int, parallel: bool):
|
||||
self.engine = engine
|
||||
self.max_calls = max_calls
|
||||
self.calls = 0
|
||||
self.cache: dict[str, Any] = {}
|
||||
self.parallel = parallel
|
||||
self.requests = 0
|
||||
self.responses = 0
|
||||
self.questions = 0
|
||||
self.states: set[str] = set()
|
||||
self.cache: dict[str, list[float]] = {}
|
||||
self.pending: dict[str, tuple] = {}
|
||||
|
||||
def noul(self, state: Any, question: Any) -> float:
|
||||
return self._ask(state, "noul", question, None)[0]
|
||||
def noul(self, state: Any, question: Any) -> Any:
|
||||
probabilities = self._ask(state, "noul", question, None)
|
||||
return UNANSWERED if probabilities is None else probabilities[0]
|
||||
|
||||
def choice(self, state: Any, question: Any, options: list | dict) -> str:
|
||||
def choice(self, state: Any, question: Any, options: list | dict) -> Any:
|
||||
probabilities = self.probabilities(state, question, options)
|
||||
return max(probabilities, key=probabilities.get)
|
||||
return UNANSWERED if probabilities is UNANSWERED else max(probabilities, key=probabilities.get)
|
||||
|
||||
def probabilities(self, state: Any, question: Any, options: list | dict) -> dict[str, float]:
|
||||
def probabilities(self, state: Any, question: Any, options: list | dict) -> Any:
|
||||
if isinstance(options, list):
|
||||
options = {str(option): None for option in options}
|
||||
return dict(zip(options, self._ask(state, "choice", question, options)))
|
||||
probabilities = self._ask(state, "choice", question, options)
|
||||
return UNANSWERED if probabilities is None else dict(zip(options, probabilities))
|
||||
|
||||
def score(self, state: Any, question: Any, levels: list) -> float:
|
||||
def score(self, state: Any, question: Any, levels: list) -> Any:
|
||||
probabilities = self._ask(state, "score", question, list(levels))
|
||||
return sum(level * p for level, p in enumerate(probabilities))
|
||||
return UNANSWERED if probabilities is None else sum(level * p for level, p in enumerate(probabilities))
|
||||
|
||||
def _ask(self, state: Any, kind: str, question: Any, criteria: Any) -> list[float]:
|
||||
key = json.dumps([state, kind, question, criteria], sort_keys=True, ensure_ascii=False)
|
||||
def _ask(self, state: Any, kind: str, question: Any, criteria: Any) -> list[float] | None:
|
||||
key = json.dumps([state, kind, question, criteria], sort_keys=True, ensure_ascii=False, default=_json_default)
|
||||
if key in self.cache:
|
||||
return self.cache[key]
|
||||
if self.calls >= self.max_calls:
|
||||
raise RuntimeError(f"Script exceeded max_jev_calls ({self.max_calls})")
|
||||
self.calls += 1
|
||||
probabilities = self.engine.ask(state, kind, question, criteria)
|
||||
self.cache[key] = probabilities
|
||||
return probabilities
|
||||
self.pending[key] = (state, kind, question, criteria)
|
||||
return None
|
||||
|
||||
def flush(self) -> None:
|
||||
groups: dict[str, list[str]] = {}
|
||||
for key, (state, *_) in self.pending.items():
|
||||
groups.setdefault(json.dumps(state, sort_keys=True, ensure_ascii=False), []).append(key)
|
||||
if self.requests + len(groups) > self.max_calls:
|
||||
raise RuntimeError(
|
||||
f"Script needs more than max_jev_calls ({self.max_calls}) requests: "
|
||||
f"{self.requests} sent, {len(groups)} more needed"
|
||||
)
|
||||
self.requests += len(groups)
|
||||
self.questions += len(self.pending)
|
||||
self.states.update(groups)
|
||||
batches = list(groups.values())
|
||||
self.pending, pending = {}, self.pending
|
||||
if self.parallel and len(batches) > 1:
|
||||
with ThreadPoolExecutor(max_workers=len(batches)) as pool:
|
||||
futures = [pool.submit(self._send, pending, keys) for keys in batches]
|
||||
errors = [future.exception() for future in futures]
|
||||
self.responses += errors.count(None)
|
||||
for error in errors:
|
||||
if error is not None:
|
||||
raise error
|
||||
return
|
||||
for keys in batches:
|
||||
self._send(pending, keys)
|
||||
self.responses += 1
|
||||
|
||||
def _send(self, pending: dict[str, tuple], keys: list[str]) -> None:
|
||||
state = pending[keys[0]][0]
|
||||
answers = self.engine.ask(state, [pending[key][1:] for key in keys])
|
||||
for key, probabilities in zip(keys, answers):
|
||||
self.cache[key] = probabilities
|
||||
|
||||
|
||||
class LocalJev:
|
||||
"""Answers with a local GGUF model. Noul is asked as Yes/No options."""
|
||||
"""Answers with a local GGUF model, one forward pass per question. Noul is asked as Yes/No options."""
|
||||
|
||||
def __init__(self, model_path: str):
|
||||
self.model_path = model_path
|
||||
|
||||
def ask(self, state: Any, kind: str, question: Any, criteria: Any) -> list[float]:
|
||||
if kind == "noul":
|
||||
options = ["Yes", "No"]
|
||||
elif kind == "choice":
|
||||
options = [label if description is None else f"{label}: {description}" for label, description in criteria.items()]
|
||||
else:
|
||||
options = criteria
|
||||
return _option_probabilities(self.model_path, state, question, options)
|
||||
def ask(self, state: Any, questions: list[tuple]) -> list[list[float]]:
|
||||
answers = []
|
||||
for kind, question, criteria in questions:
|
||||
if kind == "noul":
|
||||
options = ["Yes", "No"]
|
||||
elif kind == "choice":
|
||||
options = [label if description is None else f"{label}: {description}" for label, description in criteria.items()]
|
||||
else:
|
||||
options = criteria
|
||||
answers.append(_option_probabilities(self.model_path, state, question, options))
|
||||
return answers
|
||||
|
||||
|
||||
class TypeSafeJev:
|
||||
"""Answers with TypeSafe's Jev API. Returns probabilities in the order of the options."""
|
||||
"""Answers with TypeSafe's Jev API, all questions about one state in one request.
|
||||
|
||||
Returns probabilities in the order of the options.
|
||||
"""
|
||||
|
||||
def __init__(self, client: Any):
|
||||
self.client = client
|
||||
|
||||
def ask(self, state: Any, kind: str, question: Any, criteria: Any) -> list[float]:
|
||||
if kind == "noul":
|
||||
p = self._answer(state, Noul(instructions=question)).noul
|
||||
return [p, 1.0 - p]
|
||||
if kind == "choice":
|
||||
probabilities = self._answer(state, Choice(instructions=question, criteria=criteria)).probabilities
|
||||
return [probabilities[label] for label in criteria]
|
||||
probabilities = self._answer(state, Score(instructions=question, criteria=criteria)).probabilities
|
||||
return [probabilities[level] for level in range(len(criteria))]
|
||||
|
||||
def _answer(self, state: Any, question: Any) -> Any:
|
||||
return self.client.system_one(state=state, questions={"q": question}).answers["q"]
|
||||
def ask(self, state: Any, questions: list[tuple]) -> list[list[float]]:
|
||||
typed = {}
|
||||
for i, (kind, question, criteria) in enumerate(questions):
|
||||
if kind == "noul":
|
||||
typed[f"q{i}"] = Noul(instructions=question)
|
||||
elif kind == "choice":
|
||||
typed[f"q{i}"] = Choice(instructions=question, criteria=criteria)
|
||||
else:
|
||||
typed[f"q{i}"] = Score(instructions=question, criteria=criteria)
|
||||
answers = self.client.system_one(state=state, questions=typed).answers
|
||||
results = []
|
||||
for i, (kind, _, criteria) in enumerate(questions):
|
||||
answer = answers[f"q{i}"]
|
||||
if kind == "noul":
|
||||
results.append([answer.noul, 1.0 - answer.noul])
|
||||
elif kind == "choice":
|
||||
results.append([answer.probabilities[label] for label in criteria])
|
||||
else:
|
||||
results.append([answer.probabilities[level] for level in range(len(criteria))])
|
||||
return results
|
||||
|
||||
|
||||
def _option_probabilities(model_path: str, state: Any, question: Any, options: list) -> list[float]:
|
||||
@@ -139,11 +230,23 @@ def _option_probabilities(model_path: str, state: Any, question: Any, options: l
|
||||
if len(encoded) != 1:
|
||||
raise ValueError(f"Model tokenizer does not encode {letter!r} as one token")
|
||||
slots.append(encoded[0])
|
||||
# reset() alone keeps the recurrent state of hybrid models such as Qwen3.5
|
||||
# Each question starts from empty memory, including the recurrent state of hybrid models such as Qwen3.5.
|
||||
# Decode with the llama.cpp API directly instead of the Llama evaluation helper,
|
||||
# whose method name trips the ComfyUI Registry dynamic-execution scanner.
|
||||
llama_memory_clear(llama_get_memory(llm.ctx), True)
|
||||
llm.reset()
|
||||
llm.eval(tokens)
|
||||
# Llama.scores is not filled without logits_all in upstream llama-cpp-python, so read the context directly
|
||||
batch = llama_batch_init(len(tokens), 0, 1)
|
||||
try:
|
||||
for index, token in enumerate(tokens):
|
||||
batch.token[index] = token
|
||||
batch.pos[index] = index
|
||||
batch.n_seq_id[index] = 1
|
||||
batch.seq_id[index][0] = 0
|
||||
batch.logits[index] = int(index == len(tokens) - 1)
|
||||
batch.n_tokens = len(tokens)
|
||||
if llama_decode(llm.ctx, batch) != 0:
|
||||
raise RuntimeError("llama_decode failed for the Jev prompt")
|
||||
finally:
|
||||
llama_batch_free(batch)
|
||||
vocabulary = np.ctypeslib.as_array(llama_get_logits_ith(llm.ctx, -1), shape=(llm.n_vocab(),))
|
||||
logits = vocabulary[slots].astype(np.float64)
|
||||
weights = np.exp(logits - logits.max())
|
||||
@@ -193,7 +296,8 @@ def load_local_model(model_path: str) -> tuple[Any, Any]:
|
||||
unload_local_model()
|
||||
device = comfy.model_management.get_torch_device()
|
||||
comfy.model_management.free_memory(os.path.getsize(model_path) * 1.2, device)
|
||||
llm = Llama(model_path=model_path, n_ctx=_N_CTX, n_gpu_layers=-1, verbose=False)
|
||||
# n_batch = n_ctx so one llama_decode call can take the whole prompt
|
||||
llm = Llama(model_path=model_path, n_ctx=_N_CTX, n_batch=_N_CTX, n_gpu_layers=-1, verbose=False)
|
||||
template = llm.metadata.get("tokenizer.chat_template")
|
||||
if not template:
|
||||
llm.close()
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "ComfyUI-ScriptFlow"
|
||||
description = "Safe Python-like script node with a System One decision model: ask yes/no, choice, and score questions via TypeSafe AI's Jev API, with local GGUF LLMs as a fallback. Multiple inputs and outputs let one node handle math, logic, text editing, conditional routing, resolution selection, filename generation, and more."
|
||||
version = "1.3.0"
|
||||
version = "1.3.2"
|
||||
license = {file = "LICENSE"}
|
||||
# classifiers = [
|
||||
# # For OS-independent nodes (works on all operating systems)
|
||||
|
||||
Reference in New Issue
Block a user