Eric HissandClaude Fable 5.1
4459af9ea6
Every node id, input name and existing widget position is unchanged, so saved
workflows load; new inputs are appended only.
Wildcards
- core/wildcards: Dynamic Prompts / Impact Pack grammar, tag queries, variables,
conditionals, concept / verbatim / loose bindings, reproducible seeds.
Wired into every enhancer node; reads ComfyUI/models/wildcards.
- New nodes: Wildcard Processor, Semantic Variation (direction axes),
Concept Survival Check. Starter lists in wildcards/.
- Random choices use a private generator; global random state is untouched.
Platform profiles
- core/profiles + profiles/*.yaml: one file per model (format, length,
negative prompt or none, quality tokens, text and dialogue syntax, vendor
rewriter prompt fetched once and cached). profiles_user/ overrides by id.
- New node: Platform Prompt Rewriter (prose, tags, edit instructions, JSON,
MiniMax H3 fields, Hailuo brackets), with validators for each format.
- Text-to-Image and Image-to-Image read profiles through platforms.py;
corrected facts for FLUX, Kontext, Chroma, Wan, AuraFlow, Kolors, NoobAI,
Qwen-Image, HunyuanImage. Negative prompt is empty for models without one.
- Hosted / API models: Seedream 5, Nano Banana, GPT Image, Midjourney V8,
FLUX.2 pro, Ideogram 3, Recraft V4, Seedance 2.0 / 2.5, Kling 3.0, Veo 3.1,
Gemini Omni Flash, Runway, Wan 3.0, Luma, hosted H3, Grok, Vidu, PixVerse.
Reports say how to switch the vendor's own prompt rewriting off.
- Illustrious: spaced tags, quality tags last, 25-80 words (OnomaAI paper).
Video nodes
- target_model and duration_seconds appended to the three video expanders.
Default target is MiniMax H3; wan22_video is the original path, unchanged.
A workflow saved earlier loads with the default until the input is set.
Also included: the Alice, Ideogram Magic Prompt and prompt-database nodes that
were already registered in __init__.py but had not been committed.
Fixes: v1 {animal} / {label:a|b} wildcards were inert in the video nodes;
Image-to-Video save_to_file raised NameError; T2I padded short-prompt models.
.gitattributes added (text=auto) so CRLF working files stop showing as
whole-file changes. prompt_database.db and the cached Ideogram prompt are
git-ignored.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VpCi6nw5UECMdZ7ayYJvN
104 lines
6.2 KiB
Markdown
104 lines
6.2 KiB
Markdown
# Ideogram 4 Magic Prompt - design & rationale notes
|
|
|
|
This documents the Ideogram 4 Magic Prompt node (and the schema decisions that
|
|
affect the Composer Studio + Style node work). Kept per the "base on the official
|
|
repo, improve with tailoring, keep notes on the rationale" plan.
|
|
|
|
## What it is
|
|
`IdeogramMagicPrompt` ("Ideogram 4 Magic Prompt") expands a plain idea into an
|
|
Ideogram 4 structured caption JSON, using a local LLM through the existing
|
|
`LLMBackend` (LM Studio / Ollama / local Qwen3-VL). Optional reference image is
|
|
captioned first (vision backend) and folded into the idea.
|
|
|
|
Files:
|
|
- `ideogram_magic_core.py` - engine (load prompt, call LLM, parse/repair JSON, reorder keys, dims).
|
|
- `ideogram_magic_node.py` - the ComfyUI node.
|
|
- `ideogram_prompts/v1_official.txt` - Ideogram's official system prompt (auto-downloaded + cached).
|
|
- `ideogram_prompts/v1_tailoring.txt` - our additive tailoring (toggle via `use_tailoring`).
|
|
|
|
## Source / license of the system prompt
|
|
`v1.txt` is Ideogram's **open-source** magic-prompt system prompt, Apache-2.0,
|
|
from https://github.com/ideogram-oss/ideogram4
|
|
(`src/ideogram4/magic_prompt_system_prompts/v1.txt`). It is downloaded and cached
|
|
to `ideogram_prompts/v1_official.txt` on first run; if the machine is offline,
|
|
place that file there manually (the node prints the URL). We ship no copy in git -
|
|
we fetch the upstream source of truth so it stays current. Integration code is
|
|
under this repo's own license.
|
|
|
|
## How it mirrors Ideogram's mechanics
|
|
Faithful to their `magic_prompt.py`:
|
|
1. Parse the file's `[SYSTEM]` / `[USER]` sections; the user template substitutes
|
|
`{{aspect_ratio}}` and `{{original_prompt}}`.
|
|
2. One OpenAI-style chat call (system + user). Temperature default 1.0.
|
|
3. Strip markdown code fences from the reply.
|
|
4. Reorder keys to the canonical schema order (style_description / compositional_
|
|
deconstruction / each element), so a CaptionVerifier stays quiet.
|
|
5. Drop the top-level `aspect_ratio` (we pass width/height separately, matching the
|
|
open-weight pipeline) but use the model's committed ratio to compute dims.
|
|
6. Optionally strip element bboxes (`keep_bboxes=false`). Ideogram strips by default;
|
|
we **keep by default** because our primary use is loading into the Composer canvas.
|
|
|
|
## KEY SCHEMA FINDING - v14 vs v15 (RESOLVED: Option C)
|
|
Ideogram's official magic prompt emits **three** top-level keys:
|
|
`{"aspect_ratio", "high_level_description", "compositional_deconstruction"}` -
|
|
**no `style_description`**. Style is carried as prose (in high_level / background /
|
|
element descs), with strong tuning baked into the prompt (e.g. the word "warm" is
|
|
*banned* as a grading adjective; default to an iPhone-snapshot look rather than
|
|
DSLR bokeh; "professional photo of a person" = corporate headshot, not studio rig).
|
|
This is the **v15 prose-style** convention.
|
|
|
|
By contrast:
|
|
- The open-weight `prompting.md` documents `style_description` (with photo XOR
|
|
art_style, strict key order, color_palette) as **optional**. (v14)
|
|
- Our **Ideogram 4 Composer + Style node** currently emit `style_description`. (v14)
|
|
|
|
Both are valid inputs to the open-weight model. But Ideogram's *own* shipped tooling
|
|
chose prose-only. Open question for the wider pipeline:
|
|
- **Option A (v15 / prose):** drop `style_description`; let the Style node produce
|
|
prose style guidance folded into high_level/background. Matches Ideogram's tuned
|
|
magic prompt; loses the explicit `color_palette` control surface.
|
|
- **Option B (v14 / style_description):** keep `style_description`; gives explicit
|
|
palette + medium control (a distinctive Ideogram-4 feature). Diverges from the
|
|
magic prompt's prose tuning, so the two paths produce stylistically different
|
|
captions unless reconciled.
|
|
- **Option C (both):** Composer supports either; magic prompt stays prose; Style
|
|
node optionally emits `style_description` for palette control. Most flexible,
|
|
most surface area.
|
|
|
|
**RESOLVED - Option C is implemented (2026-06).** In `Eric_Composer_studio`:
|
|
`core/ideogram_schema.py` gained `normalize_style` + `apply_style`; IDEOGRAM_STYLE
|
|
is now a wrapper `{mode: structured|prose|off, style_description, prose, medium_tag}`
|
|
(a legacy bare `style_description` dict is still accepted as structured). The
|
|
advanced `Ideogram 4 Style` node emits any of the three modes (prose folds into
|
|
`background` + a short medium tag onto `high_level_description`). Both composers
|
|
route style through `apply_style`, so they consume whichever a caption carries, and
|
|
the canvas JSON import preserves an imported `style_description` via a hidden
|
|
`imported_style_description` in `layout_data` (fallback when no Style node is wired),
|
|
so a structured caption round-trips faithfully. The magic-prompt node here still
|
|
emits faithful **prose (v15)**; prose imports carry their style in the
|
|
high-level/background text, so they round-trip too.
|
|
|
|
## Tailoring applied (v1)
|
|
Currently one additive instruction (`v1_tailoring.txt`): require a `bbox` on every
|
|
individually-placeable element, so a caption loads cleanly onto the Composer canvas
|
|
(the official prompt allows omitting bboxes; the canvas wants them present). It does
|
|
not change the schema or the style tuning. Toggle off with `use_tailoring=false`.
|
|
|
|
Future tailoring candidates (Option C chosen), each to be logged here
|
|
with its rationale before adoption:
|
|
- optional `style_description` emission (Option B/C) for palette control;
|
|
- tightening element-count or background discipline for our specific samplers.
|
|
|
|
## Wildcard support (2026-09)
|
|
`idea` now passes through the shared wildcard engine (`core/wildcards`) before expansion, so
|
|
`__name__`, `{a|b}`, tag queries and variables work here as in the other nodes. Two appended
|
|
optional inputs: `wildcard_seed` (-1 = fresh roll each run) and `wildcard_llm_block`.
|
|
|
|
Rationale for where the selected-elements block goes: `core.expand` gained `extra_system`, which
|
|
is appended to the SYSTEM prompt after the tailoring file. It is deliberately NOT appended to the
|
|
idea text. The official user template wraps the idea as the thing to depict, so an instruction
|
|
placed there risks being treated as content, and Ideogram renders quoted text. The official
|
|
system prompt and the schema are untouched; with no wildcards in the idea, `extra_system` is
|
|
empty and the call is byte-identical to before.
|
|
|