Ways: refine with another model
refine_other_model redraws the made picture at denoise 0.4 with a second
checkpoint of its own (any family: its own CLIP and VAE, prompts copied from
the first, LoRAs added onto it with add_lora) and saves it beside it
(prefix refined), as the reference workflow's refiner keeps a loader and
LoRA stack apart. Tried on an RTX 4050 6 GB, Illustrious then NoobAI at
832x1216: about 66 s for the whole run, the composition kept, the style the
second model's.
A block node may be "like" a path ("$sampler.positive"): it takes the
settings of the node feeding it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DemonGatanjieu
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
ce4b8a8a49
commit
6ccb93e385
+1
-1
@@ -198,7 +198,7 @@ covered by the `styles.css` manifest.
|
||||
(`tests/test_reference_blocks.py`). Its links name roles that `web/modules/canvas_roles.js`
|
||||
finds on the canvas by their sockets (the sampler, its decode, the output node its picture
|
||||
reaches, what feeds an input upstream; several samplers: near_node_id); a node "like" a role
|
||||
takes its settings, one that "replaces" it its links and place too (LanPaint's sampler for the
|
||||
takes its settings (a path, "$sampler.positive", the node feeding it), one that "replaces" it its links and place too (LanPaint's sampler for the
|
||||
KSampler), before canvas_wiring splices it in (`tests/canvas_roles.mjs`). A block's "variants"
|
||||
rebuild it for the canvas model's family (`reference_blocks.build`: on Anima an LLLite model
|
||||
patch instead of a ControlNet), and an "optional" node of a pack that only improves it (KJNodes'
|
||||
|
||||
@@ -2,6 +2,9 @@
|
||||
|
||||
## v1.63.0 — unreleased
|
||||
|
||||
- 新做法 **换模型精修**：出完图以后，换另一个模型（可以是别的家族，配它自己的 LoRA）把整张图轻轻重画一遍（强度 0.4），另存一张（文件名以 refined 开头）。一个模型的构图加上另一个模型的画风和细节。新模型有自己的提示词节点，先照抄原来的提示词。实测 RTX 4050 6GB、Illustrious 出图 → NoobAI 精修，832×1216 整次约 66 秒。
|
||||
- New way **Refine with another model**: after a picture is made, another model (another family too, with its own LoRAs) redraws it lightly (0.4) and saves it beside it (prefix refined): one model's composition with another's style and detail. The new model has prompts of its own, copied from the first. About 66 s for a whole run, Illustrious then NoobAI at 832×1216, on an RTX 4050 6 GB.
|
||||
|
||||
- 新做法 **放进另一张图**：你递给 AI 两张图，一张有人物、一张是背景。AMB 抠出人物，把背景缩放裁切成同样大小，贴上人物，再把人物以外的地方轻轻重画一遍（强度 0.5），让脚下的影子、地面和光线接上，背景的样子基本不变。实测 RTX 4050 6GB、SDXL 832×1216 约 40 秒。做法现在可以接第二张图了（`add_nodes` 的 `scene`）。
|
||||
- New way **Put a character into a picture**: from two pictures handed to the AI, a character and a place, the character is cut out, the place scaled and cropped to its size, the character pasted on, and everything else lightly redrawn (0.5) so shadow, ground and light meet it while the place keeps its look. About 40 s for SDXL 832×1216 on an RTX 4050 6 GB. A block can take a second picture now (`add_nodes`' `scene`).
|
||||
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
{
|
||||
"job": "upscale",
|
||||
"rank": 6,
|
||||
"short": "other model",
|
||||
"title": {
|
||||
"zh": "换模型精修",
|
||||
"en": "Refine with another model"
|
||||
},
|
||||
"about": {
|
||||
"zh": "出完图以后，换另一个模型（可以是别的家族，配它自己的 LoRA）把整张图轻轻重画一遍（强度 0.4），存成另一张。用来把一个模型的构图和另一个模型的画风、细节拼在一起。新模型有自己的提示词节点，先照抄原来的提示词。",
|
||||
"en": "After a picture is made, another model (another family too, with its own LoRAs) redraws the whole picture lightly (0.4) and saves it beside it: one model's composition with another's style and detail. The new model has prompts of its own, copied from the first."
|
||||
},
|
||||
"source": {
|
||||
"zh": "参考工作流的「Refiner 管线」：精修时可以换一个底模加载器，配一套独立的 LoRA 堆",
|
||||
"en": "The reference workflow's refiner: a base model loader of its own for refining, with a separate LoRA stack"
|
||||
},
|
||||
"packs": {},
|
||||
"nodes": {
|
||||
"second_model": {
|
||||
"class_type": "CheckpointLoaderSimple",
|
||||
"like": "$loader",
|
||||
"inputs": {}
|
||||
},
|
||||
"second_positive": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"like": "$sampler.positive",
|
||||
"inputs": {
|
||||
"clip": [
|
||||
"second_model",
|
||||
1
|
||||
]
|
||||
}
|
||||
},
|
||||
"second_negative": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"like": "$sampler.negative",
|
||||
"inputs": {
|
||||
"clip": [
|
||||
"second_model",
|
||||
1
|
||||
]
|
||||
}
|
||||
},
|
||||
"encode": {
|
||||
"class_type": "VAEEncode",
|
||||
"inputs": {
|
||||
"pixels": [
|
||||
"$decode",
|
||||
0
|
||||
],
|
||||
"vae": [
|
||||
"second_model",
|
||||
2
|
||||
]
|
||||
}
|
||||
},
|
||||
"redraw": {
|
||||
"class_type": "KSampler",
|
||||
"like": "$sampler",
|
||||
"inputs": {
|
||||
"model": [
|
||||
"second_model",
|
||||
0
|
||||
],
|
||||
"positive": [
|
||||
"second_positive",
|
||||
0
|
||||
],
|
||||
"negative": [
|
||||
"second_negative",
|
||||
0
|
||||
],
|
||||
"latent_image": [
|
||||
"encode",
|
||||
0
|
||||
],
|
||||
"denoise": 0.4
|
||||
}
|
||||
},
|
||||
"decode": {
|
||||
"class_type": "VAEDecode",
|
||||
"inputs": {
|
||||
"samples": [
|
||||
"redraw",
|
||||
0
|
||||
],
|
||||
"vae": [
|
||||
"second_model",
|
||||
2
|
||||
]
|
||||
}
|
||||
},
|
||||
"save": {
|
||||
"class_type": "SaveImage",
|
||||
"inputs": {
|
||||
"images": [
|
||||
"decode",
|
||||
0
|
||||
],
|
||||
"filename_prefix": "refined"
|
||||
}
|
||||
}
|
||||
},
|
||||
"after": "The refine's loader (second_model) starts as the same checkpoint: set_model it to the one the user wants (ask_user, search_models), add_lora its own LoRAs onto it, and write its prompts (second_positive, second_negative, copied from the first) in that model's style. Another family may want other steps, cfg and sampler on redraw. Then run_workflow; the refined one is saved beside it (prefix refined). Denoise 0.4: lower keeps more of the first, higher takes more of the second.",
|
||||
"notes": "The source keeps a separate loader and LoRA stack for refining. Its own CLIP and VAE, so a model of another family works.",
|
||||
"models": [],
|
||||
"tested": {
|
||||
"gpu": "RTX 4050 Laptop",
|
||||
"vram_gb": 6,
|
||||
"date": "2026-10-28",
|
||||
"runs": [
|
||||
{
|
||||
"model": "SDXL (Illustrious) → NoobAI",
|
||||
"size": "832×1216",
|
||||
"seconds": 66
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -111,6 +111,9 @@ function txt2img() {
|
||||
assert.deepEqual([placed.d.kept, placed.d.feeds, placed.d.replaces],
|
||||
[{ vae: [`#${ckpt.id}`, 2] }, [{ output: 'IMAGE', to: stitch.id, input: 'inpainted_image' }], decode.id]);
|
||||
assert.deepEqual(placed.r, { class_type: 'KSampler', inputs: { denoise: 0.35 }, carry: { seed: 7, steps: 30 } }, 'like: settings only');
|
||||
pos.widgets = [{ name: 'text', value: 'a girl' }];
|
||||
assert.deepEqual(placeRoles(g, { p: { class_type: 'CLIPTextEncode', like: '$sampler.positive', inputs: {} } }).p.carry,
|
||||
{ text: 'a girl' }, 'like a path: the node feeding it');
|
||||
}
|
||||
|
||||
console.log('canvas_roles ok');
|
||||
|
||||
@@ -219,7 +219,7 @@ class BlockTests(unittest.TestCase):
|
||||
or value[0].split(".")[0] in ("$sampler", "$decode", "$save", "$loader"))
|
||||
for field in ("wire_to", "like", "replaces"):
|
||||
if field in spec:
|
||||
self.assertIn(spec[field], ("$sampler", "$decode", "$save", "$loader"))
|
||||
self.assertIn(spec[field].split(".")[0], ("$sampler", "$decode", "$save", "$loader")) # like may follow a path
|
||||
|
||||
|
||||
class AddBlockTests(unittest.TestCase):
|
||||
|
||||
@@ -107,12 +107,17 @@ export function placeRoles(g, nodes, nearId = null) {
|
||||
return path.length ? upstream(g, role(name, value[0]), path, value[0]) : [`#${role(name, value[0]).id}`, value[1]];
|
||||
};
|
||||
const target = value => (typeof value === 'string' && value.startsWith('$') ? role(value.slice(1), value).id : value);
|
||||
// "like" a path ("$sampler.positive"): the node feeding it (a second prompt with the first one's text).
|
||||
const nodeAt = (ref) => {
|
||||
const [name, ...path] = ref.slice(1).split('.');
|
||||
return path.length ? g.getNodeById(Number(upstream(g, role(name, ref), path, ref)[0].slice(1))) : role(name, ref);
|
||||
};
|
||||
const gone = new Set(Object.values(nodes).filter(spec => spec.replaces).map(spec => role(spec.replaces.slice(1), spec.replaces)));
|
||||
return Object.fromEntries(Object.entries(nodes).map(([key, { like, replaces, ...spec }]) => {
|
||||
const inputs = Object.fromEntries(Object.entries(spec.inputs || {}).map(([name, value]) => [name, link(value)]));
|
||||
const placed = { ...spec, inputs,
|
||||
...(spec.wire_to == null ? {} : { wire_to: Array.isArray(spec.wire_to) ? spec.wire_to.map(target) : target(spec.wire_to) }) };
|
||||
const model = like || replaces ? role((like || replaces).slice(1), like || replaces) : null;
|
||||
const model = like || replaces ? nodeAt(like || replaces) : null;
|
||||
if (model) placed.carry = settingsOf(model);
|
||||
if (replaces) Object.assign(placed, takenOver(g, model, inputs, gone));
|
||||
return [key, placed];
|
||||
|
||||
Reference in New Issue
Block a user