nodes/chflame163-ComfyUI_LayerStyle_Advance
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Use AutoTokenizer instead of BartTokenizerFast for broader model support

Browse Source 
Some Florence2 variants (e.g. Florence-2-Flux-Large) use a RoBERTa
tokenizer. AutoTokenizer auto-detects the correct tokenizer class.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Gang Wen
2026-04-07 19:57:01 +02:00
co-authored by Claude Opus 4.6
parent caafe797f7
commit 8d1bfd3874
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
py/florence2_ultra.py
+2 -2
View File
@@ -49,7 +49,7 @@ def _load_model_v5(model_path, attention, dtype):
"""Load Florence2 model for transformers >= 5.0.0"""
log(f"[DEBUG] _load_model_v5 called with model_path={model_path}, attention={attention}, dtype={dtype}")
from ..florence2_models.modeling_florence2 import Florence2ForConditionalGeneration, Florence2Config
from transformers import CLIPImageProcessor, BartTokenizerFast
from transformers import CLIPImageProcessor, AutoTokenizer
from ..florence2_models.processing_florence2 import Florence2Processor
from accelerate import init_empty_weights
from accelerate.utils import set_module_tensor_to_device
@@ -108,7 +108,7 @@ def _load_model_v5(model_path, attention, dtype):
image_processor.image_seq_length = 577
log(f"[DEBUG] Loading tokenizer from {model_path}")
tokenizer = BartTokenizerFast.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)
log(f"[DEBUG] Creating Florence2Processor")
processor = Florence2Processor(image_processor=image_processor, tokenizer=tokenizer)
log(f"[DEBUG] _load_model_v5 completed successfully")