nodes/city96-ComfyUI_ExtraModels
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HunYuanDiT switch to local tokenizer

Browse Source
This commit is contained in:
City
2024-05-25 17:50:31 +02:00
parent 904e5bcacf
commit 4946b5038e
9 changed files with 47127 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
HunYuanDiT/mt5_tokenizer/special_tokens_map.json
+1
View File
@@ -0,0 +1 @@
{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>"}