Compare commits

...
Author SHA1 Message Date
newtextdoc1111 98d9fdfc50 fix: Bump version to 1.6.0 2025-10-26 03:30:50 +09:00
newtextdoc1111 504b681ab8 Merge pull request #42 from newtextdoc1111/dev
Merge Dev
2025-10-26 03:27:03 +09:00
newtextdoc1111 aca0ed6326 feat: Add setting to replace underscores with spaces in tag insertion (#39) 2025-10-24 06:16:11 +09:00
newtextdoc1111 ab941ab329 chore: update actions/setup-node to version 6 2025-10-24 06:03:52 +09:00
newtextdoc1111 c4e408eede fix: update Node.js version to 22 in CI workflow 2025-10-22 08:55:04 +09:00
newtextdoc1111 f8c86203e4 chore: update babel-jest and jest to version 30.2.0 2025-10-22 08:30:18 +09:00
newtextdoc1111 5a4ded44b8 feat: Add testing support for autocomplete functions 2025-10-22 08:29:48 +09:00
newtextdoc1111 295d0c0cdc fix: Add null checks to autocomplete functions 2025-10-22 04:15:13 +09:00
newtextdoc1111 37ce793208 fix: Include hyphens in tag encoder split regex 2025-10-22 04:06:56 +09:00
newtextdoc1111 2386fc7aa0 fix: Optimize CSV loading process 2025-10-22 04:01:59 +09:00
newtextdoc1111 937c5bb283 fix: Fix TagData creation for invalid tag in RelatedTagsUI 2025-10-19 23:39:45 +09:00
newtextdoc1111 512dd88d7a docs: Add display settings documentation 2025-10-18 05:19:31 +09:00
newtextdoc1111 096314933d fix: Bump version to 1.5.0 2025-10-18 02:58:02 +09:00
newtextdoc1111 05c890ee37 Merge pull request #41 from newtextdoc1111/dev
Merge Dev
2025-10-18 02:52:46 +09:00
newtextdoc1111 d4725dfebc feat: Add option to hide aliases in autocomplete and related tags display (#33) 2025-10-18 02:38:00 +09:00
newtextdoc1111 c4d5eee89d feat: Bump version to 1.4.1 2025-10-14 23:54:20 +09:00
newtextdoc1111 fa4d87ee77 Merge pull request #40 from newtextdoc1111/dev
Merge Dev
2025-10-14 23:49:43 +09:00
newtextdoc1111 3cb5a41cc8 fix: Temporarily comment out CSV update check button until remote file detection is properly implemented 2025-10-14 23:46:41 +09:00
newtextdoc1111 16a325b798 fix: Temporarily comment out remote file modification checks in the downloader 2025-08-30 23:40:27 +09:00
newtextdoc1111 e3171940fb fix: Removed the process that periodically checks for CSV file updates. 2025-08-30 23:39:52 +09:00
newtextdoc1111 8db37967f8 fix: Bump version to 1.4.0 2025-07-28 16:01:58 +09:00
newtextdoc1111 e43e51b48a Merge pull request #31 from newtextdoc1111/feature/#26_embedding_and_lora_autocompletion
Feature/#26 embedding and lora autocompletion
2025-07-28 16:00:37 +09:00
newtextdoc1111 7b89683bfc fix: Change to toLowerCase to simplify string normalization 2025-07-28 15:45:23 +09:00
newtextdoc1111 72b9bd94ee fix: Improved the encoder to provide more accurate suggestions 2025-07-28 14:05:49 +09:00
newtextdoc1111 8a4a94bc7e docs: Added Lora and Emb suggestions to README 2025-07-28 05:36:53 +09:00
newtextdoc1111 67b7b334b8 feat: Add embeddings and loras autocompletion 2025-07-28 05:21:57 +09:00
newtextdoc1111 9cdc18e566 Merge pull request #30 from newtextdoc1111/feature/improved_fast_search_behavior
Feature/improved fast search behavior
2025-07-24 03:59:49 +09:00
19 changed files with 2532 additions and 927 deletions
+2 -2
View File
@@ -12,9 +12,9 @@ jobs:
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v6
with:
node-version: '18'
node-version: '22'
cache: 'npm'
- name: Install dependencies
+5 -2
View File
@@ -41,6 +41,7 @@ When you type in a text input area, tags that partially match the text are displ
- Tags are color-coded by category. The color-coding rules are the same as Danbooru.
- Tags that have already been entered are displayed grayed out.
- You can display Danbooru and e621 tags at the same time. You can also change the priority from the settings.
- Supports autocomplete for Lora and Embedding inputs. You can enable/disable this feature in the settings.
## Related Tags
@@ -132,6 +133,8 @@ When the browser is reloaded, you can check the list of loaded CSV files in the
- **Enable Autocomplete**: Enable/disable the autocomplete feature.
- **Max suggestions**: Maximum number of autocomplete suggestions to display.
- **Replace '_' with 'Space'**: Replaces underscores with spaces when inserting tags. This setting also affects related tag display.
- **Enable Loras and Embeddings**: Display Lora and Embedding in the suggestions.
- **Use Fast Search**: Switch autocomplete suggestions search to fast processing (see [About Fast Search for Autocomplete](#about-fast-search-for-autocomplete) for details).
### Related Tags
@@ -141,9 +144,9 @@ When the browser is reloaded, you can check the list of loaded CSV files in the
- **Default Display Position**: Default display position when ComfyUI starts.
- **Related Tags Trigger Mode**: Which action will trigger displaying related tags (click only, Ctrl+click)
### Miscellaneous
### Display
- **Check CSV updates**: Click the "Check Now" button to check if new CSV files are available in HuggingFace and download them if necessary.
- **Hide Alias**: Hide/show the Alias ​​column in autocomplete and related tags (default is show)
## Advanced Settings
+5 -2
View File
@@ -39,6 +39,7 @@
- タグのカテゴリ毎に色分けされます。色分けのルールは Danbooru と同じです
- 入力済みのタグはグレーアウトで表示されます
- Danbooruとe621のタグを同時に表示出来ます。設定から優先順位を変更できます
- LoraとEmbeddingの入力補完に対応しています。設定から有効・無効を切り替えられます
## 関連タグ
@@ -130,6 +131,8 @@ worst_quality,5,9999999,
- **Enable Autocomplete**: オートコンプリート機能の有効化/無効化
- **Max Suggestions**: オートコンプリート候補の最大表示件数
- **Replace '_' with 'Space'**: タグ挿入時にアンダースコアをスペースに置き換えます。この設定は関連タグ表示にも影響します
- **Enable Loras and Embeddings**: LoraとEmbeddingを候補に表示する
- **Use Fast Search**: オートコンプリート候補の検索を高速な処理に切り替える(詳細は [オートコンプリートの高速検索について](#オートコンプリートの高速検索について) を確認してください)
### 関連タグ
@@ -139,9 +142,9 @@ worst_quality,5,9999999,
- **Default Display Position**: ComfyUI起動時のデフォルト表示位置
- **Related Tags Trigger Mode** : 関連タグを表示する際、どの操作をトリガーとするか(クリックのみ、Ctrl+クリック)
### その他
### 表示
- **Check CSV updates**: 「Check Now」ボタンを押すと新しい CSV ファイルがHuggingFaceにあるか確認し、必要に応じてダウンロードを行います
- **Hide Alias**: オートコンプリートと関連タグで表示されるエイリアス列の非表示/表示を切り替え(デフォルトは表示です)
## 上級者向け設定
+22 -1
View File
@@ -1,7 +1,10 @@
import os
import json
import os
import folder_paths
import server
from aiohttp import web
from . import downloader as dl
# Get the absolute path to the 'data' directory
@@ -227,3 +230,21 @@ async def get_last_check_time(_request):
except (IOError, json.JSONDecodeError) as e:
print(f"[Autocomplete-Plus] Error reading csv_meta.json: {e}")
return web.json_response({"last_check_time": None, "error": str(e)}, status=500)
@server.PromptServer.instance.routes.get("/autocomplete-plus/embeddings")
async def get_embeddings(request):
"""
Returns a list of embedding files.
"""
embeddings = folder_paths.get_filename_list("embeddings")
return web.json_response(list(map(lambda a: os.path.splitext(a)[0], embeddings)))
@server.PromptServer.instance.routes.get("/autocomplete-plus/loras")
async def get_loras(request):
"""
Returns a list of lora files.
"""
loras = folder_paths.get_filename_list("loras")
return web.json_response(list(map(lambda a: os.path.splitext(a)[0], loras)))
+11 -19
View File
@@ -235,16 +235,7 @@ class Downloader:
def _check_new_csv_from_hf_dataset(self, dataset_meta: dict, now_utc: datetime, force_check: bool = False):
"""Checks HuggingFace for file updates and updates metadata."""
perform_hf_check = True
if not force_check and dataset_meta.get("last_remote_check_timestamp"):
try:
last_check_dt = datetime.fromisoformat(dataset_meta["last_remote_check_timestamp"])
if now_utc - last_check_dt < timedelta(days=7):
perform_hf_check = False
except (ValueError, KeyError, TypeError):
print(
"[Autocomplete-Plus] Invalid or missing timestamp for last_remote_check_timestamp. Will perform remote check."
)
perform_hf_check = force_check
if perform_hf_check:
huggingface_dataset_id = dataset_meta["hf_dataset_id"]
@@ -303,15 +294,16 @@ class Downloader:
return f"File {file_name} is missing or empty locally."
# Check if the last modified date on HuggingFace is newer than the last download date
try:
last_download_dt = datetime.fromisoformat(file_meta_entry["last_download"])
hf_modified_dt = datetime.fromisoformat(file_meta_entry["last_modified_on_hf"])
if hf_modified_dt > last_download_dt:
return f"Remote file {file_name} is newer (HF: {last_download_dt}, Local Download: {hf_modified_dt})."
except (ValueError, TypeError):
file_meta_entry["last_download"] = None
file_meta_entry["last_modified_on_hf"] = None
return f"Invalid timestamp format for {file_name}. Forcing download to ensure integrity."
# Temporarily comment out until changes to remote files can be detected
# try:
# last_download_dt = datetime.fromisoformat(file_meta_entry["last_download"])
# hf_modified_dt = datetime.fromisoformat(file_meta_entry["last_modified_on_hf"])
# if hf_modified_dt > last_download_dt:
# return f"Remote file {file_name} is newer (HF: {hf_modified_dt}, Local Download: {last_download_dt})."
# except (ValueError, TypeError):
# file_meta_entry["last_download"] = None
# file_meta_entry["last_modified_on_hf"] = None
# return f"Invalid timestamp format for {file_name}. Forcing download to ensure integrity."
# If the file is missing or empty, but the last download timestamp exists, we need to retry.
if not check_file_valid(local_file_path) and file_meta_entry.get("last_download") is not None:
+1501 -739
View File
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -6,9 +6,9 @@
"devDependencies": {
"@babel/core": "^7.27.1",
"@babel/preset-env": "^7.27.2",
"babel-jest": "^29.7.0",
"babel-jest": "^30.2.0",
"env-cmd": "^10.1.0",
"jest": "^29.7.0",
"jest": "^30.2.0",
"stylelint": "^16.19.1",
"stylelint-config-idiomatic-order": "^10.0.0",
"stylelint-config-standard": "^38.0.0"
+1 -1
View File
@@ -1,7 +1,7 @@
[project]
name = "comfyui-autocomplete-plus"
description = "Autocomplete and Related Tag display for ComfyUI"
version = "1.3.1"
version = "1.6.0"
license = {file = "LICENSE"}
dependencies = ["",]
+378
View File
@@ -0,0 +1,378 @@
import {
__test__
} from "../../web/js/autocomplete.js";
import {
TagData,
TagSource,
ModelTagSource,
autoCompleteData
} from "../../web/js/data.js";
import {
createFlexSearchDocument,
createFlexSearchDocumentForModel
} from "../../web/js/searchengine.js"
const {
searchCompletionCandidates,
sequentialSearch,
searchWithFlexSearch,
matchWord,
getCurrentPartialTag,
insertTagToTextArea
} = __test__;
// Helper function to create mock textarea element
function createMockTextarea(value, selectionStart, selectionEnd = -1) {
return {
value: value,
selectionStart: selectionStart,
selectionEnd: selectionEnd > selectionStart ? selectionEnd : selectionStart,
nodeName: 'TEXTAREA',
focus() { document.setFocus(this); },
setSelectionRange: () => { },
getBoundingClientRect: () => ({
top: 0,
left: 0,
width: 300,
height: 100
}),
scrollTop: 0,
scrollLeft: 0,
ownerDocument: {
defaultView: typeof window !== 'undefined' ? window : {},
documentElement: typeof document !== 'undefined' ? document.documentElement : {}
},
dispatchEvent: () => { }
};
}
// Test data setup function
function setupTestData() {
// Clear existing data
Object.keys(autoCompleteData).forEach(key => delete autoCompleteData[key]);
// Create mock AutocompleteData structure for each source
const sources = [...Object.values(TagSource), ...Object.values(ModelTagSource)];
sources.forEach(source => {
autoCompleteData[source] = {
flexSearchDocument: null,
sortedTags: [],
tagMap: new Map(),
aliasMap: new Map(),
cooccurrenceMap: new Map(),
isInitializing: false,
initialized: true,
baseLoadingProgress: {
cooccurrence: 0
}
};
});
// Add sample tag data for testing
const sampleTags = [
new TagData('1girl', 0, 1000000, ['one_girl'], TagSource.Danbooru),
new TagData('blue_hair', 0, 500000, ['blue hair'], TagSource.Danbooru),
new TagData('long_hair', 0, 800000, [], TagSource.Danbooru),
new TagData('__wildcard__', 0, 100, [], TagSource.Danbooru),
new TagData(':d', 0, 50000, [], TagSource.Danbooru),
new TagData('test_tag', 0, 1000, ['test'], TagSource.E621),
new TagData('<lora:test_model>', 0, 0, [], ModelTagSource.Lora),
new TagData('embedding:test_embedding', 0, 0, [], ModelTagSource.Embeddings)
];
// Distribute tags across sources
sampleTags.forEach(tagData => {
const source = tagData.source;
autoCompleteData[source].sortedTags.push(tagData);
autoCompleteData[source].tagMap.set(tagData.tag, tagData);
// Add aliases to alias map
if (tagData.alias && Array.isArray(tagData.alias)) {
tagData.alias.forEach(alias => {
autoCompleteData[source].aliasMap.set(alias, tagData.tag);
});
}
});
// Sort tags and build flexsearch index
sources.forEach(source => {
autoCompleteData[source].sortedTags.sort((a, b) => b.count - a.count);
const isModelSrc = Object.values(ModelTagSource).includes(source);
const doc = isModelSrc ? createFlexSearchDocumentForModel() : createFlexSearchDocument();
autoCompleteData[source].sortedTags.forEach((tagData, i) => {
doc.add(i, tagData);
});
autoCompleteData[source].flexSearchDocument = doc;
});
}
describe('Autocomplete Functions', () => {
beforeEach(() => {
setupTestData();
// Mock global document
global.document = {
execCommand: (commandId, showUI, value) => {
switch (commandId) {
case "insertText":
if (document.focusedElement) {
document.focusedElement.value += value;
}
break;
default:
throw new Error('Not implemented!');
}
},
createElement: () => ({
id: '',
style: {},
innerHTML: '',
appendChild: () => { },
getBoundingClientRect: () => ({
top: 0,
left: 0,
width: 300,
height: 100
})
}),
setFocus: (element) => {
document.focusedElement = element;
},
body: {
appendChild: () => { },
removeChild: () => { }
},
focusedElement: null
};
// Mock global window
global.window = {
getComputedStyle: () => ({
lineHeight: '20px',
fontSize: '14px',
fontFamily: 'Arial'
})
};
});
describe('matchWord', () => {
test('should match exact queries', () => {
const queries = new Set(['1girl', '1girls']);
const result = matchWord('1girl', queries);
expect(result.matched).toBe(true);
expect(result.isExactMatch).toBe(true);
});
test('should match partial queries', () => {
const queries = new Set(['girl']);
const result = matchWord('1girl', queries);
expect(result.matched).toBe(true);
expect(result.isExactMatch).toBe(false);
});
test('should handle wildcard prefixes', () => {
const queries = new Set(['__wild']);
const result = matchWord('__wildcard__', queries);
expect(result.matched).toBe(true);
expect(result.isExactMatch).toBe(false);
});
test('should handle symbol-only queries', () => {
const queries = new Set([':d']);
const result = matchWord(':d', queries);
expect(result.matched).toBe(true);
expect(result.isExactMatch).toBe(true);
});
test('should match after removing common symbols', () => {
const queries = new Set(['blue hair']);
const result = matchWord('blue_hair', queries);
expect(result.matched).toBe(true);
expect(result.isExactMatch).toBe(false);
});
test('should not match unrelated queries', () => {
const queries = new Set(['test']);
const result = matchWord('1girl', queries);
expect(result.matched).toBe(false);
expect(result.isExactMatch).toBe(false);
});
});
describe('getCurrentPartialTag', () => {
test('should extract partial tag before cursor', () => {
const textarea = createMockTextarea('1girl, blue_hair', 14);
const result = getCurrentPartialTag(textarea);
expect(typeof result).toBe('string');
expect(result).toBe('blue_ha');
});
test('should return full tag when cursor is at the end', () => {
const textarea = createMockTextarea('1girl', 5);
const result = getCurrentPartialTag(textarea);
expect(typeof result).toBe('string');
expect(result).toBe('1girl');
});
test('should handle empty textarea', () => {
const textarea = createMockTextarea('', 0);
const result = getCurrentPartialTag(textarea);
expect(typeof result).toBe('string');
expect(result).toBe('');
});
test('should handle newline separators', () => {
const textarea = createMockTextarea('1girl\nblue_h', 12);
const result = getCurrentPartialTag(textarea);
expect(typeof result).toBe('string');
expect(result).toBe('blue_h');
});
});
describe('searchCompletionCandidates', () => {
test('should return an empty array for empty input', () => {
const textarea = createMockTextarea('', 0);
const results = searchCompletionCandidates(textarea);
expect(Array.isArray(results)).toBe(true);
expect(results.length).toBe(0);
});
test('should return an empty array for null input', () => {
const results = searchCompletionCandidates(null);
expect(Array.isArray(results)).toBe(true);
expect(results.length).toBe(0);
});
test('should return an empty array for undefined input', () => {
const results = searchCompletionCandidates(undefined);
expect(Array.isArray(results)).toBe(true);
expect(results.length).toBe(0);
});
test('should return multiple candidates for valid input', () => {
const textarea = createMockTextarea('test', 4);
const results = searchCompletionCandidates(textarea);
expect(Array.isArray(results)).toBe(true);
expect(results.map(tagData => tagData.tag))
.toEqual(expect.arrayContaining([
'test_tag', '<lora:test_model>', 'embedding:test_embedding'
]));
});
test('should find wildcard tag', () => {
const textarea = createMockTextarea('__wild', 6);
const results = searchCompletionCandidates(textarea);
expect(Array.isArray(results)).toBe(true);
expect(results.map(tagData => tagData.tag))
.toEqual(expect.arrayContaining(['__wildcard__']));
});
test('should find emoticon tag', () => {
const textarea = createMockTextarea(':d', 2);
const results = searchCompletionCandidates(textarea);
expect(Array.isArray(results)).toBe(true);
expect(results.map(tagData => tagData.tag))
.toEqual(expect.arrayContaining([':d']));
});
});
describe('sequentialSearch', () => {
test('should find and return matching tags', () => {
const partialTag = 'test';
const queryVariations = new Set([partialTag.toLowerCase()]);
const results = sequentialSearch(partialTag, queryVariations);
expect(Array.isArray(results)).toBe(true);
expect(results.map(tagData => tagData.tag))
.toEqual(expect.arrayContaining([
'test_tag', '<lora:test_model>', 'embedding:test_embedding'
]));
});
});
describe('searchWithFlexSearch', () => {
test('should find and return matching tags using FlexSearch', () => {
const partialTag = 'test';
const queryVariations = new Set([partialTag.toLowerCase()]);
const results = searchWithFlexSearch(partialTag, queryVariations);
expect(Array.isArray(results)).toBe(true);
expect(results.map(tagData => tagData.tag))
.toEqual(expect.arrayContaining([
'test_tag', '<lora:test_model>', 'embedding:test_embedding'
]));
});
});
describe('insertTagToTextArea', () => {
test('should insert tag and replace underscore with space', () => {
const textarea = createMockTextarea('1girl, ', 7);
const tagData = { tag: 'blue_hair', source: 'danbooru' };
expect(() => {
insertTagToTextArea(textarea, tagData);
}).not.toThrow();
expect(textarea.value).toBe('1girl, blue hair, ');
});
test('should complete model tag and add trailing comma', () => {
const textarea = createMockTextarea('<lora:', 6);
const tagData = { tag: '<lora:my_model>', source: 'lora' };
expect(() => {
insertTagToTextArea(textarea, tagData);
}).not.toThrow();
expect(textarea.value).toBe('<lora:my_model>, ');
});
test('should handle wildcard tag insertion', () => {
const textarea = createMockTextarea('__wild', 6);
const tagData = { tag: '__wildcard__', source: 'danbooru' };
insertTagToTextArea(textarea, tagData);
expect(textarea.value).toBe('__wildcard__, ');
});
test('should handle emoticon tag insertion', () => {
const textarea = createMockTextarea(':d', 2);
const tagData = { tag: ':d', source: 'danbooru' };
insertTagToTextArea(textarea, tagData);
expect(textarea.value).toBe(':d, ');
});
});
});
+81 -18
View File
@@ -1,10 +1,11 @@
import {
import {
createFlexSearchDocument,
createFlexSearchDocumentForModel,
__test__
} from "../../web/js/searchengine.js";
} from "../../web/js/searchengine.js";
const { createTagEncoder, createCJKEncoder } = __test__;
const { createTagEncoder, createCJKEncoder, createModelEncoder } = __test__;
function parseCSVLine(line) {
const result = [];
@@ -60,34 +61,44 @@ sanshoku_dango,0,2061,"三色団子,三色团子,花見団子,花见团子"
year:1999,0,1999,
d.d.,0,1999,
copyright_(series),2,1298,"copyright,コピーライト (シリーズ),コピーライト名,コピーライト,著作"
__wildcard__,0,0,
`;
const ControlCSV = `
__wildcard__,0,1000,
<lora:my_lora1:1.0>,0,1000,
Embedding: my_embedding,0,1000,
const ModelCSV = `
<lora:my_lora1>,0,0,
<lora:日本語Lora_v1>,0,0,
embedding: my_embedding,0,0,
`;
const mockCSV = [
commonCSV, cjkAliasCSV, specialCharCSV, ControlCSV
commonCSV, cjkAliasCSV, specialCharCSV
].map(csv => csv.trim()).join('\n');
let mockTags;
let tagEncoder, cjkEncoder;
let document;
let mockTags, mockModelTags;
let tagEncoder, cjkEncoder, modelEncoder;
let document, modelDocument;
let performSearch = function (query, limit = 100) {
const results = document.search(query, {
const ids1 = document.search(query, {
field: ["tag", "alias"],
limit: limit,
limit: limit,
suggest: false,
merge: true,
});
}).map(r => r.id);
const ids = results.map(r => r.id);
const result1 = mockTags.filter(tag => ids1.includes(tag.id)).map(tag => tag.tag);
return mockTags.filter(tag => ids.includes(tag.id)).map(tag => tag.tag);
const ids2 = modelDocument.search(query, {
field: ["tag", "alias"],
limit: limit,
suggest: false,
merge: true,
}).map(r => r.id);
const result2 = mockModelTags.filter(tag => ids2.includes(tag.id)).map(tag => tag.tag);
return [...result1, ...result2];
}
beforeEach(() => {
@@ -102,6 +113,15 @@ Embedding: my_embedding,0,1000,
document = createFlexSearchDocument();
mockTags.forEach(data => document.add(data));
mockModelTags = ModelCSV.split('\n').map((line, id) => {
const [tag, category, count, alias] = parseCSVLine(line);
return { id, tag, category: parseInt(category), count: parseInt(count), alias };
});
modelEncoder = createModelEncoder();
modelDocument = createFlexSearchDocumentForModel();
mockModelTags.forEach(data => modelDocument.add(data));
});
describe('Encoder', () => {
@@ -134,6 +154,31 @@ Embedding: my_embedding,0,1000,
const encoded = tagEncoder.encode('one_two_');
expect(encoded).toEqual(['one', 'two']);
});
test('should properly encode embedding notation', () => {
let encoded = modelEncoder.encode('embedding:path/to/my_embed1');
expect(encoded).toEqual(['embedding:', 'path', 'to', 'my', 'embed1']);
expect(
modelEncoder.encode('embedding:path\\to\\my-embed1')
).toEqual(['embedding:', 'path', 'to', 'my', 'embed1']);
expect(
modelEncoder.encode('embedding:path\\to\\this is my embed. my-negative01 (v1)__by me')
).toEqual(['embedding:', 'path', 'to', 'this', 'is', 'my', 'embed', 'my', 'negative01', 'v1', 'by', 'me']);
});
test('should properly encode lora notation', () => {
expect(
modelEncoder.encode('<lora:path/to/my_lora1>')
).toEqual(['lora:', 'path', 'to', 'my', 'lora1']);
expect(
modelEncoder.encode('<lora:path\\to\\my-lora1>')
).toEqual(['lora:', 'path', 'to', 'my', 'lora1']);
expect(
modelEncoder.encode('<lora:path\\to\\this is my lora. my-style01 (v1)__by me>')
).toEqual(['lora:', 'path', 'to', 'this', 'is', 'my', 'lora', 'my', 'style01', 'v1', 'by', 'me']);
});
});
describe('Basic Search', () => {
@@ -298,9 +343,27 @@ Embedding: my_embedding,0,1000,
test('should match to lora tag', () => {
const tag = '<lora';
const results = performSearch(tag);
expect(results.length).toEqual(2);
expect(results).toContain("<lora:my_lora1>");
expect(results).toContain("<lora:日本語Lora_v1>");
});
test('should match to lora tag2', () => {
const tag = 'lora:';
const results = performSearch(tag);
expect(results.length).toEqual(2);
expect(results).toContain("<lora:my_lora1>");
expect(results).toContain("<lora:日本語Lora_v1>");
});
test('should match to lora that contain CJK characters', () => {
const word = 'lora: 日本語';
const results = performSearch(word);
expect(results.length).toEqual(1);
expect(results).toContain("<lora:my_lora1:1.0>");
expect(results).toContain("<lora:日本語Lora_v1>");
});
});
+58
View File
@@ -1,3 +1,4 @@
import { settingValues } from '../../web/js/settings.js';
import {
extractTagsFromTextArea,
normalizeTagToSearch,
@@ -135,6 +136,11 @@ describe('normalizeTagToSearch', () => {
});
describe('normalizeTagToInsert', () => {
const originalValue = settingValues.replaceUnderscoreWithSpace;
afterEach(() => {
settingValues.replaceUnderscoreWithSpace = originalValue;
});
test('should return null or empty string for invalid inputs', () => {
expect(normalizeTagToInsert(null)).toBeNull();
expect(normalizeTagToInsert(undefined)).toBeUndefined();
@@ -167,6 +173,58 @@ describe('normalizeTagToInsert', () => {
expect(normalizeTagToInsert('year:2000')).toBe('year:2000');
expect(normalizeTagToInsert('foo:bar')).toBe('foo:bar');
});
test('should respect replaceUnderscoreWithSpace setting when true', () => {
settingValues.replaceUnderscoreWithSpace = true;
expect(normalizeTagToInsert('blue_hair')).toBe('blue hair');
expect(normalizeTagToInsert('red_eyes')).toBe('red eyes');
expect(normalizeTagToInsert('long_curly_hair')).toBe('long curly hair');
});
test('should respect replaceUnderscoreWithSpace setting when false', () => {
settingValues.replaceUnderscoreWithSpace = false;
expect(normalizeTagToInsert('blue_hair')).toBe('blue_hair');
expect(normalizeTagToInsert('red_eyes')).toBe('red_eyes');
expect(normalizeTagToInsert('long_curly_hair')).toBe('long_curly_hair');
});
test('should respect replaceUnderscoreWithSpace setting with parentheses', () => {
settingValues.replaceUnderscoreWithSpace = true;
expect(normalizeTagToInsert('blue_hair(style)')).toBe('blue hair\\(style\\)');
settingValues.replaceUnderscoreWithSpace = false;
expect(normalizeTagToInsert('blue_hair(style)')).toBe('blue_hair\\(style\\)');
settingValues.replaceUnderscoreWithSpace = originalValue;
});
test('should not replace underscores in wildcard syntax regardless of setting', () => {
const originalValue = settingValues.replaceUnderscoreWithSpace;
// Wildcard syntax should preserve underscores regardless of setting
settingValues.replaceUnderscoreWithSpace = true;
expect(normalizeTagToInsert('__wildcard__')).toBe('__wildcard__');
settingValues.replaceUnderscoreWithSpace = false;
expect(normalizeTagToInsert('__wildcard__')).toBe('__wildcard__');
settingValues.replaceUnderscoreWithSpace = originalValue;
});
test('should not replace underscores in symbol-only tags regardless of setting', () => {
const originalValue = settingValues.replaceUnderscoreWithSpace;
// Symbol-only tags should not be affected by the setting
settingValues.replaceUnderscoreWithSpace = true;
expect(normalizeTagToInsert('^_^')).toBe('^_^');
settingValues.replaceUnderscoreWithSpace = false;
expect(normalizeTagToInsert('^_^')).toBe('^_^');
settingValues.replaceUnderscoreWithSpace = originalValue;
});
});
describe('getCurrentTagRange', () => {
+10 -2
View File
@@ -97,10 +97,14 @@ body.dark-theme {
display: grid;
box-shadow: 0 2px 8px rgb(0 0 0 / 30%);
grid-auto-rows: auto;
grid-template-columns: max-content 1fr auto auto;
grid-template-columns: max-content 1fr auto;
overflow-y: auto;
}
#autocomplete-plus-list.no-alias {
grid-template-columns: max-content auto;
}
.autocomplete-plus-item {
display: grid;
cursor: pointer;
@@ -292,10 +296,14 @@ body.dark-theme {
display: grid;
box-shadow: 0 2px 8px rgb(0 0 0 / 30%);
grid-auto-rows: auto;
grid-template-columns: max-content 1fr auto auto;
grid-template-columns: max-content 1fr auto;
overflow-y: auto;
}
#related-tags-list.no-alias {
grid-template-columns: max-content auto;
}
.related-tag-item {
display: grid;
cursor: pointer;
+96 -39
View File
@@ -1,4 +1,5 @@
import {
ModelTagSource,
TagCategory,
TagData,
autoCompleteData,
@@ -9,12 +10,14 @@ import {
hiraToKata,
kataToHira,
formatCountHumanReadable,
escapeHtml,
isContainsLetterOrNumber,
normalizeTagToInsert,
normalizeTagToSearch,
extractTagsFromTextArea,
getCurrentTagRange,
getViewportMargin,
getScrollbarWidth,
IconSvgHtmlString
} from './utils.js';
import { settingValues } from './settings.js';
@@ -89,7 +92,7 @@ function searchCompletionCandidates(textareaElement) {
}
// Generate Hiragana/Katakana variations if applicable
const queryVariations = new Set([partialTag, normalizeTagToSearch(partialTag)]);
const queryVariations = new Set([partialTag.toLowerCase(), normalizeTagToSearch(partialTag).toLowerCase()]);
const kataQuery = hiraToKata(partialTag);
if (kataQuery !== partialTag) {
queryVariations.add(kataQuery);
@@ -128,15 +131,14 @@ function sequentialSearch(partialTag, queryVariations) {
let matchedAlias = null;
// Check primary tag against all variations for exact/partial match
const tagMatch = matchWord(tagData.tag, queryVariations);
const tagMatch = matchWord(tagData.tag.toLowerCase(), queryVariations);
matched = tagMatch.matched;
isExactMatch = tagMatch.isExactMatch;
// If primary tag didn't match, check aliases against all variations
if (!matched && tagData.alias && Array.isArray(tagData.alias) && tagData.alias.length > 0) {
for (const alias of tagData.alias) {
const lowerAlias = alias.toLowerCase();
const aliasMatch = matchWord(lowerAlias, queryVariations);
const aliasMatch = matchWord(alias.toLowerCase(), queryVariations);
if (aliasMatch.matched) {
matched = true;
isExactMatch = aliasMatch.isExactMatch;
@@ -257,6 +259,10 @@ function searchWithFlexSearch(partialTag, queryVariations) {
* @returns {string} The current partial tag.
*/
function getCurrentPartialTag(inputElement) {
if (!inputElement) {
return "";
}
const text = inputElement.value;
const cursorPos = inputElement.selectionStart;
@@ -299,17 +305,38 @@ function getCurrentPartialTag(inputElement) {
* Inserts the selected tag into the textarea, replacing the partial tag,
* making the change undoable.
* @param {HTMLTextAreaElement} inputElement
* @param {string} tagToInsert The raw tag string to insert.
* @param {TagData} tagDataToInsert The raw tag string to insert.
*/
function insertTagToTextArea(inputElement, tagToInsert) {
function insertTagToTextArea(inputElement, tagDataToInsert) {
if (!inputElement || !tagDataToInsert) {
return;
}
const text = inputElement.value;
const cursorPos = inputElement.selectionStart;
const { start: tagStart, end: tagEnd, tag: currentTag } = getCurrentTagRange(text, cursorPos);
const tagRange = getCurrentTagRange(text, cursorPos);
let tagStart, tagEnd, currentTag;
if (!tagRange) {
// Fallback: insert at cursor position
tagStart = cursorPos;
tagEnd = cursorPos;
currentTag = '';
} else {
({ start: tagStart, end: tagEnd, tag: currentTag } = tagRange);
}
const replaceStart = Math.min(cursorPos, tagStart);
let replaceEnd = cursorPos;
const normalizedTag = normalizeTagToInsert(tagToInsert);
let normalizedTag;
if (Object.values(ModelTagSource).includes(tagDataToInsert.source)) {
// If the tag is from a model tag source, don't want to normalize it
normalizedTag = tagDataToInsert.tag;
} else {
normalizedTag = normalizeTagToInsert(tagDataToInsert.tag);
}
const currentTagAfterCursor = text.substring(cursorPos, tagEnd).trimEnd();
if (normalizedTag.lastIndexOf(currentTagAfterCursor) !== -1) {
@@ -378,7 +405,7 @@ class AutocompleteUI {
this.tagsList.addEventListener('mousedown', (e) => {
const row = e.target.closest('.autocomplete-plus-item');
if (row && row.dataset.tag) {
this.#insertTag(row.dataset.tag);
this.#insertTag(row.dataset);
e.preventDefault(); // Prevent focus loss from input
e.stopPropagation();
}
@@ -443,10 +470,12 @@ class AutocompleteUI {
this.#highlightItem();
}
/** Selects the currently highlighted item */
/** Selects the currently highlighted item
* @returns {TagData|null} The selected tag data.
*/
getSelectedTag() {
if (this.selectedIndex >= 0 && this.selectedIndex < this.candidates.length) {
return this.candidates[this.selectedIndex].tag;
return this.candidates[this.selectedIndex];
}
return null; // No valid selection
@@ -462,6 +491,8 @@ class AutocompleteUI {
return;
}
this.tagsList.classList.toggle('no-alias', settingValues.hideAlias);
const existingTags = extractTagsFromTextArea(this.target);
const currentTag = getCurrentPartialTag(this.target);
@@ -479,6 +510,7 @@ class AutocompleteUI {
*/
#createTagElement(tagData, isExisting) {
const categoryText = TagCategory[tagData.source][tagData.category] || "unknown";
const aliasText = tagData.alias.join(', ');
const tagRow = document.createElement('div');
tagRow.classList.add('autocomplete-plus-item', tagData.source);
@@ -492,9 +524,10 @@ class AutocompleteUI {
if (settingValues.tagSourceIconPosition == 'hidden') {
tagName.textContent = tagData.tag;
} else {
const escapedTag = escapeHtml(tagData.tag);
tagName.innerHTML = settingValues.tagSourceIconPosition == 'left'
? `${tagSourceIconHtml} ${tagData.tag}`
: `${tagData.tag} ${tagSourceIconHtml}`;
? `${tagSourceIconHtml} ${escapedTag}`
: `${escapedTag} ${tagSourceIconHtml}`;
}
// grayout tag name if it already exists
@@ -507,26 +540,29 @@ class AutocompleteUI {
alias.className = 'autocomplete-plus-alias';
// Display alias if available
if (tagData.alias && tagData.alias.length > 0) {
let aliasText = tagData.alias.join(', ');
if (aliasText.length > 0) {
alias.textContent = `${aliasText}`;
alias.title = tagData.alias.join(', '); // Full alias on hover
alias.title = aliasText; // Full alias on hover
}
// Category
const category = document.createElement('span');
category.className = `autocomplete-plus-category`;
category.textContent = `${categoryText.substring(0, 2)}`;
category.title = categoryText; // Full category on hover
// Count
const tagCount = document.createElement('span');
tagCount.className = `autocomplete-plus-tag-count`;
tagCount.textContent = formatCountHumanReadable(tagData.count);
// Create tooltip with more info
let tooltipText = `Count: ${tagData.count}\nCategory: ${categoryText}`;
if (aliasText.length > 0) {
tooltipText += `\nAlias: ${aliasText}`;
}
tagRow.title = tooltipText;
tagRow.appendChild(tagName);
tagRow.appendChild(alias);
tagRow.appendChild(category);
if (!settingValues.hideAlias) {
tagRow.appendChild(alias);
}
tagRow.appendChild(tagCount);
this.tagsList.appendChild(tagRow);
}
@@ -564,21 +600,14 @@ class AutocompleteUI {
let topPosition = targetElmOffset.top + ((caretTop - targetElmOffset.top) + caretLineHeight) * scale;
let leftPosition = targetElmOffset.left + (caretLeft - targetElmOffset.left) * scale;;
const maxWidth = Math.min(rootRect.width, viewportWidth / 2);
const naturalHeight = rootRect.height;
//Horizontal Collision Detection and Adjustment
if (leftPosition + maxWidth > viewportWidth - margin.right) {
leftPosition = viewportWidth - maxWidth - margin.right;
}
if (leftPosition < margin.left) {
leftPosition = margin.left;
}
// Vertical Collision Detection and Adjustment
const availableSpaceBelow = viewportHeight - topPosition - margin.bottom;
const availableSpaceAbove = caretTop - margin.top;
let needsVerticalScroll = false;
let maxHeight = rootRect.height;
if (naturalHeight <= availableSpaceBelow) {
// Fits perfectly below the caret
@@ -590,14 +619,15 @@ class AutocompleteUI {
topPosition = caretTop - naturalHeight - margin.top;
} else {
// Doesn't fit perfectly either below or above, needs scrolling.
needsVerticalScroll = true;
// Choose the position (above or below) that offers more space.
if (availableSpaceBelow >= availableSpaceAbove) {
// Scroll below: topPosition remains as initially calculated
this.tagsList.style.maxHeight = `${availableSpaceBelow}px`;
maxHeight = availableSpaceBelow;
} else {
// Scroll above: Position near the top edge and set max-height
topPosition = margin.top;
this.tagsList.style.maxHeight = `${availableSpaceAbove}px`;
maxHeight = availableSpaceAbove;
}
}
}
@@ -606,16 +636,30 @@ class AutocompleteUI {
if (topPosition < margin.top) {
topPosition = margin.top;
// If pushed down, recalculate max-height if it was set based on top alignment
if (this.tagsList.style.maxHeight && availableSpaceBelow < availableSpaceAbove) {
if (availableSpaceBelow < availableSpaceAbove) {
// Recalculate max-height based on space from the top margin
this.tagsList.style.maxHeight = `${viewportHeight - margin.top - margin.bottom}px`;
maxHeight = viewportHeight - margin.top - margin.bottom;
needsVerticalScroll = true;
}
}
// Calculate maxWidth considering scrollbar width if vertical scrolling is needed
const scrollbarWidth = needsVerticalScroll ? getScrollbarWidth() : 0;
const maxWidth = Math.min(rootRect.width + scrollbarWidth, viewportWidth / 2);
// Horizontal Collision Detection and Adjustment
if (leftPosition + maxWidth > viewportWidth - margin.right) {
leftPosition = viewportWidth - maxWidth - margin.right;
}
if (leftPosition < margin.left) {
leftPosition = margin.left;
}
// Apply the calculated position and display the element
this.root.style.left = `${leftPosition}px`;
this.root.style.top = `${topPosition}px`;
this.root.style.maxWidth = `${maxWidth}px`;
this.tagsList.style.maxHeight = `${maxHeight}px`;
}
/** Highlights the item (row) at the given index */
@@ -635,10 +679,10 @@ class AutocompleteUI {
/**
* Handles the selection of an item
* @param {string} selectedTag The tag to insert.
* @param {TagData} selectedTag The tag to insert.
*/
#insertTag(selectedTag) {
if (!this.target || !selectedTag || selectedTag.length <= 0) {
if (!this.target || !selectedTag) {
this.hide();
return;
}
@@ -1000,3 +1044,16 @@ export class AutocompleteEventHandler {
handleClick(event) {
}
}
// Export functions for testing
const isTestEnvironment = typeof process !== 'undefined' && process.env.NODE_ENV === 'test';
export const __test__ = isTestEnvironment
? {
searchCompletionCandidates,
sequentialSearch,
searchWithFlexSearch,
matchWord,
getCurrentPartialTag,
insertTagToTextArea
}
: undefined;
+137 -26
View File
@@ -1,14 +1,20 @@
import { settingValues, updateMaxTagLength } from "./settings.js";
import { createFlexSearchDocument } from "./searchengine.js";
import { createFlexSearchDocument, createFlexSearchDocumentForModel } from "./searchengine.js";
// --- Constants ---
// Tag data sources
// Tag sources for booru-like tag data.
export const TagSource = {
Danbooru: 'danbooru',
E621: 'e621',
}
// Tag sources for model based tag data.
export const ModelTagSource = {
Embeddings: 'embeddings',
Lora: 'lora'
}
export const TagCategory = {
'danbooru': [
'general',
@@ -28,6 +34,12 @@ export const TagCategory = {
'invalid',
'meta',
'lore',
],
'embeddings': [
'embeddings'
],
'lora': [
'lora'
]
}
@@ -40,19 +52,19 @@ export class TagData {
/**
* Create a tag data object
* @param {string} tag - The tag name
* @param {string[]} [alias=[]] - Array of aliases for the tag
* @param {string} [category='general'] - Category of the tag
* @param {number} [category] - Category index of the tag
* @param {number} [count=0] - Frequency count/popularity of the tag
* @param {string} [source=TagSources.Danbooru] - The source of the tag data
* @param {string[]} [alias=[]] - Array of aliases for the tag
* @param {string} [source=TagSource.Danbooru] - The source of the tag data
*/
constructor(tag, alias = [], category = 'general', count = 0, source = TagSource.Danbooru) {
constructor(tag, category, count = 0, alias = [], source = TagSource.Danbooru) {
/** @type {string} */
this.tag = tag;
/** @type {string[]} */
this.alias = alias;
/** @type {string} */
/** @type {number} */
this.category = category;
/** @type {number} */
@@ -98,24 +110,32 @@ export const autoCompleteData = {};
const TAGS_CSV_HEADER = 'tag,category,count,alias';
const TAGS_CSV_HEADER_COLUMNS = TAGS_CSV_HEADER.split(',');
const TAG_INDEX = TAGS_CSV_HEADER_COLUMNS.indexOf('tag');
const ALIAS_INDEX = TAGS_CSV_HEADER_COLUMNS.indexOf('alias');
const CATEGORY_INDEX = TAGS_CSV_HEADER_COLUMNS.indexOf('category');
const COUNT_INDEX = TAGS_CSV_HEADER_COLUMNS.indexOf('count');
const ALIAS_INDEX = TAGS_CSV_HEADER_COLUMNS.indexOf('alias');
// --- Helder Functions ---
/**
* Get the available tag sources in priority order based on the current settings.
* @returns {string[]} Array of available tag sources in priority order
*/
export function getEnabledTagSourceInPriorityOrder() {
return Object.values(TagSource)
let enabledTagSources = Object.values(TagSource)
.filter((s) => {
return settingValues.tagSource === s || settingValues.tagSource === 'all';
})
.toSorted((a, b) => {
return a === settingValues.primaryTagSource ? -1 : 1;
});
// Append Loras and Embeddings if enabled
if (settingValues.enableModels) {
enabledTagSources = [...enabledTagSources, ...Object.values(ModelTagSource)];
}
return enabledTagSources;
}
// --- Data Loading Functions ---
@@ -159,23 +179,13 @@ async function loadTags(csvUrl, siteName) {
const aliases = aliasStr ? aliasStr.split(',').map(a => a.trim()).filter(a => a.length > 0) : [];
// Create a TagData instance instead of a plain object
const tagData = new TagData(tag, aliases, category, count, siteName);
const tagData = new TagData(tag, category, count, aliases, siteName);
updateMaxTagLength(tag.length);
autoCompleteData[siteName].sortedTags.push(tagData);
} else {
console.warn(`[Autocomplete-Plus] Invalid CSV format in line ${i + 1} of ${csvUrl}: ${line}. Expected ${TAGS_CSV_HEADER_COLUMNS.length} columns, but got ${columns.length}.`);
continue;
}
}
// Sort by count in descending order
autoCompleteData[siteName].sortedTags.sort((a, b) => b.count - a.count);
// Build maps as before, but ensure not to overwrite if already processed from extra files
autoCompleteData[siteName].sortedTags.forEach(tagData => {
if (!autoCompleteData[siteName].tagMap.has(tagData.tag)) {
// Set the tag and its alias in the maps
autoCompleteData[siteName].tagMap.set(tagData.tag, tagData);
if (tagData.alias && Array.isArray(tagData.alias)) {
tagData.alias.forEach(alias => {
@@ -184,8 +194,11 @@ async function loadTags(csvUrl, siteName) {
}
});
}
} else {
console.warn(`[Autocomplete-Plus] Invalid CSV format in line ${i + 1} of ${csvUrl}: ${line}. Expected ${TAGS_CSV_HEADER_COLUMNS.length} columns, but got ${columns.length}.`);
continue;
}
});
}
} catch (error) {
console.error(`[Autocomplete-Plus] Failed to fetch or process tags from ${csvUrl}:`, error);
@@ -202,7 +215,14 @@ async function buildFlexSearchIndex(siteName) {
return;
}
const document = createFlexSearchDocument();
let document = null;
if (Object.values(TagSource).includes(siteName)) {
document = createFlexSearchDocument();
} else if (Object.values(ModelTagSource).includes(siteName)) {
document = createFlexSearchDocumentForModel();
} else {
throw new Error(`[Autocomplete-Plus] Invalid site name: ${siteName}`);
}
let startIdx = 0;
const startTime = performance.now();
@@ -341,7 +361,11 @@ function parseCSVLine(line) {
return result;
}
export async function fetchCsvList() {
/**
* Fetch the list of CSV files from the API endpoint
* @returns {Promise<void>}
*/
async function fetchCsvList() {
try {
const response = await fetch('/autocomplete-plus/csv');
if (!response.ok) {
@@ -357,9 +381,8 @@ export async function fetchCsvList() {
/**
* Initializes the autocomplete data by fetching the list of CSV files and loading them.
* This function is called when the extension is initialized.
*/
export async function initializeData(csvListData, source) {
async function initializeDataFromCSV(csvListData, source) {
if (autoCompleteData.hasOwnProperty(source) === false) {
autoCompleteData[source] = new AutocompleteData();
}
@@ -420,6 +443,9 @@ export async function initializeData(csvListData, source) {
await Promise.all([
Promise.all(tagsLoadPromiseFactories.map(factory => factory()))
.then(() => {
// Sort by count in descending order
autoCompleteData[source].sortedTags.sort((a, b) => b.count - a.count);
// Build FlexSearch index after tags are loaded
return buildFlexSearchIndex(source);
})
@@ -444,3 +470,88 @@ export async function initializeData(csvListData, source) {
autoCompleteData[source].isInitializing = false;
}
}
/**
* Load Embeddings data from the API endpoint
* @returns {Promise<void>}
*/
async function loadEmbeddings() {
try {
const response = await fetch('/autocomplete-plus/embeddings', { cache: "no-store" });
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
const embeddings = await response.json();
const source = ModelTagSource.Embeddings;
if (autoCompleteData.hasOwnProperty(source) === false) {
autoCompleteData[source] = new AutocompleteData();
}
embeddings.forEach(embedding => {
if (!autoCompleteData[source].tagMap.has(embedding)) {
const tagData = new TagData(`embedding:${embedding}`, 0, 0, [], source);
autoCompleteData[source].sortedTags.push(tagData);
autoCompleteData[source].tagMap.set(embedding, tagData);
updateMaxTagLength(embedding.length);
}
});
await buildFlexSearchIndex(ModelTagSource.Embeddings);
console.log(`[Autocomplete-Plus] Loaded ${embeddings.length} Embeddings`);
} catch (error) {
console.error(`[Autocomplete-Plus] Failed to fetch Embeddings data:`, error);
}
}
/**
* Load LoRA data from the API endpoint
* @returns {Promise<void>}
*/
async function loadLoras() {
try {
const response = await fetch('/autocomplete-plus/loras', { cache: "no-store" });
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
const loraNames = await response.json();
const source = ModelTagSource.Lora;
if (autoCompleteData.hasOwnProperty(source) === false) {
autoCompleteData[source] = new AutocompleteData();
}
loraNames.forEach(loraName => {
if (!autoCompleteData[source].tagMap.has(loraName)) {
const tagData = new TagData(`<lora:${loraName}>`, 0, 0, [], source);
autoCompleteData[source].sortedTags.push(tagData);
autoCompleteData[source].tagMap.set(loraName, tagData);
updateMaxTagLength(loraName.length);
}
});
await buildFlexSearchIndex(ModelTagSource.Lora);
console.log(`[Autocomplete-Plus] Loaded ${loraNames.length} LoRA models`);
} catch (error) {
console.error(`[Autocomplete-Plus] Failed to fetch LoRA data:`, error);
}
}
/**
* Load all data sources asynchronously.
*/
export async function loadDataAsync() {
return Promise.all([
fetchCsvList().then((csvList) => {
Object.values(TagSource).forEach((source) => {
initializeDataFromCSV(csvList, source);
});
}),
loadEmbeddings(),
loadLoras(),
]);
}
+71 -38
View File
@@ -3,7 +3,7 @@ import { $el } from "/scripts/ui.js";
import { ComfyWidgets } from "/scripts/widgets.js";
import { settingValues } from "./settings.js";
import { loadCSS } from "./utils.js";
import { TagSource, fetchCsvList, initializeData } from "./data.js";
import { TagSource, loadDataAsync } from "./data.js";
import { AutocompleteEventHandler } from "./autocomplete.js";
import { RelatedTagsEventHandler } from "./related-tags.js";
@@ -190,40 +190,41 @@ async function addExtraSettings() {
}
// Add extra setting for checking new CSV updates
app.ui.settings.addSetting({
id: id + ".check_new_csv",
defaultValue: null,
name: "Check CSV updates",
category: [name, "Misc", "Check new CSV"],
type: () => {
const lastCheckSpan = $el("span", {
textContent: lastCheckTimeText,
className: "text-sm text-gray-500",
style: {
marginRight: "16px"
}
});
// Note: Temporarily comment out until changes to remote files can be detected
// app.ui.settings.addSetting({
// id: id + ".check_new_csv",
// defaultValue: null,
// name: "Check CSV updates",
// category: [name, "Misc", "Check new CSV"],
// type: () => {
// const lastCheckSpan = $el("span", {
// textContent: lastCheckTimeText,
// className: "text-sm text-gray-500",
// style: {
// marginRight: "16px"
// }
// });
const checkButton = $el("button", {
textContent: "Check now",
className: "p-button p-component p-button-primary",
onclick: async () => {
await performUpdateCheck(checkButton, lastCheckSpan);
}
});
// const checkButton = $el("button", {
// textContent: "Check now",
// className: "p-button p-component p-button-primary",
// onclick: async () => {
// await performUpdateCheck(checkButton, lastCheckSpan);
// }
// });
return $el("div", {
className: "flex-row items-center gap-2",
}, [
$el("div", {
className: "p-component",
}, [
lastCheckSpan,
checkButton,
]),
]);
}
});
// return $el("div", {
// className: "flex-row items-center gap-2",
// }, [
// $el("div", {
// className: "p-component",
// }, [
// lastCheckSpan,
// checkButton,
// ]),
// ]);
// }
// });
}
/**
@@ -240,11 +241,7 @@ app.registerExtension({
let rootPath = import.meta.url.replace("js/main.js", "");
loadCSS(rootPath + "css/autocomplete-plus.css"); // Load CSS for autocomplete
fetchCsvList().then((csvList) => {
Object.values(TagSource).forEach((source) => {
initializeData(csvList, source);
});
});
await loadDataAsync();
},
// One the Settings Screen, displays reverse order in same category
@@ -285,6 +282,7 @@ app.registerExtension({
settingValues.tagSource = newVal;
}
},
// --- Autocomplete Settings ---
{
id: id + ".use_fast_search",
@@ -297,6 +295,28 @@ app.registerExtension({
settingValues.useFastSearch = newVal;
}
},
{
id: id + ".enable_models",
name: "Enable Loras and Embeddings",
tooltip: "Enable Lora and Embedding suggestions",
type: "boolean",
defaultValue: true,
category: [name, "Autocompletion", "Enable Loras and Embeddings"],
onChange: (newVal, oldVal) => {
settingValues.enableModels = newVal;
}
},
{
id: id + ".replace_underscore_with_space",
name: "Replace '_' with 'Space'",
tooltip: "This setting also affects related tags display.",
type: "boolean",
defaultValue: true,
category: [name, "Autocompletion", "Replace Underscore with Space"],
onChange: (newVal, oldVal) => {
settingValues.replaceUnderscoreWithSpace = newVal;
}
},
{
id: id + ".max_suggestions",
name: "Max suggestions",
@@ -375,6 +395,19 @@ app.registerExtension({
onChange: (newVal, oldVal) => {
settingValues.enableRelatedTags = newVal;
}
},
// --- Display settings ---
{
id: id + ".display_hide_alias",
name: "Hide Alias",
tooltip: "Hide alias in the autocomplete and related tags display",
type: "boolean",
defaultValue: false,
category: [name, "Display", "Hide Alias"],
onChange: (newVal, oldVal) => {
settingValues.hideAlias = newVal;
}
}
]
});
+35 -30
View File
@@ -4,6 +4,7 @@ import {
extractTagsFromTextArea,
findAllTagPositions,
getCurrentTagRange,
getScrollbarWidth,
getViewportMargin,
isLongText,
isValidTag,
@@ -259,7 +260,7 @@ class RelatedTagsUI {
this.headerControls.appendChild(this.pinBtn);
this.header.appendChild(this.headerControls);
this.root.appendChild(this.header);
// Create a tbody for the tags
@@ -410,17 +411,12 @@ class RelatedTagsUI {
.find((tagData) => tagData !== undefined);
if (!tagData) {
tagData = new TagData({
tag: this.currentTag,
source: TagSource.Danbooru,
category: 'unknown',
count: 0,
alias: [],
});
// Create a dummy TagData if not found
tagData = new TagData(this.currentTag, null, 0, [], TagSource.Danbooru);
}
const tagText = this.currentTag;
const categoryText = TagCategory[tagData.source][tagData.category];
const categoryText = TagCategory[tagData.source][tagData.category] || "unknown";
const aliasText = tagData.alias.join(', ');
// Update header text with current tag
this.headerText.innerHTML = ''; // Clear previous content
@@ -428,15 +424,15 @@ class RelatedTagsUI {
const tagName = document.createElement('span');
tagName.classList.add('related-tags-header-tag-name', tagData.source);
tagName.title = `Count: ${tagData.count}\nCategory: ${categoryText}`;
tagName.title = `Count: ${tagData.count}\nCategory: ${categoryText}\nAlias: ${aliasText}`;
tagName.dataset.tagCategory = categoryText;
if (tagData.source && ['left', 'right'].includes(settingValues.tagSourceIconPosition)) {
const tagSourceIconHtml = `<svg class="autocomplete-plus-tag-icon-svg"><use xlink:href="#autocomplete-plus-icon-${tagData.source}"></use></svg>`;
tagName.innerHTML = settingValues.tagSourceIconPosition == 'left'
? `${tagSourceIconHtml} ${tagText}`
: `${tagText} ${tagSourceIconHtml}`;
? `${tagSourceIconHtml} ${tagData.tag}`
: `${tagData.tag} ${tagSourceIconHtml}`;
} else {
tagName.textContent += tagText;
tagName.textContent += tagData.tag;
}
@@ -447,8 +443,8 @@ class RelatedTagsUI {
this.headerAlias.innerHTML = '';
// Add alias if available
if (tagData.alias?.length > 0) {
this.headerAlias.textContent = tagData.alias.join(', ');
if (aliasText.length > 0 && !settingValues.hideAlias) {
this.headerAlias.textContent = aliasText;
this.headerAlias.style.display = 'block';
}
@@ -485,6 +481,9 @@ class RelatedTagsUI {
return;
}
// Toggle column class based on settings
this.tagsContainer.classList.toggle('no-alias', settingValues.hideAlias);
const existingTags = extractTagsFromTextArea(this.target);
// Create tag rows
@@ -503,6 +502,7 @@ class RelatedTagsUI {
*/
#createTagElement(tagData, isExisting) {
const categoryText = TagCategory[tagData.source][tagData.category] || "unknown";
const aliasText = tagData.alias.join(', ');
const tagRow = document.createElement('div');
tagRow.classList.add('related-tag-item', tagData.source);
@@ -524,33 +524,30 @@ class RelatedTagsUI {
alias.className = 'related-tag-alias';
// Display alias if available
if (tagData.alias && tagData.alias.length > 0) {
let aliasText = tagData.alias.join(', ');
if (aliasText.length > 0) {
alias.textContent = `${aliasText}`;
alias.title = tagData.alias.join(', '); // Full alias on hover
alias.title = aliasText; // Full alias on hover
}
// Category
const category = document.createElement('span');
category.className = `related-tag-category`;
category.textContent = `${categoryText.substring(0, 2)}`;
// Similarity
const similarity = document.createElement('span');
similarity.className = 'related-tag-similarity';
similarity.textContent = `${(tagData.similarity * 100).toFixed(2)}%`;
// Create tooltip with more info
let tooltipText = `Tag: ${tagData.tag}\nSimilarity: ${(tagData.similarity * 100).toFixed(2)}%\nCount: ${tagData.count}`;
if (tagData.alias && tagData.alias.length > 0) {
tooltipText += `\nAlias: ${tagData.alias.join(', ')}`;
let tooltipText = `Similarity: ${(tagData.similarity * 100).toFixed(2)}%\nCount: ${tagData.count}\nCategory: ${categoryText}`;
if (aliasText.length > 0) {
tooltipText += `\nAlias: ${aliasText}`;
}
tagRow.title = tooltipText;
// Add cells to row
tagRow.appendChild(tagName);
tagRow.appendChild(alias);
tagRow.appendChild(category);
if (!settingValues.hideAlias) {
tagRow.appendChild(alias);
}
tagRow.appendChild(similarity);
return tagRow;
@@ -580,7 +577,7 @@ class RelatedTagsUI {
const newHeaderRect = this.header.getBoundingClientRect();
if(this.relatedTags.length > 0){
if (this.relatedTags.length > 0) {
this.tagsContainer.style.maxHeight = `${placementArea.height - newHeaderRect.height}px`;
}
@@ -669,6 +666,10 @@ class RelatedTagsUI {
area.y = targetRect.bottom;
}
// Calculate width considering scrollbar width if vertical scrolling is needed
const scrollbarWidth = area.height < elemHeight ? getScrollbarWidth() : 0;
area.width = Math.min(elemWidth + scrollbarWidth, maxWidth);
// Adjust x position to avoid overflow
area.x = Math.min(area.x, viewportWidth - area.width - margin.right);
} else {
@@ -685,6 +686,10 @@ class RelatedTagsUI {
area.x = targetRect.right;
}
// Calculate width considering scrollbar width if vertical scrolling is needed
const scrollbarWidth = area.height < elemHeight ? getScrollbarWidth() : 0;
area.width = Math.min(area.width + scrollbarWidth, viewportWidth - margin.left - margin.right);
// Adjust y position to avoid overflow
area.y = Math.min(area.y, viewportHeight - area.height - margin.bottom);
}
+51 -4
View File
@@ -14,7 +14,7 @@ function createTagEncoder() {
cache: true,
// filter: new Set(['and', 'to', 'be', 'on']),
replacer: [/(?<=[a-zA-Z\)])_$/, ''], // Remove trailing underscores after letters/parentheses
split: /(?<=[a-zA-Z\)])_(?=[a-zA-Z\(])|\((?=[a-zA-Z])|(?<=[a-zA-Z\)])\)|[ \n]/ // Split on underscores between words, parentheses, spaces, and newlines
split: /(?<=[a-zA-Z\)])_(?=[a-zA-Z\(])|\((?=[a-zA-Z])|(?<=[a-zA-Z\)])\)|[ \-\n]/ // Split on underscores between words, parentheses, spaces, hyphens, and newlines
});
}
@@ -35,6 +35,23 @@ function createCJKEncoder() {
});
}
/**
* Creates an encoder optimized for processing Embedding or Lora notation.
* @returns {Encoder} FlexSearch encoder
*/
function createModelEncoder() {
return new Encoder({
normalize: true,
dedupe: false,
numeric: true,
cache: true,
prepare: function (str) {
return str.replace(/^<|>$/g, '').split(/(lora:|embedding:|[^\u0000-\u007f]+)/g).filter(Boolean).join(" ").trim();
},
split: /(?<=lora:.*|embedding:.*)[_./\(\)\-\s\\]+/
});
}
/**
* Creates a FlexSearch Document instance optimized for tag searching.
* Configures separate encoders for English tags and CJK aliases with appropriate tokenization.
@@ -45,8 +62,8 @@ export function createFlexSearchDocument() {
const cjkEncoder = createCJKEncoder();
// Custom encoding function for alias field that handles mixed language content
const encodeAlias = function (term) {
return term.split(",")
const encodeAlias = function (word) {
return word.split(",")
.flatMap(str => {
if (/[^\u0000-\u007f]/.test(str)) {
// Contains non-ASCII characters (CJK text)
@@ -81,6 +98,36 @@ export function createFlexSearchDocument() {
return document;
}
/**
* Creates a FlexSearch Document instance optimized for lora or embedding searching.
* @returns {Document} Configured FlexSearch document
*/
export function createFlexSearchDocumentForModel() {
const modelEncoder = createModelEncoder();
// Configure the FlexSearch document with optimized indexing settings
// Note: alias field is not indexed for lora or embedding search
const document = new Document({
tokenize: "full", // Allow partial matching from both ends
encoder: modelEncoder,
document: {
id: "id",
index: [
{
field: "tag",
},
{
field: "alias",
}
]
}
});
return document;
}
// Export functions for testing when in test environment
const isTestEnvironment = typeof process !== 'undefined' && process.env.NODE_ENV === 'test';
export const __test__ = isTestEnvironment ? { createTagEncoder, createCJKEncoder } : undefined;
export const __test__ = isTestEnvironment ? { createTagEncoder, createCJKEncoder, createModelEncoder } : undefined;
+5
View File
@@ -7,7 +7,9 @@ export const settingValues = {
// Autocomplete feature settings
enabled: true,
maxSuggestions: 10,
enableModels: true, // Enable Lora and Embedding suggestions
useFastSearch: false,
replaceUnderscoreWithSpace: true, // Replace underscores with spaces in tag insertion
// Related tags feature settings
enableRelatedTags: true,
@@ -15,6 +17,9 @@ export const settingValues = {
relatedTagsDisplayPosition: 'horizontal', // 'horizontal' or 'vertical'
relatedTagsTriggerMode: 'click', // Options: 'click', 'ctrl+Click'
// Display settings
hideAlias: false, // Hide alias in the autocomplete and related tags display
// Internal logic settings
_useFallbackAttachmentForEventListener: false, // Fallback to attach event listener when somthing goes wrong
+61 -2
View File
@@ -109,6 +109,30 @@ export function formatCountHumanReadable(num) {
return (num / si[i].value).toFixed(1).replace(rx, "$1") + si[i].symbol;
}
/**
* Escapes HTML special characters in a string.
* @param {string} str The input string.
* @returns {string} The escaped string.
*/
export function escapeHtml(str) {
if (typeof str !== 'string') {
return str;
}
const escapeMap = {
'&': '&amp;',
'<': '&lt;',
'>': '&gt;',
'"': '&quot;',
"'": '&#x27;',
'`': '&#x60;',
'/': '&#x2F;'
};
return str.replace(/[&<>"'`/]/g, match =>
escapeMap[match]);
}
/**
* Escapes parentheses in a string for use in prompts.
* Replaces '(' with '\(' and ')' with '\)'.
@@ -208,8 +232,8 @@ export function normalizeTagToInsert(str) {
if (isContainsLetterOrNumber(str)) {
const isWildcardCall = str.startsWith('__') && str.endsWith('__') && str.length > 4;
if (!isWildcardCall) {
// If doesn't wildcard call, replace underscores with spaces
if (!isWildcardCall && settingValues.replaceUnderscoreWithSpace) {
// If doesn't wildcard call and setting is enabled, replace underscores with spaces
return escapeParentheses(str.replace(/_/g, " "));
}
}
@@ -560,3 +584,38 @@ export function getViewportMargin() {
right: rightBarRect.width,
};
}
// Cache for scrollbar width measurement
let _cachedScrollbarWidth;
/**
* Measures and returns the scrollbar width for the current browser.
* Creates a temporary element to measure the scrollbar width accurately.
* @returns {number} The scrollbar width in pixels
*/
export function getScrollbarWidth() {
// Return cached value if available
if (_cachedScrollbarWidth !== undefined) {
return _cachedScrollbarWidth;
}
// Create a temporary div with scrollbar
const outer = document.createElement('div');
outer.style.visibility = 'hidden';
outer.style.position = 'absolute';
outer.style.overflow = 'scroll';
outer.style.top = '-9999px';
outer.style.width = '100px';
document.body.appendChild(outer);
const inner = document.createElement('div');
inner.style.width = '100%';
outer.appendChild(inner);
// Cache the result
_cachedScrollbarWidth = outer.offsetWidth - inner.offsetWidth;
document.body.removeChild(outer);
return _cachedScrollbarWidth;
}