feat(security): implement PII sanitization for LLM requests with configurable privacy levels
This commit is contained in:
@@ -267,6 +267,38 @@ Configure AI analysis in the **Doctor Sidebar** → **Settings** panel:
|
||||
|
||||
**Security Note**: Your API key is transmitted securely from frontend to backend for the analysis request only. It is never logged or stored persistently.
|
||||
|
||||
### Privacy Mode (PII Sanitization)
|
||||
|
||||
ComfyUI-Doctor includes automatic **PII (Personally Identifiable Information) sanitization** to protect your privacy when sending error messages to cloud AI services.
|
||||
|
||||
**Three Privacy Levels**:
|
||||
|
||||
| Level | Description | What is Removed | Recommended For |
|
||||
| ----- | ----------- | --------------- | --------------- |
|
||||
| **None** | No sanitization | Nothing | Local LLMs (Ollama, LMStudio) |
|
||||
| **Basic** (Default) | Standard protection | User paths, API keys, emails, IP addresses | Most users with cloud LLMs |
|
||||
| **Strict** | Maximum privacy | All of Basic + IPv6, SSH fingerprints | Enterprise/compliance requirements |
|
||||
|
||||
**What is Sanitized** (Basic Level):
|
||||
|
||||
- ✅ Windows user paths: `C:\Users\john\file.py` → `<USER_PATH>\file.py`
|
||||
- ✅ Linux/macOS home: `/home/alice/test.py` → `<USER_HOME>/test.py`
|
||||
- ✅ API keys: `sk-abc123...` → `<API_KEY>`
|
||||
- ✅ Email addresses: `user@example.com` → `<EMAIL>`
|
||||
- ✅ Private IPs: `192.168.1.1` → `<PRIVATE_IP>`
|
||||
- ✅ URL credentials: `https://user:pass@host` → `https://<USER>@host`
|
||||
|
||||
**What is NOT Removed**:
|
||||
|
||||
- ❌ Error messages (needed for debugging)
|
||||
- ❌ Model names, node names
|
||||
- ❌ Workflow structure
|
||||
- ❌ Public file paths (`/usr/bin/python`)
|
||||
|
||||
**Configure Privacy Mode**: Open Doctor Sidebar → Settings → 🔒 Privacy Mode dropdown. Changes apply immediately to all AI analysis requests.
|
||||
|
||||
**GDPR Compliance**: This feature supports GDPR Article 25 (Data Protection by Design) and is recommended for enterprise deployments.
|
||||
|
||||
### Example Providers Setup
|
||||
|
||||
| Provider | Base URL | Model Example |
|
||||
|
||||
+42
-34
@@ -107,20 +107,22 @@ graph TD
|
||||
|
||||
*Sorted by priority (High → Low):*
|
||||
|
||||
- [ ] **S6**: PII Sanitization for LLM traceback - 🔴 High
|
||||
- **Backend** (`analyzer.py` preprocessing):
|
||||
- Remove user paths: `C:\Users\username\...` → `<USER_PATH>\...`
|
||||
- Sanitize Linux/macOS home: `/home/username/` → `<USER_HOME>/`
|
||||
- Optional: Email addresses, private IP addresses (regex-based)
|
||||
- Configurable sanitization levels: `none`, `basic`, `strict`
|
||||
- Zero runtime overhead, GDPR-friendly
|
||||
- [x] **S6**: PII Sanitization for LLM traceback - 🔴 High ✅ *Completed (2025-12-31)*
|
||||
- **Backend** (`sanitizer.py` module):
|
||||
- ✅ Remove user paths: `C:\Users\username\...` → `<USER_PATH>\...`
|
||||
- ✅ Sanitize Linux/macOS home: `/home/username/` → `<USER_HOME>/`
|
||||
- ✅ Email addresses, private IP addresses (regex-based)
|
||||
- ✅ Configurable sanitization levels: `none`, `basic`, `strict`
|
||||
- ✅ Zero runtime overhead, GDPR-friendly
|
||||
- **Frontend** (Privacy Controls):
|
||||
- Settings panel: "Privacy Mode" toggle (default: ON)
|
||||
- Visual indicator: 🔒 icon when sanitization active
|
||||
- "Preview" button: Show diff before sending to LLM
|
||||
- Audit log: Save sanitized requests to localStorage
|
||||
- ✅ Settings panel: "Privacy Mode" dropdown with 3 levels
|
||||
- ✅ Visual indicator: 🔒 icon in settings
|
||||
- ✅ Multi-language support (9 languages)
|
||||
- ⏸️ "Preview" button: Not implemented (deferred to future)
|
||||
- ⏸️ Audit log UI: Not implemented (backend logging only)
|
||||
- **Critical for enterprise adoption** - blocks B2B market without this
|
||||
- **Foundation for**: A6 Pipeline Stage 1 (Sanitizer)
|
||||
- **Documentation**: See `.planning/S6_PII_SANITIZATION.md`
|
||||
- [ ] **S7**: Quarterly Security Audits - 🟢 Low (recurring)
|
||||
- **Automated** (CI/CD):
|
||||
- OWASP ZAP penetration testing
|
||||
@@ -321,12 +323,14 @@ graph TD
|
||||
|
||||
**Priority**: Security → Features
|
||||
|
||||
- [ ] **S6** PII Sanitization
|
||||
- Critical for enterprise adoption (blocks B2B market)
|
||||
- Backend: Regex-based path removal in `analyzer.py`
|
||||
- Frontend: Privacy toggle, preview diff, audit log
|
||||
- Zero risk, pure preprocessing
|
||||
- Can implement on `main` branch
|
||||
- [x] **S6** PII Sanitization ✅ *Completed (2025-12-31)*
|
||||
- ✅ Critical for enterprise adoption (blocks B2B market)
|
||||
- ✅ Backend: `sanitizer.py` module with 3 sanitization levels
|
||||
- ✅ Frontend: Privacy Mode dropdown in settings (9 languages)
|
||||
- ✅ Zero risk, pure preprocessing
|
||||
- ✅ Implemented on `main` branch
|
||||
- ✅ Comprehensive unit tests (21 tests)
|
||||
- See `.planning/S6_PII_SANITIZATION.md` for details
|
||||
- [ ] **F7** Smart Parameter Injection
|
||||
- Killer UX feature - one-click fixes in chat
|
||||
- LLM prompt engineering + frontend implementation
|
||||
@@ -611,20 +615,22 @@ graph TD
|
||||
|
||||
*按優先級排序(高 → 低):*
|
||||
|
||||
- [ ] **S6**: LLM traceback PII 淨化 - 🔴 High
|
||||
- **後端**(`analyzer.py` 預處理):
|
||||
- 移除用戶路徑:`C:\Users\username\...` → `<USER_PATH>\...`
|
||||
- 淨化 Linux/macOS 家目錄:`/home/username/` → `<USER_HOME>/`
|
||||
- 可選:Email 地址、私有 IP 地址(基於正則表達式)
|
||||
- 可配置淨化等級:`none`、`basic`、`strict`
|
||||
- 零執行時開銷,符合 GDPR 規範
|
||||
- [x] **S6**: LLM traceback PII 淨化 - 🔴 High ✅ *已完成 (2025-12-31)*
|
||||
- **後端**(`sanitizer.py` 模組):
|
||||
- ✅ 移除用戶路徑:`C:\Users\username\...` → `<USER_PATH>\...`
|
||||
- ✅ 淨化 Linux/macOS 家目錄:`/home/username/` → `<USER_HOME>/`
|
||||
- ✅ Email 地址、私有 IP 地址(基於正則表達式)
|
||||
- ✅ 可配置淨化等級:`none`、`basic`、`strict`
|
||||
- ✅ 零執行時開銷,符合 GDPR 規範
|
||||
- **前端**(隱私控制):
|
||||
- 設定面板:"Privacy Mode" 開關(預設開啟)
|
||||
- 視覺指示器:🔒 圖示(淨化啟用時)
|
||||
- "Preview" 按鈕:發送前顯示 diff
|
||||
- 審核日誌:將淨化請求存至 localStorage
|
||||
- ✅ 設定面板:"Privacy Mode" 下拉選單(3 種等級)
|
||||
- ✅ 視覺指示器:🔒 圖示於設定中
|
||||
- ✅ 多語系支援(9 語言)
|
||||
- ⏸️ "Preview" 按鈕:未實作(延後至未來)
|
||||
- ⏸️ 審核日誌 UI:未實作(僅後端記錄)
|
||||
- **企業採用關鍵** - 無此功能將阻擋 B2B 市場
|
||||
- **基礎支撐**:A6 Pipeline Stage 1(Sanitizer)
|
||||
- **文件**:參見 `.planning/S6_PII_SANITIZATION.md`
|
||||
- [ ] **S7**: 季度安全稽核 - 🟢 Low(定期執行)
|
||||
- **自動化**(CI/CD):
|
||||
- OWASP ZAP 滲透測試
|
||||
@@ -825,12 +831,14 @@ graph TD
|
||||
|
||||
**優先級**: 安全性 → 功能
|
||||
|
||||
- [ ] **S6** PII 淨化
|
||||
- 企業採用關鍵需求(阻擋 B2B 市場)
|
||||
- 後端:`analyzer.py` 基於 Regex 的路徑移除
|
||||
- 前端:隱私開關、預覽 diff、審核日誌
|
||||
- 零風險,純預處理
|
||||
- 可在 `main` 分支實作
|
||||
- [x] **S6** PII 淨化 ✅ *已完成 (2025-12-31)*
|
||||
- ✅ 企業採用關鍵需求(移除 B2B 市場阻礙)
|
||||
- ✅ 後端:`sanitizer.py` 模組(3 種淨化等級)
|
||||
- ✅ 前端:隱私模式下拉選單(9 語言支援)
|
||||
- ✅ 零風險,純預處理
|
||||
- ✅ 已在 `main` 分支實作
|
||||
- ✅ 完整的單元測試(21 項測試)
|
||||
- 參見 `.planning/S6_PII_SANITIZATION.md` 了解詳情
|
||||
- [ ] **F7** 智慧參數注入
|
||||
- 殺手級 UX 功能 - 聊天內一鍵修復
|
||||
- LLM prompt 工程 + 前端實作
|
||||
|
||||
+48
-6
@@ -23,6 +23,7 @@ from .i18n import set_language, get_language, get_ui_text, SUPPORTED_LANGUAGES,
|
||||
from .config import CONFIG
|
||||
from .session_manager import SessionManager
|
||||
from .system_info import get_system_environment, format_env_for_llm
|
||||
from .sanitizer import PIISanitizer, SanitizationLevel
|
||||
|
||||
# --- LLM Environment Variable Fallbacks ---
|
||||
# These can be set in system environment to provide default values
|
||||
@@ -357,8 +358,9 @@ try:
|
||||
base_url = data.get("base_url", "https://api.openai.com/v1")
|
||||
model = data.get("model", "gpt-4o")
|
||||
language = data.get("language", "en")
|
||||
privacy_mode = data.get("privacy_mode", "basic") # S6: PII sanitization level
|
||||
|
||||
logger.info(f"Analyze API called - error_length={len(error_text) if error_text else 0}, has_workflow={bool(workflow)}, model={model}")
|
||||
logger.info(f"Analyze API called - error_length={len(error_text) if error_text else 0}, has_workflow={bool(workflow)}, model={model}, privacy={privacy_mode}")
|
||||
|
||||
# S2: SSRF protection - validate base URL
|
||||
is_valid, ssrf_error = validate_ssrf_url(base_url)
|
||||
@@ -377,11 +379,31 @@ try:
|
||||
if not error_text:
|
||||
return web.json_response({"error": "No error text provided"}, status=400)
|
||||
|
||||
# S6: PII Sanitization - Remove sensitive info before sending to LLM
|
||||
try:
|
||||
sanitization_level = SanitizationLevel(privacy_mode)
|
||||
except ValueError:
|
||||
sanitization_level = SanitizationLevel.BASIC
|
||||
|
||||
sanitizer = PIISanitizer(sanitization_level)
|
||||
|
||||
# Sanitize error text
|
||||
sanitization_result = sanitizer.sanitize(error_text)
|
||||
error_text = sanitization_result.sanitized_text
|
||||
|
||||
# Log sanitization metadata (for audit)
|
||||
if sanitization_result.pii_found:
|
||||
logger.info(f"PII sanitized: {sanitization_result.replacements}")
|
||||
|
||||
# Sanitize node context (paths, custom_node_path)
|
||||
if node_context:
|
||||
node_context = sanitizer.sanitize_dict(node_context)
|
||||
|
||||
# Truncate error text to prevent token overflow (roughly 8000 chars ≈ 2000 tokens)
|
||||
MAX_ERROR_LENGTH = 8000
|
||||
if len(error_text) > MAX_ERROR_LENGTH:
|
||||
error_text = error_text[:MAX_ERROR_LENGTH] + "\n\n[... truncated ...]"
|
||||
|
||||
|
||||
# R8: Smart workflow truncation (preserves error-related nodes)
|
||||
if workflow:
|
||||
from .truncate_workflow import truncate_workflow_smart
|
||||
@@ -559,8 +581,9 @@ try:
|
||||
stream = data.get("stream", True)
|
||||
intent = data.get("intent", "chat") # New: intent parameter
|
||||
selected_nodes = data.get("selected_nodes", []) # New: node selection context
|
||||
|
||||
logger.info(f"Chat API called - model={model}, intent={intent}, messages={len(messages)}, stream={stream}")
|
||||
privacy_mode = data.get("privacy_mode", "basic") # S6: PII sanitization level
|
||||
|
||||
logger.info(f"Chat API called - model={model}, intent={intent}, messages={len(messages)}, stream={stream}, privacy={privacy_mode}")
|
||||
|
||||
# S2: SSRF protection - validate base URL
|
||||
is_valid, ssrf_error = validate_ssrf_url(base_url)
|
||||
@@ -575,12 +598,31 @@ try:
|
||||
|
||||
if not messages:
|
||||
return web.json_response({"error": "No messages provided"}, status=400)
|
||||
|
||||
|
||||
# S6: PII Sanitization - Remove sensitive info before sending to LLM
|
||||
try:
|
||||
sanitization_level = SanitizationLevel(privacy_mode)
|
||||
except ValueError:
|
||||
sanitization_level = SanitizationLevel.BASIC
|
||||
|
||||
sanitizer = PIISanitizer(sanitization_level)
|
||||
|
||||
# Build system prompt with error context
|
||||
error_text = error_context.get("error", "")
|
||||
node_context = error_context.get("node_context", {})
|
||||
workflow = error_context.get("workflow", "")
|
||||
|
||||
|
||||
# Sanitize error context
|
||||
if error_text:
|
||||
error_text = sanitizer.sanitize(error_text).sanitized_text
|
||||
if node_context:
|
||||
node_context = sanitizer.sanitize_dict(node_context)
|
||||
|
||||
# Sanitize user messages (only user role, not assistant responses)
|
||||
for msg in messages:
|
||||
if msg.get("role") == "user" and isinstance(msg.get("content"), str):
|
||||
msg["content"] = sanitizer.sanitize(msg["content"]).sanitized_text
|
||||
|
||||
# Truncate to prevent token overflow
|
||||
MAX_ERROR_LENGTH = 4000
|
||||
if len(error_text) > MAX_ERROR_LENGTH:
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 253 KiB After Width: | Height: | Size: 318 KiB |
@@ -47,6 +47,11 @@ UI_TEXT: Dict[str, Dict[str, str]] = {
|
||||
"sidebar_hint": "Open the Doctor sidebar (left panel) to analyze with AI",
|
||||
"locate_node_btn": "Locate Node on Canvas",
|
||||
"no_errors": "No active errors detected.",
|
||||
"privacy_mode": "Privacy Mode",
|
||||
"privacy_mode_none": "None (No sanitization)",
|
||||
"privacy_mode_basic": "Basic (Recommended)",
|
||||
"privacy_mode_strict": "Strict (Maximum privacy)",
|
||||
"privacy_mode_hint": "Controls what sensitive information is removed before sending to AI",
|
||||
},
|
||||
"zh_TW": {
|
||||
"info_title": "資訊",
|
||||
@@ -56,6 +61,11 @@ UI_TEXT: Dict[str, Dict[str, str]] = {
|
||||
"sidebar_hint": "點擊左側 Doctor 側邊欄以使用 AI 分析錯誤",
|
||||
"locate_node_btn": "在畫布上定位節點",
|
||||
"no_errors": "目前沒有偵測到錯誤。",
|
||||
"privacy_mode": "隱私模式",
|
||||
"privacy_mode_none": "無(不過濾)",
|
||||
"privacy_mode_basic": "基本(建議)",
|
||||
"privacy_mode_strict": "嚴格(最大隱私)",
|
||||
"privacy_mode_hint": "控制發送給 AI 前移除哪些敏感資訊",
|
||||
},
|
||||
"zh_CN": {
|
||||
"info_title": "信息",
|
||||
@@ -65,6 +75,11 @@ UI_TEXT: Dict[str, Dict[str, str]] = {
|
||||
"sidebar_hint": "点击左侧 Doctor 侧边栏以使用 AI 分析错误",
|
||||
"locate_node_btn": "在画布上定位节点",
|
||||
"no_errors": "当前没有检测到错误。",
|
||||
"privacy_mode": "隐私模式",
|
||||
"privacy_mode_none": "无(不过滤)",
|
||||
"privacy_mode_basic": "基本(推荐)",
|
||||
"privacy_mode_strict": "严格(最大隐私)",
|
||||
"privacy_mode_hint": "控制发送给 AI 前移除哪些敏感信息",
|
||||
},
|
||||
"ja": {
|
||||
"info_title": "情報",
|
||||
@@ -74,6 +89,11 @@ UI_TEXT: Dict[str, Dict[str, str]] = {
|
||||
"sidebar_hint": "左側の Doctor サイドバーを開いて AI で分析します",
|
||||
"locate_node_btn": "キャンバス上のノードを見つける",
|
||||
"no_errors": "アクティブなエラーは検出されていません。",
|
||||
"privacy_mode": "プライバシーモード",
|
||||
"privacy_mode_none": "なし(サニタイズなし)",
|
||||
"privacy_mode_basic": "基本(推奨)",
|
||||
"privacy_mode_strict": "厳格(最大プライバシー)",
|
||||
"privacy_mode_hint": "AI に送信する前に削除される機密情報を制御",
|
||||
},
|
||||
"de": {
|
||||
"info_title": "INFO",
|
||||
@@ -83,6 +103,11 @@ UI_TEXT: Dict[str, Dict[str, str]] = {
|
||||
"sidebar_hint": "Öffnen Sie die Doctor-Seitenleiste (linkes Panel), um mit KI zu analysieren",
|
||||
"locate_node_btn": "Knoten auf Canvas finden",
|
||||
"no_errors": "Keine aktiven Fehler erkannt.",
|
||||
"privacy_mode": "Datenschutzmodus",
|
||||
"privacy_mode_none": "Keine (Keine Bereinigung)",
|
||||
"privacy_mode_basic": "Grundlegend (Empfohlen)",
|
||||
"privacy_mode_strict": "Streng (Maximaler Datenschutz)",
|
||||
"privacy_mode_hint": "Steuert, welche sensiblen Informationen vor dem Senden an die KI entfernt werden",
|
||||
},
|
||||
"fr": {
|
||||
"info_title": "INFO",
|
||||
@@ -92,6 +117,11 @@ UI_TEXT: Dict[str, Dict[str, str]] = {
|
||||
"sidebar_hint": "Ouvrez la barre latérale Doctor (panneau gauche) pour analyser avec l'IA",
|
||||
"locate_node_btn": "Localiser le nœud sur le canevas",
|
||||
"no_errors": "Aucune erreur active détectée.",
|
||||
"privacy_mode": "Mode de confidentialité",
|
||||
"privacy_mode_none": "Aucun (Pas de nettoyage)",
|
||||
"privacy_mode_basic": "De base (Recommandé)",
|
||||
"privacy_mode_strict": "Strict (Confidentialité maximale)",
|
||||
"privacy_mode_hint": "Contrôle quelles informations sensibles sont supprimées avant l'envoi à l'IA",
|
||||
},
|
||||
"it": {
|
||||
"info_title": "INFO",
|
||||
@@ -101,6 +131,11 @@ UI_TEXT: Dict[str, Dict[str, str]] = {
|
||||
"sidebar_hint": "Apri la barra laterale Doctor (pannello sinistro) per analizzare con l'IA",
|
||||
"locate_node_btn": "Trova nodo sulla tela",
|
||||
"no_errors": "Nessun errore attivo rilevato.",
|
||||
"privacy_mode": "Modalità privacy",
|
||||
"privacy_mode_none": "Nessuna (Nessuna pulizia)",
|
||||
"privacy_mode_basic": "Base (Consigliato)",
|
||||
"privacy_mode_strict": "Rigorosa (Privacy massima)",
|
||||
"privacy_mode_hint": "Controlla quali informazioni sensibili vengono rimosse prima dell'invio all'IA",
|
||||
},
|
||||
"es": {
|
||||
"info_title": "INFO",
|
||||
@@ -110,6 +145,11 @@ UI_TEXT: Dict[str, Dict[str, str]] = {
|
||||
"sidebar_hint": "Abra la barra lateral de Doctor (panel izquierdo) para analizar con IA",
|
||||
"locate_node_btn": "Localizar nodo en lienzo",
|
||||
"no_errors": "No se detectaron errores activos.",
|
||||
"privacy_mode": "Modo de privacidad",
|
||||
"privacy_mode_none": "Ninguno (Sin limpieza)",
|
||||
"privacy_mode_basic": "Básico (Recomendado)",
|
||||
"privacy_mode_strict": "Estricto (Privacidad máxima)",
|
||||
"privacy_mode_hint": "Controla qué información sensible se elimina antes de enviar a la IA",
|
||||
},
|
||||
"ko": {
|
||||
"info_title": "정보",
|
||||
@@ -119,6 +159,11 @@ UI_TEXT: Dict[str, Dict[str, str]] = {
|
||||
"sidebar_hint": "AI로 분석하려면 Doctor 사이드바(왼쪽 패널)를 여세요",
|
||||
"locate_node_btn": "캔버스에서 노드 찾기",
|
||||
"no_errors": "활성 오류가 감지되지 않았습니다.",
|
||||
"privacy_mode": "개인정보 보호 모드",
|
||||
"privacy_mode_none": "없음 (정화 없음)",
|
||||
"privacy_mode_basic": "기본 (권장)",
|
||||
"privacy_mode_strict": "엄격 (최대 개인정보 보호)",
|
||||
"privacy_mode_hint": "AI로 전송하기 전에 제거할 민감한 정보 제어",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
+313
@@ -0,0 +1,313 @@
|
||||
"""
|
||||
PII Sanitization Module for ComfyUI-Doctor.
|
||||
|
||||
Removes sensitive information (file paths, API keys, emails, IP addresses) from
|
||||
error messages before sending to LLM services. Critical for enterprise adoption
|
||||
and GDPR compliance.
|
||||
|
||||
Security Level:
|
||||
- none: No sanitization (default for local LLMs)
|
||||
- basic: Remove user paths and obvious API keys
|
||||
- strict: Remove all PII including emails, IPs, and usernames
|
||||
"""
|
||||
|
||||
import re
|
||||
import os
|
||||
from typing import Dict, Any, Optional
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class SanitizationLevel(Enum):
|
||||
"""Sanitization security levels."""
|
||||
NONE = "none"
|
||||
BASIC = "basic"
|
||||
STRICT = "strict"
|
||||
|
||||
|
||||
@dataclass
|
||||
class SanitizationResult:
|
||||
"""Result of sanitization operation with metadata."""
|
||||
sanitized_text: str
|
||||
pii_found: bool
|
||||
replacements: Dict[str, int] # Type -> count
|
||||
original_length: int
|
||||
sanitized_length: int
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert to dictionary for logging/preview."""
|
||||
return {
|
||||
"pii_found": self.pii_found,
|
||||
"replacements": self.replacements,
|
||||
"original_length": self.original_length,
|
||||
"sanitized_length": self.sanitized_length,
|
||||
"reduction_bytes": self.original_length - self.sanitized_length
|
||||
}
|
||||
|
||||
|
||||
class PIISanitizer:
|
||||
"""
|
||||
Sanitizes personally identifiable information from error tracebacks.
|
||||
|
||||
Removes:
|
||||
- Windows user paths: C:\\Users\\username\\... → <USER_PATH>\\...
|
||||
- Linux/macOS home dirs: /home/username/ → <USER_HOME>/...
|
||||
- API keys: sk-abc123... → <API_KEY>
|
||||
- Email addresses: user@example.com → <EMAIL>
|
||||
- Private IP addresses: 192.168.1.1 → <PRIVATE_IP>
|
||||
- Usernames in paths and URLs
|
||||
|
||||
Zero runtime overhead when level=NONE, GDPR-friendly.
|
||||
"""
|
||||
|
||||
# Regex patterns for different PII types
|
||||
PATTERNS = {
|
||||
# Windows user paths (C:\Users\username\...)
|
||||
"windows_user_path": (
|
||||
r'[A-Z]:\\Users\\[^\\\/\s]+',
|
||||
r'<USER_PATH>'
|
||||
),
|
||||
|
||||
# Linux/macOS home directories (/home/username/ or /Users/username/)
|
||||
"unix_home_path": (
|
||||
r'/(?:home|Users)/[^/\s]+',
|
||||
r'<USER_HOME>'
|
||||
),
|
||||
|
||||
# API keys (common patterns: sk-..., key_..., token_...)
|
||||
"api_key": (
|
||||
r'\b(?:sk-[a-zA-Z0-9_-]{20,}|key_[a-zA-Z0-9]{20,}|token_[a-zA-Z0-9]{20,}|[a-f0-9]{32,64})\b',
|
||||
r'<API_KEY>'
|
||||
),
|
||||
|
||||
# Email addresses
|
||||
"email": (
|
||||
r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
|
||||
r'<EMAIL>'
|
||||
),
|
||||
|
||||
# Private IPv4 addresses (10.x.x.x, 172.16-31.x.x, 192.168.x.x)
|
||||
"private_ipv4": (
|
||||
r'\b(?:10\.\d{1,3}\.\d{1,3}\.\d{1,3}|172\.(?:1[6-9]|2\d|3[01])\.\d{1,3}\.\d{1,3}|192\.168\.\d{1,3}\.\d{1,3})\b',
|
||||
r'<PRIVATE_IP>'
|
||||
),
|
||||
|
||||
# Localhost variants
|
||||
"localhost": (
|
||||
r'\b(?:127\.0\.0\.1|localhost|::1)\b',
|
||||
r'<LOCALHOST>'
|
||||
),
|
||||
|
||||
# Username in URLs (http://username:password@host or ssh://username@host)
|
||||
"url_credentials": (
|
||||
r'://([^:@\s]+)(?::([^@\s]+))?@',
|
||||
r'://<USER>@'
|
||||
),
|
||||
}
|
||||
|
||||
# Strict mode additional patterns
|
||||
STRICT_PATTERNS = {
|
||||
# Generic username patterns in file paths (conservative)
|
||||
"generic_username": (
|
||||
r'\\(?:Users|home)\\([A-Za-z][A-Za-z0-9_-]{2,20})\\',
|
||||
r'\\Users\\<USER>\\'
|
||||
),
|
||||
|
||||
# IPv6 private addresses (fc00::/7, fe80::/10)
|
||||
"private_ipv6": (
|
||||
r'\b(?:fc00|fe80):[0-9a-f:]+\b',
|
||||
r'<PRIVATE_IPV6>'
|
||||
),
|
||||
|
||||
# SSH keys fingerprints
|
||||
"ssh_fingerprint": (
|
||||
r'\b(?:SHA256|MD5):[A-Za-z0-9+/=]{32,}\b',
|
||||
r'<SSH_FINGERPRINT>'
|
||||
),
|
||||
}
|
||||
|
||||
def __init__(self, level: SanitizationLevel = SanitizationLevel.BASIC):
|
||||
"""
|
||||
Initialize sanitizer with specified security level.
|
||||
|
||||
Args:
|
||||
level: Sanitization level (NONE, BASIC, STRICT)
|
||||
"""
|
||||
self.level = level
|
||||
self._compiled_patterns = {}
|
||||
self._compile_patterns()
|
||||
|
||||
def _compile_patterns(self):
|
||||
"""Pre-compile all regex patterns for performance."""
|
||||
if self.level == SanitizationLevel.NONE:
|
||||
return
|
||||
|
||||
# Compile basic patterns
|
||||
for name, (pattern, _) in self.PATTERNS.items():
|
||||
self._compiled_patterns[name] = re.compile(pattern, re.IGNORECASE)
|
||||
|
||||
# Compile strict patterns if needed
|
||||
if self.level == SanitizationLevel.STRICT:
|
||||
for name, (pattern, _) in self.STRICT_PATTERNS.items():
|
||||
self._compiled_patterns[name] = re.compile(pattern, re.IGNORECASE)
|
||||
|
||||
def sanitize(self, text: str) -> SanitizationResult:
|
||||
"""
|
||||
Sanitize text by removing PII.
|
||||
|
||||
Args:
|
||||
text: Input text (error message, traceback, etc.)
|
||||
|
||||
Returns:
|
||||
SanitizationResult with sanitized text and metadata
|
||||
"""
|
||||
if self.level == SanitizationLevel.NONE or not text:
|
||||
return SanitizationResult(
|
||||
sanitized_text=text,
|
||||
pii_found=False,
|
||||
replacements={},
|
||||
original_length=len(text) if text else 0,
|
||||
sanitized_length=len(text) if text else 0
|
||||
)
|
||||
|
||||
sanitized = text
|
||||
replacements = {}
|
||||
|
||||
# Apply basic patterns
|
||||
for name, (pattern, replacement) in self.PATTERNS.items():
|
||||
compiled = self._compiled_patterns.get(name)
|
||||
if compiled:
|
||||
matches = compiled.findall(sanitized)
|
||||
if matches:
|
||||
replacements[name] = len(matches)
|
||||
sanitized = compiled.sub(replacement, sanitized)
|
||||
|
||||
# Apply strict patterns
|
||||
if self.level == SanitizationLevel.STRICT:
|
||||
for name, (pattern, replacement) in self.STRICT_PATTERNS.items():
|
||||
compiled = self._compiled_patterns.get(name)
|
||||
if compiled:
|
||||
matches = compiled.findall(sanitized)
|
||||
if matches:
|
||||
replacements[name] = len(matches)
|
||||
sanitized = compiled.sub(replacement, sanitized)
|
||||
|
||||
return SanitizationResult(
|
||||
sanitized_text=sanitized,
|
||||
pii_found=bool(replacements),
|
||||
replacements=replacements,
|
||||
original_length=len(text),
|
||||
sanitized_length=len(sanitized)
|
||||
)
|
||||
|
||||
def sanitize_dict(self, data: Dict[str, Any], keys_to_sanitize: list = None) -> Dict[str, Any]:
|
||||
"""
|
||||
Recursively sanitize string values in a dictionary.
|
||||
|
||||
Args:
|
||||
data: Dictionary to sanitize
|
||||
keys_to_sanitize: List of keys to sanitize (None = all string values)
|
||||
|
||||
Returns:
|
||||
New dictionary with sanitized values
|
||||
"""
|
||||
if self.level == SanitizationLevel.NONE:
|
||||
return data
|
||||
|
||||
if keys_to_sanitize is None:
|
||||
keys_to_sanitize = ["error", "traceback", "message", "path", "custom_node_path"]
|
||||
|
||||
sanitized_data = {}
|
||||
for key, value in data.items():
|
||||
if isinstance(value, str) and (not keys_to_sanitize or key in keys_to_sanitize):
|
||||
result = self.sanitize(value)
|
||||
sanitized_data[key] = result.sanitized_text
|
||||
elif isinstance(value, dict):
|
||||
sanitized_data[key] = self.sanitize_dict(value, keys_to_sanitize)
|
||||
elif isinstance(value, list):
|
||||
sanitized_data[key] = [
|
||||
self.sanitize_dict(item, keys_to_sanitize) if isinstance(item, dict)
|
||||
else self.sanitize(item).sanitized_text if isinstance(item, str)
|
||||
else item
|
||||
for item in value
|
||||
]
|
||||
else:
|
||||
sanitized_data[key] = value
|
||||
|
||||
return sanitized_data
|
||||
|
||||
def preview_diff(self, text: str, max_examples: int = 5) -> list:
|
||||
"""
|
||||
Generate a preview of what will be sanitized (for frontend display).
|
||||
|
||||
Args:
|
||||
text: Input text to analyze
|
||||
max_examples: Maximum number of examples per type
|
||||
|
||||
Returns:
|
||||
List of dicts with {type, original, replacement, count}
|
||||
"""
|
||||
if self.level == SanitizationLevel.NONE or not text:
|
||||
return []
|
||||
|
||||
preview = []
|
||||
all_patterns = dict(self.PATTERNS)
|
||||
if self.level == SanitizationLevel.STRICT:
|
||||
all_patterns.update(self.STRICT_PATTERNS)
|
||||
|
||||
for name, (pattern, replacement) in all_patterns.items():
|
||||
compiled = self._compiled_patterns.get(name)
|
||||
if compiled:
|
||||
matches = compiled.findall(text)
|
||||
if matches:
|
||||
# Get unique matches
|
||||
unique_matches = list(set(matches))[:max_examples]
|
||||
preview.append({
|
||||
"type": name,
|
||||
"replacement": replacement,
|
||||
"examples": unique_matches,
|
||||
"total_count": len(matches)
|
||||
})
|
||||
|
||||
return preview
|
||||
|
||||
|
||||
# Global sanitizer instance (initialized on first use)
|
||||
_global_sanitizer: Optional[PIISanitizer] = None
|
||||
|
||||
|
||||
def get_sanitizer(level: SanitizationLevel = SanitizationLevel.BASIC) -> PIISanitizer:
|
||||
"""
|
||||
Get or create the global sanitizer instance.
|
||||
|
||||
Args:
|
||||
level: Desired sanitization level
|
||||
|
||||
Returns:
|
||||
PIISanitizer instance
|
||||
"""
|
||||
global _global_sanitizer
|
||||
if _global_sanitizer is None or _global_sanitizer.level != level:
|
||||
_global_sanitizer = PIISanitizer(level)
|
||||
return _global_sanitizer
|
||||
|
||||
|
||||
def sanitize_for_llm(text: str, level: str = "basic") -> str:
|
||||
"""
|
||||
Convenience function to sanitize text for LLM transmission.
|
||||
|
||||
Args:
|
||||
text: Text to sanitize
|
||||
level: Sanitization level ("none", "basic", "strict")
|
||||
|
||||
Returns:
|
||||
Sanitized text
|
||||
"""
|
||||
try:
|
||||
sanitization_level = SanitizationLevel(level)
|
||||
except ValueError:
|
||||
sanitization_level = SanitizationLevel.BASIC
|
||||
|
||||
sanitizer = get_sanitizer(sanitization_level)
|
||||
result = sanitizer.sanitize(text)
|
||||
return result.sanitized_text
|
||||
@@ -0,0 +1,479 @@
|
||||
"""
|
||||
Unit tests for PII Sanitizer module.
|
||||
|
||||
Tests all sanitization patterns across different security levels to ensure
|
||||
sensitive information is properly removed before sending to LLMs.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from sanitizer import PIISanitizer, SanitizationLevel, sanitize_for_llm
|
||||
|
||||
|
||||
class TestPIISanitizerBasic:
|
||||
"""Tests for basic sanitization level."""
|
||||
|
||||
def test_windows_user_path_sanitization(self):
|
||||
"""Test Windows user path removal."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
test_cases = [
|
||||
(
|
||||
"Error in C:\\Users\\john_doe\\AppData\\Local\\temp.py",
|
||||
"Error in <USER_PATH>\\AppData\\Local\\temp.py"
|
||||
),
|
||||
(
|
||||
"File not found: C:\\Users\\Alice123\\Documents\\workflow.json",
|
||||
"File not found: <USER_PATH>\\Documents\\workflow.json"
|
||||
),
|
||||
(
|
||||
"Multiple paths: C:\\Users\\Bob\\test.py and C:\\Users\\Alice\\data.txt",
|
||||
"Multiple paths: <USER_PATH>\\test.py and <USER_PATH>\\data.txt"
|
||||
),
|
||||
]
|
||||
|
||||
for input_text, expected in test_cases:
|
||||
result = sanitizer.sanitize(input_text)
|
||||
assert result.sanitized_text == expected
|
||||
assert result.pii_found is True
|
||||
assert "windows_user_path" in result.replacements
|
||||
|
||||
def test_unix_home_path_sanitization(self):
|
||||
"""Test Linux/macOS home directory removal."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
test_cases = [
|
||||
(
|
||||
"Error in /home/john/comfyui/custom_nodes/test.py",
|
||||
"Error in <USER_HOME>/comfyui/custom_nodes/test.py"
|
||||
),
|
||||
(
|
||||
"File: /Users/alice/Documents/workflow.json",
|
||||
"File: <USER_HOME>/Documents/workflow.json"
|
||||
),
|
||||
(
|
||||
"Traceback: /home/bob/.config/app.conf",
|
||||
"Traceback: <USER_HOME>/.config/app.conf"
|
||||
),
|
||||
]
|
||||
|
||||
for input_text, expected in test_cases:
|
||||
result = sanitizer.sanitize(input_text)
|
||||
assert result.sanitized_text == expected
|
||||
assert result.pii_found is True
|
||||
assert "unix_home_path" in result.replacements
|
||||
|
||||
def test_api_key_sanitization(self):
|
||||
"""Test API key removal."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
test_cases = [
|
||||
(
|
||||
"Using key: sk-proj-abc123def456ghi789jkl012mno345pqr678stu901vwx234yz",
|
||||
"Using key: <API_KEY>"
|
||||
),
|
||||
(
|
||||
"Token: key_1234567890abcdef1234567890abcdef",
|
||||
"Token: <API_KEY>"
|
||||
),
|
||||
(
|
||||
"Auth with token_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"Auth with <API_KEY>"
|
||||
),
|
||||
(
|
||||
"Hash: a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2",
|
||||
"Hash: <API_KEY>"
|
||||
),
|
||||
]
|
||||
|
||||
for input_text, expected in test_cases:
|
||||
result = sanitizer.sanitize(input_text)
|
||||
assert result.sanitized_text == expected
|
||||
assert result.pii_found is True
|
||||
assert "api_key" in result.replacements
|
||||
|
||||
def test_email_sanitization(self):
|
||||
"""Test email address removal."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
test_cases = [
|
||||
(
|
||||
"Contact: user@example.com for support",
|
||||
"Contact: <EMAIL> for support"
|
||||
),
|
||||
(
|
||||
"Error from john.doe@company.org",
|
||||
"Error from <EMAIL>"
|
||||
),
|
||||
(
|
||||
"Multiple emails: alice@test.com and bob@example.net",
|
||||
"Multiple emails: <EMAIL> and <EMAIL>"
|
||||
),
|
||||
]
|
||||
|
||||
for input_text, expected in test_cases:
|
||||
result = sanitizer.sanitize(input_text)
|
||||
assert result.sanitized_text == expected
|
||||
assert result.pii_found is True
|
||||
assert "email" in result.replacements
|
||||
|
||||
def test_private_ipv4_sanitization(self):
|
||||
"""Test private IPv4 address removal."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
test_cases = [
|
||||
(
|
||||
"Connection to 192.168.1.100 failed",
|
||||
"Connection to <PRIVATE_IP> failed"
|
||||
),
|
||||
(
|
||||
"Server: 10.0.0.5:8080",
|
||||
"Server: <PRIVATE_IP>:8080"
|
||||
),
|
||||
(
|
||||
"Internal IP: 172.16.0.10",
|
||||
"Internal IP: <PRIVATE_IP>"
|
||||
),
|
||||
(
|
||||
"Multiple IPs: 192.168.0.1 and 10.1.1.1",
|
||||
"Multiple IPs: <PRIVATE_IP> and <PRIVATE_IP>"
|
||||
),
|
||||
]
|
||||
|
||||
for input_text, expected in test_cases:
|
||||
result = sanitizer.sanitize(input_text)
|
||||
assert result.sanitized_text == expected
|
||||
assert result.pii_found is True
|
||||
assert "private_ipv4" in result.replacements
|
||||
|
||||
def test_localhost_sanitization(self):
|
||||
"""Test localhost variants removal."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
test_cases = [
|
||||
(
|
||||
"Connecting to 127.0.0.1:8000",
|
||||
"Connecting to <LOCALHOST>:8000"
|
||||
),
|
||||
(
|
||||
"Server at localhost:3000",
|
||||
"Server at <LOCALHOST>:3000"
|
||||
),
|
||||
(
|
||||
"IPv6: ::1 failed",
|
||||
"IPv6: <LOCALHOST> failed"
|
||||
),
|
||||
]
|
||||
|
||||
for input_text, expected in test_cases:
|
||||
result = sanitizer.sanitize(input_text)
|
||||
assert result.sanitized_text == expected
|
||||
assert result.pii_found is True
|
||||
assert "localhost" in result.replacements
|
||||
|
||||
def test_url_credentials_sanitization(self):
|
||||
"""Test URL username/password removal."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
test_cases = [
|
||||
(
|
||||
"Git remote: https://john:secret123@github.com/repo.git",
|
||||
"Git remote: https<USER>@github.com/repo.git"
|
||||
),
|
||||
(
|
||||
"SSH: ssh://alice@server.com/path",
|
||||
"SSH: ssh<USER>@server.com/path"
|
||||
),
|
||||
(
|
||||
"FTP: ftp://user:pass@ftp.example.com/file",
|
||||
"FTP: ftp<USER>@ftp.example.com/file"
|
||||
),
|
||||
]
|
||||
|
||||
for input_text, expected in test_cases:
|
||||
result = sanitizer.sanitize(input_text)
|
||||
assert result.sanitized_text == expected
|
||||
assert result.pii_found is True
|
||||
assert "url_credentials" in result.replacements
|
||||
|
||||
def test_no_pii_found(self):
|
||||
"""Test that clean text is not modified."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
clean_texts = [
|
||||
"RuntimeError: CUDA out of memory",
|
||||
"File not found: model.safetensors",
|
||||
"Dimension mismatch at layer 5",
|
||||
"Module 'torch' has no attribute 'foo'",
|
||||
]
|
||||
|
||||
for text in clean_texts:
|
||||
result = sanitizer.sanitize(text)
|
||||
assert result.sanitized_text == text
|
||||
assert result.pii_found is False
|
||||
assert len(result.replacements) == 0
|
||||
|
||||
def test_sanitization_level_none(self):
|
||||
"""Test that NONE level does not sanitize anything."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.NONE)
|
||||
|
||||
text_with_pii = "Error in C:\\Users\\john\\test.py with key sk-abc123def456abc123def456abc123def456"
|
||||
result = sanitizer.sanitize(text_with_pii)
|
||||
|
||||
assert result.sanitized_text == text_with_pii
|
||||
assert result.pii_found is False
|
||||
assert len(result.replacements) == 0
|
||||
|
||||
|
||||
class TestPIISanitizerStrict:
|
||||
"""Tests for strict sanitization level."""
|
||||
|
||||
def test_private_ipv6_sanitization(self):
|
||||
"""Test private IPv6 address removal (strict mode only)."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.STRICT)
|
||||
|
||||
test_cases = [
|
||||
(
|
||||
"IPv6: fc00::1 connection failed",
|
||||
"IPv6: <PRIVATE_IPV6> connection failed"
|
||||
),
|
||||
(
|
||||
"Link-local: fe80::a1b2:c3d4:e5f6",
|
||||
"Link-local: <PRIVATE_IPV6>"
|
||||
),
|
||||
]
|
||||
|
||||
for input_text, expected in test_cases:
|
||||
result = sanitizer.sanitize(input_text)
|
||||
assert result.sanitized_text == expected
|
||||
assert result.pii_found is True
|
||||
assert "private_ipv6" in result.replacements
|
||||
|
||||
def test_ssh_fingerprint_sanitization(self):
|
||||
"""Test SSH fingerprint removal (strict mode only)."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.STRICT)
|
||||
|
||||
test_cases = [
|
||||
(
|
||||
"SSH fingerprint: SHA256:aBcDeFgHiJkLmNoPqRsTuVwXyZ1234567890ABCD",
|
||||
"SSH fingerprint: <SSH_FINGERPRINT>"
|
||||
),
|
||||
(
|
||||
"MD5:a1:b2:c3:d4:e5:f6:a7:b8:c9:d0:e1:f2:a3:b4:c5:d6",
|
||||
"MD5<SSH_FINGERPRINT>"
|
||||
),
|
||||
]
|
||||
|
||||
for input_text, expected in test_cases:
|
||||
result = sanitizer.sanitize(input_text)
|
||||
assert result.sanitized_text == expected
|
||||
assert result.pii_found is True
|
||||
assert "ssh_fingerprint" in result.replacements
|
||||
|
||||
|
||||
class TestPIISanitizerDict:
|
||||
"""Tests for dictionary sanitization."""
|
||||
|
||||
def test_sanitize_dict_simple(self):
|
||||
"""Test sanitizing a simple dictionary."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
data = {
|
||||
"error": "File not found at C:\\Users\\john\\test.py",
|
||||
"custom_node_path": "/home/alice/comfyui/nodes",
|
||||
"node_id": "42",
|
||||
"safe_field": "No PII here"
|
||||
}
|
||||
|
||||
result = sanitizer.sanitize_dict(data)
|
||||
|
||||
assert result["error"] == "File not found at <USER_PATH>\\test.py"
|
||||
assert result["custom_node_path"] == "<USER_HOME>/comfyui/nodes"
|
||||
assert result["node_id"] == "42" # Not sanitized (not in default keys)
|
||||
assert result["safe_field"] == "No PII here"
|
||||
|
||||
def test_sanitize_dict_nested(self):
|
||||
"""Test sanitizing nested dictionaries."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
data = {
|
||||
"error": "Error in C:\\Users\\bob\\file.py",
|
||||
"context": {
|
||||
"path": "/home/charlie/test",
|
||||
"message": "Connection to 192.168.1.1 failed"
|
||||
}
|
||||
}
|
||||
|
||||
result = sanitizer.sanitize_dict(data)
|
||||
|
||||
assert result["error"] == "Error in <USER_PATH>\\file.py"
|
||||
assert result["context"]["path"] == "<USER_HOME>/test"
|
||||
assert result["context"]["message"] == "Connection to <PRIVATE_IP> failed"
|
||||
|
||||
def test_sanitize_dict_with_list(self):
|
||||
"""Test sanitizing dictionaries containing lists."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
data = {
|
||||
"traceback": [
|
||||
"Line 1: C:\\Users\\test\\app.py",
|
||||
"Line 2: /home/user/script.py",
|
||||
"Line 3: No PII"
|
||||
]
|
||||
}
|
||||
|
||||
result = sanitizer.sanitize_dict(data)
|
||||
|
||||
assert result["traceback"][0] == "Line 1: <USER_PATH>\\app.py"
|
||||
assert result["traceback"][1] == "Line 2: <USER_HOME>/script.py"
|
||||
assert result["traceback"][2] == "Line 3: No PII"
|
||||
|
||||
|
||||
class TestPIISanitizerMetadata:
|
||||
"""Tests for sanitization metadata and preview."""
|
||||
|
||||
def test_sanitization_result_metadata(self):
|
||||
"""Test that sanitization result includes correct metadata."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
text = "Error in C:\\Users\\john\\test.py with API key sk-abc123def456abc123def456abc123def456"
|
||||
result = sanitizer.sanitize(text)
|
||||
|
||||
assert result.pii_found is True
|
||||
assert result.original_length == len(text)
|
||||
assert result.sanitized_length < result.original_length
|
||||
assert "windows_user_path" in result.replacements
|
||||
assert "api_key" in result.replacements
|
||||
assert result.replacements["windows_user_path"] == 1
|
||||
assert result.replacements["api_key"] == 1
|
||||
|
||||
def test_preview_diff(self):
|
||||
"""Test preview diff generation."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
text = "Errors: C:\\Users\\alice\\test.py and user@example.com"
|
||||
preview = sanitizer.preview_diff(text)
|
||||
|
||||
assert len(preview) == 2
|
||||
|
||||
# Check windows path preview
|
||||
path_preview = next(p for p in preview if p["type"] == "windows_user_path")
|
||||
assert path_preview["replacement"] == "<USER_PATH>"
|
||||
assert len(path_preview["examples"]) > 0
|
||||
assert path_preview["total_count"] == 1
|
||||
|
||||
# Check email preview
|
||||
email_preview = next(p for p in preview if p["type"] == "email")
|
||||
assert email_preview["replacement"] == "<EMAIL>"
|
||||
assert "user@example.com" in email_preview["examples"]
|
||||
assert email_preview["total_count"] == 1
|
||||
|
||||
def test_to_dict_serialization(self):
|
||||
"""Test SanitizationResult to_dict conversion."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
text = "Error in C:\\Users\\test\\file.py"
|
||||
result = sanitizer.sanitize(text)
|
||||
result_dict = result.to_dict()
|
||||
|
||||
assert "pii_found" in result_dict
|
||||
assert "replacements" in result_dict
|
||||
assert "original_length" in result_dict
|
||||
assert "sanitized_length" in result_dict
|
||||
assert "reduction_bytes" in result_dict
|
||||
assert result_dict["reduction_bytes"] == result.original_length - result.sanitized_length
|
||||
|
||||
|
||||
class TestConvenienceFunction:
|
||||
"""Tests for the convenience sanitize_for_llm function."""
|
||||
|
||||
def test_sanitize_for_llm_default(self):
|
||||
"""Test default (basic) sanitization."""
|
||||
text = "Error in C:\\Users\\john\\test.py"
|
||||
result = sanitize_for_llm(text)
|
||||
|
||||
assert result == "Error in <USER_PATH>\\test.py"
|
||||
|
||||
def test_sanitize_for_llm_levels(self):
|
||||
"""Test different sanitization levels."""
|
||||
text = "Error at C:\\Users\\test\\file.py with key sk-abc123abc123abc123abc123abc123abc123"
|
||||
|
||||
# None level
|
||||
result_none = sanitize_for_llm(text, "none")
|
||||
assert result_none == text
|
||||
|
||||
# Basic level
|
||||
result_basic = sanitize_for_llm(text, "basic")
|
||||
assert "<USER_PATH>" in result_basic
|
||||
assert "<API_KEY>" in result_basic
|
||||
|
||||
# Strict level
|
||||
result_strict = sanitize_for_llm(text, "strict")
|
||||
assert "<USER_PATH>" in result_strict
|
||||
assert "<API_KEY>" in result_strict
|
||||
|
||||
def test_sanitize_for_llm_invalid_level(self):
|
||||
"""Test that invalid level defaults to BASIC."""
|
||||
text = "Error in C:\\Users\\john\\test.py"
|
||||
result = sanitize_for_llm(text, "invalid_level")
|
||||
|
||||
# Should default to BASIC
|
||||
assert result == "Error in <USER_PATH>\\test.py"
|
||||
|
||||
|
||||
class TestRealWorldScenarios:
|
||||
"""Tests with real-world error messages."""
|
||||
|
||||
def test_comfyui_traceback(self):
|
||||
"""Test sanitizing a typical ComfyUI traceback."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
traceback = """Traceback (most recent call last):
|
||||
File "C:\\Users\\john_doe\\ComfyUI\\execution.py", line 152, in recursive_execute
|
||||
output_data, output_ui = get_output_data(obj, input_data_all)
|
||||
File "/home/alice/ComfyUI/custom_nodes/my_node/loader.py", line 45, in load_model
|
||||
raise FileNotFoundError(f"Model not found at {model_path}")
|
||||
FileNotFoundError: Model not found at C:\\Users\\john_doe\\models\\checkpoint.safetensors
|
||||
Contact support@example.com for help. Server IP: 192.168.1.100"""
|
||||
|
||||
result = sanitizer.sanitize(traceback)
|
||||
|
||||
# Check all PII is removed
|
||||
assert "john_doe" not in result.sanitized_text
|
||||
assert "alice" not in result.sanitized_text
|
||||
assert "support@example.com" not in result.sanitized_text
|
||||
assert "192.168.1.100" not in result.sanitized_text
|
||||
|
||||
# Check placeholders are present
|
||||
assert "<USER_PATH>" in result.sanitized_text
|
||||
assert "<USER_HOME>" in result.sanitized_text
|
||||
assert "<EMAIL>" in result.sanitized_text
|
||||
assert "<PRIVATE_IP>" in result.sanitized_text
|
||||
|
||||
# Check structure is preserved
|
||||
assert "Traceback (most recent call last):" in result.sanitized_text
|
||||
assert "FileNotFoundError" in result.sanitized_text
|
||||
|
||||
def test_api_error_with_credentials(self):
|
||||
"""Test sanitizing API error with embedded credentials."""
|
||||
sanitizer = PIISanitizer(SanitizationLevel.BASIC)
|
||||
|
||||
error = """Failed to connect to API
|
||||
URL: https://john:password123@api.example.com/v1/generate
|
||||
API Key: sk-proj-abc123def456ghi789jkl012mno345pqr678stu901vwx234yz
|
||||
Response: 401 Unauthorized
|
||||
Please check your credentials or contact admin@company.com"""
|
||||
|
||||
result = sanitizer.sanitize(error)
|
||||
|
||||
assert "john" not in result.sanitized_text
|
||||
assert "password123" not in result.sanitized_text
|
||||
assert "sk-proj-" not in result.sanitized_text
|
||||
assert "admin@company.com" not in result.sanitized_text
|
||||
|
||||
assert "<USER>" in result.sanitized_text
|
||||
assert "<API_KEY>" in result.sanitized_text
|
||||
assert "<EMAIL>" in result.sanitized_text
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
pytest.main([__file__, "-v"])
|
||||
@@ -331,6 +331,7 @@ app.registerExtension({
|
||||
const currentBaseUrl = app.ui.settings.getSettingValue("Doctor.LLM.BaseUrl", "https://api.openai.com/v1");
|
||||
const currentApiKey = app.ui.settings.getSettingValue("Doctor.LLM.ApiKey", "");
|
||||
const currentModel = app.ui.settings.getSettingValue("Doctor.LLM.Model", "");
|
||||
const currentPrivacyMode = app.ui.settings.getSettingValue("Doctor.Privacy.Mode", "basic");
|
||||
|
||||
settingsPanel.innerHTML = `
|
||||
<h4 style="margin: 0 0 10px 0; font-size: 14px; color: #ddd;">Settings</h4>
|
||||
@@ -382,6 +383,17 @@ app.registerExtension({
|
||||
</div>
|
||||
<input type="text" id="doctor-model-input-manual" value="${currentModel}" placeholder="e.g., gpt-4o, deepseek-chat" style="width: 100%; padding: 6px; background: #111; border: 1px solid #444; border-radius: 4px; color: #eee; font-size: 13px; box-sizing: border-box; margin-top: 5px; display: none;" />
|
||||
</div>
|
||||
<div style="border-top: 1px solid #444; padding-top: 10px; margin-top: 5px;">
|
||||
<label style="display: flex; align-items: center; gap: 5px; font-size: 12px; color: #aaa; margin-bottom: 3px;">
|
||||
🔒 <span id="doctor-privacy-label">Privacy Mode</span>
|
||||
</label>
|
||||
<select id="doctor-privacy-select" style="width: 100%; padding: 6px; background: #111; border: 1px solid #444; border-radius: 4px; color: #eee; font-size: 13px;">
|
||||
<option value="none" ${currentPrivacyMode === 'none' ? 'selected' : ''} id="privacy-none-option">None (No sanitization)</option>
|
||||
<option value="basic" ${currentPrivacyMode === 'basic' ? 'selected' : ''} id="privacy-basic-option">Basic (Recommended)</option>
|
||||
<option value="strict" ${currentPrivacyMode === 'strict' ? 'selected' : ''} id="privacy-strict-option">Strict (Maximum privacy)</option>
|
||||
</select>
|
||||
<div id="doctor-privacy-hint" style="font-size: 11px; color: #888; margin-top: 3px;">Controls what sensitive information is removed before sending to AI</div>
|
||||
</div>
|
||||
<button id="doctor-save-settings-btn" style="width: 100%; padding: 8px; background: #4caf50; border: none; border-radius: 4px; color: white; font-weight: bold; cursor: pointer; font-size: 13px; margin-top: 5px;">💾 Save Settings</button>
|
||||
</div>
|
||||
`;
|
||||
@@ -497,6 +509,27 @@ app.registerExtension({
|
||||
loadModels();
|
||||
}
|
||||
|
||||
// Privacy mode translations update
|
||||
const privacySelect = settingsPanel.querySelector('#doctor-privacy-select');
|
||||
const updatePrivacyTranslations = (lang) => {
|
||||
const privacyLabel = settingsPanel.querySelector('#doctor-privacy-label');
|
||||
const privacyHint = settingsPanel.querySelector('#doctor-privacy-hint');
|
||||
const noneOption = settingsPanel.querySelector('#privacy-none-option');
|
||||
const basicOption = settingsPanel.querySelector('#privacy-basic-option');
|
||||
const strictOption = settingsPanel.querySelector('#privacy-strict-option');
|
||||
|
||||
fetch(`/doctor/ui_text?lang=${lang}`)
|
||||
.then(res => res.json())
|
||||
.then(text => {
|
||||
privacyLabel.textContent = text.privacy_mode || "Privacy Mode";
|
||||
privacyHint.textContent = text.privacy_mode_hint || "Controls what sensitive information is removed before sending to AI";
|
||||
noneOption.textContent = text.privacy_mode_none || "None (No sanitization)";
|
||||
basicOption.textContent = text.privacy_mode_basic || "Basic (Recommended)";
|
||||
strictOption.textContent = text.privacy_mode_strict || "Strict (Maximum privacy)";
|
||||
})
|
||||
.catch(err => console.error('[ComfyUI-Doctor] Failed to load privacy translations:', err));
|
||||
};
|
||||
|
||||
// Save settings button
|
||||
const saveBtn = settingsPanel.querySelector('#doctor-save-settings-btn');
|
||||
saveBtn.onclick = async () => {
|
||||
@@ -514,6 +547,10 @@ app.registerExtension({
|
||||
app.ui.settings.setSettingValue("Doctor.LLM.BaseUrl", baseUrlInput.value);
|
||||
app.ui.settings.setSettingValue("Doctor.LLM.ApiKey", apiKeyInput.value);
|
||||
app.ui.settings.setSettingValue("Doctor.LLM.Model", modelValue);
|
||||
app.ui.settings.setSettingValue("Doctor.Privacy.Mode", privacySelect.value);
|
||||
|
||||
// Update privacy translations on language change
|
||||
updatePrivacyTranslations(langSelect.value);
|
||||
|
||||
// Sync language with backend
|
||||
await DoctorAPI.setLanguage(langSelect.value);
|
||||
|
||||
+10
-2
@@ -435,6 +435,9 @@ export class DoctorUI {
|
||||
|
||||
console.log('[Doctor] LLM Settings:', { baseUrl, model, language, hasApiKey: !!apiKey });
|
||||
|
||||
// S6: Get privacy mode from settings
|
||||
const privacyMode = app.ui.settings.getSettingValue("Doctor.Privacy.Mode", "basic");
|
||||
|
||||
const payload = {
|
||||
messages: [{ role: 'user', content: text }],
|
||||
error_context: this.currentErrorData || {},
|
||||
@@ -443,7 +446,8 @@ export class DoctorUI {
|
||||
api_key: apiKey,
|
||||
base_url: baseUrl,
|
||||
model: model,
|
||||
language: language
|
||||
language: language,
|
||||
privacy_mode: privacyMode
|
||||
};
|
||||
|
||||
console.log('[Doctor] Payload prepared:', payload);
|
||||
@@ -668,6 +672,9 @@ export class DoctorUI {
|
||||
// F3: Capture workflow context
|
||||
const workflowContext = this.getWorkflowContext();
|
||||
|
||||
// S6: Get privacy mode from settings
|
||||
const privacyMode = app.ui.settings.getSettingValue("Doctor.Privacy.Mode", "basic");
|
||||
|
||||
const payload = {
|
||||
error: data.last_error || "Unknown Error",
|
||||
node_context: data.node_context,
|
||||
@@ -675,7 +682,8 @@ export class DoctorUI {
|
||||
api_key: apiKey,
|
||||
base_url: baseUrl,
|
||||
model: model,
|
||||
language: language
|
||||
language: language,
|
||||
privacy_mode: privacyMode
|
||||
};
|
||||
|
||||
const result = await DoctorAPI.analyzeError(payload);
|
||||
|
||||
Reference in New Issue
Block a user