diff --git a/README.md b/README.md index 90ee648..4803dd8 100644 --- a/README.md +++ b/README.md @@ -2,17 +2,17 @@ # ComfyUI Dynamic RAM Cache Control -A custom node for ComfyUI that dynamically manages RAM usage by intelligently controlling cache purging based on available system memory. +A custom node for ComfyUI that manages RAM cache objects inside a workflow, updates RAM cache thresholds, and triggers active cache purging. ## Features -- **Dual Cache Management Modes**: +- **Dual Cache Management Modes**: - CLASSIC (No Eviction): Standard cache behavior with no automatic purging - - RAM_PRESSURE (Auto Purge): Smart cache purging when system memory falls below the threshold + - RAM_PRESSURE (Auto Purge): RAM pressure cache behavior; per-node automatic release requires the executor to start the prompt in RAM_PRESSURE - **RAM Cache Extreme Cleanup (🧹)**: Performs a one-time aggressive purge and automatically restores the previous mode and thresholds. - **Configurable Memory Thresholds**: Supports the newer `--cache-ram active inactive` RAM threshold pair -- **Seamless Cache Migration**: Preserves essential cache data when switching between modes -- **Real-time Memory Monitoring**: Continuously checks system RAM and virtual memory availability +- **Cache Migration**: Preserves essential cache data when the cache mode changes +- **Runtime Compatibility Handling**: Avoids unsafe prompt-time executor mode changes on newer ComfyUI builds ## Installation @@ -20,6 +20,8 @@ A custom node for ComfyUI that dynamically manages RAM usage by intelligently co 2. Place the `ComfyUI_Dynamic-RAMCache` folder in your ComfyUI `custom_nodes` directory 3. Restart ComfyUI +Requires a ComfyUI build that includes `RAMPressureCache`, available upstream from 2025-10-31. + ## Usage 1. Add the "DynamicRAMCacheControl" node to your workflow @@ -52,7 +54,7 @@ A custom node for ComfyUI that dynamically manages RAM usage by intelligently co - **restore behavior**: - Automatically restores the mode and both thresholds that were active before cleanup -3. Connect the node anywhere in your workflow - it will continuously monitor and manage cache based on your settings +3. Connect the node anywhere in your workflow to update cache behavior and trigger active purging when it executes 4. Optionally add the "RAMCacheExtremeCleanup" node at the end of a workflow to trigger a one-shot cleanup and restore the previous state ## How It Works @@ -66,15 +68,35 @@ In CLASSIC mode, the node maintains standard cache behavior without automatic pu ### RAM_PRESSURE Mode -In RAM_PRESSURE mode, the node actively monitors system memory and intelligently purges cache when: -1. Available system RAM falls below the specified cleanup_threshold -2. Inactive cache / pinned memory exceeds the inactive_threshold behavior in newer ComfyUI builds +In RAM_PRESSURE mode, cache purging can happen in two ways: +1. If the executor starts the prompt in RAM_PRESSURE, ComfyUI can release RAM cache after each node +2. When this node executes, it can trigger an active purge using the configured thresholds When purging, it prioritizes removing older, less recently used cache items while attempting to preserve critical data. +## Upstream Change Dates + +- 2025-10-31: ComfyUI added `RAMPressureCache`, commit `513b0c46` (Add RAM Pressure cache mode) +- 2026-04-29: `PromptExecutor.execute_async()` started directly calling the prompt-start `ram_release_callback`, commit `fce03984` (dynamicVRAM + --cache-ram 2) +- 2026-05-21: RAM cache became the default cache mode and active/inactive thresholds were added, commit `5aa5ccc9` (Multi-threaded load of models from disk) + +This node's newer ComfyUI compatibility handling mainly targets the prompt-local RAM release callback behavior after 2026-04-29 and the default RAM cache behavior after 2026-05-21. + +## Newer ComfyUI Behavior + +Some newer ComfyUI builds capture the RAM release callback when a prompt starts. A custom node runs after prompt execution has already started, so changing the executor from CLASSIC to RAM_PRESSURE during the same prompt can cause `NoneType object is not callable`. + +To avoid that error, this node detects that executor behavior: + +- Newer ComfyUI builds usually start in RAM_PRESSURE/RAM cache mode. The node keeps that state, updates thresholds, and preserves per-node automatic cache release +- If the executor starts in CLASSIC, the node enables the RAMPressureCache object, migrates cache data, and performs an active purge, but keeps the executor mode unchanged for the current prompt +- If a workflow requests CLASSIC, the node keeps RAM_PRESSURE active on newer prompt-local callback builds to avoid making later prompts start from CLASSIC +- For full per-node automatic release, keep RAM_PRESSURE active before the prompt starts. If your ComfyUI build does not default to RAM cache, launch it with `--cache-ram` + ## Compatibility -- Compatible with all ComfyUI versions +- Requires a ComfyUI build that includes `RAMPressureCache` +- Handles both old and new `--cache-ram` argument shapes - Works with both CPU and GPU processing - Supports all standard ComfyUI nodes and workflows @@ -89,12 +111,22 @@ When purging, it prioritizes removing older, less recently used cache items whil - Check the console output for any error messages related to the dynamic cache control - Try restarting ComfyUI if you encounter any persistent issues +### "executor mode kept as CLASSIC" Warning +- This means the current ComfyUI build stores a prompt-local RAM release callback +- The node keeps the executor mode unchanged to avoid the `NoneType object is not callable` error +- Threshold updates, cache migration, and active purging still run, but executor-level per-node automatic release is not enabled for that prompt + +### "CLASSIC requested, RAM_PRESSURE kept active" Warning +- Newer ComfyUI builds usually start in RAM cache mode, so the node ignores CLASSIC mode requests inside a prompt +- RAM cache remains active; if you need to disable it, do that outside prompt execution through ComfyUI startup or global settings + ## Log Information The node outputs helpful log messages to track its operations, including: - Mode changes (CLASSIC to RAM_PRESSURE or vice versa) - Cache initialization and migration - Memory threshold updates +- Compatibility warnings when the executor mode is kept unchanged on newer ComfyUI builds - Cache purging events when in RAM_PRESSURE mode ## License diff --git a/README_zh.md b/README_zh.md index 87c2469..7e59429 100644 --- a/README_zh.md +++ b/README_zh.md @@ -2,20 +2,20 @@ # ComfyUI_Dynamic_RAMCache -一个ComfyUI的自定义节点，用于动态控制和优化RAM缓存管理，支持智能内存清理以提高大模型运行效率。 +一个ComfyUI的自定义节点，用于在工作流中控制RAM缓存对象、更新清理阈值，并触发主动内存清理。 ## 功能特点 -- **动态缓存模式切换**：支持在CLASSIC（无内存回收）和RAM_PRESSURE（自动内存清理）模式之间切换 -- **智能内存管理**：自动监控和清理 RAM 缓存，当内存不足时释放不必要的缓存数据 +- **动态缓存模式控制**：支持在CLASSIC（无内存回收）和RAM_PRESSURE（自动内存清理）模式之间调整 +- **智能内存管理**：在RAM_PRESSURE可用时清理 RAM 缓存，当内存不足时释放不必要的缓存数据 - **自定义清理阈值**：支持新版 `--cache-ram active inactive` 两个 RAM 阈值 -- **无缝数据迁移**：在切换缓存模式时保留现有缓存数据，避免重复计算 +- **缓存数据迁移**：在模式变化时保留现有缓存数据，减少重复计算 - **直观的节点接口**：简单易用的参数设置，适用于不同水平的用户 - **极限清理节点**：一次性强清理（包含虚拟内存占用）并自动恢复清理前的模式与阈值 ## 安装方法 -1. 确保已安装ComfyUI（支持2025.10.31及更新版本） +1. 确保已安装包含`RAMPressureCache`的ComfyUI版本（上游从2025-10-31开始提供） 2. 克隆或下载本仓库 3. 将`ComfyUI_Dynamic-RAMCache`文件夹复制到ComfyUI的`custom_nodes`目录下 4. 重启ComfyUI @@ -36,7 +36,7 @@ ### mode（模式） - **CLASSIC (No Eviction)**：传统缓存模式，不会自动清理缓存，可能导致内存使用持续增长 -- **RAM_PRESSURE (Auto Purge)**：自动内存清理模式，当可用内存低于设定阈值时会自动清理缓存 +- **RAM_PRESSURE (Auto Purge)**：RAM_PRESSURE缓存模式。执行器以该模式开始时，每个节点后会自动清理；从CLASSIC开始时，本节点只在自身执行时主动清理 ### cleanup_threshold（active 阈值） @@ -66,11 +66,30 @@ 1. 检测ComfyUI的`PromptExecutor`实例 2. 根据所选模式在`RAMPressureCache`和`HierarchicalCache`之间切换 3. 在切换过程中保留现有的缓存数据 -4. 当处于RAM_PRESSURE模式时，按 active / inactive 两个阈值触发缓存清理 +4. 当处于RAM_PRESSURE模式时，按 active / inactive 两个阈值处理缓存清理；执行器级别的每节点自动释放只在prompt开始前已经启用RAM_PRESSURE时生效 + +## 上游变更日期 + +- 2025-10-31：ComfyUI加入`RAMPressureCache`模式，提交`513b0c46`（Add RAM Pressure cache mode） +- 2026-04-29：`PromptExecutor.execute_async()`开始直接调用prompt开始时保存的`ram_release_callback`，提交`fce03984`（dynamicVRAM + --cache-ram 2） +- 2026-05-21：RAM cache成为默认缓存模式，并加入active/inactive双阈值，提交`5aa5ccc9`（Multi-threaded load of models from disk） + +本节点的新版兼容处理主要针对2026-04-29之后的prompt-local RAM释放回调行为，以及2026-05-21之后默认启用RAM cache的行为。 + +## 新版ComfyUI行为说明 + +部分新版ComfyUI会在一个prompt开始时保存RAM释放回调。自定义节点是在prompt运行中执行的，这时再把执行器从CLASSIC改成RAM_PRESSURE，可能触发`NoneType object is not callable`。 + +为避免这个错误，节点会检测这种执行器实现： + +- 新版ComfyUI通常已经以RAM_PRESSURE/RAM cache模式开始，节点会保持这个状态，更新阈值，并保留每个节点执行后的自动清理行为 +- 如果执行器以CLASSIC开始，节点会启用RAMPressureCache对象、迁移缓存并执行一次主动清理，但不会在当前prompt内改执行器模式 +- 如果工作流请求CLASSIC，节点会在新版prompt-local回调实现中保留RAM_PRESSURE，避免把下一次prompt带入CLASSIC状态 +- 需要完整的每节点自动清理时，请保持prompt开始前已经处于RAM_PRESSURE；如果你的ComfyUI版本不是默认RAM cache，再使用`--cache-ram` ## 兼容性要求 -- **ComfyUI版本**：需要2025.10.31或更新版本（包含RAMPressureCache类） +- **ComfyUI版本**：需要包含RAMPressureCache类的版本；新版和旧版`--cache-ram`参数形式都会处理 - **Python环境**：与ComfyUI兼容的Python环境 - **依赖**：使用ComfyUI内置的caching和execution模块 @@ -87,21 +106,30 @@ - 解决方案：检查ComfyUI版本，确认是否需要更新插件 3. **内存清理不生效** - - 原因：可能清理阈值设置过高或系统内存监控不工作 - - 解决方案：尝试降低cleanup_threshold值，或检查系统内存使用情况 + - 原因：可能清理阈值设置过高，或当前prompt是从CLASSIC执行器模式开始 + - 解决方案：尝试降低cleanup_threshold值；需要每个节点执行后的自动清理时，保持prompt开始前已经处于RAM_PRESSURE + +4. **出现"executor mode kept as CLASSIC"警告** + - 原因：当前ComfyUI在prompt开始时保存RAM释放回调，节点避免在prompt运行中改变执行器模式 + - 影响：节点仍会更新阈值、迁移缓存并执行主动清理，但当前prompt不会获得执行器级别的每节点自动释放 + +5. **出现"CLASSIC requested, RAM_PRESSURE kept active"警告** + - 原因：新版ComfyUI默认已经是RAM cache模式，节点忽略工作流里的CLASSIC切换请求，避免后续prompt从CLASSIC开始 + - 影响：RAM cache继续启用；如果确实要关闭RAM cache，请在prompt外调整ComfyUI启动参数或全局设置 ## 注意事项 - 在大型工作流中，过低的清理阈值可能导致频繁的缓存清理，反而降低性能 - 建议根据系统实际RAM大小调整清理阈值，一般建议设置为总内存的10-20% -- 首次切换到RAM_PRESSURE模式时可能需要一些时间来初始化新的缓存系统 +- 首次使用RAM_PRESSURE模式时可能需要一些时间来初始化新的缓存系统 ## 日志信息 该节点会在以下情况输出日志： -- 缓存模式切换 +- 缓存模式变化 - 清理阈值更新 +- 新版ComfyUI中保留CLASSIC执行器模式的兼容警告 - 错误和警告信息 日志可在ComfyUI的控制台或日志文件中查看，前缀为`[DynamicRAMCache]` diff --git a/nodes.py b/nodes.py index b686525..8f1e0e8 100644 --- a/nodes.py +++ b/nodes.py @@ -110,20 +110,29 @@ class DynamicRAMCacheControl: logging.error("[DynamicRAMCache] HierarchicalCache class not available in caching module") return - is_currently_ram = isinstance(current_cache, RAMPressureCacheClass) - + is_currently_ram = isinstance(current_cache, RAMPressureCacheClass) + if target_mode_ram and not is_currently_ram: self._switch_to_ram_pressure(cache_set, current_cache, caching) - self._set_executor_cache_type(executor, target_mode_ram) - logging.info(self._format_headroom_log("[DynamicRAMCache] Switched mode: CLASSIC -> RAM_PRESSURE", active_headroom, inactive_headroom, supports_inactive)) + if self._can_set_executor_ram_type(executor): + self._set_executor_cache_type(executor, target_mode_ram) + logging.info(self._format_headroom_log("[DynamicRAMCache] Switched mode: CLASSIC -> RAM_PRESSURE", active_headroom, inactive_headroom, supports_inactive)) + else: + logging.warning(self._format_headroom_log("[DynamicRAMCache] RAM_PRESSURE cache active, executor mode kept as CLASSIC to avoid a prompt-local None callback", active_headroom, inactive_headroom, supports_inactive)) elif not target_mode_ram and is_currently_ram: - self._switch_to_classic(cache_set, current_cache, caching) - self._set_executor_cache_type(executor, target_mode_ram) - logging.info(f"[DynamicRAMCache] Switched mode: RAM_PRESSURE -> CLASSIC") + if self._should_keep_ram_mode(executor): + logging.warning(self._format_headroom_log("[DynamicRAMCache] CLASSIC requested, RAM_PRESSURE kept active to avoid a prompt-local callback mismatch", active_headroom, inactive_headroom, supports_inactive)) + else: + self._switch_to_classic(cache_set, current_cache, caching) + self._set_executor_cache_type(executor, target_mode_ram) + logging.info(f"[DynamicRAMCache] Switched mode: RAM_PRESSURE -> CLASSIC") elif target_mode_ram and is_currently_ram: - self._set_executor_cache_type(executor, target_mode_ram) + if self._can_set_executor_ram_type(executor): + self._set_executor_cache_type(executor, target_mode_ram) + else: + logging.warning(self._format_headroom_log("[DynamicRAMCache] RAM_PRESSURE cache active, executor mode kept as CLASSIC to avoid a prompt-local None callback", active_headroom, inactive_headroom, supports_inactive)) if old_ram_arg != active_headroom or old_ram_inactive_arg != inactive_headroom: logging.info(self._format_headroom_update_log(old_ram_arg, old_ram_inactive_arg, active_headroom, inactive_headroom, supports_inactive)) @@ -180,6 +189,35 @@ class DynamicRAMCacheControl: except (OSError, TypeError): return False + def _can_set_executor_ram_type(self, executor): + if self._is_executor_ram_type(executor): + return True + return not self._uses_prompt_local_ram_release_callback() + + def _should_keep_ram_mode(self, executor): + return self._is_executor_ram_type(executor) and self._uses_prompt_local_ram_release_callback() + + def _is_executor_ram_type(self, executor): + CacheType = getattr(execution, 'CacheType', None) + if CacheType is None: + return False + + ram_type = getattr(CacheType, 'RAM_PRESSURE', None) + return ram_type is not None and getattr(executor, 'cache_type', None) == ram_type + + def _uses_prompt_local_ram_release_callback(self): + PromptExecutor = getattr(execution, 'PromptExecutor', None) + execute_async = getattr(PromptExecutor, 'execute_async', None) + if execute_async is None: + return True + + try: + source = inspect.getsource(execute_async) + except (OSError, TypeError): + return True + + return 'ram_release_callback' in source and 'self.cache_type == CacheType.RAM_PRESSURE' in source + def _set_executor_cache_type(self, executor, target_mode_ram): CacheType = getattr(execution, 'CacheType', None) if CacheType is None: