Compatibility update for new ComfyUI
This commit is contained in:
@@ -2,17 +2,17 @@
|
||||
|
||||
# ComfyUI Dynamic RAM Cache Control
|
||||
|
||||
A custom node for ComfyUI that dynamically manages RAM usage by intelligently controlling cache purging based on available system memory.
|
||||
A custom node for ComfyUI that manages RAM cache objects inside a workflow, updates RAM cache thresholds, and triggers active cache purging.
|
||||
|
||||
## Features
|
||||
|
||||
- **Dual Cache Management Modes**:
|
||||
- **Dual Cache Management Modes**:
|
||||
- CLASSIC (No Eviction): Standard cache behavior with no automatic purging
|
||||
- RAM_PRESSURE (Auto Purge): Smart cache purging when system memory falls below the threshold
|
||||
- RAM_PRESSURE (Auto Purge): RAM pressure cache behavior; per-node automatic release requires the executor to start the prompt in RAM_PRESSURE
|
||||
- **RAM Cache Extreme Cleanup (🧹)**: Performs a one-time aggressive purge and automatically restores the previous mode and thresholds.
|
||||
- **Configurable Memory Thresholds**: Supports the newer `--cache-ram active inactive` RAM threshold pair
|
||||
- **Seamless Cache Migration**: Preserves essential cache data when switching between modes
|
||||
- **Real-time Memory Monitoring**: Continuously checks system RAM and virtual memory availability
|
||||
- **Cache Migration**: Preserves essential cache data when the cache mode changes
|
||||
- **Runtime Compatibility Handling**: Avoids unsafe prompt-time executor mode changes on newer ComfyUI builds
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -20,6 +20,8 @@ A custom node for ComfyUI that dynamically manages RAM usage by intelligently co
|
||||
2. Place the `ComfyUI_Dynamic-RAMCache` folder in your ComfyUI `custom_nodes` directory
|
||||
3. Restart ComfyUI
|
||||
|
||||
Requires a ComfyUI build that includes `RAMPressureCache`, available upstream from 2025-10-31.
|
||||
|
||||
## Usage
|
||||
|
||||
1. Add the "DynamicRAMCacheControl" node to your workflow
|
||||
@@ -52,7 +54,7 @@ A custom node for ComfyUI that dynamically manages RAM usage by intelligently co
|
||||
- **restore behavior**:
|
||||
- Automatically restores the mode and both thresholds that were active before cleanup
|
||||
|
||||
3. Connect the node anywhere in your workflow - it will continuously monitor and manage cache based on your settings
|
||||
3. Connect the node anywhere in your workflow to update cache behavior and trigger active purging when it executes
|
||||
4. Optionally add the "RAMCacheExtremeCleanup" node at the end of a workflow to trigger a one-shot cleanup and restore the previous state
|
||||
|
||||
## How It Works
|
||||
@@ -66,15 +68,35 @@ In CLASSIC mode, the node maintains standard cache behavior without automatic pu
|
||||
|
||||
### RAM_PRESSURE Mode
|
||||
|
||||
In RAM_PRESSURE mode, the node actively monitors system memory and intelligently purges cache when:
|
||||
1. Available system RAM falls below the specified cleanup_threshold
|
||||
2. Inactive cache / pinned memory exceeds the inactive_threshold behavior in newer ComfyUI builds
|
||||
In RAM_PRESSURE mode, cache purging can happen in two ways:
|
||||
1. If the executor starts the prompt in RAM_PRESSURE, ComfyUI can release RAM cache after each node
|
||||
2. When this node executes, it can trigger an active purge using the configured thresholds
|
||||
|
||||
When purging, it prioritizes removing older, less recently used cache items while attempting to preserve critical data.
|
||||
|
||||
## Upstream Change Dates
|
||||
|
||||
- 2025-10-31: ComfyUI added `RAMPressureCache`, commit `513b0c46` (Add RAM Pressure cache mode)
|
||||
- 2026-04-29: `PromptExecutor.execute_async()` started directly calling the prompt-start `ram_release_callback`, commit `fce03984` (dynamicVRAM + --cache-ram 2)
|
||||
- 2026-05-21: RAM cache became the default cache mode and active/inactive thresholds were added, commit `5aa5ccc9` (Multi-threaded load of models from disk)
|
||||
|
||||
This node's newer ComfyUI compatibility handling mainly targets the prompt-local RAM release callback behavior after 2026-04-29 and the default RAM cache behavior after 2026-05-21.
|
||||
|
||||
## Newer ComfyUI Behavior
|
||||
|
||||
Some newer ComfyUI builds capture the RAM release callback when a prompt starts. A custom node runs after prompt execution has already started, so changing the executor from CLASSIC to RAM_PRESSURE during the same prompt can cause `NoneType object is not callable`.
|
||||
|
||||
To avoid that error, this node detects that executor behavior:
|
||||
|
||||
- Newer ComfyUI builds usually start in RAM_PRESSURE/RAM cache mode. The node keeps that state, updates thresholds, and preserves per-node automatic cache release
|
||||
- If the executor starts in CLASSIC, the node enables the RAMPressureCache object, migrates cache data, and performs an active purge, but keeps the executor mode unchanged for the current prompt
|
||||
- If a workflow requests CLASSIC, the node keeps RAM_PRESSURE active on newer prompt-local callback builds to avoid making later prompts start from CLASSIC
|
||||
- For full per-node automatic release, keep RAM_PRESSURE active before the prompt starts. If your ComfyUI build does not default to RAM cache, launch it with `--cache-ram`
|
||||
|
||||
## Compatibility
|
||||
|
||||
- Compatible with all ComfyUI versions
|
||||
- Requires a ComfyUI build that includes `RAMPressureCache`
|
||||
- Handles both old and new `--cache-ram` argument shapes
|
||||
- Works with both CPU and GPU processing
|
||||
- Supports all standard ComfyUI nodes and workflows
|
||||
|
||||
@@ -89,12 +111,22 @@ When purging, it prioritizes removing older, less recently used cache items whil
|
||||
- Check the console output for any error messages related to the dynamic cache control
|
||||
- Try restarting ComfyUI if you encounter any persistent issues
|
||||
|
||||
### "executor mode kept as CLASSIC" Warning
|
||||
- This means the current ComfyUI build stores a prompt-local RAM release callback
|
||||
- The node keeps the executor mode unchanged to avoid the `NoneType object is not callable` error
|
||||
- Threshold updates, cache migration, and active purging still run, but executor-level per-node automatic release is not enabled for that prompt
|
||||
|
||||
### "CLASSIC requested, RAM_PRESSURE kept active" Warning
|
||||
- Newer ComfyUI builds usually start in RAM cache mode, so the node ignores CLASSIC mode requests inside a prompt
|
||||
- RAM cache remains active; if you need to disable it, do that outside prompt execution through ComfyUI startup or global settings
|
||||
|
||||
## Log Information
|
||||
|
||||
The node outputs helpful log messages to track its operations, including:
|
||||
- Mode changes (CLASSIC to RAM_PRESSURE or vice versa)
|
||||
- Cache initialization and migration
|
||||
- Memory threshold updates
|
||||
- Compatibility warnings when the executor mode is kept unchanged on newer ComfyUI builds
|
||||
- Cache purging events when in RAM_PRESSURE mode
|
||||
|
||||
## License
|
||||
|
||||
+40
-12
@@ -2,20 +2,20 @@
|
||||
|
||||
# ComfyUI_Dynamic_RAMCache
|
||||
|
||||
一个ComfyUI的自定义节点,用于动态控制和优化RAM缓存管理,支持智能内存清理以提高大模型运行效率。
|
||||
一个ComfyUI的自定义节点,用于在工作流中控制RAM缓存对象、更新清理阈值,并触发主动内存清理。
|
||||
|
||||
## 功能特点
|
||||
|
||||
- **动态缓存模式切换**:支持在CLASSIC(无内存回收)和RAM_PRESSURE(自动内存清理)模式之间切换
|
||||
- **智能内存管理**:自动监控和清理 RAM 缓存,当内存不足时释放不必要的缓存数据
|
||||
- **动态缓存模式控制**:支持在CLASSIC(无内存回收)和RAM_PRESSURE(自动内存清理)模式之间调整
|
||||
- **智能内存管理**:在RAM_PRESSURE可用时清理 RAM 缓存,当内存不足时释放不必要的缓存数据
|
||||
- **自定义清理阈值**:支持新版 `--cache-ram active inactive` 两个 RAM 阈值
|
||||
- **无缝数据迁移**:在切换缓存模式时保留现有缓存数据,避免重复计算
|
||||
- **缓存数据迁移**:在模式变化时保留现有缓存数据,减少重复计算
|
||||
- **直观的节点接口**:简单易用的参数设置,适用于不同水平的用户
|
||||
- **极限清理节点**:一次性强清理(包含虚拟内存占用)并自动恢复清理前的模式与阈值
|
||||
|
||||
## 安装方法
|
||||
|
||||
1. 确保已安装ComfyUI(支持2025.10.31及更新版本)
|
||||
1. 确保已安装包含`RAMPressureCache`的ComfyUI版本(上游从2025-10-31开始提供)
|
||||
2. 克隆或下载本仓库
|
||||
3. 将`ComfyUI_Dynamic-RAMCache`文件夹复制到ComfyUI的`custom_nodes`目录下
|
||||
4. 重启ComfyUI
|
||||
@@ -36,7 +36,7 @@
|
||||
### mode(模式)
|
||||
|
||||
- **CLASSIC (No Eviction)**:传统缓存模式,不会自动清理缓存,可能导致内存使用持续增长
|
||||
- **RAM_PRESSURE (Auto Purge)**:自动内存清理模式,当可用内存低于设定阈值时会自动清理缓存
|
||||
- **RAM_PRESSURE (Auto Purge)**:RAM_PRESSURE缓存模式。执行器以该模式开始时,每个节点后会自动清理;从CLASSIC开始时,本节点只在自身执行时主动清理
|
||||
|
||||
### cleanup_threshold(active 阈值)
|
||||
|
||||
@@ -66,11 +66,30 @@
|
||||
1. 检测ComfyUI的`PromptExecutor`实例
|
||||
2. 根据所选模式在`RAMPressureCache`和`HierarchicalCache`之间切换
|
||||
3. 在切换过程中保留现有的缓存数据
|
||||
4. 当处于RAM_PRESSURE模式时,按 active / inactive 两个阈值触发缓存清理
|
||||
4. 当处于RAM_PRESSURE模式时,按 active / inactive 两个阈值处理缓存清理;执行器级别的每节点自动释放只在prompt开始前已经启用RAM_PRESSURE时生效
|
||||
|
||||
## 上游变更日期
|
||||
|
||||
- 2025-10-31:ComfyUI加入`RAMPressureCache`模式,提交`513b0c46`(Add RAM Pressure cache mode)
|
||||
- 2026-04-29:`PromptExecutor.execute_async()`开始直接调用prompt开始时保存的`ram_release_callback`,提交`fce03984`(dynamicVRAM + --cache-ram 2)
|
||||
- 2026-05-21:RAM cache成为默认缓存模式,并加入active/inactive双阈值,提交`5aa5ccc9`(Multi-threaded load of models from disk)
|
||||
|
||||
本节点的新版兼容处理主要针对2026-04-29之后的prompt-local RAM释放回调行为,以及2026-05-21之后默认启用RAM cache的行为。
|
||||
|
||||
## 新版ComfyUI行为说明
|
||||
|
||||
部分新版ComfyUI会在一个prompt开始时保存RAM释放回调。自定义节点是在prompt运行中执行的,这时再把执行器从CLASSIC改成RAM_PRESSURE,可能触发`NoneType object is not callable`。
|
||||
|
||||
为避免这个错误,节点会检测这种执行器实现:
|
||||
|
||||
- 新版ComfyUI通常已经以RAM_PRESSURE/RAM cache模式开始,节点会保持这个状态,更新阈值,并保留每个节点执行后的自动清理行为
|
||||
- 如果执行器以CLASSIC开始,节点会启用RAMPressureCache对象、迁移缓存并执行一次主动清理,但不会在当前prompt内改执行器模式
|
||||
- 如果工作流请求CLASSIC,节点会在新版prompt-local回调实现中保留RAM_PRESSURE,避免把下一次prompt带入CLASSIC状态
|
||||
- 需要完整的每节点自动清理时,请保持prompt开始前已经处于RAM_PRESSURE;如果你的ComfyUI版本不是默认RAM cache,再使用`--cache-ram`
|
||||
|
||||
## 兼容性要求
|
||||
|
||||
- **ComfyUI版本**:需要2025.10.31或更新版本(包含RAMPressureCache类)
|
||||
- **ComfyUI版本**:需要包含RAMPressureCache类的版本;新版和旧版`--cache-ram`参数形式都会处理
|
||||
- **Python环境**:与ComfyUI兼容的Python环境
|
||||
- **依赖**:使用ComfyUI内置的caching和execution模块
|
||||
|
||||
@@ -87,21 +106,30 @@
|
||||
- 解决方案:检查ComfyUI版本,确认是否需要更新插件
|
||||
|
||||
3. **内存清理不生效**
|
||||
- 原因:可能清理阈值设置过高或系统内存监控不工作
|
||||
- 解决方案:尝试降低cleanup_threshold值,或检查系统内存使用情况
|
||||
- 原因:可能清理阈值设置过高,或当前prompt是从CLASSIC执行器模式开始
|
||||
- 解决方案:尝试降低cleanup_threshold值;需要每个节点执行后的自动清理时,保持prompt开始前已经处于RAM_PRESSURE
|
||||
|
||||
4. **出现"executor mode kept as CLASSIC"警告**
|
||||
- 原因:当前ComfyUI在prompt开始时保存RAM释放回调,节点避免在prompt运行中改变执行器模式
|
||||
- 影响:节点仍会更新阈值、迁移缓存并执行主动清理,但当前prompt不会获得执行器级别的每节点自动释放
|
||||
|
||||
5. **出现"CLASSIC requested, RAM_PRESSURE kept active"警告**
|
||||
- 原因:新版ComfyUI默认已经是RAM cache模式,节点忽略工作流里的CLASSIC切换请求,避免后续prompt从CLASSIC开始
|
||||
- 影响:RAM cache继续启用;如果确实要关闭RAM cache,请在prompt外调整ComfyUI启动参数或全局设置
|
||||
|
||||
## 注意事项
|
||||
|
||||
- 在大型工作流中,过低的清理阈值可能导致频繁的缓存清理,反而降低性能
|
||||
- 建议根据系统实际RAM大小调整清理阈值,一般建议设置为总内存的10-20%
|
||||
- 首次切换到RAM_PRESSURE模式时可能需要一些时间来初始化新的缓存系统
|
||||
- 首次使用RAM_PRESSURE模式时可能需要一些时间来初始化新的缓存系统
|
||||
|
||||
## 日志信息
|
||||
|
||||
该节点会在以下情况输出日志:
|
||||
|
||||
- 缓存模式切换
|
||||
- 缓存模式变化
|
||||
- 清理阈值更新
|
||||
- 新版ComfyUI中保留CLASSIC执行器模式的兼容警告
|
||||
- 错误和警告信息
|
||||
|
||||
日志可在ComfyUI的控制台或日志文件中查看,前缀为`[DynamicRAMCache]`
|
||||
|
||||
@@ -110,20 +110,29 @@ class DynamicRAMCacheControl:
|
||||
logging.error("[DynamicRAMCache] HierarchicalCache class not available in caching module")
|
||||
return
|
||||
|
||||
is_currently_ram = isinstance(current_cache, RAMPressureCacheClass)
|
||||
|
||||
is_currently_ram = isinstance(current_cache, RAMPressureCacheClass)
|
||||
|
||||
if target_mode_ram and not is_currently_ram:
|
||||
self._switch_to_ram_pressure(cache_set, current_cache, caching)
|
||||
self._set_executor_cache_type(executor, target_mode_ram)
|
||||
logging.info(self._format_headroom_log("[DynamicRAMCache] Switched mode: CLASSIC -> RAM_PRESSURE", active_headroom, inactive_headroom, supports_inactive))
|
||||
if self._can_set_executor_ram_type(executor):
|
||||
self._set_executor_cache_type(executor, target_mode_ram)
|
||||
logging.info(self._format_headroom_log("[DynamicRAMCache] Switched mode: CLASSIC -> RAM_PRESSURE", active_headroom, inactive_headroom, supports_inactive))
|
||||
else:
|
||||
logging.warning(self._format_headroom_log("[DynamicRAMCache] RAM_PRESSURE cache active, executor mode kept as CLASSIC to avoid a prompt-local None callback", active_headroom, inactive_headroom, supports_inactive))
|
||||
|
||||
elif not target_mode_ram and is_currently_ram:
|
||||
self._switch_to_classic(cache_set, current_cache, caching)
|
||||
self._set_executor_cache_type(executor, target_mode_ram)
|
||||
logging.info(f"[DynamicRAMCache] Switched mode: RAM_PRESSURE -> CLASSIC")
|
||||
if self._should_keep_ram_mode(executor):
|
||||
logging.warning(self._format_headroom_log("[DynamicRAMCache] CLASSIC requested, RAM_PRESSURE kept active to avoid a prompt-local callback mismatch", active_headroom, inactive_headroom, supports_inactive))
|
||||
else:
|
||||
self._switch_to_classic(cache_set, current_cache, caching)
|
||||
self._set_executor_cache_type(executor, target_mode_ram)
|
||||
logging.info(f"[DynamicRAMCache] Switched mode: RAM_PRESSURE -> CLASSIC")
|
||||
|
||||
elif target_mode_ram and is_currently_ram:
|
||||
self._set_executor_cache_type(executor, target_mode_ram)
|
||||
if self._can_set_executor_ram_type(executor):
|
||||
self._set_executor_cache_type(executor, target_mode_ram)
|
||||
else:
|
||||
logging.warning(self._format_headroom_log("[DynamicRAMCache] RAM_PRESSURE cache active, executor mode kept as CLASSIC to avoid a prompt-local None callback", active_headroom, inactive_headroom, supports_inactive))
|
||||
if old_ram_arg != active_headroom or old_ram_inactive_arg != inactive_headroom:
|
||||
logging.info(self._format_headroom_update_log(old_ram_arg, old_ram_inactive_arg, active_headroom, inactive_headroom, supports_inactive))
|
||||
|
||||
@@ -180,6 +189,35 @@ class DynamicRAMCacheControl:
|
||||
except (OSError, TypeError):
|
||||
return False
|
||||
|
||||
def _can_set_executor_ram_type(self, executor):
|
||||
if self._is_executor_ram_type(executor):
|
||||
return True
|
||||
return not self._uses_prompt_local_ram_release_callback()
|
||||
|
||||
def _should_keep_ram_mode(self, executor):
|
||||
return self._is_executor_ram_type(executor) and self._uses_prompt_local_ram_release_callback()
|
||||
|
||||
def _is_executor_ram_type(self, executor):
|
||||
CacheType = getattr(execution, 'CacheType', None)
|
||||
if CacheType is None:
|
||||
return False
|
||||
|
||||
ram_type = getattr(CacheType, 'RAM_PRESSURE', None)
|
||||
return ram_type is not None and getattr(executor, 'cache_type', None) == ram_type
|
||||
|
||||
def _uses_prompt_local_ram_release_callback(self):
|
||||
PromptExecutor = getattr(execution, 'PromptExecutor', None)
|
||||
execute_async = getattr(PromptExecutor, 'execute_async', None)
|
||||
if execute_async is None:
|
||||
return True
|
||||
|
||||
try:
|
||||
source = inspect.getsource(execute_async)
|
||||
except (OSError, TypeError):
|
||||
return True
|
||||
|
||||
return 'ram_release_callback' in source and 'self.cache_type == CacheType.RAM_PRESSURE' in source
|
||||
|
||||
def _set_executor_cache_type(self, executor, target_mode_ram):
|
||||
CacheType = getattr(execution, 'CacheType', None)
|
||||
if CacheType is None:
|
||||
|
||||
Reference in New Issue
Block a user