Commit Graph
1 Commits
Author SHA1 Message Date
diodiogod 565a525121 Implement stateless ChatterBox wrapper for thread-safe parallel processing
- Created StatelessChatterBoxWrapper to eliminate shared state corruption
- Wrapper calculates conditions fresh for each generation (no self.conds)
- Updated streaming processor to use stateless wrapper with batch_size control
- Modified streaming model manager to provide stateless wrappers
- Updated SRT nodes to use stateless generation in streaming mode
- Added comprehensive tensor detaching to prevent autograd issues
- Initial testing shows parallel generation working without immediate crashes

Note: Requires extensive testing to confirm stability with high worker counts
2025-08-11 22:52:15 -03:00