Compare commits

..
Author SHA1 Message Date
Mel Massadian f9a0998cc3 Merge branch 'main' into dev/scene-detect 2025-01-01 05:12:03 +01:00
Mel Massadian 4168cd5b7b fix: 🐛 better defaults 2024-12-30 18:26:23 +01:00
Mel Massadian a5f0be432c feat: ✨ add scene detect node 2024-12-29 13:56:04 +01:00
28 changed files with 925 additions and 3251 deletions
+1 -5
View File
@@ -5,20 +5,16 @@ on:
tags:
- '*'
permissions:
issues: write
jobs:
publish-node:
name: Publish Custom Node to registry
runs-on: ubuntu-latest
if: ${{ github.repository_owner == 'melMass' }}
steps:
- name: ♻️ Check out code
uses: actions/checkout@v4
with:
submodules: true
- name: 📦 Publish Custom Node
uses: Comfy-Org/publish-node-action@v1
uses: Comfy-Org/publish-node-action@main
with:
personal_access_token: ${{ secrets.COMFY_REGISTRY_TOKEN }}
+2 -238
View File
@@ -3,193 +3,10 @@
This is an automated changelog based on the commits in this repository.
Check the notes in the [releases](https://github.com/melMass/comfy_mtb/releases) for more information.
## [main] - 2025-04-16
## [main] - 2024-03-07
### Bug Fixes
- 🐛 note+ breaking wfs ([af42134](https://github.com/melMass/comfy_mtb/commit/af421340286b234e4c0cfcd4143a9d8726ebf3d1))
- 🐛 ColorCorrect clamp issue ([a85e57b](https://github.com/melMass/comfy_mtb/commit/a85e57b18c7d3c765131873ffff523244ca9be73))
- 🐛 Whisper chunks processing ([8bf3545](https://github.com/melMass/comfy_mtb/commit/8bf3545fec5b2a180607d40394b025a1e09c14b6))
- 🐛 stackImages move to device ([d87e52e](https://github.com/melMass/comfy_mtb/commit/d87e52ea2c112fd95f257dcd6a54a5db77a34fc3))
- 🐛 bbox upscale from center ([55261bd](https://github.com/melMass/comfy_mtb/commit/55261bda7c33d088b62c5483e4483201e5a9ce77))
- 🐛 add MASK support for PickFromBatch ([0d264b9](https://github.com/melMass/comfy_mtb/commit/0d264b90a78d5a6719fb3ce71f4e9a642db4c950))
- 🐛 use addDOMWidget for Debug node ([46af602](https://github.com/melMass/comfy_mtb/commit/46af6027d6c87d0c29b8bb0fd1cc1dbdae993629))
- 🐛 use "modern" notation in toDevice ([b7ca8ed](https://github.com/melMass/comfy_mtb/commit/b7ca8ed1c6e117b71afd7696f55dcc3dbd5bad08))
- 🐛 handle missing submodules ([d61da30](https://github.com/melMass/comfy_mtb/commit/d61da304099ff5e4528e4beb1ecc2eb83cabaaa1))
- 🐛 add warnings about what each IO mode can do ([6608c0b](https://github.com/melMass/comfy_mtb/commit/6608c0b6d1cf8f7a9901214096f8c78bfe17056f))
- 🐛 soft deprecate compression h264 ([a757e1c](https://github.com/melMass/comfy_mtb/commit/a757e1c98b2abbd2221a15b77e89d772e02d1d82))
- 🐛 limit packages allowed to be installed from API ([d6e004c](https://github.com/melMass/comfy_mtb/commit/d6e004cce2c32f8e48b868e66b89f82da4887dc3))
- 🐛 ensure default settings (io sidebar) ([ed17fa2](https://github.com/melMass/comfy_mtb/commit/ed17fa2ef4688aadf305a6d51b32c13a0efd22d6))
- 🐛 spawn colour picker at pointer location ([e5482ae](https://github.com/melMass/comfy_mtb/commit/e5482aee5e3de07e8f055b3edc0fccc0e0f75c14)) by [@webfiltered](https://github.com/webfiltered) in [#223](https://github.com/melMass/comfy_mtb/pull/223)
- 🐛 i/o sidebar for custom paths ([62469a4](https://github.com/melMass/comfy_mtb/commit/62469a4dd96e32509171aad74fcae8d2bb0ec593))
### Features
- ⚡ add BatchFromFolder ([9618513](https://github.com/melMass/comfy_mtb/commit/96185132b83c182032e9f6e822561eb5699af517))
- ⚡ add use_normalized to TransformBatch2D ([d4a31bf](https://github.com/melMass/comfy_mtb/commit/d4a31bf19c2863df8dfc4cb9a3cd6683304949e4))
- [**breaking**] ⚡ add support for masks in BatchFLoatMath ([fc7ba08](https://github.com/melMass/comfy_mtb/commit/fc7ba084f6ed7880e88e28eb448ab0bd7d796824))
- ✨ add use_normalized to TransformImage ([4516aa9](https://github.com/melMass/comfy_mtb/commit/4516aa9cb4fcb12c946999d6dcc1501cc09011a3))
- ✨ add regex support for String Replace ([78946b0](https://github.com/melMass/comfy_mtb/commit/78946b0fa3c3cf5dfcee8c7c4c0921b722d09d1e)) by [@poetryiii](https://github.com/poetryiii) in [#233](https://github.com/melMass/comfy_mtb/pull/233)
- ✨ update diarization to 3.1 ([c30408f](https://github.com/melMass/comfy_mtb/commit/c30408f96d4df9c7d35545654401162090a74305)) by [@numz](https://github.com/numz)
- ✨ add "workflow" query to /mtb/view endpoint ([eb7cf89](https://github.com/melMass/comfy_mtb/commit/eb7cf89f173b2342b04e7b61dca3d12cfaf65bdb))
- ✨ add stretch_x and stretch_y to TransformImage ([22fce6f](https://github.com/melMass/comfy_mtb/commit/22fce6fdda135cbb1f1aad42c86aae166cba81b5))
- ✨ add AudioDuration node ([f471497](https://github.com/melMass/comfy_mtb/commit/f47149746ac1e418cda2007c38aafbb03946ce22))
- ✨ basic whisper nodes ([83cfc5c](https://github.com/melMass/comfy_mtb/commit/83cfc5c723d1a572af67ad14b52be4f8371a3c5f))
- ✨ add BboxForDimensions ([9405784](https://github.com/melMass/comfy_mtb/commit/940578476438eaa6a42e0056f1b7b319ee585334))
- ✨ improve the debug node ([cf7a9c4](https://github.com/melMass/comfy_mtb/commit/cf7a9c41e81e8dd461ab9dfa3c05bb8e2cdf2a67))
- ✨ add BatchImageToSublist and counterpart ([00173fa](https://github.com/melMass/comfy_mtb/commit/00173fa3fbca4c5b1ff3016cc5139705ce61ec20))
- ✨ add TensorOps ([a8cf465](https://github.com/melMass/comfy_mtb/commit/a8cf4650ff5cbd4975ef954b5829c772ee53250c))
- ✨ live update outputs grid ([7f7a62f](https://github.com/melMass/comfy_mtb/commit/7f7a62f832c865a13b9181daee79d3cfc21581e2)) by [@christian-byrne](https://github.com/christian-byrne) in [#229](https://github.com/melMass/comfy_mtb/pull/229)
- ✨ add SaveImage passthrough ([0eeb707](https://github.com/melMass/comfy_mtb/commit/0eeb707f34f51142def8e0ef7d351ee5028cb5e0))
- ✨ add filtering to TransformImage ([bae26a0](https://github.com/melMass/comfy_mtb/commit/bae26a07fb02dd518c621eba28986a51c5d086bc))
- ✨ add support for video in I/O sidebar ([c92d99a](https://github.com/melMass/comfy_mtb/commit/c92d99a8a37a64cfc285296f21452c4927a22774))
- ✨ add an extra static input to Stack Images ([3f6d082](https://github.com/melMass/comfy_mtb/commit/3f6d08294096918d50101a19083f9134305cc8c9)) in [#222](https://github.com/melMass/comfy_mtb/pull/222)
- ✨ add support for subdirs (i/o sidebar) ([52bd76e](https://github.com/melMass/comfy_mtb/commit/52bd76e19c8bd7e72986900e5dbfade0457ef7e0))
- ✨ add Batch Sequence Nodes ([827c64c](https://github.com/melMass/comfy_mtb/commit/827c64c43d52ebfb8acd2e5c4491c4b66e6b8f40))
- ✨ add support for more formats (I/O sidebar) ([8c629be](https://github.com/melMass/comfy_mtb/commit/8c629bee186b5ac991058018a788e4a836eef630))
### Miscellaneous Tasks
- 🧹 bump version ([d093d76](https://github.com/melMass/comfy_mtb/commit/d093d76efd87474a3ca82858147255038060ab17))
- 🧹 small adjustments ([01107c4](https://github.com/melMass/comfy_mtb/commit/01107c45f8539ff7c579e08e2a9075d93781b9a2))
- 🤖 update publish action workflow with permissions and version constraints ([0e48aaa](https://github.com/melMass/comfy_mtb/commit/0e48aaa3e4f1e440a5d7ab42df56b728ced03aca)) by [@robinjhuang](https://github.com/robinjhuang) in [#237](https://github.com/melMass/comfy_mtb/pull/237)
- 🧹 basic standalone detection ([3a25526](https://github.com/melMass/comfy_mtb/commit/3a25526e818a1af8f886d2ad5c27101c4a0caa8b))
- 🧹 rename type ([edcb3da](https://github.com/melMass/comfy_mtb/commit/edcb3da08bff66f9adcef8dcd37c3925e64d0135))
- 🧹 update env file ([fc908ba](https://github.com/melMass/comfy_mtb/commit/fc908ba0a528523b7c1e37e34fb32f430746de0d))
- 🧹 dev ([9a94371](https://github.com/melMass/comfy_mtb/commit/9a943714aada107bfd236e00fa1063872db7a834))
- 🧹 apply formatting ([58ae89f](https://github.com/melMass/comfy_mtb/commit/58ae89f8e0f0f8b42825722a6aebc04da39847b1))
### Refactor
- 📦 add model autodownload ([147edcf](https://github.com/melMass/comfy_mtb/commit/147edcfcbc09dd27a0c787f9da568fb850c3308a))
### Wip
- 🚧 loop drawing ([ead4b34](https://github.com/melMass/comfy_mtb/commit/ead4b34e6dd03ea4ed309b246ef31c995325aa08))
## New Contributors
* [@poetryiii](https://github.com/poetryiii) made their first contribution in [#233](https://github.com/melMass/comfy_mtb/pull/233)
* [@numz](https://github.com/numz) made their first contribution in [#](https://github.com/melMass/comfy_mtb/pull/)
* [@webfiltered](https://github.com/webfiltered) made their first contribution in [#223](https://github.com/melMass/comfy_mtb/pull/223)
## [0.2.0] - 2024-12-08
### Bug Fixes
- 🐛 remove mtb sidebar ([b0d52f7](https://github.com/melMass/comfy_mtb/commit/b0d52f73051368df6de2d1e10ad28ca56df72803))
- 🐛 always enable the I/O sidebar ([ec1cb1a](https://github.com/melMass/comfy_mtb/commit/ec1cb1ac17d14670aa756dfb1ae7542397b12559))
- 🐛 ui shifts on animation builder ([ecbb220](https://github.com/melMass/comfy_mtb/commit/ecbb220de6a05f2e506ec43f2b786be983166157))
- 🐛 category for settings ([b6fa571](https://github.com/melMass/comfy_mtb/commit/b6fa571fd2096ace60d03cab42dba9ca37d0cb27)) in [#211](https://github.com/melMass/comfy_mtb/pull/211)
- 🐛 new UI issues ([f272526](https://github.com/melMass/comfy_mtb/commit/f272526bfc5da95e95d42cb4c613a0b9585b2577))
- 🐛 disable old BOOL widget (legacy) ([8596b81](https://github.com/melMass/comfy_mtb/commit/8596b8184edb484c907475a77ac1dc9e4a5c92af))
- 🐛 pass ONNX providers explicitely ([43092e4](https://github.com/melMass/comfy_mtb/commit/43092e44a4ea17f90fcfb12372da634fe4b79557))
- 🐛 typo in mtb_widgets error catch ([80b5a0c](https://github.com/melMass/comfy_mtb/commit/80b5a0ca7459763e7662421bccd8636976eefddd)) by [@christian-byrne](https://github.com/christian-byrne) in [#197](https://github.com/melMass/comfy_mtb/pull/197)
- 🐛 doc widget sidebar offset in the new ui ([81b3bc1](https://github.com/melMass/comfy_mtb/commit/81b3bc1651f06ad2fa7938f810d3f406f5e7c41c))
- 🐛 don't fallback to eval ([997d2fb](https://github.com/melMass/comfy_mtb/commit/997d2fb13af6aadf36873ea2ea3317e56f405aef))
- 🐛 rework main utils ([c99b081](https://github.com/melMass/comfy_mtb/commit/c99b0812ab4a4183ef9298fb8a7c954bc7c858b2))
- 🐛 MaskToImage ([821a0ef](https://github.com/melMass/comfy_mtb/commit/821a0ef42735a0a97ab82be22a4fdc67c9cfc80e))
### Documentation
- 📚 update wiki ([e17c6e2](https://github.com/melMass/comfy_mtb/commit/e17c6e29f5111bf5085b1fe6f764cfd1aae709f2))
- 📚 remove link ([5bc125d](https://github.com/melMass/comfy_mtb/commit/5bc125d2f08470c8900dfd89deca721835848917))
- 📚 clean readme ([333f646](https://github.com/melMass/comfy_mtb/commit/333f646ab1959d2c944fb046275cc93a545d557c))
### Features
- ✨ add h264 compression node ([e32d1e0](https://github.com/melMass/comfy_mtb/commit/e32d1e02df5e3a9351f829513f7ee3ffb2934be4))
- ✨ add postshot nodes ([27e03fa](https://github.com/melMass/comfy_mtb/commit/27e03fa23efffda461c6975b15fe3964de476cb3))
- ✨ improve the I/O sidebar ([cd9e614](https://github.com/melMass/comfy_mtb/commit/cd9e614b1a385d6b06eacfaad62def1d69f09808)) in [#193](https://github.com/melMass/comfy_mtb/pull/193)
- ✨ add UpscaleBBoxBy ([74af5c6](https://github.com/melMass/comfy_mtb/commit/74af5c6499ef5dd73ce66c4c21b8c3507d69b037))
- ✨ simplified sidebar and backend ([22f7c30](https://github.com/melMass/comfy_mtb/commit/22f7c3037345a866c9ff0b06f6689748021cee63))
- ✨ add Interpolate Condition ([0133fb9](https://github.com/melMass/comfy_mtb/commit/0133fb93bc944d0dd7593b89b36e5b2676d9397a))
- ✨ dump of wip things... ([cf7d305](https://github.com/melMass/comfy_mtb/commit/cf7d30507e7e449c4489e6a1ca159d3d0486bc55))
- ✨ use the new parser for documentations ([4e593bb](https://github.com/melMass/comfy_mtb/commit/4e593bb30be561e39f1790e3514f60bb39e5a261))
- ✨ add @mtb/markdown-parser bundles ([097ca33](https://github.com/melMass/comfy_mtb/commit/097ca33b8e7b27148e183e91712dc34d98d1a69b))
- ✨ add VitMatte nodes ([896a025](https://github.com/melMass/comfy_mtb/commit/896a025006f9c7809c5e0776393a28f908be8950))
- ✨ add ColorCorrectGPU ([9651a70](https://github.com/melMass/comfy_mtb/commit/9651a7034120589b059329b21688708e42772453))
- ✨ add Swap FG/BG colors to MaskToImage ([57683c3](https://github.com/melMass/comfy_mtb/commit/57683c3c7d299a117a26526d52de4c26f2ec0f69))
- ✨ add Extract coordinates ([f99f92e](https://github.com/melMass/comfy_mtb/commit/f99f92e8f7b2d6fac56f7f40049715910e15cfee))
- ✨ add AudioCut ([5681b46](https://github.com/melMass/comfy_mtb/commit/5681b464adce395086712b61159b2694150b8027))
- ✨ add AudioStack ([8d0fcee](https://github.com/melMass/comfy_mtb/commit/8d0fcee2f3decc1cbbf3b850332e6b2a022e1377))
- ✨ add AudioSequence node ([1078fc6](https://github.com/melMass/comfy_mtb/commit/1078fc6f0fb225b52536f25ec6a9fa0456a90595))
- ✨ add Split Bbox node ([9007a70](https://github.com/melMass/comfy_mtb/commit/9007a70aa0d6b2ead0f68f7aff8ae8e3c4f3624f))
- ✨ update lerp example ([1a0ebd5](https://github.com/melMass/comfy_mtb/commit/1a0ebd5173687784f279a9c2184c89fb3be01dc5))
### Miscellaneous Tasks
- 🧹 bump minor ([50cb6f5](https://github.com/melMass/comfy_mtb/commit/50cb6f5ed6e5d9fecb9733ef3f7852b8500005e9))
- 🧹 add worktree to gitignores ([9ccf572](https://github.com/melMass/comfy_mtb/commit/9ccf572a158caeab9bff53853e8f6fb85b76776d))
- 🧹 remove dupe code ([e099d58](https://github.com/melMass/comfy_mtb/commit/e099d581a7627c3a66d2e3e6df3a701b0e5f31b7))
- 🧹 update externs ([784fb01](https://github.com/melMass/comfy_mtb/commit/784fb0145b7421e2730b52237ce6a8b63b189191))
- 🧹 add pathlibed inputs to utils ([a825504](https://github.com/melMass/comfy_mtb/commit/a825504bdd67e3461be8118119e0becc35f8af40))
- 🧹 disable Constant ([22190cd](https://github.com/melMass/comfy_mtb/commit/22190cd25ee590595f8f19e75a9a6c539699622b))
- 🧹 new ui is default, flag for old ui ([a976adb](https://github.com/melMass/comfy_mtb/commit/a976adbb39a13b4cd76f224ebba40c604900c862))
- 🧹 add methods to shared ([f8829fc](https://github.com/melMass/comfy_mtb/commit/f8829fcb373e0f9bc4f0ad36c939f372349943bf))
- 🧹 add an old_ui flag to my launcher ([dbdf276](https://github.com/melMass/comfy_mtb/commit/dbdf27664cd207dbbc69b8d635adcd59ed8d269a))
- 🧹 move qrcode to his own file ([7d5569e](https://github.com/melMass/comfy_mtb/commit/7d5569e5c1e0f0b6ccb505a02f74640139d6aaf9))
## [0.1.6] - 2024-07-03
### Bug Fixes
- 🐛 menu callback issue ([d64fac4](https://github.com/melMass/comfy_mtb/commit/d64fac4b74e0590acde5e3b8edd4a2f715448cf5))
### Documentation
- 📚 Update requirements file in INSTALL.md ([f25f6bd](https://github.com/melMass/comfy_mtb/commit/f25f6bdcd13d50f9d383065321320b0ce6a03214)) by [@elthariel](https://github.com/elthariel) in [#186](https://github.com/melMass/comfy_mtb/pull/186)
### Features
- ✨ add alpha channel support for faceswap/restore ([d6343e1](https://github.com/melMass/comfy_mtb/commit/d6343e1860f46947e93758f8bba03857c9326b38))
### Miscellaneous Tasks
- 🧹 better classname extraction ([d687497](https://github.com/melMass/comfy_mtb/commit/d687497d8041ab5d77bd31909592def6e4d0e7f6))
- 🤖 limit release only to tags ([4eebdd8](https://github.com/melMass/comfy_mtb/commit/4eebdd8b8bff73c3db4f0248da8dac7d67cb310b))
- 🧹 runner ([fb34671](https://github.com/melMass/comfy_mtb/commit/fb34671ee6fe80b965fe576c279ed1ff77a358f2))
- 🤖 only publish on tag ([f1b4846](https://github.com/melMass/comfy_mtb/commit/f1b484617a917d38d9b3658d8920aa7dec672a79))
- 🧹 small fixes ([4507842](https://github.com/melMass/comfy_mtb/commit/4507842a706141977a6a68945c36e977c358d91a))
## New Contributors
* [@elthariel](https://github.com/elthariel) made their first contribution in [#186](https://github.com/melMass/comfy_mtb/pull/186)
## [0.1.5] - 2024-06-21
### Bug Fixes
- 🐛 keep the last model match instead of first ([1edc2cd](https://github.com/melMass/comfy_mtb/commit/1edc2cd10de81297e7a895009d358813e79b70ba))
- 🐛 properly initialize the curve value ([35622e3](https://github.com/melMass/comfy_mtb/commit/35622e3a5e58103a8f5b150556b85e97e31555e1))
- 🐛 ImageCompare improvements ([acc2d68](https://github.com/melMass/comfy_mtb/commit/acc2d687d596bf82c2075f9a24003eacf18adfe7)) by [@christian-byrne](https://github.com/christian-byrne) in [#176](https://github.com/melMass/comfy_mtb/pull/176)
- 🐛 repetitive warning ([780c52f](https://github.com/melMass/comfy_mtb/commit/780c52f03aca3079a1b695510341486720004bec)) by [@vxkj1211](https://github.com/vxkj1211) in [#177](https://github.com/melMass/comfy_mtb/pull/177)
- 🐛 add back was conversion node ([349a852](https://github.com/melMass/comfy_mtb/commit/349a8524c6f7fcab4a124cacb60bfbef1463cf1b))
- 🐛 drag lag on documentation resize handle ([15330ea](https://github.com/melMass/comfy_mtb/commit/15330eab655f66214d3c25fd237679f090175c32))
- 🐛 kwarg typo ([1571782](https://github.com/melMass/comfy_mtb/commit/1571782d012b83bce32a065e700f9a587db234d2))
- 🐛 seed of PlotBatchFloat ([5b40302](https://github.com/melMass/comfy_mtb/commit/5b4030288d43c79859c9706a12aa0f8b7dea190f))
- 🐛 forceInput for FLOAT <-> FLOATS converters ([5a0ef0d](https://github.com/melMass/comfy_mtb/commit/5a0ef0dadd01fd5937ed0715d829d6a456f96318))
- 🐛 FLOAT always need options to be set ([967e72f](https://github.com/melMass/comfy_mtb/commit/967e72fc66780685f8192cb8fe13ba66b9326f63))
- 🐛 remove doc if opened on node delete ([bee3f47](https://github.com/melMass/comfy_mtb/commit/bee3f47a14ddb92b3760098666bf75dc7d37f1e4))
- 🐛 for documentation on HiDPI ([b11346a](https://github.com/melMass/comfy_mtb/commit/b11346aba88d9f1dac3b6b42c691979cc0978b6f))
- 🐛 never remove input 0 of dynamic inputs ([30982fa](https://github.com/melMass/comfy_mtb/commit/30982fa48829c3fc2a6745ce5a07537a3d94b2f9))
- 🐛 use the same fix as dynamicInputs for debug ([92b7990](https://github.com/melMass/comfy_mtb/commit/92b79906cd2ee1b4ca3ff25378d7786b5a47cb75))
- 🐛 missing numberInput ([76f365b](https://github.com/melMass/comfy_mtb/commit/76f365b5eee165c76f3da7d2e3950786685bc08b))
- 🐛 better curve ([da67e76](https://github.com/melMass/comfy_mtb/commit/da67e766c2f700dd9e2f51a5bafe07c612904f5d))
- 🐛 prepend MTB_ to all classes ([b1d74ad](https://github.com/melMass/comfy_mtb/commit/b1d74adb15166e3e5eb9cf92d6148e4644bed346))
- 🐛 dynamic connections ([652ac3f](https://github.com/melMass/comfy_mtb/commit/652ac3f3b971582b02115177fd6f7a9d3d7295df))
- 🐛 remaining issue before merge ([100067a](https://github.com/melMass/comfy_mtb/commit/100067a645194366426f29b085bf25d0623f4fac))
- 🐛 debug issues ([7807449](https://github.com/melMass/comfy_mtb/commit/7807449e6dcc01cfdb7f0eb818569184c8b41af2))
- 🐛 errors when insightface's folder missing ([e838c04](https://github.com/melMass/comfy_mtb/commit/e838c04758402250fd3464d6cd6a6f872e8cef29))
- 🐛 typo ([e40ad7a](https://github.com/melMass/comfy_mtb/commit/e40ad7a574f961ebe1f338b97214da5cbadcc529))
- 🐛 better defaults (cont) ([1da483a](https://github.com/melMass/comfy_mtb/commit/1da483a8baa6a893f1adb05ef79b90c4412c3834))
- 🐛 better defaults for Autopan ([5eff38b](https://github.com/melMass/comfy_mtb/commit/5eff38b387d22206d39c08e435806f9d03992feb))
- 🐛 dynamic inputs ([9ab20a0](https://github.com/melMass/comfy_mtb/commit/9ab20a0ab50b1656ded9a84c13769fd2d547f2d2))
- 🐛 bundle ace editor ([7c35582](https://github.com/melMass/comfy_mtb/commit/7c3558273bebc0754c802720e705232f220a0da4))
- 🐛 image to mask ([f16d576](https://github.com/melMass/comfy_mtb/commit/f16d576f6f0e83fc2fafd2d1f29b2edeb00d3197))
- 🐛 prepend MTB to classnames ([e56508c](https://github.com/melMass/comfy_mtb/commit/e56508c2078155f053e7f11d538a048df6a5b18b))
- 🐛 allow smaller values in BatchTransform ([9a4b27d](https://github.com/melMass/comfy_mtb/commit/9a4b27d2e05e8ebe31f58a21db94bd3a54ed23d9))
- 🐛 add category for virtual note+ ([eeac8c0](https://github.com/melMass/comfy_mtb/commit/eeac8c002ad1f9e461418fb66b9338e969259e58))
- 🐛 make image feed of by default ([df0a98b](https://github.com/melMass/comfy_mtb/commit/df0a98b94a4a9388811bc8786e820ec892919c1a))
- 🐛 support batch masks (colored image node) ([2465ffb](https://github.com/melMass/comfy_mtb/commit/2465ffb0d3b052fb78559394dbb550bba59b97a3))
- 🐛 support pillow < 10 ([48f91b7](https://github.com/melMass/comfy_mtb/commit/48f91b74e2c7ef6d31c094eafa5332784a275a8b))
- 🐛 image rotation bug ([54ff658](https://github.com/melMass/comfy_mtb/commit/54ff6583ded0ed4054f8e5d7fadf0b2350259dce)) by [@hongminpark](https://github.com/hongminpark) in [#154](https://github.com/melMass/comfy_mtb/pull/154)
- 🐛 font fallback ([9fccdee](https://github.com/melMass/comfy_mtb/commit/9fccdee82d721e88c64d2292c209fec869524dd2))
- ✨ optional inputs of colored image ([cd32f26](https://github.com/melMass/comfy_mtb/commit/cd32f26b167088d6b489e43b260c187ea5e4d223)) by [@ScottNealon](https://github.com/ScottNealon) in [#147](https://github.com/melMass/comfy_mtb/pull/147)
- 📝 adds a way to not load the imagefeed ([501c330](https://github.com/melMass/comfy_mtb/commit/501c3301056b2851555cccd75ab3ff15b1ab8e0c))
@@ -230,13 +47,6 @@ Check the notes in the [releases](https://github.com/melMass/comfy_mtb/releases)
### Documentation
- 📚 update the wiki ([fa3199b](https://github.com/melMass/comfy_mtb/commit/fa3199be2b87bf3cb7484a0fee32a8ac099adc65))
- 📚 update wiki submodule ([49cea8d](https://github.com/melMass/comfy_mtb/commit/49cea8d94508b27781506e3b5509c65e1d84e80f))
- 📚 add the wiki as a submodule ([5998924](https://github.com/melMass/comfy_mtb/commit/59989249260a9c579ec851c50534b58f3f02cd61))
- 📚 missing doc ([c9836a8](https://github.com/melMass/comfy_mtb/commit/c9836a87f6823db1d53e56997417f3cbe8cc4727))
- 📚 use flat icon ([991af4f](https://github.com/melMass/comfy_mtb/commit/991af4f45ff8c660b2c45466bb219186699170ed))
- 📚 add banodoco channel link ([9ce34b4](https://github.com/melMass/comfy_mtb/commit/9ce34b47fd99b18db7997ccce44e6063f00b6801))
- 📚 udpate changelog ([8221c49](https://github.com/melMass/comfy_mtb/commit/8221c49942bd87c14d5063066315a449a1fee86e))
- 📝 add changelog ([0d817bf](https://github.com/melMass/comfy_mtb/commit/0d817bf326b4a22e2221264a414af50c3b7048b9))
- 📄 add note+ screenshot ([90d9636](https://github.com/melMass/comfy_mtb/commit/90d96366c8b7637b55d1b4f88cb9aca217c1414b))
- 📝 add cover image ([6b993b8](https://github.com/melMass/comfy_mtb/commit/6b993b84071bbb80ba1b8bd63576f31e35d05590))
@@ -250,28 +60,6 @@ Check the notes in the [releases](https://github.com/melMass/comfy_mtb/releases)
### Features
- ✨ add ModelPruner (wip) ([43d65ae](https://github.com/melMass/comfy_mtb/commit/43d65ae68c97e077117b17b7c9d1936583f965eb))
- ✨ Use dynamic contrast in Color Correct ([6abac2e](https://github.com/melMass/comfy_mtb/commit/6abac2e4706a3d937420213e01468bae10cc2017)) by [@christian-byrne](https://github.com/christian-byrne) in [#180](https://github.com/melMass/comfy_mtb/pull/180)
- ✨ StackImages add support for batch mismatch ([5060c56](https://github.com/melMass/comfy_mtb/commit/5060c561353e43624ec164cb73fce7d1d422f765))
- ✨ add BatchFloatMath ([f9d2ebf](https://github.com/melMass/comfy_mtb/commit/f9d2ebf91d09fc214fecf7501a5490b33c30aca2))
- ✨ add FLOATS to INTS ([1b7ae27](https://github.com/melMass/comfy_mtb/commit/1b7ae27cc1907bfba3c5166ec2c61547babd2e0a))
- ✨ debug dict ([63ee25d](https://github.com/melMass/comfy_mtb/commit/63ee25d001d4c94aa95dc8b39008f5d943f2ab45))
- ✨ add Swap BG/FG color menu item ([1caf7c1](https://github.com/melMass/comfy_mtb/commit/1caf7c18c372651b2be7227eb77e2251d963693d))
- ✨ BatchFloatFit the batch version of FitNumber ([ab58c36](https://github.com/melMass/comfy_mtb/commit/ab58c362124f0f4b3178534ca78cb924fb881534))
- ✨ add FloatToFloats (the counterpart) ([78a86da](https://github.com/melMass/comfy_mtb/commit/78a86daaf71dab5be34b90b13491460854718485))
- ✨ add some FLOATS batch nodes ([2159395](https://github.com/melMass/comfy_mtb/commit/2159395389429c5f7012e660b41fad48d376b39f))
- ✨ poc of the doc widget idea ([fac7529](https://github.com/melMass/comfy_mtb/commit/fac7529d1f7b6fc4b3b2e7f6022ebb23ec71169d))
- ✨ add the backend node for Constant ([dff5b22](https://github.com/melMass/comfy_mtb/commit/dff5b2201d73c1a91d4b5864e3b974e68846a011))
- ✨ add Constant node ([cbb5dd2](https://github.com/melMass/comfy_mtb/commit/cbb5dd2cf810d5648a64eae370dba610336b99d5))
- ✨ add FloatsToFloat ([6ebecfd](https://github.com/melMass/comfy_mtb/commit/6ebecfd8cf1dc3779384e565a65baa9dceb43660))
- ✨ add AutoPanEquilateral ([3513937](https://github.com/melMass/comfy_mtb/commit/35139371e84d715423015e05d1b4a6c1d88b0eb5))
- ✨ add MatchDimensions ([5db3ebe](https://github.com/melMass/comfy_mtb/commit/5db3ebedb9d38470c82544e45970775193add05c))
- ✨ add equilateral example ([8d65556](https://github.com/melMass/comfy_mtb/commit/8d65556c37f33d1c496504db92574805916dd613))
- ✨ enhance tiling tools ([ba73fc6](https://github.com/melMass/comfy_mtb/commit/ba73fc6af7039a4629a73cdc36a8c8736dc27c9d))
- ✨ add FLOATS support to blur ([92c810c](https://github.com/melMass/comfy_mtb/commit/92c810c5036f7a2b3f84a3fde8c81e6a2b046b07))
- ✨ add "tube" to Batch Shape ([f658fc3](https://github.com/melMass/comfy_mtb/commit/f658fc31e040141209384d98dfe84b766fe4ae11))
- ✨ note+ editor themes ([133da70](https://github.com/melMass/comfy_mtb/commit/133da705c94af2dfb3d2f38c0d9c2723c72cacf7))
- ✨ add ffmpeg gif export ([1b29aad](https://github.com/melMass/comfy_mtb/commit/1b29aad360116e631b7b4d34e98a5a631f134977)) by [@huanggou666](https://github.com/huanggou666) in [#159](https://github.com/melMass/comfy_mtb/pull/159)
- ✨ add "To Device" ([c28181f](https://github.com/melMass/comfy_mtb/commit/c28181f1615d2e183767aa76cc2350934330e546))
- ✨ add note+ example ([90f3bc2](https://github.com/melMass/comfy_mtb/commit/90f3bc2d953b299ea34e9e3a925f1a824b488855))
- 💄 node+ improvements ([4b29395](https://github.com/melMass/comfy_mtb/commit/4b29395000254382882c0d1be115b2ed80cd7c99))
@@ -294,19 +82,6 @@ Check the notes in the [releases](https://github.com/melMass/comfy_mtb/releases)
### Miscellaneous Tasks
- 🧹 add fields for the registry ([bb5682a](https://github.com/melMass/comfy_mtb/commit/bb5682aa6da923859db33830c2e46f24b19199a1))
- 🧹 add pre-commit ([59612fd](https://github.com/melMass/comfy_mtb/commit/59612fd8110a888f0081433242a2b5a5f7e46da6))
- 🧹 migrate from poetry to setuptools ([dfd17f6](https://github.com/melMass/comfy_mtb/commit/dfd17f6d783e784df7dab38d185c747b4c04d1d0))
- 🧹 remove logs ([1070edd](https://github.com/melMass/comfy_mtb/commit/1070edd0245fb235183d5f38cd1bebf6e0405f97))
- 🧹 add more pyproject meta ([644371e](https://github.com/melMass/comfy_mtb/commit/644371e5b5a2b8260fc5c6f699465b0bc1c81d57))
- 🤖 move at the proper location ([f3d468c](https://github.com/melMass/comfy_mtb/commit/f3d468cfc238f13905a13a7b2225e3711129c64d))
- 🤖 add CI to publish to ComfyUI Registry ([6cd448b](https://github.com/melMass/comfy_mtb/commit/6cd448b026956cdf3f1b81e93724b295316fbf09)) by [@haohaocreates](https://github.com/haohaocreates) in [#182](https://github.com/melMass/comfy_mtb/pull/182)
- 🧹 add ComfyUI registry to pyproject.toml ([5951c90](https://github.com/melMass/comfy_mtb/commit/5951c90b10f9b77b2b617e83efe0112f43c8daef)) by [@haohaocreates](https://github.com/haohaocreates) in [#181](https://github.com/melMass/comfy_mtb/pull/181)
- 🧹 update types ([96a0da9](https://github.com/melMass/comfy_mtb/commit/96a0da9dbd051d1fcf8b332c54ed2d307d8ae0dd))
- 🧹 use a gettattr fallback ([a344cdc](https://github.com/melMass/comfy_mtb/commit/a344cdcba9823ca1fb0762795068039b1e1cf0ab))
- 🧹 cleanup js ([64cc4e9](https://github.com/melMass/comfy_mtb/commit/64cc4e9649853023d645245bea1e1ceb11073f01))
- 🧹 add savedatabundle js part ([edd7c3f](https://github.com/melMass/comfy_mtb/commit/edd7c3f5d075b640e9cdb067ebfe51c42ff61791))
- 🧹 wip dynamic multitype ([71bfdd6](https://github.com/melMass/comfy_mtb/commit/71bfdd61d731ce15f9bd0bb19d65b5af208d5dcf))
- 🧹 applied some linting ([fe49312](https://github.com/melMass/comfy_mtb/commit/fe49312cbef03c6540304448fa88aa7a88391efa))
- 📝 header links not parsed ([514c0d2](https://github.com/melMass/comfy_mtb/commit/514c0d2eda9990435eb18258d4bbd1aa137feb3d))
- 📝 hardcode links in changelog ([915b744](https://github.com/melMass/comfy_mtb/commit/915b7444a9db83f349d83b636304af0d276f529f))
@@ -329,17 +104,9 @@ Check the notes in the [releases](https://github.com/melMass/comfy_mtb/releases)
### Wip
- 🚧 curve widget logic fixed ([e312b02](https://github.com/melMass/comfy_mtb/commit/e312b02ad2f8334e87654a20b0114837df229371))
- 🚧 dump3 ([eedbb4b](https://github.com/melMass/comfy_mtb/commit/eedbb4bc6581bef85c746307fe9d53360ea45bcf))
- 🚧 dump ([fa23975](https://github.com/melMass/comfy_mtb/commit/fa2397585fff4f54bcf17f0b0e0083c427b34fa8))
- 🚧 dump ([0d0fb8e](https://github.com/melMass/comfy_mtb/commit/0d0fb8e13a5da54a44a96a04607f7a349f8fdb03))
- 🚧 add text template node ([af2175a](https://github.com/melMass/comfy_mtb/commit/af2175a1fc0c2fb29ef3493f242fe45ec6fcabac))
## New Contributors
* [@haohaocreates](https://github.com/haohaocreates) made their first contribution in [#182](https://github.com/melMass/comfy_mtb/pull/182)
* [@vxkj1211](https://github.com/vxkj1211) made their first contribution in [#177](https://github.com/melMass/comfy_mtb/pull/177)
* [@huanggou666](https://github.com/huanggou666) made their first contribution in [#159](https://github.com/melMass/comfy_mtb/pull/159)
* [@hongminpark](https://github.com/hongminpark) made their first contribution in [#154](https://github.com/melMass/comfy_mtb/pull/154)
* [@ScottNealon](https://github.com/ScottNealon) made their first contribution in [#147](https://github.com/melMass/comfy_mtb/pull/147)
* [@Yurchikian](https://github.com/Yurchikian) made their first contribution in [#124](https://github.com/melMass/comfy_mtb/pull/124)
* [@M1kep](https://github.com/M1kep) made their first contribution in [#91](https://github.com/melMass/comfy_mtb/pull/91)
@@ -626,10 +393,7 @@ Check the notes in the [releases](https://github.com/melMass/comfy_mtb/releases)
- 🚀 add gh action ([572b4d5](https://github.com/melMass/comfy_mtb/commit/572b4d52bce1398660d4d7ca0c5c48c11e0128e3)) in [#4](https://github.com/melMass/comfy_mtb/pull/4)
[main]: https://github.com/melMass/comfy_mtb/compare/v0.2.0..main
[0.2.0]: https://github.com/melMass/comfy_mtb/compare/v0.1.6..v0.2.0
[0.1.6]: https://github.com/melMass/comfy_mtb/compare/v0.1.5..v0.1.6
[0.1.5]: https://github.com/melMass/comfy_mtb/compare/v0.1.4..v0.1.5
[main]: https://github.com/melMass/comfy_mtb/compare/v0.1.4..main
[0.1.4]: https://github.com/melMass/comfy_mtb/compare/v0.1.3..v0.1.4
[0.1.3]: https://github.com/melMass/comfy_mtb/compare/v0.1.2..v0.1.3
[0.1.2]: https://github.com/melMass/comfy_mtb/compare/v0.1.1..v0.1.2
-52
View File
@@ -1,52 +0,0 @@
# Code of Conduct
## Our Commitment
We are committed to creating a welcoming and inclusive community for everyone. We believe that a diverse and respectful community is essential for fostering creativity and innovation. We expect all members of our community to adhere to this Code of Conduct.
## Our Expectations
This Code of Conduct applies to all interactions within the mtb community, including:
* Public communication channels (e.g., GitHub issues, pull requests, discussions, social media)
* Private communication channels (e.g., direct messages, email)
* In-person events (if any)
We expect all members to:
* **Be respectful and considerate:** Treat others with kindness and empathy.
* **Be inclusive:** Welcome and respect people of all backgrounds, identities, and experiences.
* **Be constructive:** Focus on providing helpful and positive feedback.
* **Be mindful of your language:** Avoid using offensive, discriminatory, or harassing language.
* **Respect privacy:** Do not share personal information without consent.
## Unacceptable Behavior
The following behaviors are not tolerated:
* Offensive, discriminatory, or harassing language or conduct
* Personal attacks or insults
* Spamming or trolling
* Sharing of malicious or inappropriate content
* Disrupting the community or hindering collaboration
* Violating the privacy of others
## Reporting Violations
If you experience or witness a violation of this Code of Conduct, please report it to @melmass. All reports will be treated confidentially and investigated promptly.
## Enforcement
Violations of this Code of Conduct may result in the following actions:
* Warning
* Removal from the community
* Ban from the community
## License
[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](code_of_conduct.md)
## Contact
If you have any questions or concerns about this Code of Conduct, please contact @melmass.
-62
View File
@@ -1,62 +0,0 @@
# Contributing to mtb
Thank you for your interest in contributing to mtb! We appreciate your help in making this project better. This document outlines how you can contribute to the project.
## Project Overview
This project is a collection of custom nodes for ComfyUI, tailored specifically for animation workflows. It aims to provide a streamlined and user-friendly experience for creating animations within the ComfyUI environment.
## Ways to Contribute
We welcome all kinds of contributions! Here's how you can get involved:
* **Bug Reports:** If you encounter any issues, please create a new issue on GitHub. Please include clear steps to reproduce the bug, along with any relevant error messages, workflows or screenshots.
* **Feature Requests:** Have an idea for a new node or feature? Create a new issue to discuss it! Please describe the feature in detail, and explain how it would benefit the project.
* **Documentation Improvements:** Help us improve the documentation by fixing errors, adding examples, or clarifying explanations.
* **Code Contributions:** We welcome contributions to the codebase! Please see the "Development Setup" and "File Structure" sections below for more information.
* **Testing:** Help us ensure the stability and reliability of the project by testing new features and bug fixes.
* **Refactoring:** Help us improve the codebase by refactoring existing code to improve readability, maintainability, and performance.
## Development Setup
```sh
git clone --recursive https://github.com/melmass/comfy_mtb
```
## File Structure
Understanding the project structure is crucial for making effective contributions.
* **`./nodes/*.py`:** This directory contains the definitions for all custom nodes. Nodes are automatically registered when a file defines an array named `__nodes__` containing the node classes. Make sure your node follows the ComfyUI node definition structure.
* **`./web/*.js`:** This directory contains all the frontend JavaScript code for the extension's user interface.
* **`./wiki`:** This directory is a Git submodule that contains the project's Wiki documentation, written in Markdown. Node documentation should be created or updated in the corresponding Markdown files within this submodule. This is then referenced by the UI for in-GUI help
## Coding Style
We use **Ruff** for code formatting to ensure consistency. Please run Ruff on your code before submitting a pull request. No specific configuration is required, so the default Ruff settings will be used.
## Contribution Workflow
1. **Create a Branch:** Create a new branch for your feature or fix. Use a descriptive branch name (e.g., `feature/new-node`, `fix/bug-in-ui`). **Do not fork the main branch directly.**
2. **Make Changes:** Implement your changes in your branch.
3. **Run Tests:** (Add instructions on how to run tests if available.)
4. **Format Code:** Run Ruff on your code to ensure it is properly formatted.
5. **Create a Pull Request:** Submit a pull request to the `main` branch. Please provide a clear and concise description of your changes.
## Code of Conduct
We are committed to creating a welcoming and inclusive community. We expect all contributors to adhere to a respectful and professional code of conduct. (Consider adding a link to a CODE_OF_CONDUCT.md file or a standard code of conduct.)
## Tools and Libraries
* **Python:** The primary programming language for this project.
* **ComfyUI:** The underlying framework for the custom nodes.
## Current Focus
We are currently focused on a major refactor to clean up the project's codebase. Contributions related to this effort are particularly welcome!
## Thank You!
Thank you for considering contributing to mtb! Your contributions are greatly appreciated. We look forward to reviewing your pull requests!
+49 -190
View File
@@ -7,11 +7,9 @@
#
###
__version__ = "0.3.0"
__version__ = "0.2.1"
import os
from collections import OrderedDict
from typing import Any
from aiohttp.web_request import Request
@@ -33,16 +31,7 @@ from importlib import reload
from pathlib import Path
from aiohttp import web
IN_COMFY = False
try:
from server import PromptServer
IN_COMFY = True
except ModuleNotFoundError:
IN_COMFY = False
from server import PromptServer
from .endpoint import endlog
from .install import get_node_dependencies
@@ -242,14 +231,14 @@ if failed:
# - ENDPOINT
if IN_COMFY and hasattr(PromptServer, "instance"):
if hasattr(PromptServer, "instance"):
img_cache = None
prompt_cache = None
with contextlib.suppress(ImportError):
from cachetools import TTLCache
# img_cache = TTLCache(maxsize=100, ttl=5) # 1 min TTL
img_cache = TTLCache(maxsize=100, ttl=5) # 1 min TTL
prompt_cache = TTLCache(maxsize=100, ttl=5) # 1 min TTL
node_dependency_mapping = get_node_dependencies()
@@ -364,132 +353,29 @@ if IN_COMFY and hasattr(PromptServer, "instance"):
import asyncio
import os
import time
from asyncio import Semaphore
from concurrent.futures import ThreadPoolExecutor
from contextlib import asynccontextmanager
from io import BytesIO
from aiohttp import web
from PIL import Image
image_thread_pool = ThreadPoolExecutor(
max_workers=4, thread_name_prefix="img_worker"
)
@asynccontextmanager
async def get_image_with_timeout(
file_path, preview_params=None, channel=None, timeout=10
):
try:
result = await asyncio.wait_for(
asyncio.get_event_loop().run_in_executor(
image_thread_pool,
get_cached_image,
file_path,
preview_params,
channel,
),
timeout=timeout,
)
yield result
except asyncio.TimeoutError:
print(f"Image processing timed out for {file_path}")
raise
except Exception as e:
print(f"Error processing image {file_path}: {str(e)}")
raise
async def get_image_response(
file, filename: str, preview_info=None, channel=None
):
try:
async with get_image_with_timeout(
file, preview_info, channel
) as img:
return web.Response(
body=img,
content_type="image/webp" if preview_info else "image/png",
headers={"Content-Disposition": f'filename="{filename}"'},
)
except asyncio.TimeoutError:
return web.Response(status=504, text="Image processing timed out")
except Exception as e:
return web.Response(status=500, text=str(e))
class LRUCache:
def __init__(self, capacity: int):
self.cache = OrderedDict()
self.capacity = capacity
def get(self, key) -> Any:
if key not in self.cache:
return None
self.cache.move_to_end(key)
return self.cache[key]
def put(self, key, value: Any) -> None:
if key in self.cache:
self.cache.move_to_end(key)
self.cache[key] = value
if len(self.cache) > self.capacity:
self.cache.popitem(last=False)
img_cache = LRUCache(capacity=100)
def get_cached_image(file_path: str, preview_params=None, channel=None):
cache_key = (file_path, preview_params, channel)
try:
if img_cache and (cache_key in img_cache):
return img_cache[cache_key]
with Image.open(file_path) as img:
info = img.info
if preview_params:
img = process_preview(img, preview_params)
if channel:
img = process_channel(img, channel)
if prompt_cache:
prompt_cache[cache_key] = info
if img_cache:
cached_value = img_cache.get(cache_key)
if cached_value is not None:
return cached_value
img_cache[cache_key] = img.getvalue()
return img_cache[cache_key]
with Image.open(file_path) as img:
info = img.info
if preview_params:
img = process_preview(img, preview_params)
if channel:
img = process_channel(img, channel)
result = img.getvalue()
try:
if prompt_cache:
prompt_cache[cache_key] = info
if img_cache:
img_cache.put(cache_key, result)
except Exception as e:
print(
f"Warning: Failed to cache image {file_path}: {str(e)}"
)
return result
except Exception as e:
print(f"Error processing image {file_path}: {str(e)}")
raise
class RateLimiter:
def __init__(self, requests_per_second):
self.requests_per_second = requests_per_second
self.semaphore = Semaphore(requests_per_second)
self.timestamps = []
async def acquire(self):
await self.semaphore.acquire()
now = time.time()
self.timestamps.append(now)
# Remove old timestamps
self.timestamps = [t for t in self.timestamps if now - t < 1.0]
if len(self.timestamps) >= self.requests_per_second:
await asyncio.sleep(1.0)
def release(self):
self.semaphore.release()
rate_limiter = RateLimiter(requests_per_second=10)
return img.getvalue()
def process_preview(img: Image.Image, preview_params):
image_format, quality, width = preview_params
@@ -542,64 +428,41 @@ if IN_COMFY and hasattr(PromptServer, "instance"):
# to load workflows in the sidebar
@PromptServer.instance.routes.get("/mtb/view")
async def view_image(request: Request):
try:
import folder_paths
import folder_paths
await rate_limiter.acquire()
filename = request.rel_url.query.get("filename")
if not filename:
return web.Response(status=404)
filename = request.rel_url.query.get("filename")
if not filename:
return web.Response(status=404)
filename, output_dir = folder_paths.annotated_filepath(filename)
if filename[0] == "/" or ".." in filename:
return web.Response(status=400)
filename, output_dir = folder_paths.annotated_filepath(filename)
if filename[0] == "/" or ".." in filename:
return web.Response(status=400)
if output_dir is None:
rtype = request.rel_url.query.get("type", "output")
output_dir = folder_paths.get_directory_by_type(rtype)
if output_dir is None:
rtype = request.rel_url.query.get("type", "output")
output_dir = folder_paths.get_directory_by_type(rtype)
if output_dir is None:
return web.Response(status=400)
if output_dir is None:
return web.Response(status=400)
if "subfolder" in request.rel_url.query:
full_output_dir = os.path.join(
output_dir, request.rel_url.query["subfolder"]
)
if (
os.path.commonpath(
(os.path.abspath(full_output_dir), output_dir)
)
!= output_dir
):
return web.Response(status=403)
output_dir = full_output_dir
filename = os.path.basename(filename)
file = os.path.join(output_dir, filename)
if not os.path.isfile(file):
return web.Response(status=404)
ret_workflow = request.rel_url.query.get("workflow")
if ret_workflow:
image = Image.open(file)
prompt = image.info.get("prompt", "")
workflow = image.info.get("workflow", "")
if workflow:
workflow = json.loads(workflow)
if prompt:
prompt = json.loads(prompt)
return web.json_response(
{
"prompt": prompt,
"workflow": workflow,
}
if "subfolder" in request.rel_url.query:
full_output_dir = os.path.join(
output_dir, request.rel_url.query["subfolder"]
)
if (
os.path.commonpath(
(os.path.abspath(full_output_dir), output_dir)
)
!= output_dir
):
return web.Response(status=403)
output_dir = full_output_dir
filename = os.path.basename(filename)
file = os.path.join(output_dir, filename)
if not os.path.isfile(file):
return web.Response(status=404)
preview_info = None
if "preview" in request.rel_url.query:
@@ -617,13 +480,9 @@ if IN_COMFY and hasattr(PromptServer, "instance"):
width = request.rel_url.query.get("width")
preview_info = (image_format, quality, width)
channel = request.rel_url.query.get("channel")
channel = request.rel_url.query.get("channel")
return await get_image_response(
file, filename, preview_info, channel
)
finally:
rate_limiter.release()
return await get_image_response(file, filename, preview_info, channel)
@PromptServer.instance.routes.get("/mtb/server-info")
async def get_debug(request: Request):
+2 -7
View File
@@ -74,12 +74,7 @@ def ACTIONS_getUserImageFolders():
input_subdirs = [x.name for x in input_dir.iterdir() if x.is_dir()]
output_subdirs = [x.name for x in output_dir.iterdir() if x.is_dir()]
return {
"input_root": input_dir.as_posix(),
"input": input_subdirs,
"output": output_subdirs,
"output_root": output_dir.as_posix(),
}
return {"input": input_subdirs, "output": output_subdirs}
def ACTIONS_getUserVideos(
@@ -408,7 +403,7 @@ def render_table(table_dict: dict[str, Any], sort=True, title=None):
if "dependencies" in item:
table_rows += f"<tr><td>{name}</td><td>"
table_rows += (
f"{dependencies_button(name, item['dependencies'])}"
f"{dependencies_button(name,item['dependencies'])}"
)
table_rows += "</td></tr>"
+55 -120
View File
@@ -1,7 +1,5 @@
# NOTE: This file is only use for development you can ignore it
use private/log.nu
def get_root [--clean] {
if $clean {
$env.COMFY_CLEAN_ROOT
@@ -23,56 +21,12 @@ export def "comfy dev-web" [] {
npm run dev
}
export def "daily run" [] {
let res = (comfy update --rebase)
comfy update --clean
comfy update_extensions
daily commit $res.from $res.to
}
def short-date [] {
format date "%Y-%m-%d"
}
# was daily run today?
export def "daily was-run" [] {
let daily = ($env.COMFY_MTB | path join daily.nuon)
if ($daily | path exists) {
let last = (open $daily | sort-by date | get date | last | short-date)
let today = (date now | short-date)
return ($last == $today)
}
return false
}
export def "daily commit" [from:string, to:string] {
let daily = ($env.COMFY_MTB | path join daily.nuon)
let commit = [{date: (date now) from:$from to:$to}]
let dailies = (if ($daily | path exists) {
open $daily | append $commit
} else {
$commit
})
$dailies | save -f $daily
log success "Commited daily check"
}
# start the comfy server
export def "comfy start" [--clean,--old-ui, --listen, --skip-daily(-s)] {
if (not (daily was-run)) and not $skip_daily {
log info "Running daily checks"
daily run
}
export def "comfy start" [--clean,--old-ui, --listen] {
let root = get_root --clean=($clean)
cd $root
log info "Running Server"
MTB_DEBUG=true python main.py --port 3000 ...(if $old_ui { ["--front-end-version", "Comfy-Org/ComfyUI_legacy_frontend@latest"]} else {[ --front-end-version Comfy-Org/ComfyUI_frontend@latest]}) --preview-method auto ...(if $listen {["--listen"]} else {[]})
}
@@ -81,93 +35,75 @@ export def "comfy update" [
--clean # ??
--rebase # Rebase instead of merge
] {
let root = get_root --clean=$clean
let root = get_root --clean=($clean)
let models = $"($root)/models"
let inputs = $"($root)/input"
cd $root
let branch_name = (git rev-parse --abbrev-ref HEAD | str trim)
print $"(ansi yellow_italic)Backing up and removing models symlinks(ansi reset)"
let models = $"($root)/models"
let inputs = $"($root)/input"
cd $root
let branch_name = (git rev-parse --abbrev-ref HEAD | str trim)
let current_commit = (git rev-parse HEAD | str trim)
log info "Backing up and removing models symlinks"
# preparing root for pull
if not $clean {
git checkout pyproject.toml
cd $models
# find and store all symlinks
let links = (ls -la |
where not ($it.target | is-empty) |
select name target |
sort-by name)
if not $clean {
cd $models
# find all symlinks
let links = (ls -la |
where not ($it.target | is-empty) |
select name target |
sort-by name)
if not ($links | is-empty) {
$links | save -f links.nuon
# remove them
open links.nuon | each {|p| rm $p.name }
if not ($links | is-empty) {
$links | save -f links.nuon
# remove them
open links.nuon | each {|p| rm $p.name }
}
} else {
rm $models
rm $inputs
}
} else {
# just remove symlinks
rm $models
rm $inputs
}
cd $root
cd $root
log info $"Checking out to master"
git checkout master
print $"(ansi yellow_italic)Checking out to master(ansi reset)"
git checkout master
log info "Fetching and pulling remote updates"
if ($clean) {
# from the local base repo master
git fetch local master # $branch_name # master
git pull local master # $branch_name # master
} else {
git fetch
git pull
}
print $"(ansi yellow_italic)Fetching and pulling remote updates(ansi reset)"
if ($clean) {
git fetch local master
git pull local master
} else {
git fetch
git pull
}
let new_commit = (git rev-parse HEAD | str trim)
log info $"Back to our branch \(($branch_name)\)"
git checkout -
print $"(ansi yellow_italic)Back to our branch \(($branch_name)\)(ansi reset)"
git checkout -
if $current_commit == $new_commit {
log warn "No changes upstream"
} else {
if $rebase {
log info "Rebasing changes"
git rebase master
print $"(ansi yellow_italic)Rebasing changes(ansi reset)"
git rebase master
} else {
log info "Merging changes"
git merge master
print $"(ansi yellow_italic)Merging changes(ansi reset)"
git merge master
}
}
log info "Linking back the models"
print $"(ansi yellow_italic)Linking back the models(ansi reset)"
if not $clean {
rm pyproject.toml
cp pyproject-mel.toml pyproject.toml
cd $models
if not $clean {
cd $models
# resymlink them
open links.nuon | each {|p| link -a $p.target $p.name }
} else {
let master = (get_root)
link ($master | path join models) $models
link ($master | path join input) $inputs
}
# resymlink them
open links.nuon | each {|p| link -a $p.target $p.name }
} else {
let master = (get_root)
link ($master | path join models) $models
link ($master | path join input) $inputs
}
let commit_count = (git rev-list --count $branch_name $"^origin/($branch_name)")
let commit_count = (git rev-list --count $branch_name $"^origin/($branch_name)")
log success $"Update successful \(($commit_count) new commits\)"
return {from:$current_commit to:$new_commit}
print $"(ansi green_bold)Update successful \(($commit_count) new commits\)(ansi reset)"
}
@@ -182,11 +118,11 @@ export def "comfy toggle_extensions" [--clean] {
return
}
log info "Choices" $choices
print $choices
let filtered = $choices | wrap name | upsert enabled {|p| not ($p.name | str ends-with ".disabled")}
log info "Filtered" $filtered
print $filtered
$filtered | each {|f|
let new_name = ($f.name | str replace ".disabled" "")
@@ -195,7 +131,7 @@ export def "comfy toggle_extensions" [--clean] {
} else {
$new_name
}
log info $"Moving ($f.name) to ($new_name)"
print $"Moving ($f.name) to ($new_name)"
mv $f.name $new_name
}
}
@@ -215,7 +151,6 @@ def --env path-add [pth] {
export-env {
$env.PYTHONUTF8 = 1
$env.COMFY_MTB = ("." | path expand)
# $env.CUDA_ROOT = 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\'
+1 -1
View File
@@ -409,7 +409,7 @@ def main():
args = parser.parse_args()
print_formatted(f"Detected environment: {apply_color(mode, 'cyan')}")
print_formatted(f"Detected environment: {apply_color(mode,'cyan')}")
if args.path:
clone_dir = Path(args.path)
+15 -702
View File
@@ -1,42 +1,17 @@
from typing import Any, TypedDict
from typing import TypedDict
import torch
import torchaudio
from comfy.model_management import get_torch_device
from huggingface_hub import snapshot_download
from transformers import (
WhisperForConditionalGeneration,
WhisperProcessor,
)
# from transformers import (
# AutoFeatureExtractor,
# WhisperForConditionalGeneration,
# WhisperModel,
# WhisperProcessor,
# )
from ..log import log
from ..utils import get_model_path
WHISPER_SAMPLE_RATE = 16000
class AudioTensor(TypedDict):
class AudioDict(TypedDict):
"""Comfy's representation of AUDIO data."""
sample_rate: int
waveform: torch.Tensor
class WhisperData(TypedDict):
"""Whisper transcription data with timestamps and speaker info."""
text: str
chunks: list[dict[str, Any]]
language: str
AudioData = AudioTensor | list[AudioTensor]
AudioData = AudioDict | list[AudioDict]
class MtbAudio:
@@ -53,14 +28,10 @@ class MtbAudio:
return audios["waveform"].shape[1] == 2
@staticmethod
def resample(audio: AudioTensor, common_sample_rate: int) -> AudioTensor:
current_rate = audio["sample_rate"]
if current_rate != common_sample_rate:
log.debug(
f"Resampling audio from {current_rate} to {common_sample_rate}"
)
def resample(audio: AudioDict, common_sample_rate: int) -> AudioDict:
if audio["sample_rate"] != common_sample_rate:
resampler = torchaudio.transforms.Resample(
orig_freq=current_rate, new_freq=common_sample_rate
orig_freq=audio["sample_rate"], new_freq=common_sample_rate
)
return {
"sample_rate": common_sample_rate,
@@ -70,7 +41,7 @@ class MtbAudio:
return audio
@staticmethod
def to_stereo(audio: AudioTensor) -> AudioTensor:
def to_stereo(audio: AudioDict) -> AudioDict:
if audio["waveform"].shape[1] == 1:
return {
"sample_rate": audio["sample_rate"],
@@ -83,8 +54,8 @@ class MtbAudio:
@classmethod
def preprocess_audios(
cls, audios: list[AudioTensor]
) -> tuple[list[AudioTensor], bool, int]:
cls, audios: list[AudioDict]
) -> tuple[list[AudioDict], bool, int]:
max_sample_rate = max([audio["sample_rate"] for audio in audios])
resampled_audios = [
@@ -98,383 +69,6 @@ class MtbAudio:
return (audios, is_stereo, max_sample_rate)
class WhisperPipeline(TypedDict):
"""Whisper model pipeline."""
processor: WhisperProcessor
model: WhisperForConditionalGeneration
class MTB_LoadWhisper:
"""Load Whisper model and processor."""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"model_size": (
[
"tiny",
"small",
"medium",
"medium.en",
"base",
"large",
"large-v2",
"large-v3",
"large-v3-turbo",
],
{"default": "tiny"},
),
},
"optional": {
"download_missing": (
"BOOLEAN",
{
"default": False,
"tooltip": (
"Download missing models if missing,"
"otherwise they must be in ComfyUI/models/whisper"
),
},
),
},
}
RETURN_TYPES = ("WHISPER_PIPELINE",)
RETURN_NAMES = ("pipeline",)
CATEGORY = "mtb/audio"
FUNCTION = "load"
def load(self, model_size="tiny", download_missing=False):
"""Load Whisper model and processor."""
whisper_dir = get_model_path("whisper")
tag = f"whisper-{model_size}"
model_dir = whisper_dir / tag
if not (whisper_dir.exists() or model_dir.exists()):
if not download_missing:
raise RuntimeError(
"Models not found and download_missing=False"
)
else:
whisper_dir.mkdir(exist_ok=True)
model_dir.mkdir(exist_ok=True)
snapshot_download(
repo_id=f"openai/{tag}",
resume_download=True,
ignore_patterns=["*.msgpack", "*.bin", "*.h5"],
local_dir=model_dir.as_posix(),
local_dir_use_symlinks=False,
)
device = get_torch_device()
log.debug(
f"Loading Whisper model {model_size} on {device} from {model_dir}"
)
processor = WhisperProcessor.from_pretrained(model_dir.as_posix())
model = WhisperForConditionalGeneration.from_pretrained(
model_dir.as_posix()
).to(device)
model.eval()
model.requires_grad_(False)
return ({"processor": processor, "model": model},)
class MTB_AudioToText(MtbAudio):
"""Transcribe audio to text using Whisper."""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"pipeline": ("WHISPER_PIPELINE",),
"audio": ("AUDIO",),
"language": (
["auto"]
+ sorted(
[
"en",
"fr",
"es",
"de",
"it",
"pt",
"nl",
"ru",
"zh",
"ja",
"ko",
]
),
{"default": "auto"},
),
"return_timestamps": ("BOOLEAN", {"default": True}),
},
}
RETURN_TYPES = ("STRING", "WHISPER_OUTPUT")
FUNCTION = "transcribe"
CATEGORY = "mtb/audio"
def transcribe(
self,
pipeline: WhisperPipeline,
audio: AudioTensor,
language="auto",
return_timestamps=True,
):
"""Transcribe audio to text using Whisper."""
processor = pipeline["processor"]
model = pipeline["model"]
device = model.device
audio = self.resample(audio, WHISPER_SAMPLE_RATE)
waveform = audio["waveform"]
log.debug(f"Processed waveform shape: {waveform.shape}")
# - Mono: [1, 1, samples] or [1, samples] or [samples]
# - Stereo: [1, 2, samples] or [2, samples] or [samples, 2]
if len(waveform.shape) == 3:
waveform = waveform.squeeze(0)
if len(waveform.shape) == 2:
if waveform.shape[0] == 2: # [channels, samples]
waveform = waveform.mean(dim=0)
elif waveform.shape[1] == 2: # [samples, channels]
waveform = waveform.mean(dim=1)
else: # mono
waveform = waveform.squeeze(0)
sample_rate = audio["sample_rate"]
chunk_duration = 30
chunk_samples = chunk_duration * sample_rate
total_samples = waveform.shape[-1]
total_duration = total_samples / sample_rate
log.debug(f"Audio duration: {total_duration:.2f}s")
all_tokens = []
all_text = []
chunk_offsets = []
last_time = 0.0
accumulated_offset = 0.0
for chunk_start in range(0, total_samples, chunk_samples):
chunk_end = min(chunk_start + chunk_samples, total_samples)
chunk_waveform = waveform[chunk_start:chunk_end]
chunk_offset = chunk_start / sample_rate
chunk_offsets.append(chunk_offset)
log.debug(
f"Processing chunk {chunk_offset:.1f}s - {chunk_end / sample_rate:.1f}s"
)
max_length = model.config.max_length or 448
attention_mask = torch.ones((1, max_length))
input_features = processor(
chunk_waveform,
sampling_rate=sample_rate,
return_tensors="pt",
).input_features.to(device)
with torch.no_grad():
predicted_ids = model.generate(
input_features,
attention_mask=attention_mask.to(device),
task="transcribe",
language=None if language == "auto" else language,
return_timestamps=return_timestamps,
no_repeat_ngram_size=3,
num_beams=5,
length_penalty=1.0,
max_length=max_length,
)
chunk_tokens = processor.tokenizer.convert_ids_to_tokens(
predicted_ids[0]
)
adjusted_tokens = []
for token in chunk_tokens:
if token.startswith("<|") and token.endswith("|>"):
try:
time_str = token[2:-2]
if time_str.replace(".", "").isdigit():
time_val = float(time_str)
# If this timestamp is less than the last one, we've started a new sequence
if time_val < last_time:
accumulated_offset += last_time
adjusted_time = time_val + accumulated_offset
adjusted_tokens.append(f"<|{adjusted_time:.2f}|>")
last_time = time_val
else:
adjusted_tokens.append(token)
except ValueError:
adjusted_tokens.append(token)
else:
adjusted_tokens.append(token)
all_tokens.extend(adjusted_tokens)
chunk_text = processor.batch_decode(
predicted_ids, skip_special_tokens=True
)[0]
all_text.append(chunk_text)
detected_language = "en"
if language == "auto":
try:
log.debug("Detecting language")
with torch.no_grad():
first_chunk_features = processor(
waveform[:chunk_samples],
sampling_rate=sample_rate,
return_tensors="pt",
).input_features.to(device)
predicted_probs = model.detect_language(
first_chunk_features
)[0]
language_token = processor.tokenizer.convert_ids_to_tokens(
predicted_probs.argmax(-1).item()
)
detected_language = (
language_token[2:-2]
if language_token.startswith("<|")
else "en"
)
log.debug(f"Detected language: {detected_language}")
except Exception as e:
log.warning(f"Language detection failed: {e}")
full_transcription = " ".join(all_text)
whisper_output = {
"text": full_transcription,
"language": detected_language,
"tokens": all_tokens,
"audio": audio,
"chunk_offsets": chunk_offsets,
}
return full_transcription, whisper_output
class MTB_ProcessWhisperOutput:
"""Process Whisper output into timestamped chunks."""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"whisper_output": ("WHISPER_OUTPUT",),
"min_chunk_length": (
"FLOAT",
{"default": 0.0, "min": 0.0, "max": 10.0, "step": 0.1},
),
},
}
RETURN_TYPES = ("STRING", "WHISPER_CHUNKS")
FUNCTION = "process"
CATEGORY = "mtb/audio"
def process(self, whisper_output, min_chunk_length=0.0):
"""Process Whisper output into timestamped chunks."""
tokens = whisper_output["tokens"]
audio = whisper_output["audio"]
timestamp_tokens = []
audio_duration = audio["waveform"].shape[-1] / audio["sample_rate"]
log.debug(f"Audio duration: {audio_duration:.2f}s")
for i, token in enumerate(tokens):
if token.startswith("<|") and token.endswith("|>"):
try:
time_str = token[2:-2]
if time_str.replace(".", "").isdigit():
time_val = float(time_str)
if 0 <= time_val <= audio_duration:
timestamp_tokens.append((i, time_val))
log.debug(f"Token {i}: {time_val}")
except ValueError:
continue
chunks = []
if len(timestamp_tokens) > 1:
for i in range(len(timestamp_tokens) - 1):
start_pos, start_time = timestamp_tokens[i]
end_pos, end_time = timestamp_tokens[i + 1]
if end_time - start_time < min_chunk_length:
continue
chunk_tokens = tokens[start_pos + 1 : end_pos]
text = " ".join(
t
for t in chunk_tokens
if not (t.startswith("<|") and t.endswith("|>"))
)
if text.strip():
chunks.append(
{
"text": text.strip(),
"timestamp": [start_time, end_time],
}
)
if timestamp_tokens:
start_pos, start_time = timestamp_tokens[-1]
if start_pos < len(tokens) - 1:
text = " ".join(
t
for t in tokens[start_pos + 1 :]
if not (t.startswith("<|") and t.endswith("|>"))
)
if text.strip():
if chunks:
prev_chunk = chunks[-1]
prev_duration = (
prev_chunk["timestamp"][1]
- prev_chunk["timestamp"][0]
)
end_time = min(
start_time + prev_duration, audio_duration
)
else:
end_time = audio_duration
if (
end_time > start_time
and end_time - start_time >= min_chunk_length
):
chunks.append(
{
"text": text.strip(),
"timestamp": [start_time, end_time],
}
)
result = {
"text": whisper_output["text"],
"chunks": chunks,
"language": whisper_output["language"],
}
return whisper_output["text"], result
class MTB_AudioCut(MtbAudio):
"""Basic audio cutter, values are in ms."""
@@ -504,7 +98,7 @@ class MTB_AudioCut(MtbAudio):
CATEGORY = "mtb/audio"
FUNCTION = "cut"
def cut(self, audio: AudioTensor, length: float, offset: float):
def cut(self, audio: AudioDict, length: float, offset: float):
sample_rate = audio["sample_rate"]
start_idx = int(offset * sample_rate / 1000)
end_idx = min(
@@ -523,6 +117,7 @@ class MTB_AudioCut(MtbAudio):
class MTB_AudioStack(MtbAudio):
"""Stack/Overlay audio inputs (dynamic inputs).
- pad audios to the longest inputs.
- resample audios to the highest sample rate in the inputs.
- convert them all to stereo if one of the inputs is.
@@ -537,7 +132,7 @@ class MTB_AudioStack(MtbAudio):
CATEGORY = "mtb/audio"
FUNCTION = "stack"
def stack(self, **kwargs: AudioTensor) -> tuple[AudioTensor]:
def stack(self, **kwargs: AudioDict) -> tuple[AudioDict]:
audios, is_stereo, max_rate = self.preprocess_audios(
list(kwargs.values())
)
@@ -568,6 +163,7 @@ class MTB_AudioStack(MtbAudio):
class MTB_AudioSequence(MtbAudio):
"""Sequence audio inputs (dynamic inputs).
- adding silence_duration between each segment
can now also be negative to overlap the clips, safely bound
to the the input length.
@@ -591,7 +187,7 @@ class MTB_AudioSequence(MtbAudio):
CATEGORY = "mtb/audio"
FUNCTION = "sequence"
def sequence(self, silence_duration: float, **kwargs: AudioTensor):
def sequence(self, silence_duration: float, **kwargs: AudioDict):
audios, is_stereo, max_rate = self.preprocess_audios(
list(kwargs.values())
)
@@ -636,287 +232,4 @@ class MTB_AudioSequence(MtbAudio):
)
class MTB_AudioResample(MtbAudio):
"""Resample audio to a different sample rate."""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"audio": ("AUDIO",),
"sample_rate": (
"INT",
{
"default": 16000,
"min": 1000,
"max": 192000,
"step": 100,
"tooltip": "Target sample rate in Hz. Whisper requires 16000.",
},
),
}
}
RETURN_TYPES = ("AUDIO",)
RETURN_NAMES = ("resampled_audio",)
CATEGORY = "mtb/audio"
FUNCTION = "resample_audio"
def resample_audio(
self, audio: AudioTensor, sample_rate: int
) -> tuple[AudioTensor]:
resampled = self.resample(audio, sample_rate)
return (resampled,)
class MTB_AudioIsolateSpeaker(MtbAudio):
"""Isolate or mute specific speakers using WhisperData"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"audio": ("AUDIO",),
"whisper_data": ("WHISPER_CHUNKS",),
"target_speaker": ("STRING", {"default": "SPEAKER_00"}),
"mode": (["isolate", "mute"], {"default": "isolate"}),
"fade_ms": (
"FLOAT",
{
"default": 100.0,
"min": 0.0,
"max": 1000.0,
"step": 10,
"tooltip": "Fade duration in milliseconds to avoid clicks",
},
),
},
}
RETURN_TYPES = ("AUDIO",)
RETURN_NAMES = ("processed_audio",)
CATEGORY = "mtb/audio"
FUNCTION = "process_audio"
def process_audio(
self,
audio: AudioTensor,
whisper_data: WhisperData,
target_speaker: str,
mode: str = "isolate",
fade_ms: float = 100.0,
) -> tuple[AudioTensor]:
fade_samples = int((fade_ms / 1000.0) * audio["sample_rate"])
mask = (
torch.zeros_like(audio["waveform"])
if mode == "isolate"
else torch.ones_like(audio["waveform"])
)
for chunk in whisper_data["chunks"]:
if not chunk.get("speaker"):
continue
speaker_present = target_speaker in chunk["speaker"]
if (mode == "isolate" and speaker_present) or (
mode == "mute" and not speaker_present
):
start_sample = int(
chunk["timestamp"][0] * audio["sample_rate"]
)
end_sample = int(chunk["timestamp"][1] * audio["sample_rate"])
mask[:, start_sample:end_sample] = 1.0
if fade_samples > 0:
fade = torch.linspace(0, 1, fade_samples)
transitions = torch.where(mask[0, 1:] != mask[0, :-1])[0] + 1
for trans_idx in transitions:
if (
trans_idx >= fade_samples
and trans_idx <= mask.shape[1] - fade_samples
):
if mask[0, trans_idx] == 1:
mask[:, trans_idx : trans_idx + fade_samples] *= fade
else:
mask[:, trans_idx - fade_samples : trans_idx] *= (
fade.flip(0)
)
processed_waveform = audio["waveform"] * mask
return (
{
"sample_rate": audio["sample_rate"],
"waveform": processed_waveform,
},
)
class MTB_ProcessWhisperDiarization:
"""Process Whisper chunks with speaker diarization using either pyannote or NeMo."""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"whisper_chunks": ("WHISPER_CHUNKS",),
"audio": ("AUDIO",),
"backend": (["pyannote", "nemo"], {"default": "pyannote"}),
"num_speakers": (
"INT",
{"default": 2, "min": 1, "max": 10, "step": 1},
),
},
"optional": {
"device": (["cuda", "cpu"], {"default": "cuda"}),
},
}
RETURN_TYPES = ("WHISPER_CHUNKS",)
FUNCTION = "process"
CATEGORY = "mtb/audio"
def process_pyannote(self, audio, num_speakers, device):
"""Process audio using pyannote backend."""
try:
from pyannote.audio import Pipeline
from pyannote.audio.pipelines.utils.hook import ProgressHook
except ImportError:
raise ImportError(
"pyannote.audio not found. Install with: pip install pyannote.audio"
)
pipeline = Pipeline.from_pretrained(
"pyannote/speaker-diarization-3.1", use_auth_token=None
)
pipeline.to(torch.device(device))
with ProgressHook() as hook:
diarization = pipeline(
{
"waveform": audio["waveform"][0],
"sample_rate": audio["sample_rate"],
},
num_speakers=num_speakers,
hook=hook,
)
speaker_segments = []
for turn, _, speaker in diarization.itertracks(yield_label=True):
speaker_segments.append(
{
"start": turn.start,
"end": turn.end,
"speaker": speaker,
}
)
return speaker_segments
def process_nemo(self, audio, num_speakers, device):
"""Process audio using NeMo backend."""
try:
import nemo.collections.asr as nemo_asr
except ImportError:
raise ImportError(
"NeMo not found. Install with: pip install nemo_toolkit[asr]"
)
model = nemo_asr.models.ClusteringDiarizer.from_pretrained(
"nvidia/speakerverification_en_titanet_large"
).to(device)
diarization = model.diarize(
audio=audio["waveform"][0],
sample_rate=audio["sample_rate"],
num_speakers=num_speakers,
)
speaker_segments = []
for segment in diarization:
speaker_segments.append(
{
"start": segment["start"],
"end": segment["end"],
"speaker": f"SPEAKER_{segment['speaker']}",
}
)
return speaker_segments
def process(
self,
whisper_chunks,
audio,
backend="pyannote",
num_speakers=2,
device="cuda",
):
if backend == "pyannote":
speaker_segments = self.process_pyannote(
audio, num_speakers, device
)
else: # nemo
speaker_segments = self.process_nemo(audio, num_speakers, device)
for chunk in whisper_chunks["chunks"]:
chunk_start, chunk_end = chunk["timestamp"]
chunk_speakers = set()
for segment in speaker_segments:
if (
segment["start"] <= chunk_end
and segment["end"] >= chunk_start
):
chunk_speakers.add(segment["speaker"])
if chunk_speakers:
chunk["speaker"] = list(chunk_speakers)[0]
else:
chunk["speaker"] = "unknown"
return (whisper_chunks,)
class MTB_AudioDuration:
"""Get audio duration in milliseconds."""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"audio": ("AUDIO",),
},
}
RETURN_TYPES = ("INT",)
RETURN_NAMES = ("duration_ms",)
FUNCTION = "get_duration"
CATEGORY = "mtb/audio"
def get_duration(self, audio):
waveform = audio["waveform"]
sample_rate = audio["sample_rate"]
duration_ms = int((waveform.shape[-1] / sample_rate) * 1000)
log.debug(
f"Audio duration: {duration_ms}ms ({duration_ms / 1000:.2f}s)"
)
return (duration_ms,)
__nodes__ = [
MTB_AudioSequence,
MTB_AudioStack,
MTB_AudioCut,
MTB_AudioResample,
MTB_AudioIsolateSpeaker,
MTB_LoadWhisper,
MTB_AudioToText,
MTB_ProcessWhisperOutput,
MTB_ProcessWhisperDiarization,
MTB_AudioDuration,
]
__nodes__ = [MTB_AudioSequence, MTB_AudioStack, MTB_AudioCut]
+7 -292
View File
@@ -1,18 +1,13 @@
import os
import random
from io import BytesIO
from pathlib import Path
from typing import Literal
import comfy.utils
import cv2
import folder_paths
import numpy as np
import torch
from PIL import Image
from ..log import log
from ..utils import EASINGS, apply_easing, glob_multiple, pil2tensor
from ..utils import EASINGS, apply_easing, pil2tensor
from .transform import MTB_TransformImage
@@ -52,7 +47,7 @@ class MTB_BatchFloatMath:
for v in vals:
if len(v) != ref_count:
raise ValueError(
f"All values must have the same length (current: {len(v)}, ref: {ref_count})"
f"All values must have the same length (current: {len(v)}, ref: {ref_count}"
)
match operation:
@@ -175,124 +170,6 @@ class MTB_BatchTimeWrap:
return (warped_tensor, interpolated_curve)
class MTB_ImageBatchToSublist:
"""
# Image Batch To Sublist 🔄
Splits a large batched tensor into smaller sub-batches for memory-efficient processing.
"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"sub_batch_size": (
"INT",
{"default": 1, "min": 1, "max": 1000, "step": 1},
),
},
"optional": {
"image": ("IMAGE",),
"mask": ("MASK",),
},
}
RETURN_TYPES = ("IMAGE", "MASK", "INT")
RETURN_NAMES = ("image_list", "mask_list", "item_count")
OUTPUT_IS_LIST = (True, True)
FUNCTION = "split_batch"
CATEGORY = "batch_processing"
def split_batch(
self,
sub_batch_size: int,
image: torch.Tensor | None = None,
mask: torch.Tensor | None = None,
):
if image is None and mask is None:
raise ValueError(
"You must either pass mask or image, none received"
)
image_count = 0
if image is not None:
image_count = image.size(0)
mask_count = 0
if mask is not None:
mask_count = mask.size(0)
if image_count > 0 and mask_count > 0 and mask_count != image_count:
raise ValueError(
f"When providing image and mask, batch size must match (got {mask.size(0)} mask and {image.size(0)} images)"
)
batch_size = max(image_count, mask_count)
num_full_batches = batch_size // sub_batch_size
im_batches = []
mask_batches = []
for i in range(num_full_batches):
start_idx = i * sub_batch_size
end_idx = start_idx + sub_batch_size
if image_count > 0:
im_batches.append(image[start_idx:end_idx, ...])
if mask_count > 0:
mask_batches.append(mask[start_idx:end_idx, ...])
if batch_size % sub_batch_size != 0:
remaining_start = num_full_batches * sub_batch_size
if image_count > 0:
im_batches.append(image[remaining_start:, ...])
if mask_count > 0:
mask_batches.append(mask[remaining_start:, ...])
return (im_batches, mask_batches, len(im_batches))
class MTB_SublistToImageBatch:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"tensors": ("IMAGE",),
}
}
INPUT_IS_LIST = True
RETURN_TYPES = ("IMAGE",)
FUNCTION = "merge_batches"
CATEGORY = "batch_processing"
DOCUMENTATION = """# Sublist to Image Batch 🔄
Merges a list of sub-batched tensors back into a single large batch.
"""
def merge_batches(self, tensors: list[torch.Tensor]):
if len(tensors) <= 1:
return (tensors[0],)
result = tensors[0]
for next_tensor in tensors[1:]:
if result.shape[1:] != next_tensor.shape[1:]:
next_tensor = comfy.utils.common_upscale(
next_tensor.movedim(-1, 1),
result.shape[2],
result.shape[1],
"lanczos",
"center",
).movedim(1, -1)
result = torch.cat((result, next_tensor), dim=0)
return (result,)
class MTB_BatchMake:
"""Simply duplicates the input frame as a batch"""
@@ -302,22 +179,18 @@ class MTB_BatchMake:
"required": {
"image": ("IMAGE",),
"count": ("INT", {"default": 1}),
},
"optional": {"mask": ("MASK",)},
}
}
RETURN_TYPES = ("IMAGE",)
FUNCTION = "generate_batch"
CATEGORY = "mtb/batch"
def generate_batch(self, image: torch.Tensor, count, mask=None):
def generate_batch(self, image: torch.Tensor, count):
if len(image.shape) == 3:
image = image.unsqueeze(0)
return (
image.repeat(count, 1, 1, 1),
mask.repeat(count, 1, 1) if mask else mask,
)
return (image.repeat(count, 1, 1, 1),)
class MTB_BatchShape:
@@ -502,14 +375,8 @@ class MTB_BatchFloat:
{"default": "Steps"},
),
"count": ("INT", {"default": 2}),
"min": (
"FLOAT",
{"default": 0.0, "min": -1e4, "max": 1e4, "step": 0.001},
),
"max": (
"FLOAT",
{"default": 1.0, "min": -1e4, "max": 1e4, "step": 0.001},
),
"min": ("FLOAT", {"default": 0.0, "step": 0.001}),
"max": ("FLOAT", {"default": 1.0, "step": 0.001}),
"easing": (
[
"Linear",
@@ -850,13 +717,6 @@ class MTB_Batch2dTransform:
"zoom": ("FLOATS",),
"angle": ("FLOATS",),
"shear": ("FLOATS",),
"use_normalized": (
"BOOLEAN",
{
"default": False,
"tooltip": "If true, transform values will be scaled to image dimensions.",
},
),
},
}
@@ -885,7 +745,6 @@ class MTB_Batch2dTransform:
zoom: list[float] | None = None,
angle: list[float] | None = None,
shear: list[float] | None = None,
use_normalized: bool = False,
):
if all(
self.get_num_elements(param) <= 0
@@ -937,7 +796,6 @@ class MTB_Batch2dTransform:
keyframes["shear"][i],
border_handling,
constant_color,
use_normalized=use_normalized,
)[0]
for i in range(image.shape[0])
]
@@ -1381,146 +1239,6 @@ class MTB_BatchShake:
return (shaken_images, x_translations, y_translations, rotations)
class MTB_BatchFromFolder:
"""Load images from a folder with options for latest, oldest, or random selection."""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"enable": (
"BOOLEAN",
{
"default": True,
"tooltip": "Enable or disable the node. If disabled, returns passthrough_image or an empty tensor.",
},
),
"folder_path": (
"STRING",
{
"default": "",
"tooltip": "Path to the folder containing images. Relative paths are resolved to the ComfyUI output directory.",
},
),
"mode": (
["latest", "oldest", "random"],
{
"default": "latest",
"tooltip": "How to select images: latest, oldest, or random.",
},
),
"count": (
"INT",
{
"default": 10,
"min": 1,
"max": 1000,
"tooltip": "Number of images to load from the folder.",
},
),
"filter": (
"STRING",
{
"default": "*",
"tooltip": "Glob filter for image filenames (e.g. *.png).",
},
),
},
"optional": {
"passthrough_image": (
"IMAGE",
{
"tooltip": "If provided and node is disabled, this image is passed through instead of returning an empty tensor."
},
),
},
}
RETURN_TYPES = ("IMAGE",)
RETURN_NAMES = ("images",)
CATEGORY = "mtb/batch"
FUNCTION = "load_from_folder"
def load_from_folder(
self,
enable: bool,
folder_path: str,
mode: str,
count: int,
filter: str,
passthrough_image=None,
):
"""Load images from a folder with the specified selection mode."""
if not enable:
if passthrough_image is not None:
log.debug(
"MTB_BatchFromFolder: Using passthrough image (disabled)"
)
return (passthrough_image,)
log.debug(
"MTB_BatchFromFolder: Disabled and no passthrough_image provided, returning empty tensor"
)
return (torch.zeros(0, 0, 0, 3),)
path_obj = Path(folder_path)
if not path_obj.is_absolute():
output_dir = Path(folder_paths.get_output_directory())
path_obj = output_dir / folder_path
path_obj = path_obj.resolve()
if not path_obj.exists():
log.error(f"Folder path does not exist: {path_obj}")
return (torch.zeros(0, 0, 0, 3),)
if not path_obj.is_dir():
log.error(f"Path is not a directory: {path_obj}")
return (torch.zeros(0, 0, 0, 3),)
patterns = [filter] if filter else ["*"]
files = glob_multiple(path_obj, patterns)
image_extensions = [".png", ".jpg", ".jpeg", ".bmp", ".webp", ".tiff"]
image_files = [
f for f in files if f.suffix.lower() in image_extensions
]
if not image_files:
log.warning(
f"No image files found in {path_obj} with filter {filter}"
)
return (torch.zeros(0, 0, 0, 3),)
if mode == "latest":
image_files.sort(key=lambda x: os.path.getmtime(x), reverse=True)
elif mode == "oldest":
image_files.sort(key=lambda x: os.path.getmtime(x))
elif mode == "random":
random.shuffle(image_files)
selected_files = image_files[:count]
if len(selected_files) < count:
log.warning(
f"Requested {count} images but only found {len(selected_files)}"
)
loaded_images = []
for file_path in selected_files:
try:
img = Image.open(file_path)
if img.mode != "RGB":
img = img.convert("RGB")
loaded_images.append(img)
except Exception as e:
log.error(f"Error loading image {file_path}: {e}")
if not loaded_images:
log.error("Failed to load any images")
return (torch.zeros(0, 0, 0, 3),)
return (pil2tensor(loaded_images),)
__nodes__ = [
MTB_Batch2dTransform,
MTB_BatchFloat,
@@ -1529,7 +1247,6 @@ __nodes__ = [
MTB_BatchFloatFit,
MTB_BatchFloatMath,
MTB_BatchFloatNormalize,
MTB_BatchFromFolder,
MTB_BatchMake,
MTB_BatchMerge,
MTB_BatchSequence,
@@ -1538,6 +1255,4 @@ __nodes__ = [
MTB_BatchShape,
MTB_BatchTimeWrap,
MTB_PlotBatchFloat,
MTB_SublistToImageBatch,
MTB_ImageBatchToSublist,
]
+7 -67
View File
@@ -78,17 +78,14 @@ class MTB_UpscaleBboxBy:
self, bbox: tuple[int, int, int, int], scale: float
) -> tuple[tuple[int, int, int, int]]:
x, y, width, height = bbox
# scaled = (x * scale, y * scale, width * scale, height * scale)
scaled = (
int(x * scale),
int(y * scale),
int(width * scale),
int(height * scale),
)
center_x = x + width // 2
center_y = y + height // 2
new_width = int(width * scale)
new_height = int(height * scale)
new_x = center_x - new_width // 2
new_y = center_y - new_height // 2
scaled = (new_x, new_y, new_width, new_height)
return (scaled,)
@@ -375,62 +372,6 @@ class MTB_Uncrop:
return (pil2tensor(out_images),)
class MTB_BBoxForceDimensions:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"bbox": ("BBOX",),
"width": ("INT", {"default": 512, "min": 1, "max": 8192}),
"height": ("INT", {"default": 512, "min": 1, "max": 8192}),
},
"optional": {
"image": ("IMAGE",),
},
}
CATEGORY = "mtb/crop"
RETURN_TYPES = ("BBOX",)
FUNCTION = "force_dimensions"
def force_dimensions(
self,
bbox: tuple[int, int, int, int],
width: int,
height: int,
image: torch.Tensor = None,
) -> tuple[tuple[int, int, int, int]]:
x, y, curr_width, curr_height = bbox
center_x = x + curr_width // 2
center_y = y + curr_height // 2
new_x = center_x - width // 2
new_y = center_y - height // 2
if image is not None:
img_height, img_width = image.shape[1:3]
x_overflow = max(0, new_x + width - img_width) + min(0, new_x)
y_overflow = max(0, new_y + height - img_height) + min(0, new_y)
if width > img_width or height > img_height:
x_exceed = width - img_width if width > img_width else 0
y_exceed = height - img_height if height > img_height else 0
raise ValueError(
f"Target bbox dimensions ({width}x{height}) exceed image bounds ({img_width}x{img_height}) "
f"by {x_exceed}px horizontally and {y_exceed}px vertically"
)
if x_overflow > 0 or x_overflow < 0:
new_x -= x_overflow
if y_overflow > 0:
new_y -= y_overflow
elif y_overflow < 0:
new_y -= y_overflow # Add the negative overflow
return ((int(new_x), int(new_y), width, height),)
__nodes__ = [
MTB_BboxFromMask,
MTB_Bbox,
@@ -438,5 +379,4 @@ __nodes__ = [
MTB_Uncrop,
MTB_SplitBbox,
MTB_UpscaleBboxBy,
MTB_BBoxForceDimensions,
]
+24 -101
View File
@@ -2,6 +2,7 @@ import base64
import io
import json
from pathlib import Path
from typing import Optional
import folder_paths
import torch
@@ -10,66 +11,13 @@ from ..log import log
from ..utils import tensor2pil
def get_detailed_type_info(obj):
type_info = []
type_name = type(obj).__name__
type_info.append(f"Type: {type_name}")
if isinstance(obj, torch.Tensor):
type_info.extend(
[
f"Shape: {obj.shape}",
f"Dtype: {obj.dtype}",
f"Device: {obj.device}",
f"Requires grad: {obj.requires_grad}",
f"Stride: {obj.stride()}",
f"Contiguous: {obj.is_contiguous()}",
]
)
elif isinstance(obj, (list, tuple)):
type_info.extend(
[
f"Length: {len(obj)}",
f"Container type: {type_name}",
]
)
if obj:
type_info.append(f"Element type: {type(obj[0]).__name__}")
elif isinstance(obj, dict):
type_info.extend(
[
f"Length: {len(obj)}",
f"Keys: {list(obj.keys())}",
]
)
elif hasattr(obj, "__dict__"):
attributes = [attr for attr in dir(obj) if not attr.startswith("_")]
type_info.append(f"Attributes: {attributes}")
return type_info
# region processors
def process_tensor(tensor: torch.Tensor, as_type=False):
def process_tensor(tensor):
log.debug(f"Tensor: {tensor.shape}")
if as_type:
return {
"text": [f"Tensor of shape {tensor.shape} of type {tensor.dtype}"]
}
is_mask = len(tensor.shape) == 3
if is_mask:
tensor = tensor.unsqueeze(-1).repeat(1, 1, 1, 3)
image = tensor2pil(tensor)
b64_imgs = []
for im in image:
if is_mask:
im = im.convert("L")
buffered = io.BytesIO()
im.save(buffered, format="PNG")
b64_imgs.append(
@@ -80,16 +28,11 @@ def process_tensor(tensor: torch.Tensor, as_type=False):
return {"b64_images": b64_imgs}
def process_list(anything, as_type=False):
def process_list(anything):
text = []
if not anything:
return {"text": []}
if as_type:
type_info = get_detailed_type_info(anything)
type_info.extend(get_detailed_type_info(anything[0]))
return {"text": type_info}
first_element = anything[0]
if (
isinstance(first_element, list)
@@ -111,41 +54,25 @@ def process_list(anything, as_type=False):
return {"text": text}
def process_dict(anything, as_type=False):
def process_dict(anything):
text = []
if as_type:
return {"text": get_detailed_type_info(anything)}
if "samples" in anything:
is_empty = (
"(empty)" if torch.count_nonzero(anything["samples"]) == 0 else ""
)
text.append(f"Latent Samples: {anything['samples'].shape} {is_empty}")
elif "waveform" in anything:
is_empty = (
"(empty) " if torch.count_nonzero(anything["samples"]) == 0 else ""
)
text.append(
f"Audio Samples: {anything['waveform'].shape}{is_empty} | sample rate {anything['sample_rate']}"
)
else:
log.debug(f"Unhandled dict: {anything.keys()}")
text.append(json.dumps(anything, indent=2))
return {"text": text}
def process_bool(anything, as_type=False):
def process_bool(anything):
return {"text": ["True" if anything else "False"]}
def process_text(anything, as_type=False):
if as_type:
return {"text": get_detailed_type_info(anything)}
def process_text(anything):
return {"text": [str(anything)]}
@@ -162,7 +89,6 @@ class MTB_Debug:
def INPUT_TYPES(cls):
return {
"required": {"output_to_console": ("BOOLEAN", {"default": False})},
"optional": {"as_detailed_types": ("BOOLEAN", {"default": False})},
}
RETURN_TYPES = ()
@@ -170,25 +96,29 @@ class MTB_Debug:
CATEGORY = "mtb/debug"
OUTPUT_NODE = True
def do_debug(
self, output_to_console: bool, as_detailed_types: bool, **kwargs
):
output = {"ui": {"items": []}}
def do_debug(self, output_to_console: bool, **kwargs):
output = {
"ui": {"b64_images": [], "text": []},
# "result": ("A"),
}
processors = {
torch.Tensor: process_tensor,
list: process_list,
dict: process_dict,
bool: process_bool,
}
if output_to_console:
for k, v in kwargs.items():
log.info(f"{k}: {v}")
for input_name, anything in kwargs.items():
for anything in kwargs.values():
processor = processors.get(type(anything), process_text)
processed = processor(anything, as_detailed_types)
processed_data = processor(anything)
item = {
"input": input_name,
**processed,
}
output["ui"]["items"].append(item)
for ui_key, ui_value in processed_data.items():
output["ui"][ui_key].extend(ui_value)
return output
@@ -224,9 +154,9 @@ class MTB_SaveTensors:
def save(
self,
filename_prefix,
image: torch.Tensor | None = None,
mask: torch.Tensor | None = None,
latent: torch.Tensor | None = None,
image: Optional[torch.Tensor] = None,
mask: Optional[torch.Tensor] = None,
latent: Optional[torch.Tensor] = None,
):
(
full_output_folder,
@@ -258,11 +188,4 @@ class MTB_SaveTensors:
return f"{filename_prefix}_{counter:05}"
processors = {
torch.Tensor: process_tensor,
list: process_list,
dict: process_dict,
bool: process_bool,
}
__nodes__ = [MTB_Debug, MTB_SaveTensors]
+4 -25
View File
@@ -2,16 +2,13 @@ import tempfile
from pathlib import Path
import numpy as np
# torch must be imported prior to onnx for the CUDAProvider.
import torch # isort:skip
import onnxruntime as ort
import torch
from PIL import Image
from ..errors import ModelNotFound
from ..log import mklog
from ..utils import (
download_model,
get_model_path,
tensor2pil,
tiles_infer,
@@ -26,12 +23,7 @@ log = mklog(__name__)
# - COLOR to NORMALS
def color_to_normals(
color_img,
overlap,
progress_callback,
*,
save_temp=False,
auto_download=False,
color_img, overlap, progress_callback, *, save_temp=False
):
"""Compute a normal map from the given color map.
@@ -75,13 +67,7 @@ def color_to_normals(
log.debug("DeepBump Color → Normals : loading model")
model = get_model_path("deepbump", "deepbump256.onnx")
if not model or not model.exists():
if not auto_download:
raise ModelNotFound(f"deepbump ({model})")
log.debug("Downloading models...")
download_model(
"https://github.com/HugoTini/DeepBump/raw/master/deepbump256.onnx",
"deepbump",
)
raise ModelNotFound(f"deepbump ({model})")
providers = [
"TensorrtExecutionProvider",
@@ -365,9 +351,6 @@ class MTB_DeepBump:
),
"normals_to_height_seamless": ("BOOLEAN", {"default": True}),
},
"optional": {
"auto_download": ("BOOLEAN", {"default": True}),
},
}
RETURN_TYPES = ("IMAGE",)
@@ -383,7 +366,6 @@ class MTB_DeepBump:
color_to_normals_overlap="SMALL",
normals_to_curvature_blur_radius="SMALL",
normals_to_height_seamless=True,
auto_download=False,
):
images = tensor2pil(image)
out_images = []
@@ -398,10 +380,7 @@ class MTB_DeepBump:
# Apply processing
if mode == "Color to Normals":
out_img = color_to_normals(
in_img,
color_to_normals_overlap,
None,
auto_download=auto_download,
in_img, color_to_normals_overlap, None
)
if mode == "Normals to Curvature":
out_img = normals_to_curvature(
+52 -191
View File
@@ -1,8 +1,4 @@
import io
import requests
import torch
from PIL import Image, ImageDraw, ImageFont
from PIL import Image
from ..log import log
from ..utils import comfy_dir, font_path, pil2tensor
@@ -85,6 +81,10 @@ class MTB_UnsplashImage:
CATEGORY = "mtb/generate"
def do_unsplash_image(self, width, height, random_seed, keyword=None):
import io
import requests
base_url = "https://source.unsplash.com/random/"
if width and height:
@@ -213,90 +213,14 @@ by default it fallsback to a default font.
"INT",
{"default": 100, "min": 1, "max": 100, "step": 1},
),
},
"optional": {
"whisper_chunks": ("WHISPER_CHUNKS",),
"fps": (
"INT",
{"default": 24, "min": 1, "max": 60, "step": 1},
),
"fade_duration": (
"FLOAT",
{"default": 0.5, "min": 0.0, "max": 5.0, "step": 0.1},
),
},
}
}
RETURN_TYPES = ("IMAGE",)
RETURN_NAMES = ("image",)
FUNCTION = "text_to_image"
CATEGORY = "mtb/generate"
def create_animation_frames(
self,
chunks,
base_image,
font,
font_size,
color,
width,
height,
fps,
fade_duration,
):
"""Create animation frames from Whisper chunks."""
if not chunks or not chunks.get("chunks"):
return [base_image]
frames = []
total_duration = chunks["chunks"][-1]["timestamp"][1]
frame_count = int(total_duration * fps)
fade_frames = int(fade_duration * fps)
for frame_idx in range(frame_count):
time = frame_idx / fps
frame = base_image.copy()
draw = ImageDraw.Draw(frame)
active_chunks = []
for chunk in chunks["chunks"]:
start, end = chunk["timestamp"]
if start <= time <= end:
fade_in_alpha = min(
1.0, (time - start) * fps / fade_frames
)
fade_out_alpha = min(1.0, (end - time) * fps / fade_frames)
alpha = min(fade_in_alpha, fade_out_alpha)
active_chunks.append((chunk["text"], alpha))
y = height // 4
for text, alpha in active_chunks:
# Create a temporary image for the text with alpha
text_img = Image.new("RGBA", (width, height), (0, 0, 0, 0))
text_draw = ImageDraw.Draw(text_img)
text_draw.text(
(width // 2, y),
text,
font=font,
fill=color,
anchor="mm",
)
text_img.putalpha(
Image.fromarray(
(torch.ones((height, width)) * (alpha * 255))
.byte()
.numpy()
)
)
frame = Image.alpha_composite(frame, text_img)
y += font_size * 1.5
frames.append(frame)
return frames
def text_to_image(
self,
text: str,
@@ -314,121 +238,58 @@ by default it fallsback to a default font.
h_offset=0,
v_offset=0,
h_coverage=100,
whisper_chunks=None,
fps=24,
fade_duration=0.5,
):
"""Convert text to image, with optional animation support."""
import textwrap
from PIL import ImageColor
from PIL import Image, ImageDraw, ImageFont
font_path = self.fonts[font]
font = ImageFont.truetype(font_path, size=font_size)
try:
if isinstance(color, str):
color = ImageColor.getrgb(color)
if isinstance(background, str):
background = ImageColor.getrgb(background)
if len(color) == 3:
color = color + (255,)
if len(background) == 3:
background = background + (255,)
except ValueError as e:
log.error(f"Color parsing error: {e}")
color = (255, 255, 255, 255)
background = (0, 0, 0, 255)
def render_text(text_to_render, alpha=None):
if trim:
text_to_render = (
text_to_render.encode("ascii", "ignore").decode().strip()
)
if wrap:
wrap_width = (((width / 100) * h_coverage) / font_size) * 2
lines = textwrap.wrap(text_to_render, width=wrap_width)
else:
lines = [text_to_render]
img = Image.new("RGBA", (width, height), (0, 0, 0, 0))
draw = ImageDraw.Draw(img)
line_height_px = line_height * font_size
if v_align == "top":
y_text = v_offset
elif v_align == "center":
y_text = (
(height - (line_height_px * len(lines))) // 2
) + v_offset
else:
y_text = (height - (line_height_px * len(lines))) - v_offset
def get_width(line):
if hasattr(font, "getsize"):
return font.getsize(line)[0]
else:
return font.getlength(line)
for line in lines:
line_width = get_width(line)
if h_align == "left":
x_text = h_offset
elif h_align == "center":
x_text = ((width - line_width) // 2) + h_offset
else:
x_text = (width - line_width) - h_offset
text_color = color
if alpha is not None:
text_color = tuple(
list(color[:3]) + [int(alpha * color[3])]
)
draw.text((x_text, y_text), line, fill=text_color, font=font)
y_text += line_height_px
return img
base_img = Image.new("RGBA", (width, height), background)
if whisper_chunks and whisper_chunks.get("chunks"):
frames = []
total_duration = whisper_chunks["chunks"][-1]["timestamp"][1]
frame_count = int(total_duration * fps)
fade_frames = int(fade_duration * fps)
for frame_idx in range(frame_count):
time = frame_idx / fps
frame = base_img.copy()
active_chunks = []
for chunk in whisper_chunks["chunks"]:
start, end = chunk["timestamp"]
if start <= time <= end:
fade_in_alpha = min(
1.0, (time - start) * fps / fade_frames
)
fade_out_alpha = min(
1.0, (end - time) * fps / fade_frames
)
alpha = min(fade_in_alpha, fade_out_alpha)
active_chunks.append((chunk["text"], alpha))
for chunk_text, alpha in active_chunks:
chunk_img = render_text(chunk_text, alpha)
frame = Image.alpha_composite(frame, chunk_img)
frames.append(frame)
frame_tensors = [pil2tensor(frame) for frame in frames]
return (torch.cat(frame_tensors, dim=0),)
text = (
text.encode("ascii", "ignore").decode().strip() if trim else text
)
# Handle word wrapping
if wrap:
wrap_width = (((width / 100) * h_coverage) / font_size) * 2
lines = textwrap.wrap(text, width=wrap_width)
else:
text_img = render_text(text)
result = Image.alpha_composite(base_img, text_img)
return (pil2tensor(result),)
lines = [text]
font = ImageFont.truetype(font_path, size=font_size)
log.debug(f"Lines: {lines}")
img = Image.new("RGBA", (width, height), background)
draw = ImageDraw.Draw(img)
line_height_px = line_height * font_size
# Vertical alignment
if v_align == "top":
y_text = v_offset
elif v_align == "center":
y_text = ((height - (line_height_px * len(lines))) // 2) + v_offset
else: # bottom
y_text = (height - (line_height_px * len(lines))) - v_offset
def get_width(line):
if hasattr(font, "getsize"):
return font.getsize(line)[0]
else:
return font.getlength(line)
# Draw each line of text
for line in lines:
line_width = get_width(line)
# Horizontal alignment
if h_align == "left":
x_text = h_offset
elif h_align == "center":
x_text = ((width - line_width) // 2) + h_offset
else: # right
x_text = (width - line_width) - h_offset
draw.text((x_text, y_text), line, fill=color, font=font)
y_text += line_height_px
return (pil2tensor(img),)
__nodes__ = [
+3 -194
View File
@@ -1,6 +1,5 @@
import io
import json
import re
import urllib.parse
import urllib.request
from math import pi
@@ -472,7 +471,7 @@ class MTB_AnyToString:
class MTB_StringReplace:
"""Basic string replacement with regex support."""
"""Basic string replacement."""
@classmethod
def INPUT_TYPES(cls):
@@ -481,7 +480,6 @@ class MTB_StringReplace:
"string": ("STRING", {"forceInput": True}),
"old": ("STRING", {"default": ""}),
"new": ("STRING", {"default": ""}),
"use_regex": ("BOOLEAN", {"default": False}),
}
}
@@ -489,19 +487,12 @@ class MTB_StringReplace:
RETURN_TYPES = ("STRING",)
CATEGORY = "mtb/string"
def replace_str(self, string: str, old: str, new: str, use_regex: bool):
def replace_str(self, string: str, old: str, new: str):
log.debug(f"Current string: {string}")
log.debug(f"Find string: {old}")
log.debug(f"Replace string: {new}")
log.debug(f"Use regex: {use_regex}")
if use_regex:
try:
string = re.sub(old, new, string)
except re.error as e:
raise ValueError(f"Regex error: {e}") from e
else:
string = string.replace(old, new)
string = string.replace(old, new)
log.debug(f"New string: {string}")
@@ -686,187 +677,6 @@ class MTB_ConcatImages:
return (concatenated,)
class MTB_TensorOps:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"tensor": ("IMAGE",),
"operation": (
[
"multiply",
"divide",
"add",
"subtract",
"power",
"clamp",
"abs",
"log",
"exp",
"convert_dtype",
"normalize_range",
"normalize_per_channel",
],
{"default": "multiply"},
),
"value": (
"FLOAT",
{
"default": 1.0,
"min": -1000000.0,
"max": 1000000.0,
"step": 0.01,
},
),
"source_min": (
"FLOAT",
{
"default": 0.0,
"min": -1000000.0,
"max": 1000000.0,
"step": 0.01,
},
),
"source_max": (
"FLOAT",
{
"default": 1.0,
"min": -1000000.0,
"max": 1000000.0,
"step": 0.01,
},
),
"target_min": (
"FLOAT",
{
"default": 0.0,
"min": -1000000.0,
"max": 1000000.0,
"step": 0.01,
},
),
"target_max": (
"FLOAT",
{
"default": 16.0,
"min": -1000000.0,
"max": 1000000.0,
"step": 0.01,
},
),
"dtype": (
["uint8", "float32", "float16", "bfloat16"],
{"default": "float32"},
),
"use_mean": ("BOOLEAN", {"default": False}),
},
"optional": {
"target_tensor": ("IMAGE",),
},
}
RETURN_TYPES = ("IMAGE",)
FUNCTION = "apply"
CATEGORY = "mtb/tensor_ops"
def apply(
self,
tensor,
operation="multiply",
value=1.0,
source_min=0.0,
source_max=1.0,
target_min=0.0,
target_max=1.0,
dtype="float32",
use_mean=False,
target_tensor=None,
):
log.debug(
f"Input tensor stats: shape={tensor.shape}, dtype={tensor.dtype}, range=[{tensor.min().item():.6f}, {tensor.max().item():.6f}]"
)
if operation == "normalize_per_channel":
if target_tensor is None:
raise ValueError(
"Target tensor required for per-channel normalization"
)
result = tensor.clone()
for c in range(tensor.shape[-1]):
if use_mean:
source_mean = tensor[..., c].mean()
target_mean = target_tensor[..., c].mean()
scale = target_mean / source_mean
result[..., c] = tensor[..., c] * scale
else:
source_min = tensor[..., c].min()
source_max = tensor[..., c].max()
target_min = target_tensor[..., c].min()
target_max = target_tensor[..., c].max()
normalized = (tensor[..., c] - source_min) / (
source_max - source_min
)
result[..., c] = (
normalized * (target_max - target_min) + target_min
)
log.debug(
f"Channel {c} - Scale: source=[{source_min:.6f}, {source_max:.6f}], target=[{target_min:.6f}, {target_max:.6f}]"
)
elif operation == "normalize_range":
if target_tensor is not None:
target_min = target_tensor.min().item()
target_max = target_tensor.max().item()
log.debug(
f"Using target tensor range: [{target_min:.6f}, {target_max:.6f}]"
)
normalized = (tensor - source_min) / (source_max - source_min)
result = normalized * (target_max - target_min) + target_min
elif operation == "convert_dtype":
if dtype == "float32":
result = tensor.float()
elif dtype == "float16":
result = tensor.half()
elif dtype == "bfloat16":
result = tensor.bfloat16()
else:
result = tensor
if operation == "multiply":
result = tensor * value
elif operation == "divide":
result = tensor / value if value != 0 else tensor
elif operation == "add":
result = tensor + value
elif operation == "subtract":
result = tensor - value
elif operation == "power":
result = torch.pow(tensor, value)
elif operation == "clamp":
if target_tensor is not None:
result = torch.clamp(
tensor,
target_tensor.min().item(),
target_tensor.max().item(),
)
else:
result = torch.clamp(tensor, source_min, source_max)
elif operation == "abs":
result = torch.abs(tensor)
elif operation == "log":
result = torch.log(tensor.clamp(min=1e-10))
elif operation == "exp":
result = torch.exp(tensor)
log.debug(
f"Output tensor stats: shape={result.shape}, dtype={result.dtype}, range=[{result.min().item():.6f}, {result.max().item():.6f}]"
)
return (result,)
__nodes__ = [
MTB_StringReplace,
MTB_FitNumber,
@@ -881,5 +691,4 @@ __nodes__ = [
MTB_FloatsToFloat,
MTB_FloatToFloats,
MTB_FloatsToInts,
MTB_TensorOps,
]
+1 -1
View File
@@ -543,7 +543,7 @@ class MTB_ColorCorrect:
adjusted = self.hsv_adjustment(adjusted, hue, saturation, value)
if clamp:
adjusted = torch.clamp(adjusted, 0.0, 1.0)
adjusted = torch.clamp(image, 0.0, 1.0)
result = (
adjusted
+5 -18
View File
@@ -39,13 +39,9 @@ class MTB_StackImages:
f"{'vertically' if vertical else 'horizontally'}"
)
target_device = tensors[0].device
normalized_tensors = [
self.normalize_to_rgba(tensor.to(target_device))
for tensor in tensors
self.normalize_to_rgba(tensor) for tensor in tensors
]
max_batch_size = max(tensor.shape[0] for tensor in normalized_tensors)
normalized_tensors = [
self.duplicate_frames(tensor, max_batch_size)
@@ -171,34 +167,25 @@ class MTB_PickFromBatch:
"image": ("IMAGE",),
"from_direction": (["end", "start"], {"default": "start"}),
"count": ("INT", {"default": 1}),
},
"optional": {
"mask": ("MASK",),
},
}
}
RETURN_TYPES = ("IMAGE", "MASK")
RETURN_TYPES = ("IMAGE",)
FUNCTION = "pick_from_batch"
CATEGORY = "mtb/image utils"
def pick_from_batch(self, image, from_direction, count, mask=None):
def pick_from_batch(self, image, from_direction, count):
batch_size = image.size(0)
# Limit count to the available number of images in the batch
count = min(count, batch_size)
selected_masks = None
if from_direction == "end":
selected_tensors = image[-count:]
if mask is not None:
selected_masks = mask[-count:]
else:
selected_tensors = image[:count]
if mask is not None:
selected_masks = mask[:count]
return (selected_tensors, selected_masks)
return (selected_tensors,)
import folder_paths
+16 -67
View File
@@ -57,21 +57,6 @@ class MTB_TransformImage:
],
{"default": "bilinear"},
),
"stretch_x": (
"FLOAT",
{"default": 1.0, "min": 0.001, "max": 10.0, "step": 0.01},
),
"stretch_y": (
"FLOAT",
{"default": 1.0, "min": 0.001, "max": 10.0, "step": 0.01},
),
"use_normalized": (
"BOOLEAN",
{
"default": False,
"tooltip": "If true, transform values are scaled to image dimensions.",
},
),
},
}
@@ -90,9 +75,6 @@ class MTB_TransformImage:
border_handling="edge",
constant_color=None,
filter_type="nearest",
stretch_x=1.0,
stretch_y=1.0,
use_normalized: bool = False,
):
filter_map = {
"nearest": Image.NEAREST,
@@ -104,21 +86,20 @@ class MTB_TransformImage:
}
resampling_filter = filter_map[filter_type]
_, frame_height, frame_width, _ = image.size()
if use_normalized:
x = float(x) * frame_width
y = float(y) * frame_height
x = int(x)
y = int(y)
angle = int(angle)
log.debug(
f"Zoom: {zoom} | x: {x}, y: {y}, angle: {angle}, shear: {shear} | stretch_x: {stretch_x}, stretch_y: {stretch_y}"
f"Zoom: {zoom} | x: {x}, y: {y}, angle: {angle}, shear: {shear}"
)
if image.size(0) == 0:
return (torch.zeros(0),)
transformed_images = []
frames_count, frame_height, frame_width, frame_channel_count = (
image.size()
)
new_height, new_width = (
int(frame_height * zoom),
@@ -149,55 +130,23 @@ class MTB_TransformImage:
for img in tensor2pil(image):
img = TF.pad(
img,
img, # transformed_frame,
padding=padding,
padding_mode=border_handling,
fill=constant_color or 0,
)
if stretch_x != 1.0 or stretch_y != 1.0:
img = cast(
Image.Image,
TF.affine(
img,
angle=angle,
scale=zoom,
translate=[x, y],
shear=shear,
interpolation=resampling_filter,
),
)
width, height = img.size
center = (width // 2, height // 2)
stretch_x_factor = 1.0 / stretch_x
stretch_y_factor = 1.0 / stretch_y
matrix = [
stretch_x_factor,
0,
center[0] - center[0] * stretch_x_factor,
0,
stretch_y_factor,
center[1] - center[1] * stretch_y_factor,
]
img = img.transform(
img.size, Image.AFFINE, matrix, resampling_filter
)
else:
img = cast(
Image.Image,
TF.affine(
img,
angle=angle,
scale=zoom,
translate=[x, y],
shear=shear,
interpolation=resampling_filter,
),
)
img = cast(
Image.Image,
TF.affine(
img,
angle=angle,
scale=zoom,
translate=[x, y],
shear=shear,
interpolation=resampling_filter,
),
)
left = abs(padding[0])
upper = abs(padding[1])
+458
View File
@@ -0,0 +1,458 @@
import comfy.utils
import torch
import torch.nn.functional as F
from ..log import log
class MTB_SceneCutDetector:
"""Detects scene cuts in a video using various methods (content, histogram, hash, or adaptive)"""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"frames": (
"IMAGE",
{"tooltip": "The frames used for processing"},
),
"method": (
["content", "histogram", "hash", "adaptive"],
{
"default": "histogram",
"tooltip": "only histogram works properly for now",
},
),
"downsample": (
["0.1x", "0.25x", "0.5x", "0.75x", "1.0x"],
{
"default": "0.1x",
"tooltip": "Downsample 'frames' (only for processing)",
},
),
"min_scene_length": (
"INT",
{
"default": 15,
"min": 1,
"max": 1000,
"tooltip": "the minimum number of frames a cut can be",
},
),
# content
"content_threshold": (
"FLOAT",
{"default": 0.1, "min": 0.0, "max": 1.0, "step": 0.001},
),
# histogram
"histogram_threshold": (
"FLOAT",
{"default": 0.20, "min": 0.0, "max": 1.0, "step": 0.001},
),
"histogram_bins": (
"INT",
{"default": 32, "min": 2, "max": 256},
),
# hash
"hash_threshold": (
"FLOAT",
{"default": 0.395, "min": 0.0, "max": 1.0, "step": 0.001},
),
"hash_size": ("INT", {"default": 16, "min": 8, "max": 64}),
# adaptive
"adaptive_threshold": (
"FLOAT",
{"default": 3.0, "min": 0.0, "max": 10.0, "step": 0.001},
),
"window_width": ("INT", {"default": 2, "min": 1, "max": 10}),
"min_content_val": (
"FLOAT",
{"default": 15.0, "min": 0.0, "max": 100.0},
),
},
"optional": {
"original_frames": (
"IMAGE",
{
"tooltip": "If provided the returned list will use these frames."
},
),
},
}
FUNCTION = "detect_cuts"
RETURN_TYPES = ("IMAGE",)
RETURN_NAMES = ("sequences",)
OUTPUT_IS_LIST = (True,)
CATEGORY = "mtb/video"
def detect_cuts(
self,
frames: torch.Tensor,
method: str,
min_scene_length: int,
content_threshold: float = 27.0,
histogram_threshold: float = 0.05,
histogram_bins: int = 64,
hash_threshold: float = 0.395,
hash_size: int = 16,
adaptive_threshold: float = 3.0,
window_width: int = 2,
min_content_val: float = 15.0,
downsample: str = "1.0x",
original_frames: torch.Tensor | None = None,
) -> tuple[list[torch.Tensor]]:
processing_frames = frames
frames_to_split = (
original_frames if original_frames is not None else frames
)
if downsample != "1.0x":
scale = float(downsample.replace("x", ""))
h, w = frames.shape[1:3]
new_h, new_w = int(h * scale), int(w * scale)
processing_frames = F.interpolate(
frames.permute(0, 3, 1, 2), # [B,C,H,W] for interpolate
size=(new_h, new_w),
mode="bilinear",
align_corners=False,
).permute(0, 2, 3, 1) # Back to [B,H,W,C]
cuts = []
if method == "content":
cuts = self.detect_content_cuts(
processing_frames, content_threshold, min_scene_length
)
elif method == "histogram":
cuts = self.detect_histogram_cuts(
processing_frames,
histogram_threshold,
histogram_bins,
min_scene_length,
)
elif method == "hash":
cuts = self.detect_hash_cuts(
processing_frames, hash_threshold, hash_size, min_scene_length
)
elif method == "adaptive":
cuts = self.detect_adaptive_cuts(
processing_frames,
adaptive_threshold,
window_width,
min_content_val,
min_scene_length,
)
# always include end
cuts.append(frames.shape[0])
# split into list
sequences = [
frames_to_split[cuts[i] : cuts[i + 1]]
for i in range(len(cuts) - 1)
]
log.debug(f"Found {len(sequences)} cuts")
return (sequences,)
def detect_content_cuts(
self,
frames: torch.Tensor,
threshold: float,
min_scene_length: int,
) -> list[int]:
"""Content-based cut detection using frame differences"""
num_frames = frames.shape[0]
device = frames.device
cuts = [0]
last_cut = 0
total = (
max(0, (num_frames - min_scene_length) - min_scene_length)
+ num_frames
)
pbar = comfy.utils.ProgressBar(total)
differences = torch.zeros(num_frames - 1, device=device)
for i in range(num_frames - 1):
differences[i] = self.compute_content_difference(
frames[i], frames[i + 1]
)
pbar.update(1)
# temporal smoothing
kernel_size = 3
differences = F.pad(
differences.unsqueeze(0).unsqueeze(0),
((kernel_size - 1) // 2, (kernel_size - 1) // 2),
mode="replicate",
)
differences = F.avg_pool1d(
differences, kernel_size, stride=1
).squeeze()
for i in range(min_scene_length, num_frames - min_scene_length):
pbar.update(1)
if i - last_cut >= min_scene_length and differences[i] > threshold:
cuts.append(i)
last_cut = i
return cuts
def detect_histogram_cuts(
self,
frames: torch.Tensor,
threshold: float,
bins: int,
min_scene_length: int,
) -> list[int]:
"""Histogram-based cut detection"""
num_frames = frames.shape[0]
# device = frames.device
cuts = [0]
last_cut = 0
pbar = comfy.utils.ProgressBar(num_frames)
for i in range(1, num_frames):
pbar.update(1)
if i - last_cut < min_scene_length:
continue
# Convert to YUV and get Y channel
yuv1 = (
0.299 * frames[i - 1, ..., 0]
+ 0.587 * frames[i - 1, ..., 1]
+ 0.114 * frames[i - 1, ..., 2]
)
yuv2 = (
0.299 * frames[i, ..., 0]
+ 0.587 * frames[i, ..., 1]
+ 0.114 * frames[i, ..., 2]
)
# Compute histograms
hist1 = torch.histc(yuv1, bins=bins, min=0, max=1)
hist2 = torch.histc(yuv2, bins=bins, min=0, max=1)
# Normalize histograms
hist1 = hist1 / hist1.sum()
hist2 = hist2 / hist2.sum()
# Compute histogram difference
diff = torch.sum(torch.abs(hist1 - hist2))
if diff > threshold:
cuts.append(i)
last_cut = i
return cuts
def detect_hash_cuts(
self,
frames: torch.Tensor,
threshold: float,
hash_size: int,
min_scene_length: int,
) -> list[int]:
"""Perceptual hash based cut detection"""
num_frames = frames.shape[0]
# device = frames.device
cuts = [0]
last_cut = 0
pbar = comfy.utils.ProgressBar(num_frames)
def compute_frame_hash(frame):
# Convert to grayscale
gray = (
0.299 * frame[..., 0]
+ 0.587 * frame[..., 1]
+ 0.114 * frame[..., 2]
)
gray = F.interpolate(
gray.unsqueeze(0).unsqueeze(0),
size=(hash_size, hash_size),
mode="bilinear",
align_corners=False,
).squeeze()
dct = torch.fft.rfft2(gray)
dct = dct[: hash_size // 2, : hash_size // 2]
return dct > dct.median()
for i in range(1, num_frames):
pbar.update(1)
if i - last_cut < min_scene_length:
continue
hash1 = compute_frame_hash(frames[i - 1])
hash2 = compute_frame_hash(frames[i])
diff = torch.mean((hash1 != hash2).float())
if diff > threshold:
cuts.append(i)
last_cut = i
return cuts
def detect_adaptive_cuts(
self,
frames: torch.Tensor,
adaptive_threshold: float,
window_width: int,
min_content_val: float,
min_scene_length: int,
) -> list[int]:
"""Adaptive threshold based cut detection"""
num_frames = frames.shape[0]
device = frames.device
cuts = [0]
last_cut = 0
total = num_frames + max(0, (num_frames - window_width) - window_width)
pbar = comfy.utils.ProgressBar(total)
content_vals = torch.zeros(num_frames - 1, device=device)
for i in range(num_frames - 1):
content_vals[i] = self.compute_content_difference(
frames[i], frames[i + 1]
)
pbar.update(1)
for i in range(window_width, num_frames - window_width):
pbar.update(1)
if i - last_cut < min_scene_length:
continue
target_score = content_vals[i]
window_scores = content_vals[
i - window_width : i + window_width + 1
]
surrounding_scores = torch.cat(
[
window_scores[:window_width],
window_scores[window_width + 1 :],
]
)
average_score = surrounding_scores.mean()
if average_score > 1e-5:
adaptive_ratio = min(target_score / average_score, 255.0)
elif target_score >= min_content_val:
adaptive_ratio = 255.0
else:
adaptive_ratio = 0.0
if (
adaptive_ratio >= adaptive_threshold
and target_score >= min_content_val
):
cuts.append(i)
last_cut = i
return cuts
def compute_content_difference(
self, frame1: torch.Tensor, frame2: torch.Tensor
) -> torch.Tensor:
"""
Computes content difference between frames using multiple metrics:
- Structural similarity
- Color distribution changes
- Edge differences
"""
device = frame1.device
if frame1.dtype != torch.float32:
frame1 = frame1.float()
frame2 = frame2.float()
def ssim(x, y):
c1, c2 = 0.01**2, 0.03**2
mu_x = F.avg_pool2d(x, kernel_size=11, stride=1, padding=5)
mu_y = F.avg_pool2d(y, kernel_size=11, stride=1, padding=5)
mu_x_sq = mu_x.pow(2)
mu_y_sq = mu_y.pow(2)
mu_xy = mu_x * mu_y
sigma_x = (
F.avg_pool2d(x.pow(2), kernel_size=11, stride=1, padding=5)
- mu_x_sq
)
sigma_y = (
F.avg_pool2d(y.pow(2), kernel_size=11, stride=1, padding=5)
- mu_y_sq
)
sigma_xy = (
F.avg_pool2d(x * y, kernel_size=11, stride=1, padding=5)
- mu_xy
)
ssim_map = ((2 * mu_xy + c1) * (2 * sigma_xy + c2)) / (
(mu_x_sq + mu_y_sq + c1) * (sigma_x + sigma_y + c2)
)
return 1 - ssim_map.mean()
def color_change(x, y):
bins = 64
x_hist = torch.stack(
[
torch.histc(x[..., i], bins=bins, min=0, max=1)
for i in range(3)
]
)
y_hist = torch.stack(
[
torch.histc(y[..., i], bins=bins, min=0, max=1)
for i in range(3)
]
)
x_hist = x_hist / x_hist.sum(dim=1, keepdim=True).clamp(min=1e-6)
y_hist = y_hist / y_hist.sum(dim=1, keepdim=True).clamp(min=1e-6)
return torch.mean(torch.abs(x_hist - y_hist))
def edge_change(x, y):
sobel_x = torch.tensor(
[[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]], device=device
).float()
sobel_y = torch.tensor(
[[-1, -2, -1], [0, 0, 0], [1, 2, 1]], device=device
).float()
def detect_edges(img):
gray = (
0.2989 * img[..., 0]
+ 0.5870 * img[..., 1]
+ 0.1140 * img[..., 2]
)
gray = gray.unsqueeze(0).unsqueeze(0)
gx = F.conv2d(gray, sobel_x.view(1, 1, 3, 3), padding=1)
gy = F.conv2d(gray, sobel_y.view(1, 1, 3, 3), padding=1)
return torch.sqrt(gx.pow(2) + gy.pow(2)).squeeze()
edges1 = detect_edges(frame1)
edges2 = detect_edges(frame2)
return torch.mean(torch.abs(edges1 - edges2))
struct_diff = ssim(frame1, frame2)
color_diff = color_change(frame1, frame2)
edge_diff = edge_change(frame1, frame2)
weights = torch.tensor([0.4, 0.3, 0.3], device=device)
combined_diff = (
weights[0] * struct_diff
+ weights[1] * color_diff
+ weights[2] * edge_diff
)
return combined_diff
__nodes__ = [MTB_SceneCutDetector]
+7 -8
View File
@@ -4,9 +4,9 @@ build-backend = "setuptools.build_meta"
[project]
name = "comfy-mtb"
version = "0.3.0"
version = "0.2.1"
description = "Animation oriented nodes pack for ComfyUI."
license = { text = "MIT" }
license = "MIT"
readme = "README.md"
# repository = ""
# url = "https://github.com/melMass/comfy_mtb"
@@ -63,7 +63,7 @@ DisplayName = "comfy-mtb"
Icon = "https://avatars.githubusercontent.com/u/7041726?v=4"
[tool.bumpversion]
current_version = "0.3.0"
current_version = "0.2.1"
parse = "(?P<major>\\d+)\\.(?P<minor>\\d+)\\.(?P<patch>\\d+)"
serialize = ["{major}.{minor}.{patch}"]
search = "{current_version}"
@@ -150,9 +150,6 @@ show_contexts = true
[tool.ruff]
line-length = 79
extend-exclude = ["./docs/conf.py", "notebooks", "stubs"]
[tool.ruff.lint]
select = ["A", "B", "C", "D", "E", "F", "FBT", "I", "N", "S", "SIM", "UP", "W"]
# NOTE:
# D102 - undocumented-public-method (noisy)
@@ -160,14 +157,16 @@ select = ["A", "B", "C", "D", "E", "F", "FBT", "I", "N", "S", "SIM", "UP", "W"]
# D100 - undocumented-public-module (noisy)
# N802 - invalid-function-name (forced by comfy's arch)
ignore = ["D103", "D102", "D100", "N802"]
# exclude auto generated file
extend-exclude = ["./docs/conf.py"]
[tool.ruff.lint.per-file-ignores]
[tool.ruff.per-file-ignores]
# imported but unused
"__init__.py" = ["F401"]
# use of assert detected
"tests/*" = ["S101"]
[tool.ruff.lint.pydocstyle]
[tool.ruff.pydocstyle]
convention = "numpy"
[tool.mypy]
-1
View File
@@ -9,4 +9,3 @@ rich_argparse
matplotlib
pillow
cachetools
transformers
-47
View File
@@ -16,50 +16,3 @@
* @typedef {import("./shared.d.ts").INodeOutputSlot} INodeOutputSlot
*/
/**
* @typedef {Object} ResultItem
* @property {string} [filename] - The filename of the item.
* @property {string} [subfolder] - The subfolder of the item.
* @property {string} [type] - The type of the item.
*/
/**
* @typedef {Object} Outputs
* @property {ResultItem[]} [audio] - Audio result items.
* @property {ResultItem[]} [images] - Image result items.
* @property {ResultItem[]} [animated] - Animated result items.
*/
/**
* @typedef {Record<string, Outputs>} TaskOutput
* - A record mapping Node IDs to their Outputs.
*/
/**
* @typedef {Array} TaskPrompt
* @property {QueueIndex} [0] - The queue index.
* @property {PromptId} [1] - The unique prompt ID.
* @property {PromptInputs} [2] - The prompt inputs.
* @property {ExtraData} [3] - Extra data.
* @property {OutputsToExecute} [4] - The outputs to execute.
*/
/**
* @typedef {Object} HistoryTaskItem
* @property {'History'} taskType - The type of task.
* @property {TaskPrompt} prompt - The task prompt.
* @property {Status} [status] - The status of the task.
* @property {TaskOutput} outputs - The task outputs.
* @property {TaskMeta} [meta] - Optional task metadata.
*/
/**
* @typedef {Object} ExecInfo
* @property {number} queue_remaining - The number of items remaining in the queue.
*/
/**
* @typedef {Object} StatusWsMessageStatus
* @property {ExecInfo} exec_info - Execution information.
*/
+2 -58
View File
@@ -15,9 +15,7 @@ from enum import Enum
from functools import reduce
from pathlib import Path
from typing import TypeVar
from urllib.parse import urlparse
import comfy.utils
import folder_paths
import numpy as np
import numpy.typing as npt
@@ -546,7 +544,9 @@ PIL_FILTER_MAP = {
# region TENSOR Utilities
def to_numpy(image: torch.Tensor) -> npt.NDArray[np.uint8]:
"""Converts a tensor to a ndarray with proper scaling and type conversion."""
log.debug(f"Converting tensor to numpy array with shape {image.shape}")
np_array = np.clip(255.0 * image.cpu().numpy(), 0, 255).astype(np.uint8)
log.debug(f"Numpy array shape after conversion: {np_array.shape}")
return np_array
@@ -858,62 +858,6 @@ def tiles_split(img, tile_size, stride_size):
# region MODEL Utilities
def download_model(model_url: str, destination: str):
if isinstance(model_url, list):
for url in model_url:
download_model(url, destination)
return
filename = Path(urlparse(model_url).path).name
if "drive.google.com" in model_url:
try:
import gdown
except ImportError:
log.info("Installing gdown")
subprocess.check_call(
[
sys.executable,
"-m",
"pip",
"install",
"gdown",
]
)
import gdown
if "/folders/" in model_url:
# download folder
try:
gdown.download_folder(
model_url, output=destination, resume=True
)
except TypeError:
gdown.download_folder(model_url, output=destination)
return
# download from google drive
gdown.download(model_url, destination, quiet=False, resume=True)
return True
response = requests.get(model_url, stream=True)
total_size = int(response.headers.get("content-length", 0))
destination_path = get_model_path(destination, filename)
destination_path.parent.mkdir(exist_ok=True)
pbar = comfy.utils.ProgressBar(total_size)
with open(destination_path, "wb") as file:
for data in response.iter_content(chunk_size=4096):
file.write(data)
pbar.update(len(data))
log.info(
f"Downloaded model from {model_url} to {destination_path}",
)
def download_antelopev2():
antelopev2_url = (
"https://drive.google.com/uc?id=18wEUfMNohBJ4K3Ly5wpTejPfDzp-8fI8"
-69
View File
@@ -12,9 +12,6 @@
import { app } from '../../scripts/app.js'
import { api } from '../../scripts/api.js'
if (!window.MTB) {
window.MTB = {}
}
// #region base utils
// - crude uuid
@@ -279,10 +276,6 @@ export const getNamedWidget = (node, ...names) => {
* @returns {{to:LGraphNode, from:LGraphNode, type:'error' | 'incoming' | 'outgoing'}}
*/
export const nodesFromLink = (node, link) => {
if (typeof link === 'number') {
console.log('Resolving link from id', link)
link = app.graph.links[link]
}
const fromNode = app.graph.getNodeById(link.origin_id)
const toNode = app.graph.getNodeById(link.target_id)
@@ -1075,66 +1068,6 @@ export const addDocumentation = (
// #endregion
// #region canvas / drawing
// calculate convex hull (Graham)
export function getConvexHull(points) {
if (points.length < 3) return points
// find the bottommost point (and leftmost if tied)
let bottom = 0
for (let i = 1; i < points.length; i++) {
if (
points[i][1] < points[bottom][1] ||
(points[i][1] === points[bottom][1] && points[i][0] < points[bottom][0])
) {
bottom = i
}
}
// swap bottom point to first position
;[points[0], points[bottom]] = [points[bottom], points[0]]
// sort points by polar angle with respect to base point
const basePoint = points[0]
points.sort((a, b) => {
if (a === basePoint) return -1
if (b === basePoint) return 1
const angleA = Math.atan2(a[1] - basePoint[1], a[0] - basePoint[0])
const angleB = Math.atan2(b[1] - basePoint[1], b[0] - basePoint[0])
if (angleA < angleB) return -1
if (angleA > angleB) return 1
// if angles are equal, sort by distance
const distA = (a[0] - basePoint[0]) ** 2 + (a[1] - basePoint[1]) ** 2
const distB = (b[0] - basePoint[0]) ** 2 + (b[1] - basePoint[1]) ** 2
return distA - distB
})
// build convex hull
const stack = [points[0], points[1]]
for (let i = 2; i < points.length; i++) {
while (
stack.length > 1 &&
!isLeftTurn(stack[stack.length - 2], stack[stack.length - 1], points[i])
) {
stack.pop()
}
stack.push(points[i])
}
return stack
}
function isLeftTurn(p1, p2, p3) {
return (
(p2[0] - p1[0]) * (p3[1] - p1[1]) - (p2[1] - p1[1]) * (p3[0] - p1[0]) > 0
)
}
// #endregion
// #region node extensions
/**
@@ -1209,8 +1142,6 @@ export const runAction = async (name, ...args) => {
const res = await req.json()
return res.result
}
window.MTB.run = runAction
export const getServerInfo = async () => {
const res = await api.fetchApi('/mtb/server-info')
return await res.json()
+47 -106
View File
@@ -11,9 +11,13 @@
/// <reference path="../types/typedefs.js" />
import { app } from '../../scripts/app.js'
import * as shared from './comfy_shared.js'
import { MtbWidgets } from './mtb_widgets.js'
import * as mtb_ui from './mtb_ui.js'
// TODO: respect inputs order...
function escapeHtml(unsafe) {
return unsafe
.replace(/&/g, '&amp;')
@@ -22,54 +26,6 @@ function escapeHtml(unsafe) {
.replace(/"/g, '&quot;')
.replace(/'/g, '&#039;')
}
function createDebugSection(title) {
const section = mtb_ui.makeElement('div', {
margin: '8px 0',
padding: '8px',
borderRadius: '4px',
backgroundColor: 'rgba(0,0,0,0.2)'
})
const header = mtb_ui.makeElement('h3', {
margin: '0 0 8px 0',
padding: '4px 0',
borderBottom: '1px solid rgba(255,255,255,0.1)',
fontSize: '14px',
fontWeight: 'bold',
color: '#9f9'
})
header.textContent = title
section.appendChild(header)
return section
}
function createDebugContent(content, type) {
const wrapper = mtb_ui.makeElement('div', {
margin: '4px 0'
})
if (type === 'text') {
const text = mtb_ui.makeElement('p', {
margin: '2px 0',
fontFamily: 'monospace',
whiteSpace: 'pre-wrap'
})
text.innerHTML = content
wrapper.appendChild(text)
} else if (type === 'image') {
const img = mtb_ui.makeElement('img', {
width: '100%',
borderRadius: '2px'
})
img.src = content
wrapper.appendChild(img)
}
return wrapper
}
app.registerExtension({
name: 'mtb.Debug',
@@ -128,78 +84,63 @@ app.registerExtension({
onExecuted?.apply(this, args)
const [data, ..._rest] = args
const prefix = 'anything_'
if (this.widgets) {
let tgt_len = this.widgets.length
for (let i = 0; i < this.widgets.length; i++) {
if (
this.widgets[i].name !== 'output_to_console' &&
this.widgets[i].name !== 'as_detailed_types'
) {
if (this.widgets[i].name !== 'output_to_console') {
this.widgets[i].onRemove?.()
this.widgets[i].onRemoved?.()
tgt_len -= 1
}
}
this.widgets.length = tgt_len
this.widgets.length = 1
}
const inputData = {}
const uiData = data.ui || data
if (uiData.items) {
uiData.items.forEach(item => {
const inputName = item.input
if (!inputData[inputName]) {
inputData[inputName] = { text: [], b64_images: [] }
}
if (item.text) {
inputData[inputName].text.push(...item.text)
}
if (item.b64_images) {
inputData[inputName].b64_images.push(...item.b64_images)
}
})
}
let widgetI = 1
for (const [inputName, content] of Object.entries(inputData)) {
if (content.text.length === 0 && content.b64_images.length === 0) {
continue
// console.log(message)
if (data.text) {
for (const txt of data.text) {
const textDom = mtb_ui.makeElement('p', { fontFamily: 'monospace' })
textDom.innerHTML = txt
this.addDOMWidget(
`${prefix}_${widgetI}`,
'CUSTOM_TEXT',
textDom,
{},
)
widgetI++
}
const section = createDebugSection(inputName)
if (content.text.length > 0) {
content.text.forEach(text => {
section.appendChild(createDebugContent(text, 'text'))
})
}
if (content.b64_images.length > 0) {
content.b64_images.forEach(img => {
section.appendChild(createDebugContent(img, 'image'))
})
}
this.addDOMWidget(
`debug_section_${widgetI}`,
'CUSTOM',
section,
{}
)
widgetI++
}
if (data.b64_images) {
for (const img of data.b64_images) {
const imgDom = mtb_ui.makeElement('img', { width: '100%' })
imgDom.src = img
this.addDOMWidget(
`${prefix}_${widgetI}`,
'CUSTOM_IMG_B64',
mtb_ui.wrapElement(imgDom, {
overflow: 'hidden',
}),
{},
)
widgetI++
}
}
// this.setSize(this.computeSize())
this.onRemoved = function () {
for (const widget of this.widgets) {
if (widget.canvas) {
widget.canvas.remove()
// When removing this node we need to remove the input from the DOM
for (const y in this.widgets) {
if (this.widgets[y].canvas) {
this.widgets[y].canvas.remove()
}
widget.onRemoved?.()
widget.onRemove?.()
shared.cleanupNode(this)
this.widgets[y].onRemoved?.()
this.widgets[y].onRemove?.()
}
shared.cleanupNode(this)
}
}
}
+164 -541
View File
@@ -1,8 +1,5 @@
/// <reference path="../types/typedefs.js" />
import { app } from '../../scripts/app.js'
import { api } from '../../scripts/api.js'
import { infoLogger, successLogger, errorLogger } from './comfy_shared.js'
import * as shared from './comfy_shared.js'
@@ -13,470 +10,122 @@ import {
makeSelect,
makeSlider,
renderSidebar,
ContextMenu,
} from './mtb_ui.js'
let currentAbortController = null
/** cursor/offset of where we are at */
const offset = 0
/** width of the images in the grid */
let currentWidth = 200
let currentMode = 'input'
let subfolder = ''
let currentSort = 'None'
let clientOnce = false
/** reference to the dom element receiving the images */
let imgGrid = undefined
/** currently loaded image (as object urls) */
let loaded_images = undefined
/**
* stores the user's full local path to input/output directory
* This is then used to feed VHS Load Image (from path)
*/
let userDirectories = undefined
// const IMAGE_NODES = ['LoadImage', 'VHS_LoadImagePath']
const IMAGE_NODES = ['LoadImage', 'VHS_LoadImagePath']
const VIDEO_NODES = ['VHS_LoadVideo']
const PROCESSED_PROMPT_IDS = new Set()
let contextMenu = undefined
function debounce(func, wait) {
let timeout
return function executedFunction(...args) {
const later = () => {
infoLogger('Debouncing method')
clearTimeout(timeout)
func(...args)
}
clearTimeout(timeout)
timeout = setTimeout(later, wait)
}
}
const debouncedGetUrls = async (ms = 250) => {
if (loaded_images === undefined) {
return await getUrls(subfolder)
}
debounce(async (subfolder) => {
const urls = await getUrls(subfolder)
infoLogger('Loaded URLs (debounced): ', urls)
if (urls) {
loaded_images = await getImgsFromUrls(urls, imgGrid)
infoLogger('Loaded Images (debounced): ', loaded_images)
}
}, ms)
return loaded_images
}
/** Callback on clicking an image in the grid */
const updateImage = (node, image) => {
switch (node.type) {
case 'LoadImage': {
if (subfolder && subfolder !== '') {
app.extensionManager.toast.add({
severity: 'warn',
summary: 'Subfolder not supported',
detail: "The LoadImage node doesn't support subfolders",
life: 5000,
})
return
if (IMAGE_NODES.includes(node.type)) {
const w = node.widgets?.find((w) => w.name === 'image')
if (w) {
w.value = image
w.callback()
}
} else if (VIDEO_NODES.includes(node.type)) {
const w = node.widgets?.find((w) => w.name === 'video')
if (w) {
node.updateParameters({ filename: image }, true)
}
} else {
console.warn('No method to update', node.type)
}
}
const getImgsFromUrls = (urls, target) => {
const imgs = []
if (urls === undefined) {
return imgs
}
const elem = currentMode === 'video' ? 'video' : 'img'
for (const [key, url] of Object.entries(urls)) {
const a = makeElement(elem)
a.src = url
a.width = currentWidth
if (currentMode === 'input') {
a.onclick = (_e) => {
if (subfolder !== '') {
app.extensionManager.toast.add({
severity: 'warn',
summary: 'Subfolder not supported',
detail: "The LoadImage node doesn't support subfolders",
life: 5000,
})
return
}
const selected = app.canvas.selected_nodes
if (selected && Object.keys(selected).length === 0) {
app.extensionManager.toast.add({
severity: 'warn',
summary: 'No node selected!',
detail:
'For now the only action when clicking images in the sidebar is to set the image on all selected LoadImage nodes.',
life: 5000,
})
return
}
for (const [_id, node] of Object.entries(app.canvas.selected_nodes)) {
updateImage(node, key)
}
}
if (currentMode === 'output') {
} else if (currentMode === 'output') {
a.onclick = (_e) => {
// window.MTB?.notify?.("Output import isn't supported yet...", 5000)
if (subfolder !== '') {
app.extensionManager.toast.add({
severity: 'warn',
summary: 'Subfolder not supported',
detail: "The LoadImage node doesn't support subfolders",
life: 5000,
})
return
}
app.extensionManager.toast.add({
severity: 'warn',
summary: 'Outputs not supported',
detail:
"The LoadImage node doesn't support loading outputs, use VHS Load Image Path and I'll resolve the full path.",
'For now only inputs can be clicked to load the image on the active LoadImage node.',
life: 5000,
})
return
}
// if (IMAGE_NODES.includes(node.type)) {
const w = node.widgets?.find((w) => w.name === 'image')
if (w) {
w.value = image
w.callback()
}
//}
break
}
case 'VHS_LoadImagePath': {
let value = image
} else {
a.autoplay = true
if (!userDirectories?.output) {
app.extensionManager.toast.add({
severity: 'warn',
summary: 'User output directory not resolved',
detail: "We couldn't resolve the image full path.",
life: 5000,
})
return
}
if (subfolder && subfolder !== '') {
value = `${subfolder}/${image}`
}
value = `${userDirectories.output}/${value}`
const w = node.widgets?.find((w) => w.name === 'image')
if (w) {
console.log(w)
w.value = value
// TODO: VHS needs explicity value passsed here
w.callback(value)
}
break
}
case VIDEO_NODES.includes(node.type): {
const w = node.widgets?.find((w) => w.name === 'video')
if (w) {
node.updateParameters({ filename: image }, true)
}
break
}
default: {
console.warn('No method to update', node.type)
}
}
}
/**
* Converts a result item to a request url.
* @param {ResultItem} resultItem
* @returns {string} - The request URL.
*/
const resultItemToQuery = (resultItem) =>
[
`/mtb/view?filename=${resultItem.filename}`,
`width=512`,
`type=${resultItem.type}`,
`subfolder=${resultItem.subfolder}`,
`preview=`,
].join('&')
/**
* Retrieves the unique prompt ID from a history task item.
* @param {HistoryTaskItem} historyTaskItem
* @returns {string} - The prompt ID.
*/
const getPromptId = (historyTaskItem) => `${historyTaskItem.prompt[1]}`
/**
* Process and return any new/unseen outputs from the most recent history item.
* @param {HistoryTaskItem} mostRecentTask - The most recent history task item.
* @returns {Object<string, string>} - A map of task outputs URLs.
*/
const getNewOutputUrls = (mostRecentTask) => {
if (!mostRecentTask) return
const promptId = getPromptId(mostRecentTask)
if (PROCESSED_PROMPT_IDS.has(promptId)) return
const urls = {}
for (const nodeOutputs of Object.values(mostRecentTask.outputs)) {
const { images, audio, animated } = nodeOutputs
if (images) {
const imageOutputs = Object.values(nodeOutputs.images)
imageOutputs.forEach(
(resultItem) =>
(urls[resultItem.filename] = resultItemToQuery(resultItem))
)
}
// Can process `animated` and `audio` outputs here.
}
const foundNewOutputs = Object.keys(urls).length > 0
if (!foundNewOutputs) return null
PROCESSED_PROMPT_IDS.add(promptId)
return urls
}
/** Fetch history and update the grid with any new ouput images. */
const updateOutputsGrid = async () => {
try {
const history = await api.getHistory(/** maxSize: */ 1)
const mostRcentTask = history.History[0]
const newUrls = getNewOutputUrls(mostRcentTask)
if (newUrls) {
const imgGrid = document.querySelector('.mtb_img_grid')
getImgsFromUrls(newUrls, imgGrid, { prepend: true })
}
} catch (error) {
console.error('Error fetching history:', error)
}
}
const getImgsFromUrls = (urls, target, options = { prepend: false }) => {
if (currentAbortController) {
currentAbortController.abort()
}
infoLogger('getting images from urls', urls)
currentAbortController = new AbortController()
const { signal } = currentAbortController
const imgs = []
if (!urls) return imgs
const loadingIndicator = document.createElement('div')
loadingIndicator.className = 'mtb-loading-indicator'
if (target) target.appendChild(loadingIndicator)
const totalImages = Object.keys(urls).length
let loadedCount = 0
const updateLoadingStatus = () => {
loadingIndicator.textContent = `Loaded ${loadedCount} of ${totalImages} images`
}
updateLoadingStatus()
try {
const loadImage = async (key, url) => {
try {
const response = await fetch(url, { signal })
if (!response.ok) {
console.warn(`Failed to fetch ${key}: ${response.status}`)
return null
}
// throw new Error(`HTTP error! status: ${response.status}`)
const blob = await response.blob()
const imgUrl = URL.createObjectURL(blob)
const elem = makeElement(currentMode === 'video' ? 'video' : 'img')
elem.src = imgUrl
elem.width = currentWidth
// cleanup
elem.onload = () => URL.revokeObjectURL(imgUrl)
elem.onerror = () => URL.revokeObjectURL(imgUrl)
// Add click handler for input mode
// if (currentMode === 'input') {
// elem.onclick = (_e) => {
// Your existing click handler code
// }
// }
// Add context menu
elem.addEventListener('contextmenu', (e) => {
e.preventDefault()
const contextMenuItems = [
{
label: 'Add Node with Image',
icon: '🖼',
action: () => {
const node = app.graph.createNode('LoadImage')
updateImage(node, key)
},
},
{
label: 'Load Workflow from Image',
icon: '📋',
action: async () => {
try {
const response = await fetch(url)
const data = await response.blob()
// Assuming you have a function to extract workflow from image metadata
const workflow = await extractWorkflowFromImage(data)
if (workflow) {
app.loadGraphData(workflow)
}
} catch (error) {
app.extensionManager.toast.add({
severity: 'error',
summary: 'Error',
detail: 'Failed to load workflow from image',
life: 3000,
})
}
},
},
{
label: 'View Full Image',
icon: '🔍',
action: () => {
window.open(url, '_blank')
},
},
]
contextMenu.show(e.pageX, e.pageY, contextMenuItems, {
elem,
key,
url,
a.muted = true
a.loop = true
a.onclick = (_e) => {
const selected = app.canvas.selected_nodes
if (selected && Object.keys(selected).length === 0) {
app.extensionManager.toast.add({
severity: 'warn',
summary: 'No node selected!',
detail:
"For now the only action when clicking videos in the sidebar is to set the video on all selected 'Load Video (Upload)' nodes.",
life: 5000,
})
})
elem.onclick = (_e) => {
const selected = app.canvas.selected_nodes
if (!selected || Object.keys(selected).length === 0) {
app.extensionManager.toast.add({
severity: 'warn',
summary: 'No node selected!',
detail: 'Please select a node first.',
life: 5000,
})
return
}
for (const [_id, node] of Object.entries(selected)) {
updateImage(node, key)
}
return
}
loadedCount++
updateLoadingStatus()
return elem
} catch (error) {
if (error.name === 'AbortError') {
console.log('Fetch aborted')
return null
}
console.error('Error loading image:', error)
return null
}
}
const BATCH_SIZE = 20
for (let i = 0; i < Object.entries(urls).length; i += BATCH_SIZE) {
const batch = Object.entries(urls).slice(i, i + BATCH_SIZE)
const loadedImages = await Promise.all(
batch.map(([key, url]) => loadImage(key, url)),
)
const validImages = loadedImages.filter((img) => img !== null)
imgs.push(...validImages)
if (target) {
target.append(...validImages)
}
}
return imgs
// return
// const elem = currentMode === 'video' ? 'video' : 'img'
for (const [key, url] of Object.entries(urls)) {
const a = makeElement(elem)
a.src = url
a.width = currentWidth
const selected = app.canvas.selected_nodes
if (currentMode === 'input') {
a.onclick = (_e) => {
// if (subfolder !== '') {
// app.extensionManager.toast.add({
// severity: 'warn',
// summary: 'Subfolder not supported',
// detail: "The LoadImage node doesn't support subfolders",
// life: 5000,
// })
// return
// }
if (selected && Object.keys(selected).length === 0) {
app.extensionManager.toast.add({
severity: 'warn',
summary: 'No node selected!',
detail:
'For now the only action when clicking images in the sidebar is to set the image on all selected LoadImage nodes.',
life: 5000,
})
return
}
for (const [_id, node] of Object.entries(app.canvas.selected_nodes)) {
updateImage(node, key)
}
}
} else if (currentMode === 'output') {
a.onclick = (_e) => {
if (selected && Object.keys(selected).length === 0) {
return
}
for (const [_id, node] of Object.entries(app.canvas.selected_nodes)) {
updateImage(node, key)
}
// window.MTB?.notify?.("Output import isn't supported yet...", 5000)
// if (subfolder !== '') {
// app.extensionManager.toast.add({
// severity: 'warn',
// summary: 'Subfolder not supported',
// detail: "The LoadImage node doesn't support subfolders",
// life: 5000,
// })
// return
// }
//
// app.extensionManager.toast.add({
// severity: 'warn',
// summary: 'Outputs not supported',
// detail:
// 'For now only inputs can be clicked to load the image on the active LoadImage node.',
// life: 5000,
// })
}
} else {
a.autoplay = true
a.muted = true
a.loop = true
a.onclick = (_e) => {
const selected = app.canvas.selected_nodes
if (selected && Object.keys(selected).length === 0) {
app.extensionManager.toast.add({
severity: 'warn',
summary: 'No node selected!',
detail:
"For now the only action when clicking videos in the sidebar is to set the video on all selected 'Load Video (Upload)' nodes.",
life: 5000,
})
return
}
for (const [_id, node] of Object.entries(app.canvas.selected_nodes)) {
updateImage(node, key)
}
for (const [_id, node] of Object.entries(app.canvas.selected_nodes)) {
updateImage(node, key)
}
}
imgs.push(a)
}
if (target !== undefined) {
if (options.prepend) target.prepend(...imgs)
else target.append(...imgs)
}
return imgs
} finally {
// Keep loading indicator visible for a moment after completion
setTimeout(() => {
if (target && loadingIndicator.parentNode === target) {
loadingIndicator.remove()
}
}, 2000)
imgs.push(a)
}
}
// Helper function to extract workflow from image metadata
async function extractWorkflowFromImage(blob) {
// Implementation depends on how the workflow data is stored in the image
// This is just a placeholder
try {
// You might need to use ExifReader or similar library to extract metadata
return null
} catch (error) {
console.error('Failed to extract workflow:', error)
return null
if (target !== undefined) {
target.append(...imgs)
}
return imgs
}
const getModes = async () => {
@@ -485,7 +134,7 @@ const getModes = async () => {
}
const getUrls = async (subfolder) => {
const count = (await api.getSetting('mtb.io-sidebar.count')) || 1000
console.debug('Sidebar count', count)
console.log('Sidebar count', count)
if (currentMode === 'video') {
const output = await shared.runAction(
'getUserVideos',
@@ -508,99 +157,6 @@ const getUrls = async (subfolder) => {
return output || {}
}
const build_ui = async (el) => {
if (el.parentNode) {
el.parentNode.style.overflowY = 'clip'
}
const allModes = await getModes()
const input_modes = allModes.input.map((m) => `input - ${m}`)
const output_modes = allModes.output.map((m) => `output - ${m}`)
if (!userDirectories) {
userDirectories = {
input: allModes.input_root,
output: allModes.output_root,
}
infoLogger('User directories', userDirectories)
}
// const urls = await getUrls()
// const urls = await debouncedGetUrls(subfolder)
const cont = makeElement('div.mtb_sidebar')
contextMenu = new ContextMenu(cont)
imgGrid = makeElement('div.mtb_img_grid')
const selector = makeSelect(
['input', 'output', 'video', ...output_modes, ...input_modes],
currentMode,
)
selector.addEventListener('change', async (e) => {
let newMode = e.target.value
let changed = false
let newSub = ''
if (newMode !== 'input' && newMode !== 'output') {
if (newMode.startsWith('input - ')) {
newSub = newMode.replace('input - ', '')
newMode = 'input'
} else if (newMode.startsWith('output - ')) {
newSub = newMode.replace('output - ', '')
newMode = 'output'
}
}
changed = newMode !== currentMode || newSub !== subfolder
currentMode = newMode
subfolder = newSub
if (changed) {
imgGrid.innerHTML = ''
// const urls = await getUrls(subfolder)
debouncedGetUrls(subfolder)
// if (urls) {
// loaded_images = getImgsFromUrls(urls, imgGrid)
// }
}
})
const imgTools = makeElement('div.mtb_tools')
const orderSelect = makeSelect(
['None', 'Modified', 'Modified-Reverse', 'Name', 'Name-Reverse'],
currentSort,
)
orderSelect.addEventListener('change', async (e) => {
const newSort = e.target.value
const changed = newSort !== currentSort
currentSort = newSort
if (changed) {
imgGrid.innerHTML = ''
// const urls = await getUrls(subfolder)
// const urls = debouncedGetUrls(subfolder)
// const urls = await getUrls(subfolder)
debouncedGetUrls(subfolder)
// if (urls) {
// loaded_images = getImgsFromUrls(urls, imgGrid)
// }
}
})
const sizeSlider = makeSlider(64, 1024, currentWidth, 1)
imgTools.appendChild(orderSelect)
imgTools.appendChild(sizeSlider)
loaded_images = getImgsFromUrls(urls, imgGrid)
// infoLogger({ loaded_images })
sizeSlider.addEventListener('input', (e) => {
currentWidth = e.target.value
for (const img of loaded_images) {
img.style.width = `${e.target.value}px`
}
})
handle = renderSidebar(el, cont, [selector, imgGrid, imgTools])
}
//NOTE: do not load if using the old ui
if (window?.__COMFYUI_FRONTEND_VERSION__) {
// NOTE: removed this for now since I'm not actually exposing anything a client
@@ -620,8 +176,8 @@ if (window?.__COMFYUI_FRONTEND_VERSION__) {
// },
init: () => {
let handle
// const version = window?.__COMFYUI_FRONTEND_VERSION__
// console.log(`%c ${version}`, 'background: orange; color: white;')
const version = window?.__COMFYUI_FRONTEND_VERSION__
console.log(`%c ${version}`, 'background: orange; color: white;')
ensureMTBStyles()
@@ -693,19 +249,86 @@ if (window?.__COMFYUI_FRONTEND_VERSION__) {
handle = undefined
}
if (!loaded_images) {
await build_ui(el)
if (el.parentNode) {
el.parentNode.style.overflowY = 'clip'
}
app.api.addEventListener('status', async () => {
if (currentMode !== 'output') return
updateOutputsGrid()
const allModes = await getModes()
const input_modes = allModes.input.map((m) => `input - ${m}`)
const output_modes = allModes.output.map((m) => `output - ${m}`)
const urls = await getUrls()
let imgs = {}
const cont = makeElement('div.mtb_sidebar')
const imgGrid = makeElement('div.mtb_img_grid')
const selector = makeSelect(
['input', 'output', 'video', ...output_modes, ...input_modes],
currentMode,
)
selector.addEventListener('change', async (e) => {
let newMode = e.target.value
let changed = false
let newSub = ''
if (newMode !== 'input' && newMode !== 'output') {
if (newMode.startsWith('input - ')) {
newSub = newMode.replace('input - ', '')
newMode = 'input'
} else if (newMode.startsWith('output - ')) {
newSub = newMode.replace('output - ', '')
newMode = 'output'
}
}
changed = newMode !== currentMode || newSub !== subfolder
currentMode = newMode
subfolder = newSub
if (changed) {
imgGrid.innerHTML = ''
const urls = await getUrls(subfolder)
if (urls) {
imgs = getImgsFromUrls(urls, imgGrid)
}
}
})
const imgTools = makeElement('div.mtb_tools')
const orderSelect = makeSelect(
['None', 'Modified', 'Modified-Reverse', 'Name', 'Name-Reverse'],
currentSort,
)
orderSelect.addEventListener('change', async (e) => {
const newSort = e.target.value
const changed = newSort !== currentSort
currentSort = newSort
if (changed) {
imgGrid.innerHTML = ''
const urls = await getUrls(subfolder)
if (urls) {
imgs = getImgsFromUrls(urls, imgGrid)
}
}
})
const sizeSlider = makeSlider(64, 1024, currentWidth, 1)
imgTools.appendChild(orderSelect)
imgTools.appendChild(sizeSlider)
imgs = getImgsFromUrls(urls, imgGrid)
sizeSlider.addEventListener('input', (e) => {
currentWidth = e.target.value
for (const img of imgs) {
img.style.width = `${e.target.value}px`
}
})
handle = renderSidebar(el, cont, [selector, imgGrid, imgTools])
},
destroy: () => {
if (handle) {
handle.unregister()
handle = undefined
app.api.removeEventListener('status')
}
},
})
-85
View File
@@ -174,101 +174,16 @@ export const ensureMTBStyles = () => {
.mtb_slider[type="range"]:active::-webkit-slider-thumb {
background-color: ${S.accent};
}
`
const contextMenus = `
.mtb_context_menu {
position: fixed;
background: var(--comfy-input-bg);
border: 1px solid var(--border-color);
border-radius: 4px;
padding: 4px 0;
min-width: 150px;
z-index: 1000;
box-shadow: 0 2px 5px rgba(0,0,0,0.2);
}
.mtb-context-menu-item {
padding: 6px 12px;
cursor: pointer;
display: flex;
align-items: center;
gap: 8px;
}
.mtb-context-menu-item:hover {
background: var(--comfy-input-hover);
}
.mtb-loading-indicator {
position: sticky;
bottom: 0;
left: 0;
right: 0;
background: var(--comfy-input-bg);
padding: 8px;
text-align: center;
border-top: 1px solid var(--border-color);
z-index: 100;
}
`
addNamedStyleSheet(
'mtb_ui',
`
${common}
${inputs}
${contextMenus}
`,
)
}
export class ContextMenu {
constructor(parent) {
this.menu = makeElement('div.mtb_context_menu', { display: 'none' })
const body = parent || document.body
body.appendChild(this.menu)
document.addEventListener('click', (e) => {
if (!this.menu.contains(e.target)) {
this.hide()
}
})
}
show(x, y, items, context) {
this.menu.innerHTML = ''
for (const item of items) {
const menuItem = makeElement('div.mtb-context-menu-item')
if (item.icon) {
const icon = makeElement(`i.${item.icon}`)
menuItem.appendChild(icon)
}
menuItem.appendChild(document.createTextNode(item.label))
menuItem.onclick = () => {
item.action(context)
this.hide()
}
this.menu.appendChild(menuItem)
}
this.menu.style.display = 'block'
const rect = this.menu.getBoundingClientRect()
const viewportWidth = window.innerWidth
const viewportHeight = window.innerHeight
x = Math.min(x, viewportWidth - rect.width)
y = Math.min(y, viewportHeight - rect.height)
this.menu.style.left = `${x}px`
this.menu.style.top = `${y}px`
}
hide() {
this.menu.style.display = 'none'
}
}
/**
* Wrap an element with a div
*
+3 -3
View File
@@ -600,9 +600,9 @@ class NotePlus extends LiteGraph.LGraphNode {
// this.updateHTML(this.html_widget.value) // widget is populated here since we called super
// this.updateCSS(this.css_widget.value)
}
// onRemoved() {
// infoLogger('Node removed', this?.uuid)
// }
onRemoved() {
infoLogger('Node removed', this.uuid)
}
getExtraMenuOptions() {
const currentMode = this.edit_mode_widget.value
const newMode = currentMode === 'html' ? 'markdown' : 'html'