Commit Graph

Select branches

Hide Pull Requests

main

old

sd3

ti_cross_attn_loss

Mono Color

• 4959c87ee7 rename workflows xander 2024-08-07 21:50:57 +02:00
• cc467f04b7 put comfyui workflows in separate folder xander 2024-08-07 21:07:40 +02:00
• f5b2569646 simplify xander 2024-08-07 21:03:57 +02:00
• 7f80ca88b3 simplify xander 2024-08-07 20:55:09 +02:00
• 5a2283b4c0 update defaults xander 2024-08-05 13:44:51 +02:00
• 2c1bb4a3f9 bugfix xander 2024-08-05 13:13:38 +02:00
• ac2dcf787c remove objects xander 2024-08-05 04:07:18 +02:00
• ff165f32db more cleanup and tweaking xander 2024-08-05 04:06:23 +02:00
• 8e71457f07 massive refactor and attention regularization optimizations xander 2024-08-04 20:10:40 +02:00
• ef61d5f0e4 update defaults xander 2024-08-03 04:01:47 +02:00
• 47a0d37b45 tweak params xander 2024-08-03 03:49:00 +02:00
• 32836aa82f massive upgrades to token attention reg, add florence captioning xander 2024-08-03 03:26:39 +02:00
• 2b9f9c0f0a Merge branch 'main' of https://github.com/edenartlab/trainer into main xander 2024-08-02 18:46:49 +02:00
• 479465b47a add token attention regularization xander 2024-08-02 18:46:42 +02:00
• fb8ee98a1c penalize mean ti_cross_attn_loss mayukhdeb 2024-08-02 04:31:01 -07:00
• 7ff02a35ce smaller daam loss scale mayukhdeb 2024-08-02 04:13:23 -07:00
• 21200b41b9 naive distribution loss, did not work :( mayukhdeb 2024-08-02 03:17:32 -07:00
• c21605bb95 also plot token as string on title mayukhdeb 2024-08-01 00:49:47 -07:00
• 4880e3f096 vis token wise daam maps mayukhdeb 2024-08-01 00:39:49 -07:00
• 35e76fdef0 wip reproduce daam mayukhdeb 2024-07-30 23:01:14 -07:00
• 871e425f3e obtain single layer heatmap mayukhdeb 2024-07-30 22:50:56 -07:00
• e6534f8d1d baby steps mayukhdeb 2024-07-30 22:50:22 -07:00
• b5ad0b659e watch daam loss and plot norms into heatmaps dir mayukhdeb 2024-07-29 10:46:11 -07:00
• d6a205cc7b ahijack AttnProcessor2_0 mayukhdeb 2024-07-29 10:12:43 -07:00
• f21efd7840 train TI on 2 tokens mayukhdeb 2024-07-29 10:11:15 -07:00
• 29855d94ba ignore notebook checkpoints folder mayukhdeb 2024-07-22 12:03:25 -07:00
• 006a3b9750 add ti config xander 2024-07-22 20:53:49 +02:00
• b166f7c274 merge aiXander 2024-07-22 11:47:38 -07:00
• f6cfd6d0bc fix typo aiXander 2024-07-22 11:46:33 -07:00
• 6e4c86ea0c keep a copy sd3 mayukhdeb 2024-07-22 11:07:14 -07:00
• 7e30d33dc2 find image filenames recursively in folder + train on big style dataset mayukhdeb 2024-07-21 03:56:42 -07:00
• fbd050cb0c cleaner preprocess fn mayukhdeb 2024-07-21 03:25:07 -07:00
• 70c8e88195 migrate to huggingface script mayukhdeb 2024-07-21 02:05:30 -07:00
• 2f6f0cacd2 ignore stuff mayukhdeb 2024-07-21 02:03:42 -07:00
• 6a4fa1ef8d add progressbar to node xander 2024-07-18 21:04:43 +02:00
• e8932f30ef start with a different embed string mayukhdeb 2024-07-18 09:57:00 -07:00
• 3cd17086f3 trainer comfyui v1 xander 2024-07-18 05:50:04 +02:00
• 75137a7539 Merge branch 'main' of https://github.com/edenartlab/sd-lora-trainer into main xander 2024-07-18 04:07:07 +02:00
• ee91f8c0c7 update naming conventions xander 2024-07-18 04:07:00 +02:00
• 598c4204be add test config xander 2024-07-18 04:05:10 +02:00
• 8da6c91af8 Merge branch 'main' of https://github.com/edenartlab/sd-lora-trainer into main xander 2024-07-18 04:00:30 +02:00
• 01523350e1 cleanup save dir xander 2024-07-18 04:00:21 +02:00
• c891203365 update gitignore xander 2024-07-18 04:00:08 +02:00
• 85599dc725 minor changes xander 2024-07-18 03:43:43 +02:00
• af16d1edcc update configs xander 2024-07-18 03:26:51 +02:00
• b3db4ff3bb Merge branch 'main' of https://github.com/edenartlab/trainer into main xander 2024-07-18 03:16:43 +02:00
• 2a0761ae75 use gpt-4o xander 2024-07-18 03:16:41 +02:00
• d8ee9ee522 use gpt-4o xander 2024-07-18 03:14:31 +02:00
• 4f084d251c updates for comfyui node xander 2024-07-18 03:12:11 +02:00
• fb521e9dbd add print flush aiXander 2024-07-16 04:39:29 -07:00
• f937106f6c push changes aiXander 2024-07-16 04:04:34 -07:00
• 5f8fe5c4c8 Update requirements.txt Gene Kogan 2024-07-16 03:44:59 -07:00
• 2f5eaeba7a Update cog.yaml Gene Kogan 2024-07-16 03:44:35 -07:00
• b7dd176536 add adamw8bit mayukhdeb 2024-07-15 07:48:07 -07:00
• 5a72aca596 useful wandb name mayukhdeb 2024-07-15 07:47:34 -07:00
• 85264bd004 wandb mayukhdeb 2024-07-15 07:38:27 -07:00
• d46976f3dd sweep params set mayukhdeb 2024-07-15 07:33:45 -07:00
• 8ed5996c1f train and inference on same device mayukhdeb 2024-07-15 07:32:27 -07:00
• 8247b4b9dd fix OOM during inference (vae.decode) mayukhdeb 2024-07-15 07:20:13 -07:00
• dc28fcb988 inference on same device mayukhdeb 2024-07-15 03:28:31 -07:00
• 3c0a3be55f save different sh files for each gpu mayukhdeb 2024-07-15 03:28:20 -07:00
• 7d8c845765 update yaml and print deps in config aiXander 2024-07-12 05:16:06 -07:00
• 274ec5c90b disable textual inversion if config.ti_lr is None mayukhdeb 2024-07-11 12:46:52 -07:00
• fc54948eff more tweaks mayukhdeb 2024-07-11 12:39:30 -07:00
• 32a7ae248d prompts for sweep mayukhdeb 2024-07-11 12:34:42 -07:00
• 3cbabd2634 sweep dry runs mayukhdeb 2024-07-11 12:26:07 -07:00
• 1cdacbc63a remove old todos mayukhdeb 2024-07-11 11:16:22 -07:00
• 340336b53d print config pre training start aiXander 2024-07-11 06:11:33 -07:00
• 5abed1487f setup lora_scale automation for validation grid xander 2024-07-11 13:36:11 +02:00
• 4668f755ec switch to adamw_8bit for sd3 transformer lora params mayukhdeb 2024-07-11 00:15:09 -07:00
• b5cf857b1f push small changes xander 2024-07-09 18:33:52 +02:00
• dd093616b5 fix inference prompts bug mayukhdeb 2024-07-09 09:09:56 -07:00
• 2c824d6807 more progress mayukhdeb 2024-07-09 05:35:21 -07:00
• 022d51f53a inference fixed mayukhdeb 2024-07-09 00:25:02 -07:00
• ecacf815ec re-impl textual inversion for first 2 text encoders mayukhdeb 2024-07-08 12:45:55 -07:00
• e1813c75d1 updates to avoid OOM when plotting hist xander 2024-07-06 22:18:47 +02:00
• 4fde1b7dd6 small tweaks xander 2024-07-06 18:45:38 +02:00
• 892cf61c52 add disable_ti flag xander 2024-07-06 18:43:03 +02:00
• fdd531d3c5 fix full finetuning and add 8bitadam xander 2024-07-06 17:42:49 +02:00
• ef6635fafb update train cmd xander 2024-07-05 17:11:06 +02:00
• faf864ce15 tiny tweak to learning rates for SDXL xander 2024-07-05 17:07:33 +02:00
• 90d2269572 testing textual inversion training with frozen transformer mayukhdeb 2024-07-04 07:40:41 -07:00
• 4955261cae new checkpoint + deterministic inference mayukhdeb 2024-07-03 04:28:06 -07:00
• e84932af29 some small changes to text with main_sd3.py mayukhdeb 2024-07-03 04:06:21 -07:00
• 115da83e2d cleaner output dir with checkpoints and generated samples in one folder mayukhdeb 2024-07-03 03:43:30 -07:00
• 98b78cd6d6 cleanup + save training samples in output dir mayukhdeb 2024-07-03 03:11:56 -07:00
• 7c5e0949ba keep changes mayukhdeb 2024-07-02 05:19:34 -07:00
• ba2b7532ad more tweaks mayukhdeb 2024-07-02 05:16:22 -07:00
• 2c0939a733 run inference less often mayukhdeb 2024-07-02 04:25:26 -07:00
• 3b41479aa1 bfloat16 training + inference mayukhdeb 2024-07-02 04:12:23 -07:00
• 6e589c76a8 impement some upstream changes and save a sample every 10 train steps mayukhdeb 2024-07-02 01:50:14 -07:00
• 95f78d7b91 completely comement out TI for now mayukhdeb 2024-07-01 01:56:49 -07:00
• 3c0ebd7d70 apply mask to loss + some hardcoding for banny debugging mayukhdeb 2024-06-28 04:38:43 -07:00
• 6350b5344b more progress mayukhdeb 2024-06-28 03:22:49 -07:00
• 2c4bd43044 temporarily remove ti grad norms mayukhdeb 2024-06-28 03:12:16 -07:00
• b850a1615d watch grad norms mayukhdeb 2024-06-28 03:04:40 -07:00
• 6d6dd98bfa dynamic ti lr mayukhdeb 2024-06-28 02:03:34 -07:00
• 0444e35729 clip grad norms mayukhdeb 2024-06-26 03:01:34 -07:00
• 0748e1d14e better prompt mayukhdeb 2024-06-22 02:32:19 -07:00
• db7507c849 handle T5EncoderModel mayukhdeb 2024-06-22 01:38:48 -07:00