Add LCM info to readme
This commit is contained in:
@@ -48,6 +48,8 @@ That's it! You're now ready to start enhancing your ComfyUI workflows with Core
|
||||
- **VAE**: Variational Autoencoder. A model that learns a latent representation of images. It's used as a prior in
|
||||
Stable Diffusion.
|
||||
- **Checkpoint**: A file that contains the weights of a model. It's used to load models in Stable Diffusion.
|
||||
- **LCM**: [Latent Consistency Model](https://latent-consistency-models.github.io/). A type of model designed to
|
||||
generate images with as few steps as possible.
|
||||
|
||||
> [!NOTE]
|
||||
> Note on Compute Units:
|
||||
@@ -64,6 +66,8 @@ These custom nodes come with a host of features, including:
|
||||
- Support for ANE (Apple Neural Engine)
|
||||
- Support for CPU and GPU
|
||||
- Support for `mlmodelc` and `mlpackage` files
|
||||
- Support for LCM models
|
||||
- LCM -> Core ML conversion
|
||||
|
||||
> [!NOTE]
|
||||
> Please note that using Core ML models can take a bit longer to load initially.
|
||||
@@ -139,6 +143,30 @@ resulting latent as you normally would in your workflow.
|
||||
- **LATENT**: The latent image output by the Core ML model. This can be decoded using a VAE Decoder or used as input
|
||||
to the next node in your workflow.
|
||||
|
||||
#### LCM Converter
|
||||
|
||||

|
||||
|
||||
This node converts [SimianLuo/LCM_Dreamshaper_v7](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7) model to Core
|
||||
ML. The converted model is stored in the `models/unet` directory and can be used with the Core ML UNet Loader. The
|
||||
conversion parameteres are encoded in the node name, so if the model already exists, the node will not convert it again.
|
||||
|
||||
- **Inputs**:
|
||||
- **height**: The desired height of the image generated by the model. The default is 512. Must be a multiple of 8.
|
||||
- **width**: The desired width of the image generated by the model. The default is 512. Must be a multiple of 8.
|
||||
- **batch_size**: The batch size of generated images. If you're planning to generate batches of images, you can try
|
||||
increasing this value to speed up the generation process. The default is 1.
|
||||
- **compute_unit**: The hardware on which the model should run. This is used only when loading the model and
|
||||
doesn't affect the conversion process.
|
||||
- **controlnet_support**: For the model to support ControlNet, it must be converted with this option set to True.
|
||||
The default is False.
|
||||
|
||||
> [!NOTE]
|
||||
> The conversion process can take a while, so please be patient.
|
||||
|
||||
> [!NOTE]
|
||||
> When using the LCM model with Core ML Sampler, please set _sampler_name_ to `lcm` and _scheduler_ to `sgm_uniform`.
|
||||
|
||||
#### Core ML Adapter (Experimental) (`CoreMLModelAdapter`)
|
||||
|
||||

|
||||
@@ -197,6 +225,13 @@ The ControlNet model used in this workflow is available
|
||||
Once downloaded, place the model in the `models/controlnet` directory.
|
||||

|
||||
|
||||
#### LCM conversion with ControlNet
|
||||
|
||||
This workflow uses LCM converter to
|
||||
convert [SimianLuo/LCM_Dreamshaper_v7](https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7)
|
||||
model to Core ML. The converted model can then be used with or without ControlNet to generate images.
|
||||

|
||||
|
||||
## Limitations
|
||||
|
||||
- Core ML models are fixed in terms of their inputs and outputs.
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 476 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 51 KiB |
Reference in New Issue
Block a user