71 lines
4.1 KiB
Markdown
71 lines
4.1 KiB
Markdown
# ComfyUI_CartoonSegmentation
|
|
|
|
Front end ComfyUI nodes for CartoonSegmentation
|
|
based upon the work of the [CartoonSegmentation](https://github.com/CartoonSegmentation/CartoonSegmentation) repository this project will provide a front end to some of the features. For notation purposes this project will be referred to as ComfyUI_CartoonSegmentation of briefly as "CfyCS" and the non UI code that provides the power to this project will be referred to as "CartoonSegmentation".
|
|
|
|
## Run Segmentation
|
|
This will create segmentations for an input image. An example of final anime segmentation result is [workflows and outputs](examples/workflows_and_outputs/cartoon_segmentation.png) folder. The image source used is in the [[examples/source_image/standing_in_rain.png]]
|
|
|
|
## Run 3d Kenburns
|
|
This will create a short video of a zoom in and out from a single image. There are two finished output examples show in the [[examples/workflows_and_outputs/]] folder based on the images warrior and warrior_blue in the [example source image](examples/source_image/) folder. This workflow includes a required image creation at the end of workflow to pass ComfyUI validation. The image is a screenshot from the half way through the video.
|
|
|
|
## Install
|
|
### Initial Setup
|
|
```bash
|
|
# Create a venv and active it
|
|
# The root of the environment will be noted as [venv_path]
|
|
# Start in ComfyUI Root Directory
|
|
cd custom_nodes
|
|
git clone https://github.com/Nlar/ComfyUI_CartoonSegmentation.git
|
|
cd ComfyUI_CartoonSegmentation
|
|
# This is inteded as a superset of the requirments for CartoonSegmentation
|
|
pip install -r requirement.txt
|
|
```
|
|
### Addtional Files
|
|
#### libpatchmatch
|
|
Follow the [instructions](https://github.com/CartoonSegmentation/CartoonSegmentation#compile-patchmatch) from the Cartoon Segmentation project.
|
|
|
|
With the nodes new directory structure the file should be placed in
|
|
./custom_nodes/ComfyUI_CartoonSegmentation/CartoonSegmentation/data/libs/libpatchmatch_inpaint.so
|
|
|
|
#### Models
|
|
Follow the [instructions](https://github.com/CartoonSegmentation/CartoonSegmentation#download-models) to download the models from Hugging Face after changing the local directory.
|
|
```bash
|
|
# The ckpt files should be in the directory [comfy_root]/custom_nodes/ComfyUI_CartoonSegmentation/CartoonSegmentation/models/AnimeInstanceSegmentation
|
|
cd custom_nodes/ComfyUI_CartoonSegmentation/CartoonSegmentation
|
|
# Follow the instructions from CartoonSegmentation repository
|
|
```
|
|
**Currently only the rtmdetl_e60.ckpt model is supported.**
|
|
|
|
#### res101.pth
|
|
The original hosting provider no longer stores this annotator. It can be found on [HuggingFace](https://huggingface.co/lllyasviel/Annotators/blob/af19c34529d974eb965a00250f7b743431d56047/res101.pth) and should be placed in the "[comfy_root]/custom_nodes/ComfyUI_CartoonSegmentation/CartoonSegmentation/models/leres" directory.
|
|
```bash
|
|
cd [comfy_root]/custom_nodes/ComfyUI_CartoonSegmentation/CartoonSegmentation/models
|
|
mkdir leres
|
|
# Place file in the leres directory
|
|
```
|
|
|
|
### Troubleshooting
|
|
|
|
#### ModuleNotFoundError: No module named 'mmcv._ext'
|
|
The requirements for mmcv are fairly specific and this message indicates that will have to manually install the package. The current version of mmcv (2.1.0) can be used with this module. The best case install can be completed by downloading mmcv from the [mmcv release](https://github.com/open-mmlab/mmcv/releases/tag/v2.1.0) and installed through pip.
|
|
|
|
```bash
|
|
pip install [extracted_directory]/mmcv
|
|
```
|
|
|
|
#### OSError: [venv_path]/lib/python3.11/site-packages/torch/lib/libgomp-a34b3233.so.1: version `GOMP_5.0' not found (required by /usr/lib/libvtkCommonCore.so.1)
|
|
The installed version of gcc-libs (13.2.1-3) provided a file from /usr/lib/libgomp.so. This file was able to resolve this issue.
|
|
```bash
|
|
cd [venv_path]/lib/python3.1*/site-packages/torch/lib/
|
|
mv libgomp-a34b3233.so.1 libgomp-a34b3233.so.1.bak
|
|
# Symlink below can be replaced with a copy command.
|
|
ln -s /usr/lib/libgomp.so libgomp-a34b3233.so.1
|
|
```
|
|
|
|
|
|
## Roadmap
|
|
- ~~Correct output folder issue~~
|
|
- ~~Verify functionality beyond KenBurns Config Loader~~
|
|
- ~~Ouptut masks for the input image during anime segmentation.~~
|