modelscope-scepter/docs/en/tasks/largen.md

Locate, Assign, Refine: Taming Customized Image Inpainting with Text-Subject Guidance

Yulin Pan · Chaojie Mao · Zeyinzi Jiang · Zhen Han · Jingfeng Zhang

LARGen is a unified image inpainting framework that supports text-guided, subject-guided and text-subject-guided inpainting simutaneously. Four LARGen-based fantastic applications are now supported by SCEPTER Studio:

1. Zoom Out
2. Virtual Try On
3. Text-Guided Inpainting
4. Text-Subject-Guided Inpainting

Basic Usage

Here's a demo showcasing the use of LARGen-based functions.

Gallery

LAR-Gen: Zoom Out

Origin Image Prompt: a temple on fire	Zoom-Out CenterAround:0.75	Zoom-Out CenterAround:0.75	Zoom-Out CenterAround:0.75	Zoom-Out CenterAround:0.75

LAR-Gen: Virtual Try-on

Model Image	Model Mask	Clothing Image	Clothing Mask	Try-on Output

LAR-Gen: Inpainting (Text guided)

Origin Image Prompt: a blue and white porcelain	Inpainting Mask1	Inpainting Output1	Inpainting Mask2 Prompt: a clock	Inpainting Output2

LAR-Gen: Inpainting (Text and Subject guided)

Origin Image Prompt: a dog wearing sunglasses	Origin Mask	Reference Image	Reference Mask	Inpainting Output

Features

Model	Locate	Assign	Refine
SD v1.5	⏳	⏳	⏳
SD XL	🪄	🪄	⏳

• 🪄 denotes that the feature has been supported.
• ⏳ denotes that the feature has not been integrated currently.

Pretrained Models

Model	URL
largen-sdxl-s22k	ModelScope

BibTeX

If our work is useful for your research, please consider citing:

@article{pan2024locate,
  title={Locate, Assign, Refine: Taming Customized Image Inpainting with Text-Subject Guidance},
  author={Pan, Yulin and Mao, Chaojie and Jiang, Zeyinzi and Han, Zhen and Zhang, Jingfeng},
  journal={arXiv preprint arXiv:2403.19534},
  year={2024}
}