Locate, Assign, Refine: Taming Customized Image Inpainting with Text-Subject Guidance

Yulin Pan · Chaojie Mao · Zeyinzi Jiang · Zhen Han · Jingfeng Zhang

LARGen is a unified image inpainting framework that supports text-guided, subject-guided and text-subject-guided inpainting simutaneously. Four LARGen-based fantastic applications are now supported by SCEPTER Studio: 1. Zoom Out 2. Virtual Try On 3. Text-Guided Inpainting 4. Text-Subject-Guided Inpainting ## Basic Usage Here's a demo showcasing the use of LARGen-based functions.

## Gallery ### LAR-Gen: Zoom Out

Origin Image Prompt: a temple on fire	Zoom-Out CenterAround:0.75	Zoom-Out CenterAround:0.75	Zoom-Out CenterAround:0.75	Zoom-Out CenterAround:0.75

### LAR-Gen: Virtual Try-on

Model Image	Model Mask	Clothing Image	Clothing Mask	Try-on Output

### LAR-Gen: Inpainting (Text guided)

Origin Image Prompt: a blue and white porcelain	Inpainting Mask1	Inpainting Output1	Inpainting Mask2 Prompt: a clock	Inpainting Output2

### LAR-Gen: Inpainting (Text and Subject guided)

Origin Image Prompt: a dog wearing sunglasses	Origin Mask	Reference Image	Reference Mask	Inpainting Output

## Features | **Model** | **Locate** | **Assign** | **Refine** | |:---------:|:----------:|:----------:|:----------:| | SD v1.5 | ⏳ | ⏳ | ⏳ | | SD XL | 🪄 | 🪄 | ⏳ | - 🪄 denotes that the feature has been supported. - ⏳ denotes that the feature has not been integrated currently. ## Pretrained Models | **Model** | **URL** | |:----------:|:-------:| | largen-sdxl-s22k | [ModelScope](https://www.modelscope.cn/models/iic/LARGEN/summary) | ## BibTeX If our work is useful for your research, please consider citing: ```bibtex @article{pan2024locate, title={Locate, Assign, Refine: Taming Customized Image Inpainting with Text-Subject Guidance}, author={Pan, Yulin and Mao, Chaojie and Jiang, Zeyinzi and Han, Zhen and Zhang, Jingfeng}, journal={arXiv preprint arXiv:2403.19534}, year={2024} } ```