Update README.md
Browse files
README.md
CHANGED
|
@@ -11,6 +11,10 @@ tags:
|
|
| 11 |
|
| 12 |
[TokenCompose_SD21_B](https://mlpc-ucsd.github.io/TokenCompose/) is a [latent text-to-image diffusion model](https://arxiv.org/abs/2112.10752) finetuned from the [**Stable-Diffusion-v2-1**](https://huggingface.co/stabilityai/stable-diffusion-2-1) checkpoint at resolution 768x768 on the [VSR](https://github.com/cambridgeltl/visual-spatial-reasoning) split of [COCO image-caption pairs](https://cocodataset.org/#download) for 32,000 steps with a learning rate of 5e-6. The training objective involves token-level grounding terms in addition to denoising loss for enhanced multi-category instance composition and photorealism. The "_A/B" postfix indicates different finetuning runs of the model using the same above configurations.
|
| 13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
# 🧨Example Usage
|
| 15 |
|
| 16 |
We strongly recommend using the [🤗Diffuser](https://github.com/huggingface/diffusers) library to run our model.
|
|
@@ -39,4 +43,13 @@ image.save("cat_and_wine_glass.png")
|
|
| 39 |
| TokenCompose (SD21) | 60.10 | 80.48 | 36.69 | 5.71 | 79.51 | 39.59 | 8.13 | 19.15 |
|
| 40 |
|
| 41 |
# 📰 Citation
|
| 42 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
[TokenCompose_SD21_B](https://mlpc-ucsd.github.io/TokenCompose/) is a [latent text-to-image diffusion model](https://arxiv.org/abs/2112.10752) finetuned from the [**Stable-Diffusion-v2-1**](https://huggingface.co/stabilityai/stable-diffusion-2-1) checkpoint at resolution 768x768 on the [VSR](https://github.com/cambridgeltl/visual-spatial-reasoning) split of [COCO image-caption pairs](https://cocodataset.org/#download) for 32,000 steps with a learning rate of 5e-6. The training objective involves token-level grounding terms in addition to denoising loss for enhanced multi-category instance composition and photorealism. The "_A/B" postfix indicates different finetuning runs of the model using the same above configurations.
|
| 13 |
|
| 14 |
+
# 📄 Paper
|
| 15 |
+
|
| 16 |
+
Please follow [this](https://arxiv.org/abs/2312.03626) link.
|
| 17 |
+
|
| 18 |
# 🧨Example Usage
|
| 19 |
|
| 20 |
We strongly recommend using the [🤗Diffuser](https://github.com/huggingface/diffusers) library to run our model.
|
|
|
|
| 43 |
| TokenCompose (SD21) | 60.10 | 80.48 | 36.69 | 5.71 | 79.51 | 39.59 | 8.13 | 19.15 |
|
| 44 |
|
| 45 |
# 📰 Citation
|
| 46 |
+
```bibtex
|
| 47 |
+
@misc{wang2023tokencompose,
|
| 48 |
+
title={TokenCompose: Grounding Diffusion with Token-level Supervision},
|
| 49 |
+
author={Zirui Wang and Zhizhou Sha and Zheng Ding and Yilin Wang and Zhuowen Tu},
|
| 50 |
+
year={2023},
|
| 51 |
+
eprint={2312.03626},
|
| 52 |
+
archivePrefix={arXiv},
|
| 53 |
+
primaryClass={cs.CV}
|
| 54 |
+
}
|
| 55 |
+
```
|