JaehaerysNL/StableDiffusion
1
0
Fork 0
mirror of https://github.com/Stability-AI/stablediffusion.git synced 2024-12-22 15:44:58 +00:00
Code Issues Projects Releases Packages Wiki Activity

final ckpt links for unclip

Browse source
This commit is contained in:
Robin Rombach 2023-03-20 14:28:06 +01:00
parent 88553b6da4
commit e04300bb4f
3 changed files with 6 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
README.md
View file

@ -13,7 +13,7 @@ new checkpoints. The following list provides an overview of all currently availa
*Stable UnCLIP 2.1* *Stable UnCLIP 2.1*
- New stable diffusion finetune (_Stable unCLIP 2.1_, [HuggingFace](https://huggingface.co/stabilityai/)) at 768x768 resolution, - New stable diffusion finetune (_Stable unCLIP 2.1_, [HuggingFace](https://huggingface.co/stabilityai/)) at 768x768 resolution,
based on SD2.1-768. This model allows for image variations and mixing operations as described in [*Hierarchical Text-Conditional Image Generation with CLIP Latents*](https://arxiv.org/abs/2204.06125), and, thanks to its modularity, can be combined with other models based on SD2.1-768. This model allows for image variations and mixing operations as described in [*Hierarchical Text-Conditional Image Generation with CLIP Latents*](https://arxiv.org/abs/2204.06125), and, thanks to its modularity, can be combined with other models
such as [KARLO](https://github.com/kakaobrain/karlo). Comes in two variants: [*Stable unCLIP-L*](TODO) and [*Stable unCLIP-H*](TODO), which are conditioned on CLIP such as [KARLO](https://github.com/kakaobrain/karlo). Comes in two variants: [*Stable unCLIP-L*](https://huggingface.co/stabilityai/stable-diffusion-2-1-unclip/blob/main/sd21-unclip-l.ckpt) and [*Stable unCLIP-H*](https://huggingface.co/stabilityai/stable-diffusion-2-1-unclip/blob/main/sd21-unclip-h.ckpt), which are conditioned on CLIP
ViT-L and ViT-H image embeddings, respectively. Instructions are available [here](doc/UNCLIP.MD). ViT-L and ViT-H image embeddings, respectively. Instructions are available [here](doc/UNCLIP.MD).
**December 7, 2022** **December 7, 2022**

5
doc/UNCLIP.MD
View file

@ -5,7 +5,8 @@ trained to invert CLIP image embeddings.
We finetuned SD 2.1 to accept a CLIP ViT-L/14 image embedding in addition to the text encodings. We finetuned SD 2.1 to accept a CLIP ViT-L/14 image embedding in addition to the text encodings.
This means that the model can be used to produce image variations, but can also be combined with a text-to-image This means that the model can be used to produce image variations, but can also be combined with a text-to-image
embedding prior to yield a full text-to-image model at 768x768 resolution. embedding prior to yield a full text-to-image model at 768x768 resolution.
We provide two models, trained on OpenAI CLIP-L and OpenCLIP-H image embeddings, respectively, available from [https://huggingface.co/stabilityai/](TODO). We provide two models, trained on OpenAI CLIP-L and OpenCLIP-H image embeddings, respectively,
available from [https://huggingface.co/stabilityai/stable-diffusion-2-1-unclip](https://huggingface.co/stabilityai/stable-diffusion-2-1-unclip/tree/main).
To use them, download from Hugging Face, and put and the weights into the `checkpoints` folder. To use them, download from Hugging Face, and put and the weights into the `checkpoints` folder.
#### Image Variations #### Image Variations
![image-variations-l-1](../assets/stable-samples/stable-unclip/unclip-variations.png) ![image-variations-l-1](../assets/stable-samples/stable-unclip/unclip-variations.png)
@ -37,7 +38,7 @@ wget https://arena.kakaocdn.net/brainrepo/models/karlo-public/v1.0.0.alpha/0b623
wget https://arena.kakaocdn.net/brainrepo/models/karlo-public/v1.0.0.alpha/85626483eaca9f581e2a78d31ff905ca/prior-ckpt-step%3D01000000-of-01000000.ckpt wget https://arena.kakaocdn.net/brainrepo/models/karlo-public/v1.0.0.alpha/85626483eaca9f581e2a78d31ff905ca/prior-ckpt-step%3D01000000-of-01000000.ckpt
cd ../../ cd ../../
``` ```
and the finetuned SD2.1 unCLIP-L checkpoint from [https://huggingface.co/stabilityai/](https://huggingface.co/stabilityai/TODO), and put the ckpt into the `checkpoints folder` and the finetuned SD2.1 unCLIP-L checkpoint from [here](https://huggingface.co/stabilityai/stable-diffusion-2-1-unclip/blob/main/sd21-unclip-l.ckpt), and put the ckpt into the `checkpoints folder`
Then, run Then, run

4
scripts/streamlit/stableunclip.py
View file

@ -198,11 +198,11 @@ def init(version="Stable unCLIP-L", load_karlo_prior=False):
if not "model" in state: if not "model" in state:
if version == "Stable unCLIP-L": if version == "Stable unCLIP-L":
config = "configs/stable-diffusion/v2-1-stable-unclip-l-inference.yaml" config = "configs/stable-diffusion/v2-1-stable-unclip-l-inference.yaml"
ckpt = "checkpoints/v2-1-stable-unclip-l-ft.ckpt" ckpt = "checkpoints/sd21-unclip-l.ckpt"
elif version == "Stable unOpenCLIP-H": elif version == "Stable unOpenCLIP-H":
config = "configs/stable-diffusion/v2-1-stable-unclip-h-inference.yaml" config = "configs/stable-diffusion/v2-1-stable-unclip-h-inference.yaml"
ckpt = "checkpoints/v2-1-stable-unclip-h-ft.ckpt" ckpt = "checkpoints/sd21-unclip-h.ckpt"
elif version == "Full Karlo": elif version == "Full Karlo":
from ldm.modules.karlo.kakao.sampler import T2ISampler from ldm.modules.karlo.kakao.sampler import T2ISampler