Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
renzhc
diffusers_dcu
Commits
95ea538c
Unverified
Commit
95ea538c
authored
Jun 21, 2023
by
YiYi Xu
Committed by
GitHub
Jun 21, 2023
Browse files
Add ddpm kandinsky (#3783)
* update doc --------- Co-authored-by:
yiyixuxu
<
yixu310@gmail,com
>
parent
ef3844d3
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
17 additions
and
6 deletions
+17
-6
docs/source/en/api/pipelines/kandinsky.mdx
docs/source/en/api/pipelines/kandinsky.mdx
+14
-0
src/diffusers/pipelines/kandinsky/pipeline_kandinsky.py
src/diffusers/pipelines/kandinsky/pipeline_kandinsky.py
+3
-6
No files found.
docs/source/en/api/pipelines/kandinsky.mdx
View file @
95ea538c
...
...
@@ -55,6 +55,20 @@ t2i_pipe = DiffusionPipeline.from_pretrained("kandinsky-community/kandinsky-2-1"
t2i_pipe.to("cuda")
```
<Tip warning={true}>
By default, the text-to-image pipeline use [`DDIMScheduler`], you can change the scheduler to [`DDPMScheduler`]
```py
scheduler = DDPMScheduler.from_pretrained("kandinsky-community/kandinsky-2-1", subfolder="ddpm_scheduler")
t2i_pipe = DiffusionPipeline.from_pretrained(
"kandinsky-community/kandinsky-2-1", scheduler=scheduler, torch_dtype=torch.float16
)
t2i_pipe.to("cuda")
```
</Tip>
Now we pass the prompt through the prior to generate image embeddings. The prior
returns both the image embeddings corresponding to the prompt and negative/unconditional image
embeddings corresponding to an empty string.
...
...
src/diffusers/pipelines/kandinsky/pipeline_kandinsky.py
View file @
95ea538c
...
...
@@ -22,7 +22,7 @@ from transformers import (
from
...models
import
UNet2DConditionModel
,
VQModel
from
...pipelines
import
DiffusionPipeline
from
...pipelines.pipeline_utils
import
ImagePipelineOutput
from
...schedulers
import
DDIMScheduler
from
...schedulers
import
DDIMScheduler
,
DDPMScheduler
from
...utils
import
(
is_accelerate_available
,
is_accelerate_version
,
...
...
@@ -88,7 +88,7 @@ class KandinskyPipeline(DiffusionPipeline):
Frozen text-encoder.
tokenizer ([`XLMRobertaTokenizer`]):
Tokenizer of class
scheduler (
[`DDI
MScheduler`]):
scheduler (
Union[`DDIMScheduler`,`DDP
MScheduler`]):
A scheduler to be used in combination with `unet` to generate image latents.
unet ([`UNet2DConditionModel`]):
Conditional U-Net architecture to denoise the image embedding.
...
...
@@ -101,7 +101,7 @@ class KandinskyPipeline(DiffusionPipeline):
text_encoder
:
MultilingualCLIP
,
tokenizer
:
XLMRobertaTokenizer
,
unet
:
UNet2DConditionModel
,
scheduler
:
DDIMScheduler
,
scheduler
:
Union
[
DDIMScheduler
,
DDPMScheduler
],
movq
:
VQModel
,
):
super
().
__init__
()
...
...
@@ -439,9 +439,6 @@ class KandinskyPipeline(DiffusionPipeline):
noise_pred
,
t
,
latents
,
# YiYi notes: only reason this pipeline can't work with unclip scheduler is that can't pass down this argument
# need to use DDPM scheduler instead
# prev_timestep=prev_timestep,
generator
=
generator
,
).
prev_sample
# post-processing
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment