[docs] More API fixes (#3640)

* part 2 of api fixes

* move randn_tensor

* add to toctree

* apply feedback

* more feedback

docs/source/en/_toctree.yml

@@ -144,12 +144,16 @@
       title: Outputs
     - local: api/loaders
       title: Loaders
+    - local: api/utilities
+      title: Utilities
     title: Main Classes
   - sections:
     - local: api/pipelines/overview
       title: Overview
     - local: api/pipelines/alt_diffusion
       title: AltDiffusion
+    - local: api/pipelines/attend_and_excite
+      title: Attend and Excite
     - local: api/pipelines/audio_diffusion
       title: Audio Diffusion
     - local: api/pipelines/audioldm
@@ -164,24 +168,32 @@
       title: DDIM
     - local: api/pipelines/ddpm
       title: DDPM
+    - local: api/pipelines/diffedit
+      title: DiffEdit
     - local: api/pipelines/dit
       title: DiT
     - local: api/pipelines/if
       title: IF
+    - local: api/pipelines/pix2pix
+      title: InstructPix2Pix
     - local: api/pipelines/kandinsky
       title: Kandinsky
     - local: api/pipelines/latent_diffusion
       title: Latent Diffusion
+    - local: api/pipelines/panorama
+      title: MultiDiffusion Panorama
     - local: api/pipelines/paint_by_example
       title: PaintByExample
+    - local: api/pipelines/pix2pix_zero
+      title: Pix2Pix Zero
     - local: api/pipelines/pndm
       title: PNDM
     - local: api/pipelines/repaint
       title: RePaint
-    - local: api/pipelines/stable_diffusion_safe
-      title: Safe Stable Diffusion
     - local: api/pipelines/score_sde_ve
       title: Score SDE VE
+    - local: api/pipelines/self_attention_guidance
+      title: Self-Attention Guidance
     - local: api/pipelines/semantic_stable_diffusion
       title: Semantic Guidance
     - local: api/pipelines/spectrogram_diffusion
@@ -199,31 +211,21 @@
         title: Depth-to-Image
       - local: api/pipelines/stable_diffusion/image_variation
         title: Image-Variation
-      - local: api/pipelines/stable_diffusion/upscale
-        title: Super-Resolution
+      - local: api/pipelines/stable_diffusion/stable_diffusion_safe
+        title: Safe Stable Diffusion
+      - local: api/pipelines/stable_diffusion/stable_diffusion_2
+        title: Stable Diffusion 2
       - local: api/pipelines/stable_diffusion/latent_upscale
         title: Stable-Diffusion-Latent-Upscaler
-      - local: api/pipelines/stable_diffusion/pix2pix
-        title: InstructPix2Pix
-      - local: api/pipelines/stable_diffusion/attend_and_excite
-        title: Attend and Excite
-      - local: api/pipelines/stable_diffusion/pix2pix_zero
-        title: Pix2Pix Zero
-      - local: api/pipelines/stable_diffusion/self_attention_guidance
-        title: Self-Attention Guidance
-      - local: api/pipelines/stable_diffusion/panorama
-        title: MultiDiffusion Panorama
-      - local: api/pipelines/stable_diffusion/model_editing
-        title: Text-to-Image Model Editing
-      - local: api/pipelines/stable_diffusion/diffedit
-        title: DiffEdit
+      - local: api/pipelines/stable_diffusion/upscale
+        title: Super-Resolution
       title: Stable Diffusion
-    - local: api/pipelines/stable_diffusion_2
-      title: Stable Diffusion 2
     - local: api/pipelines/stable_unclip
       title: Stable unCLIP
     - local: api/pipelines/stochastic_karras_ve
       title: Stochastic Karras VE
+    - local: api/pipelines/model_editing
+      title: Text-to-Image Model Editing
     - local: api/pipelines/text_to_video
       title: Text-to-Video
     - local: api/pipelines/text_to_video_zero

docs/source/en/api/diffusion_pipeline.mdx

@@ -12,41 +12,25 @@ specific language governing permissions and limitations under the License.
 
 # Pipelines
 
-The [`DiffusionPipeline`] is the easiest way to load any pretrained diffusion pipeline from the [Hub](https://huggingface.co/models?library=diffusers) and to use it in inference.
+The [`DiffusionPipeline`] is the easiest way to load any pretrained diffusion pipeline from the [Hub](https://huggingface.co/models?library=diffusers) and use it for inference.
 
 <Tip>
 	
-	One should not use the Diffusion Pipeline class for training or fine-tuning a diffusion model. Individual 
-	components of diffusion pipelines are usually trained individually, so we suggest to directly work 
-	with [`UNetModel`] and [`UNetConditionModel`].
+You shouldn't use the [`DiffusionPipeline`] class for training or finetuning a diffusion model. Individual 
+components (for example, [`UNetModel`] and [`UNetConditionModel`]) of diffusion pipelines are usually trained individually, so we suggest directly working with instead.
 
 </Tip>
 
-Any diffusion pipeline that is loaded with [`~DiffusionPipeline.from_pretrained`] will automatically 
-detect the pipeline type, *e.g.* [`StableDiffusionPipeline`] and consequently load each component of the 
-pipeline and pass them into the `__init__` function of the pipeline, *e.g.* [`~StableDiffusionPipeline.__init__`].
+The pipeline type (for example [`StableDiffusionPipeline`]) of any diffusion pipeline loaded with [`~DiffusionPipeline.from_pretrained`] is automatically 
+detected and pipeline components are loaded and passed to the `__init__` function of the pipeline.
 
 Any pipeline object can be saved locally with [`~DiffusionPipeline.save_pretrained`].
 
 ## DiffusionPipeline
+
 [[autodoc]] DiffusionPipeline
 	- all
 	- __call__
 	- device
 	- to
 	- components
-
-## ImagePipelineOutput
-By default diffusion pipelines return an object of class
-
-[[autodoc]] pipelines.ImagePipelineOutput
-
-## AudioPipelineOutput
-By default diffusion pipelines return an object of class
-
-[[autodoc]] pipelines.AudioPipelineOutput
-
-## ImageTextPipelineOutput
-By default diffusion pipelines return an object of class
-
-[[autodoc]] ImageTextPipelineOutput

docs/source/en/api/outputs.mdx

@@ -12,11 +12,11 @@ specific language governing permissions and limitations under the License.
 
 # BaseOutputs
 
-All models have outputs that are instances of subclasses of [`~utils.BaseOutput`]. Those are
-data structures containing all the information returned by the model, but that can also be used as tuples or
+All models have outputs that are subclasses of [`~utils.BaseOutput`]. Those are
+data structures containing all the information returned by the model, but they can also be used as tuples or
 dictionaries.
 
-Let's see how this looks in an example:
+For example:
 
 ```python
 from diffusers import DDIMPipeline
@@ -25,31 +25,45 @@ pipeline = DDIMPipeline.from_pretrained("google/ddpm-cifar10-32")
 outputs = pipeline()
 ```
 
-The `outputs` object is a [`~pipelines.ImagePipelineOutput`], as we can see in the
-documentation of that class below, it means it has an image attribute.
+The `outputs` object is a [`~pipelines.ImagePipelineOutput`] which means it has an image attribute.
 
-You can access each attribute as you would usually do, and if that attribute has not been returned by the model, you will get `None`:
+You can access each attribute as you normally would or with a keyword lookup, and if that attribute is not returned by the model, you will get `None`:
 
 ```python
 outputs.images
-```
-
-or via keyword lookup
-
-```python
 outputs["images"]
 ```
 
-When considering our `outputs` object as tuple, it only considers the attributes that don't have `None` values.
-Here for instance, we could retrieve images via indexing:
+When considering the `outputs` object as a tuple, it only considers the attributes that don't have `None` values.
+For instance, retrieving an image by indexing into it returns the tuple `(outputs.images)`:
 
 ```python
 outputs[:1]
 ```
 
-which will return the tuple `(outputs.images)` for instance.
+<Tip>
+
+To check a specific pipeline or model output, refer to its corresponding API documentation.
+
+</Tip>
 
 ## BaseOutput
 
 [[autodoc]] utils.BaseOutput
     - to_tuple
+
+## ImagePipelineOutput
+
+[[autodoc]] pipelines.ImagePipelineOutput
+
+## FlaxImagePipelineOutput
+
+[[autodoc]] pipelines.pipeline_flax_utils.FlaxImagePipelineOutput
+
+## AudioPipelineOutput
+
+[[autodoc]] pipelines.AudioPipelineOutput
+
+## ImageTextPipelineOutput
+
+[[autodoc]] ImageTextPipelineOutput
\ No newline at end of file

docs/source/en/api/utilities.mdx

+# Utilities
+
+Utility and helper functions for working with 🤗 Diffusers.
+
+## randn_tensor
+
+[[autodoc]] diffusers.utils.randn_tensor
+
+## numpy_to_pil
+
+[[autodoc]] utils.pil_utils.numpy_to_pil
+
+## pt_to_pil
+
+[[autodoc]] utils.pil_utils.pt_to_pil
+
+## load_image
+
+[[autodoc]] utils.testing_utils.load_image
+
+## export_to_video
+
+[[autodoc]] utils.testing_utils.export_to_video
\ No newline at end of file

docs/source/en/using-diffusers/reproducibility.mdx

@@ -111,7 +111,7 @@ print(np.abs(image).sum())
 
 The result is not the same even though you're using an identical seed because the GPU uses a different random number generator than the CPU.
 
-To circumvent this problem, 🧨 Diffusers has a [`randn_tensor`](#diffusers.utils.randn_tensor) function for creating random noise on the CPU, and then moving the tensor to a GPU if necessary. The `randn_tensor` function is used everywhere inside the pipeline, allowing the user to **always** pass a CPU `Generator` even if the pipeline is run on a GPU. 
+To circumvent this problem, 🧨 Diffusers has a [`~diffusers.utils.randn_tensor`] function for creating random noise on the CPU, and then moving the tensor to a GPU if necessary. The `randn_tensor` function is used everywhere inside the pipeline, allowing the user to **always** pass a CPU `Generator` even if the pipeline is run on a GPU. 
 
 You'll see the results are much closer now!
 
@@ -147,9 +147,6 @@ susceptible to precision error propagation. Don't expect similar results across
 different GPU hardware or PyTorch versions. In this case, you'll need to run 
 exactly the same hardware and PyTorch version for full reproducibility.
 
-### randn_tensor
-[[autodoc]] diffusers.utils.randn_tensor
-
 ## Deterministic algorithms
 
 You can also configure PyTorch to use deterministic algorithms to create a reproducible pipeline. However, you should be aware that deterministic algorithms may be slower than nondeterministic ones and you may observe a decrease in performance. But if reproducibility is important to you, then this is the way to go!

src/diffusers/configuration_utils.py

@@ -160,7 +160,7 @@ class ConfigMixin:
     @classmethod
     def from_config(cls, config: Union[FrozenDict, Dict[str, Any]] = None, return_unused_kwargs=False, **kwargs):
         r"""
-        Instantiate a Python class from a config dictionary
+        Instantiate a Python class from a config dictionary.
 
         Parameters:
             config (`Dict[str, Any]`):
@@ -170,9 +170,13 @@ class ConfigMixin:
                 Whether kwargs that are not consumed by the Python class should be returned or not.
 
             kwargs (remaining dictionary of keyword arguments, *optional*):
-                Can be used to update the configuration object (after it being loaded) and initiate the Python class.
-                `**kwargs` will be directly passed to the underlying scheduler/model's `__init__` method and eventually
-                overwrite same named arguments of `config`.
+                Can be used to update the configuration object (after it is loaded) and initiate the Python class.
+                `**kwargs` are directly passed to the underlying scheduler/model's `__init__` method and eventually
+                overwrite same named arguments in `config`.
+
+        Returns:
+            [`ModelMixin`] or [`SchedulerMixin`]:
+                A model or scheduler object instantiated from a config dictionary.
 
         Examples:
 
@@ -258,59 +262,57 @@ class ConfigMixin:
         **kwargs,
     ) -> Tuple[Dict[str, Any], Dict[str, Any]]:
         r"""
-        Instantiate a Python class from a config dictionary
+        Load a model or scheduler configuration.
 
         Parameters:
             pretrained_model_name_or_path (`str` or `os.PathLike`, *optional*):
                 Can be either:
 
-                    - A string, the *model id* of a model repo on huggingface.co. Valid model ids should have an
-                      organization name, like `google/ddpm-celebahq-256`.
-                    - A path to a *directory* containing model weights saved using [`~ConfigMixin.save_config`], e.g.,
-                      `./my_model_directory/`.
+                    - A string, the *model id* (for example `google/ddpm-celebahq-256`) of a pretrained model hosted on
+                      the Hub.
+                    - A path to a *directory* (for example `./my_model_directory`) containing model weights saved with
+                      [`~ConfigMixin.save_config`].
 
             cache_dir (`Union[str, os.PathLike]`, *optional*):
-                Path to a directory in which a downloaded pretrained model configuration should be cached if the
-                standard cache should not be used.
+                Path to a directory where a downloaded pretrained model configuration is cached if the standard cache
+                is not used.
             force_download (`bool`, *optional*, defaults to `False`):
                 Whether or not to force the (re-)download of the model weights and configuration files, overriding the
                 cached versions if they exist.
             resume_download (`bool`, *optional*, defaults to `False`):
-                Whether or not to delete incompletely received files. Will attempt to resume the download if such a
-                file exists.
+                Whether or not to resume downloading the model weights and configuration files. If set to False, any
+                incompletely downloaded files are deleted.
             proxies (`Dict[str, str]`, *optional*):
-                A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128',
+                A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128',
                 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
             output_loading_info(`bool`, *optional*, defaults to `False`):
                 Whether or not to also return a dictionary containing missing keys, unexpected keys and error messages.
             local_files_only(`bool`, *optional*, defaults to `False`):
-                Whether or not to only look at local files (i.e., do not try to download the model).
+                Whether to only load local model weights and configuration files or not. If set to True, the model
+                won’t be downloaded from the Hub.
             use_auth_token (`str` or *bool*, *optional*):
-                The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
-                when running `transformers-cli login` (stored in `~/.huggingface`).
+                The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from
+                `diffusers-cli login` (stored in `~/.huggingface`) is used.
             revision (`str`, *optional*, defaults to `"main"`):
-                The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
-                git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
-                identifier allowed by git.
+                The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier
+                allowed by Git.
             subfolder (`str`, *optional*, defaults to `""`):
-                In case the relevant files are located inside a subfolder of the model repo (either remote in
-                huggingface.co or downloaded locally), you can specify the folder name here.
+                The subfolder location of a model file within a larger model repository on the Hub or locally.
             return_unused_kwargs (`bool`, *optional*, defaults to `False):
-                Whether unused keyword arguments of the config shall be returned.
+                Whether unused keyword arguments of the config are returned.
             return_commit_hash (`bool`, *optional*, defaults to `False):
-                Whether the commit_hash of the loaded configuration shall be returned.
-
-        <Tip>
+                Whether the `commit_hash` of the loaded configuration are returned.
 
-         It is required to be logged in (`huggingface-cli login`) when you want to use private or [gated
-         models](https://huggingface.co/docs/hub/models-gated#gated-models).
-
-        </Tip>
+        Returns:
+            `dict`:
+                A dictionary of all the parameters stored in a JSON configuration file.
 
         <Tip>
 
-        Activate the special ["offline-mode"](https://huggingface.co/transformers/installation.html#offline-mode) to
-        use this method in a firewalled environment.
+        To use private or [gated models](https://huggingface.co/docs/hub/models-gated#gated-models), log-in with
+        `huggingface-cli login`. You can also activate the special
+        ["offline-mode"](https://huggingface.co/transformers/installation.html#offline-mode) to use this method in a
+        firewalled environment.
 
         </Tip>
         """

src/diffusers/pipelines/pipeline_utils.py

@@ -1111,95 +1111,78 @@ class DiffusionPipeline(ConfigMixin):
     @classmethod
     def download(cls, pretrained_model_name, **kwargs) -> Union[str, os.PathLike]:
         r"""
-        Download and cache a PyTorch diffusion pipeline from pre-trained pipeline weights.
+        Download and cache a PyTorch diffusion pipeline from pretrained pipeline weights.
 
         Parameters:
             pretrained_model_name (`str` or `os.PathLike`, *optional*):
-                Should be a string, the *repo id* of a pretrained pipeline hosted inside a model repo on
-                https://huggingface.co/ Valid repo ids have to be located under a user or organization name, like
-                `CompVis/ldm-text2im-large-256`.
+                A string, the repository id (for example `CompVis/ldm-text2im-large-256`) of a pretrained pipeline
+                hosted on the Hub.
             custom_pipeline (`str`, *optional*):
-
-                <Tip warning={true}>
-
-                    This is an experimental feature and is likely to change in the future.
-
-                </Tip>
-
                 Can be either:
 
-                    - A string, the *repo id* of a custom pipeline hosted inside a model repo on
-                      https://huggingface.co/. Valid repo ids have to be located under a user or organization name,
-                      like `hf-internal-testing/diffusers-dummy-pipeline`.
-
-                        <Tip>
-
-                         It is required that the model repo has a file, called `pipeline.py` that defines the custom
-                         pipeline.
-
-                        </Tip>
+                    - A string, the repository id (for example `CompVis/ldm-text2im-large-256`) of a pretrained
+                      pipeline hosted on the Hub. The repository must contain a file called `pipeline.py` that defines
+                      the custom pipeline.
 
                     - A string, the *file name* of a community pipeline hosted on GitHub under
-                      https://github.com/huggingface/diffusers/tree/main/examples/community. Valid file names have to
-                      match exactly the file name without `.py` located under the above link, *e.g.*
-                      `clip_guided_stable_diffusion`.
+                      [Community](https://github.com/huggingface/diffusers/tree/main/examples/community). Valid file
+                      names must match the file name and not the pipeline script (`clip_guided_stable_diffusion`
+                      instead of `clip_guided_stable_diffusion.py`). Community pipelines are always loaded from the
+                      current `main` branch of GitHub.
 
-                        <Tip>
-
-                         Community pipelines are always loaded from the current `main` branch of GitHub.
-
-                        </Tip>
-
-                    - A path to a *directory* containing a custom pipeline, e.g., `./my_pipeline_directory/`.
+                    - A path to a *directory* (`./my_pipeline_directory/`) containing a custom pipeline. The directory
+                      must contain a file called `pipeline.py` that defines the custom pipeline.
 
-                        <Tip>
+                <Tip warning={true}>
 
-                         It is required that the directory has a file, called `pipeline.py` that defines the custom
-                         pipeline.
+                🧪 This is an experimental feature and may change in the future.
 
-                        </Tip>
+                </Tip>
 
-                For more information on how to load and create custom pipelines, please have a look at [Loading and
-                Adding Custom
-                Pipelines](https://huggingface.co/docs/diffusers/using-diffusers/custom_pipeline_overview)
+                For more information on how to load and create custom pipelines, take a look at [How to contribute a
+                community pipeline](https://huggingface.co/docs/diffusers/main/en/using-diffusers/contribute_pipeline).
 
             force_download (`bool`, *optional*, defaults to `False`):
                 Whether or not to force the (re-)download of the model weights and configuration files, overriding the
                 cached versions if they exist.
             resume_download (`bool`, *optional*, defaults to `False`):
-                Whether or not to delete incompletely received files. Will attempt to resume the download if such a
-                file exists.
+                Whether or not to resume downloading the model weights and configuration files. If set to False, any
+                incompletely downloaded files are deleted.
             proxies (`Dict[str, str]`, *optional*):
-                A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128',
+                A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128',
                 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
             output_loading_info(`bool`, *optional*, defaults to `False`):
                 Whether or not to also return a dictionary containing missing keys, unexpected keys and error messages.
             local_files_only(`bool`, *optional*, defaults to `False`):
-                Whether or not to only look at local files (i.e., do not try to download the model).
+                Whether to only load local model weights and configuration files or not. If set to True, the model
+                won’t be downloaded from the Hub.
             use_auth_token (`str` or *bool*, *optional*):
-                The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
-                when running `huggingface-cli login` (stored in `~/.huggingface`).
+                The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from
+                `diffusers-cli login` (stored in `~/.huggingface`) is used.
             revision (`str`, *optional*, defaults to `"main"`):
-                The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
-                git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
-                identifier allowed by git.
+                The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier
+                allowed by Git.
             custom_revision (`str`, *optional*, defaults to `"main"` when loading from the Hub and to local version of
             `diffusers` when loading from GitHub):
                 The specific model version to use. It can be a branch name, a tag name, or a commit id similar to
                 `revision` when loading a custom pipeline from the Hub. It can be a diffusers version when loading a
                 custom pipeline from GitHub.
             mirror (`str`, *optional*):
-                Mirror source to accelerate downloads in China. If you are from China and have an accessibility
-                problem, you can set this option to resolve it. Note that we do not guarantee the timeliness or safety.
-                Please refer to the mirror site for more information. specify the folder name here.
+                Mirror source to resolve accessibility issues if you're downloading a model in China. We do not
+                guarantee the timeliness or safety of the source, and you should refer to the mirror site for more
+                information.
             variant (`str`, *optional*):
-                If specified load weights from `variant` filename, *e.g.* pytorch_model.<variant>.bin. `variant` is
-                ignored when using `from_flax`.
+                Load weights from a specified variant filename such as `"fp16"` or `"ema"`. This is ignored when
+                loading `from_flax`.
+
+        Returns:
+            `os.PathLike`:
+                A path to the downloaded pipeline.
 
         <Tip>
 
-         It is required to be logged in (`huggingface-cli login`) when you want to use private or [gated
-         models](https://huggingface.co/docs/hub/models-gated#gated-models)
+        To use private or [gated models](https://huggingface.co/docs/hub/models-gated#gated-models), log-in with
+        `huggingface-cli login`.
 
         </Tip>
 

src/diffusers/utils/pil_utils.py

@@ -23,6 +23,9 @@ else:
 
 
 def pt_to_pil(images):
+    """
+    Convert a torch image to a PIL image.
+    """
     images = (images / 2 + 0.5).clamp(0, 1)
     images = images.cpu().permute(0, 2, 3, 1).float().numpy()
     images = numpy_to_pil(images)

src/diffusers/utils/testing_utils.py

@@ -261,12 +261,14 @@ def load_pt(url: str):
 
 def load_image(image: Union[str, PIL.Image.Image]) -> PIL.Image.Image:
     """
-    Args:
     Loads `image` to a PIL Image.
+
+    Args:
         image (`str` or `PIL.Image.Image`):
             The image to convert to the PIL Image format.
     Returns:
-        `PIL.Image.Image`: A PIL Image.
+        `PIL.Image.Image`:
+            A PIL Image.
     """
     if isinstance(image, str):
         if image.startswith("http://") or image.startswith("https://"):

src/diffusers/utils/torch_utils.py

@@ -40,9 +40,9 @@ def randn_tensor(
     dtype: Optional["torch.dtype"] = None,
     layout: Optional["torch.layout"] = None,
 ):
-    """This is a helper function that allows to create random tensors on the desired `device` with the desired `dtype`. When
-    passing a list of generators one can seed each batched size individually. If CPU generators are passed the tensor
-    will always be created on CPU.
+    """A helper function to create random tensors on the desired `device` with the desired `dtype`. When
+    passing a list of generators, you can seed each batch size individually. If CPU generators are passed, the tensor
+    is always created on the CPU.
     """
     # device on which tensor is created defaults to device
     rand_device = device