update codes

.gitignore

+*.egg-info
+build/
+./dist/
+*.pyc
+dist/
+*.data
+*.log
+*.tar
+*.tar.gz
+*.zip
+docs/site
+__pycache__
\ No newline at end of file

LICENSE

+Copyright (c) 2022 Robin Rombach and Patrick Esser and contributors
+
+CreativeML Open RAIL-M
+dated August 22, 2022
+
+Section I: PREAMBLE
+
+Multimodal generative models are being widely adopted and used, and have the potential to transform the way artists, among other individuals, conceive and benefit from AI or ML technologies as a tool for content creation.
+
+Notwithstanding the current and potential benefits that these artifacts can bring to society at large, there are also concerns about potential misuses of them, either due to their technical limitations or ethical considerations.
+
+In short, this license strives for both the open and responsible downstream use of the accompanying model. When it comes to the open character, we took inspiration from open source permissive licenses regarding the grant of IP rights. Referring to the downstream responsible use, we added use-based restrictions not permitting the use of the Model in very specific scenarios, in order for the licensor to be able to enforce the license in case potential misuses of the Model may occur. At the same time, we strive to promote open and responsible research on generative models for art and content generation.
+
+Even though downstream derivative versions of the model could be released under different licensing terms, the latter will always have to include - at minimum - the same use-based restrictions as the ones in the original license (this license). We believe in the intersection between open and responsible AI development; thus, this License aims to strike a balance between both in order to enable responsible open-science in the field of AI.
+
+This License governs the use of the model (and its derivatives) and is informed by the model card associated with the model.
+
+NOW THEREFORE, You and Licensor agree as follows:
+
+1. Definitions
+
+- "License" means the terms and conditions for use, reproduction, and Distribution as defined in this document.
+- "Data" means a collection of information and/or content extracted from the dataset used with the Model, including to train, pretrain, or otherwise evaluate the Model. The Data is not licensed under this License.
+- "Output" means the results of operating a Model as embodied in informational content resulting therefrom.
+- "Model" means any accompanying machine-learning based assemblies (including checkpoints), consisting of learnt weights, parameters (including optimizer states), corresponding to the model architecture as embodied in the Complementary Material, that have been trained or tuned, in whole or in part on the Data, using the Complementary Material.
+- "Derivatives of the Model" means all modifications to the Model, works based on the Model, or any other model which is created or initialized by transfer of patterns of the weights, parameters, activations or output of the Model, to the other model, in order to cause the other model to perform similarly to the Model, including - but not limited to - distillation methods entailing the use of intermediate data representations or methods based on the generation of synthetic data by the Model for training the other model.
+- "Complementary Material" means the accompanying source code and scripts used to define, run, load, benchmark or evaluate the Model, and used to prepare data for training or evaluation, if any. This includes any accompanying documentation, tutorials, examples, etc, if any.
+- "Distribution" means any transmission, reproduction, publication or other sharing of the Model or Derivatives of the Model to a third party, including providing the Model as a hosted service made available by electronic or other remote means - e.g. API-based or web access.
+- "Licensor" means the copyright owner or entity authorized by the copyright owner that is granting the License, including the persons or entities that may have rights in the Model and/or distributing the Model.
+- "You" (or "Your") means an individual or Legal Entity exercising permissions granted by this License and/or making use of the Model for whichever purpose and in any field of use, including usage of the Model in an end-use application - e.g. chatbot, translator, image generator.
+- "Third Parties" means individuals or legal entities that are not under common control with Licensor or You.
+- "Contribution" means any work of authorship, including the original version of the Model and any modifications or additions to that Model or Derivatives of the Model thereof, that is intentionally submitted to Licensor for inclusion in the Model by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Model, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution."
+- "Contributor" means Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Model.
+
+Section II: INTELLECTUAL PROPERTY RIGHTS
+
+Both copyright and patent grants apply to the Model, Derivatives of the Model and Complementary Material. The Model and Derivatives of the Model are subject to additional terms as described in Section III.
+
+2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare, publicly display, publicly perform, sublicense, and distribute the Complementary Material, the Model, and Derivatives of the Model.
+3. Grant of Patent License. Subject to the terms and conditions of this License and where and as applicable, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this paragraph) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Model and the Complementary Material, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Model to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Model and/or Complementary Material or a Contribution incorporated within the Model and/or Complementary Material constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for the Model and/or Work shall terminate as of the date such litigation is asserted or filed.
+
+Section III: CONDITIONS OF USAGE, DISTRIBUTION AND REDISTRIBUTION
+
+4. Distribution and Redistribution. You may host for Third Party remote access purposes (e.g. software-as-a-service), reproduce and distribute copies of the Model or Derivatives of the Model thereof in any medium, with or without modifications, provided that You meet the following conditions:
+Use-based restrictions as referenced in paragraph 5 MUST be included as an enforceable provision by You in any type of legal agreement (e.g. a license) governing the use and/or distribution of the Model or Derivatives of the Model, and You shall give notice to subsequent users You Distribute to, that the Model or Derivatives of the Model are subject to paragraph 5. This provision does not apply to the use of Complementary Material.
+You must give any Third Party recipients of the Model or Derivatives of the Model a copy of this License;
+You must cause any modified files to carry prominent notices stating that You changed the files;
+You must retain all copyright, patent, trademark, and attribution notices excluding those notices that do not pertain to any part of the Model, Derivatives of the Model.
+You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions - respecting paragraph 4.a. - for use, reproduction, or Distribution of Your modifications, or for any such Derivatives of the Model as a whole, provided Your use, reproduction, and Distribution of the Model otherwise complies with the conditions stated in this License.
+5. Use-based restrictions. The restrictions set forth in Attachment A are considered Use-based restrictions. Therefore You cannot use the Model and the Derivatives of the Model for the specified restricted uses. You may use the Model subject to this License, including only for lawful purposes and in accordance with the License. Use may include creating any content with, finetuning, updating, running, training, evaluating and/or reparametrizing the Model. You shall require all of Your users who use the Model or a Derivative of the Model to comply with the terms of this paragraph (paragraph 5).
+6. The Output You Generate. Except as set forth herein, Licensor claims no rights in the Output You generate using the Model. You are accountable for the Output you generate and its subsequent uses. No use of the output can contravene any provision as stated in the License.
+
+Section IV: OTHER PROVISIONS
+
+7. Updates and Runtime Restrictions. To the maximum extent permitted by law, Licensor reserves the right to restrict (remotely or otherwise) usage of the Model in violation of this License, update the Model through electronic means, or modify the Output of the Model based on updates. You shall undertake reasonable efforts to use the latest version of the Model.
+8. Trademarks and related. Nothing in this License permits You to make use of Licensors’ trademarks, trade names, logos or to otherwise suggest endorsement or misrepresent the relationship between the parties; and any rights not expressly granted herein are reserved by the Licensors.
+9. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Model and the Complementary Material (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Model, Derivatives of the Model, and the Complementary Material and assume any risks associated with Your exercise of permissions under this License.
+10. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Model and the Complementary Material (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.
+11. Accepting Warranty or Additional Liability. While redistributing the Model, Derivatives of the Model and the Complementary Material thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
+12. If any provision of this License is held to be invalid, illegal or unenforceable, the remaining provisions shall be unaffected thereby and remain valid as if such provision had not been set forth herein.
+
+END OF TERMS AND CONDITIONS
+
+
+
+
+Attachment A
+
+Use Restrictions
+
+You agree not to use the Model or Derivatives of the Model:
+- In any way that violates any applicable national, federal, state, local or international law or regulation;
+- For the purpose of exploiting, harming or attempting to exploit or harm minors in any way;
+- To generate or disseminate verifiably false information and/or content with the purpose of harming others;
+- To generate or disseminate personal identifiable information that can be used to harm an individual;
+- To defame, disparage or otherwise harass others;
+- For fully automated decision making that adversely impacts an individual’s legal rights or otherwise creates or modifies a binding, enforceable obligation;
+- For any use intended to or which has the effect of discriminating against or harming individuals or groups based on online or offline social behavior or known or predicted personal or personality characteristics;
+- To exploit any of the vulnerabilities of a specific group of persons based on their age, social, physical or mental characteristics, in order to materially distort the behavior of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological harm;
+- For any use intended to or which has the effect of discriminating against individuals or groups based on legally protected characteristics or categories;
+- To provide medical advice and medical results interpretation;
+- To generate or disseminate information for the purpose to be used for administration of justice, law enforcement, immigration or asylum processes, such as predicting an individual will commit fraud/crime commitment (e.g. by text profiling, drawing causal relationships between assertions made in documents, indiscriminate and arbitrarily-targeted use).
\ No newline at end of file

README.md

-# ImageGenerate_paddle
+# ImageGenerate by paddle 
+## 模型介绍
+文图生成模型-【使用文生图、图生图进行图像的生成】
+主要分享如何定制自己的文图生成模型，只需要几张图片，即可定制开发自己想要的文图生成模型。
+文图生成任务要求模型根据所提供的描述性文本生成一张与之相对应的图片。这极大地释放了AI的想象力，也激发了人类的创意，给视觉内容创作者、文字内容创作者和大众用户带来了方便。用户可以生成多样化创意图片，并从中汲取创意灵感，打破创意瓶颈，从而可以进行创作出更优质的作品。
+## 模型结构
+文生图、图生图使用模型[MoososCap/NOVEL-MODEL]：  
+默认使用该模型进行文生图、图生图等功能实现，也可切换其他模型
+- MoososCap/NOVEL-MODEL:  
+    - feature_extractor/preprocessor_config.json  
+    - model_index.json  
+    - scheduler/scheduler_config.json  
+    - text_encoder/[model_config.json,model_state.pdparams] 
+    - tokenizer/[added_tokens.json,merges.txt,special_tokens_map.json,tokenizer_config.json,vocab.json]  
+    - unet/[config.json,model_state.pdparams] 
+    - vae/[config.json, model_state.pdparams]  
 
-A simple way to generate image by typing text or add image, also easy for training custom's key word to generate custom's images elements.
+超分模型:  
\ No newline at end of file
+falsr_a,falsr_b,falsr_c,falsr 三个超分模型  
+[论文地址](https://arxiv.org/abs/1901.07261)  
+![模型图](./resources/pic_readme/falsr_network.png)
+
+## 推理和训练
+### 环境配置
+提供[光源](https://www.sourcefind.cn/#/service-details)拉取的训练以及推理的docker镜像：
+* 推理和训练镜像：
+
+* python依赖安装：
+```bash
+    pip install -r requirement.txt  
+```
+### 测试
+测试命令：
+- 测试环境、功能是否够能够正常
+```python
+python varify.py 
+```
+
+### 推理\训练代码
+【使用jupyter进行推理】  
+命令-jupyter：
+- 运行main.ipynb
+- 根据提示步骤运行推理、训练代码
+
+### 预训练模型
+预训练模型采用[MoososCap/NOVEL-MODEL]
+- MoososCap/NOVEL-MODEL:  
+    - feature_extractor/preprocessor_config.json  
+    - model_index.json  
+    - scheduler/scheduler_config.json  
+    - text_encoder/[model_config.json,model_state.pdparams] 
+    - tokenizer/[added_tokens.json,merges.txt,special_tokens_map.json,tokenizer_config.json,vocab.json]  
+    - unet/[config.json,model_state.pdparams] 
+    - vae/[config.json, model_state.pdparams]  
+
+## 生成效果 
+
+ | 文生图  | 图生图  | 定制化训练 |
+ | :----: | :-----: | :------:  |
+ ![文生图](./resources/pic_readme/txt2img.jpg)  | ![图生图](./resources/pic_readme/img2img.jpg) | ![定制化训练](./resources/pic_readme/train.jpg)   
+  
+
+
+
+## 源码仓库及问题反馈
+* https://developer.hpccube.com/codes/modelzoo/imagegenerate_paddle
+## 参考
+* https://aistudio.baidu.com/aistudio/projectdetail/4905623

main.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2022-10-27T03:28:46.042713Z",
+     "iopub.status.busy": "2022-10-27T03:28:46.041900Z",
+     "iopub.status.idle": "2022-10-27T03:28:46.056473Z",
+     "shell.execute_reply": "2022-10-27T03:28:46.055255Z",
+     "shell.execute_reply.started": "2022-10-27T03:28:46.042679Z"
+    },
+    "tags": []
+   },
+   "source": [
+    "# 想定制自己的文图生成模型吗？想画什么画什么\n",
+    "\n",
+    "**文图生成有多火，已经不用介绍了。今天主要来分享如何定制自己的文图生成模型，只需要几张图片，即可定制开发自己想要的文图生成模型哦。有问题的话，文末有交流群，欢迎加入！**\n",
+    "\n",
+    "文图生成任务要求模型根据所提供的描述性文本生成一张与之相对应的图片。这极大地释放了AI的想象力，也激发了人类的创意，给视觉内容创作者、文字内容创作者和大众用户带来了方便。用户可以生成多样化创意图片，并从中汲取创意灵感，打破创意瓶颈，从而可以进行创作出更优质的作品。\n",
+    "\n",
+    "## 感谢 @风飏 开发者开发的UI界面！\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 一、Fork项目，进入 Notebook！\n",
+    "\n",
+    "点击“运行一下”后，再点击“启动环境\"，选择合适的GPU后即可进入项目。 AI Studio每天自动**赠送 8 小时**的GPU算力，显存更大的GPU能够生成尺寸更大的图片哦。\n",
+    "<div>\n",
+    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/9319c725e55445458e0536a254e0825fa09f332bef524855899801ba0b12879b\" width=\"40%\" height=\"40%\"> \n",
+    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/d07ebc951af14889bbb100e2714fe5c7203f0d2f2cf94f4680146d612e7e9d81\" width=\"40%\" height=\"40%\">\n",
+    "</div>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 二、运行下面的代码，并且重启内核！\n",
+    "\n",
+    "进入之后，点击下边的框里左上角的“点击运行”按钮（或者点进下面的框内用快捷键 Ctrl + Enter）。\n",
+    "\n",
+    "**提示**：下面安装环境的代码，只需要在你**第一次进入本项目**时运行！\n",
+    "\n",
+    "等到显示“加载完毕, 请重启内核”后，请重启内核。\n",
+    "<div>\n",
+    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/12bb4a5a24fa4111b70b85aa89d64bceb7c98ea24fa54d0dba32c0e8610b412d\" width=\"40%\" height=\"40%\"> \n",
+    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/d9487d2c0b9d4467ad34a96abd60d6349802ced3ca2d483cb76bffe8415a24aa\" width=\"40%\" height=\"40%\">\n",
+    "</div>\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2023-01-12T14:35:14.706990Z",
+     "iopub.status.busy": "2023-01-12T14:35:14.706668Z",
+     "iopub.status.idle": "2023-01-12T14:35:19.300615Z",
+     "shell.execute_reply": "2023-01-12T14:35:19.299887Z",
+     "shell.execute_reply.started": "2023-01-12T14:35:14.706966Z"
+    },
+    "scrolled": true,
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "加载完毕, 请重启内核\r\n"
+     ]
+    }
+   ],
+   "source": [
+    "from IPython.display import clear_output\n",
+    "from ui.utils import diffusers_auto_update\n",
+    "diffusers_auto_update()\n",
+    "clear_output() # 清理很长的内容\n",
+    "print('加载完毕, 请重启内核')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "# 三、运行下面的代码快速体验\n",
+    "> 最后一步，点击左上角的“点击运行”后，就会自动运行下面的代码，等几秒加载模型就可以玩耍啦~ **以后每次进来这个项目，就可以直接从这里开始运行啦~**\n",
+    "\n",
+    "**下面推荐了部分二次元模型，想要了解更多的模型可以 [点开这里的链接](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/ppdiffusers#ppdiffusers%E6%A8%A1%E5%9E%8B%E6%94%AF%E6%8C%81%E7%9A%84%E6%9D%83%E9%87%8D)！**\n",
+    "\n",
+    "| ppdiffusers支持的模型名称    | huggingface对应的模型地址                           | Tips备注                                                     |\n",
+    "| ---------------------------------------- | ---------------------------------------------------------- | -------------------------------------------------------------- |\n",
+    "| **Linaqruf/anything-v3.0**       （推荐！） | https://huggingface.co/Linaqruf/anything-v3.0 | 二次元模型！ |\n",
+    "| **hakurei/waifu-diffusion-v1-3**         （推荐！）            | https://huggingface.co/hakurei/waifu-diffusion             | Waifu v1-3的模型，主要适合画二次元图像！（对比v1-2更好！）         |\n",
+    "| **MoososCap/NOVEL-MODEL**        （推荐！）| https://huggingface.co/MoososCap/NOVEL-MODEL | 二次元模型！ |\n",
+    "| **Baitian/momocha**       （推荐！） | 无 | 二次元模型！ |\n",
+    "| **Baitian/momoco**         （推荐！）            | 无             | 二次元模型！         |\n",
+    "| **hequanshaguo/monoko-e**         （推荐！）            | 无             | 二次元模型！         |\n",
+    "\n",
+    "> Tips:\n",
+    "！！！🔥 删除原有的模型名称后，我们可以手动输入新的模型名称时会出现智能提示！例如：我输入了any，他会自动提示出 Linaqruf/anything-v3.0 和 ruisi/anything。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3.0 在线转换 ckpt 或者 safetensors 权重\n",
+    "我们可以使用下面的**UI**，在线转换**pytorch**的**ckpt权重**，当前仅支持**SD-V1**的权重进行转换！"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2023-02-18T05:48:23.313309Z",
+     "iopub.status.busy": "2023-02-18T05:48:23.312544Z",
+     "iopub.status.idle": "2023-02-18T05:48:23.324141Z",
+     "shell.execute_reply": "2023-02-18T05:48:23.323413Z",
+     "shell.execute_reply.started": "2023-02-18T05:48:23.313249Z"
+    },
+    "scrolled": true,
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "a1e71a804ae548cd8095dd4596ba69e5",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Box(children=(Box(children=(Text(value='', description='ckpt或safetensors模型文件位置', description_tooltip='你要转换的模型位…"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from ui import gui_convert # 点开始可能没反应，多等一会就好了，千万别多点\n",
+    "display(gui_convert.gui)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3.1 文生图UI使用\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2023-02-18T05:49:03.271708Z",
+     "iopub.status.busy": "2023-02-18T05:49:03.271128Z",
+     "iopub.status.idle": "2023-02-18T05:49:03.299380Z",
+     "shell.execute_reply": "2023-02-18T05:49:03.298796Z",
+     "shell.execute_reply.started": "2023-02-18T05:49:03.271665Z"
+    },
+    "scrolled": true,
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "f6990bc9e654499ca46aa3c0e379e9fd",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Box(children=(HTML(value='<style>\\n@media (max-width:576px) {\\n    .StableDiffusionUI_txt2imgF651 {\\n        m…"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from ui import gui_txt2img # 文生图\n",
+    "display(gui_txt2img.gui) # 生成的图片自动保存到左侧的 outputs/txt2img 的文件夹里\n",
+    "# 记得点击“生成图片”，才能出图"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3.2 图生图UI使用\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2022-12-28T11:20:59.613296Z",
+     "iopub.status.busy": "2022-12-28T11:20:59.612695Z",
+     "iopub.status.idle": "2022-12-28T11:20:59.665567Z",
+     "shell.execute_reply": "2022-12-28T11:20:59.664643Z",
+     "shell.execute_reply.started": "2022-12-28T11:20:59.613256Z"
+    },
+    "scrolled": true,
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "56e47acaa1534a198e9255eba7b11ee6",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Box(children=(HTML(value=\"<style>\\n@media (max-width:576px) {\\n    .StableDiffusionUI_img2img91AD {\\n        m…"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from ui import gui_img2img # 图生图, 在左侧上传图片, 然后修改 \"需要转换的图片路径\"\n",
+    "display(gui_img2img.gui) # 生成的图片自动保存到左侧的 outputs/img2img 的文件夹里\n",
+    "\n",
+    "# 记得点击“生成图片”，才能出图"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3.3 超分UI使用"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2022-12-28T11:21:15.413637Z",
+     "iopub.status.busy": "2022-12-28T11:21:15.412816Z",
+     "iopub.status.idle": "2022-12-28T11:21:15.423444Z",
+     "shell.execute_reply": "2022-12-28T11:21:15.422625Z",
+     "shell.execute_reply.started": "2022-12-28T11:21:15.413590Z"
+    },
+    "scrolled": true,
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "78cafc2bdc934ba0a6e2f242852c8fe1",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Box(children=(Text(value='resources/image_Kurisu.png', description='需要超分的图片路径', layout=Layout(align_items='cen…"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from ui import gui_superres # 超分 (图片放大一倍), 在左侧“文件”目录上传图片, 然后修改 \"需要超分的图片路径\"\n",
+    "display(gui_superres.gui) # 生成的图片自动保存到左侧的 outputs/highres 的文件夹里"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3.4 定制化训练（让模型认识新的物体或学习到新的风格）\n",
+    "\n",
+    "除了直接调用外，**PaddleNLP**还提供好用的二次开发能力，可以参照**下方**或[**PaddleNLP仓库**](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/ppdiffusers/examples)的教程，只需要几张图片，就可以定制属于自己的文图生成模型。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "在下面**GIF**例子中，我们进行了如下操作，更多信息可进入[PaddleNLP examples](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/ppdiffusers/examples/textual_inversion)中查看更多例子。\n",
+    "\n",
+    "**(1)** 输入了代表该人物的**新单词**：\\<Alice\\>，新词用<>括起来，主要是为了避免与已有的单词混淆；\n",
+    "\n",
+    "**(2)** 指定了与该人物**比较接近的单词**：**girl**, 该单词作为先验（已知）知识，可以帮助模型快速理解新单词，从而加快模型训练；\n",
+    "\n",
+    "**(3)** 提供了含有下面**6张图片**的文件夹地址：**resources/Alices**，在这里我们指定了resources目录下的Alices文件夹。\n",
+    "\n",
+    "<center> <img src=\"https://ai-studio-static-online.cdn.bcebos.com/dcdfa7f8c35f4d5f9e0eeab7e590f5f4b576bb1728e94bb4a889b34d833397d2\" width=\"40%\" height=\"40%\"> <br /></center>\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2022-12-28T11:21:18.513434Z",
+     "iopub.status.busy": "2022-12-28T11:21:18.512755Z",
+     "iopub.status.idle": "2022-12-28T11:21:18.533021Z",
+     "shell.execute_reply": "2022-12-28T11:21:18.531913Z",
+     "shell.execute_reply.started": "2022-12-28T11:21:18.513398Z"
+    },
+    "scrolled": true,
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "7489b2c29d1440398723c69fec907f38",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Box(children=(Box(children=(Dropdown(description='训练目标', description_tooltip='训练目标是什么？风格还是实体？', index=1, layou…"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from ui import gui_train_text_inversion # 训练\n",
+    "display(gui_train_text_inversion.gui) # 训练结果会保存到 outputs/textual_inversion 的文件夹"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**模型训练完成之后，可以使用新模型预测了！**\n",
+    "\n",
+    "现在模型已经认识 \\<Alice\\> 这个object（人物、物体，everything...）了。\n",
+    "\n",
+    "比如输入Prompt：\\<Alice\\> at the lake，就可以生成一张站在湖边的 \\<Alice\\> 图像了。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2022-12-28T11:21:21.214243Z",
+     "iopub.status.busy": "2022-12-28T11:21:21.213559Z",
+     "iopub.status.idle": "2022-12-28T11:21:21.231119Z",
+     "shell.execute_reply": "2022-12-28T11:21:21.230250Z",
+     "shell.execute_reply.started": "2022-12-28T11:21:21.214206Z"
+    },
+    "scrolled": true
+   },
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "7489b2c29d1440398723c69fec907f38",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Box(children=(Box(children=(Dropdown(description='训练目标', description_tooltip='训练目标是什么？风格还是实体？', index=1, layou…"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from ui import gui_train_text_inversion # 使用训练好的权重进行加载预测\n",
+    "display(gui_train_text_inversion.gui) # 生成的图片自动保存到左侧的 outputs/text_inversion_txt2img 的文件夹里\n",
+    "# 输入 <Alice> 相关的Prompt试试吧，例如：<Alice> at the lake，看看模型是否学到了<Alice>这个object。记得删除默认给出的negative_prompt反面描述。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "下面是Dreambooth训练部分\n",
+    "可以参考 https://github.com/PaddlePaddle/PaddleNLP/tree/develop/ppdiffusers/examples/dreambooth 配置下面的选项\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2022-12-28T11:39:34.406174Z",
+     "iopub.status.busy": "2022-12-28T11:39:34.405796Z",
+     "iopub.status.idle": "2022-12-28T11:39:39.239954Z",
+     "shell.execute_reply": "2022-12-28T11:39:39.238856Z",
+     "shell.execute_reply.started": "2022-12-28T11:39:34.406148Z"
+    },
+    "scrolled": true,
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "0ec6a2aba73d4b099a11f65685342e5b",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Box(children=(Box(children=(Combobox(value='MoososCap/NOVEL-MODEL', description='需要训练的模型名称', layout=Layout(ali…"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from ui import gui_dreambooth #训练dreambooth\n",
+    "display(gui_dreambooth.gui) "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**现在你已经学会了如何让模型认识新的物体，模型不仅可以学习新物体还可以学习新风格。风格的学习与物体的学习方法与流程一致，我们仅需在UI界面中设置学习“style”，开始训练即可。**\n",
+    "\n",
+    "**欢迎各位贡献自己训练好的模型权重到PaddleNLP，比如\"xxx艺术家风格的模型\"，让更多开发者了解、使用自己的作品。欢迎加入交流群，了解如何贡献。**"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 四、 加入交流群，一起创造吧\n",
+    "\n",
+    "以上实现基于PaddleNLP，开源不易，希望大家多多支持~ \n",
+    "**如果对您有帮助，记得给[PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP)点个小小的Star⭐，收藏起来，不易走丢~**\n",
+    "\n",
+    "GitHub地址：[https://github.com/PaddlePaddle/PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP)\n",
+    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/7f287240b16f4c4081d0b57ca808dce62769b13369434c64b9a26cfc48b6a978\" width=\"80%\" height=\"80%\"> <br />\n",
+    "\n",
+    "- **加入交流群，一起创造吧**\n",
+    "\n",
+    "如果对NLP模型二次开发技术感兴趣，欢迎加入PaddleNLP**微信群**交流：\n",
+    "\n",
+    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/0be17ea733e341638404261e2321d4304c9fbfade5ed426389884bf8a7ff5867\" width=\"25%\" height=\"25%\"> <br />\n",
+    "\n",
+    "也可以加入**QQ群**，交流文图趣味应用。\n",
+    "\n",
+    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/de62713000a641d28c74d3174aaf517d78a4801e422f4e2788d5f890b272760b\" width=\"25%\" height=\"25%\"> <br />\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 五、 更多相关产品与服务\n",
+    "\n",
+    "## [文心一格](https://yige.baidu.com/)\n",
+    "\n",
+    "文心一格是百度推出的AI艺术和创意辅助平台，是百度依托于飞桨、文心大模型持续的技术创新，在“AI作图”方面发布的产品和技术工具。定位为面向有设计需求和创意的人群，基于文心大模型智能生成多样化AI创意图片，辅助创意设计，打破创意瓶颈。用户只需输入自己的创想文字，并选择期望的画作风格，即可快速获取由一格生成的相应画作。\n",
+    "\n",
+    "- 文心一格官网：[yige.baidu.com](https://yige.baidu.com/)\n",
+    "\n",
+    "## [文心 ERNIE-ViLG AI 作画大模型](https://wenxin.baidu.com/ernie-vilg)\n",
+    "\n",
+    "文心 ERNIE-ViLG 是目前全球规模最大的中文跨模态生成模型，可通过自然语言实现图像生成与编辑。开发者可以通过申请相关API，实现在自己的项目中添加文生图、参考图等文心ERNIE-ViLG AI作画相关能力。\n",
+    "\n",
+    "- 前往文心 ERNIE-ViLG 官网，申请API服务：[https://wenxin.baidu.com/ernie-vilg](https://wenxin.baidu.com/ernie-vilg)\n",
+    "\n",
+    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/99da398946094eb8baeb3362a63ded8dfd34e1dbb07e44ac923922b54be64810\" width=\"10%\" height=\"10%\"> <br />\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 六、参考资料\n",
+    "\n",
+    "- https://github.com/PaddlePaddle/PaddleNLP\n",
+    "- https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features\n",
+    "- https://github.com/huggingface/diffusers\n",
+    "- https://github.com/CompVis/stable-diffusion\n",
+    "- https://arxiv.org/pdf/2006.11239.pdf\n",
+    "- https://arxiv.org/pdf/2208.01618.pdf\n",
+    "- https://huggingface.co/runwayml/stable-diffusion-v1-5 (请注意里面使用的LICENSE)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 七、鸣谢\n",
+    "感谢 **@风飏** 开发者开发的UI界面！这个界面真的太好用啦！！！"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "py35-paddle1.2.0"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.4"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "02171df25bfaf2d4dd046004a91568a270b3a8e32f8815a8efc9d94f8a69beff"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

model.properties

+# 模型名称
+modelName=ImageGenerate_paddle
+# 模型描述
+modelDescription=文生图、图生图模型是一种基于深度学习的图像生成算法，可应用于艺术创作提供创作思路
+# 应用场景
+appScenario=推理,训练,CV,文生图,图生图
+# 框架类型
+frameType=paddle
\ No newline at end of file

requirement.txt

+OmegaConf
+safetensors
+ppdiffusers==0.9.0
+paddlenlp==2.4.9
+ipython==8.14.0
+ipywidgets==8.0.7
+pillow==9.5.0
+tqdm==4.65.0
+urllib3==1.26.12
\ No newline at end of file