Skip to content

GitLab

Commit aad7b6c7 authored by chenzk's avatar chenzk
Browse files

v1.0

parents
Pipeline #2416 canceled with stages
Expand all Show whitespace changes
Inline Side-by-side
Showing with 1222 additions and 0 deletions
LICENSE 0 → 100644
View file @ aad7b6c7
TENCENT HUNYUAN 3D 2.0 COMMUNITY LICENSE AGREEMENT
Tencent Hunyuan 3D 2.0 Release Date: January 21, 2025
THIS LICENSE AGREEMENT DOES NOT APPLY IN THE EUROPEAN UNION, UNITED KINGDOM AND SOUTH KOREA AND IS EXPRESSLY LIMITED TO THE TERRITORY, AS DEFINED BELOW.
By clicking to agree or by using, reproducing, modifying, distributing, performing or displaying any portion or element of the Tencent Hunyuan 3D 2.0 Works, including via any Hosted Service, You will be deemed to have recognized and accepted the content of this Agreement, which is effective immediately.
1. DEFINITIONS.
a. “Acceptable Use Policy” shall mean the policy made available by Tencent as set forth in the Exhibit A.
b. “Agreement” shall mean the terms and conditions for use, reproduction, distribution, modification, performance and displaying of Tencent Hunyuan 3D 2.0 Works or any portion or element thereof set forth herein.
c. “Documentation” shall mean the specifications, manuals and documentation for Tencent Hunyuan 3D 2.0 made publicly available by Tencent.
d. “Hosted Service” shall mean a hosted service offered via an application programming interface (API), web access, or any other electronic or remote means.
e. “Licensee,” “You” or “Your” shall mean a natural person or legal entity exercising the rights granted by this Agreement and/or using the Tencent Hunyuan 3D 2.0 Works for any purpose and in any field of use.
f. “Materials” shall mean, collectively, Tencent’s proprietary Tencent Hunyuan 3D 2.0 and Documentation (and any portion thereof) as made available by Tencent under this Agreement.
g. “Model Derivatives” shall mean all: (i) modifications to Tencent Hunyuan 3D 2.0 or any Model Derivative of Tencent Hunyuan 3D 2.0; (ii) works based on Tencent Hunyuan 3D 2.0 or any Model Derivative of Tencent Hunyuan 3D 2.0; or (iii) any other machine learning model which is created by transfer of patterns of the weights, parameters, operations, or Output of Tencent Hunyuan 3D 2.0 or any Model Derivative of Tencent Hunyuan 3D 2.0, to that model in order to cause that model to perform similarly to Tencent Hunyuan 3D 2.0 or a Model Derivative of Tencent Hunyuan 3D 2.0, including distillation methods, methods that use intermediate data representations, or methods based on the generation of synthetic data Outputs by Tencent Hunyuan 3D 2.0 or a Model Derivative of Tencent Hunyuan 3D 2.0 for training that model. For clarity, Outputs by themselves are not deemed Model Derivatives.
h. “Output” shall mean the information and/or content output of Tencent Hunyuan 3D 2.0 or a Model Derivative that results from operating or otherwise using Tencent Hunyuan 3D 2.0 or a Model Derivative, including via a Hosted Service.
i. “Tencent,” “We” or “Us” shall mean THL A29 Limited.
j. “Tencent Hunyuan 3D 2.0” shall mean the 3D generation models and their software and algorithms, including trained model weights, parameters (including optimizer states), machine-learning model code, inference-enabling code, training-enabling code, fine-tuning enabling code and other elements of the foregoing made publicly available by Us at https://github.com/Tencent/Hunyuan3D-2.
k. “Tencent Hunyuan 3D 2.0 Works” shall mean: (i) the Materials; (ii) Model Derivatives; and (iii) all derivative works thereof.
l. “Territory” shall mean the worldwide territory, excluding the territory of the European Union, United Kingdom and South Korea.
m. “Third Party” or “Third Parties” shall mean individuals or legal entities that are not under common control with Us or You.
n. “including” shall mean including but not limited to.
2. GRANT OF RIGHTS.
We grant You, for the Territory only, a non-exclusive, non-transferable and royalty-free limited license under Tencent’s intellectual property or other rights owned by Us embodied in or utilized by the Materials to use, reproduce, distribute, create derivative works of (including Model Derivatives), and make modifications to the Materials, only in accordance with the terms of this Agreement and the Acceptable Use Policy, and You must not violate (or encourage or permit anyone else to violate) any term of this Agreement or the Acceptable Use Policy.
3. DISTRIBUTION.
You may, subject to Your compliance with this Agreement, distribute or make available to Third Parties the Tencent Hunyuan 3D 2.0 Works, exclusively in the Territory, provided that You meet all of the following conditions:
a. You must provide all such Third Party recipients of the Tencent Hunyuan 3D 2.0 Works or products or services using them a copy of this Agreement;
b. You must cause any modified files to carry prominent notices stating that You changed the files;
c. You are encouraged to: (i) publish at least one technology introduction blogpost or one public statement expressing Your experience of using the Tencent Hunyuan 3D 2.0 Works; and (ii) mark the products or services developed by using the Tencent Hunyuan 3D 2.0 Works to indicate that the product/service is “Powered by Tencent Hunyuan”; and
d. All distributions to Third Parties (other than through a Hosted Service) must be accompanied by a “Notice” text file that contains the following notice: “Tencent Hunyuan 3D 2.0 is licensed under the Tencent Hunyuan 3D 2.0 Community License Agreement, Copyright © 2025 Tencent. All Rights Reserved. The trademark rights of “Tencent Hunyuan” are owned by Tencent or its affiliate.”
You may add Your own copyright statement to Your modifications and, except as set forth in this Section and in Section 5, may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Model Derivatives as a whole, provided Your use, reproduction, modification, distribution, performance and display of the work otherwise complies with the terms and conditions of this Agreement (including as regards the Territory). If You receive Tencent Hunyuan 3D 2.0 Works from a Licensee as part of an integrated end user product, then this Section 3 of this Agreement will not apply to You.
4. ADDITIONAL COMMERCIAL TERMS.
If, on the Tencent Hunyuan 3D 2.0 version release date, the monthly active users of all products or services made available by or for Licensee is greater than 1 million monthly active users in the preceding calendar month, You must request a license from Tencent, which Tencent may grant to You in its sole discretion, and You are not authorized to exercise any of the rights under this Agreement unless or until Tencent otherwise expressly grants You such rights.
Subject to Tencent's written approval, you may request a license for the use of Tencent Hunyuan 3D 2.0 by submitting the following information to hunyuan3d@tencent.com:
a. Your company’s name and associated business sector that plans to use Tencent Hunyuan 3D 2.0.
b. Your intended use case and the purpose of using Tencent Hunyuan 3D 2.0.
5. RULES OF USE.
a. Your use of the Tencent Hunyuan 3D 2.0 Works must comply with applicable laws and regulations (including trade compliance laws and regulations) and adhere to the Acceptable Use Policy for the Tencent Hunyuan 3D 2.0 Works, which is hereby incorporated by reference into this Agreement. You must include the use restrictions referenced in these Sections 5(a) and 5(b) as an enforceable provision in any agreement (e.g., license agreement, terms of use, etc.) governing the use and/or distribution of Tencent Hunyuan 3D 2.0 Works and You must provide notice to subsequent users to whom You distribute that Tencent Hunyuan 3D 2.0 Works are subject to the use restrictions in these Sections 5(a) and 5(b).
b. You must not use the Tencent Hunyuan 3D 2.0 Works or any Output or results of the Tencent Hunyuan 3D 2.0 Works to improve any other AI model (other than Tencent Hunyuan 3D 2.0 or Model Derivatives thereof).
c. You must not use, reproduce, modify, distribute, or display the Tencent Hunyuan 3D 2.0 Works, Output or results of the Tencent Hunyuan 3D 2.0 Works outside the Territory. Any such use outside the Territory is unlicensed and unauthorized under this Agreement.
6. INTELLECTUAL PROPERTY.
a. Subject to Tencent’s ownership of Tencent Hunyuan 3D 2.0 Works made by or for Tencent and intellectual property rights therein, conditioned upon Your compliance with the terms and conditions of this Agreement, as between You and Tencent, You will be the owner of any derivative works and modifications of the Materials and any Model Derivatives that are made by or for You.
b. No trademark licenses are granted under this Agreement, and in connection with the Tencent Hunyuan 3D 2.0 Works, Licensee may not use any name or mark owned by or associated with Tencent or any of its affiliates, except as required for reasonable and customary use in describing and distributing the Tencent Hunyuan 3D 2.0 Works. Tencent hereby grants You a license to use “Tencent Hunyuan” (the “Mark”) in the Territory solely as required to comply with the provisions of Section 3(c), provided that You comply with any applicable laws related to trademark protection. All goodwill arising out of Your use of the Mark will inure to the benefit of Tencent.
c. If You commence a lawsuit or other proceedings (including a cross-claim or counterclaim in a lawsuit) against Us or any person or entity alleging that the Materials or any Output, or any portion of any of the foregoing, infringe any intellectual property or other right owned or licensable by You, then all licenses granted to You under this Agreement shall terminate as of the date such lawsuit or other proceeding is filed. You will defend, indemnify and hold harmless Us from and against any claim by any Third Party arising out of or related to Your or the Third Party’s use or distribution of the Tencent Hunyuan 3D 2.0 Works.
d. Tencent claims no rights in Outputs You generate. You and Your users are solely responsible for Outputs and their subsequent uses.
7. DISCLAIMERS OF WARRANTY AND LIMITATIONS OF LIABILITY.
a. We are not obligated to support, update, provide training for, or develop any further version of the Tencent Hunyuan 3D 2.0 Works or to grant any license thereto.
b. UNLESS AND ONLY TO THE EXTENT REQUIRED BY APPLICABLE LAW, THE TENCENT HUNYUAN 3D 2.0 WORKS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED “AS IS” WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES OF ANY KIND INCLUDING ANY WARRANTIES OF TITLE, MERCHANTABILITY, NONINFRINGEMENT, COURSE OF DEALING, USAGE OF TRADE, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING, REPRODUCING, MODIFYING, PERFORMING, DISPLAYING OR DISTRIBUTING ANY OF THE TENCENT HUNYUAN 3D 2.0 WORKS OR OUTPUTS AND ASSUME ANY AND ALL RISKS ASSOCIATED WITH YOUR OR A THIRD PARTY’S USE OR DISTRIBUTION OF ANY OF THE TENCENT HUNYUAN 3D 2.0 WORKS OR OUTPUTS AND YOUR EXERCISE OF RIGHTS AND PERMISSIONS UNDER THIS AGREEMENT.
c. TO THE FULLEST EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL TENCENT OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, FOR ANY DAMAGES, INCLUDING ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, EXEMPLARY, CONSEQUENTIAL OR PUNITIVE DAMAGES, OR LOST PROFITS OF ANY KIND ARISING FROM THIS AGREEMENT OR RELATED TO ANY OF THE TENCENT HUNYUAN 3D 2.0 WORKS OR OUTPUTS, EVEN IF TENCENT OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
8. SURVIVAL AND TERMINATION.
a. The term of this Agreement shall commence upon Your acceptance of this Agreement or access to the Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein.
b. We may terminate this Agreement if You breach any of the terms or conditions of this Agreement. Upon termination of this Agreement, You must promptly delete and cease use of the Tencent Hunyuan 3D 2.0 Works. Sections 6(a), 6(c), 7 and 9 shall survive the termination of this Agreement.
9. GOVERNING LAW AND JURISDICTION.
a. This Agreement and any dispute arising out of or relating to it will be governed by the laws of the Hong Kong Special Administrative Region of the People’s Republic of China, without regard to conflict of law principles, and the UN Convention on Contracts for the International Sale of Goods does not apply to this Agreement.
b. Exclusive jurisdiction and venue for any dispute arising out of or relating to this Agreement will be a court of competent jurisdiction in the Hong Kong Special Administrative Region of the People’s Republic of China, and Tencent and Licensee consent to the exclusive jurisdiction of such court with respect to any such dispute.
 
EXHIBIT A
ACCEPTABLE USE POLICY
Tencent reserves the right to update this Acceptable Use Policy from time to time.
Last modified: November 5, 2024
Tencent endeavors to promote safe and fair use of its tools and features, including Tencent Hunyuan 3D 2.0. You agree not to use Tencent Hunyuan 3D 2.0 or Model Derivatives:
1. Outside the Territory;
2. In any way that violates any applicable national, federal, state, local, international or any other law or regulation;
3. To harm Yourself or others;
4. To repurpose or distribute output from Tencent Hunyuan 3D 2.0 or any Model Derivatives to harm Yourself or others;
5. To override or circumvent the safety guardrails and safeguards We have put in place;
6. For the purpose of exploiting, harming or attempting to exploit or harm minors in any way;
7. To generate or disseminate verifiably false information and/or content with the purpose of harming others or influencing elections;
8. To generate or facilitate false online engagement, including fake reviews and other means of fake online engagement;
9. To intentionally defame, disparage or otherwise harass others;
10. To generate and/or disseminate malware (including ransomware) or any other content to be used for the purpose of harming electronic systems;
11. To generate or disseminate personal identifiable information with the purpose of harming others;
12. To generate or disseminate information (including images, code, posts, articles), and place the information in any public context (including –through the use of bot generated tweets), without expressly and conspicuously identifying that the information and/or content is machine generated;
13. To impersonate another individual without consent, authorization, or legal right;
14. To make high-stakes automated decisions in domains that affect an individual’s safety, rights or wellbeing (e.g., law enforcement, migration, medicine/health, management of critical infrastructure, safety components of products, essential services, credit, employment, housing, education, social scoring, or insurance);
15. In a manner that violates or disrespects the social ethics and moral standards of other countries or regions;
16. To perform, facilitate, threaten, incite, plan, promote or encourage violent extremism or terrorism;
17. For any use intended to discriminate against or harm individuals or groups based on protected characteristics or categories, online or offline social behavior or known or predicted personal or personality characteristics;
18. To intentionally exploit any of the vulnerabilities of a specific group of persons based on their age, social, physical or mental characteristics, in order to materially distort the behavior of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological harm;
19. For military purposes;
20. To engage in the unauthorized or unlicensed practice of any profession including, but not limited to, financial, legal, medical/health, or other professional practices.
NOTICE 0 → 100644
View file @ aad7b6c7
This diff is collapsed.
README.md 0 → 100644
View file @ aad7b6c7
# Hunyuan3D-2
腾讯提出Hunyuan3D-2，用于生成带有高分辨率纹理贴图的高保真度3D模型。
## 论文
`无`
- 技术文档参考：[Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation](./assets/report/Tencent_Hunyuan3D_2_0.pdf )
## 模型结构
Hunyuan3D-DiT采用了具有双流和单流块的 transformer 架构，是一种基于流的扩散模型，旨在根据给定的图像提示生成高保真和高分辨率的3D形状。
<div align=center>
<img src="./doc/Hunyuan3DDiT.png"/>
</div>
Hunyuan3D-Paint通过双流图像条件参考网络、多任务注意力机制以及几何和视图条件策略，实现图像对齐、几何跟随和多视图一致性。
<div align=center>
<img src="./doc/Hunyuan3DPaint.png"/>
</div>
## 算法原理
Hunyuan3D 2.0包括两个基础组件：一个大规模形状生成模型Hunyuan3D-DiT，和一个大规模纹理合成模型Hunyuan3D-Paint。形状生成模型基于可扩展的基于流的扩散变换器，旨在创建与给定条件图像正确对齐的几何形状，纹理合成模型基于生成的或手工制作的网格生成高分辨率和生动的纹理图。
<div align=center>
<img src="./doc/algorithm.png"/>
</div>
## 环境配置
```
mv Hunyuan3D-2_pytorch Hunyuan3D-2 # 去框架名后缀
```
### Docker（方法一）
```
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.3.0-py3.10-dtk24.04.3-ubuntu20.04
# <your IMAGE ID>为以上拉取的docker的镜像ID替换，本镜像为：b272aae8ec72
docker run -it --shm-size=64G -v $PWD/Hunyuan3D-2:/home/Hunyuan3D-2 -v /opt/hyhal:/opt/hyhal:ro --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add hy2 --name hy2 <your IMAGE ID> bash
cd /home/Hunyuan3D-2
pip install -r requirements.txt
cd hy3dgen/texgen/custom_rasterizer
python3 setup.py install
cd ../../..
cd hy3dgen/texgen/differentiable_renderer
python3 setup.py install
```
### Dockerfile（方法二）
```
cd /home/Hunyuan3D-2/docker
docker build --no-cache -t hy2:latest .
docker run --shm-size=64G --name hy2 -v /opt/hyhal:/opt/hyhal:ro --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video -v $PWD/../../Hunyuan3D-2:/home/Hunyuan3D-2 -it hy2 bash
# 若遇到Dockerfile启动的方式安装环境需要长时间等待，可注释掉里面的pip安装，启动容器后再安装python库：pip install -r requirements.txt。
cd hy3dgen/texgen/custom_rasterizer
python3 setup.py install
cd ../../..
cd hy3dgen/texgen/differentiable_renderer
python3 setup.py install
```
### Anaconda（方法三）
1、关于本项目DCU显卡所需的特殊深度学习库可从光合开发者社区下载安装：
- https://developer.hpccube.com/tool/
```
DTK驱动:dtk24.04.3
python:python3.10
torch:2.3.0
torchvision:0.18.1
torchaudio:2.1.2
triton:2.1.0
vllm:0.6.2
flash-attn:2.6.1
deepspeed:0.14.2
apex:1.3.0
xformers:0.0.25
transformers:4.48.0
```
`Tips：以上dtk驱动、python、torch等DCU相关工具版本需要严格一一对应。`
2、其它非特殊库参照requirements.txt安装
```
cd /home/Hunyuan3D-2
pip install -r requirements.txt
cd hy3dgen/texgen/custom_rasterizer
python3 setup.py install
cd ../../..
cd hy3dgen/texgen/differentiable_renderer
python3 setup.py install
```
## 数据集
`无`
## 训练
`无`
## 推理
### 单机单卡
预训练权重目录结构：
```
/home/Hunyuan3D-2/
├── 3tencent/Hunyuan3D-2/
├── hunyuan3d-delight-v2-0
├── hunyuan3d-dit-v2-0
├── hunyuan3d-dit-v2-0-fast
└── hunyuan3d-paint-v2-0
├── tabilityai/stable-diffusion-x4-upscaler
└── u2net.onnx
mv u2net.onnx /root/.u2net/
```
```
python minimal_demo.py
# 目前DCU仅支持3D形状生成，其它功能敬请期待后续开放。
```
更多资料可参考源项目的[`README_origin`](./README_origin.md)
## result
`输入: `
```
图片: assets/demo.png
```
`输出:`
```
3D结构: mesh.glb
```
在线3D模型结构查看工具：[gltf-viewer](https://gltf-viewer.donmccurdy.com/)
源作者提供的效果示例：
<div align=center>
<img src="assets/images/e2e-1.gif" height=250>
<img src="assets/images/e2e-2.gif" height=250>
</div>
### 精度
DCU与GPU精度一致，推理框架：pytorch。
## 应用场景
### 算法类别
`3D生成`
### 热点应用行业
`动漫,广媒,影视,制造,医疗,家居,教育`
## 预训练权重
预训练权重快速下载中心：[SCNet AIModels](http://113.200.138.88:18080/aimodels) ，项目中的预训练权重可从快速下载通道下载：[Hunyuan3D-2](http://113.200.138.88:18080/aimodels/tencent/hunyuan3d-2-hf.git)、[stable-diffusion-x4-upscaler](http://113.200.138.88:18080/aimodels/stable-diffusion-x4-upscaler.git)、[rembg-u2net](http://113.200.138.88:18080/aimodels/findsource-dependency/rembg-u2net.git)
Hugging Face下载地址为：[Hunyuan3D-2](https://huggingface.co/tencent/Hunyuan3D-2)、[stable-diffusion-x4-upscaler](https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler)、[rembg-u2net](https://github.com/danielgatis/rembg/releases/download/v0.0.0/u2net.onnx)
## 源码仓库及问题反馈
- http://developer.sourcefind.cn/codes/modelzoo/Hunyuan3D-2_pytorch.git
## 参考资料
- https://github.com/Tencent/Hunyuan3D-2.git
README_ja_jp.md 0 → 100644
View file @ aad7b6c7
[Read in English](README.md)
[中文阅读](README_zh_cn.md)
<p align="center">
<img src="./assets/images/teaser.jpg">
</p>
<div align="center">
<a href=https://3d.hunyuan.tencent.com target="_blank"><img src=https://img.shields.io/badge/Official%20Site-black.svg?logo=homepage height=22px></a>
<a href=https://huggingface.co/spaces/tencent/Hunyuan3D-2 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Demo-276cb4.svg height=22px></a>
<a href=https://huggingface.co/tencent/Hunyuan3D-2 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Models-d96902.svg height=22px></a>
<a href=https://3d-models.hunyuan.tencent.com/ target="_blank"><img src= https://img.shields.io/badge/Page-bb8a2e.svg?logo=github height=22px></a>
<a href=https://discord.gg/GuaWYwzKbX target="_blank"><img src= https://img.shields.io/badge/Discord-white.svg?logo=discord height=22px></a>
<a href=https://github.com/Tencent/Hunyuan3D-2/blob/main/assets/report/Tencent_Hunyuan3D_2_0.pdf target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>
</div>
[//]: # ( <a href=# target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>)
[//]: # ( <a href=# target="_blank"><img src= https://img.shields.io/badge/Colab-8f2628.svg?logo=googlecolab height=22px></a>)
[//]: # ( <a href="#"><img alt="PyPI - Downloads" src="https://img.shields.io/pypi/v/mulankit?logo=pypi" height=22px></a>)
<br>
<p align="center">
“ 3Dアセットの作成と操作において、すべての人の想像力を実現します。”
</p>
## 🔥 ニュース
- 2025年2月14日: 🛠️ テクスチャ強化モジュールをリリースしました。HD テクスチャ生成を [体験](minimal_demo.py) してください。
- 2025年1月21日: 💬 私たちのウェブサイト [Hunyuan3D Studio](https://3d.hunyuan.tencent.com) でエキサイティングな3D生成を楽しんでください！
- 2025年1月21日: 💬 [Hunyuan3D 2.0](https://huggingface.co/tencent/Hunyuan3D-2) の推論コードと事前学習モデルをリリースしました。
- 2025年1月21日: 💬 Hunyuan3D 2.0をリリースしました。 [huggingface space](https://huggingface.co/spaces/tencent/Hunyuan3D-2) や [公式サイト](https://3d.hunyuan.tencent.com) でお試しください！
## **概要**
Hunyuan3D 2.0は、高解像度のテクスチャ付き3Dアセットを生成するための高度な大規模3D合成システムです。
このシステムには、2つの基盤コンポーネントが含まれています：大規模形状生成モデル - Hunyuan3D-DiT、および大規模
テクスチャ合成モデル - Hunyuan3D-Paint。
形状生成モデルは、スケーラブルなフローに基づく拡散トランスフォーマーに基づいて構築されており、与えられた条件画像に適切に
一致するジオメトリを作成することを目的としており、下流のアプリケーションのための堅固な基盤を提供します。
テクスチャ合成モデルは、強力なジオメトリおよび拡散の事前知識を活用して、生成されたまたは手作りのメッシュのために高解像度で鮮やかな
テクスチャマップを生成します。
さらに、Hunyuan3D-Studioを構築しました。これは、3Dアセットの再作成プロセスを簡素化する多用途で使いやすい制作プラットフォームです。
プロフェッショナルおよびアマチュアユー���ーの両方がメッシュを効率的に操作したり、アニメーション化したりすることができます。
私たちはモデルを体系的に評価し、Hunyuan3D 2.0が以前の最先端モデルを上回ることを示しました。
オープンソースモデルとクローズドソースモデルの両方で、ジオメトリの詳細、条件の整合性、テクスチャの品質などの点で優れています。
<p align="center">
<img src="assets/images/system.jpg">
</p>
## ☯️ **Hunyuan3D 2.0**
### アーキテクチャ
Hunyuan3D 2.0は、ベアメッシュの作成から始まり、そのメッシュのテクスチャマップの合成に至る2段階の生成パイプライン���特徴としています。
この戦略は、形状とテクスチャの生成の難しさを分離するのに効果的であり、生成されたメッシュまたは手作りのメッシュのテクスチャリングに柔軟性を提供します。
<p align="left">
<img src="assets/images/arch.jpg">
</p>
### パフォーマンス
Hunyuan3D 2.0を他のオープンソースおよびクローズドソースの3D生成方法と比較して評価しました。
数値結果は、Hunyuan3D 2.0が生成されたテクスチャ付き3Dアセットの品質と条件の遵守能力においてすべてのベースラインを上回っていることを示しています。
| モデル | CMMD(⬇) | FID_CLIP(⬇) | FID(⬇) | CLIP-score(⬆) |
|-------------------------|-----------|-------------|-------------|---------------|
| トップオープンソースモデル1 | 3.591 | 54.639 | 289.287 | 0.787 |
| トップクローズドソースモデル1 | 3.600 | 55.866 | 305.922 | 0.779 |
| トップクローズドソースモデル2 | 3.368 | 49.744 | 294.628 | 0.806 |
| トップクローズドソースモデル3 | 3.218 | 51.574 | 295.691 | 0.799 |
| Hunyuan3D 2.0 | **3.193** | **49.165** | **282.429** | **0.809** |
Hunyuan3D 2.0の生成結果：
<p align="left">
<img src="assets/images/e2e-1.gif" height=250>
<img src="assets/images/e2e-2.gif" height=250>
</p>
### 事前学習モデル
| モデル | 日付 | Huggingface |
|----------------------|------------|--------------------------------------------------------|
| Hunyuan3D-DiT-v2-0 | 2025-01-21 | [ダウンロード](https://huggingface.co/tencent/Hunyuan3D-2) |
| Hunyuan3D-Paint-v2-0 | 2025-01-21 | [ダウンロード](https://huggingface.co/tencent/Hunyuan3D-2) |
## 🤗 Hunyuan3D 2.0の使い方
次の手順に従って、コードまたはGradioアプリを使用してHunyuan3D 2.0を使用できます。
### 必要なものをインストール
公式サイトからPytorchをインストールしてください。次に、他の必要なものを以下の方法でインストールします。
```bash
pip install -r requirements.txt
# for texture
cd hy3dgen/texgen/custom_rasterizer
python3 setup.py install
cd hy3dgen/texgen/differentiable_renderer
python3 setup.py install
```
### APIの使い方
形状生成モデル - Hunyuan3D-DiTおよびテクスチャ合成モデル - Hunyuan3D-Paintを使用するためのdiffusersのようなAPIを設計しました。
**Hunyuan3D-DiT**にアクセスするには、次のようにします：
```python
from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline
pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2')
mesh = pipeline(image='assets/demo.png')[0]
```
出力メッシュは[trimeshオブジェクト](https://trimesh.org/trimesh.html)であり、glb/obj（または他の形式）ファイルに保存できます。
**Hunyuan3D-Paint**の場合は、次のようにします：
```python
from hy3dgen.texgen import Hunyuan3DPaintPipeline
from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline
# まずメッシュを生成しましょう
pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2')
mesh = pipeline(image='assets/demo.png')[0]
pipeline = Hunyuan3DPaintPipeline.from_pretrained('tencent/Hunyuan3D-2')
mesh = pipeline(mesh, image='assets/demo.png')
```
より高度な使用法については、[minimal_demo.py](minimal_demo.py)を参照してください。例えば、**テキストから3D**や**手作りメッシュのテクスチャ生成**などです。
### Gradioアプリ
次の方法で自分のコンピュータで[Gradio](https://www.gradio.app/)アプリをホストすることもできます：
```bash
python3 gradio_app.py
```
自分でホストしたくない場合は、[Hunyuan3D](https://3d.hunyuan.tencent.com)を訪れてすぐに使用してください。
## 📑 オープンソース計画
- [x] 推論コード
- [x] モデルチェックポイント
- [x] 技術報告書
- [ ] ComfyUI
- [ ] TensorRTバージョン
## 🔗 BibTeX
このリポジトリが役に立った場合は、以下の方法で報告書を引用してください：
```bibtex
@misc{hunyuan3d22025tencent,
title={Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation},
author={Tencent Hunyuan3D Team},
year={2025},
eprint={2501.12202},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{yang2024hunyuan3d,
title={Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation},
author={Tencent Hunyuan3D Team},
year={2024},
eprint={2411.02293},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
## 謝辞
[DINOv2](https://github.com/facebookresearch/dinov2), [Stable Diffusion](https://github.com/Stability-AI/stablediffusion), [FLUX](https://github.com/black-forest-labs/flux), [diffusers](https://github.com/huggingface/diffusers), [HuggingFace](https://huggingface.co), [CraftsMan3D](https://github.com/wyysf-98/CraftsMan3D), and [Michelangelo](https://github.com/NeuralCarver/Michelangelo/tree/main) リポジトリの貢献者に感謝します。
## スター履歴
<a href="https://star-history.com/#Tencent/Hunyuan3D-2&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date" />
</picture>
</a>
README_origin.md 0 → 100644
View file @ aad7b6c7
[中文阅读](README_zh_cn.md)
[日本語で読む](README_ja_jp.md)
<p align="center">
<img src="./assets/images/teaser.jpg">
</p>
<div align="center">
<a href=https://3d.hunyuan.tencent.com target="_blank"><img src=https://img.shields.io/badge/Official%20Site-333399.svg?logo=homepage height=22px></a>
<a href=https://huggingface.co/spaces/tencent/Hunyuan3D-2 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Demo-276cb4.svg height=22px></a>
<a href=https://huggingface.co/tencent/Hunyuan3D-2 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Models-d96902.svg height=22px></a>
<a href=https://3d-models.hunyuan.tencent.com/ target="_blank"><img src= https://img.shields.io/badge/Page-bb8a2e.svg?logo=github height=22px></a>
<a href=https://discord.gg/GuaWYwzKbX target="_blank"><img src= https://img.shields.io/badge/Discord-white.svg?logo=discord height=22px></a>
<a href=https://arxiv.org/abs/2501.12202 target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>
<a href=https://x.com/txhunyuan target="_blank"><img src=https://img.shields.io/badge/Hunyuan-black.svg?logo=x height=22px></a>
<a href="#community-resources" target="_blank"><img src=https://img.shields.io/badge/Community-lavender.svg?logo=homeassistantcommunitystore height=22px></a>
</div>
[//]: # ( <a href=# target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>)
[//]: # ( <a href=# target="_blank"><img src= https://img.shields.io/badge/Colab-8f2628.svg?logo=googlecolab height=22px></a>)
[//]: # ( <a href="#"><img alt="PyPI - Downloads" src="https://img.shields.io/pypi/v/mulankit?logo=pypi" height=22px></a>)
<br>
> Join our **[Wechat](#)** and **[Discord](https://discord.gg/GuaWYwzKbX)** group to discuss and find help from us.
| Wechat Group | Xiaohongshu | X | Discord |
|--------------------------------------------------|-------------------------------------------------------|---------------------------------------------|---------------------------------------------------|
| <img src="assets/qrcode/wechat.png" height=140> | <img src="assets/qrcode/xiaohongshu.png" height=140> | <img src="assets/qrcode/x.png" height=140> | <img src="assets/qrcode/discord.png" height=140> |
---
<p align="center">
“ Living out everyone’s imagination on creating and manipulating 3D assets.”
</p>
## 🔥 News
- Feb 14, 2025: 🛠️ Release texture enhancement module, please obtain high-definition textures via [here](minimal_demo.py)!
- Feb 3, 2025: 🐎
Release [Hunyuan3D-DiT-v2-0-Fast](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-dit-v2-0-fast), our
guidance distillation model that could half the dit inference time, see [here](minimal_demo.py) for usage.
- Jan 27, 2025: 🛠️ Release Blender addon for Hunyuan3D 2.0, Check it out [here](#blender-addon).
- Jan 23, 2025: 💬 We thank community members for
creating [Windows installation tool](https://github.com/YanWenKun/Hunyuan3D-2-WinPortable), ComfyUI support
with [ComfyUI-Hunyuan3DWrapper](https://github.com/kijai/ComfyUI-Hunyuan3DWrapper)
and [ComfyUI-3D-Pack](https://github.com/MrForExample/ComfyUI-3D-Pack) and other
awesome [extensions](#community-resources).
- Jan 21, 2025: 💬 Enjoy exciting 3D generation on our website [Hunyuan3D Studio](https://3d.hunyuan.tencent.com)!
- Jan 21, 2025: 🤗 Release inference code and pretrained models
of [Hunyuan3D 2.0](https://huggingface.co/tencent/Hunyuan3D-2). Please give it a try
via [huggingface space](https://huggingface.co/spaces/tencent/Hunyuan3D-2) and
our [official site](https://3d.hunyuan.tencent.com)!
## **Abstract**
We present Hunyuan3D 2.0, an advanced large-scale 3D synthesis system for generating high-resolution textured 3D assets.
This system includes two foundation components: a large-scale shape generation model - Hunyuan3D-DiT, and a large-scale
texture synthesis model - Hunyuan3D-Paint.
The shape generative model, built on a scalable flow-based diffusion transformer, aims to create geometry that properly
aligns with a given condition image, laying a solid foundation for downstream applications.
The texture synthesis model, benefiting from strong geometric and diffusion priors, produces high-resolution and vibrant
texture maps for either generated or hand-crafted meshes.
Furthermore, we build Hunyuan3D-Studio - a versatile, user-friendly production platform that simplifies the re-creation
process of 3D assets. It allows both professional and amateur users to manipulate or even animate their meshes
efficiently.
We systematically evaluate our models, showing that Hunyuan3D 2.0 outperforms previous state-of-the-art models,
including the open-source models and closed-source models in geometry details, condition alignment, texture quality, and
e.t.c.
<p align="center">
<img src="assets/images/system.jpg">
</p>
## ☯️ **Hunyuan3D 2.0**
### Architecture
Hunyuan3D 2.0 features a two-stage generation pipeline, starting with the creation of a bare mesh, followed by the
synthesis of a texture map for that mesh. This strategy is effective for decoupling the difficulties of shape and
texture generation and also provides flexibility for texturing either generated or handcrafted meshes.
<p align="left">
<img src="assets/images/arch.jpg">
</p>
### Performance
We have evaluated Hunyuan3D 2.0 with other open-source as well as close-source 3d-generation methods.
The numerical results indicate that Hunyuan3D 2.0 surpasses all baselines in the quality of generated textured 3D assets
and the condition following ability.
| Model | CMMD(⬇) | FID_CLIP(⬇) | FID(⬇) | CLIP-score(⬆) |
|-------------------------|-----------|-------------|-------------|---------------|
| Top Open-source Model1 | 3.591 | 54.639 | 289.287 | 0.787 |
| Top Close-source Model1 | 3.600 | 55.866 | 305.922 | 0.779 |
| Top Close-source Model2 | 3.368 | 49.744 | 294.628 | 0.806 |
| Top Close-source Model3 | 3.218 | 51.574 | 295.691 | 0.799 |
| Hunyuan3D 2.0 | **3.193** | **49.165** | **282.429** | **0.809** |
Generation results of Hunyuan3D 2.0:
<p align="left">
<img src="assets/images/e2e-1.gif" height=250>
<img src="assets/images/e2e-2.gif" height=250>
</p>
## 🎁 Models Zoo
It takes 11.5 GB VRAM for shape generation and 24.5 GB for shape and texture generation in total.
| Model | Description | Date | Size | Huggingface |
|-------------------------|-----------------------------|------------|------|------------------------------------------------------------------------------------------|
| Hunyuan3D-DiT-v2-0-Fast | Guidance Distillation Model | 2025-02-03 | 2.6B | [Download](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-dit-v2-0-fast) |
| Hunyuan3D-DiT-v2-0 | Image to Shape Model | 2025-01-21 | 2.6B | [Download](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-dit-v2-0) |
| Hunyuan3D-Paint-v2-0 | Texture Generation Model | 2025-01-21 | 1.3B | [Download](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-paint-v2-0) |
| Hunyuan3D-Delight-v2-0 | Image Delight Model | 2025-01-21 | 1.3B | [Download](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-delight-v2-0) |
## 🤗 Get Started with Hunyuan3D 2.0
You may follow the next steps to use Hunyuan3D 2.0 via:
- [Code](#code-usage)
- [Gradio App](#gradio-app)
- [API Server](#api-server)
- [Blender Addon](#blender-addon)
- [Official Site](#official-site)
### Install Requirements
Please install Pytorch via the [official](https://pytorch.org/) site. Then install the other requirements via
```bash
pip install -r requirements.txt
# for texture
cd hy3dgen/texgen/custom_rasterizer
python3 setup.py install
cd ../../..
cd hy3dgen/texgen/differentiable_renderer
python3 setup.py install
```
### Code Usage
We designed a diffusers-like API to use our shape generation model - Hunyuan3D-DiT and texture synthesis model -
Hunyuan3D-Paint.
You could assess **Hunyuan3D-DiT** via:
```python
from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline
pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2')
mesh = pipeline(image='assets/demo.png')[0]
```
The output mesh is a [trimesh object](https://trimesh.org/trimesh.html), which you could save to glb/obj (or other
format) file.
For **Hunyuan3D-Paint**, do the following:
```python
from hy3dgen.texgen import Hunyuan3DPaintPipeline
from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline
# let's generate a mesh first
pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2')
mesh = pipeline(image='assets/demo.png')[0]
pipeline = Hunyuan3DPaintPipeline.from_pretrained('tencent/Hunyuan3D-2')
mesh = pipeline(mesh, image='assets/demo.png')
```
Please visit [minimal_demo.py](minimal_demo.py) for more advanced usage, such as **text to 3D** and **texture generation
for handcrafted mesh**.
### Gradio App
You could also host a [Gradio](https://www.gradio.app/) App in your own computer via:
```bash
python3 gradio_app.py
```
### API Server
You could launch an API server locally, which you could post web request for Image/Text to 3D, Texturing existing mesh,
and e.t.c.
```bash
python api_server.py --host 0.0.0.0 --port 8080
```
A demo post request for image to 3D without texture.
```bash
img_b64_str=$(base64 -i assets/demo.png)
curl -X POST "http://localhost:8080/generate" \
-H "Content-Type: application/json" \
-d '{
"image": "'"$img_b64_str"'",
}' \
-o test2.glb
```
### Blender Addon
With an API server launched, you could also directly use Hunyuan3D 2.0 in your blender with
our [Blender Addon](blender_addon.py). Please follow our tutorial to install and use.
https://github.com/user-attachments/assets/8230bfb5-32b1-4e48-91f4-a977c54a4f3e
### Official Site
Don't forget to visit [Hunyuan3D](https://3d.hunyuan.tencent.com) for quick use, if you don't want to host yourself.
## 📑 Open-Source Plan
- [x] Inference Code
- [x] Model Checkpoints
- [x] Technical Report
- [ ] ComfyUI
- [ ] TensorRT Version
## 🔗 BibTeX
If you found this repository helpful, please cite our reports:
```bibtex
@misc{hunyuan3d22025tencent,
title={Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation},
author={Tencent Hunyuan3D Team},
year={2025},
eprint={2501.12202},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{yang2024hunyuan3d,
title={Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation},
author={Tencent Hunyuan3D Team},
year={2024},
eprint={2411.02293},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
## Community Resources
Thanks for the contributions of community members, here we have these great extensions of Hunyuan3D 2.0:
- [ComfyUI-3D-Pack](https://github.com/MrForExample/ComfyUI-3D-Pack)
- [ComfyUI-Hunyuan3DWrapper](https://github.com/kijai/ComfyUI-Hunyuan3DWrapper)
- [Hunyuan3D-2-for-windows](https://github.com/sdbds/Hunyuan3D-2-for-windows)
- [📦 A bundle for running on Windows | 整合包](https://github.com/YanWenKun/Hunyuan3D-2-WinPortable)
- [Hunyuan3D-2GP](https://github.com/deepbeepmeep/Hunyuan3D-2GP)
## Acknowledgements
We would like to thank the contributors to
the [DINOv2](https://github.com/facebookresearch/dinov2), [Stable Diffusion](https://github.com/Stability-AI/stablediffusion), [FLUX](https://github.com/black-forest-labs/flux), [diffusers](https://github.com/huggingface/diffusers), [HuggingFace](https://huggingface.co), [CraftsMan3D](https://github.com/wyysf-98/CraftsMan3D),
and [Michelangelo](https://github.com/NeuralCarver/Michelangelo/tree/main) repositories, for their open research and
exploration.
## Star History
<a href="https://star-history.com/#Tencent/Hunyuan3D-2&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date" />
</picture>
</a>
README_zh_cn.md 0 → 100644
View file @ aad7b6c7
[Read in English](README.md)
[日本語で読む](README_ja_jp.md)
<p align="center">
<img src="./assets/images/teaser.jpg">
</p>
<div align="center">
<a href=https://3d.hunyuan.tencent.com target="_blank"><img src=https://img.shields.io/badge/Official%20Site-333399.svg?logo=homepage height=22px></a>
<a href=https://huggingface.co/spaces/tencent/Hunyuan3D-2 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Demo-276cb4.svg height=22px></a>
<a href=https://huggingface.co/tencent/Hunyuan3D-2 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Models-d96902.svg height=22px></a>
<a href=https://3d-models.hunyuan.tencent.com/ target="_blank"><img src= https://img.shields.io/badge/Page-bb8a2e.svg?logo=github height=22px></a>
<a href=https://discord.gg/GuaWYwzKbX target="_blank"><img src= https://img.shields.io/badge/Discord-white.svg?logo=discord height=22px></a>
<a href=https://arxiv.org/abs/2501.12202 target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>
<a href=https://x.com/txhunyuan target="_blank"><img src=https://img.shields.io/badge/Hunyuan-black.svg?logo=x height=22px></a>
</div>
[//]: # ( <a href=# target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>)
[//]: # ( <a href=# target="_blank"><img src= https://img.shields.io/badge/Colab-8f2628.svg?logo=googlecolab height=22px></a>)
[//]: # ( <a href="#"><img alt="PyPI - Downloads" src="https://img.shields.io/pypi/v/mulankit?logo=pypi" height=22px></a>)
<br>
> 新年快乐!
![happynewyear](https://github.com/user-attachments/assets/69aa40a7-8657-4c2b-8efd-99eda6c26fe4)
> 加入我们的 **[微信群](#)** and **[Discord 社区](https://discord.gg/GuaWYwzKbX)** 讨论，获取最新进展以及帮助吧.
| Wechat Group | Xiaohongshu | X | Discord |
| ------------------------------------------------ | ----------------------------------------------------- | ------------------------------------------- | ------------------------------------------------- |
| <img src="assets/qrcode/wechat.png" height=140> | <img src="assets/qrcode/xiaohongshu.png" height=140> | <img src="assets/qrcode/x.png" height=140> | <img src="assets/qrcode/discord.png" height=140> |
---
<br>
<p align="center">
“通过 3D 创作与编辑让每个人的想象变成现实。”
</p>
## 🔥 最新消息
- Feb 14, 2025: 🛠️ 发布纹理增强模块，欢迎[体验](minimal_demo.py)高清纹理生成.
- Jan 27, 2025: 🛠️ 发布 Blender 插件，欢迎[体验](#blender-addon).
- Jan 23, 2025: 💬 感谢社区成员的 [Windows 安装工具](https://github.com/YanWenKun/Hunyuan3D-2-WinPortable), ComfyUI 支持 [ComfyUI-Hunyuan3DWrapper](https://github.com/kijai/ComfyUI-Hunyuan3DWrapper)， [ComfyUI-3D-Pack](https://github.com/MrForExample/ComfyUI-3D-Pack) 以及其他出色的 [扩展功能](#community-resources).
- Jan 21, 2025: 💬 欢迎来我们的门户网站 [Hunyuan3D Studio](https://3d.hunyuan.tencent.com) 体验更多3D生成功能!
- Jan 21, 2025: 💬 我们开源了 [Hunyuan3D 2.0](https://huggingface.co/tencent/Hunyuan3D-2)的推理代码和预训练权重.
- Jan 21, 2025: 💬 我们发布了 [Hunyuan3D 2.0](https://huggingface.co/spaces/tencent/Hunyuan3D-2). 快来试试吧!
## 概览
混元 3D 2.0 是一款先进的大规模 3D 资产创作系统，它可以用于生成带有高分辨率纹理贴图的高保真度3D模型。该系统包含两个基础组件：一个大规模几何生成模型 — 混元 3D-DiT，以及一个大规模纹理生成模型 — 混元 3D-Paint。
几何生成模型基于流扩散的扩散模型构建，旨在生成与给定条件图像精确匹配的几何模型，为下游应用奠定坚实基础。
纹理生成模型得益于强大的几何和扩散模型先验知识，能够为AI生成的或手工制作的网格模型生成高分辨率且生动逼真的纹理贴图。
此外，我们打造了混元 3D 功能矩阵，一个功能多样、易于使用的创作平台，简化了 3D 模型的制作以及修改过程。它使专业用户和业余爱好者都能高效地对3D模型进行操作，甚至制作动画。
我们对该系统进行了系统评估，结果表明混元 3D 2.0 在几何细节、条件匹配、纹理质量等方面均优于以往的最先进的开源以及闭源模型。
<p align="center">
<img src="assets/images/system.jpg">
</p>
## ☯️ **Hunyuan3D 2.0**
### 模型架构
混元 3D 2.0 采用了一个两阶段的生成过程，它首先创建一个无纹理的几何模型，然后为该几何模型合成纹理贴图。这种策略有效地将形状生成和纹理生成的难点分离开来，同时也为生成的几何模型或手工制作的几何模型进行纹理处理提供了灵活性。
<p align="left">
<img src="assets/images/arch.jpg">
</p>
### 性能评估
我们将混元 3D 2.0 与其他开源及闭源的 3D 生成方法进行了评估对比。
数值结果表明，在生成的带纹理 3D 模型的质量以及对给定条件的遵循能力方面，混元 3D 2.0 超越了所有的基准模型。
| Model | CMMD(⬇) | FID_CLIP(⬇) | FID(⬇) | CLIP-score(⬆) |
| ----------------------- | --------- | ----------- | ----------- | ------------- |
| Top Open-source Model1 | 3.591 | 54.639 | 289.287 | 0.787 |
| Top Close-source Model1 | 3.600 | 55.866 | 305.922 | 0.779 |
| Top Close-source Model2 | 3.368 | 49.744 | 294.628 | 0.806 |
| Top Close-source Model3 | 3.218 | 51.574 | 295.691 | 0.799 |
| Hunyuan3D 2.0 | **3.193** | **49.165** | **282.429** | **0.809** |
一些 Hunyuan3D 2.0 的生成结果:
<p align="left">
<img src="assets/images/e2e-1.gif" height=300>
<img src="assets/images/e2e-2.gif" height=300>
</p>
### 预训练模型
| 模型名称 | 发布日期 | 参数 | Huggingface |
| ---------------------- | ---------- | ---- | ----------------------------------------------------------------------------------- |
| Hunyuan3D-DiT-v2-0 | 2025-01-21 | 2.6B | [下载](https://huggingface.co/tencent/Hunyuan3D-2) |
| Hunyuan3D-Paint-v2-0 | 2025-01-21 | 1.3B | [下载](https://huggingface.co/tencent/Hunyuan3D-2) |
| Hunyuan3D-Delight-v2-0 | 2025-01-21 | 1.3B | [下载](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-delight-v2-0) |
## 🤗快速入门 Hunyuan3D 2.0
你可以按照以下步骤，通过代码或 Gradio 来使用混元 3D 2.0。
- [代码使用](#代码使用方法)
- [Gradio](#gradio-app-使用方法)
- [API服务器](#api-服务器)
- [Blender插件](#blender-插件)
- [官方网站](#官方网站)
### 依赖包安装
请通过官方网站安装 PyTorch。然后通过以下方式安装其他所需的依赖项。
```bash
pip install -r requirements.txt
# for texture
cd hy3dgen/texgen/custom_rasterizer
python3 setup.py install
cd ../../..
cd hy3dgen/texgen/differentiable_renderer
python3 setup.py install
```
### 代码使用方法
我们设计了一个类似于 diffusers 的 API 来使用我们的几何生成模型 — 混元 3D-DiT 和纹理合成模型 — 混元 3D-Paint。
你可以通过以下方式使用 混元 3D-DiT：
```python
from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline
pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2')
mesh = pipeline(image='assets/demo.png')[0]
```
输出的网格是一个 Trimesh 对象，你可以将其保存为 glb/obj（或其他格式）文件。
对于 混元 3D-Paint，请执行以下操作：
```python
from hy3dgen.texgen import Hunyuan3DPaintPipeline
from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline
# let's generate a mesh first
pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2')
mesh = pipeline(image='assets/demo.png')[0]
pipeline = Hunyuan3DPaintPipeline.from_pretrained('tencent/Hunyuan3D-2')
mesh = pipeline(mesh, image='assets/demo.png')
```
请访问 [minimal_demo.py](minimal_demo.py) 以了解更多高级用法，例如 文本转 3D 以及 为手工制作的网格生成纹理。
### Gradio App 使用方法
你也可以通过以下方式在自己的计算机上托管一个Gradio应用程序：
```bash
python3 gradio_app.py
```
### API 服务器
You could launch an API server locally, which you could post web request for Image/Text to 3D, Texturing existing mesh, and e.t.c.
```bash
python api_server.py --host 0.0.0.0 --port 8080
```
A demo post request for image to 3D without texture.
```bash
img_b64_str=$(base64 -i assets/demo.png)
curl -X POST "http://localhost:8080/generate" \
-H "Content-Type: application/json" \
-d '{
"image": "'"$img_b64_str"'",
}' \
-o test2.glb
```
### Blender 插件
With an API server launched, you could also directly use Hunyuan3D 2.0 in your blender with our [Blender Addon](blender_addon.py). Please follow our tutorial to install and use.
https://github.com/user-attachments/assets/8230bfb5-32b1-4e48-91f4-a977c54a4f3e
### 官方网站
如果你不想自己托管，别忘了访问[混元 3D](https://3d.hunyuan.tencent.com)进行快速使用。
## 📑 开源计划
- [x] 推理代码
- [x] 模型权重
- [x] 技术报告
- [ ] ComfyUI
- [ ] TensorRT 量化
## 🔗 引用
如果你发现我们的工作有帮助，你可以以下面的方式引用我们的报告：
```bibtex
@misc{hunyuan3d22025tencent,
title={Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation},
author={Tencent Hunyuan3D Team},
year={2025},
eprint={2501.12202},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{yang2024hunyuan3d,
title={Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation},
author={Tencent Hunyuan3D Team},
year={2024},
eprint={2411.02293},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
## 致谢
We would like to thank the contributors to
the [DINOv2](https://github.com/facebookresearch/dinov2), [Stable Diffusion](https://github.com/Stability-AI/stablediffusion), [FLUX](https://github.com/black-forest-labs/flux), [diffusers](https://github.com/huggingface/diffusers), [HuggingFace](https://huggingface.co), [CraftsMan3D](https://github.com/wyysf-98/CraftsMan3D), and [Michelangelo](https://github.com/NeuralCarver/Michelangelo/tree/main) repositories, for their open research and exploration.
## Star 历史
<a href="https://star-history.com/#Tencent/Hunyuan3D-2&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date" />
</picture>
</a>
api_server.py 0 → 100644
View file @ aad7b6c7
"""
A model worker executes the model.
"""
import argparse
import asyncio
import base64
import logging
import logging.handlers
import os
import sys
import tempfile
import threading
import traceback
import uuid
from io import BytesIO
import torch
import trimesh
import uvicorn
from PIL import Image
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse, FileResponse
from hy3dgen.rembg import BackgroundRemover
from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline, FloaterRemover, DegenerateFaceRemover, FaceReducer
from hy3dgen.texgen import Hunyuan3DPaintPipeline
from hy3dgen.text2image import HunyuanDiTPipeline
LOGDIR = '.'
server_error_msg = "**NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.**"
moderation_msg = "YOUR INPUT VIOLATES OUR CONTENT MODERATION GUIDELINES. PLEASE TRY AGAIN."
handler = None
def build_logger(logger_name, logger_filename):
global handler
formatter = logging.Formatter(
fmt="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
# Set the format of root handlers
if not logging.getLogger().handlers:
logging.basicConfig(level=logging.INFO)
logging.getLogger().handlers[0].setFormatter(formatter)
# Redirect stdout and stderr to loggers
stdout_logger = logging.getLogger("stdout")
stdout_logger.setLevel(logging.INFO)
sl = StreamToLogger(stdout_logger, logging.INFO)
sys.stdout = sl
stderr_logger = logging.getLogger("stderr")
stderr_logger.setLevel(logging.ERROR)
sl = StreamToLogger(stderr_logger, logging.ERROR)
sys.stderr = sl
# Get logger
logger = logging.getLogger(logger_name)
logger.setLevel(logging.INFO)
# Add a file handler for all loggers
if handler is None:
os.makedirs(LOGDIR, exist_ok=True)
filename = os.path.join(LOGDIR, logger_filename)
handler = logging.handlers.TimedRotatingFileHandler(
filename, when='D', utc=True, encoding='UTF-8')
handler.setFormatter(formatter)
for name, item in logging.root.manager.loggerDict.items():
if isinstance(item, logging.Logger):
item.addHandler(handler)
return logger
class StreamToLogger(object):
"""
Fake file-like stream object that redirects writes to a logger instance.
"""
def __init__(self, logger, log_level=logging.INFO):
self.terminal = sys.stdout
self.logger = logger
self.log_level = log_level
self.linebuf = ''
def __getattr__(self, attr):
return getattr(self.terminal, attr)
def write(self, buf):
temp_linebuf = self.linebuf + buf
self.linebuf = ''
for line in temp_linebuf.splitlines(True):
# From the io.TextIOWrapper docs:
# On output, if newline is None, any '\n' characters written
# are translated to the system default line separator.
# By default sys.stdout.write() expects '\n' newlines and then
# translates them so this is still cross platform.
if line[-1] == '\n':
self.logger.log(self.log_level, line.rstrip())
else:
self.linebuf += line
def flush(self):
if self.linebuf != '':
self.logger.log(self.log_level, self.linebuf.rstrip())
self.linebuf = ''
def pretty_print_semaphore(semaphore):
if semaphore is None:
return "None"
return f"Semaphore(value={semaphore._value}, locked={semaphore.locked()})"
SAVE_DIR = 'gradio_cache'
os.makedirs(SAVE_DIR, exist_ok=True)
worker_id = str(uuid.uuid4())[:6]
logger = build_logger("controller", f"{SAVE_DIR}/controller.log")
def load_image_from_base64(image):
return Image.open(BytesIO(base64.b64decode(image)))
class ModelWorker:
def __init__(self, model_path='tencent/Hunyuan3D-2', device='cuda'):
self.model_path = model_path
self.worker_id = worker_id
self.device = device
logger.info(f"Loading the model {model_path} on worker {worker_id} ...")
self.rembg = BackgroundRemover()
self.pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained(model_path, device=device)
self.pipeline_t2i = HunyuanDiTPipeline('Tencent-Hunyuan/HunyuanDiT-v1.1-Diffusers-Distilled',
device=device)
self.pipeline_tex = Hunyuan3DPaintPipeline.from_pretrained(model_path)
def get_queue_length(self):
if model_semaphore is None:
return 0
else:
return args.limit_model_concurrency - model_semaphore._value + (len(
model_semaphore._waiters) if model_semaphore._waiters is not None else 0)
def get_status(self):
return {
"speed": 1,
"queue_length": self.get_queue_length(),
}
@torch.inference_mode()
def generate(self, uid, params):
if 'image' in params:
image = params["image"]
image = load_image_from_base64(image)
else:
if 'text' in params:
text = params["text"]
image = self.pipeline_t2i(text)
else:
raise ValueError("No input image or text provided")
image = self.rembg(image)
params['image'] = image
if 'mesh' in params:
mesh = trimesh.load(BytesIO(base64.b64decode(params["mesh"])), file_type='glb')
else:
seed = params.get("seed", 1234)
params['generator'] = torch.Generator(self.device).manual_seed(seed)
params['octree_resolution'] = params.get("octree_resolution", 256)
params['num_inference_steps'] = params.get("num_inference_steps", 30)
params['guidance_scale'] = params.get('guidance_scale', 7.5)
params['mc_algo'] = 'mc'
mesh = self.pipeline(**params)[0]
if params.get('texture', False):
mesh = FloaterRemover()(mesh)
mesh = DegenerateFaceRemover()(mesh)
mesh = FaceReducer()(mesh, max_facenum=params.get('face_count', 40000))
mesh = self.pipeline_tex(mesh, image)
with tempfile.NamedTemporaryFile(suffix='.glb', delete=False) as temp_file:
mesh.export(temp_file.name)
mesh = trimesh.load(temp_file.name)
temp_file.close()
os.unlink(temp_file.name)
save_path = os.path.join(SAVE_DIR, f'{str(uid)}.glb')
mesh.export(save_path)
torch.cuda.empty_cache()
return save_path, uid
app = FastAPI()
@app.post("/generate")
async def generate(request: Request):
logger.info("Worker generating...")
params = await request.json()
uid = uuid.uuid4()
try:
file_path, uid = worker.generate(uid, params)
return FileResponse(file_path)
except ValueError as e:
traceback.print_exc()
print("Caught ValueError:", e)
ret = {
"text": server_error_msg,
"error_code": 1,
}
return JSONResponse(ret, status_code=404)
except torch.cuda.CudaError as e:
print("Caught torch.cuda.CudaError:", e)
ret = {
"text": server_error_msg,
"error_code": 1,
}
return JSONResponse(ret, status_code=404)
except Exception as e:
print("Caught Unknown Error", e)
traceback.print_exc()
ret = {
"text": server_error_msg,
"error_code": 1,
}
return JSONResponse(ret, status_code=404)
@app.post("/send")
async def generate(request: Request):
logger.info("Worker send...")
params = await request.json()
uid = uuid.uuid4()
threading.Thread(target=worker.generate, args=(uid, params,)).start()
ret = {"uid": str(uid)}
return JSONResponse(ret, status_code=200)
@app.get("/status/{uid}")
async def status(uid: str):
save_file_path = os.path.join(SAVE_DIR, f'{uid}.glb')
print(save_file_path, os.path.exists(save_file_path))
if not os.path.exists(save_file_path):
response = {'status': 'processing'}
return JSONResponse(response, status_code=200)
else:
base64_str = base64.b64encode(open(save_file_path, 'rb').read()).decode()
response = {'status': 'completed', 'model_base64': base64_str}
return JSONResponse(response, status_code=200)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--host", type=str, default="0.0.0.0")
parser.add_argument("--port", type=int, default=8081)
parser.add_argument("--model_path", type=str, default='tencent/Hunyuan3D-2')
parser.add_argument("--device", type=str, default="cuda")
parser.add_argument("--limit-model-concurrency", type=int, default=5)
args = parser.parse_args()
logger.info(f"args: {args}")
model_semaphore = asyncio.Semaphore(args.limit_model_concurrency)
worker = ModelWorker(model_path=args.model_path, device=args.device)
uvicorn.run(app, host=args.host, port=args.port, log_level="info")
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment