Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
text-generation-inference
Commits
3729a8a4
"vscode:/vscode.git/clone" did not exist on "41ce92bb2287f99d66cd614b4c838af056a289ce"
Commit
3729a8a4
authored
Oct 31, 2024
by
xuxzh1
🎱
Browse files
update
parent
efd602c8
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
21 deletions
+5
-21
README.md
README.md
+5
-21
No files found.
README.md
View file @
3729a8a4
...
...
@@ -21,36 +21,20 @@ Text Generation Inference(TGI)是一个用 Rust 和 Python 编写的框架
## python支持
+
Python 3.9.
+
Python 3.10.
+
DTK24.04.2
### 使用源码编译方式安装
#### 编译环境准备
有两种方式安装准备环境
##### 方式一(建议方式):
基于光源pytorch2.1.0基础镜像环境:镜像下载地址:
[
https://sourcefind.cn/#/image/dcu/pytorch
](
https://sourcefind.cn/#/image/dcu/pytorch
)
,根据pytorch2.1.0、python、dtk及系统下载对应的镜像版本。pytorch2.1.0镜像里已经安装了trition,flash-attn
##### 方式一:
##### 方式二:
基于现有python环境自己安装pytorch,triton,flash-att包:
**安装pytorch**
安装pytorch2.1.0,pytorch whl包下载目录:
[
https://cancon.hpccube.com:65024/4/main/pytorch
](
https://cancon.hpccube.com:65024/4/main/pytorch
)
,根据python、dtk版本,下载对应pytorch2.1.0的whl包。安装命令如下:
```
bash
pip
install
torch
*
(
下载的torch的whl包
)
pip
install
setuptools wheel
```
**安装triton**
triton whl包下载:
[
https://cancon.hpccube.com:65024/4/main/triton
](
https://cancon.hpccube.com:65024/4/main/triton
)
,需要根据python、dtk版本,下载对应triton 2.1的whl包
```
bash
pip
install
triton
*
(
下载的triton的whl包
)
```
### **TODO**
**安装flash-attn**
flash_attn包下载:
[
https://cancon.hpccube.com:65024/4/main/flash_attn
](
https://cancon.hpccube.com:65024/4/main/flash_attn
)
,需要根据python、dtk版本,下载对应flash_attn 2.0.4的whl包
```
bash
pip
install
flash_attn
*
(
下载的triton的whl包
)
```
##### 方式二:
#### 源码编译安装流程
基于光源pytorch2.1.0基础镜像环境:镜像下载地址:
[
https://sourcefind.cn/#/image/dcu/pytorch
](
https://sourcefind.cn/#/image/dcu/pytorch
)
,根据pytorch2.1.0、python、dtk及系统下载对应的镜像版本。pytorch2.1.0镜像里已经安装了trition,flash-attn
1.
安装Rust
```
shell
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment