Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
llama_tgi
Commits
8b75592e
Commit
8b75592e
authored
Oct 31, 2024
by
xuxzh1
🎱
Browse files
update readme
parent
7c6fd42e
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
17 deletions
+4
-17
README.md
README.md
+4
-17
No files found.
README.md
View file @
8b75592e
...
@@ -20,30 +20,17 @@ LLama是一个基础语言模型的集合,参数范围从7B到65B。在数万亿
...
@@ -20,30 +20,17 @@ LLama是一个基础语言模型的集合,参数范围从7B到65B。在数万亿
## 环境配置
## 环境配置
### Docker(方法一)
### Docker(方法一)
TODO
##
# Dockerfile(方法二)
##
**TODO**
```
### 源码编译安装(方法二)
cd ./text-generation-inference
docker build -f Dockerfile_dcu -t tgi:latest --ulimit nofile=2048:2048 .
[
readme中方法二
](
https://developer.sourcefind.cn/codes/OpenDAS/text-generation-inference/-/blob/v2.1.1/README.md
)
# <Host Path>主机端路径
# <Container Path>容器映射路径
docker run -it --name llama_tgi --privileged --shm-size=64G --device=/dev/kfd --device=/dev/dri/ --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --ulimit memlock=-1:-1 --ipc=host --network host --group-add video -v /opt/hyhal:/opt/hyhal:ro -v <Host Path>:<Container Path> tgi:latest /bin/bash
```
## 数据集
## 数据集
无
无
## 推理
## 推理
### 源码编译安装
参考源码里的
[
README
](
./text-generation-inference/README.md
)
源码编译部分。
本项目源码编译需要的工具包、深度学习库等均可从
[
光合
](
https://developer.hpccube.com/tool/
)
开发者社区下载安装。
-
[
DTK 24.04
](
https://cancon.hpccube.com:65024/1/main/DTK-24.04
)
-
[
Pytorch 2.1.0
](
https://cancon.hpccube.com:65024/4/main/pytorch/DAS1.0
)
-
[
Flash_attn 2.0.4
](
https://cancon.hpccube.com:65024/4/main/flash_attn/DAS1.0
)
-
[
Triton 2.1.0
](
https://cancon.hpccube.com:65024/4/main/triton/DAS1.0
)
### 模型下载
### 模型下载
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment