Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
ChatGLM_vllm
Commits
4f8680ee
Commit
4f8680ee
authored
Aug 30, 2024
by
zhuwenwen
Browse files
update readme
parent
460e2662
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
5 additions
and
7 deletions
+5
-7
README.md
README.md
+5
-7
icon.png
icon.png
+0
-0
No files found.
README.md
View file @
4f8680ee
...
@@ -2,7 +2,7 @@
...
@@ -2,7 +2,7 @@
*
@Author: zhuww
*
@Author: zhuww
*
@email: zhuww@sugon.com
*
@email: zhuww@sugon.com
*
@Date: 2024-06-13 14:38:07
*
@Date: 2024-06-13 14:38:07
*
@LastEditTime: 2024-0
6-2
0 0
8
:35:01
*
@LastEditTime: 2024-0
8-3
0 0
9
:35:01
-->
-->
## 论文
## 论文
...
@@ -22,7 +22,6 @@ ChatGLM-6B 是清华大学开源的开源的、支持中英双语的对话语言
...
@@ -22,7 +22,6 @@ ChatGLM-6B 是清华大学开源的开源的、支持中英双语的对话语言
| 模型名称 | 隐含层维度 | 层数 | 头数 | 词表大小 | 位置编码 | 最大序列长度 |
| 模型名称 | 隐含层维度 | 层数 | 头数 | 词表大小 | 位置编码 | 最大序列长度 |
| ----------- | ---------- | ---- | ---- | -------- | -------- | ------------ |
| ----------- | ---------- | ---- | ---- | -------- | -------- | ------------ |
| ChatGLM-6B | 4096 | 28 | 32 | 130528 | RoPE | 2048 |
| ChatGLM2-6B | 4096 | 28 | 32 | 65024 | RoPE | 8192 |
| ChatGLM2-6B | 4096 | 28 | 32 | 65024 | RoPE | 8192 |
| ChatGLM3-6B | 4096 | 28 | 32 | 65024 | RoPE | 8192 |
| ChatGLM3-6B | 4096 | 28 | 32 | 65024 | RoPE | 8192 |
| glm-4-9b | 4096 | 40 | 32 | 151552 | RoPE | 131072 |
| glm-4-9b | 4096 | 40 | 32 | 151552 | RoPE | 131072 |
...
@@ -57,7 +56,7 @@ pip install aiohttp==3.9.1 outlines==0.0.37 openai==1.23.3 -i http://mirrors.ali
...
@@ -57,7 +56,7 @@ pip install aiohttp==3.9.1 outlines==0.0.37 openai==1.23.3 -i http://mirrors.ali
# <Host Path>主机端路径
# <Host Path>主机端路径
# <Container Path>容器映射路径
# <Container Path>容器映射路径
docker build -t chatglm:latest .
docker build -t chatglm:latest .
docker run -it --name chatglm_vllm --privileged --shm-size=64G --device=/dev/kfd --device=/dev/dri/ --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --ulimit memlock=-1:-1 --ipc=host --network host --group-add video -v /opt/hyhal:/opt/hyhal -v <Host Path>:<Container Path> llama:latest /bin/bash
docker run -it --name chatglm_vllm --privileged --shm-size=64G --device=/dev/kfd --device=/dev/dri/ --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --ulimit memlock=-1:-1 --ipc=host --network host --group-add video -v /opt/hyhal:/opt/hyhal
:ro
-v <Host Path>:<Container Path> llama:latest /bin/bash
```
```
`Tips:若在K100/Z100L上使用,需要替换flash_attn,下载链接:https://forum.hpccube.com/thread/515`
`Tips:若在K100/Z100L上使用,需要替换flash_attn,下载链接:https://forum.hpccube.com/thread/515`
...
@@ -67,7 +66,7 @@ conda create -n chatglm_vllm python=3.10
...
@@ -67,7 +66,7 @@ conda create -n chatglm_vllm python=3.10
pip install aiohttp==3.9.1 outlines==0.0.37 openai==1.23.3
pip install aiohttp==3.9.1 outlines==0.0.37 openai==1.23.3
```
```
关于本项目DCU显卡所需的特殊深度学习库可从
[
光合
](
https://developer.hpccube.com/tool/
)
开发者社区下载安装。
关于本项目DCU显卡所需的特殊深度学习库可从
[
光合
](
https://developer.hpccube.com/tool/
)
开发者社区下载安装。
*
DTK驱动:dtk24.04
*
DTK驱动:dtk24.04
.1
*
Pytorch: 2.1.0
*
Pytorch: 2.1.0
*
triton:2.1.0
*
triton:2.1.0
*
vllm: 0.3.3
*
vllm: 0.3.3
...
@@ -97,9 +96,8 @@ cd dist && pip install vllm*
...
@@ -97,9 +96,8 @@ cd dist && pip install vllm*
| chat模型 | 长文本模型 |
| chat模型 | 长文本模型 |
| ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
|
[
chatglm-6b
](
https://huggingface.co/THUDM/chatglm-6b
)
|
|
[
chatglm2-6b
](
http://113.200.138.88:18080/aimodels/chatglm2-6b
)
|
[
chatglm2-6b-32k
](
https://huggingface.co/THUDM/chatglm2-6b-32k
)
|
|
[
chatglm2-6b
](
https://huggingface.co/THUDM/chatglm2-6b
)
|
[
chatglm2-6b-32k
](
https://huggingface.co/THUDM/chatglm2-6b-32k
)
|
|
[
chatglm3-6b
](
http://113.200.138.88:18080/aimodels/chatglm3-6b
)
|
[
chatglm3-6b-32k
](
http://113.200.138.88:18080/aimodels/chatglm3-6b-32k
)
|
|
[
chatglm3-6b
](
https://huggingface.co/THUDM/chatglm3-6b
)
|
[
chatglm3-6b-32k
](
https://huggingface.co/THUDM/chatglm3-6b-32k
)
|
|
[
glm-4-9b-chat
](
https://huggingface.co/THUDM/glm-4-9b-chat
)
|
|
[
glm-4-9b-chat
](
https://huggingface.co/THUDM/glm-4-9b-chat
)
|
...
...
icon.png
0 → 100644
View file @
4f8680ee
62.1 KB
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment