Skip to content

GitLab

Commit 5a160e1c authored by zhuwenwen's avatar zhuwenwen
Browse files

update readme

parent c0bc033b
Hide whitespace changes
Inline Side-by-side
Showing with 12 additions and 12 deletions
README.md
View file @ 5a160e1c
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
* @Author: zhuww * @Author: zhuww
* @email: zhuww@sugon.com * @email: zhuww@sugon.com
* @Date: 2024-05-24 14:15:07 * @Date: 2024-05-24 14:15:07
* @LastEditTime: 2024-06-20 08:40:01 * @LastEditTime: 2024-08-23 14:40:01
--> -->
# Qwen1.5 # Qwen1.5
...@@ -42,7 +42,7 @@ pip install aiohttp==3.9.1 outlines==0.0.37 openai==1.23.3 -i http://mirrors.ali ...@@ -42,7 +42,7 @@ pip install aiohttp==3.9.1 outlines==0.0.37 openai==1.23.3 -i http://mirrors.ali
# <Host Path>主机端路径 # <Host Path>主机端路径
# <Container Path>容器映射路径 # <Container Path>容器映射路径
docker build -t qwen1.5:latest . docker build -t qwen1.5:latest .
docker run -it --name qwen1.5_vllm --privileged --shm-size=64G --device=/dev/kfd --device=/dev/dri/ --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --ulimit memlock=-1:-1 --ipc=host --network host --group-add video -v /opt/hyhal:/opt/hyhal -v <Host Path>:<Container Path> qwen1.5:latest /bin/bash docker run -it --name qwen1.5_vllm --privileged --shm-size=64G --device=/dev/kfd --device=/dev/dri/ --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --ulimit memlock=-1:-1 --ipc=host --network host --group-add video -v /opt/hyhal:/opt/hyhal:ro -v <Host Path>:<Container Path> qwen1.5:latest /bin/bash
``` ```
`Tips：若在K100/Z100L上使用，需要替换flash_attn，下载链接：https://forum.hpccube.com/thread/515` `Tips：若在K100/Z100L上使用，需要替换flash_attn，下载链接：https://forum.hpccube.com/thread/515`
...@@ -52,7 +52,7 @@ conda create -n qwen1.5_vllm python=3.10 ...@@ -52,7 +52,7 @@ conda create -n qwen1.5_vllm python=3.10
pip install aiohttp==3.9.1 outlines==0.0.37 openai==1.23.3 pip install aiohttp==3.9.1 outlines==0.0.37 openai==1.23.3
``` ```
关于本项目DCU显卡所需的特殊深度学习库可从[光合](https://developer.hpccube.com/tool/)开发者社区下载安装。 关于本项目DCU显卡所需的特殊深度学习库可从[光合](https://developer.hpccube.com/tool/)开发者社区下载安装。
* DTK驱动：dtk24.04 * DTK驱动：dtk24.04.1
* Pytorch: 2.1.0 * Pytorch: 2.1.0
* triton:2.1.0 * triton:2.1.0
* vllm: 0.3.3 * vllm: 0.3.3
...@@ -83,16 +83,16 @@ cd dist && pip install vllm* ...@@ -83,16 +83,16 @@ cd dist && pip install vllm*
| 基座模型 | chat模型 | GPTQ模型 | | 基座模型 | chat模型 | GPTQ模型 |
| ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | | ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| [Qwen-7B](https://huggingface.co/Qwen/Qwen-7B) | [Qwen-7B-Chat](https://huggingface.co/Qwen/Qwen-7B-Chat) | | [Qwen-7B](https://huggingface.co/Qwen/Qwen-7B) | [Qwen-7B-Chat](http://113.200.138.88:18080/aimodels/Qwen-7B-Chat) |
| [Qwen-14B](https://huggingface.co/Qwen/Qwen-14B) | [Qwen-14B-Chat](https://huggingface.co/Qwen/Qwen-14B-Chat) | | [Qwen-14B](https://huggingface.co/Qwen/Qwen-14B) | [Qwen-14B-Chat](http://113.200.138.88:18080/aimodels/Qwen-14B-Chat) |
| [Qwen-72B](https://huggingface.co/Qwen/Qwen-72B) | [Qwen-72B-Chat](https://huggingface.co/Qwen/Qwen-72B-Chat) | | [Qwen-72B](http://113.200.138.88:18080/aimodels/qwen/Qwen-72B) | [Qwen-72B-Chat](http://113.200.138.88:18080/aimodels/Qwen-72B-Chat) |
| [Qwen1.5-7B](https://huggingface.co/Qwen/Qwen1.5-7B) | [Qwen1.5-7B-Chat](https://huggingface.co/Qwen/Qwen1.5-7B-Chat) | [Qwen1.5-7B-Chat-GPTQ-Int4](https://huggingface.co/Qwen/Qwen1.5-7B-Chat-GPTQ-Int4) | | [Qwen1.5-7B](https://huggingface.co/Qwen/Qwen1.5-7B) | [Qwen1.5-7B-Chat](https://huggingface.co/Qwen/Qwen1.5-7B-Chat) | [Qwen1.5-7B-Chat-GPTQ-Int4](https://huggingface.co/Qwen/Qwen1.5-7B-Chat-GPTQ-Int4) |
| [Qwen1.5-14B](https://huggingface.co/Qwen/Qwen1.5-14B) | [Qwen1.5-14B-Chat](https://huggingface.co/Qwen/Qwen1.5-14B-Chat) | [Qwen1.5-14B-Chat-GPTQ-Int4](https://huggingface.co/Qwen/Qwen1.5-14B-Chat-GPTQ-Int4) | | [Qwen1.5-14B](https://huggingface.co/Qwen/Qwen1.5-14B) | [Qwen1.5-14B-Chat](http://113.200.138.88:18080/aimodels/qwen/Qwen1.5-14B-Chat) | [Qwen1.5-14B-Chat-GPTQ-Int4](https://huggingface.co/Qwen/Qwen1.5-14B-Chat-GPTQ-Int4) |
| [Qwen1.5-32B](https://huggingface.co/Qwen/Qwen1.5-32B) | [Qwen1.5-32B-Chat](https://huggingface.co/Qwen/Qwen1.5-32B-Chat) | [Qwen1.5-32B-Chat-GPTQ-Int4](https://huggingface.co/Qwen/Qwen1.5-32B-Chat-GPTQ-Int4) | | [Qwen1.5-32B](http://113.200.138.88:18080/aimodels/Qwen1.5-32B) | [Qwen1.5-32B-Chat](http://113.200.138.88:18080/aimodels/Qwen1.5-32B-Chat) | [Qwen1.5-32B-Chat-GPTQ-Int4](http://113.200.138.88:18080/aimodels/Qwen1.5-32B-Chat-GPTQ-Int4) |
| [Qwen1.5-72B](https://huggingface.co/Qwen/Qwen1.5-72B) | [Qwen1.5-72B-Chat](https://huggingface.co/Qwen/Qwen1.5-72B-Chat) | [Qwen1.5-72B-Chat-GPTQ-Int4](https://huggingface.co/Qwen/Qwen1.5-72B-Chat-GPTQ-Int4) | | [Qwen1.5-72B](http://113.200.138.88:18080/aimodels/Qwen1.5-72B) | [Qwen1.5-72B-Chat](http://113.200.138.88:18080/aimodels/Qwen1.5-72B-Chat) | [Qwen1.5-72B-Chat-GPTQ-Int4](https://huggingface.co/Qwen/Qwen1.5-72B-Chat-GPTQ-Int4) |
| [Qwen1.5-110B](https://huggingface.co/Qwen/Qwen1.5-110B) | [Qwen1.5-110B-Chat](https://huggingface.co/Qwen/Qwen1.5-110B-Chat) | [Qwen1.5-110B-Chat-GPTQ-Int4](https://huggingface.co/Qwen/Qwen1.5-110B-Chat-GPTQ-Int4) | | [Qwen1.5-110B](http://113.200.138.88:18080/aimodels/Qwen1.5-110B) | [Qwen1.5-110B-Chat](http://113.200.138.88:18080/aimodels/Qwen1.5-110B-Chat) | [Qwen1.5-110B-Chat-GPTQ-Int4](https://huggingface.co/Qwen/Qwen1.5-110B-Chat-GPTQ-Int4) |
| [Qwen2-7B](https://huggingface.co/Qwen/Qwen2-7B) | [Qwen2-7B-Instruct](https://huggingface.co/Qwen/Qwen2-7B-Instruct) | [Qwen2-7B-Instruct-GPTQ-Int4](https://huggingface.co/Qwen/Qwen2-7B-Instruct-GPTQ-Int4) | | [Qwen2-7B](http://113.200.138.88:18080/aimodels/Qwen2-7B) | [Qwen2-7B-Instruct](http://113.200.138.88:18080/aimodels/Qwen2-7B-Instruct) | [Qwen2-7B-Instruct-GPTQ-Int4](https://huggingface.co/Qwen/Qwen2-7B-Instruct-GPTQ-Int4) |
| [Qwen2-72B](https://huggingface.co/Qwen/Qwen2-72B) | [Qwen2-72B-Instruct](https://huggingface.co/Qwen/Qwen2-72B-Instruct) | [Qwen2-72B-Instruct-GPTQ-Int4](https://huggingface.co/Qwen/Qwen2-72B-Instruct-GPTQ-Int4) | | [Qwen2-72B](http://113.200.138.88:18080/aimodels/Qwen2-72B) | [Qwen2-72B-Instruct](http://113.200.138.88:18080/aimodels/Qwen2-72B-Instruct) | [Qwen2-72B-Instruct-GPTQ-Int4](https://huggingface.co/Qwen/Qwen2-72B-Instruct-GPTQ-Int4) |
......
icon.png 0 → 100644
View file @ 5a160e1c
icon.png

53.8 KB

Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment