Commit b886f1bf authored by dengjb's avatar dengjb
Browse files

update

parent c10a3c70
......@@ -25,7 +25,7 @@ Seed-Coder,这是一个强大的、透明的、参数高效的开源代码模
| flash_attn | 2.6.1+das.opt1.dtk2504 |
| flash_mla | 1.0.0+das.opt1.dtk25042 |
当前仅支持镜像:
推荐使用镜像:
- 挂载地址`-v`根据实际模型情况修改
```bash
......@@ -50,8 +50,6 @@ docker run -it --shm-size 60g --network=host --name seed_coder --privileged --de
export ALLREDUCE_STREAM_WITH_COMPUTE=1
export VLLM_MLA_DISABLE=0
export VLLM_USE_FLASH_MLA=1
vllm serve /path/of/ByteDance/Seed-Coder-8B-Instruct/ \
--trust-remote-code \
--max-model-len 32768 \
......@@ -84,7 +82,7 @@ DCU与GPU精度一致,推理框架:vllm。
## 预训练权重
| 模型名称 | 权重大小 | DCU型号 | 最低卡数需求 |下载地址|
|:-----:|:----------:|:----------:|:---------------------:|:----------:|
| Seed-Coder-8B-Instruct | 8B | K100AI | 1| [下载地址](https://huggingface.co/ByteDance-Seed/Seed-Coder-8B-Instruct) |
| Seed-Coder-8B-Instruct | 8B | K100AI | 1| [huggingface](https://huggingface.co/ByteDance-Seed/Seed-Coder-8B-Instruct) |
## 源码仓库及问题反馈
- https://developer.sourcefind.cn/codes/modelzoo/seedcoder_vllm
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment