Commit 76716153 authored by chenzk's avatar chenzk
Browse files

Update url.md

parent 1aaa0554
...@@ -97,7 +97,7 @@ export VLLM_RANK7_NUMA=7 ...@@ -97,7 +97,7 @@ export VLLM_RANK7_NUMA=7
| 基座模型 | | | 基座模型 | |
| ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------- | | ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------- |
| [Mixtral-8x7B-Instruct-v0.1](http://113.200.138.88:18080/aimodels/Mixtral-8x7B-Instruct-v0.1) | [Mixtral-8x22B-Instruct-v0.1](http://113.200.138.88:18080/aimodels/mistralai/Mixtral-8x22B-Instruct-v0.1.git) | | [Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) | [Mixtral-8x22B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1) |
### 离线批量推理 ### 离线批量推理
...@@ -121,9 +121,7 @@ python benchmarks/benchmark_throughput.py --num-prompts 1 --input-len 32 --outpu ...@@ -121,9 +121,7 @@ python benchmarks/benchmark_throughput.py --num-prompts 1 --input-len 32 --outpu
2、使用数据集 2、使用数据集
下载数据集: 下载数据集:
```bash [sharegpt_v3_unfiltered_cleaned_split](https://huggingface.co/datasets/learnanything/sharegpt_v3_unfiltered_cleaned_split)
wget http://113.200.138.88:18080/aidatasets/vllm_data/-/raw/main/ShareGPT_V3_unfiltered_cleaned_split.json
```
```bash ```bash
python benchmarks/benchmark_throughput.py --num-prompts 1 --model mixtral/Mixtral-8x7B-Instruct-v0.1 --dataset ShareGPT_V3_unfiltered_cleaned_split.json -tp 4 --trust-remote-code --enforce-eager --dtype float16 python benchmarks/benchmark_throughput.py --num-prompts 1 --model mixtral/Mixtral-8x7B-Instruct-v0.1 --dataset ShareGPT_V3_unfiltered_cleaned_split.json -tp 4 --trust-remote-code --enforce-eager --dtype float16
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment