Commit 25144e8b authored by laibao's avatar laibao
Browse files

Update README.md

parent c9354e8e
......@@ -87,14 +87,14 @@ conda create -n qwen2.5_vllm python=3.10
| [Qwen2.5-32B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-32B) | [Qwen2.5-32B-Instruct](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-32B-Instruct) | [Qwen2.5-32B-Instruct-GPTQ-Int4](https://huggingface.co/Qwen/Qwen2.5-32B-Instruct-GPTQ-Int4) | [Qwen2.5-32B-Instruct-AWQ](https://huggingface.co/Qwen/Qwen2.5-32B-Instruct-AWQ) |
| [Qwen2.5-72B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-72B) | [Qwen2.5-72B-Instruct](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-72B-Instruct) | [Qwen2.5-72B-Instruct-GPTQ-Int4](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-72B-Instruct-GPTQ-Int4) | [Qwen2.5-72B-Instruct-AWQ](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-72B-Instruct-AWQ) |
| [ Qwen2.5 Coder 1.5B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-1.5B) | [Qwen2.5-Coder-1.5B-Instruct](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-1.5B-Instruct) | [Qwen2.5-Coder-1.5B-Instruct-GPTQ-Int4](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-1.5B-Instruct-GPTQ-Int4) | [Qwen2.5-Coder-1.5B-Instruct-AWQ](http://113.200.138.88:18080/aimodels/qwen/qwen2.5-coder-1.5b-instruct-awq) |
| [Qwen2.5 Coder 7B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-7B) | [Qwen2.5 Coder 7B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-7B) | [Qwen2.5 Coder 7B Instruct GPTQ Int4](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-7B-Instruct-GPTQ-Int4) | [Qwen2.5 Coder 7B Instruct AWQ](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-7B-Instruct-AWQ) |
| [Qwen2.5 Coder 7B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-7B) | [Qwen2.5 Coder 7B Instruct](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-7B-Instruct) | [Qwen2.5 Coder 7B Instruct GPTQ Int4](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-7B-Instruct-GPTQ-Int4) | [Qwen2.5 Coder 7B Instruct AWQ](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-7B-Instruct-AWQ) |
| [Qwen2.5 Math 1.5B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Math-1.5B) | [Qwen2.5 Math 1.5B Instruct](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Math-1.5B-Instruct) | | |
| [ Qwen2.5 Math 7B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Math-7B) | [Qwen2.5-Math-7B-Instruct](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Math-7B-Instruct) | | |
### 离线批量推理
```bash
python examples/offline_inference.py
-python examples/offline_inference.py
```
其中,`prompts`为提示词;`temperature`为控制采样随机性的值,值越小模型生成越确定,值变高模型生成更随机,0表示贪婪采样,默认为1;`max_tokens=16`为生成长度,默认为1;
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment