Skip to content

GitLab

Commit 68e9352f authored by laibao's avatar laibao
Browse files

No commit message 

No commit message
parent 721cbee2
Hide whitespace changes
Inline Side-by-side
Showing with 15 additions and 2 deletions
README.md
View file @ 68e9352f
...@@ -101,9 +101,22 @@ python examples/llava_example.py ...@@ -101,9 +101,22 @@ python examples/llava_example.py
* **去除了AWS CLI 下载逻辑** ： * **去除了AWS CLI 下载逻辑** ：
* **移除对 `subprocess` 和 `os` 模块的部分依赖** * **移除对 `subprocess` 和 `os` 模块的部分依赖**
### 离线批量推理性能测试 ### result
使用的加速卡:单卡K100_AI 模型：[llava-v1.5-7b](http://113.200.138.88:18080/aimodels/llava-v1.5-7b)
输入：
images:
<div align="center">
<img src="./doc/images.png" width="300" height="200"/>
</div>
1、指定输入输出
```bash ```bash
python benchmarks/benchmark_throughput.py --num-prompts 1 --input-len 32 --output-len 128 --model Qwen/Qwen1.5-7B-Chat -tp 1 --trust-remote-code --enforce-eager --dtype float16 python benchmarks/benchmark_throughput.py --num-prompts 1 --input-len 32 --output-len 128 --model Qwen/Qwen1.5-7B-Chat -tp 1 --trust-remote-code --enforce-eager --dtype float16
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment