Commit 764f75fe authored by PanZezhong's avatar PanZezhong
Browse files

update readme

parent b4e3f49f
...@@ -15,13 +15,23 @@ xmake && xmake install ...@@ -15,13 +15,23 @@ xmake && xmake install
- 运行模型推理测试 - 运行模型推理测试
```bash ```bash
python jiuge.py [--cpu | --nvidia | --cambricon | --ascend | --metax | --moore] <path/to/model_dir> [n_device] python scripts/jiuge.py [--cpu | --nvidia | --cambricon | --ascend | --metax | --moore] path/to/model_dir [n_device]
``` ```
- 部署模型推理服务 - 部署模型推理服务
```bash ```bash
launch_server.py [-h] [--dev {cpu,nvidia,cambricon,ascend,metax,moore}] python scripts/launch_server.py --model-path MODEL_PATH [-h] [--dev {cpu,nvidia,cambricon,ascend,metax,moore}] [--ndev NDEV] [--max-batch MAX_BATCH] [--max-tokens MAX_TOKENS]
[--model-path MODEL_PATH] [--ndev NDEV] [--max-batch MAX_BATCH] ```
[--max-tokens MAX_TOKENS]
- 测试模型推理服务性能
```bash
python scripts/test_perf.py
```
- 使用推理服务测试模型困惑度(Perplexity)
```bash
python scripts/test_ppl.py --model-path MODEL_PATH [--ndev NDEV] [--max-batch MAX_BATCH] [--max-tokens MAX_TOKENS]
``` ```
...@@ -90,7 +90,7 @@ if __name__ == "__main__": ...@@ -90,7 +90,7 @@ if __name__ == "__main__":
dataset = load_dataset("wikitext", "wikitext-2-raw-v1", split="test") dataset = load_dataset("wikitext", "wikitext-2-raw-v1", split="test")
texts = dataset["text"] texts = dataset["text"]
texts = [t for t in texts if len(t.strip()) > 0] texts = [t.strip() for t in texts if len(t.strip()) > 0]
input_ids_list = [] input_ids_list = []
for text in texts: for text in texts:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment