Commit 764f75fe authored by PanZezhong's avatar PanZezhong
Browse files

update readme

parent b4e3f49f
......@@ -15,13 +15,23 @@ xmake && xmake install
- 运行模型推理测试
```bash
python jiuge.py [--cpu | --nvidia | --cambricon | --ascend | --metax | --moore] <path/to/model_dir> [n_device]
python scripts/jiuge.py [--cpu | --nvidia | --cambricon | --ascend | --metax | --moore] path/to/model_dir [n_device]
```
- 部署模型推理服务
```bash
launch_server.py [-h] [--dev {cpu,nvidia,cambricon,ascend,metax,moore}]
[--model-path MODEL_PATH] [--ndev NDEV] [--max-batch MAX_BATCH]
[--max-tokens MAX_TOKENS]
python scripts/launch_server.py --model-path MODEL_PATH [-h] [--dev {cpu,nvidia,cambricon,ascend,metax,moore}] [--ndev NDEV] [--max-batch MAX_BATCH] [--max-tokens MAX_TOKENS]
```
- 测试模型推理服务性能
```bash
python scripts/test_perf.py
```
- 使用推理服务测试模型困惑度(Perplexity)
```bash
python scripts/test_ppl.py --model-path MODEL_PATH [--ndev NDEV] [--max-batch MAX_BATCH] [--max-tokens MAX_TOKENS]
```
......@@ -90,7 +90,7 @@ if __name__ == "__main__":
dataset = load_dataset("wikitext", "wikitext-2-raw-v1", split="test")
texts = dataset["text"]
texts = [t for t in texts if len(t.strip()) > 0]
texts = [t.strip() for t in texts if len(t.strip()) > 0]
input_ids_list = []
for text in texts:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment