Skip to content

GitLab

"git@developer.sourcefind.cn:zhaoyu6/sglang.git" did not exist on "c827c671f74438b32243d2d9add27cdf29511ab0"
Unverified Commit 86311c23 authored by Hailey Schoelkopf's avatar Hailey Schoelkopf Committed by GitHub
Browse files

Update README.md

parent 6409fe6c
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
README.md
View file @ 86311c23
...@@ -133,7 +133,7 @@ We also support vLLM for faster inference on [supported model types](https://doc ...@@ -133,7 +133,7 @@ We also support vLLM for faster inference on [supported model types](https://doc
```bash ```bash
lm_eval --model vllm \ lm_eval --model vllm \
--model_args pretrained={model_name},tensor_parallel_size={GPUs_per_model},dtype=auto,gpu_memory_utilization=0.8,data_parallel={model_replicas} \ --model_args pretrained={model_name},tensor_parallel_size={GPUs_per_model},dtype=auto,gpu_memory_utilization=0.8,data_parallel_size={model_replicas} \
--tasks lambada_openai \ --tasks lambada_openai \
--batch_size auto --batch_size auto
``` ```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment