"vscode:/vscode.git/clone" did not exist on "4240b045e6cd354ecc69a047f6f3e95ddca581c2"
Commit 82dafb34 authored by baberabb's avatar baberabb
Browse files

nits

parent f45f67e4
...@@ -113,7 +113,7 @@ We also support vLLM for faster inference on [supported model types](https://doc ...@@ -113,7 +113,7 @@ We also support vLLM for faster inference on [supported model types](https://doc
```bash ```bash
lm_eval --model vllm \ lm_eval --model vllm \
--model_args pretrained={model_name},tensor_parallel_size={number of GPUs to use},dtype=auto,gpu_memory_utilization=0.8 \ --model_args pretrained={model_name},tensor_parallel_size={GPUs_per_model},dtype=auto,gpu_memory_utilization=0.8,data_parallel={model_replicas} \
--tasks lambada_openai \ --tasks lambada_openai \
--batch_size auto --batch_size auto
``` ```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment