Unverified Commit 9ccc6ded authored by Reid's avatar Reid Committed by GitHub
Browse files

[doc] add missing import (#18133)


Signed-off-by: default avatarreidliu41 <reid201711@gmail.com>
Co-authored-by: default avatarreidliu41 <reid201711@gmail.com>
parent d62a076e
...@@ -74,6 +74,8 @@ Tensor parallelism (`tensor_parallel_size` option) can be used to split the mode ...@@ -74,6 +74,8 @@ Tensor parallelism (`tensor_parallel_size` option) can be used to split the mode
The following code splits the model across 2 GPUs. The following code splits the model across 2 GPUs.
```python ```python
from vllm import LLM
llm = LLM(model="ibm-granite/granite-3.1-8b-instruct", llm = LLM(model="ibm-granite/granite-3.1-8b-instruct",
tensor_parallel_size=2) tensor_parallel_size=2)
``` ```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment