"vllm/vscode:/vscode.git/clone" did not exist on "49bcd893e753d89a1c2a95a1c2649819309c1e1b"
inference_params.md 281 Bytes
Newer Older
1
# Inference Parameters
2

3
Inference parameters for vLLM APIs.
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

(sampling-params)=

## Sampling Parameters

```{eval-rst}
.. autoclass:: vllm.SamplingParams
    :members:
```

(pooling-params)=

## Pooling Parameters

```{eval-rst}
.. autoclass:: vllm.PoolingParams
    :members:
```