Skip to content

GitLab

Unverified Commit 0c0c2015 authored by Robert Shaw's avatar Robert Shaw Committed by GitHub
Browse files

Update openai_compatible_server.md (#11536) 


Co-authored-by: default avatarSimon Mo <simon.mo@hey.com>
parent 82d24f7a
Show whitespace changes
Inline Side-by-side
Showing with 7 additions and 1 deletion
docs/source/serving/openai_compatible_server.md
View file @ 0c0c2015
......@@ -112,7 +112,13 @@ completion = client.chat.completions.create(
## Extra HTTP Headers
Only `X-Request-Id` HTTP request header is supported for now.
Only `X-Request-Id` HTTP request header is supported for now. It can be enabled
with `--enable-request-id-headers`.
> Note that enablement of the headers can impact performance significantly at high QPS
> rates. We recommend implementing HTTP headers at the router level (e.g. via Istio),
> rather than within the vLLM layer for this reason.
> See https://github.com/vllm-project/vllm/pull/11529 for more details.
```python
completion = client.chat.completions.create(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment