Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
5aefd6ac
Unverified
Commit
5aefd6ac
authored
Mar 25, 2025
by
daniel-salib
Committed by
GitHub
Mar 25, 2025
Browse files
Fix raw_request extraction in load_aware_call decorator (#15382)
Signed-off-by:
Daniel Salib
<
danielsalib@meta.com
>
parent
6c663dfd
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
10 additions
and
3 deletions
+10
-3
vllm/entrypoints/utils.py
vllm/entrypoints/utils.py
+10
-3
No files found.
vllm/entrypoints/utils.py
View file @
5aefd6ac
...
@@ -68,13 +68,20 @@ def decrement_server_load(request: Request):
...
@@ -68,13 +68,20 @@ def decrement_server_load(request: Request):
def
load_aware_call
(
func
):
def
load_aware_call
(
func
):
@
functools
.
wraps
(
func
)
@
functools
.
wraps
(
func
)
async
def
wrapper
(
*
args
,
raw_request
:
Request
,
**
kwargs
):
async
def
wrapper
(
*
args
,
**
kwargs
):
raw_request
=
kwargs
.
get
(
"raw_request"
,
args
[
1
]
if
len
(
args
)
>
1
else
None
)
if
raw_request
is
None
:
raise
ValueError
(
"raw_request required when server load tracking is enabled"
)
if
not
raw_request
.
app
.
state
.
enable_server_load_tracking
:
if
not
raw_request
.
app
.
state
.
enable_server_load_tracking
:
return
await
func
(
*
args
,
raw_request
=
raw_request
,
**
kwargs
)
return
await
func
(
*
args
,
**
kwargs
)
raw_request
.
app
.
state
.
server_load_metrics
+=
1
raw_request
.
app
.
state
.
server_load_metrics
+=
1
try
:
try
:
response
=
await
func
(
*
args
,
raw_request
=
raw_request
,
**
kwargs
)
response
=
await
func
(
*
args
,
**
kwargs
)
except
Exception
:
except
Exception
:
raw_request
.
app
.
state
.
server_load_metrics
-=
1
raw_request
.
app
.
state
.
server_load_metrics
-=
1
raise
raise
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment