Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
f4c2187e
Unverified
Commit
f4c2187e
authored
Nov 08, 2024
by
Cyrus Leung
Committed by
GitHub
Nov 08, 2024
Browse files
[Misc] Fix typo in #5895 (#10145)
Signed-off-by:
DarkLight1337
<
tlleungac@connect.ust.hk
>
parent
aea6ad62
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
benchmarks/backend_request_func.py
benchmarks/backend_request_func.py
+2
-2
No files found.
benchmarks/backend_request_func.py
View file @
f4c2187e
...
...
@@ -256,7 +256,7 @@ async def async_request_openai_completions(
async
with
session
.
post
(
url
=
api_url
,
json
=
payload
,
headers
=
headers
)
as
response
:
if
response
.
status
==
200
:
first_
valid_
chunk_received
=
False
first_chunk_received
=
False
async
for
chunk_bytes
in
response
.
content
:
chunk_bytes
=
chunk_bytes
.
strip
()
if
not
chunk_bytes
:
...
...
@@ -275,7 +275,7 @@ async def async_request_openai_completions(
if
data
[
"choices"
][
0
][
"text"
]:
timestamp
=
time
.
perf_counter
()
# First token
if
not
first_
valid_
chunk_received
:
if
not
first_chunk_received
:
first_chunk_received
=
True
ttft
=
time
.
perf_counter
()
-
st
output
.
ttft
=
ttft
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment