Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
5e0391c0
Unverified
Commit
5e0391c0
authored
May 16, 2024
by
Alex Wu
Committed by
GitHub
May 17, 2024
Browse files
[Frontend] Separate OpenAI Batch Runner usage from API Server (#4851)
parent
dbc0754d
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
2 additions
and
1 deletion
+2
-1
vllm/entrypoints/openai/run_batch.py
vllm/entrypoints/openai/run_batch.py
+1
-1
vllm/usage/usage_lib.py
vllm/usage/usage_lib.py
+1
-0
No files found.
vllm/entrypoints/openai/run_batch.py
View file @
5e0391c0
...
...
@@ -101,7 +101,7 @@ async def main(args):
engine_args
=
AsyncEngineArgs
.
from_cli_args
(
args
)
engine
=
AsyncLLMEngine
.
from_engine_args
(
engine_args
,
usage_context
=
UsageContext
.
OPENAI_
API_SERV
ER
)
engine_args
,
usage_context
=
UsageContext
.
OPENAI_
BATCH_RUNN
ER
)
# When using single vLLM without engine_use_ray
model_config
=
await
engine
.
get_model_config
()
...
...
vllm/usage/usage_lib.py
View file @
5e0391c0
...
...
@@ -90,6 +90,7 @@ class UsageContext(str, Enum):
LLM_CLASS
=
"LLM_CLASS"
API_SERVER
=
"API_SERVER"
OPENAI_API_SERVER
=
"OPENAI_API_SERVER"
OPENAI_BATCH_RUNNER
=
"OPENAI_BATCH_RUNNER"
ENGINE_CONTEXT
=
"ENGINE_CONTEXT"
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment