Skip to content

GitLab

"tests/tool_parsers/test_pythonic_tool_parser.py" did not exist on "34a984274eae2f8fb9d1d6413abd08d7fcde741c"
Unverified Commit 10152d21 authored by Patrick von Platen's avatar Patrick von Platen Committed by GitHub
Browse files

[Realtime API] Adds minimal realtime API based on websockets (#33187) 


Signed-off-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: default avatarNick Hill <nickhill123@gmail.com>
parent 1a7894db
Hide whitespace changes
Inline Side-by-side
Showing with 4 additions and 0 deletions
vllm/v1/worker/gpu_model_runner.py
View file @ 10152d21
...@@ -68,6 +68,7 @@ from vllm.model_executor.models.interfaces import ( ...@@ -68,6 +68,7 @@ from vllm.model_executor.models.interfaces import (
supports_eagle3, supports_eagle3,
supports_mrope, supports_mrope,
supports_multimodal_pruning, supports_multimodal_pruning,
supports_realtime,
supports_transcription, supports_transcription,
supports_xdrope, supports_xdrope,
) )
...@@ -2541,6 +2542,9 @@ class GPUModelRunner( ...@@ -2541,6 +2542,9 @@ class GPUModelRunner(
supported_tasks.append("transcription") supported_tasks.append("transcription")
if supports_realtime(model):
supported_tasks.append("realtime")
return supported_tasks return supported_tasks
def get_supported_pooling_tasks(self) -> list[PoolingTask]: def get_supported_pooling_tasks(self) -> list[PoolingTask]:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment