Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
67a6882d
Unverified
Commit
67a6882d
authored
Oct 27, 2024
by
科英
Committed by
GitHub
Oct 27, 2024
Browse files
[Misc] SpecDecodeWorker supports profiling (#9719)
Signed-off-by:
Abatom
<
abatom@163.com
>
parent
6650e6a9
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
8 additions
and
0 deletions
+8
-0
vllm/spec_decode/spec_decode_worker.py
vllm/spec_decode/spec_decode_worker.py
+8
-0
No files found.
vllm/spec_decode/spec_decode_worker.py
View file @
67a6882d
...
...
@@ -1038,6 +1038,14 @@ class SpecDecodeWorker(LoraNotSupportedWorkerBase):
"""
raise
NotImplementedError
def
start_profile
(
self
):
if
isinstance
(
self
.
scorer_worker
,
Worker
):
self
.
scorer_worker
.
start_profile
()
def
stop_profile
(
self
):
if
isinstance
(
self
.
scorer_worker
,
Worker
):
self
.
scorer_worker
.
stop_profile
()
def
split_num_cache_blocks_evenly
(
scorer_cache_block_size_bytes
:
int
,
proposer_cache_block_size_bytes
:
int
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment