Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
8f20fc04
Unverified
Commit
8f20fc04
authored
Apr 19, 2024
by
Uranus
Committed by
GitHub
Apr 19, 2024
Browse files
[Misc] fix docstrings (#4191)
Co-authored-by:
Zhong Wang
<
wangzhong@infini-ai.com
>
parent
221d93ec
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
6 deletions
+3
-6
vllm/sequence.py
vllm/sequence.py
+3
-6
No files found.
vllm/sequence.py
View file @
8f20fc04
...
@@ -160,7 +160,7 @@ class SequenceData:
...
@@ -160,7 +160,7 @@ class SequenceData:
self
.
_stage
=
SequenceStage
.
PREFILL
self
.
_stage
=
SequenceStage
.
PREFILL
def
get_num_uncomputed_tokens
(
self
)
->
int
:
def
get_num_uncomputed_tokens
(
self
)
->
int
:
"""Return the number of prefil tokens that are not computed."""
"""Return the number of prefil
l
tokens that are not computed."""
# we use `get_len()` which includes prompt_len + output_len instead
# we use `get_len()` which includes prompt_len + output_len instead
# of prompt_len here. This is because during recompute we need to
# of prompt_len here. This is because during recompute we need to
# prefill for both prompt and output.
# prefill for both prompt and output.
...
@@ -345,12 +345,9 @@ class Sequence:
...
@@ -345,12 +345,9 @@ class Sequence:
def
get_num_new_tokens
(
self
)
->
int
:
def
get_num_new_tokens
(
self
)
->
int
:
"""Get the number of new tokens to be computed.
"""Get the number of new tokens to be computed.
Args:
remainig_token_budget: The remaining token budgets.
Returns:
Returns:
The new number of tokens to be computed. I.e., 1 for decode, prompt
The new number of tokens to be computed. I.e., 1 for decode, or
size for prefill. If there's not enough remainig_token_budget, it
the remaining prompt size for prefill.
can return the chunked number of new tokens.
"""
"""
if
self
.
data
.
stage
==
SequenceStage
.
DECODE
:
if
self
.
data
.
stage
==
SequenceStage
.
DECODE
:
return
1
return
1
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment