Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
befc6beb
Unverified
Commit
befc6beb
authored
Nov 11, 2024
by
Lianmin Zheng
Committed by
GitHub
Nov 11, 2024
Browse files
Fix a typo in io_struct.py (#2008)
parent
59a5ba9b
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
16 additions
and
16 deletions
+16
-16
.github/workflows/pr-test.yml
.github/workflows/pr-test.yml
+14
-14
python/sglang/srt/managers/io_struct.py
python/sglang/srt/managers/io_struct.py
+1
-1
python/sglang/srt/managers/schedule_batch.py
python/sglang/srt/managers/schedule_batch.py
+1
-1
No files found.
.github/workflows/pr-test.yml
View file @
befc6beb
...
@@ -114,25 +114,25 @@ jobs:
...
@@ -114,25 +114,25 @@ jobs:
run
:
|
run
:
|
bash scripts/ci_install_dependency.sh
bash scripts/ci_install_dependency.sh
-
name
:
Benchmark
S
ingle
L
atency
-
name
:
Benchmark
s
ingle
l
atency
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
python3 -m unittest test_bench_latency.TestBenchLatency.test_default
python3 -m unittest test_bench_latency.TestBenchLatency.test_default
-
name
:
Benchmark
O
nline
L
atency
-
name
:
Benchmark
o
nline
l
atency
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
python3 -m unittest test_bench_serving.TestBenchServing.test_online_latency_default
python3 -m unittest test_bench_serving.TestBenchServing.test_online_latency_default
-
name
:
Benchmark
O
ffline
T
hroughput
-
name
:
Benchmark
o
ffline
t
hroughput
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
python3 -m unittest test_bench_serving.TestBenchServing.test_offline_throughput_default
python3 -m unittest test_bench_serving.TestBenchServing.test_offline_throughput_default
-
name
:
Benchmark
O
ffline
T
hroughput (Non-streaming, small batch size)
-
name
:
Benchmark
o
ffline
t
hroughput (Non-streaming, small batch size)
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
...
@@ -149,19 +149,19 @@ jobs:
...
@@ -149,19 +149,19 @@ jobs:
run
:
|
run
:
|
bash scripts/ci_install_dependency.sh
bash scripts/ci_install_dependency.sh
-
name
:
Benchmark
O
ffline
T
hroughput (w/o RadixAttention)
-
name
:
Benchmark
o
ffline
t
hroughput (w/o RadixAttention)
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
python3 -m unittest test_bench_serving.TestBenchServing.test_offline_throughput_without_radix_cache
python3 -m unittest test_bench_serving.TestBenchServing.test_offline_throughput_without_radix_cache
-
name
:
Benchmark
O
ffline
T
hroughput (w/ Triton)
-
name
:
Benchmark
o
ffline
t
hroughput (w/ Triton)
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
python3 -m unittest test_bench_serving.TestBenchServing.test_offline_throughput_with_triton_attention_backend
python3 -m unittest test_bench_serving.TestBenchServing.test_offline_throughput_with_triton_attention_backend
-
name
:
Benchmark
O
ffline
T
hroughput (w/ FP8)
-
name
:
Benchmark
o
ffline
t
hroughput (w/ FP8)
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
...
@@ -178,19 +178,19 @@ jobs:
...
@@ -178,19 +178,19 @@ jobs:
run
:
|
run
:
|
bash scripts/ci_install_dependency.sh
bash scripts/ci_install_dependency.sh
-
name
:
Benchmark
O
ffline
T
hroughput (TP=2)
-
name
:
Benchmark
o
ffline
t
hroughput (TP=2)
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
python3 -m unittest test_bench_serving.TestBenchServing.test_moe_offline_throughput_default
python3 -m unittest test_bench_serving.TestBenchServing.test_moe_offline_throughput_default
-
name
:
Benchmark
O
ffline
T
hroughput (w/o RadixAttention) (TP=2)
-
name
:
Benchmark
o
ffline
t
hroughput (w/o RadixAttention) (TP=2)
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
python3 -m unittest test_bench_serving.TestBenchServing.test_moe_offline_throughput_without_radix_cache
python3 -m unittest test_bench_serving.TestBenchServing.test_moe_offline_throughput_without_radix_cache
-
name
:
Benchmark
S
ingle
L
atency (TP=2)
-
name
:
Benchmark
s
ingle
l
atency (TP=2)
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
...
@@ -211,7 +211,7 @@ jobs:
...
@@ -211,7 +211,7 @@ jobs:
cd human-eval
cd human-eval
pip install -e .
pip install -e .
-
name
:
Evaluate
A
ccuracy
-
name
:
Evaluate
a
ccuracy
timeout-minutes
:
20
timeout-minutes
:
20
run
:
|
run
:
|
cd test/srt
cd test/srt
...
@@ -232,20 +232,20 @@ jobs:
...
@@ -232,20 +232,20 @@ jobs:
cd human-eval
cd human-eval
pip install -e .
pip install -e .
-
name
:
Evaluate
A
ccuracy (TP=2)
-
name
:
Evaluate
a
ccuracy (TP=2)
timeout-minutes
:
20
timeout-minutes
:
20
run
:
|
run
:
|
cd test/srt
cd test/srt
python3 test_moe_eval_accuracy_large.py
python3 test_moe_eval_accuracy_large.py
-
name
:
Evaluate MLA
A
ccuracy (TP=2)
-
name
:
Evaluate MLA
a
ccuracy (TP=2)
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
python3 test_mla.py
python3 test_mla.py
python3 test_mla_fp8.py
python3 test_mla_fp8.py
-
name
:
Evaluate
D
ata
P
arallelism
A
ccuracy (DP=2)
-
name
:
Evaluate
d
ata
p
arallelism
a
ccuracy (DP=2)
timeout-minutes
:
10
timeout-minutes
:
10
run
:
|
run
:
|
cd test/srt
cd test/srt
...
...
python/sglang/srt/managers/io_struct.py
View file @
befc6beb
...
@@ -184,7 +184,7 @@ class TokenizedGenerateReqInput:
...
@@ -184,7 +184,7 @@ class TokenizedGenerateReqInput:
input_text
:
str
input_text
:
str
# The input token ids
# The input token ids
input_ids
:
List
[
int
]
input_ids
:
List
[
int
]
# The image input
# The image input
s
image_inputs
:
dict
image_inputs
:
dict
# The sampling parameters
# The sampling parameters
sampling_params
:
SamplingParams
sampling_params
:
SamplingParams
...
...
python/sglang/srt/managers/schedule_batch.py
View file @
befc6beb
...
@@ -212,7 +212,7 @@ class Req:
...
@@ -212,7 +212,7 @@ class Req:
# this does not include the jump forward tokens.
# this does not include the jump forward tokens.
self
.
completion_tokens_wo_jump_forward
=
0
self
.
completion_tokens_wo_jump_forward
=
0
# For
vision
inputs
# For
multimodal
inputs
self
.
image_inputs
:
Optional
[
ImageInputs
]
=
None
self
.
image_inputs
:
Optional
[
ImageInputs
]
=
None
# Prefix info
# Prefix info
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment