Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
6c7cb903
Unverified
Commit
6c7cb903
authored
Sep 10, 2024
by
Lianmin Zheng
Committed by
GitHub
Sep 11, 2024
Browse files
[Minor] improve kill scripts and torchao import (#1375)
parent
dff2860a
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
9 additions
and
7 deletions
+9
-7
python/sglang/srt/layers/torchao_utils.py
python/sglang/srt/layers/torchao_utils.py
+8
-6
test/killall_sglang.sh
test/killall_sglang.sh
+1
-1
No files found.
python/sglang/srt/layers/torchao_utils.py
View file @
6c7cb903
...
@@ -3,15 +3,17 @@ Common utilities for torchao.
...
@@ -3,15 +3,17 @@ Common utilities for torchao.
"""
"""
import
torch
import
torch
from
torchao.quantization
import
(
int4_weight_only
,
int8_dynamic_activation_int8_weight
,
int8_weight_only
,
quantize_
,
)
def
torchao_quantize_param_data
(
param
,
torchao_config
):
def
torchao_quantize_param_data
(
param
,
torchao_config
):
# Lazy import to suppress some warnings
from
torchao.quantization
import
(
int4_weight_only
,
int8_dynamic_activation_int8_weight
,
int8_weight_only
,
quantize_
,
)
dummy_linear
=
torch
.
nn
.
Linear
(
param
.
shape
[
1
],
param
.
shape
[
0
],
bias
=
False
)
dummy_linear
=
torch
.
nn
.
Linear
(
param
.
shape
[
1
],
param
.
shape
[
0
],
bias
=
False
)
dummy_linear
.
weight
=
param
dummy_linear
.
weight
=
param
if
"int8wo"
in
torchao_config
:
if
"int8wo"
in
torchao_config
:
...
...
test/killall_sglang.sh
View file @
6c7cb903
kill
-9
$(
ps aux |
grep
'sglang'
|
grep
-v
'grep'
|
awk
'{print $2}'
)
kill
-9
$(
ps aux |
grep
'sglang
.launch_server
'
|
grep
-v
'grep'
|
awk
'{print $2}'
)
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment