Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
4ea92f83
Unverified
Commit
4ea92f83
authored
Jan 29, 2024
by
Lianmin Zheng
Committed by
GitHub
Jan 29, 2024
Browse files
Format code (#118)
parent
6b0af285
Changes
6
Hide whitespace changes
Inline
Side-by-side
Showing
6 changed files
with
6 additions
and
1 deletion
+6
-1
README.md
README.md
+1
-1
python/sglang/srt/models/llama2.py
python/sglang/srt/models/llama2.py
+1
-0
python/sglang/srt/models/llava.py
python/sglang/srt/models/llava.py
+1
-0
python/sglang/srt/models/mixtral.py
python/sglang/srt/models/mixtral.py
+1
-0
python/sglang/srt/models/qwen.py
python/sglang/srt/models/qwen.py
+1
-0
python/sglang/srt/models/qwen2.py
python/sglang/srt/models/qwen2.py
+1
-0
No files found.
README.md
View file @
4ea92f83
...
...
@@ -324,7 +324,7 @@ python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port
-
Mixtral
-
LLaVA
-
`python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000`
-
Qwen
-
Qwen
/ Qwen 2
-
AWQ quantization
## Benchmark And Performance
...
...
python/sglang/srt/models/llama2.py
View file @
4ea92f83
...
...
@@ -319,4 +319,5 @@ class LlamaForCausalLM(nn.Module):
weight_loader
=
getattr
(
param
,
"weight_loader"
,
default_weight_loader
)
weight_loader
(
param
,
loaded_weight
)
EntryClass
=
LlamaForCausalLM
python/sglang/srt/models/llava.py
View file @
4ea92f83
...
...
@@ -331,4 +331,5 @@ def monkey_path_clip_vision_embed_forward():
clip_vision_embed_forward
,
)
EntryClass
=
LlavaLlamaForCausalLM
python/sglang/srt/models/mixtral.py
View file @
4ea92f83
...
...
@@ -377,4 +377,5 @@ class MixtralForCausalLM(nn.Module):
weight_loader
=
getattr
(
param
,
"weight_loader"
,
default_weight_loader
)
weight_loader
(
param
,
loaded_weight
)
EntryClass
=
MixtralForCausalLM
python/sglang/srt/models/qwen.py
View file @
4ea92f83
...
...
@@ -259,4 +259,5 @@ class QWenLMHeadModel(nn.Module):
weight_loader
=
getattr
(
param
,
"weight_loader"
,
default_weight_loader
)
weight_loader
(
param
,
loaded_weight
)
EntryClass
=
QWenLMHeadModel
python/sglang/srt/models/qwen2.py
View file @
4ea92f83
...
...
@@ -315,4 +315,5 @@ class Qwen2ForCausalLM(nn.Module):
weight_loader
=
getattr
(
param
,
"weight_loader"
,
default_weight_loader
)
weight_loader
(
param
,
loaded_weight
)
EntryClass
=
Qwen2ForCausalLM
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment