Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
925dbb32
Unverified
Commit
925dbb32
authored
Sep 16, 2025
by
Zaili Wang
Committed by
GitHub
Sep 16, 2025
Browse files
[CPU] fix CPU backend sel. issue for Llama4 (#10511)
parent
8df7353a
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
3 additions
and
1 deletion
+3
-1
docs/platforms/cpu_server.md
docs/platforms/cpu_server.md
+2
-0
python/sglang/srt/server_args.py
python/sglang/srt/server_args.py
+1
-1
No files found.
docs/platforms/cpu_server.md
View file @
925dbb32
...
@@ -81,6 +81,8 @@ git clone https://github.com/sgl-project/sglang.git
...
@@ -81,6 +81,8 @@ git clone https://github.com/sgl-project/sglang.git
cd
sglang
cd
sglang
git checkout <YOUR-DESIRED-VERSION>
git checkout <YOUR-DESIRED-VERSION>
# Use dedicated toml file
cp
python/pyproject_other.toml python/pyproject.toml
# Install SGLang dependent libs, and build SGLang main package
# Install SGLang dependent libs, and build SGLang main package
pip
install
--upgrade
pip setuptools
pip
install
--upgrade
pip setuptools
conda
install
-y
libsqlite
==
3.48.0 gperftools tbb libnuma numactl
conda
install
-y
libsqlite
==
3.48.0 gperftools tbb libnuma numactl
...
...
python/sglang/srt/server_args.py
View file @
925dbb32
...
@@ -2648,7 +2648,7 @@ class ServerArgs:
...
@@ -2648,7 +2648,7 @@ class ServerArgs:
# use bf16 for mxfp4 triton kernels
# use bf16 for mxfp4 triton kernels
self
.
dtype
=
"bfloat16"
self
.
dtype
=
"bfloat16"
elif
"Llama4"
in
model_arch
:
elif
"Llama4"
in
model_arch
and
self
.
device
!=
"cpu"
:
assert
self
.
attention_backend
in
{
assert
self
.
attention_backend
in
{
"fa3"
,
"fa3"
,
"aiter"
,
"aiter"
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment