Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
8deb3ea4
Unverified
Commit
8deb3ea4
authored
Jun 03, 2025
by
ishandhanani
Committed by
GitHub
Jun 03, 2025
Browse files
chore: bump sgl and update dp attn yaml (#1362)
parent
ac53c0bb
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
4 additions
and
3 deletions
+4
-3
container/Dockerfile.sglang
container/Dockerfile.sglang
+3
-2
examples/sglang/configs/disagg-dp-attention.yaml
examples/sglang/configs/disagg-dp-attention.yaml
+1
-1
No files found.
container/Dockerfile.sglang
View file @
8deb3ea4
...
@@ -136,8 +136,9 @@ RUN if [ "$ARCH" = "arm64" ]; then \
...
@@ -136,8 +136,9 @@ RUN if [ "$ARCH" = "arm64" ]; then \
# Install sglang
# Install sglang
# Once either 0.4.6post6 or 0.4.7 is released, we can switch back to using the published version
# Once either 0.4.6post6 or 0.4.7 is released, we can switch back to using the published version
# This commit references a fix for DP attention and NIXL https://github.com/sgl-project/sglang/pull/6473
# This commit references multiple perf fixes for DP attention and NIXL https://github.com/sgl-project/sglang/pull/6780
ARG SGLANG_COMMIT="e806f708c954020bda7d1cc98035a44fd6a4eb96"
# 6/2(ishan) - moving to ToT for performance purposes
ARG SGLANG_COMMIT="6376b632eb4daef306b89ede0eabdcb89ddff728"
RUN --mount=type=cache,target=/root/.cache/uv \
RUN --mount=type=cache,target=/root/.cache/uv \
git clone https://github.com/sgl-project/sglang.git && \
git clone https://github.com/sgl-project/sglang.git && \
cd sglang && \
cd sglang && \
...
...
examples/sglang/configs/disagg-dp-attention.yaml
View file @
8deb3ea4
...
@@ -14,7 +14,7 @@
...
@@ -14,7 +14,7 @@
# limitations under the License.
# limitations under the License.
Frontend
:
Frontend
:
served_model_name
:
deepseek-ai
/DeepSeek-R1-
Disti
ll-
L
la
ma-8B
served_model_name
:
silence09
/DeepSeek-R1-
Sma
ll-
2
la
yers
endpoint
:
dynamo.SGLangWorker.generate
endpoint
:
dynamo.SGLangWorker.generate
port
:
8000
port
:
8000
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment