Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
2491cc92
Unverified
Commit
2491cc92
authored
Feb 11, 2025
by
Zachary Streeter
Committed by
GitHub
Feb 12, 2025
Browse files
add deepseek-v3 amd docker command (#3495)
parent
67c5de92
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
18 additions
and
0 deletions
+18
-0
docs/references/amd.md
docs/references/amd.md
+18
-0
No files found.
docs/references/amd
_configure
.md
→
docs/references/amd.md
View file @
2491cc92
...
...
@@ -98,3 +98,21 @@ drun sglang_image \
```
With your AMD system properly configured and SGLang installed, you can now fully leverage AMD hardware to power SGLang’s machine learning capabilities.
## Running DeepSeek-V3
The only difference in running DeepSeek-V3 is when starting the server.
```
bash
drun
-p
30000:30000
\
-v
~/.cache/huggingface:/root/.cache/huggingface
\
--ipc
=
host
\
--env
"HF_TOKEN=<secret>"
\
sglang_image
\
python3
-m
sglang.launch_server
\
--model
deepseek-ai/DeepSeek-V3
# <- here \
--tp
8
\
--trust-remote-code
\
--host
0.0.0.0
\
--port
30000
```
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment