Unverified Commit 2491cc92 authored by Zachary Streeter's avatar Zachary Streeter Committed by GitHub
Browse files

add deepseek-v3 amd docker command (#3495)

parent 67c5de92
...@@ -98,3 +98,21 @@ drun sglang_image \ ...@@ -98,3 +98,21 @@ drun sglang_image \
``` ```
With your AMD system properly configured and SGLang installed, you can now fully leverage AMD hardware to power SGLang’s machine learning capabilities. With your AMD system properly configured and SGLang installed, you can now fully leverage AMD hardware to power SGLang’s machine learning capabilities.
## Running DeepSeek-V3
The only difference in running DeepSeek-V3 is when starting the server.
```bash
drun -p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--ipc=host \
--env "HF_TOKEN=<secret>" \
sglang_image \
python3 -m sglang.launch_server \
--model deepseek-ai/DeepSeek-V3 # <- here \
--tp 8 \
--trust-remote-code \
--host 0.0.0.0 \
--port 30000
```
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment