Unverified Commit 7d270333 authored by ishandhanani's avatar ishandhanani Committed by GitHub
Browse files

feat: h100 wideep instructions fix (#2594)

parent 08a3763a
...@@ -11,7 +11,7 @@ SGLang allows you to deploy multi-node sized models by adding in the `dist-init- ...@@ -11,7 +11,7 @@ SGLang allows you to deploy multi-node sized models by adding in the `dist-init-
**Step 1**: Use the provided helper script to generate commands to start NATS/ETCD on your head prefill node. This script will also give you environment variables to export on each other node. You will need the IP addresses of your head prefill and head decode node to run this script. **Step 1**: Use the provided helper script to generate commands to start NATS/ETCD on your head prefill node. This script will also give you environment variables to export on each other node. You will need the IP addresses of your head prefill and head decode node to run this script.
```bash ```bash
./components/backends/sglang/src/dynamo/sglang/utils/gen_env_vars.sh ./utils/gen_env_vars.sh
``` ```
**Step 2**: Ensure that your configuration file has the required arguments. Here's an example configuration that runs prefill and the model in TP16: **Step 2**: Ensure that your configuration file has the required arguments. Here's an example configuration that runs prefill and the model in TP16:
...@@ -78,7 +78,7 @@ python3 -m dynamo.sglang \ ...@@ -78,7 +78,7 @@ python3 -m dynamo.sglang \
Node 4: Run the remaining 8 shards of the decode worker Node 4: Run the remaining 8 shards of the decode worker
```bash ```bash
python3 -m dynamo.sglang.decode_worker \ python3 -m dynamo.sglang \
--model-path /model/ \ --model-path /model/ \
--served-model-name deepseek-ai/DeepSeek-R1 \ --served-model-name deepseek-ai/DeepSeek-R1 \
--tp 16 \ --tp 16 \
......
...@@ -97,7 +97,7 @@ RUN cd dynamo/lib/bindings/python && \ ...@@ -97,7 +97,7 @@ RUN cd dynamo/lib/bindings/python && \
cd /sgl-workspace/dynamo && \ cd /sgl-workspace/dynamo && \
pip install --break-system-packages . pip install --break-system-packages .
RUN pip install --break-system-packages sglang-router==0.1.5 RUN pip install --break-system-packages sglang-router==0.1.9
RUN wget --tries=3 --waitretry=5 \ RUN wget --tries=3 --waitretry=5 \
https://github.com/nats-io/nats-server/releases/download/v2.10.28/\ https://github.com/nats-io/nats-server/releases/download/v2.10.28/\
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment