Skip to content

GitLab

Unverified Commit c8a9e791 authored by Dr. Artificial曾小健's avatar Dr. Artificial曾小健 Committed by GitHub
Browse files

Fix readme (#1236)

parent 3602692c
Hide whitespace changes
Inline Side-by-side
Showing with 4 additions and 0 deletions
README.md
View file @ c8a9e791
...@@ -83,6 +83,7 @@ docker run --gpus all \ ...@@ -83,6 +83,7 @@ docker run --gpus all \
### Method 4: Using docker compose ### Method 4: Using docker compose
<details> <details>
<summary>More</summary>
> This method is recommended if you plan to serve it as a service. > This method is recommended if you plan to serve it as a service.
> A better approach is to use the [k8s-sglang-service.yaml](./docker/k8s-sglang-service.yaml). > A better approach is to use the [k8s-sglang-service.yaml](./docker/k8s-sglang-service.yaml).
...@@ -94,6 +95,7 @@ docker run --gpus all \ ...@@ -94,6 +95,7 @@ docker run --gpus all \
### Method 5: Run on Kubernetes or Clouds with SkyPilot ### Method 5: Run on Kubernetes or Clouds with SkyPilot
<details> <details>
<summary>More</summary>
To deploy on Kubernetes or 12+ clouds, you can use [SkyPilot](https://github.com/skypilot-org/skypilot). To deploy on Kubernetes or 12+ clouds, you can use [SkyPilot](https://github.com/skypilot-org/skypilot).
...@@ -262,6 +264,7 @@ Instructions for supporting a new model are [here](https://github.com/sgl-projec ...@@ -262,6 +264,7 @@ Instructions for supporting a new model are [here](https://github.com/sgl-projec
#### Use Models From ModelScope #### Use Models From ModelScope
<details> <details>
<summary>More</summary>
To use a model from [ModelScope](https://www.modelscope.cn), set the environment variable SGLANG_USE_MODELSCOPE. To use a model from [ModelScope](https://www.modelscope.cn), set the environment variable SGLANG_USE_MODELSCOPE.
``` ```
...@@ -276,6 +279,7 @@ SGLANG_USE_MODELSCOPE=true python -m sglang.launch_server --model-path qwen/Qwen ...@@ -276,6 +279,7 @@ SGLANG_USE_MODELSCOPE=true python -m sglang.launch_server --model-path qwen/Qwen
#### Run Llama 3.1 405B #### Run Llama 3.1 405B
<details> <details>
<summary>More</summary>
```bash ```bash
# Run 405B (fp8) on a single node # Run 405B (fp8) on a single node
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment