Unverified Commit 4d2a88bd authored by Baizhou Zhang's avatar Baizhou Zhang Committed by GitHub
Browse files

[Docs]Add instruction for manually stopping nsys profiler (#3795)


Co-authored-by: default avatarzhaochenyang20 <zhaochen20@outlook.com>
parent 45360b2f
......@@ -71,14 +71,29 @@ apt install nsight-systems-cli
2. To profile a server, e.g.
```bash
# server
# set the delay and duration times according to needs
# launch the server, set the delay and duration times according to needs
# after the duration time has been used up, server will be killed by nsys
nsys profile --trace-fork-before-exec=true --cuda-graph-trace=node -o sglang.out --delay 60 --duration 70 python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --disable-radix-cache
# client
python3 -m sglang.bench_serving --backend sglang --num-prompts 1000 --dataset-name random --random-input 1024 --random-output 512
```
In practice, we recommend users to set `--duration` argument to a large value. Whenever user wants the server to stop profiling. Firstly run:
```bash
nsys sessions list
```
to get the session id in the form of `profile-XXXXX`, then run:
```bash
nsys stop --session=profile-XXXXX
```
to manually kill the profiler and generate `nsys-rep` files instantly.
3. Use NVTX to annotate code regions, e.g. to see their execution time.
```bash
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment