Unverified Commit fb7abfc1 authored by Angela Yi's avatar Angela Yi Committed by GitHub
Browse files

[docs] Improve tlparse section (#33211)


Signed-off-by: default avatarangelayi <yiangela7@gmail.com>
parent 5d3d6e44
...@@ -39,10 +39,9 @@ For more details on the design, please see the following resources: ...@@ -39,10 +39,9 @@ For more details on the design, please see the following resources:
## Use tlparse ## Use tlparse
Use [tlparse](https://github.com/meta-pytorch/tlparse) to acquire torch.compile logs. These logs show all stages of the compilation process, Use [tlparse](https://github.com/meta-pytorch/tlparse) to view torch.compile
including the fused kernels that torch.compile produces. logs. These logs show all stages of the compilation process, including the fused
If you can, we recommend sending these or pieces of these along with any bug reports -- kernels that torch.compile produces.
they are very helpful.
Install tlparse: Install tlparse:
...@@ -50,11 +49,16 @@ Install tlparse: ...@@ -50,11 +49,16 @@ Install tlparse:
pip install tlparse pip install tlparse
``` ```
To enable the torch.compile logs, you can set the envvar `TORCH_TRACE=<dir>`.
During tracing, a file per rank will be created inside of that directory, with
each file containing the artifacts during compilation. If you can, we recommend
sending these log files along with bug reports -- they are very helpful.
Usage (offline inference) Usage (offline inference)
```sh ```sh
TORCH_TRACE=~/trace_dir python my_script.py TORCH_TRACE=~/trace_dir python my_script.py
tlparse ~/trace_dir/<the_first_log_file> tlparse ~/trace_dir/<rank_0_log_file>
``` ```
Usage (serving) Usage (serving)
...@@ -62,10 +66,11 @@ Usage (serving) ...@@ -62,10 +66,11 @@ Usage (serving)
```sh ```sh
TORCH_TRACE=~/trace_dir vllm serve TORCH_TRACE=~/trace_dir vllm serve
# ctrl-c out of the server # ctrl-c out of the server
tlparse ~/trace_dir/<the_first_log_file> tlparse ~/trace_dir/<rank_0_log_file>
``` ```
The `tlparse` command outputs some HTML files (perhaps into e.g. `./tl_out/index.html`). Given one of the log files, the `tlparse` command outputs some HTML files
(perhaps into e.g. `./tl_out/index.html`).
Open it to see the logs. It'll look something like the following: Open it to see the logs. It'll look something like the following:
![tlparse example](../assets/design/debug_vllm_compile/tlparse_inductor.png) ![tlparse example](../assets/design/debug_vllm_compile/tlparse_inductor.png)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment