Unverified Commit 5010e0d2 authored by HAI's avatar HAI Committed by GitHub
Browse files

[3rdparty, document] Add 3rdparty/amd, with profiling and tuning instructions to be added (#1822)

parent 5e6c3265
## Profiling SGLang Infer System with AMD GPUs
This AppNote describes the SGLang profiling technical, code augment and running steps for systems with AMD Instinct GPUs, nevertheless the same procedure may work with Nvidia GPUs too.
Examples and steps are provided in detail, to facilitate easy reproduce and use to localize performance problem towards optimizations.
Two primary methods are covered:
- [RPD](https://github.com/ROCm/rocmProfileData.git)
- [Torch Profiler](https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html)
## Tuning SGLang Infer System with AMD GPUs
This AppNote describes the SGLang performance tuning technical, code harness and running steps for systems with AMD Instinct GPUs.
Harness code, examples and steps are provided in detail, to facilitate easy reproduce & use to tune performance towards workloads.
Three primary runtime areas are covered:
- Triton Kernels
- Torch Tunable Ops
- Torch Compile
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment