Commits · b7f3fec13c0280d341096a2db302f81ba4dbaf74 · zhaoyu6 / sglang

"vscode:/vscode.git/clone" did not exist on "92ab0a2055889d4a72edf4fb2dc1e4143161828f"

15 Jan, 2025 9 commits
- minor: rename bench for sgl kernel (#2909) · b7f3fec1
  Yineng Zhang authored Jan 16, 2025
  
  b7f3fec1
- minor: update pr test (#2908) · 58f42b1d
  Yineng Zhang authored Jan 16, 2025
  
  58f42b1d
- adapt custom allreduce for tensorrt llm (#2511) · 767c9dec
  yizhang2077 authored Jan 16, 2025
  
  767c9dec
- fix: sgl-kernel link cuda (#2906) · a53454c5
  Yineng Zhang authored Jan 16, 2025
  
  a53454c5
- optimize custom allreduce kernel (#2904) · 6cb3974e
  yizhang2077 authored Jan 16, 2025
  
  6cb3974e
- Remove normalized_prompt_logprobs from the engine to make code easier to maintain (#2902) · f65c13b5
  Lianmin Zheng authored Jan 15, 2025
  
  f65c13b5
- Disable graceful shutdown of tokenizer manager when not in the main thread (#2872) · b803b395
  Cody Yu authored Jan 15, 2025
  
  b803b395
- Add ut for w8a8 int8 quantization (#2897) · bfbda62c
  Ke Bao authored Jan 15, 2025
  
  bfbda62c
- chore: bump v0.4.1.post6 (#2899) · b3e99dfb
  Yineng Zhang authored Jan 15, 2025
  
  b3e99dfb
14 Jan, 2025 6 commits
- introduce CUB in sgl-kernel (#2887) · f005758f
  Xiaoyu Zhang authored Jan 14, 2025
  
  f005758f
- feat: support internlm 3 dense (#2888) · f5c6c667
  Yineng Zhang authored Jan 14, 2025
  
  f5c6c667
- Support w8a8 int8 quantization config (#2881) · cc0485be
  Ke Bao authored Jan 14, 2025
  
  cc0485be
- update ROCm docker for layernorm kernel optimization (#2885) · b8cd09f2
  kk authored Jan 14, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
```
  b8cd09f2
- Adjust flashinfer workspace size for Qwen2 models (#2879) · c19d8482
  Ke Bao authored Jan 14, 2025
  
  c19d8482
- docs: update README (#2878) · 80002562
  Yineng Zhang authored Jan 14, 2025
  
  80002562
13 Jan, 2025 25 commits
- Add a new api configure_logging to allow dumping the requests (#2875) · 46d44318
  Lianmin Zheng authored Jan 13, 2025
  
  46d44318
- CUDA-graph-compatible releasing and resuming KV cache and model weight memory (#2630) · 923f5183
  fzyzcjy authored Jan 14, 2025
  
  923f5183
- Sampling penalties memory interface (#2870) · d08c77c4
  Xiaoyu Zhang authored Jan 13, 2025
  
  d08c77c4
- Revert "Dump requests to a folder" (#2869) · c1e097ca
  Lianmin Zheng authored Jan 13, 2025
  
  c1e097ca
- add qwen2 eagle model (#2863) · 6ec75e62
  Lzhang-hub authored Jan 13, 2025
  
  6ec75e62
- minor: fix release docs (#2868) · d855653b
  Yineng Zhang authored Jan 13, 2025
  
  d855653b
- Fix typos in io_struct.py (#2867) · 336ff5b9
  Lianmin Zheng authored Jan 13, 2025
  
  336ff5b9
- Dump requests (#2862) · 3b141e15
  Lianmin Zheng authored Jan 13, 2025
  
  3b141e15
- Revert "Integration of TurboMind AWQ" (#2866) · 6249e4a1
  Lianmin Zheng authored Jan 13, 2025
  
  6249e4a1
- Fix quant kernel accuracy issue (#2865) · f3516c28
  Ke Bao authored Jan 13, 2025
  
  f3516c28
- Integration of TurboMind AWQ (#2828) · 17de02f9
  bjmsong authored Jan 13, 2025
```
Co-authored-by: root <bjmsong@126.com>
```
  17de02f9
- Collect more metrics: num_requests_total (#2859) · 51ab3ccf
  Lianmin Zheng authored Jan 13, 2025
  
  51ab3ccf
- Use only one GPU for MLA CI tests (#2858) · 67008f4b
  Lianmin Zheng authored Jan 13, 2025
  
  67008f4b
- minor: use ubuntu-latest instead of self-hosted runner for amd build (#2861) · 4536d724
  Yineng Zhang authored Jan 13, 2025
  
  4536d724
- docs: update link (#2857) · 41d7e5b7
  Yineng Zhang authored Jan 13, 2025
  
  41d7e5b7
- fix: not delete CNAME (#2860) · 20a9f5df
  Yineng Zhang authored Jan 13, 2025
  
  20a9f5df
- Unify sglang coding style (#2856) · 42f39099
  kk authored Jan 13, 2025
```
Co-authored-by: Lin, Soga <soga.lin@amd.com>
```
  42f39099
- Fix linear.py and improve weight loading (#2851) · 72c77763
  Lianmin Zheng authored Jan 13, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  72c77763
- [Fix]eagle2 health_generate is first request,apiserver will core (#2853) · 4093aa46
  justdoit authored Jan 13, 2025
  
  4093aa46
- Integrate ROCm ater package for ck moe function feasibility (#2854) · e808c1df
  kk authored Jan 13, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Lin, Soga <soga.lin@amd.com>
```
  e808c1df
- Update base image for ROCm (#2852) · a18ab81d
  sogalin authored Jan 13, 2025
```
Co-authored-by: HAI <hixiao@gmail.com>
```
  a18ab81d
- Support FP8 E4M3 KV Cache (#2786) · 0bb0f763
  bjmsong authored Jan 13, 2025
```
Co-authored-by: root <bjmsong@126.com>
```
  0bb0f763
- Add int8 quant kernel (#2848) · 85b2e057
  Ke Bao authored Jan 13, 2025
  
  85b2e057
- fix sgl-kernel build (#2850) · a879c2fb
  Yineng Zhang authored Jan 13, 2025
  
  a879c2fb
- add sampling_scaling_penalties kernel (#2846) · e2b16c47
  Xiaoyu Zhang authored Jan 13, 2025
  
  e2b16c47