Commits · 0fe7c13be18d1edd8682747ce558b430a1aa1c9e · change / sglang · GitLab

08 Mar, 2025 7 commits
- Fix bench_serving flush cache not recognizing OPENAI_API_KEY (#4181) · 0fe7c13b
  Mingshan authored Mar 08, 2025
```
Signed-off-by: Mingshan <git@brighill.com>
```
  0fe7c13b
- lazy import attn backends (#4200) · 08c4d764
  Lianmin Zheng authored Mar 08, 2025
  
  08c4d764
- Revert "Minor improvement to per_tensor_quant_fp8 (#4197)" (#4198) · 96d0e37f
  Yineng Zhang authored Mar 07, 2025
  
  96d0e37f
- Minor improvement to per_tensor_quant_fp8 (#4197) · 90bb2be2
  Rex authored Mar 07, 2025
  
  90bb2be2
- Remove the vllm dependency from the moe_align function (#4164) · b93ef5e5
  lukec authored Mar 08, 2025
```
Co-authored-by: Hongbosherlock <hongbosherlock@gmail.com>
```
  b93ef5e5
- [EAGLE] many fixes for eagle (#4195) · d4017a6b
  Lianmin Zheng authored Mar 07, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  d4017a6b
- New clang format for sgl kernel (#4194) · d052f4c8
  Lianmin Zheng authored Mar 07, 2025
  
  d052f4c8
07 Mar, 2025 19 commits
- Update amd ci docker image to v0.4.3.post4-rocm630. (#4189) · e1aaa79a
  saienduri authored Mar 07, 2025
  
  e1aaa79a
- Fix eagle hang issue for max_new_tokens=1 (#4185) · 20c81199
  Ke Bao authored Mar 08, 2025
  
  20c81199
- use same version for ci and pyproject (#4187) · 70866b6f
  Yineng Zhang authored Mar 07, 2025
  
  70866b6f
- Revert "ROCm: Flex Attention Enablement with custom backends (#4178)" (#4186) · eb61f5c9
  Yineng Zhang authored Mar 07, 2025
  
  eb61f5c9
- ROCm: Flex Attention Enablement with custom backends (#4178) · 0beea450
  HAI authored Mar 07, 2025
```
Co-authored-by: linsun12 <linsun12@amd.com>
```
  0beea450
- [Docs] Improve bullets appearance and grammar (#4174) · c827c671
  Michael Yao authored Mar 07, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  c827c671
- fix int8 doc link (#4179) · b55a621f
  Yineng Zhang authored Mar 07, 2025
  
  b55a621f
- Add an example of using deepseekv3 int8 sglang. (#4177) · ffa1b3e3
  lukec authored Mar 07, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  ffa1b3e3
- update release-pypi-kernel · 7e3bb527
  Yineng Zhang authored Mar 07, 2025
  
  7e3bb527
- chore: bump v0.0.3.post7 for sgl-kernel (#4176) · 96263f27
  Yineng Zhang authored Mar 07, 2025
  
  96263f27
- Memory pool fix for upstream change about eagle (#4170) · 9376ac36
  Zhiqiang Xie authored Mar 07, 2025
  
  9376ac36
- Put utils in ifndef USE_ROCM to fix CI (#4167) (#4168) · 94a2b9d3
  Yineng Zhang authored Mar 07, 2025
```
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
```
  94a2b9d3
- Remove non-existent AMD header include (#4166) · 3c3eb374
  Stefan He authored Mar 06, 2025
  
  3c3eb374
- [Docs] Fix links and grammar issues (#4162) · d557319a
  Michael Yao authored Mar 07, 2025
  
  d557319a
- [Refactor] Reducing code duplication across FP8 CUDA quantization kernels (#4163) · 95085d65
  Stefan He authored Mar 06, 2025
  
  95085d65
- [Feature] DeepSeek V3/R1 INT8 Quantization (channel-wise) (#3888) · c7f25446
  HandH1998 authored Mar 07, 2025
```
Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: b0urnee <2769086541@qq.com>
```
  c7f25446
- Add sgl_per_token_quant_fp8 (#4089) · 63ee26d1
  Stefan He authored Mar 06, 2025
  
  63ee26d1
- [quant kernel] sgl-kernel support per_tensor_quant fp8 (#3786) · ad55f171
  Xiaoyu Zhang authored Mar 07, 2025
  
  ad55f171
- Add Support for Qwen2-VL Multi-modal Embedding Models (#3694) · 361971b8
  uylnap authored Mar 07, 2025
  
  361971b8
06 Mar, 2025 14 commits
- ROCm: enable trillion-parameter MoE models with INT4-FP8 single node (#4152) · 13bc39c5
  HAI authored Mar 06, 2025
  
  13bc39c5
- Hot fix small vocal eagle in docs (#4154) · 9854a18a
  Chayenne authored Mar 06, 2025
```
Co-authored-by: ybyang <ybyang7@iflytek.com>
```
  9854a18a
- Docs: add torch compile cache (#4151) · ebddb65a
  Chayenne authored Mar 06, 2025
```
Co-authored-by: ybyang <ybyang7@iflytek.com>
```
  ebddb65a
- [docs] fix HF reference script command (#4148) · 19fd57bc
  Adarsh Shirawalmath authored Mar 07, 2025
  
  19fd57bc
- Release v0.4.3.post4 (#4140) · 9c58e68b
  Lianmin Zheng authored Mar 06, 2025
  
  9c58e68b
- Fix constrained generation errors by adding datasets dependency (#4142) · d03b3467
  Oliver Stanley authored Mar 06, 2025
  
  d03b3467
- Fix nightly ci Gsm8k & Fix flashinfer backend kvcache quant (#4147) · ab7fba0e
  yinfan98 authored Mar 07, 2025
  
  ab7fba0e
- Fix a draft model accuracy bug in eagle; support step=1; return logprob in eagle (#4134) · bc1534ff
  Lianmin Zheng authored Mar 06, 2025
```
Co-authored-by: Sehoon Kim <kssteven418@gmail.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  bc1534ff
- fix bench serving bug (#4135) · 3a391812
  Lzhang-hub authored Mar 06, 2025
  
  3a391812
- Update CODEOWNER (#4138) · 800bf018
  Lianmin Zheng authored Mar 06, 2025
  
  800bf018
- AMD/ROCm: update base image string (#4137) · b16af90b
  kk authored Mar 06, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: yichiche <yichiche@amd.com>
```
  b16af90b
- [Minor] make the `__init__` function of model_runner.py shorter (#4132) · 98c73d71
  Lianmin Zheng authored Mar 06, 2025
  
  98c73d71
- Split the __init__ of scheduler as smaller functions. Improve the eagle tests (#4128) · fcc2e37f
  Lianmin Zheng authored Mar 06, 2025
  
  fcc2e37f
- remove unused max_jobs in setup_rocm.py (#4126) · 0804dd11
  Liu Jinjie authored Mar 06, 2025
```
Signed-off-by: Jinjie Liu <jinjie.liu@usc.edu>
```
  0804dd11