Commits · 89ccb533ad390d9cf9c75dacf6eec4130901aede · change / sglang · GitLab

09 Mar, 2025 3 commits
- use sgl-kernel 0.0.4 (#4224) · 89ccb533
  Yineng Zhang authored Mar 08, 2025
  
  89ccb533
- Lazily import lora backends (#4225) · 1361ab9e
  Lianmin Zheng authored Mar 08, 2025
  
  1361ab9e
- Rename files in sgl kernel to avoid nested folder structure (#4213) · 8abf74e3
  Lianmin Zheng authored Mar 08, 2025
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  8abf74e3
08 Mar, 2025 4 commits
- Fix bench_serving flush cache not recognizing OPENAI_API_KEY (#4181) · 0fe7c13b
  Mingshan authored Mar 08, 2025
```
Signed-off-by: Mingshan <git@brighill.com>
```
  0fe7c13b
- lazy import attn backends (#4200) · 08c4d764
  Lianmin Zheng authored Mar 08, 2025
  
  08c4d764
- [EAGLE] many fixes for eagle (#4195) · d4017a6b
  Lianmin Zheng authored Mar 07, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  d4017a6b
- New clang format for sgl kernel (#4194) · d052f4c8
  Lianmin Zheng authored Mar 07, 2025
  
  d052f4c8
07 Mar, 2025 6 commits
- Fix eagle hang issue for max_new_tokens=1 (#4185) · 20c81199
  Ke Bao authored Mar 08, 2025
  
  20c81199
- Revert "ROCm: Flex Attention Enablement with custom backends (#4178)" (#4186) · eb61f5c9
  Yineng Zhang authored Mar 07, 2025
  
  eb61f5c9
- ROCm: Flex Attention Enablement with custom backends (#4178) · 0beea450
  HAI authored Mar 07, 2025
```
Co-authored-by: linsun12 <linsun12@amd.com>
```
  0beea450
- Memory pool fix for upstream change about eagle (#4170) · 9376ac36
  Zhiqiang Xie authored Mar 07, 2025
  
  9376ac36
- [Feature] DeepSeek V3/R1 INT8 Quantization (channel-wise) (#3888) · c7f25446
  HandH1998 authored Mar 07, 2025
```
Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: b0urnee <2769086541@qq.com>
```
  c7f25446
- Add Support for Qwen2-VL Multi-modal Embedding Models (#3694) · 361971b8
  uylnap authored Mar 07, 2025
  
  361971b8
06 Mar, 2025 14 commits
- ROCm: enable trillion-parameter MoE models with INT4-FP8 single node (#4152) · 13bc39c5
  HAI authored Mar 06, 2025
  
  13bc39c5
- Release v0.4.3.post4 (#4140) · 9c58e68b
  Lianmin Zheng authored Mar 06, 2025
  
  9c58e68b
- Fix constrained generation errors by adding datasets dependency (#4142) · d03b3467
  Oliver Stanley authored Mar 06, 2025
  
  d03b3467
- Fix nightly ci Gsm8k & Fix flashinfer backend kvcache quant (#4147) · ab7fba0e
  yinfan98 authored Mar 07, 2025
  
  ab7fba0e
- Fix a draft model accuracy bug in eagle; support step=1; return logprob in eagle (#4134) · bc1534ff
  Lianmin Zheng authored Mar 06, 2025
```
Co-authored-by: Sehoon Kim <kssteven418@gmail.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  bc1534ff
- fix bench serving bug (#4135) · 3a391812
  Lzhang-hub authored Mar 06, 2025
  
  3a391812
- [Minor] make the `__init__` function of model_runner.py shorter (#4132) · 98c73d71
  Lianmin Zheng authored Mar 06, 2025
  
  98c73d71
- Split the __init__ of scheduler as smaller functions. Improve the eagle tests (#4128) · fcc2e37f
  Lianmin Zheng authored Mar 06, 2025
  
  fcc2e37f
- Add a pointer to the real KV cache pool (#4113) · aee30630
  Zhiqiang Xie authored Mar 05, 2025
  
  aee30630
- Remove prefill-only-one-req (#4117) · 286e6540
  Lianmin Zheng authored Mar 05, 2025
  
  286e6540
- [Hoxfix] Fix incomplete token_to_kv_pool refactor (#4121) · 718c391f
  Wenxuan Tan authored Mar 05, 2025
  
  718c391f
- chore: bump v0.4.3.post3 (#4114) · fc671f66
  Yineng Zhang authored Mar 05, 2025
  
  fc671f66
- Online serving benchmarks of real datasets for hierarchical KV caching (#3211) · 25482edb
  Yueyang Pan authored Mar 06, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  25482edb
- Debug radixcache: refactor recursive helper methods (#3029) · 62b362b1
  luzengxiangcn authored Mar 06, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  62b362b1
05 Mar, 2025 9 commits
- Add update_weights_from_disk endpoint to Engine (#4102) · 70b3c6ee
  Jhin authored Mar 05, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  70b3c6ee
- Fix triton kernel illegal memory issue for eagle (#4100) · ef9d3b3c
  Ke Bao authored Mar 06, 2025
  
  ef9d3b3c
- [Revision] Add fast decode plan for flashinfer mla (#4012) · fc91d08a
  Baizhou Zhang authored Mar 05, 2025
  
  fc91d08a
- Fix the moe padding conditional logic (#4081) · 71ab0dab
  HAI authored Mar 05, 2025
  
  71ab0dab
- [Eagle] Refactor eagle speculative decoding (#3986) · d3d4d767
  Ying Sheng authored Mar 05, 2025
```
Co-authored-by: Ke Bao <ISPObaoke@163.com>
```
  d3d4d767
- ROCM: AITER BLOCK GEMM (#4075) · 5be8f1ed
  yigex authored Mar 05, 2025
  
  5be8f1ed
- [QUANT] Add GPTQModel Dynamic Quantization + `lm_head` Quantization (#3790) · 56a724eb
  Qubitium-ModelCloud authored Mar 05, 2025
```
Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>
Co-authored-by: ZX-ModelCloud <zx@modelcloud.ai>
```
  56a724eb
- example: add vlm to token in & out example (#3941) · 583d6af7
  Mick authored Mar 05, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  583d6af7
- [Minor] more code cleanup (#4077) · e074d84e
  Lianmin Zheng authored Mar 04, 2025
  
  e074d84e
04 Mar, 2025 4 commits
- Simplify eagle tests and TP sync in grammar backend (#4066) · 77a3954b
  Lianmin Zheng authored Mar 04, 2025
  
  77a3954b
- Revert "Fix nightly-test CI" (#4065) · 2dd7d0c5
  Lianmin Zheng authored Mar 04, 2025
  
  2dd7d0c5
- [Feature] Add test for speculative_token_map (#4016) · 0d4e3228
  William authored Mar 04, 2025
  
  0d4e3228
- fix: support gelu_new activation function in gpt2 (#3712) · 9545bfb2
  Xiuyu Li authored Mar 04, 2025
  
  9545bfb2