Commits · 2c4f5ccac1aecee0eb86ea1bd3f908b6728b133b · zhaoyu6 / sglang

".github/workflows/test-ampere.yaml" did not exist on "b7fecb353f8efdf13e4e53f2b2a8acc370ca5f7e"

16 Mar, 2025 1 commit
- Fix minor style (#4460) · 2c4f5cca
  Lianmin Zheng authored Mar 15, 2025
  
  2c4f5cca
15 Mar, 2025 1 commit
- Fix test_create_kvindices unit test (#4452) · 21d485f8
  lukec authored Mar 16, 2025
  
  21d485f8
14 Mar, 2025 1 commit

Hierarchical Caching supports MLA (#4009) · 0e0ec702

Lu Changqi authored Mar 14, 2025


Signed-off-by: Changqi Lu <luchangqi.123@bytedance.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

0e0ec702

13 Mar, 2025 5 commits
- Add a dummy grok test case (#4399) · f0afaf52
  Lianmin Zheng authored Mar 13, 2025
  
  f0afaf52
- Fix the output of hidden states after HTTP requests (#4269) · 85d2365d
  Qiaolin Yu authored Mar 13, 2025
  
  85d2365d
- Fix auto merge & add back get_flat_data_by_layer (#4393) · a5a892ff
  Lianmin Zheng authored Mar 13, 2025
  
  a5a892ff
- Improve DP attention (#4390) · 8e66fbec
  Lianmin Zheng authored Mar 13, 2025
```
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  8e66fbec
- Support page size > 1 (#4356) · c76040e3
  Lianmin Zheng authored Mar 12, 2025
  
  c76040e3
12 Mar, 2025 2 commits
- Update nightly tests (#4352) · d40ee62b
  Lianmin Zheng authored Mar 12, 2025
  
  d40ee62b
- model: Support Janus-pro (#3203) · 01090e8a
  Mick authored Mar 13, 2025
  
  01090e8a
11 Mar, 2025 2 commits
- refactor: move image processors to separate files (#4229) · ff2ce0b8
  Mick authored Mar 12, 2025
  
  ff2ce0b8
- linear support deepgemm (#4199) · dce303e2
  lukec authored Mar 11, 2025
```
Co-authored-by: yinfan98 <1106310035@qq.com>
```
  dce303e2
10 Mar, 2025 7 commits
- Fix nightly eval for neuralmagic/Mixtral-8x7B-Instruct-v0.1-FP8 (#4279) · 5524e7d0
  Lianmin Zheng authored Mar 10, 2025
  
  5524e7d0
- Amd test fp8 (#4261) · 2ac189ed
  HandH1998 authored Mar 11, 2025
  
  2ac189ed
- Fix quantization and nightly tests (#4258) · 00d25a7f
  Lianmin Zheng authored Mar 10, 2025
  
  00d25a7f
- Simplify tests & Fix trtllm custom allreduce registration (#4252) · aa957102
  Lianmin Zheng authored Mar 10, 2025
  
  aa957102
- Clean up fp8 support (#4230) · e8a69e4d
  Lianmin Zheng authored Mar 09, 2025
  
  e8a69e4d
- Auto balance CI tests (#4238) · fbd56002
  Lianmin Zheng authored Mar 09, 2025
  
  fbd56002
- Minor style fix for sgl-kernel (#4243) · 730d084f
  Lianmin Zheng authored Mar 09, 2025
  
  730d084f
09 Mar, 2025 2 commits
- Fix test of flashinfer mla with nextn (#4237) · 9dfafa74
  Baizhou Zhang authored Mar 09, 2025
  
  9dfafa74
- Support nextn for flashinfer mla attention backend (#4218) · 9fb48f95
  Baizhou Zhang authored Mar 09, 2025
  
  9fb48f95
08 Mar, 2025 4 commits
- Split test_mla.py into two files (#4216) · 48473684
  Lianmin Zheng authored Mar 08, 2025
  
  48473684
- Test no vllm custom allreduce (#4210) · 2cadd51d
  Lianmin Zheng authored Mar 08, 2025
  
  2cadd51d
- lazy import attn backends (#4200) · 08c4d764
  Lianmin Zheng authored Mar 08, 2025
  
  08c4d764
- [EAGLE] many fixes for eagle (#4195) · d4017a6b
  Lianmin Zheng authored Mar 07, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  d4017a6b
07 Mar, 2025 2 commits
- [Feature] DeepSeek V3/R1 INT8 Quantization (channel-wise) (#3888) · c7f25446
  HandH1998 authored Mar 07, 2025
```
Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: b0urnee <2769086541@qq.com>
```
  c7f25446
- Add Support for Qwen2-VL Multi-modal Embedding Models (#3694) · 361971b8
  uylnap authored Mar 07, 2025
  
  361971b8
06 Mar, 2025 3 commits
- Fix a draft model accuracy bug in eagle; support step=1; return logprob in eagle (#4134) · bc1534ff
  Lianmin Zheng authored Mar 06, 2025
```
Co-authored-by: Sehoon Kim <kssteven418@gmail.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  bc1534ff
- Split the __init__ of scheduler as smaller functions. Improve the eagle tests (#4128) · fcc2e37f
  Lianmin Zheng authored Mar 06, 2025
  
  fcc2e37f
- Remove prefill-only-one-req (#4117) · 286e6540
  Lianmin Zheng authored Mar 05, 2025
  
  286e6540
05 Mar, 2025 4 commits
- Add update_weights_from_disk endpoint to Engine (#4102) · 70b3c6ee
  Jhin authored Mar 05, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  70b3c6ee
- [Eagle] Refactor eagle speculative decoding (#3986) · d3d4d767
  Ying Sheng authored Mar 05, 2025
```
Co-authored-by: Ke Bao <ISPObaoke@163.com>
```
  d3d4d767
- [QUANT] Add GPTQModel Dynamic Quantization + `lm_head` Quantization (#3790) · 56a724eb
  Qubitium-ModelCloud authored Mar 05, 2025
```
Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>
Co-authored-by: ZX-ModelCloud <zx@modelcloud.ai>
```
  56a724eb
- example: add vlm to token in & out example (#3941) · 583d6af7
  Mick authored Mar 05, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  583d6af7
04 Mar, 2025 4 commits
- Simplify eagle tests and TP sync in grammar backend (#4066) · 77a3954b
  Lianmin Zheng authored Mar 04, 2025
  
  77a3954b
- Update nextn ci test (#4071) · 03b0364f
  Ke Bao authored Mar 05, 2025
  
  03b0364f
- [Feature] Add test for speculative_token_map (#4016) · 0d4e3228
  William authored Mar 04, 2025
  
  0d4e3228
- Reasoning parser (#4000) · 95575aa7
  Xihuai Wang authored Mar 04, 2025
```
Co-authored-by: Lucas Pickup <lupickup@microsoft.com>
```
  95575aa7
03 Mar, 2025 2 commits

Misc clean up; Remove the support of jump forward (#4032) · 935cda94
Lianmin Zheng authored Mar 03, 2025

935cda94

Support penalty in overlap mode; return logprob with chunked prefill; improve... · ac238727

Lianmin Zheng authored Mar 03, 2025


Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

ac238727