Commits · dce303e279e11258dff57e1c25410b2f570a7a2f · change / sglang

11 Mar, 2025 1 commit
- linear support deepgemm (#4199) · dce303e2
  lukec authored Mar 11, 2025
```
Co-authored-by: yinfan98 <1106310035@qq.com>
```
  dce303e2
10 Mar, 2025 7 commits
- Fix nightly eval for neuralmagic/Mixtral-8x7B-Instruct-v0.1-FP8 (#4279) · 5524e7d0
  Lianmin Zheng authored Mar 10, 2025
  
  5524e7d0
- Amd test fp8 (#4261) · 2ac189ed
  HandH1998 authored Mar 11, 2025
  
  2ac189ed
- Fix quantization and nightly tests (#4258) · 00d25a7f
  Lianmin Zheng authored Mar 10, 2025
  
  00d25a7f
- Simplify tests & Fix trtllm custom allreduce registration (#4252) · aa957102
  Lianmin Zheng authored Mar 10, 2025
  
  aa957102
- Clean up fp8 support (#4230) · e8a69e4d
  Lianmin Zheng authored Mar 09, 2025
  
  e8a69e4d
- Auto balance CI tests (#4238) · fbd56002
  Lianmin Zheng authored Mar 09, 2025
  
  fbd56002
- Minor style fix for sgl-kernel (#4243) · 730d084f
  Lianmin Zheng authored Mar 09, 2025
  
  730d084f
09 Mar, 2025 2 commits
- Fix test of flashinfer mla with nextn (#4237) · 9dfafa74
  Baizhou Zhang authored Mar 09, 2025
  
  9dfafa74
- Support nextn for flashinfer mla attention backend (#4218) · 9fb48f95
  Baizhou Zhang authored Mar 09, 2025
  
  9fb48f95
08 Mar, 2025 4 commits
- Split test_mla.py into two files (#4216) · 48473684
  Lianmin Zheng authored Mar 08, 2025
  
  48473684
- Test no vllm custom allreduce (#4210) · 2cadd51d
  Lianmin Zheng authored Mar 08, 2025
  
  2cadd51d
- lazy import attn backends (#4200) · 08c4d764
  Lianmin Zheng authored Mar 08, 2025
  
  08c4d764
- [EAGLE] many fixes for eagle (#4195) · d4017a6b
  Lianmin Zheng authored Mar 07, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  d4017a6b
07 Mar, 2025 2 commits
- [Feature] DeepSeek V3/R1 INT8 Quantization (channel-wise) (#3888) · c7f25446
  HandH1998 authored Mar 07, 2025
```
Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: b0urnee <2769086541@qq.com>
```
  c7f25446
- Add Support for Qwen2-VL Multi-modal Embedding Models (#3694) · 361971b8
  uylnap authored Mar 07, 2025
  
  361971b8
06 Mar, 2025 3 commits
- Fix a draft model accuracy bug in eagle; support step=1; return logprob in eagle (#4134) · bc1534ff
  Lianmin Zheng authored Mar 06, 2025
```
Co-authored-by: Sehoon Kim <kssteven418@gmail.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  bc1534ff
- Split the __init__ of scheduler as smaller functions. Improve the eagle tests (#4128) · fcc2e37f
  Lianmin Zheng authored Mar 06, 2025
  
  fcc2e37f
- Remove prefill-only-one-req (#4117) · 286e6540
  Lianmin Zheng authored Mar 05, 2025
  
  286e6540
05 Mar, 2025 4 commits
- Add update_weights_from_disk endpoint to Engine (#4102) · 70b3c6ee
  Jhin authored Mar 05, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  70b3c6ee
- [Eagle] Refactor eagle speculative decoding (#3986) · d3d4d767
  Ying Sheng authored Mar 05, 2025
```
Co-authored-by: Ke Bao <ISPObaoke@163.com>
```
  d3d4d767
- [QUANT] Add GPTQModel Dynamic Quantization + `lm_head` Quantization (#3790) · 56a724eb
  Qubitium-ModelCloud authored Mar 05, 2025
```
Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>
Co-authored-by: ZX-ModelCloud <zx@modelcloud.ai>
```
  56a724eb
- example: add vlm to token in & out example (#3941) · 583d6af7
  Mick authored Mar 05, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  583d6af7
04 Mar, 2025 4 commits
- Simplify eagle tests and TP sync in grammar backend (#4066) · 77a3954b
  Lianmin Zheng authored Mar 04, 2025
  
  77a3954b
- Update nextn ci test (#4071) · 03b0364f
  Ke Bao authored Mar 05, 2025
  
  03b0364f
- [Feature] Add test for speculative_token_map (#4016) · 0d4e3228
  William authored Mar 04, 2025
  
  0d4e3228
- Reasoning parser (#4000) · 95575aa7
  Xihuai Wang authored Mar 04, 2025
```
Co-authored-by: Lucas Pickup <lupickup@microsoft.com>
```
  95575aa7
03 Mar, 2025 2 commits

Misc clean up; Remove the support of jump forward (#4032) · 935cda94
Lianmin Zheng authored Mar 03, 2025

935cda94

Support penalty in overlap mode; return logprob with chunked prefill; improve... · ac238727

Lianmin Zheng authored Mar 03, 2025


Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

ac238727

02 Mar, 2025 2 commits
- Add accuracy test for TP torch compile (#3994) · d3fe9bae
  Ke Bao authored Mar 03, 2025
  
  d3fe9bae
- Refactor: Move return_hidden_states to the generate input (#3985) · 40782f05
  Qiaolin Yu authored Mar 01, 2025
```
Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
```
  40782f05
01 Mar, 2025 1 commit

[Feature]Support ragged prefill in flashinfer mla backend (#3967) · 90a4b7d9

Baizhou Zhang authored Feb 28, 2025


Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>

90a4b7d9

28 Feb, 2025 3 commits
- [Feature] SPMD for SGLang + Verl (#3852) · e3e0bc50
  fzyzcjy authored Mar 01, 2025
  
  e3e0bc50
- [Bugfix] Fix tokenizer_manager not getting 400 when req is too long (#3678) · eec3f6d1
  Chang Su authored Feb 27, 2025
```
Co-authored-by: voidxb <unkown>
```
  eec3f6d1
- [feat] Add Vertex AI compatible prediction route for /generate (#3866) · bc20e93f
  KCFindstr authored Feb 27, 2025
  
  bc20e93f
27 Feb, 2025 1 commit

Add return hidden state in the native API (#3897) · d6898dd2

Qiaolin Yu authored Feb 27, 2025


Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>

d6898dd2

26 Feb, 2025 1 commit
- [Feature] Support llguidance for constrained decoding (#3298) · 7551498a
  JC1DA authored Feb 26, 2025
  
  7551498a
25 Feb, 2025 2 commits
- [ROCm] Enable Fused MLA Triton kernel for DeepSeekV3 (#3237) · 6ce9dbe8
  Chaitanya Sri Krishna Lolla authored Feb 25, 2025
```
Co-authored-by: HAI <hixiao@gmail.com>
```
  6ce9dbe8
- Fix CI and install docs (#3821) · d7934cde
  Lianmin Zheng authored Feb 24, 2025
  
  d7934cde
24 Feb, 2025 1 commit
- Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730) · 1a6e9757
  laixin authored Feb 24, 2025
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  1a6e9757