Commits · 3a3918121f22c7a936df7cc248fa1a1b9507a0bc · change / sglang

06 Mar, 2025 2 commits
- Split the __init__ of scheduler as smaller functions. Improve the eagle tests (#4128) · fcc2e37f
  Lianmin Zheng authored Mar 06, 2025
  
  fcc2e37f
- Remove prefill-only-one-req (#4117) · 286e6540
  Lianmin Zheng authored Mar 05, 2025
  
  286e6540
05 Mar, 2025 4 commits
- Add update_weights_from_disk endpoint to Engine (#4102) · 70b3c6ee
  Jhin authored Mar 05, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  70b3c6ee
- [Eagle] Refactor eagle speculative decoding (#3986) · d3d4d767
  Ying Sheng authored Mar 05, 2025
```
Co-authored-by: Ke Bao <ISPObaoke@163.com>
```
  d3d4d767
- [QUANT] Add GPTQModel Dynamic Quantization + `lm_head` Quantization (#3790) · 56a724eb
  Qubitium-ModelCloud authored Mar 05, 2025
```
Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>
Co-authored-by: ZX-ModelCloud <zx@modelcloud.ai>
```
  56a724eb
- example: add vlm to token in & out example (#3941) · 583d6af7
  Mick authored Mar 05, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  583d6af7
04 Mar, 2025 4 commits
- Simplify eagle tests and TP sync in grammar backend (#4066) · 77a3954b
  Lianmin Zheng authored Mar 04, 2025
  
  77a3954b
- Update nextn ci test (#4071) · 03b0364f
  Ke Bao authored Mar 05, 2025
  
  03b0364f
- [Feature] Add test for speculative_token_map (#4016) · 0d4e3228
  William authored Mar 04, 2025
  
  0d4e3228
- Reasoning parser (#4000) · 95575aa7
  Xihuai Wang authored Mar 04, 2025
```
Co-authored-by: Lucas Pickup <lupickup@microsoft.com>
```
  95575aa7
03 Mar, 2025 2 commits

Misc clean up; Remove the support of jump forward (#4032) · 935cda94
Lianmin Zheng authored Mar 03, 2025

935cda94

Support penalty in overlap mode; return logprob with chunked prefill; improve... · ac238727

Lianmin Zheng authored Mar 03, 2025


Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

ac238727

02 Mar, 2025 2 commits
- Add accuracy test for TP torch compile (#3994) · d3fe9bae
  Ke Bao authored Mar 03, 2025
  
  d3fe9bae
- Refactor: Move return_hidden_states to the generate input (#3985) · 40782f05
  Qiaolin Yu authored Mar 01, 2025
```
Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
```
  40782f05
01 Mar, 2025 1 commit

[Feature]Support ragged prefill in flashinfer mla backend (#3967) · 90a4b7d9

Baizhou Zhang authored Feb 28, 2025


Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>

90a4b7d9

28 Feb, 2025 3 commits
- [Feature] SPMD for SGLang + Verl (#3852) · e3e0bc50
  fzyzcjy authored Mar 01, 2025
  
  e3e0bc50
- [Bugfix] Fix tokenizer_manager not getting 400 when req is too long (#3678) · eec3f6d1
  Chang Su authored Feb 27, 2025
```
Co-authored-by: voidxb <unkown>
```
  eec3f6d1
- [feat] Add Vertex AI compatible prediction route for /generate (#3866) · bc20e93f
  KCFindstr authored Feb 27, 2025
  
  bc20e93f
27 Feb, 2025 1 commit

Add return hidden state in the native API (#3897) · d6898dd2

Qiaolin Yu authored Feb 27, 2025


Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>

d6898dd2

26 Feb, 2025 1 commit
- [Feature] Support llguidance for constrained decoding (#3298) · 7551498a
  JC1DA authored Feb 26, 2025
  
  7551498a
25 Feb, 2025 2 commits
- [ROCm] Enable Fused MLA Triton kernel for DeepSeekV3 (#3237) · 6ce9dbe8
  Chaitanya Sri Krishna Lolla authored Feb 25, 2025
```
Co-authored-by: HAI <hixiao@gmail.com>
```
  6ce9dbe8
- Fix CI and install docs (#3821) · d7934cde
  Lianmin Zheng authored Feb 24, 2025
  
  d7934cde
24 Feb, 2025 2 commits
- Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730) · 1a6e9757
  laixin authored Feb 24, 2025
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  1a6e9757
- Fix dependency (#3813) · 27a46317
  Lianmin Zheng authored Feb 24, 2025
  
  27a46317
21 Feb, 2025 2 commits
- Add support for nvidia modelopt fp8 kv cache (#3223) · c66b2c9c
  Zhiyu authored Feb 21, 2025
  
  c66b2c9c
- feat: Add SageMaker support (#3740) · 1df6eabd
  Andrew Smith authored Feb 21, 2025
  
  1df6eabd
20 Feb, 2025 1 commit
- [Fix] Fix bugs and refactor codes in lora for better scalability. (#3652) · e79f7420
  aoshen524 authored Feb 20, 2025
```
Co-authored-by: ShenAo1111 <1377693092@qq.com>
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  e79f7420
17 Feb, 2025 3 commits
- chore: bump v0.4.3.post1 (#3638) · e782eb7e
  Yineng Zhang authored Feb 17, 2025
  
  e782eb7e
- update unit test (#3636) · e319153b
  Yineng Zhang authored Feb 17, 2025
  
  e319153b
- add mtp unit test (#3634) · 32b44d2f
  Yineng Zhang authored Feb 17, 2025
  
  32b44d2f
16 Feb, 2025 1 commit
- Model: Support Qwen 2.5 vl (#3258) · bcc213df
  Mick authored Feb 16, 2025
  
  bcc213df
15 Feb, 2025 1 commit
- fix eagle unit test (#3591) · 6718b109
  Yineng Zhang authored Feb 15, 2025
  
  6718b109
14 Feb, 2025 2 commits
- chore: bump v0.4.3 (#3556) · e0b9a423
  Yineng Zhang authored Feb 14, 2025
  
  e0b9a423
- feat: support flashinfer mla attention for deepseek v3 (#3550) · 70f894b8
  Yineng Zhang authored Feb 14, 2025
  
  70f894b8
11 Feb, 2025 1 commit
- Support Eagle cuda graph for Triton backend (#3500) · 7e6d5fc6
  Ke Bao authored Feb 12, 2025
  
  7e6d5fc6
10 Feb, 2025 3 commits
- [Feat] Return hidden states (experimental) (#3364) · 5f0e7de3
  Jackmin801 authored Feb 10, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  5f0e7de3
- Support Eagle2 for Triton backend (#3466) · 2d611323
  Ke Bao authored Feb 10, 2025
  
  2d611323
- [Fix] Fix accuracy bug and refactor codes for lora (#3413) · c45cab1c
  Baizhou Zhang authored Feb 09, 2025
  
  c45cab1c
09 Feb, 2025 2 commits
- minor: cleanup test_eagle_infer (#3415) · 60abdb3e
  Yineng Zhang authored Feb 09, 2025
  
  60abdb3e
- [Fix] Fix eagle with disable cuda graph (#3411) · 7b4e61ff
  Ying Sheng authored Feb 08, 2025
  
  7b4e61ff