Commits · 0194948fd96802ea41d6baa3673e9d42cfa5dc8c · change / sglang · GitLab

03 Mar, 2025 4 commits
- Fix nightly-test CI (#3826) · b4d34cd3
  yinfan98 authored Mar 03, 2025
  
  b4d34cd3
- Revert "Add fast decode plan for flashinfer mla" (#4008) · 9e1014cf
  Lianmin Zheng authored Mar 02, 2025
  
  9e1014cf
- Add fast decode plan for flashinfer mla (#3987) · fa561067
  Baizhou Zhang authored Mar 02, 2025
  
  fa561067
- [feat] add small vocab table for eagle's draft model[1]. (#3822) · 7fbab730
  Zhousx authored Mar 03, 2025
```
Co-authored-by: Achazwl <323163497@qq.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  7fbab730
02 Mar, 2025 3 commits
- Enable custom AR for AMD GPUs and maintain it in sgl-kernel (#3406) · 9cf40772
  Hubert Lu authored Mar 02, 2025
  
  9cf40772
- Fix all gather torch compile (#3992) · 00ce7e31
  Ke Bao authored Mar 02, 2025
```
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  00ce7e31
- Refactor: Move return_hidden_states to the generate input (#3985) · 40782f05
  Qiaolin Yu authored Mar 01, 2025
```
Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
```
  40782f05
01 Mar, 2025 3 commits
- rename FunctionCallReqInput to ParseFunctionCallReq (#3976) · 930da877
  Chayenne authored Feb 28, 2025
  
  930da877
- [Feature]Support ragged prefill in flashinfer mla backend (#3967) · 90a4b7d9
  Baizhou Zhang authored Feb 28, 2025
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>
```
  90a4b7d9
- update flashinfer-python version · f3b99f73
  Yineng Zhang authored Feb 28, 2025
  
  f3b99f73
28 Feb, 2025 7 commits
- Remove unused imports from rocm mla kernel. (#3963) · 77a6c9d2
  Chaitanya Sri Krishna Lolla authored Feb 28, 2025
  
  77a6c9d2
- [Feature] SPMD for SGLang + Verl (#3852) · e3e0bc50
  fzyzcjy authored Mar 01, 2025
  
  e3e0bc50
- [Feature] integrate Structural Tag in xgrammar backend for function calling (#3566) · bac414ab
  mlmz authored Feb 28, 2025
```
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
```
  bac414ab
- [Bugfix] Fix tokenizer_manager not getting 400 when req is too long (#3678) · eec3f6d1
  Chang Su authored Feb 27, 2025
```
Co-authored-by: voidxb <unkown>
```
  eec3f6d1
- set a strict sgl-kernel version (#3950) · 90bc26a8
  Chayenne authored Feb 27, 2025
  
  90bc26a8
- Fix bench_serving not recognizing OPENAI_API_KEY (#3870) · ec0a72c2
  Kebe authored Feb 28, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  ec0a72c2
- [feat] Add Vertex AI compatible prediction route for /generate (#3866) · bc20e93f
  KCFindstr authored Feb 27, 2025
  
  bc20e93f
27 Feb, 2025 6 commits
- Fix the doc link for sampling params (#3861) · d3887852
  Qiaolin Yu authored Feb 27, 2025
  
  d3887852
- upgrade flashinfer v0.2.2.post1 (#3934) · 564bdf29
  Yineng Zhang authored Feb 27, 2025
  
  564bdf29
- Improve: Support xgrammar 0.1.14 (#3593) · d2815879
  Enrique Shockwave authored Feb 27, 2025
  
  d2815879
- Tuning Script for Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3922) · b0df5d24
  laixin authored Feb 27, 2025
```
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
```
  b0df5d24
- Add return hidden state in the native API (#3897) · d6898dd2
  Qiaolin Yu authored Feb 27, 2025
```
Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  d6898dd2
- [Rocm] Fix to the rocm_mla_decode_rope.py returning random result (#3898) · 8b681d77
  Tianxing Wu authored Feb 27, 2025
  
  8b681d77
26 Feb, 2025 2 commits
- [Feature] Support llguidance for constrained decoding (#3298) · 7551498a
  JC1DA authored Feb 26, 2025
  
  7551498a
- Expert Parallelism (EP) Support for DeepSeek V3/R1 (#3602) · 21463e32
  lukec authored Feb 26, 2025
```
Co-authored-by: laixin <xielx@shanghaitech.edu.cn>
Co-authored-by: HandH1998 <1335248067@qq.com>
Co-authored-by: laixin <q865809639@gmail.com>
```
  21463e32
25 Feb, 2025 9 commits
- [Bug]: Fix maximum recursion depth triggered on exception exit (#3519) · 60524920
  Kebe authored Feb 26, 2025
  
  60524920
- [BugFix] Fix crash when receive a req with structed output in DP attention mode. (#3841) · 10771026
  IAN authored Feb 26, 2025
  
  10771026
- Bug: fix capture_bs (#3857) · 4606e2a3
  who who who authored Feb 26, 2025
  
  4606e2a3
- Fix allgather ops inside cuda graphs (#3709) · 127998cc
  Nicolas Castet authored Feb 25, 2025
  
  127998cc
- [improve] made timeout configurable (#3803) · c0bb9eb3
  Shenggui Li authored Feb 25, 2025
  
  c0bb9eb3
- [Bug]: Add missing clamp to llavavid (#3787) · 7036d6fc
  Yueyang Pan authored Feb 25, 2025
  
  7036d6fc
- [ROCm] Enable Fused MLA Triton kernel for DeepSeekV3 (#3237) · 6ce9dbe8
  Chaitanya Sri Krishna Lolla authored Feb 25, 2025
```
Co-authored-by: HAI <hixiao@gmail.com>
```
  6ce9dbe8
- Improve: fix typos (#3801) · 60b771c8
  Wang Ran (汪然) authored Feb 25, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  60b771c8
- Fix CI and install docs (#3821) · d7934cde
  Lianmin Zheng authored Feb 24, 2025
  
  d7934cde
24 Feb, 2025 6 commits
- Revert "Extract generation_manager from tokenizer_manager" (#3829) · 62bbd343
  Lianmin Zheng authored Feb 24, 2025
  
  62bbd343
- Revert "Rename TokenizerManager to StdOrchestrator" (#3828) · f2388f6b
  Lianmin Zheng authored Feb 24, 2025
  
  f2388f6b
- Fix pandas dependency in CI (#3818) · c9745ee0
  Lianmin Zheng authored Feb 24, 2025
  
  c9745ee0
- Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730) · 1a6e9757
  laixin authored Feb 24, 2025
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  1a6e9757
- Refactor flashinfer logic for deepseek v3 and fix accuracy bug (#3785) · b1100846
  Baizhou Zhang authored Feb 24, 2025
  
  b1100846
- Fix dependency (#3813) · 27a46317
  Lianmin Zheng authored Feb 24, 2025
  
  27a46317