Commits · eec3f6d1eb65fbdb6490bad2d867acc96b676cda · zhaoyu6 / sglang

"docs/source/vscode:/vscode.git/clone" did not exist on "dbc16f4404eca4a75459683d5135f6accea35a02"

28 Feb, 2025 4 commits
- [Bugfix] Fix tokenizer_manager not getting 400 when req is too long (#3678) · eec3f6d1
  Chang Su authored Feb 27, 2025
```
Co-authored-by: voidxb <unkown>
```
  eec3f6d1
- set a strict sgl-kernel version (#3950) · 90bc26a8
  Chayenne authored Feb 27, 2025
  
  90bc26a8
- Fix bench_serving not recognizing OPENAI_API_KEY (#3870) · ec0a72c2
  Kebe authored Feb 28, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  ec0a72c2
- [feat] Add Vertex AI compatible prediction route for /generate (#3866) · bc20e93f
  KCFindstr authored Feb 27, 2025
  
  bc20e93f
27 Feb, 2025 6 commits
- Fix the doc link for sampling params (#3861) · d3887852
  Qiaolin Yu authored Feb 27, 2025
  
  d3887852
- upgrade flashinfer v0.2.2.post1 (#3934) · 564bdf29
  Yineng Zhang authored Feb 27, 2025
  
  564bdf29
- Improve: Support xgrammar 0.1.14 (#3593) · d2815879
  Enrique Shockwave authored Feb 27, 2025
  
  d2815879
- Tuning Script for Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3922) · b0df5d24
  laixin authored Feb 27, 2025
```
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
```
  b0df5d24
- Add return hidden state in the native API (#3897) · d6898dd2
  Qiaolin Yu authored Feb 27, 2025
```
Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  d6898dd2
- [Rocm] Fix to the rocm_mla_decode_rope.py returning random result (#3898) · 8b681d77
  Tianxing Wu authored Feb 27, 2025
  
  8b681d77
26 Feb, 2025 2 commits
- [Feature] Support llguidance for constrained decoding (#3298) · 7551498a
  JC1DA authored Feb 26, 2025
  
  7551498a
- Expert Parallelism (EP) Support for DeepSeek V3/R1 (#3602) · 21463e32
  lukec authored Feb 26, 2025
```
Co-authored-by: laixin <xielx@shanghaitech.edu.cn>
Co-authored-by: HandH1998 <1335248067@qq.com>
Co-authored-by: laixin <q865809639@gmail.com>
```
  21463e32
25 Feb, 2025 9 commits
- [Bug]: Fix maximum recursion depth triggered on exception exit (#3519) · 60524920
  Kebe authored Feb 26, 2025
  
  60524920
- [BugFix] Fix crash when receive a req with structed output in DP attention mode. (#3841) · 10771026
  IAN authored Feb 26, 2025
  
  10771026
- Bug: fix capture_bs (#3857) · 4606e2a3
  who who who authored Feb 26, 2025
  
  4606e2a3
- Fix allgather ops inside cuda graphs (#3709) · 127998cc
  Nicolas Castet authored Feb 25, 2025
  
  127998cc
- [improve] made timeout configurable (#3803) · c0bb9eb3
  Shenggui Li authored Feb 25, 2025
  
  c0bb9eb3
- [Bug]: Add missing clamp to llavavid (#3787) · 7036d6fc
  Yueyang Pan authored Feb 25, 2025
  
  7036d6fc
- [ROCm] Enable Fused MLA Triton kernel for DeepSeekV3 (#3237) · 6ce9dbe8
  Chaitanya Sri Krishna Lolla authored Feb 25, 2025
```
Co-authored-by: HAI <hixiao@gmail.com>
```
  6ce9dbe8
- Improve: fix typos (#3801) · 60b771c8
  Wang Ran (汪然) authored Feb 25, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  60b771c8
- Fix CI and install docs (#3821) · d7934cde
  Lianmin Zheng authored Feb 24, 2025
  
  d7934cde
24 Feb, 2025 7 commits
- Revert "Extract generation_manager from tokenizer_manager" (#3829) · 62bbd343
  Lianmin Zheng authored Feb 24, 2025
  
  62bbd343
- Revert "Rename TokenizerManager to StdOrchestrator" (#3828) · f2388f6b
  Lianmin Zheng authored Feb 24, 2025
  
  f2388f6b
- Fix pandas dependency in CI (#3818) · c9745ee0
  Lianmin Zheng authored Feb 24, 2025
  
  c9745ee0
- Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730) · 1a6e9757
  laixin authored Feb 24, 2025
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  1a6e9757
- Refactor flashinfer logic for deepseek v3 and fix accuracy bug (#3785) · b1100846
  Baizhou Zhang authored Feb 24, 2025
  
  b1100846
- Fix dependency (#3813) · 27a46317
  Lianmin Zheng authored Feb 24, 2025
  
  27a46317
- Hierarchical Caching for SGLang (#2693) · 6c7a152c
  Zhiqiang Xie authored Feb 23, 2025
```
Co-authored-by: Wenxuan Tan <wenxuan.tan@wisc.edu>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  6c7a152c
23 Feb, 2025 2 commits
- Improve: Rename TokenizerManager to StdOrchestrator (#3116) · 45360b2f
  fzyzcjy authored Feb 23, 2025
  
  45360b2f
- Improve: Extract generation_manager from tokenizer_manager (#3115) · 3f41b184
  fzyzcjy authored Feb 23, 2025
  
  3f41b184
22 Feb, 2025 5 commits
- bench: Add MMMU benchmark for vLM (#3562) · 45205d88
  Mick authored Feb 23, 2025
  
  45205d88
- Improve: Use TypeBasedDispatcher in DetokenizerManager (#3117) · 90876940
  fzyzcjy authored Feb 22, 2025
  
  90876940
- Bug: Fix weight loader error when LM head weights are tied (#3766) · a3339d8c
  fzyzcjy authored Feb 22, 2025
  
  a3339d8c
- Bug: fix lm head weights in Qwen models (#3777) · 14d90617
  Chayenne authored Feb 21, 2025
  
  14d90617
- Improve: Tiny fix Olmo2 (#3348) · d37f9551
  fzyzcjy authored Feb 22, 2025
  
  d37f9551
21 Feb, 2025 5 commits
- Add support for nvidia modelopt fp8 kv cache (#3223) · c66b2c9c
  Zhiyu authored Feb 21, 2025
  
  c66b2c9c
- Model: Support Qwen 72B RM model. (#3772) · 20b765a2
  simveit authored Feb 21, 2025
  
  20b765a2
- [bug] fixed batch api for DeepSeek V3/R1 (#3754) · 9af0e21e
  Shenggui Li authored Feb 22, 2025
  
  9af0e21e
- [Fix] OpenAI API adapter tokenizer encoding (#3432) · c7c79b16
  Shi Shuai authored Feb 21, 2025
  
  c7c79b16
- feat: Add SageMaker support (#3740) · 1df6eabd
  Andrew Smith authored Feb 21, 2025
  
  1df6eabd