Commits · d6898dd2534a3faa801a7166062722288cc5c92a · zhaoyu6 / sglang · GitLab

27 Feb, 2025 1 commit

Add return hidden state in the native API (#3897) · d6898dd2

Qiaolin Yu authored Feb 27, 2025


Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>

d6898dd2

26 Feb, 2025 1 commit
- [Feature] Support llguidance for constrained decoding (#3298) · 7551498a
  JC1DA authored Feb 26, 2025
  
  7551498a
25 Feb, 2025 1 commit
- [improve] made timeout configurable (#3803) · c0bb9eb3
  Shenggui Li authored Feb 25, 2025
  
  c0bb9eb3
24 Feb, 2025 3 commits
- Revert "Rename TokenizerManager to StdOrchestrator" (#3828) · f2388f6b
  Lianmin Zheng authored Feb 24, 2025
  
  f2388f6b
- Fix pandas dependency in CI (#3818) · c9745ee0
  Lianmin Zheng authored Feb 24, 2025
  
  c9745ee0
- Fix dependency (#3813) · 27a46317
  Lianmin Zheng authored Feb 24, 2025
  
  27a46317
23 Feb, 2025 1 commit
- Improve: Rename TokenizerManager to StdOrchestrator (#3116) · 45360b2f
  fzyzcjy authored Feb 23, 2025
  
  45360b2f
14 Feb, 2025 2 commits
- Support NextN (MTP) speculative decoding for DeepSeek-V3/R1 (#3582) · 862dd76c
  Ke Bao authored Feb 15, 2025
  
  862dd76c
- feat: support flashinfer mla attention for deepseek v3 (#3550) · 70f894b8
  Yineng Zhang authored Feb 14, 2025
  
  70f894b8
11 Feb, 2025 1 commit
- Make NCCL NVLS configurable (#3502) · b8318aec
  Ata Fatahi authored Feb 11, 2025
  
  b8318aec
10 Feb, 2025 2 commits
- [Feat] Return hidden states (experimental) (#3364) · 5f0e7de3
  Jackmin801 authored Feb 10, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  5f0e7de3
- refine some typo (#3473) · 2f47d710
  Xiaoyu Zhang authored Feb 10, 2025
  
  2f47d710
04 Feb, 2025 1 commit
- [Feature] Define backends and add Triton backend for Lora (#3161) · 70817a7e
  Baizhou Zhang authored Feb 03, 2025
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  70817a7e
02 Feb, 2025 1 commit
- Tune paged attention parameters for AMD GPU. (#3255) · d9eb9358
  Wen-Heng (Jack) Chung authored Feb 01, 2025
  
  d9eb9358
27 Jan, 2025 2 commits
- Sanity check to prevent performance regression (#3171) · 08104b56
  Zhiqiang Xie authored Jan 27, 2025
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  08104b56
- Improve weight loading and code style (#3174) · 53cef815
  Lianmin Zheng authored Jan 27, 2025
  
  53cef815
26 Jan, 2025 1 commit

Feature/function calling update (#2700) · b045841b

YAMY authored Jan 26, 2025


Co-authored-by: Mingyuan Ma <mamingyuan2001@berkeley.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: shuaills <shishuaiuoe@gmail.com>

b045841b

23 Jan, 2025 1 commit
- Support loading of larger models with on-the-fly quantization (#3061) · 862bcff8
  Ke Wen authored Jan 22, 2025
  
  862bcff8
20 Jan, 2025 1 commit
- Sync distributed package from vllm 0.6.4.post1 (#3010) · 73401fd0
  Lianmin Zheng authored Jan 20, 2025
  
  73401fd0
19 Jan, 2025 1 commit
- [Feature] Add sampler custom logits processor (#2396) · e403d237
  Hongpeng Guo authored Jan 19, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  e403d237
17 Jan, 2025 1 commit
- Enable CPU device on SGLang (#2806) · 63051738
  Chunyuan WU authored Jan 17, 2025
  
  63051738
16 Jan, 2025 3 commits
- [Frontend] Fix request length check and add option to disallow auto truncation in scheduler (#2876) · a8ccacc8
  Chang Su authored Jan 16, 2025
  
  a8ccacc8
- Improve type annotation and styles (#2926) · bc6915e3
  Lianmin Zheng authored Jan 16, 2025
  
  bc6915e3
- Support multi-node DP attention (#2925) · 8b6ce52e
  Lianmin Zheng authored Jan 16, 2025
```
Co-authored-by: dhou-xai <dhou@x.ai>
```
  8b6ce52e
14 Jan, 2025 1 commit
- Support w8a8 int8 quantization config (#2881) · cc0485be
  Ke Bao authored Jan 14, 2025
  
  cc0485be
13 Jan, 2025 7 commits
- Add a new api configure_logging to allow dumping the requests (#2875) · 46d44318
  Lianmin Zheng authored Jan 13, 2025
  
  46d44318
- CUDA-graph-compatible releasing and resuming KV cache and model weight memory (#2630) · 923f5183
  fzyzcjy authored Jan 14, 2025
  
  923f5183
- Revert "Dump requests to a folder" (#2869) · c1e097ca
  Lianmin Zheng authored Jan 13, 2025
  
  c1e097ca
- Dump requests (#2862) · 3b141e15
  Lianmin Zheng authored Jan 13, 2025
  
  3b141e15
- Revert "Integration of TurboMind AWQ" (#2866) · 6249e4a1
  Lianmin Zheng authored Jan 13, 2025
  
  6249e4a1
- Integration of TurboMind AWQ (#2828) · 17de02f9
  bjmsong authored Jan 13, 2025
```
Co-authored-by: root <bjmsong@126.com>
```
  17de02f9
- Support FP8 E4M3 KV Cache (#2786) · 0bb0f763
  bjmsong authored Jan 13, 2025
```
Co-authored-by: root <bjmsong@126.com>
```
  0bb0f763
11 Jan, 2025 1 commit
- Improve: Token-In Token-Out Usage for RLHF (#2843) · c4f9707e
  Shi Shuai authored Jan 11, 2025
  
  c4f9707e
10 Jan, 2025 1 commit
- Fix port number overflow (#2826) · b1706469
  TianYu GUO authored Jan 11, 2025
  
  b1706469
07 Jan, 2025 2 commits
- Misc fix for min_p_sampling, --cuda-graph-bs (#2761) · bdc1acf6
  Lianmin Zheng authored Jan 07, 2025
  
  bdc1acf6
- Remove --modelopt-config in server_args (#2758) · 9dec582d
  Lianmin Zheng authored Jan 06, 2025
  
  9dec582d
06 Jan, 2025 1 commit
- Enable Nvidia's ModelOpt fp8 quantized models (#2535) · 287427e2
  Zhiyu authored Jan 06, 2025
  
  287427e2
02 Jan, 2025 1 commit
- Eagle speculative decoding part 3: small modifications to the general scheduler (#2709) · ad20b795
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  ad20b795
31 Dec, 2024 1 commit
- Eagle speculative decoding part 2: Fix cuda graph + DP attention hanging (#2684) · b0524c37
  Lianmin Zheng authored Dec 31, 2024
```
Co-authored-by: yukavio <kavioyu@gmail.com>
```
  b0524c37
30 Dec, 2024 1 commit
- [Docs] clean up structured outputs docs (#2654) · 8c3b420e
  Lianmin Zheng authored Dec 29, 2024
  
  8c3b420e