Commits · c7f254468fcae6a2df91765f9229aeeaf1d53613 · change / sglang

07 Mar, 2025 2 commits
- [Feature] DeepSeek V3/R1 INT8 Quantization (channel-wise) (#3888) · c7f25446
  HandH1998 authored Mar 07, 2025
```
Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: b0urnee <2769086541@qq.com>
```
  c7f25446
- Add Support for Qwen2-VL Multi-modal Embedding Models (#3694) · 361971b8
  uylnap authored Mar 07, 2025
  
  361971b8
05 Mar, 2025 1 commit
- [QUANT] Add GPTQModel Dynamic Quantization + `lm_head` Quantization (#3790) · 56a724eb
  Qubitium-ModelCloud authored Mar 05, 2025
```
Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>
Co-authored-by: ZX-ModelCloud <zx@modelcloud.ai>
```
  56a724eb
04 Mar, 2025 1 commit
- Reasoning parser (#4000) · 95575aa7
  Xihuai Wang authored Mar 04, 2025
```
Co-authored-by: Lucas Pickup <lupickup@microsoft.com>
```
  95575aa7
03 Mar, 2025 1 commit

Support penalty in overlap mode; return logprob with chunked prefill; improve... · ac238727

Lianmin Zheng authored Mar 03, 2025


Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

ac238727

01 Mar, 2025 1 commit

[Feature]Support ragged prefill in flashinfer mla backend (#3967) · 90a4b7d9

Baizhou Zhang authored Feb 28, 2025


Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>

90a4b7d9

28 Feb, 2025 1 commit
- [feat] Add Vertex AI compatible prediction route for /generate (#3866) · bc20e93f
  KCFindstr authored Feb 27, 2025
  
  bc20e93f
24 Feb, 2025 1 commit
- Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730) · 1a6e9757
  laixin authored Feb 24, 2025
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  1a6e9757
20 Feb, 2025 1 commit
- [Fix] Fix bugs and refactor codes in lora for better scalability. (#3652) · e79f7420
  aoshen524 authored Feb 20, 2025
```
Co-authored-by: ShenAo1111 <1377693092@qq.com>
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  e79f7420
17 Feb, 2025 1 commit
- update unit test (#3636) · e319153b
  Yineng Zhang authored Feb 17, 2025
  
  e319153b
10 Feb, 2025 1 commit
- [Feat] Return hidden states (experimental) (#3364) · 5f0e7de3
  Jackmin801 authored Feb 10, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  5f0e7de3
04 Feb, 2025 2 commits
- upgrade flashinfer v0.2.0.post2 (#3288) · d39899e8
  Yineng Zhang authored Feb 04, 2025
```
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>
```
  d39899e8
- [Feature] Define backends and add Triton backend for Lora (#3161) · 70817a7e
  Baizhou Zhang authored Feb 03, 2025
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  70817a7e
28 Jan, 2025 2 commits
- [Fix] Address remaining issues of supporting MiniCPMV (#2977) · 9f635ea5
  Mick authored Jan 28, 2025
  
  9f635ea5
- [test] deduplicate test_session_control (#3183) · 27aeb4b7
  Byron Hsu authored Jan 27, 2025
  
  27aeb4b7
27 Jan, 2025 1 commit
- add unit test for block wise fp8 (#3156) · 1e3e5215
  yizhang2077 authored Jan 27, 2025
  
  1e3e5215
26 Jan, 2025 1 commit
- Add a test case for cached_tokens (#3145) · d1a08632
  Lianmin Zheng authored Jan 26, 2025
  
  d1a08632
20 Jan, 2025 1 commit
- Improve metrics, logging, and importing orders (#2992) · cd493b5a
  Lianmin Zheng authored Jan 19, 2025
  
  cd493b5a
19 Jan, 2025 1 commit
- support regex in xgrammar backend (#2983) · 3bcf5ece
  Enrique Shockwave authored Jan 19, 2025
  
  3bcf5ece
18 Jan, 2025 1 commit
- support e4m3 kvcache in qwen2 & add kv scaling facotr json (#2894) · d3024f4f
  bjmsong authored Jan 18, 2025
```
Co-authored-by: bjmsong <bjmsong@126.com>
```
  d3024f4f
17 Jan, 2025 1 commit
- Add ut for qwen model (#2947) · d47c5101
  Ke Bao authored Jan 18, 2025
  
  d47c5101
16 Jan, 2025 1 commit
- [Frontend] Fix request length check and add option to disallow auto truncation in scheduler (#2876) · a8ccacc8
  Chang Su authored Jan 16, 2025
  
  a8ccacc8
15 Jan, 2025 2 commits
- adapt custom allreduce for tensorrt llm (#2511) · 767c9dec
  yizhang2077 authored Jan 16, 2025
  
  767c9dec
- Add ut for w8a8 int8 quantization (#2897) · bfbda62c
  Ke Bao authored Jan 15, 2025
  
  bfbda62c
13 Jan, 2025 2 commits
- CUDA-graph-compatible releasing and resuming KV cache and model weight memory (#2630) · 923f5183
  fzyzcjy authored Jan 14, 2025
  
  923f5183
- Use only one GPU for MLA CI tests (#2858) · 67008f4b
  Lianmin Zheng authored Jan 13, 2025
  
  67008f4b
11 Jan, 2025 1 commit
- Improve: Token-In Token-Out Usage for RLHF (#2843) · c4f9707e
  Shi Shuai authored Jan 11, 2025
  
  c4f9707e
08 Jan, 2025 2 commits
- Fix nightly accuracy tests (#2780) · b22f3f64
  Lianmin Zheng authored Jan 07, 2025
  
  b22f3f64
- Disable math eval on nightly CI temporarily (#2779) · 6fb57683
  Lianmin Zheng authored Jan 07, 2025
  
  6fb57683
02 Jan, 2025 1 commit

Eagle speculative decoding part 4: Add EAGLE2 worker (#2150) · 815dce05

yukavio authored Jan 02, 2025


Co-authored-by: kavioyu <kavioyu@tencent.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>

815dce05

01 Jan, 2025 1 commit
- [feat]: Add math eval to CI nightly run (#2663) · a4d6d6f1
  Xiaotong Jiang authored Jan 01, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  a4d6d6f1
29 Dec, 2024 1 commit
- [Feature] Get Token IDs with Engine.generate() (#2636) · 35bdb485
  Shi Shuai authored Dec 29, 2024
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  35bdb485
28 Dec, 2024 2 commits
- Add `update_weights_from_tensor` (#2631) · fd28640d
  fzyzcjy authored Dec 29, 2024
  
  fd28640d
- [CI] Fix nightly test and raise better error message (#2626) · 855d0ba3
  Lianmin Zheng authored Dec 27, 2024
```
Co-authored-by: Sangbin <rkooo567@gmail.com>
```
  855d0ba3
26 Dec, 2024 1 commit
- Fix test and benchmark scripts (#2598) · dc3bee48
  Lianmin Zheng authored Dec 26, 2024
  
  dc3bee48
16 Dec, 2024 1 commit
- Temporarily disable unit test of torch native attention backend (#2492) · 9cd9dc83
  Lianmin Zheng authored Dec 16, 2024
  
  9cd9dc83
09 Dec, 2024 3 commits
- [feat] Enable chunked prefill for llava-onevision (#2412) · 8586b72d
  Ying Sheng authored Dec 09, 2024
  
  8586b72d
- [Minor] Improve code style (#2422) · 641b7d0a
  Lianmin Zheng authored Dec 09, 2024
  
  641b7d0a
- Add a unittest for fused_moe (#2416) · 3844feb9
  Xiaoyu Zhang authored Dec 09, 2024
  
  3844feb9
03 Dec, 2024 1 commit
- Revert "[feat] Enable chunked prefill for llava-onevision" (#2329) · aa47f642
  Ying Sheng authored Dec 02, 2024
  
  aa47f642