Commits · 6ec75e626d5949dfca49069cd778cd4eb29d02b1 · change / sglang · GitLab

13 Jan, 2025 4 commits
- Dump requests (#2862) · 3b141e15
  Lianmin Zheng authored Jan 13, 2025
  
  3b141e15
- Revert "Integration of TurboMind AWQ" (#2866) · 6249e4a1
  Lianmin Zheng authored Jan 13, 2025
  
  6249e4a1
- Integration of TurboMind AWQ (#2828) · 17de02f9
  bjmsong authored Jan 13, 2025
```
Co-authored-by: root <bjmsong@126.com>
```
  17de02f9
- Support FP8 E4M3 KV Cache (#2786) · 0bb0f763
  bjmsong authored Jan 13, 2025
```
Co-authored-by: root <bjmsong@126.com>
```
  0bb0f763
11 Jan, 2025 1 commit
- Improve: Token-In Token-Out Usage for RLHF (#2843) · c4f9707e
  Shi Shuai authored Jan 11, 2025
  
  c4f9707e
10 Jan, 2025 1 commit
- Fix port number overflow (#2826) · b1706469
  TianYu GUO authored Jan 11, 2025
  
  b1706469
07 Jan, 2025 2 commits
- Misc fix for min_p_sampling, --cuda-graph-bs (#2761) · bdc1acf6
  Lianmin Zheng authored Jan 07, 2025
  
  bdc1acf6
- Remove --modelopt-config in server_args (#2758) · 9dec582d
  Lianmin Zheng authored Jan 06, 2025
  
  9dec582d
06 Jan, 2025 1 commit
- Enable Nvidia's ModelOpt fp8 quantized models (#2535) · 287427e2
  Zhiyu authored Jan 06, 2025
  
  287427e2
02 Jan, 2025 1 commit
- Eagle speculative decoding part 3: small modifications to the general scheduler (#2709) · ad20b795
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  ad20b795
31 Dec, 2024 1 commit
- Eagle speculative decoding part 2: Fix cuda graph + DP attention hanging (#2684) · b0524c37
  Lianmin Zheng authored Dec 31, 2024
```
Co-authored-by: yukavio <kavioyu@gmail.com>
```
  b0524c37
30 Dec, 2024 1 commit
- [Docs] clean up structured outputs docs (#2654) · 8c3b420e
  Lianmin Zheng authored Dec 29, 2024
  
  8c3b420e
29 Dec, 2024 2 commits
- [Feature] Get Token IDs with Engine.generate() (#2636) · 35bdb485
  Shi Shuai authored Dec 29, 2024
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  35bdb485
- Clean up wrapper in flashinfer backend (#2638) · 3815b23c
  Lianmin Zheng authored Dec 29, 2024
  
  3815b23c
16 Dec, 2024 1 commit
- Simplify pytorch sampling kernel and logit processor (#2491) · 7a1aecb9
  Lianmin Zheng authored Dec 16, 2024
  
  7a1aecb9
14 Dec, 2024 1 commit
- Remove cuda graph batch size adjustment for dp attention (#2484) · 0ba2c589
  Ke Bao authored Dec 14, 2024
  
  0ba2c589
11 Dec, 2024 1 commit
- Fix model loader for more quantization formats (#2448) · 959735fc
  Lianmin Zheng authored Dec 11, 2024
  
  959735fc
08 Dec, 2024 1 commit
- Optimize Triton decoding kernel for long context (#2394) · 7dc66fcb
  Ke Bao authored Dec 08, 2024
  
  7dc66fcb
06 Dec, 2024 3 commits
- Add more support for intel Gaudi accelerators (#2357) · 37ee906f
  Qun Yang authored Dec 06, 2024
  
  37ee906f
- optimize cuda graph max_bs_settings on low-end gpus (#2360) · 34b364e0
  Xiaoyu Zhang authored Dec 06, 2024
  
  34b364e0
- Resubmit MoE-EP (#2371) · 3d32e4a3
  xiaobochen authored Dec 06, 2024
  
  3d32e4a3
04 Dec, 2024 1 commit
- Check gpu availability at server args creation (#2340) · ed45e509
  Ata Fatahi authored Dec 04, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  ed45e509
03 Dec, 2024 1 commit
- [Minor] Fix logger and style (#2325) · 3ddb1c46
  Lianmin Zheng authored Dec 02, 2024
  
  3ddb1c46
02 Dec, 2024 1 commit
- Update model_loader deps and qqq quantization deps (#2220) (#2318) · 85e1a6f3
  Yineng Zhang authored Dec 02, 2024
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  85e1a6f3
01 Dec, 2024 3 commits
- Add a simple torch native attention backend (#2241) · 62c516ac
  Qun Yang authored Dec 01, 2024
  
  62c516ac
- Revert "Revert "[FEAT] Support GGUF format"" (#2287) · 4936be8a
  Lianmin Zheng authored Nov 30, 2024
  
  4936be8a
- Revert "[FEAT] Support GGUF format" (#2285) · 7e4c6dd8
  Lianmin Zheng authored Nov 30, 2024
  
  7e4c6dd8
30 Nov, 2024 2 commits
- [FEAT] Support GGUF format (#2215) · 883c9554
  Yang Zheng authored Nov 30, 2024
```
Co-authored-by: Yang Zheng(SW)(Alex) <you@example.com>
```
  883c9554
- Fix the default chunked prefill size (#2268) · 94e167ea
  Lianmin Zheng authored Nov 29, 2024
  
  94e167ea
29 Nov, 2024 3 commits
- [benchmark] Add fused_moe_triton benchmark and tuning tools (#2225) · 262e370f
  Xiaoyu Zhang authored Nov 30, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
```
  262e370f
- Revert "Revert "Add simple CPU offloading support"" (#2253) · 8b48496a
  Ying Sheng authored Nov 28, 2024
```
Co-authored-by: Jani Monoses <jani.monoses@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  8b48496a
- Revert "Add simple CPU offloading support" (#2252) · 4057ea82
  Ying Sheng authored Nov 28, 2024
```
We'll re-add the commit to correctly ack Kaichao's authorship
```
  4057ea82
28 Nov, 2024 1 commit
- Fix chunked prefill size for bench_offline_throughput (#2234) · 09798b36
  Lianmin Zheng authored Nov 27, 2024
  
  09798b36
24 Nov, 2024 1 commit
- Fix mixed chunked prefill in overlap mode (#2158) · 731146f6
  Lianmin Zheng authored Nov 24, 2024
  
  731146f6
23 Nov, 2024 2 commits
- Add simple CPU offloading support. (#2081) · d98fa1e9
  Jani Monoses authored Nov 23, 2024
  
  d98fa1e9
- Add initial support for intel Gaudi accelerators (#2121) · 865233e2
  Ankur Neog authored Nov 23, 2024
  
  865233e2
22 Nov, 2024 2 commits
- docs: fix module docstrings and copyright headers (#2077) · 62a4a339
  Xuehai Pan authored Nov 22, 2024
  
  62a4a339
- [router] add base_gpu_id server args & merged radix tree python reference (#2115) · 30af7dfb
  Byron Hsu authored Nov 21, 2024
  
  30af7dfb
21 Nov, 2024 1 commit
- Error out when torchao-config option is not recognized (#2107) · 5c6a41fa
  Jerry Zhang authored Nov 20, 2024
  
  5c6a41fa
20 Nov, 2024 1 commit
- Enable overlap scheduler by default for the triton attention backend (#2105) · 722530fa
  Lianmin Zheng authored Nov 20, 2024
  
  722530fa