Commits · 8b6a4486ecbf83c915c6a9d3c727d188a22f455e · change / sglang · GitLab

19 Jan, 2025 3 commits
- fix missing revision arg when loading tokenizer (#2982) · 8b6a4486
  giorgiopiatti-dfinity authored Jan 19, 2025
  
  8b6a4486
- Refactor to add TypeBasedDispatcher to simplify dispatching (#2958) · 81d27c8e
  fzyzcjy authored Jan 19, 2025
  
  81d27c8e
- Frontend: better error message handling for FINISH_ABORT in scheduler.py (#2956) · 4d4cdb3f
  Chang Su authored Jan 18, 2025
  
  4d4cdb3f
18 Jan, 2025 2 commits
- [Feature] Support minicpmv v2.6 (#2785) · 3d93f84a
  Mick authored Jan 19, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  3d93f84a
- Query remaining memory dynamically for PrefillAdder (#2941) · 8af7048d
  Zhiqiang Xie authored Jan 17, 2025
  
  8af7048d
17 Jan, 2025 1 commit
- Enable CPU device on SGLang (#2806) · 63051738
  Chunyuan WU authored Jan 17, 2025
  
  63051738
16 Jan, 2025 5 commits
- [Frontend] Fix request length check and add option to disallow auto truncation in scheduler (#2876) · a8ccacc8
  Chang Su authored Jan 16, 2025
  
  a8ccacc8
- Fix zmq binding (#2930) · 0427416b
  Lianmin Zheng authored Jan 16, 2025
```
Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
```
  0427416b
- Improve type annotation and styles (#2926) · bc6915e3
  Lianmin Zheng authored Jan 16, 2025
  
  bc6915e3
- Support multi-node DP attention (#2925) · 8b6ce52e
  Lianmin Zheng authored Jan 16, 2025
```
Co-authored-by: dhou-xai <dhou@x.ai>
```
  8b6ce52e
- Improve benchmark scripts and error message printing (#2922) · 8f2c522a
  Lianmin Zheng authored Jan 16, 2025
  
  8f2c522a
15 Jan, 2025 1 commit
- Remove normalized_prompt_logprobs from the engine to make code easier to maintain (#2902) · f65c13b5
  Lianmin Zheng authored Jan 15, 2025
  
  f65c13b5
13 Jan, 2025 3 commits
- Add a new api configure_logging to allow dumping the requests (#2875) · 46d44318
  Lianmin Zheng authored Jan 13, 2025
  
  46d44318
- CUDA-graph-compatible releasing and resuming KV cache and model weight memory (#2630) · 923f5183
  fzyzcjy authored Jan 14, 2025
  
  923f5183
- Fix linear.py and improve weight loading (#2851) · 72c77763
  Lianmin Zheng authored Jan 13, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  72c77763
11 Jan, 2025 1 commit
- Improve: Token-In Token-Out Usage for RLHF (#2843) · c4f9707e
  Shi Shuai authored Jan 11, 2025
  
  c4f9707e
07 Jan, 2025 1 commit
- Misc fix for min_p_sampling, --cuda-graph-bs (#2761) · bdc1acf6
  Lianmin Zheng authored Jan 07, 2025
  
  bdc1acf6
06 Jan, 2025 1 commit
- Clean up eagle code (#2756) · b8574f69
  Lianmin Zheng authored Jan 06, 2025
  
  b8574f69
03 Jan, 2025 1 commit
- Fix package loss for small models (#2717) · 0f9cc6d8
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: sdli1995 < mmlmonkey@163.com>
```
  0f9cc6d8
02 Jan, 2025 1 commit
- Eagle speculative decoding part 3: small modifications to the general scheduler (#2709) · ad20b795
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  ad20b795
30 Dec, 2024 1 commit
- Refactor logprob computation to return the real logprob used in sampling (#2664) · 9c6ba248
  Lianmin Zheng authored Dec 30, 2024
  
  9c6ba248
29 Dec, 2024 3 commits
- [Feature] Get Token IDs with Engine.generate() (#2636) · 35bdb485
  Shi Shuai authored Dec 29, 2024
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  35bdb485
- [Session] Update session control interface (#2635) · e0e09fce
  Ying Sheng authored Dec 29, 2024
  
  e0e09fce
- Clean up wrapper in flashinfer backend (#2638) · 3815b23c
  Lianmin Zheng authored Dec 29, 2024
  
  3815b23c
28 Dec, 2024 1 commit
- Add `update_weights_from_tensor` (#2631) · fd28640d
  fzyzcjy authored Dec 29, 2024
  
  fd28640d
27 Dec, 2024 2 commits
- [minor] clean up docs and eos id (#2622) · 751e5ca2
  Lianmin Zheng authored Dec 27, 2024
  
  751e5ca2
- [FIX] Update EOS from config (#2475) · 7a7ac6be
  Yang Zheng authored Dec 28, 2024
  
  7a7ac6be
26 Dec, 2024 4 commits
- Fix duplicated handling of GetWeightsByNameReqInput (#2565) · 3169e66c
  fzyzcjy authored Dec 26, 2024
  
  3169e66c
- Fix logprob_start_len for multi modal models (#2597) · 77395154
  Lianmin Zheng authored Dec 26, 2024
```
Co-authored-by: libra <lihu723@gmail.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: Wang, Haoyu <haoyu.wang@intel.com>
```
  77395154
- [Feature] Support new parameter - EBNF in xgrammar (#2526) · acb34072
  Adarsh Shirawalmath authored Dec 26, 2024
  
  acb34072
- Fix cache hit rate when chunked prefill (#2555) · e7ebecf8
  Liangsheng Yin authored Dec 26, 2024
  
  e7ebecf8
22 Dec, 2024 1 commit
- [Misc] Fix metrics, weight update lock, request logging (#2543) · 84967019
  Lianmin Zheng authored Dec 22, 2024
  
  84967019
11 Dec, 2024 1 commit
- [Core] in batch prefix caching by delay scheduling (#2442) · 9208618b
  SangBin Cho authored Dec 11, 2024
  
  9208618b
09 Dec, 2024 1 commit
- [Minor] Improve code style (#2419) · 0ce091a8
  Lianmin Zheng authored Dec 09, 2024
  
  0ce091a8
08 Dec, 2024 4 commits
- Simplify stream_output (#2398) · a6ca736c
  Lianmin Zheng authored Dec 08, 2024
  
  a6ca736c
- Fix recv_requests (#2405) · cc858953
  Lianmin Zheng authored Dec 08, 2024
  
  cc858953
- Fix a bug with logprob streaming + chunked prefill (#2403) · a2486eb5
  Lianmin Zheng authored Dec 08, 2024
  
  a2486eb5
- nit: Remove busy waiting on scheduler (#2382) · 1f09e84b
  SangBin Cho authored Dec 08, 2024
  
  1f09e84b
06 Dec, 2024 2 commits
- Fix the overlap for xgrammar (#2377) · 0e7409ad
  Lianmin Zheng authored Dec 06, 2024
  
  0e7409ad
- Add more support for intel Gaudi accelerators (#2357) · 37ee906f
  Qun Yang authored Dec 06, 2024
  
  37ee906f