Commits · a7591ecf6967aebc13ad53d8927ed7465970ab12 · change / sglang

"vscode:/vscode.git/clone" did not exist on "25caa7a8a98e1bf73335973b29adbadb108f0ac7"

21 Apr, 2025 24 commits
- Update protocol.py to fix #4589 (#4590) · a7591ecf
  moontidef authored Apr 21, 2025
  
  a7591ecf
- Modify metrics service endpoint (#3443) · c44f2869
  lambert0312 authored Apr 21, 2025
  
  c44f2869
- Tiny add warning when cannot recognize bool env var (#5348) · 685d8980
  fzyzcjy authored Apr 21, 2025
  
  685d8980
- upstream hicache fixes (#5570) · 70645f4d
  Zhiqiang Xie authored Apr 20, 2025
  
  70645f4d
- Add Speculative Decoding Eagle3 topk > 1 (#5318) · 188f0955
  Qingquan Song authored Apr 20, 2025
```
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
```
  188f0955
- Fix flush cache (#5590) · eef9433b
  Lianmin Zheng authored Apr 20, 2025
  
  eef9433b
- [misc] remove is_cuda_available (#5319) · 97cb762b
  JieXin Liang authored Apr 21, 2025
  
  97cb762b
- Tiny add Engine.flush_cache API (#5241) · 11951820
  fzyzcjy authored Apr 21, 2025
  
  11951820
- Speedup shared expert weight construction by avoid cloning (#5188) · 5239d795
  fzyzcjy authored Apr 21, 2025
  
  5239d795
- Perform Batch Tokenization. (#5141) · f0815419
  Sundara Raman Ramachandran authored Apr 20, 2025
  
  f0815419
- Correct grafana heatmap. (#5019) · 2b3bdc93
  mac0ne authored Apr 21, 2025
  
  2b3bdc93
- Add sanity check for max_running_requests (#5016) · 5fc4b600
  fzyzcjy authored Apr 21, 2025
  
  5fc4b600
- Fix one more issue reported by torchfix (#4859) · b868526d
  Brayden Zhong authored Apr 20, 2025
  
  b868526d
- compressed_tensors: port w8a16 fp8 from vllm (#4852) · 502524e2
  Juwan Yoo authored Apr 21, 2025
  
  502524e2
- check marlin format before attempting conversion (#4675) · 4c764007
  Enrique Shockwave authored Apr 21, 2025
  
  4c764007
- Feat: Implement JSON Mode (response_format.type="json_object") (#4733) · 9f3bd2ad
  kyle-pena-kuzco authored Apr 20, 2025
```
Co-authored-by: Kyle Pena <kylepena@kyles-macbook-pro.turkey-marlin.ts.net>
```
  9f3bd2ad
- smaller and non gated models for docs (#5378) · 8de53da9
  simveit authored Apr 21, 2025
  
  8de53da9
- add function call parser for DeepSeek V3 (#5224) · fac17acf
  Yi Zhou authored Apr 21, 2025
  
  fac17acf
- [Feature] Prefill assistant response - add continue_final_message parameter (#4226) · 8b39274e
  Adarsh Shirawalmath authored Apr 21, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  8b39274e
- Add test config yamls for Deepseek v3 (#5433) · 5156d5a4
  Baizhou Zhang authored Apr 20, 2025
  
  5156d5a4
- [PD] Fix large page size + chunk prefill (#5588) · c951d312
  Byron Hsu authored Apr 20, 2025
  
  c951d312
- Fix ChatCompletionMessageGenericParam to allow for None content (#5452) · dcb82325
  AmadeusW authored Apr 21, 2025
  
  dcb82325
- fix: use fa3 for gemma2 (#5586) · 66c0ff9e
  Yineng Zhang authored Apr 20, 2025
  
  66c0ff9e
- Fix enable chunked prefill for Llama4 (#5575) · 9a7e83e8
  tarinkk authored Apr 20, 2025
  
  9a7e83e8
20 Apr, 2025 16 commits
- [Feat] upgrade pytorch2.6 (#5417) · 417b44eb
  lukec authored Apr 21, 2025
  
  417b44eb
- [PD] Fix server crash when using batch requests (#5531) · 475e2e37
  fzyzcjy authored Apr 21, 2025
  
  475e2e37
- Tiny improve error message (#5526) · fba86b6b
  fzyzcjy authored Apr 21, 2025
  
  fba86b6b
- Add document for LoRA serving (#5521) · 072b4d03
  Baizhou Zhang authored Apr 20, 2025
  
  072b4d03
- Super tiny fix typo (#5559) · 9c434777
  fzyzcjy authored Apr 21, 2025
  
  9c434777
- Fix torch memory saver not enabled in DP scenario (#5560) · fa2f677e
  fzyzcjy authored Apr 21, 2025
  
  fa2f677e
- Fix DeepEP cannot run on latest master (#5567) · 463d4b74
  fzyzcjy authored Apr 21, 2025
  
  463d4b74
- Fix bench_serving fail when zero warmup requests (#5574) · 9924bbe1
  fzyzcjy authored Apr 21, 2025
  
  9924bbe1
- Release v0.4.5.post2 (#5582) · fbdc94ba
  Lianmin Zheng authored Apr 20, 2025
  
  fbdc94ba
- [Doc]Add instruction for profiling with bench_one_batch (#5581) · b54b5a96
  Baizhou Zhang authored Apr 20, 2025
  
  b54b5a96
- [Fix] fix outlines and xgrammar (#4947) · bca832c7
  JieXin Liang authored Apr 21, 2025
  
  bca832c7
- enable DeepSeek V3 shared_experts_fusion in sm90 (#5571) · d9dd5298
  Xiaoyu Zhang authored Apr 21, 2025
  
  d9dd5298
- Fix BumpAllocator error when no input_ids (#5564) · 0a0dd34e
  fzyzcjy authored Apr 20, 2025
  
  0a0dd34e
- [PD] Fix DeepSeek cannot be run on latest master (#5568) · 80ac527d
  fzyzcjy authored Apr 20, 2025
  
  80ac527d
- [perf] introduce deep gemm group_gemm_masked as bmm (#5432) · 99456bca
  JieXin Liang authored Apr 20, 2025
  
  99456bca
- Fix bench_one_batch producing unnatural results for expert parallel (#5149) · d07e797a
  fzyzcjy authored Apr 20, 2025
  
  d07e797a