Commits · 97cb762bb65ebf05025eb342de03c184660427a3 · change / sglang · GitLab

21 Apr, 2025 18 commits
- [misc] remove is_cuda_available (#5319) · 97cb762b
  JieXin Liang authored Apr 21, 2025
  
  97cb762b
- Tiny add Engine.flush_cache API (#5241) · 11951820
  fzyzcjy authored Apr 21, 2025
  
  11951820
- Speedup shared expert weight construction by avoid cloning (#5188) · 5239d795
  fzyzcjy authored Apr 21, 2025
  
  5239d795
- Perform Batch Tokenization. (#5141) · f0815419
  Sundara Raman Ramachandran authored Apr 20, 2025
  
  f0815419
- Correct grafana heatmap. (#5019) · 2b3bdc93
  mac0ne authored Apr 21, 2025
  
  2b3bdc93
- Add sanity check for max_running_requests (#5016) · 5fc4b600
  fzyzcjy authored Apr 21, 2025
  
  5fc4b600
- Fix one more issue reported by torchfix (#4859) · b868526d
  Brayden Zhong authored Apr 20, 2025
  
  b868526d
- compressed_tensors: port w8a16 fp8 from vllm (#4852) · 502524e2
  Juwan Yoo authored Apr 21, 2025
  
  502524e2
- check marlin format before attempting conversion (#4675) · 4c764007
  Enrique Shockwave authored Apr 21, 2025
  
  4c764007
- Feat: Implement JSON Mode (response_format.type="json_object") (#4733) · 9f3bd2ad
  kyle-pena-kuzco authored Apr 20, 2025
```
Co-authored-by: Kyle Pena <kylepena@kyles-macbook-pro.turkey-marlin.ts.net>
```
  9f3bd2ad
- smaller and non gated models for docs (#5378) · 8de53da9
  simveit authored Apr 21, 2025
  
  8de53da9
- add function call parser for DeepSeek V3 (#5224) · fac17acf
  Yi Zhou authored Apr 21, 2025
  
  fac17acf
- [Feature] Prefill assistant response - add continue_final_message parameter (#4226) · 8b39274e
  Adarsh Shirawalmath authored Apr 21, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  8b39274e
- Add test config yamls for Deepseek v3 (#5433) · 5156d5a4
  Baizhou Zhang authored Apr 20, 2025
  
  5156d5a4
- [PD] Fix large page size + chunk prefill (#5588) · c951d312
  Byron Hsu authored Apr 20, 2025
  
  c951d312
- Fix ChatCompletionMessageGenericParam to allow for None content (#5452) · dcb82325
  AmadeusW authored Apr 21, 2025
  
  dcb82325
- fix: use fa3 for gemma2 (#5586) · 66c0ff9e
  Yineng Zhang authored Apr 20, 2025
  
  66c0ff9e
- Fix enable chunked prefill for Llama4 (#5575) · 9a7e83e8
  tarinkk authored Apr 20, 2025
  
  9a7e83e8
20 Apr, 2025 20 commits
- [Feat] upgrade pytorch2.6 (#5417) · 417b44eb
  lukec authored Apr 21, 2025
  
  417b44eb
- [PD] Fix server crash when using batch requests (#5531) · 475e2e37
  fzyzcjy authored Apr 21, 2025
  
  475e2e37
- Tiny improve error message (#5526) · fba86b6b
  fzyzcjy authored Apr 21, 2025
  
  fba86b6b
- Add document for LoRA serving (#5521) · 072b4d03
  Baizhou Zhang authored Apr 20, 2025
  
  072b4d03
- Super tiny fix typo (#5559) · 9c434777
  fzyzcjy authored Apr 21, 2025
  
  9c434777
- Fix torch memory saver not enabled in DP scenario (#5560) · fa2f677e
  fzyzcjy authored Apr 21, 2025
  
  fa2f677e
- Fix DeepEP cannot run on latest master (#5567) · 463d4b74
  fzyzcjy authored Apr 21, 2025
  
  463d4b74
- Fix bench_serving fail when zero warmup requests (#5574) · 9924bbe1
  fzyzcjy authored Apr 21, 2025
  
  9924bbe1
- Release v0.4.5.post2 (#5582) · fbdc94ba
  Lianmin Zheng authored Apr 20, 2025
  
  fbdc94ba
- [Doc]Add instruction for profiling with bench_one_batch (#5581) · b54b5a96
  Baizhou Zhang authored Apr 20, 2025
  
  b54b5a96
- [Fix] fix outlines and xgrammar (#4947) · bca832c7
  JieXin Liang authored Apr 21, 2025
  
  bca832c7
- enable DeepSeek V3 shared_experts_fusion in sm90 (#5571) · d9dd5298
  Xiaoyu Zhang authored Apr 21, 2025
  
  d9dd5298
- Fix BumpAllocator error when no input_ids (#5564) · 0a0dd34e
  fzyzcjy authored Apr 20, 2025
  
  0a0dd34e
- [PD] Fix DeepSeek cannot be run on latest master (#5568) · 80ac527d
  fzyzcjy authored Apr 20, 2025
  
  80ac527d
- [perf] introduce deep gemm group_gemm_masked as bmm (#5432) · 99456bca
  JieXin Liang authored Apr 20, 2025
  
  99456bca
- Fix bench_one_batch producing unnatural results for expert parallel (#5149) · d07e797a
  fzyzcjy authored Apr 20, 2025
  
  d07e797a
- Minor update for ROCm variable style (#5562) · c555d794
  Zhaoyi Li authored Apr 20, 2025
  
  c555d794
- fix hicache write back (#5543) · e2574ee9
  Zhiqiang Xie authored Apr 19, 2025
  
  e2574ee9
- [PD] Support page size > 1 (#5561) · ab4b5606
  Byron Hsu authored Apr 19, 2025
  
  ab4b5606
- Fix sampler nan check when calling top_k_top_p_sampling_from_probs (#5546) · 20f1c8e3
  Yubo Wang authored Apr 19, 2025
  
  20f1c8e3
19 Apr, 2025 2 commits
- Remove one kernel in per_tensor_quant_mla_fp8 (#5549) · 613b197e
  fzyzcjy authored Apr 20, 2025
  
  613b197e
- simplify the control logic for using shared experts fusion (#5504) · d58e3544
  Xiaoyu Zhang authored Apr 20, 2025
  
  d58e3544