Commits · 4d253057000eaf5a4b9a8cc9932e884c6ecdfca0 · change / sglang

"magic_pdf/vscode:/vscode.git/clone" did not exist on "34f8965007615a98453959fa351db1530d0a38ff"

23 Mar, 2025 4 commits
- Move mem_state update into debug mode (#4525) · 4d253057
  Zhiqiang Xie authored Mar 23, 2025
  
  4d253057
- refactor: bug fixes and refactor for vlm (#4661) · 11577ced
  Mick authored Mar 23, 2025
  
  11577ced
- Support async in DeepEP (#4610) · ca75741e
  fzyzcjy authored Mar 23, 2025
```
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
```
  ca75741e
- Multiple tiny code cleanups (#4608) · c6d549e7
  fzyzcjy authored Mar 23, 2025
  
  c6d549e7
22 Mar, 2025 15 commits
- close gemma2 in test_verl_engine.py temporarily (#4685) · 3c09548d
  Yi Zhang authored Mar 23, 2025
  
  3c09548d
- fix typo SGLang supports three grammar backends (#4679) · 8796cebb
  BroadbentJim authored Mar 22, 2025
  
  8796cebb
- Optimize Permute Kernel in DeepEP (#4643) · c2bd094d
  xutizhou authored Mar 23, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  c2bd094d
- [Bug Fix] Add partial rotary factor support for Phi-4 and upgrade to transformers v4.50.0 (#3984) · f8f9244a
  Adarsh Shirawalmath authored Mar 23, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  f8f9244a
- Bug fix for metrics counter (#4660) · ecbfe58b
  Zhiqiang Xie authored Mar 22, 2025
  
  ecbfe58b
- Add EAGLE mtbench benchmark script (#4676) · 8f163b16
  Ke Bao authored Mar 23, 2025
```
Co-authored-by: chromecast56 <jamesll@mit.edu>
```
  8f163b16
- fix flaky ut (#4670) · e7a8610d
  Yineng Zhang authored Mar 22, 2025
  
  e7a8610d
- [CI fix] test skipping modelopt on AMD (#4677) · a2cc62a6
  Adarsh Shirawalmath authored Mar 23, 2025
  
  a2cc62a6
- [Docs] Update docs for gemma3 and VLM chat templates (#4674) · fb888603
  Adarsh Shirawalmath authored Mar 22, 2025
  
  fb888603
- [1/3] fix dsv3 awq issue (#4556) · 321ab756
  AniZpZ authored Mar 22, 2025
```
Co-authored-by: leoneo <1320612015@qq.com>
```
  321ab756
- Correcting default configuration when benchmarking fused_moe (#4665) · 38f25e87
  penguin_wwy authored Mar 22, 2025
  
  38f25e87
- [quantization] fix channelwise conversion with scalar weight scale (#4596) · 8cd42504
  Yun Dai authored Mar 22, 2025
  
  8cd42504
- Speed up per token and per tensor quant by 15% (#4639) · 6a384d5c
  Chunan Zeng authored Mar 22, 2025
  
  6a384d5c
- update news for README (#4664) · f69e0696
  Yineng Zhang authored Mar 21, 2025
  
  f69e0696
- fix: fix ipython running error for Engine due to outlines nest_asyncio (#4582) · f6ab4ca6
  mlmz authored Mar 22, 2025
```
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
```
  f6ab4ca6
21 Mar, 2025 5 commits
- [PD] Release initial code (#4654) · c7c7dbeb
  Byron Hsu authored Mar 21, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Ying1123 <sqy1415@gmail.com>
Co-authored-by: merrymercy <lianminzheng@gmail.com>
Co-authored-by: makro
Co-authored-by: dhou-xai
```
  c7c7dbeb
- Align completion and chat_completion response to OpenAI API (#4637) · 417fc72f
  Yuhong Guo authored Mar 21, 2025
  
  417fc72f
- [docs] Add links and fix grammars in deploy_on_k8s.md (#4641) · c6ec7029
  Michael Yao authored Mar 21, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  c6ec7029
- Set deepgemm to the default value in the hopper architecture. (#4613) · 4c56e5db
  lukec authored Mar 21, 2025
  
  4c56e5db
- fix SUPPORT_CUTLASS_BLOCK_FP8 flag (#4640) · 7b5fc719
  Cheng Wan authored Mar 21, 2025
  
  7b5fc719
20 Mar, 2025 5 commits
- Support fp8 gemm for blackwell (#4558) · ad4e58bf
  Shu Wang authored Mar 20, 2025
  
  ad4e58bf
- Update doc for MTP and DP attention (#4622) · bfb03c61
  Ke Bao authored Mar 21, 2025
  
  bfb03c61
- Enable setting sglang logger from Env Variable `SGLANG_LOGGING_CONFIG_PATH ` (#4592) · b36ab493
  Yuhong Guo authored Mar 20, 2025
```
Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com>
```
  b36ab493
- [fix] fix illegal mem access and clean up triton attention backend (#4571) · 9e93ef3f
  JieXin Liang authored Mar 20, 2025
  
  9e93ef3f
- Support `n` in OpenAI API completions (#3446) · fad86a68
  Chuyue Sun authored Mar 19, 2025
```
Co-authored-by: Shan Yu <shanyu1@g.ucla.edu>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: chuyue sun <chuyue@lmsys.us-northcentral1-a.compute.internal>
```
  fad86a68
19 Mar, 2025 9 commits
- avoid cudaStreamSynchronize in DeepSeekV2AttentionMLA (#4577) · df7014a8
  strgrb authored Mar 20, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  df7014a8
- [fix] fix initialization of _ENABLE_TORCH_INFERENCE_MODE (#4549) · 49420741
  JieXin Liang authored Mar 20, 2025
  
  49420741
- Add clang-format to pre-commit config (#4583) · ba52fd18
  Hongbo Xu authored Mar 20, 2025
  
  ba52fd18
- Support FlashMLA backend cuda graph (#4514) · b6944f97
  lukec authored Mar 19, 2025
```
Co-authored-by: yinfan98 <1106310035@qq.com>
Co-authored-by: Hongbosherlock <hongbosherlock@gmail.com>
Co-authored-by: ispobock <ispobaoke@163.com>
```
  b6944f97
- [Feature] Integrate DeepEP into SGLang (#4232) · f44db16c
  Jinyan Chen authored Mar 19, 2025
```
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
```
  f44db16c
- Create col-major and tma-aligned x_scale for deep_gemm.gemm_fp8_fp8_bf16_nt (#4515) · f9c53cbb
  strgrb authored Mar 19, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  f9c53cbb
- [Fix] Fix raw_bs bug when using flashinfer mla and eagle (#4557) · 90532b76
  Baizhou Zhang authored Mar 18, 2025
  
  90532b76
- Optimize Triton decoding kernel for dynamic workload (#4553) · c0e9a36c
  JieXin Liang authored Mar 19, 2025
  
  c0e9a36c
- [Feature] Support Tensor Parallelism and Weight Slicing for Lora (#4274) · 588865f0
  aoshen524 authored Mar 18, 2025
```
Co-authored-by: ShenAo1111 <1377693092@qq.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  588865f0
18 Mar, 2025 2 commits
- Reduce computation and communication in DP attention (#4521) · 3196999f
  Cheng Wan authored Mar 18, 2025
  
  3196999f
- [Feature] Support EAGLE 3 (#4247) · 9e0186f3
  James Liu authored Mar 18, 2025
  
  9e0186f3