Commits · fe6a445d1e1f4162fa62475a62474ff19f1b2f3c · zhaoyu6 / sglang

"host/host_tensor/include/gemm_common.hpp" did not exist on "627d8ef35a6da8ad268b5197e3045ccdfb4ac684"

27 Jul, 2025 1 commit
- [NVIDIA] Add Flashinfer MoE blockscale fp8 backend (#8036) · 85486b6f
  Kaixi Hou authored Jul 27, 2025
  
  85486b6f
26 Jul, 2025 1 commit
- Tiny assert EPLB is used together with expert parallel (#8381) · da0c0260
  fzyzcjy authored Jul 26, 2025
  
  da0c0260
25 Jul, 2025 3 commits
- Fix FP4 MoE accuracy from missing routed_scaling_factor (#8333) · 58c468f4
  Trevor Morris authored Jul 25, 2025
  
  58c468f4
- Clean up server_args, triton cache manager (#8332) · ed2e313e
  Lianmin Zheng authored Jul 25, 2025
  
  ed2e313e
- [Bugfix][Feat] Add XML-ish grammar in EBNFComposer and fix misc bugs in Qwen3 detector (#8357) · f8260f25
  Chang Su authored Jul 25, 2025
  
  f8260f25
23 Jul, 2025 1 commit
- Introduce Stable LoRA ID System for Overlapped Updates and Prefix Caching (#8261) · 8abd3e77
  Lifu Huang authored Jul 23, 2025
  
  8abd3e77
22 Jul, 2025 1 commit
- Preliminary Support for Qwen3XMLDetector (#8260) · 0dfe2491
  yhyang201 authored Jul 23, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  0dfe2491
19 Jul, 2025 4 commits
- Support start up LoRA server without initial adapters (#8019) · 4e3defe5
  Lifu Huang authored Jul 19, 2025
  
  4e3defe5
- Clean up server args (#8161) · bb0e8a32
  Lianmin Zheng authored Jul 19, 2025
  
  bb0e8a32
- Support NVFP4 quantized dense models on AMD CDNA2/CDNA3 GPUs (#7302) · d918ab79
  Haohui Mai authored Jul 18, 2025
```
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
```
  d918ab79
- chore: tune mem fraction static for vlm (#6881) · 3964b352
  Mick authored Jul 19, 2025
  
  3964b352
18 Jul, 2025 1 commit
- Hicache Storage Layer Prototype (#7704) · 9d33fcfb
  Zhiqiang Xie authored Jul 18, 2025
  
  9d33fcfb
17 Jul, 2025 1 commit
- feat: add tp_rank, pp_rank and dp_rank labels for scheduler metrics (#7597) · 795668dc
  Yingchun Lai authored Jul 17, 2025
```
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
```
  795668dc
14 Jul, 2025 2 commits
- [Feature]TP Group Switching for PD-Multiplexing (#7653) · d4d0c7c3
  ykcombat authored Jul 15, 2025
  
  d4d0c7c3
- Refactor dynamic LoRA update to fix incorrect handling of variant weight shapes (#7844) · e2ed9d04
  Lifu Huang authored Jul 13, 2025
  
  e2ed9d04
13 Jul, 2025 2 commits
- SWA Prefix Cache (#7367) · 9379da77
  Hanming Lu authored Jul 13, 2025
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  9379da77
- [minor] Add server_args check for Llama4 with hybrid (#7988) · 42fc4410
  Ying Sheng authored Jul 12, 2025
  
  42fc4410
11 Jul, 2025 2 commits
- [feature] kv transfer support of ascend npu (#7795) · 86044712
  ronnie_zheng authored Jul 11, 2025
```
Co-authored-by: liupeng <liupeng374@huawei.com>
```
  86044712
- Support Kimi K2 (#7940) · 61555307
  Atream authored Jul 11, 2025
  
  61555307
09 Jul, 2025 1 commit
- Fixes typo in assertion message (#7895) · e5888edd
  Xinyuan Tong authored Jul 09, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  e5888edd
07 Jul, 2025 3 commits
- feat: support DeepSeek-R1-W4AFP8 model with ep-moe mode (#7762) · cb9d91ea
  SijiaYang authored Jul 08, 2025
```
Signed-off-by: yangsijia.614 <yangsijia.614@bytedance.com>
```
  cb9d91ea
- Kernels for efficient KV cache IO (#7313) · 2fc824b8
  Zhiqiang Xie authored Jul 06, 2025
  
  2fc824b8
- Integrate triton moe kernel (#7689) · 253454de
  Yuan Luo authored Jul 07, 2025
```
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
```
  253454de
05 Jul, 2025 2 commits
- DP Attention with Auto DeepEP Dispatch (#7222) · 8fc910db
  Cheng Wan authored Jul 05, 2025
  
  8fc910db
- [RL] Add --nccl-port to prevent port conflict (#7418) · 16a6b1d8
  Zilin Zhu authored Jul 05, 2025
  
  16a6b1d8
04 Jul, 2025 1 commit
- Move mem_fraction_static adjustment for multimodal models to `server_args.py`... · 14229ccf
  Lianmin Zheng authored Jul 04, 2025
```
Move mem_fraction_static adjustment for multimodal models to `server_args.py` & Fix session control & Other cleanups (#7748)
```
  14229ccf
03 Jul, 2025 3 commits
- Ascend attention backend(PA&MLA) (#7722) · 1e0e5497
  ronnie_zheng authored Jul 03, 2025
```
Co-authored-by: Maksim <makcum888e@mail.ru>
Co-authored-by: VDV1985 <vladdv85@mail.ru>
```
  1e0e5497
- [b200] support trt-llm allreduce fuse rms_norm_add kernel (#7621) · 8e64140e
  Xiaoyu Zhang authored Jul 03, 2025
  
  8e64140e
- [RL] add --skip-warmup (#7416) · 09e699bb
  Zilin Zhu authored Jul 03, 2025
  
  09e699bb
02 Jul, 2025 1 commit
- Improve error handling for requests with unloaded LoRA path(s) (#7642) · 1a08358a
  Lifu Huang authored Jul 01, 2025
  
  1a08358a
30 Jun, 2025 2 commits
- Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632) · 22352d47
  Lianmin Zheng authored Jun 29, 2025
```
Co-authored-by: Kan Wu <wukanustc@gmail.com>
```
  22352d47
- [EAGLE] remove a wrong adjustment for page_size > 1 & topk > 1 in server_args.py (#7643) · 78700893
  Lianmin Zheng authored Jun 29, 2025
  
  78700893
28 Jun, 2025 1 commit

Hybrid kv cache for LLaMA4 (#6563) · eb6c2c16

tarinkk authored Jun 27, 2025


Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: tarinkk <rt572@physics.rutger.edu>
Co-authored-by: tarinkk <rt572@rutgers.physics.edu>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>

eb6c2c16

25 Jun, 2025 1 commit
- feat: send kvmetrics from sglang scheduler (#6721) · f3cbd245
  zixuanzhang226 authored Jun 25, 2025
  
  f3cbd245
24 Jun, 2025 2 commits
- fix: Add `--model` as an alias for `--model-path` in server_args (#7505) · 3562256b
  Chang Su authored Jun 24, 2025
  
  3562256b
- Support multi-thread model weight loading (#7277) · 9f1787fa
  xianzhiT authored Jun 25, 2025
  
  9f1787fa
23 Jun, 2025 2 commits
- Support weight loading without mmap (#7469) · e5afb88b
  Yuhong Guo authored Jun 24, 2025
  
  e5afb88b
- Fix a bug in BatchTokenIDOut & Misc style and dependency updates (#7457) · 55e03b10
  Lianmin Zheng authored Jun 23, 2025
  
  55e03b10
22 Jun, 2025 1 commit

FlashInfer NVFP4 MoE with EP & 2-stream shared expert (#7327) · 5962e70d

Trevor Morris authored Jun 22, 2025


Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
Co-authored-by: alcanderian <alcanderian@gmail.com>

5962e70d

20 Jun, 2025 1 commit
- [Feature] Comprehensive Hybrid Parallelism Support (#6389) · e879d8b7
  Cheng Wan authored Jun 20, 2025
  
  e879d8b7