Commits · 1be6956d1bd7188cc9ecb1f43bcf978f9fa5152c · change / sglang · GitLab

18 May, 2025 30 commits
- [Bugfix] Fix field error in v1_embedding_request (#6400) · 1be6956d
  Chang Su authored May 18, 2025
  
  1be6956d
- vlm: tensor hash kernel (#5974) · 626ccb7d
  Mick authored May 19, 2025
  
  626ccb7d
- Refactor DeepSeek MoE layer to unify the two forward branches (#6325) · 72bfb0ba
  fzyzcjy authored May 19, 2025
  
  72bfb0ba
- refactor: Extract repeated member variables in KVCache subclasses to base class. (#6323) · 15521495
  wangxiyu191 authored May 19, 2025
  
  15521495
- [Misc] Implement RankZeroFilter for rank-specific logging in model_runner.py (#6333) · ebe58d54
  Chang Su authored May 18, 2025
  
  ebe58d54
- [OAI] Add rid tracing for v1/embeddings and fix rid type in Chat (#6397) · 066cf445
  Chang Su authored May 18, 2025
  
  066cf445
- Add missing model to doc (#6396) · 6dc6b306
  applesaucethebun authored May 18, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  6dc6b306
- [fix] fix fa3 forward_decode with spec_decode (#6395) · 1f30c05d
  JieXin Liang authored May 19, 2025
```
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
```
  1f30c05d
- Add fp8 shared_expert kernel for CPU in sgl-kernel and add UT (#6339) · 5dd62c3a
  Chunyuan WU authored May 19, 2025
```
Co-authored-by: Jiang, Yanbing <yanbing.jiang@intel.com>
Co-authored-by: mingfeima <mingfei.ma@intel.com>
```
  5dd62c3a
- Add 4-GPU runner tests and split existing tests (#6383) · f11481b9
  fzyzcjy authored May 19, 2025
  
  f11481b9
- chore: tiny remove duplicated code (#6392) · 9d24c3ff
  doujiang24 authored May 18, 2025
```
Signed-off-by: doujiang24 <doujiang24@gmail.com>
```
  9d24c3ff
- The Gemma template is missing a newline after the user role. (#6331) · 24161c59
  Yury Sulsky authored May 18, 2025
```
Co-authored-by: Yury Sulsky <ysulsky@tesla.com>
```
  24161c59
- feat: add long context example (#6391) · eabcf82a
  Yineng Zhang authored May 18, 2025
  
  eabcf82a
- Clean up AMD CI (#6365) · c47a51db
  Sai Enduri authored May 18, 2025
  
  c47a51db
- Add pipeline parallelism for Qwen2 and Qwen3 Model (#6250) · 11553c1a
  libra authored May 18, 2025
  
  11553c1a
- refactor: minor refactors regarding multimodal processing (#6187) · 01dd39ba
  Mick authored May 18, 2025
  
  01dd39ba
- Do not use FA3 for mistral (#6379) · b3f3d610
  Lianmin Zheng authored May 17, 2025
  
  b3f3d610
- chore: upgrade sgl-kernel v0.1.3 (#6377) · f07c6a00
  Yineng Zhang authored May 17, 2025
  
  f07c6a00
- Fix CI tests (#6362) · 4bb816d4
  Lianmin Zheng authored May 17, 2025
  
  4bb816d4
- [Fix Chat API] add request id for chat/completion for tracing (#6364) · c250939e
  ybyang authored May 18, 2025
  
  c250939e
- fix: allow `launch_dummy_health_check_server` to start inside of running asyncio loop (#6330) · b6909aa2
  ishandhanani authored May 17, 2025
  
  b6909aa2
- Add expert distribution APIs for engine (#6290) · f8728357
  fzyzcjy authored May 18, 2025
  
  f8728357
- Reland tiny refactor DefaultModelLoader.Source (#6041) · 73187152
  fzyzcjy authored May 18, 2025
  
  73187152
- Fix expert distribution recorder and profiler command stuck forever (#6284) · 40865665
  fzyzcjy authored May 18, 2025
  
  40865665
- Support custom DeepEP tuning config (#6257) · fd08c048
  fzyzcjy authored May 18, 2025
  
  fd08c048
- Tiny refactor bench_serving to extract RequestFuncOutput.init_new (#6108) · 26ebb849
  fzyzcjy authored May 18, 2025
  
  26ebb849
- Tiny refactor bench_serving to improve extensibility (#6134) · 02973cd9
  fzyzcjy authored May 18, 2025
  
  02973cd9
- Support outputing details for bench_serving (#6107) · 6d95a35a
  fzyzcjy authored May 18, 2025
  
  6d95a35a
- Fix stop_profile does not wait for finishing (#4741) · 01d2838c
  fzyzcjy authored May 18, 2025
  
  01d2838c
- [fix] illegal memory in _fwd_kernel_ep_scatter_2 and _fwd_kernel_ep_gather (#6348) · e3b8a722
  xutizhou authored May 18, 2025
  
  e3b8a722
17 May, 2025 9 commits
- Use monotonic clock for interval measurement (#6211) · 3cf1473a
  Lifu Huang authored May 17, 2025
```
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
```
  3cf1473a
- Speed up when having padding tokens in DeepEP (#6175) · 27168308
  fzyzcjy authored May 18, 2025
  
  27168308
- [router] Add /list_workers endpoint to router (#6366) · e3bed74a
  Zilin Zhu authored May 18, 2025
  
  e3bed74a
- docs: Update the MD files (#6373) · e9ef39d2
  Vincent Zhong authored May 17, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  e9ef39d2
- perf: Optimize local attention memory allocation in FlashAttentionBackend (#6356) · 205d5cb4
  Chang Su authored May 17, 2025
  
  205d5cb4
- chore: bump sgl-kernel v0.1.3 (#6368) · 3d7f7a43
  Yineng Zhang authored May 17, 2025
  
  3d7f7a43
- Minor code cleanup refactor for DeepSeek models (#6324) · 2df9d40a
  fzyzcjy authored May 17, 2025
  
  2df9d40a
- Fix one wasted kernel in DeepSeek and minor refactor (#6316) · 8dc191f2
  fzyzcjy authored May 17, 2025
  
  8dc191f2
- model(vlm): mistral 3.1 (#5099) · 64825b83
  Kiv Chen authored May 16, 2025
```
Co-authored-by: KivenChen <sleigh-queue-0y@icloud.com>
```
  64825b83
16 May, 2025 1 commit
- docs: update readme (#6361) · 69748d08
  Yineng Zhang authored May 16, 2025
  
  69748d08