Commits · 25549433e8ff9965cbbbb8efe9d450753ca23356 · zhaoyu6 / sglang

"host/include/reference_conv_fwd.hpp" did not exist on "6d92959ad3642754d0f6de85388a922d33651578"

23 Jun, 2025 9 commits
- Fix prefill OOM due to wrong token calculation when page > 1 (#7397) · 25549433
  Liangsheng Yin authored Jun 24, 2025
  
  25549433
- [CI] Upgrade mooncake to 0.3.4.post1 to fix 8 gpu tests (#7472) · d6dddc19
  Shangming Cai authored Jun 24, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  d6dddc19
- Fix a bug in BatchTokenIDOut & Misc style and dependency updates (#7457) · 55e03b10
  Lianmin Zheng authored Jun 23, 2025
  
  55e03b10
- Solve docker build failed in the virtual machine (#7290) · 8aa68ed5
  kk authored Jun 23, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
```
  8aa68ed5
- feat: integrate deepgemm into EPMoE (#6821) · 506c4928
  xutizhou authored Jun 23, 2025
```
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: TianQiLin666666 <1834987979@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  506c4928
- Update hyperparameter_tuning.md (#7454) · 30ceccc7
  Lianmin Zheng authored Jun 22, 2025
  
  30ceccc7
- Fix CUDA Graph Check under Deepep with DP FFN (#7451) · ac5010e0
  Cheng Wan authored Jun 22, 2025
  
  ac5010e0
- add fused moe config for qwen3 in triton3.3.1 (#7445) · 3cee035e
  Yi Zhang authored Jun 23, 2025
  
  3cee035e
- [misc] Add PD service discovery support in router (#7361) · 30f2a44a
  Simo Lin authored Jun 22, 2025
  
  30f2a44a
22 Jun, 2025 6 commits
- Fix torch compile run (#7391) · bd4f5818
  kk authored Jun 23, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
```
  bd4f5818
- Remove copy after bmm (#7441) · 50f1b6d6
  Ke Bao authored Jun 23, 2025
  
  50f1b6d6
- FlashInfer NVFP4 MoE with EP & 2-stream shared expert (#7327) · 5962e70d
  Trevor Morris authored Jun 22, 2025
```
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
Co-authored-by: alcanderian <alcanderian@gmail.com>
```
  5962e70d
- Tiny add logging for GC (#7406) · edc21cc8
  fzyzcjy authored Jun 22, 2025
  
  edc21cc8
- [minor] simplify the `TokenToKVPoolAllocator` (#7414) · 05c9bc89
  Liangsheng Yin authored Jun 22, 2025
  
  05c9bc89
- refactor(test): reorganize OpenAI test file structure (#7408) · b7a2df0a
  Chang Su authored Jun 21, 2025
  
  b7a2df0a
21 Jun, 2025 10 commits
- Refactor LoRAManager and LoRAMemoryPool state management logic for dynamic... · 1998ce40
  Lifu Huang authored Jun 21, 2025
```
Refactor LoRAManager and LoRAMemoryPool state management logic for dynamic LoRA loading support (#7412)
```
  1998ce40
- feat(oai refactor): Replace `openai_api` with `entrypoints/openai` (#7351) · 72676cd6
  Chang Su authored Jun 21, 2025
```
Co-authored-by: Jin Pan <jpan236@wisc.edu>
```
  72676cd6
- [fix] PD disaggregation when enable mtp and tp!=dp (#7420) · 02bf31ef
  Atream authored Jun 22, 2025
  
  02bf31ef
- Fix CPU offloading for MLA memory pool (#7409) · 5ea5d221
  Liangsheng Yin authored Jun 22, 2025
  
  5ea5d221
- fix: Fix CI test_function_call_parser.py (#7425) · fdfd5224
  Chang Su authored Jun 21, 2025
  
  fdfd5224
- fix overlap pagecount (#6984) · 7f3ee861
  pansicheng authored Jun 21, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  7f3ee861
- [BugFix]fix qwen25 invoke function call streaming responses with curly braces... · bec58910
  ehuaa authored Jun 21, 2025
```
[BugFix]fix qwen25 invoke function call streaming responses with curly braces as the starting indicator (#7394)
```
  bec58910
- [BugFix]: fix EmbeddingReqInput single input error (#7396) · 9edf6608
  woodx authored Jun 21, 2025
  
  9edf6608
- Remove batches api in docs & example (#7400) · ab74f8f0
  Jinn authored Jun 20, 2025
  
  ab74f8f0
- [OAI Server Refactor] [ChatCompletions & Completions] Support Return Hidden State (#7329) · 5e7fdc79
  Keyang Ru authored Jun 20, 2025
```
Signed-off-by: keru <rukeyang@gmail.com>
```
  5e7fdc79
20 Jun, 2025 15 commits
- chore: upgrade mooncake-transfer-engine 0.3.4 (#7401) · 4d8d9b8e
  Yineng Zhang authored Jun 20, 2025
  
  4d8d9b8e
- Fix 7285 Merge Conflicts (#7403) · 5041df2d
  Cheng Wan authored Jun 20, 2025
  
  5041df2d
- Update usage_processor.py (#7402) · 256801e9
  Cheng Wan authored Jun 20, 2025
  
  256801e9
- Optimize DP attn scheduling for speculative decoding (#7285) · 73b13e69
  Cheng Wan authored Jun 20, 2025
  
  73b13e69
- Fix All-Gather under world size one (#7219) · 8609e637
  Cheng Wan authored Jun 20, 2025
  
  8609e637
- [OAI Server Refactor] [ChatCompletions & Completions] Implement UsageInfo Processor (#7360) · dea2b84b
  yhyang201 authored Jun 21, 2025
```
Co-authored-by: Chang Su <chang.s.su@oracle.com>
```
  dea2b84b
- [OAI refactor] Add rerank and score serving (#7399) · cfb2fb5a
  woodx authored Jun 21, 2025
```
Co-authored-by: Chang Su <chang.s.su@oracle.com>
```
  cfb2fb5a
- [DeepSeekNextN] fix: residual of head norm can be None (#7398) · 22bfed75
  Cheng Wan authored Jun 20, 2025
  
  22bfed75
- [Feature] Comprehensive Hybrid Parallelism Support (#6389) · e879d8b7
  Cheng Wan authored Jun 20, 2025
  
  e879d8b7
- Refine OpenAI serving entrypoint to remove batch requests (#7372) · 09988080
  Xinyuan Tong authored Jun 20, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Chang Su <csu272@usc.edu>
```
  09988080
- Support THUDM/GLM-4-0414 (GLM-Z1) Glm4ForCausalLM architecture. (#5485) · 794be55a
  Wenbo Yang authored Jun 21, 2025
  
  794be55a
- [PD] Optimize custom mem pool usage and bump mooncake version (#7393) · 187b85b7
  Shangming Cai authored Jun 21, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  187b85b7
- support return logprobs for pipeline (#7356) · ceba0ce4
  strgrb authored Jun 20, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  ceba0ce4
- Purge VerlEngine (#7326) · 1ab6be1b
  Ata Fatahi authored Jun 19, 2025
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  1ab6be1b
- Feat/refactor embedding server (#7322) · 4df5fc21
  woodx authored Jun 20, 2025
  
  4df5fc21