Commits · 1998ce40466577a9563149e1449f997dcd162cee · change / sglang · GitLab

21 Jun, 2025 9 commits
- Refactor LoRAManager and LoRAMemoryPool state management logic for dynamic... · 1998ce40
  Lifu Huang authored Jun 21, 2025
```
Refactor LoRAManager and LoRAMemoryPool state management logic for dynamic LoRA loading support (#7412)
```
  1998ce40
- feat(oai refactor): Replace `openai_api` with `entrypoints/openai` (#7351) · 72676cd6
  Chang Su authored Jun 21, 2025
```
Co-authored-by: Jin Pan <jpan236@wisc.edu>
```
  72676cd6
- [fix] PD disaggregation when enable mtp and tp!=dp (#7420) · 02bf31ef
  Atream authored Jun 22, 2025
  
  02bf31ef
- Fix CPU offloading for MLA memory pool (#7409) · 5ea5d221
  Liangsheng Yin authored Jun 22, 2025
  
  5ea5d221
- fix: Fix CI test_function_call_parser.py (#7425) · fdfd5224
  Chang Su authored Jun 21, 2025
  
  fdfd5224
- fix overlap pagecount (#6984) · 7f3ee861
  pansicheng authored Jun 21, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  7f3ee861
- [BugFix]fix qwen25 invoke function call streaming responses with curly braces... · bec58910
  ehuaa authored Jun 21, 2025
```
[BugFix]fix qwen25 invoke function call streaming responses with curly braces as the starting indicator (#7394)
```
  bec58910
- [BugFix]: fix EmbeddingReqInput single input error (#7396) · 9edf6608
  woodx authored Jun 21, 2025
  
  9edf6608
- [OAI Server Refactor] [ChatCompletions & Completions] Support Return Hidden State (#7329) · 5e7fdc79
  Keyang Ru authored Jun 20, 2025
```
Signed-off-by: keru <rukeyang@gmail.com>
```
  5e7fdc79
20 Jun, 2025 17 commits
- Fix 7285 Merge Conflicts (#7403) · 5041df2d
  Cheng Wan authored Jun 20, 2025
  
  5041df2d
- Update usage_processor.py (#7402) · 256801e9
  Cheng Wan authored Jun 20, 2025
  
  256801e9
- Optimize DP attn scheduling for speculative decoding (#7285) · 73b13e69
  Cheng Wan authored Jun 20, 2025
  
  73b13e69
- Fix All-Gather under world size one (#7219) · 8609e637
  Cheng Wan authored Jun 20, 2025
  
  8609e637
- [OAI Server Refactor] [ChatCompletions & Completions] Implement UsageInfo Processor (#7360) · dea2b84b
  yhyang201 authored Jun 21, 2025
```
Co-authored-by: Chang Su <chang.s.su@oracle.com>
```
  dea2b84b
- [OAI refactor] Add rerank and score serving (#7399) · cfb2fb5a
  woodx authored Jun 21, 2025
```
Co-authored-by: Chang Su <chang.s.su@oracle.com>
```
  cfb2fb5a
- [DeepSeekNextN] fix: residual of head norm can be None (#7398) · 22bfed75
  Cheng Wan authored Jun 20, 2025
  
  22bfed75
- [Feature] Comprehensive Hybrid Parallelism Support (#6389) · e879d8b7
  Cheng Wan authored Jun 20, 2025
  
  e879d8b7
- Refine OpenAI serving entrypoint to remove batch requests (#7372) · 09988080
  Xinyuan Tong authored Jun 20, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Chang Su <csu272@usc.edu>
```
  09988080
- Support THUDM/GLM-4-0414 (GLM-Z1) Glm4ForCausalLM architecture. (#5485) · 794be55a
  Wenbo Yang authored Jun 21, 2025
  
  794be55a
- [PD] Optimize custom mem pool usage and bump mooncake version (#7393) · 187b85b7
  Shangming Cai authored Jun 21, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  187b85b7
- support return logprobs for pipeline (#7356) · ceba0ce4
  strgrb authored Jun 20, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  ceba0ce4
- Purge VerlEngine (#7326) · 1ab6be1b
  Ata Fatahi authored Jun 19, 2025
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  1ab6be1b
- Feat/refactor embedding server (#7322) · 4df5fc21
  woodx authored Jun 20, 2025
  
  4df5fc21
- Fix judgment condition for enabling Deepseek V3/R1 shared expert fusion optimization (#7371) · a06912ad
  Li Hui authored Jun 20, 2025
  
  a06912ad
- [Bugfix]Fix hang bug using dp attention with HiRadixCache (#7159) · 1d6515ef
  Huang Long authored Jun 20, 2025
```
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
```
  1d6515ef
- Clean unused import for mimo mtp model (#7370) · dea8aa7a
  Li Hui authored Jun 20, 2025
  
  dea8aa7a
19 Jun, 2025 6 commits
- [Bugfix][PD] Set conclude state before clear when failure happens (#7362) · f88e7085
  Shangming Cai authored Jun 20, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  f88e7085
- [PD] Transfer hidden states for mtp when disaggregation (#7242) · 4f838c09
  Atream authored Jun 20, 2025
  
  4f838c09
- Put `_normalize_rid` before other normalization in `io_struct` (#7363) · d20a073b
  Chang Su authored Jun 19, 2025
  
  d20a073b
- [Refactor] Clean up radix cache related API (#7303) · 47367b76
  DarkSharpness authored Jun 19, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  47367b76
- Reintroduce tiny fix sampler error when prob is not contiguous (#7354) · 650127a1
  fzyzcjy authored Jun 20, 2025
  
  650127a1
- Multi-Stage Awake: Support Resume and Pause KV Cache and Weights separately (#7099) · 3774f078
  Stefan He authored Jun 19, 2025
  
  3774f078
18 Jun, 2025 8 commits
- Add more refactored openai test & in CI (#7284) · ffd1a26e
  Jinn authored Jun 18, 2025
  
  ffd1a26e
- Merge PDLB (Prefill-Decode Load Balancer) into SGLang Router (#7096) · 09ae5b20
  Simo Lin authored Jun 18, 2025
  
  09ae5b20
- Fix profiler error when there are idle passes (#7003) · 9c6a0656
  fzyzcjy authored Jun 19, 2025
  
  9c6a0656
- Fix mini_lb for PD with long output: limit chunk size of decode response (#7301) · 2ae809c5
  ch-tiger1 authored Jun 19, 2025
```
Signed-off-by: ch-tiger1 <xyz@ch-tech.ip-ddns.com>
Co-authored-by: ch-tiger1 <xyz@ch-tech.ip-ddns.com>
```
  2ae809c5
- chore: change logs from`INFO` to `DEBUG` for dp and add force quit for tokenizer manager (#7251) · 31fccf5a
  ishandhanani authored Jun 18, 2025
  
  31fccf5a
- Update python API of activation, topk, norm and rope and remove vllm dependency (#6614) · 094c116f
  YanbingJiang authored Jun 18, 2025
```
Co-authored-by: Wu, Chunyuan <chunyuan.wu@intel.com>
Co-authored-by: jianan-gu <jianan.gu@intel.com>
Co-authored-by: sdp <sdp@gnr799219.jf.intel.com>
```
  094c116f
- Upstreaming hicache bug fixes (#7267) · e56685ac
  Zhiqiang Xie authored Jun 17, 2025
  
  e56685ac
- [PD] Add custom memory pool option to support Mooncake PD with NVLink (#7264) · c26d7349
  shangmingc authored Jun 18, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  c26d7349