Commits · dea2b84bc302ec757958d615ba5489ad5a1fe652 · change / sglang · GitLab

20 Jun, 2025 15 commits
- [OAI Server Refactor] [ChatCompletions & Completions] Implement UsageInfo Processor (#7360) · dea2b84b
  yhyang201 authored Jun 21, 2025
```
Co-authored-by: Chang Su <chang.s.su@oracle.com>
```
  dea2b84b
- [OAI refactor] Add rerank and score serving (#7399) · cfb2fb5a
  woodx authored Jun 21, 2025
```
Co-authored-by: Chang Su <chang.s.su@oracle.com>
```
  cfb2fb5a
- [DeepSeekNextN] fix: residual of head norm can be None (#7398) · 22bfed75
  Cheng Wan authored Jun 20, 2025
  
  22bfed75
- [Feature] Comprehensive Hybrid Parallelism Support (#6389) · e879d8b7
  Cheng Wan authored Jun 20, 2025
  
  e879d8b7
- Refine OpenAI serving entrypoint to remove batch requests (#7372) · 09988080
  Xinyuan Tong authored Jun 20, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Chang Su <csu272@usc.edu>
```
  09988080
- Support THUDM/GLM-4-0414 (GLM-Z1) Glm4ForCausalLM architecture. (#5485) · 794be55a
  Wenbo Yang authored Jun 21, 2025
  
  794be55a
- [PD] Optimize custom mem pool usage and bump mooncake version (#7393) · 187b85b7
  Shangming Cai authored Jun 21, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  187b85b7
- support return logprobs for pipeline (#7356) · ceba0ce4
  strgrb authored Jun 20, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  ceba0ce4
- Purge VerlEngine (#7326) · 1ab6be1b
  Ata Fatahi authored Jun 19, 2025
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  1ab6be1b
- Feat/refactor embedding server (#7322) · 4df5fc21
  woodx authored Jun 20, 2025
  
  4df5fc21
- Fix judgment condition for enabling Deepseek V3/R1 shared expert fusion optimization (#7371) · a06912ad
  Li Hui authored Jun 20, 2025
  
  a06912ad
- [Doc] add embedding rerank doc (#7364) · 97011abc
  woodx authored Jun 20, 2025
  
  97011abc
- [Bugfix]Fix hang bug using dp attention with HiRadixCache (#7159) · 1d6515ef
  Huang Long authored Jun 20, 2025
```
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
```
  1d6515ef
- Clean unused import for mimo mtp model (#7370) · dea8aa7a
  Li Hui authored Jun 20, 2025
  
  dea8aa7a
- [Docker] optimize dockerfile remove deepep and blackwell merge it to… (#7343) · 906dbc34
  ybyang authored Jun 20, 2025
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  906dbc34
19 Jun, 2025 9 commits
- docs: update installation (#7366) · fadf18fd
  Yineng Zhang authored Jun 19, 2025
  
  fadf18fd
- [Bugfix][PD] Set conclude state before clear when failure happens (#7362) · f88e7085
  Shangming Cai authored Jun 20, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  f88e7085
- [PD] Transfer hidden states for mtp when disaggregation (#7242) · 4f838c09
  Atream authored Jun 20, 2025
  
  4f838c09
- Put `_normalize_rid` before other normalization in `io_struct` (#7363) · d20a073b
  Chang Su authored Jun 19, 2025
  
  d20a073b
- [Refactor] Clean up radix cache related API (#7303) · 47367b76
  DarkSharpness authored Jun 19, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  47367b76
- Reintroduce tiny fix sampler error when prob is not contiguous (#7354) · 650127a1
  fzyzcjy authored Jun 20, 2025
  
  650127a1
- Multi-Stage Awake: Support Resume and Pause KV Cache and Weights separately (#7099) · 3774f078
  Stefan He authored Jun 19, 2025
  
  3774f078
- add seed in CPU UTs to avoid flaky failure (#7333) · 9179ea15
  Chunyuan WU authored Jun 19, 2025
  
  9179ea15
- fix: resolve blackwell deepep image issue (#7331) · 20a503c7
  Yineng Zhang authored Jun 18, 2025
  
  20a503c7
18 Jun, 2025 12 commits
- Add more refactored openai test & in CI (#7284) · ffd1a26e
  Jinn authored Jun 18, 2025
  
  ffd1a26e
- Merge PDLB (Prefill-Decode Load Balancer) into SGLang Router (#7096) · 09ae5b20
  Simo Lin authored Jun 18, 2025
  
  09ae5b20
- [pd] optimize dockerfile for pd disaggregation (#7319) · 712bf9ec
  ybyang authored Jun 19, 2025
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  712bf9ec
- Fix profiler error when there are idle passes (#7003) · 9c6a0656
  fzyzcjy authored Jun 19, 2025
  
  9c6a0656
- Fix mini_lb for PD with long output: limit chunk size of decode response (#7301) · 2ae809c5
  ch-tiger1 authored Jun 19, 2025
```
Signed-off-by: ch-tiger1 <xyz@ch-tech.ip-ddns.com>
Co-authored-by: ch-tiger1 <xyz@ch-tech.ip-ddns.com>
```
  2ae809c5
- update invalid link in doc (#7297) · 1de4db9b
  linzhuo authored Jun 18, 2025
  
  1de4db9b
- chore: change logs from`INFO` to `DEBUG` for dp and add force quit for tokenizer manager (#7251) · 31fccf5a
  ishandhanani authored Jun 18, 2025
  
  31fccf5a
- Fix hicache benchmark script bug - some sampled input_request is [] (#7300) · b783c1cb
  Binyao Jiang authored Jun 17, 2025
  
  b783c1cb
- Update python API of activation, topk, norm and rope and remove vllm dependency (#6614) · 094c116f
  YanbingJiang authored Jun 18, 2025
```
Co-authored-by: Wu, Chunyuan <chunyuan.wu@intel.com>
Co-authored-by: jianan-gu <jianan.gu@intel.com>
Co-authored-by: sdp <sdp@gnr799219.jf.intel.com>
```
  094c116f
- Upstreaming hicache bug fixes (#7267) · e56685ac
  Zhiqiang Xie authored Jun 17, 2025
  
  e56685ac
- [PD] Add custom memory pool option to support Mooncake PD with NVLink (#7264) · c26d7349
  shangmingc authored Jun 18, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  c26d7349
- [PD] Support get local ip from NIC for PD disaggregation (#7237) · ceaa85c9
  shangmingc authored Jun 18, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  ceaa85c9
17 Jun, 2025 4 commits
- fix: only enable flash_attn test on sm80 sm90 (#7289) · 0650e517
  Yineng Zhang authored Jun 17, 2025
  
  0650e517
- ci: Fix test_ebnf_generate_all_optional_function_params (#7288) · fc554105
  Chang Su authored Jun 17, 2025
  
  fc554105
- fix: resolve b200 dsv3 mtp issue (#7286) · 4f204db5
  Yineng Zhang authored Jun 17, 2025
  
  4f204db5
- Fix AWQ Dequant and Weight Loading of deepseek v2 (#6842) · 3eb4a800
  AniZpZ authored Jun 18, 2025
  
  3eb4a800