Commits · 73401fd0161caef9681e34f36dfead3134edd549 · change / sglang · GitLab

20 Jan, 2025 14 commits
- Sync distributed package from vllm 0.6.4.post1 (#3010) · 73401fd0
  Lianmin Zheng authored Jan 20, 2025
  
  73401fd0
- Roll back to use vllm custom allreduce (#3006) · 89cd9235
  Lianmin Zheng authored Jan 20, 2025
  
  89cd9235
- Fix perf regression on small batch sizes (#3008) · dc188132
  Lianmin Zheng authored Jan 20, 2025
  
  dc188132
- fix moe align blocks benchmark (#3003) · 10bfce71
  yiakwy-xpu-ml-framework-team authored Jan 20, 2025
  
  10bfce71
- [Enhancement] Custom Logit Processor Improvement (#2998) · 583697cd
  Hongpeng Guo authored Jan 20, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  583697cd
- Docs: Add Performance Demonstaration for DPA (#3005) · 2584f6d9
  Chayenne authored Jan 20, 2025
  
  2584f6d9
- Skip flaky custom_logit_processor tests (#3004) · 51e87f6f
  Lianmin Zheng authored Jan 20, 2025
  
  51e87f6f
- Update TypeBasedDispatcher and balance CI tests (#3001) · 09bcbe01
  Lianmin Zheng authored Jan 19, 2025
  
  09bcbe01
- Separate two entry points: Engine and HTTP server (#2996) · 03464890
  Lianmin Zheng authored Jan 19, 2025
```
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
```
  03464890
- keep rotary_embedding only (#2997) · 44a96697
  Yineng Zhang authored Jan 20, 2025
  
  44a96697
- Remove dependency of pynvml on ROCm (#2995) · 1a820e38
  Chaitanya Sri Krishna Lolla authored Jan 20, 2025
  
  1a820e38
- Docs: Only use X-Grammar in structed output (#2991) · 0ffcfdf4
  Chayenne authored Jan 19, 2025
  
  0ffcfdf4
- Improve metrics, logging, and importing orders (#2992) · cd493b5a
  Lianmin Zheng authored Jan 19, 2025
  
  cd493b5a
- Move sgl.Runtime under sglang/lang (#2990) · 61f42b57
  Lianmin Zheng authored Jan 19, 2025
  
  61f42b57
19 Jan, 2025 22 commits
- [Feature] Add sampler custom logits processor (#2396) · e403d237
  Hongpeng Guo authored Jan 19, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  e403d237
- support regex in xgrammar backend (#2983) · 3bcf5ece
  Enrique Shockwave authored Jan 19, 2025
  
  3bcf5ece
- fix custom op version compatibility (#2988) · 2c05f81f
  Yineng Zhang authored Jan 20, 2025
  
  2c05f81f
- [#2812] Make the decode status dict capcity adjustable by a CLI param (#2839) · d77caa2b
  Seungduk Kim authored Jan 20, 2025
  
  d77caa2b
- fix missing revision arg when loading tokenizer (#2982) · 8b6a4486
  giorgiopiatti-dfinity authored Jan 19, 2025
  
  8b6a4486
- cleanup unused header in sgl_kernel (#2986) · a69cb5cf
  Yineng Zhang authored Jan 20, 2025
  
  a69cb5cf
- docs: update supported_models (#2987) · def5c318
  Yineng Zhang authored Jan 20, 2025
  
  def5c318
- update docker dev image (#2985) · 3fc2b625
  Yineng Zhang authored Jan 19, 2025
  
  3fc2b625
- feat: check for is_cuda for sgl_kernel import (#2984) · 6ada05d0
  Yineng Zhang authored Jan 19, 2025
  
  6ada05d0
- add config to swtich from vllm custom allreduce to sgl_kernel custom allreduce (#2981) · 24cafe31
  yizhang2077 authored Jan 19, 2025
  
  24cafe31
- fix deepseek v2 with cpu device (#2975) · 5a176c92
  Yineng Zhang authored Jan 19, 2025
  
  5a176c92
- [router] Fix sgl router path for release (#2980) · 4719c1d0
  Byron Hsu authored Jan 19, 2025
  
  4719c1d0
- [router] Allow empty worker list for sglang.launch_router (#2979) · ef18b0ed
  Byron Hsu authored Jan 19, 2025
  
  ef18b0ed
- [devcontainer] Fix mount and GPU & Support rust dev (#2978) · 53cc91e5
  Byron Hsu authored Jan 19, 2025
  
  53cc91e5
- remove cub and add cccl (#2976) · d33cbb7e
  Yineng Zhang authored Jan 19, 2025
  
  d33cbb7e
- Simplify logits processor (#2974) · 23196d52
  Lianmin Zheng authored Jan 18, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  23196d52
- Fix the request loggings to make it fully able to be easily replayed (#2973) · 93b77c8e
  Lianmin Zheng authored Jan 18, 2025
  
  93b77c8e
- Remove the unused write_with_records (#2972) · 7906d1d2
  Lianmin Zheng authored Jan 18, 2025
  
  7906d1d2
- Refactor to add TypeBasedDispatcher to simplify dispatching (#2958) · 81d27c8e
  fzyzcjy authored Jan 19, 2025
  
  81d27c8e
- Frontend: better error message handling for FINISH_ABORT in scheduler.py (#2956) · 4d4cdb3f
  Chang Su authored Jan 18, 2025
  
  4d4cdb3f
- Memory pool: Minor optimize to avoid to (#2901) · 2bd18e2d
  Yang Zheng authored Jan 19, 2025
  
  2bd18e2d
- fix file name spelling mistake and useless variable in minmax-text-01-lightning_attention (#2971) · 83452dbb
  Xiaoyu Zhang authored Jan 19, 2025
  
  83452dbb
18 Jan, 2025 4 commits
- [Feature] Support minicpmv v2.6 (#2785) · 3d93f84a
  Mick authored Jan 19, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  3d93f84a
- optimize MiniMax-Text-01 lightning_attn_decode triton (#2966) · c2f212d6
  Xiaoyu Zhang authored Jan 18, 2025
  
  c2f212d6
- upgrade cutlass v3.7.0 (#2967) · e2cdc8a5
  Yineng Zhang authored Jan 18, 2025
  
  e2cdc8a5
- feat: remove vllm get_rope (#2964) · 2add697d
  Yineng Zhang authored Jan 18, 2025
  
  2add697d