Commits · 6ada05d0ed52f099ec8ffb49c7f7aa7efc31cd49 · change / sglang · GitLab

19 Jan, 2025 14 commits
- feat: check for is_cuda for sgl_kernel import (#2984) · 6ada05d0
  Yineng Zhang authored Jan 19, 2025
  
  6ada05d0
- add config to swtich from vllm custom allreduce to sgl_kernel custom allreduce (#2981) · 24cafe31
  yizhang2077 authored Jan 19, 2025
  
  24cafe31
- fix deepseek v2 with cpu device (#2975) · 5a176c92
  Yineng Zhang authored Jan 19, 2025
  
  5a176c92
- [router] Fix sgl router path for release (#2980) · 4719c1d0
  Byron Hsu authored Jan 19, 2025
  
  4719c1d0
- [router] Allow empty worker list for sglang.launch_router (#2979) · ef18b0ed
  Byron Hsu authored Jan 19, 2025
  
  ef18b0ed
- [devcontainer] Fix mount and GPU & Support rust dev (#2978) · 53cc91e5
  Byron Hsu authored Jan 19, 2025
  
  53cc91e5
- remove cub and add cccl (#2976) · d33cbb7e
  Yineng Zhang authored Jan 19, 2025
  
  d33cbb7e
- Simplify logits processor (#2974) · 23196d52
  Lianmin Zheng authored Jan 18, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  23196d52
- Fix the request loggings to make it fully able to be easily replayed (#2973) · 93b77c8e
  Lianmin Zheng authored Jan 18, 2025
  
  93b77c8e
- Remove the unused write_with_records (#2972) · 7906d1d2
  Lianmin Zheng authored Jan 18, 2025
  
  7906d1d2
- Refactor to add TypeBasedDispatcher to simplify dispatching (#2958) · 81d27c8e
  fzyzcjy authored Jan 19, 2025
  
  81d27c8e
- Frontend: better error message handling for FINISH_ABORT in scheduler.py (#2956) · 4d4cdb3f
  Chang Su authored Jan 18, 2025
  
  4d4cdb3f
- Memory pool: Minor optimize to avoid to (#2901) · 2bd18e2d
  Yang Zheng authored Jan 19, 2025
  
  2bd18e2d
- fix file name spelling mistake and useless variable in minmax-text-01-lightning_attention (#2971) · 83452dbb
  Xiaoyu Zhang authored Jan 19, 2025
  
  83452dbb
18 Jan, 2025 9 commits
- [Feature] Support minicpmv v2.6 (#2785) · 3d93f84a
  Mick authored Jan 19, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  3d93f84a
- optimize MiniMax-Text-01 lightning_attn_decode triton (#2966) · c2f212d6
  Xiaoyu Zhang authored Jan 18, 2025
  
  c2f212d6
- upgrade cutlass v3.7.0 (#2967) · e2cdc8a5
  Yineng Zhang authored Jan 18, 2025
  
  e2cdc8a5
- feat: remove vllm get_rope (#2964) · 2add697d
  Yineng Zhang authored Jan 18, 2025
  
  2add697d
- fix sgl-kernel setup.py (#2963) · 6f98c586
  lukec authored Jan 18, 2025
  
  6f98c586
- Remove fp8 monkey patch (#2960) · 656dcc1a
  Ke Bao authored Jan 18, 2025
  
  656dcc1a
- Query remaining memory dynamically for PrefillAdder (#2941) · 8af7048d
  Zhiqiang Xie authored Jan 17, 2025
  
  8af7048d
- support e4m3 kvcache in qwen2 & add kv scaling facotr json (#2894) · d3024f4f
  bjmsong authored Jan 18, 2025
```
Co-authored-by: bjmsong <bjmsong@126.com>
```
  d3024f4f
- Multi-turn benchmark for hierarchical caching (#2942) · 13387e6b
  Zhiqiang Xie authored Jan 17, 2025
  
  13387e6b
17 Jan, 2025 13 commits
- Fix Llama-3.1-405B References Docs (#2944) · 120c3634
  Wen Sun authored Jan 18, 2025
  
  120c3634
- feat: use get_rope for gemma2 (#2954) · 78e5b22f
  Yineng Zhang authored Jan 18, 2025
  
  78e5b22f
- cleanup models unused import 2/n (#2952) · 7a15e9ad
  Yineng Zhang authored Jan 18, 2025
  
  7a15e9ad
- Update pr template (#2951) · dc2ac0cb
  Ke Bao authored Jan 18, 2025
  
  dc2ac0cb
- Add ut for qwen model (#2947) · d47c5101
  Ke Bao authored Jan 18, 2025
  
  d47c5101
- cleanup models dependencies 1/n (#2948) · 033c715b
  Yineng Zhang authored Jan 17, 2025
  
  033c715b
- update ci install dependency (#2949) · d06c1ab5
  Yineng Zhang authored Jan 17, 2025
  
  d06c1ab5
- docs: add Cursor for adoption and sponsorship (#2950) · c5644cac
  Yineng Zhang authored Jan 17, 2025
  
  c5644cac
- Fix qwen accuracy issue (#2945) · 53e6552f
  Ke Bao authored Jan 17, 2025
  
  53e6552f
- feat: remove vllm distributed (#2907) · 5dc54f1a
  Yineng Zhang authored Jan 17, 2025
```
Co-authored-by: Zhangyi <1109276519@qq.com>
```
  5dc54f1a
- Fix sgl-kernel ci (#2938) · f3e9b489
  Ke Bao authored Jan 17, 2025
  
  f3e9b489
- Update release-docs.yml (#2937) · 6a7973ad
  Lianmin Zheng authored Jan 17, 2025
  
  6a7973ad
- Enable CPU device on SGLang (#2806) · 63051738
  Chunyuan WU authored Jan 17, 2025
  
  63051738
16 Jan, 2025 4 commits
- [Frontend] Fix request length check and add option to disallow auto truncation in scheduler (#2876) · a8ccacc8
  Chang Su authored Jan 16, 2025
  
  a8ccacc8
- Fix zmq binding (#2930) · 0427416b
  Lianmin Zheng authored Jan 16, 2025
```
Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
```
  0427416b
- Docs: Update pull_request_template.md (#2928) · bf3edc2c
  Chayenne authored Jan 16, 2025
  
  bf3edc2c
- [kernel] MiniMax-Text-01 decode lightning_attn with triton (#2920) · 78e974b2
  Xiaoyu Zhang authored Jan 17, 2025
  
  78e974b2