Commits · 03464890e0e0d048ebc1aa407e5235d7338f6aff · change / sglang · GitLab

20 Jan, 2025 6 commits
- Separate two entry points: Engine and HTTP server (#2996) · 03464890
  Lianmin Zheng authored Jan 19, 2025
```
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
```
  03464890
- keep rotary_embedding only (#2997) · 44a96697
  Yineng Zhang authored Jan 20, 2025
  
  44a96697
- Remove dependency of pynvml on ROCm (#2995) · 1a820e38
  Chaitanya Sri Krishna Lolla authored Jan 20, 2025
  
  1a820e38
- Docs: Only use X-Grammar in structed output (#2991) · 0ffcfdf4
  Chayenne authored Jan 19, 2025
  
  0ffcfdf4
- Improve metrics, logging, and importing orders (#2992) · cd493b5a
  Lianmin Zheng authored Jan 19, 2025
  
  cd493b5a
- Move sgl.Runtime under sglang/lang (#2990) · 61f42b57
  Lianmin Zheng authored Jan 19, 2025
  
  61f42b57
19 Jan, 2025 22 commits
- [Feature] Add sampler custom logits processor (#2396) · e403d237
  Hongpeng Guo authored Jan 19, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  e403d237
- support regex in xgrammar backend (#2983) · 3bcf5ece
  Enrique Shockwave authored Jan 19, 2025
  
  3bcf5ece
- fix custom op version compatibility (#2988) · 2c05f81f
  Yineng Zhang authored Jan 20, 2025
  
  2c05f81f
- [#2812] Make the decode status dict capcity adjustable by a CLI param (#2839) · d77caa2b
  Seungduk Kim authored Jan 20, 2025
  
  d77caa2b
- fix missing revision arg when loading tokenizer (#2982) · 8b6a4486
  giorgiopiatti-dfinity authored Jan 19, 2025
  
  8b6a4486
- cleanup unused header in sgl_kernel (#2986) · a69cb5cf
  Yineng Zhang authored Jan 20, 2025
  
  a69cb5cf
- docs: update supported_models (#2987) · def5c318
  Yineng Zhang authored Jan 20, 2025
  
  def5c318
- update docker dev image (#2985) · 3fc2b625
  Yineng Zhang authored Jan 19, 2025
  
  3fc2b625
- feat: check for is_cuda for sgl_kernel import (#2984) · 6ada05d0
  Yineng Zhang authored Jan 19, 2025
  
  6ada05d0
- add config to swtich from vllm custom allreduce to sgl_kernel custom allreduce (#2981) · 24cafe31
  yizhang2077 authored Jan 19, 2025
  
  24cafe31
- fix deepseek v2 with cpu device (#2975) · 5a176c92
  Yineng Zhang authored Jan 19, 2025
  
  5a176c92
- [router] Fix sgl router path for release (#2980) · 4719c1d0
  Byron Hsu authored Jan 19, 2025
  
  4719c1d0
- [router] Allow empty worker list for sglang.launch_router (#2979) · ef18b0ed
  Byron Hsu authored Jan 19, 2025
  
  ef18b0ed
- [devcontainer] Fix mount and GPU & Support rust dev (#2978) · 53cc91e5
  Byron Hsu authored Jan 19, 2025
  
  53cc91e5
- remove cub and add cccl (#2976) · d33cbb7e
  Yineng Zhang authored Jan 19, 2025
  
  d33cbb7e
- Simplify logits processor (#2974) · 23196d52
  Lianmin Zheng authored Jan 18, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  23196d52
- Fix the request loggings to make it fully able to be easily replayed (#2973) · 93b77c8e
  Lianmin Zheng authored Jan 18, 2025
  
  93b77c8e
- Remove the unused write_with_records (#2972) · 7906d1d2
  Lianmin Zheng authored Jan 18, 2025
  
  7906d1d2
- Refactor to add TypeBasedDispatcher to simplify dispatching (#2958) · 81d27c8e
  fzyzcjy authored Jan 19, 2025
  
  81d27c8e
- Frontend: better error message handling for FINISH_ABORT in scheduler.py (#2956) · 4d4cdb3f
  Chang Su authored Jan 18, 2025
  
  4d4cdb3f
- Memory pool: Minor optimize to avoid to (#2901) · 2bd18e2d
  Yang Zheng authored Jan 19, 2025
  
  2bd18e2d
- fix file name spelling mistake and useless variable in minmax-text-01-lightning_attention (#2971) · 83452dbb
  Xiaoyu Zhang authored Jan 19, 2025
  
  83452dbb
18 Jan, 2025 9 commits
- [Feature] Support minicpmv v2.6 (#2785) · 3d93f84a
  Mick authored Jan 19, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  3d93f84a
- optimize MiniMax-Text-01 lightning_attn_decode triton (#2966) · c2f212d6
  Xiaoyu Zhang authored Jan 18, 2025
  
  c2f212d6
- upgrade cutlass v3.7.0 (#2967) · e2cdc8a5
  Yineng Zhang authored Jan 18, 2025
  
  e2cdc8a5
- feat: remove vllm get_rope (#2964) · 2add697d
  Yineng Zhang authored Jan 18, 2025
  
  2add697d
- fix sgl-kernel setup.py (#2963) · 6f98c586
  lukec authored Jan 18, 2025
  
  6f98c586
- Remove fp8 monkey patch (#2960) · 656dcc1a
  Ke Bao authored Jan 18, 2025
  
  656dcc1a
- Query remaining memory dynamically for PrefillAdder (#2941) · 8af7048d
  Zhiqiang Xie authored Jan 17, 2025
  
  8af7048d
- support e4m3 kvcache in qwen2 & add kv scaling facotr json (#2894) · d3024f4f
  bjmsong authored Jan 18, 2025
```
Co-authored-by: bjmsong <bjmsong@126.com>
```
  d3024f4f
- Multi-turn benchmark for hierarchical caching (#2942) · 13387e6b
  Zhiqiang Xie authored Jan 17, 2025
  
  13387e6b
17 Jan, 2025 3 commits
- Fix Llama-3.1-405B References Docs (#2944) · 120c3634
  Wen Sun authored Jan 18, 2025
  
  120c3634
- feat: use get_rope for gemma2 (#2954) · 78e5b22f
  Yineng Zhang authored Jan 18, 2025
  
  78e5b22f
- cleanup models unused import 2/n (#2952) · 7a15e9ad
  Yineng Zhang authored Jan 18, 2025
  
  7a15e9ad