Commits · 6c856b4f3a4e63a25f5adc3388bf79ac2a6e4f72 · change / sglang · GitLab

21 Jan, 2025 6 commits
- minor: update Makefile for sgl-kernel (#3025) · 6c856b4f
  Yineng Zhang authored Jan 21, 2025
  
  6c856b4f
- Misc fixes for eagle (flush_cache, CPU overhead) (#3014) · 287d07a6
  Lianmin Zheng authored Jan 20, 2025
  
  287d07a6
- Enable Cohere2 Models (#3018) · d2571dd5
  Hui Liu authored Jan 20, 2025
  
  d2571dd5
- [EAGLE] Fix some boundary situation when retract reqs and req's max token = 1 (#2939) · b730aa6b
  996_icu authored Jan 21, 2025
```
Co-authored-by: josephyou <josephyou@tencent.com>
```
  b730aa6b
- Fix flaky tests in test_programs.py (#3022) · 60b2a44a
  Lianmin Zheng authored Jan 20, 2025
  
  60b2a44a
- [Doc] Update doc of custom logit processor (#3021) · 949b3fbf
  Hongpeng Guo authored Jan 20, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  949b3fbf
20 Jan, 2025 23 commits
- enable kv_scale remap (#3017) · da4e8b38
  Hui Liu authored Jan 20, 2025
  
  da4e8b38
- deepseek v3 and r1 chat template (#3015) · af6c5357
  Enrique Shockwave authored Jan 20, 2025
  
  af6c5357
- bump router to 0.1.3 (#3020) · 3ad4cd49
  Byron Hsu authored Jan 20, 2025
  
  3ad4cd49
- [router] Expose worker startup interval (#3019) · 3a8428ec
  Byron Hsu authored Jan 20, 2025
  
  3a8428ec
- [router] Expose worker startup secs & Return error instead of panic for router init (#3016) · 0311ce8e
  Byron Hsu authored Jan 20, 2025
  
  0311ce8e
- Add compile flags for cutlass 3.x (#3013) · 5dfcacfc
  Ke Bao authored Jan 21, 2025
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  5dfcacfc
- Add clang-format check to sgl-kernel ci (#3012) · 41a0ccd4
  Ke Bao authored Jan 20, 2025
  
  41a0ccd4
- chore: bump v0.4.1.post7 (#3009) · e94fb7cb
  Yineng Zhang authored Jan 20, 2025
  
  e94fb7cb
- [kernel] port rope cuda kernel to sgl-kernel (#2993) · b5caa22d
  Byron Hsu authored Jan 20, 2025
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  b5caa22d
- Sync distributed package from vllm 0.6.4.post1 (#3010) · 73401fd0
  Lianmin Zheng authored Jan 20, 2025
  
  73401fd0
- Roll back to use vllm custom allreduce (#3006) · 89cd9235
  Lianmin Zheng authored Jan 20, 2025
  
  89cd9235
- Fix perf regression on small batch sizes (#3008) · dc188132
  Lianmin Zheng authored Jan 20, 2025
  
  dc188132
- fix moe align blocks benchmark (#3003) · 10bfce71
  yiakwy-xpu-ml-framework-team authored Jan 20, 2025
  
  10bfce71
- [Enhancement] Custom Logit Processor Improvement (#2998) · 583697cd
  Hongpeng Guo authored Jan 20, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  583697cd
- Docs: Add Performance Demonstaration for DPA (#3005) · 2584f6d9
  Chayenne authored Jan 20, 2025
  
  2584f6d9
- Skip flaky custom_logit_processor tests (#3004) · 51e87f6f
  Lianmin Zheng authored Jan 20, 2025
  
  51e87f6f
- Update TypeBasedDispatcher and balance CI tests (#3001) · 09bcbe01
  Lianmin Zheng authored Jan 19, 2025
  
  09bcbe01
- Separate two entry points: Engine and HTTP server (#2996) · 03464890
  Lianmin Zheng authored Jan 19, 2025
```
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
```
  03464890
- keep rotary_embedding only (#2997) · 44a96697
  Yineng Zhang authored Jan 20, 2025
  
  44a96697
- Remove dependency of pynvml on ROCm (#2995) · 1a820e38
  Chaitanya Sri Krishna Lolla authored Jan 20, 2025
  
  1a820e38
- Docs: Only use X-Grammar in structed output (#2991) · 0ffcfdf4
  Chayenne authored Jan 19, 2025
  
  0ffcfdf4
- Improve metrics, logging, and importing orders (#2992) · cd493b5a
  Lianmin Zheng authored Jan 19, 2025
  
  cd493b5a
- Move sgl.Runtime under sglang/lang (#2990) · 61f42b57
  Lianmin Zheng authored Jan 19, 2025
  
  61f42b57
19 Jan, 2025 11 commits
- [Feature] Add sampler custom logits processor (#2396) · e403d237
  Hongpeng Guo authored Jan 19, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  e403d237
- support regex in xgrammar backend (#2983) · 3bcf5ece
  Enrique Shockwave authored Jan 19, 2025
  
  3bcf5ece
- fix custom op version compatibility (#2988) · 2c05f81f
  Yineng Zhang authored Jan 20, 2025
  
  2c05f81f
- [#2812] Make the decode status dict capcity adjustable by a CLI param (#2839) · d77caa2b
  Seungduk Kim authored Jan 20, 2025
  
  d77caa2b
- fix missing revision arg when loading tokenizer (#2982) · 8b6a4486
  giorgiopiatti-dfinity authored Jan 19, 2025
  
  8b6a4486
- cleanup unused header in sgl_kernel (#2986) · a69cb5cf
  Yineng Zhang authored Jan 20, 2025
  
  a69cb5cf
- docs: update supported_models (#2987) · def5c318
  Yineng Zhang authored Jan 20, 2025
  
  def5c318
- update docker dev image (#2985) · 3fc2b625
  Yineng Zhang authored Jan 19, 2025
  
  3fc2b625
- feat: check for is_cuda for sgl_kernel import (#2984) · 6ada05d0
  Yineng Zhang authored Jan 19, 2025
  
  6ada05d0
- add config to swtich from vllm custom allreduce to sgl_kernel custom allreduce (#2981) · 24cafe31
  yizhang2077 authored Jan 19, 2025
  
  24cafe31
- fix deepseek v2 with cpu device (#2975) · 5a176c92
  Yineng Zhang authored Jan 19, 2025
  
  5a176c92