Commits · 7e0976133ca435d5b2fa4bddff25794ddf64dabf · change / sglang · GitLab

26 Jan, 2025 7 commits
- udpate sgl-kernel version for srt (#3150) · 7e097613
  Yineng Zhang authored Jan 26, 2025
  
  7e097613
- Add a test case for cached_tokens (#3145) · d1a08632
  Lianmin Zheng authored Jan 26, 2025
  
  d1a08632
- Add CPU affinity setting to latency benchmark (#3085) · f8b28e46
  Hubert Lu authored Jan 25, 2025
  
  f8b28e46
- Fix repetition penalty (#3139) · 4f118a39
  Lianmin Zheng authored Jan 25, 2025
  
  4f118a39
- [Fix] Not skip NVML Check on AMD Platform (#3135) · 66283dbc
  yigex authored Jan 26, 2025
  
  66283dbc
- enable kv_scale for Gemma2 (#3113) · 8e48ca8c
  Hui Liu authored Jan 25, 2025
  
  8e48ca8c
- Use torch.compile for scaling penalty (#3133) · 27acf63b
  Lianmin Zheng authored Jan 25, 2025
  
  27acf63b
23 Jan, 2025 3 commits
- Revert "disable custom allreduce on HIP" (#3067) · ea535dc5
  Lianmin Zheng authored Jan 22, 2025
  
  ea535dc5
- Support loading of larger models with on-the-fly quantization (#3061) · 862bcff8
  Ke Wen authored Jan 22, 2025
  
  862bcff8
- Fix tp token sync for dp attention (#3062) · 8b84e69f
  Lianmin Zheng authored Jan 22, 2025
  
  8b84e69f
22 Jan, 2025 5 commits
- Add some flags to allow sync token ids across TP ranks (#3060) · 022614d2
  Lianmin Zheng authored Jan 22, 2025
  
  022614d2
- Fix the FP8 E4M3 parsing offline scales failure bug (#3045) · b8ab989f
  lukec authored Jan 23, 2025
  
  b8ab989f
- disable custom allreduce on HIP (#3058) · ddc2001f
  Hui Liu authored Jan 22, 2025
  
  ddc2001f
- fix rotary_embedding rope_scaling for phi (#3055) · 0d2148ef
  nstream-ai-devx authored Jan 22, 2025
  
  0d2148ef
- Use int64 as indices for set_kv_buffer (#3039) · 3d8f1c9b
  Lianmin Zheng authored Jan 21, 2025
  
  3d8f1c9b
21 Jan, 2025 6 commits
- Add accuracy and latency tests of eagle into CI (#3027) · a4331cd2
  Lianmin Zheng authored Jan 21, 2025
  
  a4331cd2
- Misc fixes for eagle (flush_cache, CPU overhead) (#3014) · 287d07a6
  Lianmin Zheng authored Jan 20, 2025
  
  287d07a6
- Enable Cohere2 Models (#3018) · d2571dd5
  Hui Liu authored Jan 20, 2025
  
  d2571dd5
- [EAGLE] Fix some boundary situation when retract reqs and req's max token = 1 (#2939) · b730aa6b
  996_icu authored Jan 21, 2025
```
Co-authored-by: josephyou <josephyou@tencent.com>
```
  b730aa6b
- Fix flaky tests in test_programs.py (#3022) · 60b2a44a
  Lianmin Zheng authored Jan 20, 2025
  
  60b2a44a
- [Doc] Update doc of custom logit processor (#3021) · 949b3fbf
  Hongpeng Guo authored Jan 20, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  949b3fbf
20 Jan, 2025 13 commits
- enable kv_scale remap (#3017) · da4e8b38
  Hui Liu authored Jan 20, 2025
  
  da4e8b38
- deepseek v3 and r1 chat template (#3015) · af6c5357
  Enrique Shockwave authored Jan 20, 2025
  
  af6c5357
- chore: bump v0.4.1.post7 (#3009) · e94fb7cb
  Yineng Zhang authored Jan 20, 2025
  
  e94fb7cb
- Sync distributed package from vllm 0.6.4.post1 (#3010) · 73401fd0
  Lianmin Zheng authored Jan 20, 2025
  
  73401fd0
- Roll back to use vllm custom allreduce (#3006) · 89cd9235
  Lianmin Zheng authored Jan 20, 2025
  
  89cd9235
- Fix perf regression on small batch sizes (#3008) · dc188132
  Lianmin Zheng authored Jan 20, 2025
  
  dc188132
- [Enhancement] Custom Logit Processor Improvement (#2998) · 583697cd
  Hongpeng Guo authored Jan 20, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  583697cd
- Update TypeBasedDispatcher and balance CI tests (#3001) · 09bcbe01
  Lianmin Zheng authored Jan 19, 2025
  
  09bcbe01
- Separate two entry points: Engine and HTTP server (#2996) · 03464890
  Lianmin Zheng authored Jan 19, 2025
```
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
```
  03464890
- keep rotary_embedding only (#2997) · 44a96697
  Yineng Zhang authored Jan 20, 2025
  
  44a96697
- Remove dependency of pynvml on ROCm (#2995) · 1a820e38
  Chaitanya Sri Krishna Lolla authored Jan 20, 2025
  
  1a820e38
- Improve metrics, logging, and importing orders (#2992) · cd493b5a
  Lianmin Zheng authored Jan 19, 2025
  
  cd493b5a
- Move sgl.Runtime under sglang/lang (#2990) · 61f42b57
  Lianmin Zheng authored Jan 19, 2025
  
  61f42b57
19 Jan, 2025 6 commits
- [Feature] Add sampler custom logits processor (#2396) · e403d237
  Hongpeng Guo authored Jan 19, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  e403d237
- support regex in xgrammar backend (#2983) · 3bcf5ece
  Enrique Shockwave authored Jan 19, 2025
  
  3bcf5ece
- fix custom op version compatibility (#2988) · 2c05f81f
  Yineng Zhang authored Jan 20, 2025
  
  2c05f81f
- [#2812] Make the decode status dict capcity adjustable by a CLI param (#2839) · d77caa2b
  Seungduk Kim authored Jan 20, 2025
  
  d77caa2b
- fix missing revision arg when loading tokenizer (#2982) · 8b6a4486
  giorgiopiatti-dfinity authored Jan 19, 2025
  
  8b6a4486
- feat: check for is_cuda for sgl_kernel import (#2984) · 6ada05d0
  Yineng Zhang authored Jan 19, 2025
  
  6ada05d0