Commits · b49d6d0fee3cf83d72ed658bd9f514bd87fcaa56 · zhaoyu6 / sglang

"src/vscode:/vscode.git/clone" did not exist on "cef0e3677e9a32fcf27cdfccfdf809f689a6f908"

31 Jan, 2025 2 commits
- Fix block wise fp8 torch compile (#3232) · c02e3139
  Ke Bao authored Jan 31, 2025
  
  c02e3139
- [fix] Clamp logprob with dtype min to prevent `-inf` (#3224) · 734daedd
  Byron Hsu authored Jan 31, 2025
  
  734daedd
28 Jan, 2025 3 commits
- [Fix] Address remaining issues of supporting MiniCPMV (#2977) · 9f635ea5
  Mick authored Jan 28, 2025
  
  9f635ea5
- [kernel] Use sgl_kernel rope (#3169) · 988d0a4b
  Byron Hsu authored Jan 27, 2025
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  988d0a4b
- Docs fix about EAGLE and streaming output (#3166) · 7b9b4f44
  Jhin authored Jan 27, 2025
```
Co-authored-by: Chayenne <zhaochenyang@ucla.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Jhin <jhinpan@umich.edu>
```
  7b9b4f44
27 Jan, 2025 6 commits
- Sanity check to prevent performance regression (#3171) · 08104b56
  Zhiqiang Xie authored Jan 27, 2025
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  08104b56
- chore: bump v0.4.2 (#3180) · 4ab43cfb
  Yineng Zhang authored Jan 27, 2025
  
  4ab43cfb
- feat: use sgl-kernel 0.0.3 in sglang (#3179) · 2f79f588
  Yineng Zhang authored Jan 27, 2025
  
  2f79f588
- Improve weight loading and code style (#3174) · 53cef815
  Lianmin Zheng authored Jan 27, 2025
  
  53cef815
- add dsv3 mi300 triton config for block scale (#3146) · 351a72d4
  yigex authored Jan 27, 2025
  
  351a72d4
- Add activation parameters to fused_moe (#3170) · 52c03f16
  Lianmin Zheng authored Jan 27, 2025
  
  52c03f16
26 Jan, 2025 9 commits
- Feature/function calling update (#2700) · b045841b
  YAMY authored Jan 26, 2025
```
Co-authored-by: Mingyuan Ma <mamingyuan2001@berkeley.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
```
  b045841b
- Return more infos for computing average acceptance length (#3152) · 1dda8c5e
  Lianmin Zheng authored Jan 26, 2025
  
  1dda8c5e
- udpate sgl-kernel version for srt (#3150) · 7e097613
  Yineng Zhang authored Jan 26, 2025
  
  7e097613
- Add a test case for cached_tokens (#3145) · d1a08632
  Lianmin Zheng authored Jan 26, 2025
  
  d1a08632
- Add CPU affinity setting to latency benchmark (#3085) · f8b28e46
  Hubert Lu authored Jan 25, 2025
  
  f8b28e46
- Fix repetition penalty (#3139) · 4f118a39
  Lianmin Zheng authored Jan 25, 2025
  
  4f118a39
- [Fix] Not skip NVML Check on AMD Platform (#3135) · 66283dbc
  yigex authored Jan 26, 2025
  
  66283dbc
- enable kv_scale for Gemma2 (#3113) · 8e48ca8c
  Hui Liu authored Jan 25, 2025
  
  8e48ca8c
- Use torch.compile for scaling penalty (#3133) · 27acf63b
  Lianmin Zheng authored Jan 25, 2025
  
  27acf63b
23 Jan, 2025 3 commits
- Revert "disable custom allreduce on HIP" (#3067) · ea535dc5
  Lianmin Zheng authored Jan 22, 2025
  
  ea535dc5
- Support loading of larger models with on-the-fly quantization (#3061) · 862bcff8
  Ke Wen authored Jan 22, 2025
  
  862bcff8
- Fix tp token sync for dp attention (#3062) · 8b84e69f
  Lianmin Zheng authored Jan 22, 2025
  
  8b84e69f
22 Jan, 2025 5 commits
- Add some flags to allow sync token ids across TP ranks (#3060) · 022614d2
  Lianmin Zheng authored Jan 22, 2025
  
  022614d2
- Fix the FP8 E4M3 parsing offline scales failure bug (#3045) · b8ab989f
  lukec authored Jan 23, 2025
  
  b8ab989f
- disable custom allreduce on HIP (#3058) · ddc2001f
  Hui Liu authored Jan 22, 2025
  
  ddc2001f
- fix rotary_embedding rope_scaling for phi (#3055) · 0d2148ef
  nstream-ai-devx authored Jan 22, 2025
  
  0d2148ef
- Use int64 as indices for set_kv_buffer (#3039) · 3d8f1c9b
  Lianmin Zheng authored Jan 21, 2025
  
  3d8f1c9b
21 Jan, 2025 6 commits
- Add accuracy and latency tests of eagle into CI (#3027) · a4331cd2
  Lianmin Zheng authored Jan 21, 2025
  
  a4331cd2
- Misc fixes for eagle (flush_cache, CPU overhead) (#3014) · 287d07a6
  Lianmin Zheng authored Jan 20, 2025
  
  287d07a6
- Enable Cohere2 Models (#3018) · d2571dd5
  Hui Liu authored Jan 20, 2025
  
  d2571dd5
- [EAGLE] Fix some boundary situation when retract reqs and req's max token = 1 (#2939) · b730aa6b
  996_icu authored Jan 21, 2025
```
Co-authored-by: josephyou <josephyou@tencent.com>
```
  b730aa6b
- Fix flaky tests in test_programs.py (#3022) · 60b2a44a
  Lianmin Zheng authored Jan 20, 2025
  
  60b2a44a
- [Doc] Update doc of custom logit processor (#3021) · 949b3fbf
  Hongpeng Guo authored Jan 20, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  949b3fbf
20 Jan, 2025 6 commits
- enable kv_scale remap (#3017) · da4e8b38
  Hui Liu authored Jan 20, 2025
  
  da4e8b38
- deepseek v3 and r1 chat template (#3015) · af6c5357
  Enrique Shockwave authored Jan 20, 2025
  
  af6c5357
- chore: bump v0.4.1.post7 (#3009) · e94fb7cb
  Yineng Zhang authored Jan 20, 2025
  
  e94fb7cb
- Sync distributed package from vllm 0.6.4.post1 (#3010) · 73401fd0
  Lianmin Zheng authored Jan 20, 2025
  
  73401fd0
- Roll back to use vllm custom allreduce (#3006) · 89cd9235
  Lianmin Zheng authored Jan 20, 2025
  
  89cd9235
- Fix perf regression on small batch sizes (#3008) · dc188132
  Lianmin Zheng authored Jan 20, 2025
  
  dc188132