Commits · 0d8d97b8e6d7b3f35c2c2bce9508bdd5dd83cc9b · change / sglang

"tools/vscode:/vscode.git/clone" did not exist on "85c94dc1e4a82a94adb7f386dc62df3cabded454"

31 Dec, 2024 6 commits
- h200 tuning fused_moe_triton config for Mixtral 8x7B/8x22B and Qwen2 57BA14B (#2689) · 286cad3e
  Xiaoyu Zhang authored Dec 31, 2024
  
  286cad3e
- [Fix] fix openai adapter (#2685) · dc7eb01f
  Ying Sheng authored Dec 31, 2024
  
  dc7eb01f
- Eagle speculative decoding part 2: Fix cuda graph + DP attention hanging (#2684) · b0524c37
  Lianmin Zheng authored Dec 31, 2024
```
Co-authored-by: yukavio <kavioyu@gmail.com>
```
  b0524c37
- feat: use CUDA 12.4 by default (for FA3) (#2682) · d49b13c6
  Yineng Zhang authored Dec 31, 2024
  
  d49b13c6
- Support target model verification in the attention backend (#2678) · f44d1439
  Lianmin Zheng authored Dec 30, 2024
```
Co-authored-by: yukavio <kavioyu@gmail.com>
```
  f44d1439
- Improve the computation for time_per_output_token Prometheus metrics (#2674) · 339c69a2
  Lianmin Zheng authored Dec 30, 2024
  
  339c69a2
30 Dec, 2024 6 commits
- Minor follow-up fixes for the logprob refactor (#2670) · 21ec66e5
  Lianmin Zheng authored Dec 30, 2024
  
  21ec66e5
- AMD DeepSeek_V3 FP8 Numerical fix (#2667) · c5210dfa
  HAI authored Dec 30, 2024
  
  c5210dfa
- Add GemLite caching after each capture (#2669) · a29dd950
  mobicham authored Dec 30, 2024
  
  a29dd950
- Refactor logprob computation to return the real logprob used in sampling (#2664) · 9c6ba248
  Lianmin Zheng authored Dec 30, 2024
  
  9c6ba248
- [Docs] clean up structured outputs docs (#2654) · 8c3b420e
  Lianmin Zheng authored Dec 29, 2024
  
  8c3b420e
- fix typo in python/sglang/srt/layers/quantization/fp8.py (#2655) · e6f523b5
  HAI authored Dec 29, 2024
  
  e6f523b5
29 Dec, 2024 10 commits
- Release 0.4.1.post3 - upload the config.json to PyPI (#2647) · 03d5fbfd
  Lianmin Zheng authored Dec 29, 2024
  
  03d5fbfd
- CI: Fix unittest for engine input token ids and output token ids (#2646) · fad29f7f
  Shi Shuai authored Dec 29, 2024
  
  fad29f7f
- [Feature] Get Token IDs with Engine.generate() (#2636) · 35bdb485
  Shi Shuai authored Dec 29, 2024
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  35bdb485
- chore: bump v0.4.1.post2 (#2643) · 3ccf566b
  Yineng Zhang authored Dec 30, 2024
  
  3ccf566b
- Update Triton configs for block fp8 kernels (#2641) · afa0341e
  HandH1998 authored Dec 29, 2024
  
  afa0341e
- AMD: set weights and scaling numbers properly for block FP8 (#2637) · 30828e71
  HAI authored Dec 29, 2024
  
  30828e71
- [Session] Update session control interface (#2635) · e0e09fce
  Ying Sheng authored Dec 29, 2024
  
  e0e09fce
- Add llama_eagle.py (#2640) · 9c05c689
  Lianmin Zheng authored Dec 29, 2024
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  9c05c689
- Clean up wrapper in flashinfer backend (#2638) · 3815b23c
  Lianmin Zheng authored Dec 29, 2024
  
  3815b23c
- [Feature] Function Calling (#2544) · 8ee9a850
  Tanjiro authored Dec 29, 2024
```
Co-authored-by: Haoyu Wang <120358163+HaoyuWang4188@users.noreply.github.com>
```
  8ee9a850
28 Dec, 2024 4 commits
- Add `update_weights_from_tensor` (#2631) · fd28640d
  fzyzcjy authored Dec 29, 2024
  
  fd28640d
- add configs for block fp8 related kernels (#2628) · 7863e436
  Yineng Zhang authored Dec 28, 2024
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  7863e436
- [CI] Fix nightly test and raise better error message (#2626) · 855d0ba3
  Lianmin Zheng authored Dec 27, 2024
```
Co-authored-by: Sangbin <rkooo567@gmail.com>
```
  855d0ba3
- avoid fused_moe_triton `padding` circular import (#2624) · 9254a33a
  Xiaoyu Zhang authored Dec 28, 2024
  
  9254a33a
27 Dec, 2024 8 commits
- [minor] clean up docs and eos id (#2622) · 751e5ca2
  Lianmin Zheng authored Dec 27, 2024
  
  751e5ca2
- [FIX] Update EOS from config (#2475) · 7a7ac6be
  Yang Zheng authored Dec 28, 2024
  
  7a7ac6be
- chore: bump v0.4.1.post1 (#2616) · ef5b0ff9
  Yineng Zhang authored Dec 28, 2024
  
  ef5b0ff9
- update sgl_moe_align_block_size usage (#2617) · 6e530515
  HandH1998 authored Dec 28, 2024
  
  6e530515
- Change extend attention kernel launch parameter for ROCm platform to … (#2610) · 70dc2fbe
  kk authored Dec 27, 2024
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
```
  70dc2fbe
- Fused moe triton cfg opt for rocm (#2612) · 7ca751ff
  kk authored Dec 27, 2024
```
Co-authored-by: wunhuang <wunhuang@amd.com>
```
  7ca751ff
- Regression fix to AMD/ROCm from recent change (#2606) · 7722c11c
  HAI authored Dec 26, 2024
  
  7722c11c
- Tiny code cleanup in tokenizer_manager.py (#2586) · b2ed5c8e
  fzyzcjy authored Dec 27, 2024
  
  b2ed5c8e
26 Dec, 2024 6 commits
- Super tiny typo fix (#2564) · 44f011d2
  fzyzcjy authored Dec 27, 2024
  
  44f011d2
- fix: package data missing (#2521) · 531d6ea9
  yudian0504 authored Dec 27, 2024
  
  531d6ea9
- Fix test and benchmark scripts (#2598) · dc3bee48
  Lianmin Zheng authored Dec 26, 2024
  
  dc3bee48
- Fix duplicated handling of GetWeightsByNameReqInput (#2565) · 3169e66c
  fzyzcjy authored Dec 26, 2024
  
  3169e66c
- Fix logprob_start_len for multi modal models (#2597) · 77395154
  Lianmin Zheng authored Dec 26, 2024
```
Co-authored-by: libra <lihu723@gmail.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: Wang, Haoyu <haoyu.wang@intel.com>
```
  77395154
- [Feature] Support new parameter - EBNF in xgrammar (#2526) · acb34072
  Adarsh Shirawalmath authored Dec 26, 2024
  
  acb34072