Commits · ef5b0ff90b526e4f8dc301553f3a1ce04e9dd71d · change / sglang · GitLab

27 Dec, 2024 6 commits
- chore: bump v0.4.1.post1 (#2616) · ef5b0ff9
  Yineng Zhang authored Dec 28, 2024
  
  ef5b0ff9
- update sgl_moe_align_block_size usage (#2617) · 6e530515
  HandH1998 authored Dec 28, 2024
  
  6e530515
- Change extend attention kernel launch parameter for ROCm platform to … (#2610) · 70dc2fbe
  kk authored Dec 27, 2024
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
```
  70dc2fbe
- Fused moe triton cfg opt for rocm (#2612) · 7ca751ff
  kk authored Dec 27, 2024
```
Co-authored-by: wunhuang <wunhuang@amd.com>
```
  7ca751ff
- Regression fix to AMD/ROCm from recent change (#2606) · 7722c11c
  HAI authored Dec 26, 2024
  
  7722c11c
- Tiny code cleanup in tokenizer_manager.py (#2586) · b2ed5c8e
  fzyzcjy authored Dec 27, 2024
  
  b2ed5c8e
26 Dec, 2024 10 commits
- Super tiny typo fix (#2564) · 44f011d2
  fzyzcjy authored Dec 27, 2024
  
  44f011d2
- fix: package data missing (#2521) · 531d6ea9
  yudian0504 authored Dec 27, 2024
  
  531d6ea9
- Fix test and benchmark scripts (#2598) · dc3bee48
  Lianmin Zheng authored Dec 26, 2024
  
  dc3bee48
- Fix duplicated handling of GetWeightsByNameReqInput (#2565) · 3169e66c
  fzyzcjy authored Dec 26, 2024
  
  3169e66c
- Fix logprob_start_len for multi modal models (#2597) · 77395154
  Lianmin Zheng authored Dec 26, 2024
```
Co-authored-by: libra <lihu723@gmail.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: Wang, Haoyu <haoyu.wang@intel.com>
```
  77395154
- [Feature] Support new parameter - EBNF in xgrammar (#2526) · acb34072
  Adarsh Shirawalmath authored Dec 26, 2024
  
  acb34072
- Error occurs when loading the gemma model in bitsandbytes format. (#2557) · 08effbff
  Sangchun Ha (Patrick) authored Dec 26, 2024
  
  08effbff
- Fix cache hit rate when chunked prefill (#2555) · e7ebecf8
  Liangsheng Yin authored Dec 26, 2024
  
  e7ebecf8
- h100 tuning fused_moe_triton for qwen2 moe (#2560) · 9a23c484
  Xiaoyu Zhang authored Dec 26, 2024
  
  9a23c484
- docs: update deepseek v3 example (#2592) · 635a0426
  Yineng Zhang authored Dec 26, 2024
  
  635a0426
25 Dec, 2024 3 commits
- chore: bump v0.4.1 (#2582) · efc52f85
  Yineng Zhang authored Dec 26, 2024
  
  efc52f85
- use sgl-kernel moe_align_block_size (#2581) · 60e2fdcf
  Yineng Zhang authored Dec 26, 2024
```
Co-authored-by: ispobock <ispobaoke@163.com>
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  60e2fdcf
- Refactor MoE (#2575) · 53aed988
  HandH1998 authored Dec 26, 2024
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  53aed988
24 Dec, 2024 1 commit
- [Bench] Flush cache before benchmarking (#2566) · 8a56b431
  Ying Sheng authored Dec 23, 2024
  
  8a56b431
23 Dec, 2024 2 commits
- Reorg moe code (#2563) · e835a500
  Ke Bao authored Dec 24, 2024
  
  e835a500
- Fix gemlite import (#2553) · 23e5e50f
  Lianmin Zheng authored Dec 22, 2024
  
  23e5e50f
22 Dec, 2024 3 commits
- A better aio rwlock that guarantees the order (#2547) · 41b1db69
  Lianmin Zheng authored Dec 22, 2024
  
  41b1db69
- [Misc] Fix metrics, weight update lock, request logging (#2543) · 84967019
  Lianmin Zheng authored Dec 22, 2024
  
  84967019
- Add lora_paths to v1_chat_generate_request (#2529) · 19ba2b0e
  Lei authored Dec 22, 2024
  
  19ba2b0e
21 Dec, 2024 1 commit
- chore: bump v0.4.0.post2 (#2525) · 8f4d04e5
  Yineng Zhang authored Dec 21, 2024
  
  8f4d04e5
20 Dec, 2024 1 commit
- Add integration with gemlite weight only quant (#2528) · feb2b768
  Jerry Zhang authored Dec 20, 2024
  
  feb2b768
19 Dec, 2024 3 commits
- fix: continue to use flashinfer 0.1.6 temporarily (#2517) · 4b83db24
  Yineng Zhang authored Dec 19, 2024
  
  4b83db24
- docs: update README (#2516) · 64456cf0
  Yineng Zhang authored Dec 19, 2024
  
  64456cf0
- feat: add llama3 eval (#2515) · bb4a9220
  Yineng Zhang authored Dec 19, 2024
  
  bb4a9220
17 Dec, 2024 6 commits
- Print progress bar during cuda graph capture (#2502) · 21e9e63a
  Lianmin Zheng authored Dec 17, 2024
  
  21e9e63a
- Fix openai protocols and pass top_k, min_p (#2499) · 361ea8d9
  Lianmin Zheng authored Dec 17, 2024
  
  361ea8d9
- Add lora_path to chat completion (#2438) · 33c5ff28
  Lei authored Dec 17, 2024
  
  33c5ff28
- ROCm support for sglang.check_env (#2426) · 5ce9daea
  Hui Liu authored Dec 17, 2024
  
  5ce9daea
- Small fix for the order of apply_torchao_config (#2495) · bd619616
  Lianmin Zheng authored Dec 16, 2024
  
  bd619616
- Add a benchmark script for in-batch prefix caching (#2494) · 56198b45
  Lianmin Zheng authored Dec 16, 2024
  
  56198b45
16 Dec, 2024 4 commits
- Revert "Small fixes for torchao quant" (#2493) · ba36b552
  Lianmin Zheng authored Dec 16, 2024
  
  ba36b552
- Simplify pytorch sampling kernel and logit processor (#2491) · 7a1aecb9
  Lianmin Zheng authored Dec 16, 2024
  
  7a1aecb9
- Small fixes for torchao quant (#2476) · 82699474
  Jerry Zhang authored Dec 16, 2024
  
  82699474
- fix moe-ep accuracy issue for fp8 (#2489) · b532a5fd
  xiaobochen authored Dec 16, 2024
  
  b532a5fd