Commits · 773951548ddd2d9f98f062788bc1c13aecbcf66d · change / sglang

"vscode:/vscode.git/clone" did not exist on "b75faf5a48a60c094ed37d18fd5bb03b02a8c9a6"

26 Dec, 2024 10 commits
- Fix logprob_start_len for multi modal models (#2597) · 77395154
  Lianmin Zheng authored Dec 26, 2024
```
Co-authored-by: libra <lihu723@gmail.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: Wang, Haoyu <haoyu.wang@intel.com>
```
  77395154
- update readme of DeepSeek V3 (#2596) · 637de9e8
  fsygd authored Dec 26, 2024
  
  637de9e8
- [Feature] Support new parameter - EBNF in xgrammar (#2526) · acb34072
  Adarsh Shirawalmath authored Dec 26, 2024
  
  acb34072
- Error occurs when loading the gemma model in bitsandbytes format. (#2557) · 08effbff
  Sangchun Ha (Patrick) authored Dec 26, 2024
  
  08effbff
- Update README.md (#2594) · 60bd3272
  Lianmin Zheng authored Dec 26, 2024
  
  60bd3272
- Fix cache hit rate when chunked prefill (#2555) · e7ebecf8
  Liangsheng Yin authored Dec 26, 2024
  
  e7ebecf8
- h100 tuning fused_moe_triton for qwen2 moe (#2560) · 9a23c484
  Xiaoyu Zhang authored Dec 26, 2024
  
  9a23c484
- docs: update deepseek v3 example (#2592) · 635a0426
  Yineng Zhang authored Dec 26, 2024
  
  635a0426
- fix: only enable moe_align_block_size for now (#2590) · 2dccecf4
  Yineng Zhang authored Dec 26, 2024
  
  2dccecf4
- docs: add deepseek v3 launch instructions (#2589) · 75ad0a14
  Yineng Zhang authored Dec 26, 2024
  
  75ad0a14
25 Dec, 2024 5 commits
- chore: bump v0.4.1 (#2582) · efc52f85
  Yineng Zhang authored Dec 26, 2024
  
  efc52f85
- use sgl-kernel moe_align_block_size (#2581) · 60e2fdcf
  Yineng Zhang authored Dec 26, 2024
```
Co-authored-by: ispobock <ispobaoke@163.com>
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  60e2fdcf
- chore: bump 0.0.2.post8 for sgl-kernel (#2580) · d7c0e872
  Yineng Zhang authored Dec 26, 2024
  
  d7c0e872
- fix moe_align_block_size_kernel for shared memory issue (#2579) · 31548116
  Yineng Zhang authored Dec 26, 2024
```
Co-authored-by: ispobock <ispobaoke@163.com>
```
  31548116
- Refactor MoE (#2575) · 53aed988
  HandH1998 authored Dec 26, 2024
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  53aed988
24 Dec, 2024 1 commit
- [Bench] Flush cache before benchmarking (#2566) · 8a56b431
  Ying Sheng authored Dec 23, 2024
  
  8a56b431
23 Dec, 2024 3 commits
- Reorg moe code (#2563) · e835a500
  Ke Bao authored Dec 24, 2024
  
  e835a500
- Fix gemlite import (#2553) · 23e5e50f
  Lianmin Zheng authored Dec 22, 2024
  
  23e5e50f
- Doc: Update Grammar Backend (#2545) · 25e5d589
  Shi Shuai authored Dec 23, 2024
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  25e5d589
22 Dec, 2024 5 commits
- A better aio rwlock that guarantees the order (#2547) · 41b1db69
  Lianmin Zheng authored Dec 22, 2024
  
  41b1db69
- [Misc] Fix metrics, weight update lock, request logging (#2543) · 84967019
  Lianmin Zheng authored Dec 22, 2024
  
  84967019
- [kernel optimize] benchmark write_req_to_token_pool_triton and optimize kernel (#2509) · 7d672d27
  Xiaoyu Zhang authored Dec 22, 2024
  
  7d672d27
- docs: update sponsorship (DataCrunch) (#2523) · d4b17481
  Yineng Zhang authored Dec 22, 2024
  
  d4b17481
- Add lora_paths to v1_chat_generate_request (#2529) · 19ba2b0e
  Lei authored Dec 22, 2024
  
  19ba2b0e
21 Dec, 2024 2 commits
- fix #2528 (#2541) · 4e1e3cff
  Yineng Zhang authored Dec 22, 2024
  
  4e1e3cff
- chore: bump v0.4.0.post2 (#2525) · 8f4d04e5
  Yineng Zhang authored Dec 21, 2024
  
  8f4d04e5
20 Dec, 2024 1 commit
- Add integration with gemlite weight only quant (#2528) · feb2b768
  Jerry Zhang authored Dec 20, 2024
  
  feb2b768
19 Dec, 2024 4 commits
- fix followup #2517 (#2524) · d95a5f5b
  Yineng Zhang authored Dec 19, 2024
  
  d95a5f5b
- fix: continue to use flashinfer 0.1.6 temporarily (#2517) · 4b83db24
  Yineng Zhang authored Dec 19, 2024
  
  4b83db24
- docs: update README (#2516) · 64456cf0
  Yineng Zhang authored Dec 19, 2024
  
  64456cf0
- feat: add llama3 eval (#2515) · bb4a9220
  Yineng Zhang authored Dec 19, 2024
  
  bb4a9220
17 Dec, 2024 9 commits
- Print progress bar during cuda graph capture (#2502) · 21e9e63a
  Lianmin Zheng authored Dec 17, 2024
  
  21e9e63a
- Update readme (#2500) · 1fc84cf6
  Lianmin Zheng authored Dec 17, 2024
```
Co-authored-by: Ravi Theja <ravi03071991@gmail.com>
Co-authored-by: “yixin-huang1” <yixinhuang1@berkeley.edu>
```
  1fc84cf6
- Fix openai protocols and pass top_k, min_p (#2499) · 361ea8d9
  Lianmin Zheng authored Dec 17, 2024
  
  361ea8d9
- Add lora_path to chat completion (#2438) · 33c5ff28
  Lei authored Dec 17, 2024
  
  33c5ff28
- ROCm support for sglang.check_env (#2426) · 5ce9daea
  Hui Liu authored Dec 17, 2024
  
  5ce9daea
- Clean up GPU memory after killing sglang processes (#2457) · ce094a5d
  Ata Fatahi authored Dec 17, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  ce094a5d
- benchmark decoding attention kernel with cudnn (#2467) · e2102669
  bjmsong authored Dec 17, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  e2102669
- Small fix for the order of apply_torchao_config (#2495) · bd619616
  Lianmin Zheng authored Dec 16, 2024
  
  bd619616
- Add a benchmark script for in-batch prefix caching (#2494) · 56198b45
  Lianmin Zheng authored Dec 16, 2024
  
  56198b45