Commits · f67723940d92dc7e78b79897bbefc42c3bec6a80 · change / sglang · GitLab

11 Dec, 2024 5 commits
- decoding attention kernel benchmark (#2425) · f6772394
  bjmsong authored Dec 11, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  f6772394
- chore: update ao v0.7.0 (#2447) · 626a99ac
  Yineng Zhang authored Dec 11, 2024
  
  626a99ac
- Make torch TP composable with torchao (#2436) · ece72491
  Ke Wen authored Dec 11, 2024
  
  ece72491
- [router] Use borrow if possible to save cost (#2441) · 0fb88aaa
  Byron Hsu authored Dec 11, 2024
  
  0fb88aaa
- [router] Refactor: decouple select and send stage (#2440) · d4de9a62
  Byron Hsu authored Dec 11, 2024
  
  d4de9a62
10 Dec, 2024 4 commits
- fix: compatible with PEP 440 (#2435) · 7310aede
  Yineng Zhang authored Dec 11, 2024
  
  7310aede
- fix: use manylinux2014_x86_64 tag (#2434) · 5de9a58e
  Yineng Zhang authored Dec 11, 2024
  
  5de9a58e
- feat: support sgl-kernel PyPI (#2433) · 56fcd8e8
  Yineng Zhang authored Dec 11, 2024
```
Co-authored-by: Zhangyi <1109276519@qq.com>
```
  56fcd8e8
- Typo fix in router.md (#2424) · 2b340adf
  Adarsh Shirawalmath authored Dec 10, 2024
  
  2b340adf
09 Dec, 2024 8 commits
- [feat] Enable chunked prefill for llava-onevision (#2412) · 8586b72d
  Ying Sheng authored Dec 09, 2024
  
  8586b72d
- [Minor] Improve code style (#2422) · 641b7d0a
  Lianmin Zheng authored Dec 09, 2024
  
  641b7d0a
- [Minor] Improve code style (#2419) · 0ce091a8
  Lianmin Zheng authored Dec 09, 2024
  
  0ce091a8
- Migrate llama_classification to use the /classify interface (#2417) · 835f8afc
  Lianmin Zheng authored Dec 08, 2024
  
  835f8afc
- Add a unittest for fused_moe (#2416) · 3844feb9
  Xiaoyu Zhang authored Dec 09, 2024
  
  3844feb9
- reduce watchdog interval to 5s (#2410) · 27f7bed7
  Byron Hsu authored Dec 08, 2024
  
  27f7bed7
- [router] add health checking in router init (#2393) · 6387098f
  Byron Hsu authored Dec 08, 2024
  
  6387098f
- [Router] fix interrupt from terminal (#2413) · 2a717c50
  Byron Hsu authored Dec 08, 2024
  
  2a717c50
08 Dec, 2024 14 commits
- [router] Improve cleanup logic (#2411) · a1e697b2
  Byron Hsu authored Dec 08, 2024
  
  a1e697b2
- Simplify stream_output (#2398) · a6ca736c
  Lianmin Zheng authored Dec 08, 2024
  
  a6ca736c
- minor: add random flashinfer vs triton use case (#2409) · f62055b5
  Yineng Zhang authored Dec 09, 2024
  
  f62055b5
- minor: add random use case (#2408) · 74bc9184
  Yineng Zhang authored Dec 09, 2024
  
  74bc9184
- feat: support custom task runner (#2407) · 0f8eb153
  Yineng Zhang authored Dec 09, 2024
  
  0f8eb153
- minor: update correct measurement unit (#2406) · 67470bbb
  Yineng Zhang authored Dec 08, 2024
  
  67470bbb
- Fix recv_requests (#2405) · cc858953
  Lianmin Zheng authored Dec 08, 2024
  
  cc858953
- fix: specify dtype with begin_forward aka plan (#2404) · 6128f7cf
  Yineng Zhang authored Dec 08, 2024
  
  6128f7cf
- Fix a bug with logprob streaming + chunked prefill (#2403) · a2486eb5
  Lianmin Zheng authored Dec 08, 2024
  
  a2486eb5
- Remove unused vars in the triton backend (#2401) · 61dec545
  Ke Bao authored Dec 08, 2024
  
  61dec545
- Update killall_sglang.sh (#2397) · 96db0f66
  Lianmin Zheng authored Dec 08, 2024
  
  96db0f66
- Optimize Triton decoding kernel for long context (#2394) · 7dc66fcb
  Ke Bao authored Dec 08, 2024
  
  7dc66fcb
- nit: Remove busy waiting on scheduler (#2382) · 1f09e84b
  SangBin Cho authored Dec 08, 2024
  
  1f09e84b
- Fix shape error that occurred when loading lora weight of gemma2 model. (#2330) · 63dfab1b
  Sangchun Ha (Patrick) authored Dec 08, 2024
  
  63dfab1b
07 Dec, 2024 6 commits
- [router] Health check on worker before adding to the router (#2392) · ef995dae
  Byron Hsu authored Dec 07, 2024
  
  ef995dae
- minor: update killall script (#2391) · 75ae9689
  Yineng Zhang authored Dec 08, 2024
  
  75ae9689
- Fp8 MoE optimizations on AMD (#2388) · 95f93f49
  HAI authored Dec 07, 2024
  
  95f93f49
- fix: update xgrammar v0.1.6 (#2390) · aaac33fd
  Yineng Zhang authored Dec 07, 2024
  
  aaac33fd
- fix: resolve fp8 moe issue (#2387) · d332aa3b
  Yineng Zhang authored Dec 07, 2024
  
  d332aa3b
- [router] Add remove worker api (#2380) · c36736c8
  Byron Hsu authored Dec 06, 2024
  
  c36736c8
06 Dec, 2024 3 commits
- [router] add remove tenant method in the radix tree (#2379) · 1bf9e347
  Byron Hsu authored Dec 06, 2024
  
  1bf9e347
- [Router] remove duplicate char count (#2378) · 499c85f1
  Byron Hsu authored Dec 06, 2024
  
  499c85f1
- Release v0.4.0.post1 (#2375) · e5f227c0
  Lianmin Zheng authored Dec 06, 2024
  
  e5f227c0