Commits · 8586b72da0fe8a9d32a76770eba4b83a893b5ddf · change / sglang · GitLab

09 Dec, 2024 8 commits
- [feat] Enable chunked prefill for llava-onevision (#2412) · 8586b72d
  Ying Sheng authored Dec 09, 2024
  
  8586b72d
- [Minor] Improve code style (#2422) · 641b7d0a
  Lianmin Zheng authored Dec 09, 2024
  
  641b7d0a
- [Minor] Improve code style (#2419) · 0ce091a8
  Lianmin Zheng authored Dec 09, 2024
  
  0ce091a8
- Migrate llama_classification to use the /classify interface (#2417) · 835f8afc
  Lianmin Zheng authored Dec 08, 2024
  
  835f8afc
- Add a unittest for fused_moe (#2416) · 3844feb9
  Xiaoyu Zhang authored Dec 09, 2024
  
  3844feb9
- reduce watchdog interval to 5s (#2410) · 27f7bed7
  Byron Hsu authored Dec 08, 2024
  
  27f7bed7
- [router] add health checking in router init (#2393) · 6387098f
  Byron Hsu authored Dec 08, 2024
  
  6387098f
- [Router] fix interrupt from terminal (#2413) · 2a717c50
  Byron Hsu authored Dec 08, 2024
  
  2a717c50
08 Dec, 2024 14 commits
- [router] Improve cleanup logic (#2411) · a1e697b2
  Byron Hsu authored Dec 08, 2024
  
  a1e697b2
- Simplify stream_output (#2398) · a6ca736c
  Lianmin Zheng authored Dec 08, 2024
  
  a6ca736c
- minor: add random flashinfer vs triton use case (#2409) · f62055b5
  Yineng Zhang authored Dec 09, 2024
  
  f62055b5
- minor: add random use case (#2408) · 74bc9184
  Yineng Zhang authored Dec 09, 2024
  
  74bc9184
- feat: support custom task runner (#2407) · 0f8eb153
  Yineng Zhang authored Dec 09, 2024
  
  0f8eb153
- minor: update correct measurement unit (#2406) · 67470bbb
  Yineng Zhang authored Dec 08, 2024
  
  67470bbb
- Fix recv_requests (#2405) · cc858953
  Lianmin Zheng authored Dec 08, 2024
  
  cc858953
- fix: specify dtype with begin_forward aka plan (#2404) · 6128f7cf
  Yineng Zhang authored Dec 08, 2024
  
  6128f7cf
- Fix a bug with logprob streaming + chunked prefill (#2403) · a2486eb5
  Lianmin Zheng authored Dec 08, 2024
  
  a2486eb5
- Remove unused vars in the triton backend (#2401) · 61dec545
  Ke Bao authored Dec 08, 2024
  
  61dec545
- Update killall_sglang.sh (#2397) · 96db0f66
  Lianmin Zheng authored Dec 08, 2024
  
  96db0f66
- Optimize Triton decoding kernel for long context (#2394) · 7dc66fcb
  Ke Bao authored Dec 08, 2024
  
  7dc66fcb
- nit: Remove busy waiting on scheduler (#2382) · 1f09e84b
  SangBin Cho authored Dec 08, 2024
  
  1f09e84b
- Fix shape error that occurred when loading lora weight of gemma2 model. (#2330) · 63dfab1b
  Sangchun Ha (Patrick) authored Dec 08, 2024
  
  63dfab1b
07 Dec, 2024 6 commits
- [router] Health check on worker before adding to the router (#2392) · ef995dae
  Byron Hsu authored Dec 07, 2024
  
  ef995dae
- minor: update killall script (#2391) · 75ae9689
  Yineng Zhang authored Dec 08, 2024
  
  75ae9689
- Fp8 MoE optimizations on AMD (#2388) · 95f93f49
  HAI authored Dec 07, 2024
  
  95f93f49
- fix: update xgrammar v0.1.6 (#2390) · aaac33fd
  Yineng Zhang authored Dec 07, 2024
  
  aaac33fd
- fix: resolve fp8 moe issue (#2387) · d332aa3b
  Yineng Zhang authored Dec 07, 2024
  
  d332aa3b
- [router] Add remove worker api (#2380) · c36736c8
  Byron Hsu authored Dec 06, 2024
  
  c36736c8
06 Dec, 2024 12 commits
- [router] add remove tenant method in the radix tree (#2379) · 1bf9e347
  Byron Hsu authored Dec 06, 2024
  
  1bf9e347
- [Router] remove duplicate char count (#2378) · 499c85f1
  Byron Hsu authored Dec 06, 2024
  
  499c85f1
- Release v0.4.0.post1 (#2375) · e5f227c0
  Lianmin Zheng authored Dec 06, 2024
  
  e5f227c0
- Fix the overlap for xgrammar (#2377) · 0e7409ad
  Lianmin Zheng authored Dec 06, 2024
  
  0e7409ad
- docs: Improve instructions for supporting new models (#2363) · 3cde5eb6
  vchzls authored Dec 06, 2024
```
Co-authored-by: zhaohoulong <zhaohoulong@xiaomi.com>
```
  3cde5eb6
- Use proc.join instead of busy waiting (#2374) · f5b2a3aa
  Lianmin Zheng authored Dec 06, 2024
  
  f5b2a3aa
- docs: update adoption (Meituan) (#2373) · f6817596
  Yineng Zhang authored Dec 06, 2024
  
  f6817596
- [router] support `/add_worker` api (#2369) · 67b65794
  Byron Hsu authored Dec 06, 2024
  
  67b65794
- Add more support for intel Gaudi accelerators (#2357) · 37ee906f
  Qun Yang authored Dec 06, 2024
  
  37ee906f
- optimize cuda graph max_bs_settings on low-end gpus (#2360) · 34b364e0
  Xiaoyu Zhang authored Dec 06, 2024
  
  34b364e0
- Move FP8 to SGLang (#2370) · 84d96b3a
  Yineng Zhang authored Dec 06, 2024
```
Co-authored-by: HaiShaw <hixiao@gmail.com>
```
  84d96b3a
- Resubmit MoE-EP (#2371) · 3d32e4a3
  xiaobochen authored Dec 06, 2024
  
  3d32e4a3