Commits · 2d60a5ee75a8506bf7ee97090afc3f0d1baf88d4 · change / sglang

"scripts/stat_utils/cal_flops.py" did not exist on "0d4db43f32cb3c472da832ad8586517f670235e2"

11 Dec, 2024 14 commits
- Update v0.1.0.md · 2d60a5ee
  Byron Hsu authored Dec 11, 2024
  
  2d60a5ee
- [router] Release router 0.1.0 with dynamic scaling and fault tolerance (#2455) · 2e4a5907
  Byron Hsu authored Dec 11, 2024
  
  2e4a5907
- [router] Update doc for dynamic scaling and fault tolerance (#2454) · c0ee46fe
  Byron Hsu authored Dec 11, 2024
  
  c0ee46fe
- [Core] in batch prefix caching by delay scheduling (#2442) · 9208618b
  SangBin Cho authored Dec 11, 2024
  
  9208618b
- [router] remove main.rs because only lib.rs is used for py binding (#2453) · 864bf2ba
  Byron Hsu authored Dec 11, 2024
  
  864bf2ba
- [router] Add retries based fault tolerance (#2452) · a4cca7fc
  Byron Hsu authored Dec 11, 2024
  
  a4cca7fc
- Add support for IBM Granite 3.x models (#2437) · 993956c6
  Fred Reiss authored Dec 11, 2024
  
  993956c6
- Fix warmup in bench_offline_throughput.py (#2449) · f8548295
  Lianmin Zheng authored Dec 11, 2024
  
  f8548295
- Fix model loader for more quantization formats (#2448) · 959735fc
  Lianmin Zheng authored Dec 11, 2024
  
  959735fc
- decoding attention kernel benchmark (#2425) · f6772394
  bjmsong authored Dec 11, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  f6772394
- chore: update ao v0.7.0 (#2447) · 626a99ac
  Yineng Zhang authored Dec 11, 2024
  
  626a99ac
- Make torch TP composable with torchao (#2436) · ece72491
  Ke Wen authored Dec 11, 2024
  
  ece72491
- [router] Use borrow if possible to save cost (#2441) · 0fb88aaa
  Byron Hsu authored Dec 11, 2024
  
  0fb88aaa
- [router] Refactor: decouple select and send stage (#2440) · d4de9a62
  Byron Hsu authored Dec 11, 2024
  
  d4de9a62
10 Dec, 2024 4 commits
- fix: compatible with PEP 440 (#2435) · 7310aede
  Yineng Zhang authored Dec 11, 2024
  
  7310aede
- fix: use manylinux2014_x86_64 tag (#2434) · 5de9a58e
  Yineng Zhang authored Dec 11, 2024
  
  5de9a58e
- feat: support sgl-kernel PyPI (#2433) · 56fcd8e8
  Yineng Zhang authored Dec 11, 2024
```
Co-authored-by: Zhangyi <1109276519@qq.com>
```
  56fcd8e8
- Typo fix in router.md (#2424) · 2b340adf
  Adarsh Shirawalmath authored Dec 10, 2024
  
  2b340adf
09 Dec, 2024 8 commits
- [feat] Enable chunked prefill for llava-onevision (#2412) · 8586b72d
  Ying Sheng authored Dec 09, 2024
  
  8586b72d
- [Minor] Improve code style (#2422) · 641b7d0a
  Lianmin Zheng authored Dec 09, 2024
  
  641b7d0a
- [Minor] Improve code style (#2419) · 0ce091a8
  Lianmin Zheng authored Dec 09, 2024
  
  0ce091a8
- Migrate llama_classification to use the /classify interface (#2417) · 835f8afc
  Lianmin Zheng authored Dec 08, 2024
  
  835f8afc
- Add a unittest for fused_moe (#2416) · 3844feb9
  Xiaoyu Zhang authored Dec 09, 2024
  
  3844feb9
- reduce watchdog interval to 5s (#2410) · 27f7bed7
  Byron Hsu authored Dec 08, 2024
  
  27f7bed7
- [router] add health checking in router init (#2393) · 6387098f
  Byron Hsu authored Dec 08, 2024
  
  6387098f
- [Router] fix interrupt from terminal (#2413) · 2a717c50
  Byron Hsu authored Dec 08, 2024
  
  2a717c50
08 Dec, 2024 14 commits
- [router] Improve cleanup logic (#2411) · a1e697b2
  Byron Hsu authored Dec 08, 2024
  
  a1e697b2
- Simplify stream_output (#2398) · a6ca736c
  Lianmin Zheng authored Dec 08, 2024
  
  a6ca736c
- minor: add random flashinfer vs triton use case (#2409) · f62055b5
  Yineng Zhang authored Dec 09, 2024
  
  f62055b5
- minor: add random use case (#2408) · 74bc9184
  Yineng Zhang authored Dec 09, 2024
  
  74bc9184
- feat: support custom task runner (#2407) · 0f8eb153
  Yineng Zhang authored Dec 09, 2024
  
  0f8eb153
- minor: update correct measurement unit (#2406) · 67470bbb
  Yineng Zhang authored Dec 08, 2024
  
  67470bbb
- Fix recv_requests (#2405) · cc858953
  Lianmin Zheng authored Dec 08, 2024
  
  cc858953
- fix: specify dtype with begin_forward aka plan (#2404) · 6128f7cf
  Yineng Zhang authored Dec 08, 2024
  
  6128f7cf
- Fix a bug with logprob streaming + chunked prefill (#2403) · a2486eb5
  Lianmin Zheng authored Dec 08, 2024
  
  a2486eb5
- Remove unused vars in the triton backend (#2401) · 61dec545
  Ke Bao authored Dec 08, 2024
  
  61dec545
- Update killall_sglang.sh (#2397) · 96db0f66
  Lianmin Zheng authored Dec 08, 2024
  
  96db0f66
- Optimize Triton decoding kernel for long context (#2394) · 7dc66fcb
  Ke Bao authored Dec 08, 2024
  
  7dc66fcb
- nit: Remove busy waiting on scheduler (#2382) · 1f09e84b
  SangBin Cho authored Dec 08, 2024
  
  1f09e84b
- Fix shape error that occurred when loading lora weight of gemma2 model. (#2330) · 63dfab1b
  Sangchun Ha (Patrick) authored Dec 08, 2024
  
  63dfab1b