Commits · e04d3f289753b942bc2d201988df408d01baf73c · change / sglang · GitLab

15 Dec, 2024 1 commit
- adapt tensorrt llm custom all reduce to sgl-kernel (#2481) · e04d3f28
  yizhang2077 authored Dec 15, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  e04d3f28
14 Dec, 2024 4 commits
- hotfix: checking for HIP (#2485) · 5f2595be
  Yineng Zhang authored Dec 15, 2024
  
  5f2595be
- Remove cuda graph batch size adjustment for dp attention (#2484) · 0ba2c589
  Ke Bao authored Dec 14, 2024
  
  0ba2c589
- format: add clang-format for sgl-kernel (#2483) · fccbfa37
  Yineng Zhang authored Dec 14, 2024
  
  fccbfa37
- Fix correctness issue for triton decoding kernel (#2479) · 2f9bd0fa
  Ke Bao authored Dec 14, 2024
  
  2f9bd0fa
12 Dec, 2024 9 commits
- [Minor] Fix grok model loader (#2473) · 5282a473
  Lianmin Zheng authored Dec 12, 2024
  
  5282a473
- feat: support dev image (#2469) · f0ed9c35
  Yineng Zhang authored Dec 13, 2024
  
  f0ed9c35
- Rename rust folder to sgl-router (#2464) · e3b3acfa
  Ata Fatahi authored Dec 12, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  e3b3acfa
- fix: set runtime path (#2466) · 2673fa29
  Yineng Zhang authored Dec 12, 2024
  
  2673fa29
- minor: update pypi tag (#2463) · dedaf8cd
  Yineng Zhang authored Dec 12, 2024
  
  dedaf8cd
- chore: bump v0.0.2 for sgl-kernel (#2462) · 32ed0160
  Yineng Zhang authored Dec 12, 2024
  
  32ed0160
- Bump sglang-router to 0.1.1 (#2459) · 6efa9e4a
  Ata Fatahi authored Dec 11, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  6efa9e4a
- Include version info into the router package (#2456) · 7791fd99
  Ata Fatahi authored Dec 11, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  7791fd99
- Make request payload size configurable (#2444) · 2ac36b9a
  Ata Fatahi authored Dec 11, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  2ac36b9a
11 Dec, 2024 14 commits
- Update v0.1.0.md · 2d60a5ee
  Byron Hsu authored Dec 11, 2024
  
  2d60a5ee
- [router] Release router 0.1.0 with dynamic scaling and fault tolerance (#2455) · 2e4a5907
  Byron Hsu authored Dec 11, 2024
  
  2e4a5907
- [router] Update doc for dynamic scaling and fault tolerance (#2454) · c0ee46fe
  Byron Hsu authored Dec 11, 2024
  
  c0ee46fe
- [Core] in batch prefix caching by delay scheduling (#2442) · 9208618b
  SangBin Cho authored Dec 11, 2024
  
  9208618b
- [router] remove main.rs because only lib.rs is used for py binding (#2453) · 864bf2ba
  Byron Hsu authored Dec 11, 2024
  
  864bf2ba
- [router] Add retries based fault tolerance (#2452) · a4cca7fc
  Byron Hsu authored Dec 11, 2024
  
  a4cca7fc
- Add support for IBM Granite 3.x models (#2437) · 993956c6
  Fred Reiss authored Dec 11, 2024
  
  993956c6
- Fix warmup in bench_offline_throughput.py (#2449) · f8548295
  Lianmin Zheng authored Dec 11, 2024
  
  f8548295
- Fix model loader for more quantization formats (#2448) · 959735fc
  Lianmin Zheng authored Dec 11, 2024
  
  959735fc
- decoding attention kernel benchmark (#2425) · f6772394
  bjmsong authored Dec 11, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  f6772394
- chore: update ao v0.7.0 (#2447) · 626a99ac
  Yineng Zhang authored Dec 11, 2024
  
  626a99ac
- Make torch TP composable with torchao (#2436) · ece72491
  Ke Wen authored Dec 11, 2024
  
  ece72491
- [router] Use borrow if possible to save cost (#2441) · 0fb88aaa
  Byron Hsu authored Dec 11, 2024
  
  0fb88aaa
- [router] Refactor: decouple select and send stage (#2440) · d4de9a62
  Byron Hsu authored Dec 11, 2024
  
  d4de9a62
10 Dec, 2024 4 commits
- fix: compatible with PEP 440 (#2435) · 7310aede
  Yineng Zhang authored Dec 11, 2024
  
  7310aede
- fix: use manylinux2014_x86_64 tag (#2434) · 5de9a58e
  Yineng Zhang authored Dec 11, 2024
  
  5de9a58e
- feat: support sgl-kernel PyPI (#2433) · 56fcd8e8
  Yineng Zhang authored Dec 11, 2024
```
Co-authored-by: Zhangyi <1109276519@qq.com>
```
  56fcd8e8
- Typo fix in router.md (#2424) · 2b340adf
  Adarsh Shirawalmath authored Dec 10, 2024
  
  2b340adf
09 Dec, 2024 8 commits
- [feat] Enable chunked prefill for llava-onevision (#2412) · 8586b72d
  Ying Sheng authored Dec 09, 2024
  
  8586b72d
- [Minor] Improve code style (#2422) · 641b7d0a
  Lianmin Zheng authored Dec 09, 2024
  
  641b7d0a
- [Minor] Improve code style (#2419) · 0ce091a8
  Lianmin Zheng authored Dec 09, 2024
  
  0ce091a8
- Migrate llama_classification to use the /classify interface (#2417) · 835f8afc
  Lianmin Zheng authored Dec 08, 2024
  
  835f8afc
- Add a unittest for fused_moe (#2416) · 3844feb9
  Xiaoyu Zhang authored Dec 09, 2024
  
  3844feb9
- reduce watchdog interval to 5s (#2410) · 27f7bed7
  Byron Hsu authored Dec 08, 2024
  
  27f7bed7
- [router] add health checking in router init (#2393) · 6387098f
  Byron Hsu authored Dec 08, 2024
  
  6387098f
- [Router] fix interrupt from terminal (#2413) · 2a717c50
  Byron Hsu authored Dec 08, 2024
  
  2a717c50