Commits · ce094a5d79aa3794f0cdb86ebf03e4897764e1bd · change / sglang · GitLab

17 Dec, 2024 4 commits
- Clean up GPU memory after killing sglang processes (#2457) · ce094a5d
  Ata Fatahi authored Dec 17, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  ce094a5d
- benchmark decoding attention kernel with cudnn (#2467) · e2102669
  bjmsong authored Dec 17, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  e2102669
- Small fix for the order of apply_torchao_config (#2495) · bd619616
  Lianmin Zheng authored Dec 16, 2024
  
  bd619616
- Add a benchmark script for in-batch prefix caching (#2494) · 56198b45
  Lianmin Zheng authored Dec 16, 2024
  
  56198b45
16 Dec, 2024 6 commits
- Revert "Small fixes for torchao quant" (#2493) · ba36b552
  Lianmin Zheng authored Dec 16, 2024
  
  ba36b552
- Temporarily disable unit test of torch native attention backend (#2492) · 9cd9dc83
  Lianmin Zheng authored Dec 16, 2024
  
  9cd9dc83
- Simplify pytorch sampling kernel and logit processor (#2491) · 7a1aecb9
  Lianmin Zheng authored Dec 16, 2024
  
  7a1aecb9
- Small fixes for torchao quant (#2476) · 82699474
  Jerry Zhang authored Dec 16, 2024
  
  82699474
- minor: update flashinfer nightly (#2490) · 7154b4b1
  Yineng Zhang authored Dec 16, 2024
  
  7154b4b1
- fix moe-ep accuracy issue for fp8 (#2489) · b532a5fd
  xiaobochen authored Dec 16, 2024
  
  b532a5fd
15 Dec, 2024 3 commits
- [Benchmark] add a benchmark for hf/vllm/sglang rmsnorm (#2486) · a0592c05
  Xiaoyu Zhang authored Dec 15, 2024
  
  a0592c05
- fix typo (#2487) · e8dbdf75
  Yineng Zhang authored Dec 15, 2024
  
  e8dbdf75
- adapt tensorrt llm custom all reduce to sgl-kernel (#2481) · e04d3f28
  yizhang2077 authored Dec 15, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  e04d3f28
14 Dec, 2024 4 commits
- hotfix: checking for HIP (#2485) · 5f2595be
  Yineng Zhang authored Dec 15, 2024
  
  5f2595be
- Remove cuda graph batch size adjustment for dp attention (#2484) · 0ba2c589
  Ke Bao authored Dec 14, 2024
  
  0ba2c589
- format: add clang-format for sgl-kernel (#2483) · fccbfa37
  Yineng Zhang authored Dec 14, 2024
  
  fccbfa37
- Fix correctness issue for triton decoding kernel (#2479) · 2f9bd0fa
  Ke Bao authored Dec 14, 2024
  
  2f9bd0fa
12 Dec, 2024 9 commits
- [Minor] Fix grok model loader (#2473) · 5282a473
  Lianmin Zheng authored Dec 12, 2024
  
  5282a473
- feat: support dev image (#2469) · f0ed9c35
  Yineng Zhang authored Dec 13, 2024
  
  f0ed9c35
- Rename rust folder to sgl-router (#2464) · e3b3acfa
  Ata Fatahi authored Dec 12, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  e3b3acfa
- fix: set runtime path (#2466) · 2673fa29
  Yineng Zhang authored Dec 12, 2024
  
  2673fa29
- minor: update pypi tag (#2463) · dedaf8cd
  Yineng Zhang authored Dec 12, 2024
  
  dedaf8cd
- chore: bump v0.0.2 for sgl-kernel (#2462) · 32ed0160
  Yineng Zhang authored Dec 12, 2024
  
  32ed0160
- Bump sglang-router to 0.1.1 (#2459) · 6efa9e4a
  Ata Fatahi authored Dec 11, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  6efa9e4a
- Include version info into the router package (#2456) · 7791fd99
  Ata Fatahi authored Dec 11, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  7791fd99
- Make request payload size configurable (#2444) · 2ac36b9a
  Ata Fatahi authored Dec 11, 2024
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  2ac36b9a
11 Dec, 2024 14 commits
- Update v0.1.0.md · 2d60a5ee
  Byron Hsu authored Dec 11, 2024
  
  2d60a5ee
- [router] Release router 0.1.0 with dynamic scaling and fault tolerance (#2455) · 2e4a5907
  Byron Hsu authored Dec 11, 2024
  
  2e4a5907
- [router] Update doc for dynamic scaling and fault tolerance (#2454) · c0ee46fe
  Byron Hsu authored Dec 11, 2024
  
  c0ee46fe
- [Core] in batch prefix caching by delay scheduling (#2442) · 9208618b
  SangBin Cho authored Dec 11, 2024
  
  9208618b
- [router] remove main.rs because only lib.rs is used for py binding (#2453) · 864bf2ba
  Byron Hsu authored Dec 11, 2024
  
  864bf2ba
- [router] Add retries based fault tolerance (#2452) · a4cca7fc
  Byron Hsu authored Dec 11, 2024
  
  a4cca7fc
- Add support for IBM Granite 3.x models (#2437) · 993956c6
  Fred Reiss authored Dec 11, 2024
  
  993956c6
- Fix warmup in bench_offline_throughput.py (#2449) · f8548295
  Lianmin Zheng authored Dec 11, 2024
  
  f8548295
- Fix model loader for more quantization formats (#2448) · 959735fc
  Lianmin Zheng authored Dec 11, 2024
  
  959735fc
- decoding attention kernel benchmark (#2425) · f6772394
  bjmsong authored Dec 11, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  f6772394
- chore: update ao v0.7.0 (#2447) · 626a99ac
  Yineng Zhang authored Dec 11, 2024
  
  626a99ac
- Make torch TP composable with torchao (#2436) · ece72491
  Ke Wen authored Dec 11, 2024
  
  ece72491
- [router] Use borrow if possible to save cost (#2441) · 0fb88aaa
  Byron Hsu authored Dec 11, 2024
  
  0fb88aaa
- [router] Refactor: decouple select and send stage (#2440) · d4de9a62
  Byron Hsu authored Dec 11, 2024
  
  d4de9a62