Commits · e00e5385e0d9592a55381684292fbcb437c7c70b · change / sglang

16 Jan, 2025 1 commit
- add profiling to bench_one_batch script (#2821) · e00e5385
  Yun Dai authored Jan 16, 2025
  
  e00e5385
02 Jan, 2025 1 commit
- Eagle speculative decoding part 3: small modifications to the general scheduler (#2709) · ad20b795
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  ad20b795
23 Dec, 2024 1 commit
- Fix gemlite import (#2553) · 23e5e50f
  Lianmin Zheng authored Dec 22, 2024
  
  23e5e50f
20 Dec, 2024 1 commit
- Add integration with gemlite weight only quant (#2528) · feb2b768
  Jerry Zhang authored Dec 20, 2024
  
  feb2b768
02 Dec, 2024 1 commit
- Update model_loader deps and qqq quantization deps (#2220) (#2318) · 85e1a6f3
  Yineng Zhang authored Dec 02, 2024
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  85e1a6f3
28 Nov, 2024 1 commit
- Crash the server correctly during error (#2231) · d4fc1a70
  Lianmin Zheng authored Nov 28, 2024
  
  d4fc1a70
27 Nov, 2024 1 commit
- Release v0.3.6.post2 (#2214) · fed4c694
  Lianmin Zheng authored Nov 27, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  fed4c694
24 Nov, 2024 1 commit
- Update CI threshold & Improve code style (#2159) · 5652c565
  Lianmin Zheng authored Nov 24, 2024
  
  5652c565
23 Nov, 2024 1 commit
- Add initial support for intel Gaudi accelerators (#2121) · 865233e2
  Ankur Neog authored Nov 23, 2024
  
  865233e2
22 Nov, 2024 1 commit
- Rename sglang.bench_latency to sglang.bench_one_batch (#2118) · dfec7fca
  Lianmin Zheng authored Nov 21, 2024
  
  dfec7fca
20 Nov, 2024 1 commit
- Enable overlap by default (#2067) · 7d671e4a
  Lianmin Zheng authored Nov 19, 2024
  
  7d671e4a
16 Nov, 2024 1 commit
- Add Tensor Parallel to torch_native_llama (#1876) · cf248976
  Ke Wen authored Nov 15, 2024
  
  cf248976
03 Nov, 2024 1 commit
- Unify the model type checking (#1905) · 0abbf289
  Lianmin Zheng authored Nov 03, 2024
  
  0abbf289
27 Oct, 2024 1 commit
- Add a watch dog thread (#1816) · 86fc0d79
  Lianmin Zheng authored Oct 27, 2024
  
  86fc0d79
21 Oct, 2024 1 commit
- Llama3.2 vision model support (#1551) · 94cde109
  Liangsheng Yin authored Oct 21, 2024
  
  94cde109
14 Oct, 2024 1 commit
- Add output_ids into ScheduleBatch (#1659) · da1ffed6
  Lianmin Zheng authored Oct 13, 2024
  
  da1ffed6
12 Oct, 2024 1 commit
- [Feature, Hardware] Enable SGLang on XPU GPUs via PyTorch (#1480) · 5d638c92
  Zhang, Liangang authored Oct 13, 2024
  
  5d638c92
11 Oct, 2024 2 commits
- Add back data parallelism (#1635) · 23cc66f7
  Lianmin Zheng authored Oct 11, 2024
  
  23cc66f7
- Fix the correctness test in bench_latency.py when tp > 1 and test_generation_models.py (#1631) · aba9eae4
  Lianmin Zheng authored Oct 11, 2024
  
  aba9eae4
07 Oct, 2024 1 commit
- Fix the port_args in bench_latency (#1597) · 2b302b93
  Lianmin Zheng authored Oct 07, 2024
  
  2b302b93
06 Oct, 2024 1 commit
- [Bug] Fix decode stats error on output_len 1 (#1585) · 4d086719
  HAI authored Oct 06, 2024
  
  4d086719
05 Oct, 2024 1 commit
- Add llama implementation with no tensor parallel linears (#1561) · 9b0926ce
  Jerry Zhang authored Oct 05, 2024
  
  9b0926ce
04 Oct, 2024 1 commit
- Use ipc instead of tcp in zmq (#1566) · 114bbc86
  Lianmin Zheng authored Oct 04, 2024
  
  114bbc86
30 Sep, 2024 4 commits
- Clean up batch data structures: Introducing ModelWorkerBatch (#1544) · 63ba2f8d
  Lianmin Zheng authored Sep 30, 2024
  
  63ba2f8d
- Rename InputMetadata -> ForwardBatch (#1543) · 36d5acfc
  Lianmin Zheng authored Sep 30, 2024
  
  36d5acfc
- Let ModelRunner take InputMetadata as input, instead of ScheduleBatch (#1541) · 3f0fe08d
  Lianmin Zheng authored Sep 29, 2024
  
  3f0fe08d
- Move scheduler code from tp_worker.py to scheduler.py (#1538) · f86c1e61
  Lianmin Zheng authored Sep 29, 2024
  
  f86c1e61
25 Sep, 2024 1 commit
- Simplify bench_latency.py (#1503) · 067d8e16
  Lianmin Zheng authored Sep 24, 2024
  
  067d8e16
23 Sep, 2024 1 commit
- Fix the overhead due to penalizer in bench_latency (#1496) · 2854a5ea
  Lianmin Zheng authored Sep 23, 2024
  
  2854a5ea
19 Sep, 2024 1 commit
- Fix env vars in bench_latency (#1472) · 2cd7e181
  Lianmin Zheng authored Sep 19, 2024
  
  2cd7e181
18 Sep, 2024 1 commit
- Add bench_server_latency.py (#1452) · 5e62a6b7
  Lianmin Zheng authored Sep 18, 2024
  
  5e62a6b7
15 Sep, 2024 2 commits
- Remove deprecated configs (#1431) · 899cf5c4
  Lianmin Zheng authored Sep 15, 2024
  
  899cf5c4
- [Fix] Fix logprob and normalized_logprob (#1428) · 9ba1f097
  Lianmin Zheng authored Sep 15, 2024
  
  9ba1f097
14 Sep, 2024 2 commits
- Enable torch.compile for triton backend (#1422) · 9463bc13
  Lianmin Zheng authored Sep 14, 2024
  
  9463bc13
- Optimize conflicts between CUDA graph and vocab mask tensors (#1392) · 70b68029
  Liangsheng Yin authored Sep 13, 2024
  
  70b68029
10 Sep, 2024 1 commit
- [Minor] move triton attention kernels into a separate folder (#1379) · 3a6e8b6d
  Lianmin Zheng authored Sep 10, 2024
  
  3a6e8b6d
09 Sep, 2024 2 commits
- Unify forward mode (#1360) · 69b3bb9a
  Liangsheng Yin authored Sep 09, 2024
  
  69b3bb9a
- [server] Passing `model_override_args` to `launch_server` via the CLI. (#1298) · c9b75917
  Kai-Hsun Chen authored Sep 09, 2024
```
Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com>
```
  c9b75917
01 Sep, 2024 1 commit
- [CI] Add more multi-gpu tests (#1280) · 1b5d56f7
  Lianmin Zheng authored Sep 01, 2024
  
  1b5d56f7
30 Aug, 2024 1 commit
- Report median instead of mean in bench_latency.py (#1269) · 79ece2c5
  Lianmin Zheng authored Aug 30, 2024
  
  79ece2c5