Commits · 8bee20f80b47f1b79eb87e3d53b117d84d4ff948 · zhaoyu6 / sglang

20 Oct, 2024 1 commit
- Update vllm to 0.6.3 (#1711) (#1720) · 8bee20f8
  Yineng Zhang authored Oct 19, 2024
```
Co-authored-by: Ke Bao <ISPObaoke@163.com>
```
  8bee20f8
19 Oct, 2024 2 commits
- Simplify the nan detection and greedy check in sampler (#1709) · f0f8a769
  Lianmin Zheng authored Oct 18, 2024
  
  f0f8a769
- Skip unnecessary penalizer (#1707) · 2bcfba1b
  Lianmin Zheng authored Oct 18, 2024
  
  2bcfba1b
18 Oct, 2024 2 commits
- Add dtype for more operations (#1705) · 392f2863
  Lianmin Zheng authored Oct 18, 2024
  
  392f2863
- Simplify flashinfer utilities (#1704) · 6d0fa73e
  Lianmin Zheng authored Oct 17, 2024
  
  6d0fa73e
14 Oct, 2024 1 commit
- Support double sparsity (#1459) · 061e5463
  Shuo Yang authored Oct 14, 2024
  
  061e5463
13 Oct, 2024 1 commit
- Add an option to disable penalizer (#1651) · 9da5a60b
  Lianmin Zheng authored Oct 12, 2024
  
  9da5a60b
12 Oct, 2024 1 commit
- [Feature, Hardware] Enable SGLang on XPU GPUs via PyTorch (#1480) · 5d638c92
  Zhang, Liangang authored Oct 13, 2024
  
  5d638c92
11 Oct, 2024 2 commits
- Add back data parallelism (#1635) · 23cc66f7
  Lianmin Zheng authored Oct 11, 2024
  
  23cc66f7
- Add device support (#1607) · 8275049c
  Zhang, Liangang authored Oct 11, 2024
  
  8275049c
09 Oct, 2024 1 commit
- [Minor] Fix logging typo (#1615) · c996e8cc
  Amos You authored Oct 08, 2024
  
  c996e8cc
04 Oct, 2024 2 commits
- Make input_ids a torch.Tensor (#1568) · 45473d4b
  Lianmin Zheng authored Oct 04, 2024
  
  45473d4b
- Organize sampling batch info better (#1562) · 32eb6e96
  Lianmin Zheng authored Oct 03, 2024
  
  32eb6e96
03 Oct, 2024 1 commit
- Move status check in the memory pool to CPU (#1557) · 4ae0969c
  Lianmin Zheng authored Oct 02, 2024
  
  4ae0969c
01 Oct, 2024 1 commit
- Simplify flashinfer dispatch (#1552) · 100f5b8b
  Liangsheng Yin authored Oct 01, 2024
  
  100f5b8b
30 Sep, 2024 5 commits
- Organize Attention Backends (#1547) · 99ec439d
  Liangsheng Yin authored Sep 30, 2024
  
  99ec439d
- Clean up batch data structures: Introducing ModelWorkerBatch (#1544) · 63ba2f8d
  Lianmin Zheng authored Sep 30, 2024
  
  63ba2f8d
- Rename InputMetadata -> ForwardBatch (#1543) · 36d5acfc
  Lianmin Zheng authored Sep 30, 2024
  
  36d5acfc
- Let ModelRunner take InputMetadata as input, instead of ScheduleBatch (#1541) · 3f0fe08d
  Lianmin Zheng authored Sep 29, 2024
  
  3f0fe08d
- Move scheduler code from tp_worker.py to scheduler.py (#1538) · f86c1e61
  Lianmin Zheng authored Sep 29, 2024
  
  f86c1e61
29 Sep, 2024 2 commits
- Improve process creation (#1534) · 04868543
  Lianmin Zheng authored Sep 29, 2024
  
  04868543
- Organize image inputs (#1531) · fd9ad817
  Liangsheng Yin authored Sep 28, 2024
  
  fd9ad817
25 Sep, 2024 1 commit
- [Fix] Ignore import error (#1513) · 9ae1db0b
  Lianmin Zheng authored Sep 25, 2024
  
  9ae1db0b
24 Sep, 2024 1 commit
- MoE torch compile (#1497) · 8d4ed42a
  Ke Bao authored Sep 24, 2024
  
  8d4ed42a
23 Sep, 2024 1 commit
- Fix the overhead due to penalizer in bench_latency (#1496) · 2854a5ea
  Lianmin Zheng authored Sep 23, 2024
  
  2854a5ea
22 Sep, 2024 1 commit
- Update dockerfile to include datamodel_code_generator (#1492) · 39bb49d1
  Lianmin Zheng authored Sep 22, 2024
  
  39bb49d1
19 Sep, 2024 1 commit
- Fix padding in the cuda graph (#1469) · 2d346a57
  Lianmin Zheng authored Sep 19, 2024
  
  2d346a57
18 Sep, 2024 1 commit
- Fuse top_k and top_k in the sampler (#1457) · 7f24ea95
  Lianmin Zheng authored Sep 18, 2024
  
  7f24ea95
17 Sep, 2024 5 commits
- Fix attention backend (#1448) · b3710d2c
  Ke Bao authored Sep 17, 2024
  
  b3710d2c
- Enable MLA by default (#1447) · c6b6d2e7
  Ke Bao authored Sep 17, 2024
  
  c6b6d2e7
- Fix torch compile for deepseek-v2 (#1442) · 76524b70
  Ke Bao authored Sep 17, 2024
  
  76524b70
- Simplify sampler and its error handling (#1441) · 2fa5cec7
  Lianmin Zheng authored Sep 16, 2024
  
  2fa5cec7
- Clean up model loader (#1440) · 27b557ae
  Lianmin Zheng authored Sep 16, 2024
  
  27b557ae
15 Sep, 2024 1 commit
- [Fix] Fix logprob and normalized_logprob (#1428) · 9ba1f097
  Lianmin Zheng authored Sep 15, 2024
  
  9ba1f097
14 Sep, 2024 1 commit
- Optimize conflicts between CUDA graph and vocab mask tensors (#1392) · 70b68029
  Liangsheng Yin authored Sep 13, 2024
  
  70b68029
12 Sep, 2024 2 commits
- [Feature] Initial support for multi-LoRA serving (#1307) · 71221692
  Ying Sheng authored Sep 12, 2024
  
  71221692
- Support cuda graph in the triton attention backend (#1401) · 3efa7981
  Lianmin Zheng authored Sep 12, 2024
  
  3efa7981
11 Sep, 2024 3 commits
- Refactor attention backend (#1381) · fec185ce
  Lianmin Zheng authored Sep 11, 2024
  
  fec185ce
- Organize flashinfer indices update (#1378) · 144bc70f
  Liangsheng Yin authored Sep 10, 2024
  
  144bc70f
- Deprecate --disable-flashinfer and introduce --attention-backend (#1380) · 46094e0c
  Lianmin Zheng authored Sep 10, 2024
  
  46094e0c