Commits · 2d346a57c2fb753d9353f346dff3bbc5b208bd0b · change / sglang · GitLab

19 Sep, 2024 1 commit
- Fix padding in the cuda graph (#1469) · 2d346a57
  Lianmin Zheng authored Sep 19, 2024
  
  2d346a57
18 Sep, 2024 6 commits
- Fuse top_k and top_k in the sampler (#1457) · 7f24ea95
  Lianmin Zheng authored Sep 18, 2024
  
  7f24ea95
- Fix oom issues with fp8 for llama (#1454) · 1acccb36
  Lianmin Zheng authored Sep 18, 2024
  
  1acccb36
- [Bugfix] Enable SGLang on AMD GPUs via PyTorch for ROCm (#1419) (#1453) · aa2750be
  HAI authored Sep 18, 2024
  
  aa2750be
- Add bench_server_latency.py (#1452) · 5e62a6b7
  Lianmin Zheng authored Sep 18, 2024
  
  5e62a6b7
- Fixed n>1 causing list index out of range with VLM (#1449) · 5752f25e
  Xiao Yu authored Sep 18, 2024
  
  5752f25e
- Fix schedule bug (#1451) · 7c162fa9
  Liangsheng Yin authored Sep 17, 2024
  
  7c162fa9
17 Sep, 2024 9 commits
- fix schedule bug (#1450) · 36078fb2
  Liangsheng Yin authored Sep 17, 2024
  
  36078fb2
- Fix attention backend (#1448) · b3710d2c
  Ke Bao authored Sep 17, 2024
  
  b3710d2c
- Enable MLA by default (#1447) · c6b6d2e7
  Ke Bao authored Sep 17, 2024
  
  c6b6d2e7
- Release 0.3.1.post1 (#1445) · 90a26be3
  Lianmin Zheng authored Sep 17, 2024
  
  90a26be3
- Add OLMoE model (#1444) · 1f4b5f77
  Jani Monoses authored Sep 17, 2024
  
  1f4b5f77
- Fix torch compile for deepseek-v2 (#1442) · 76524b70
  Ke Bao authored Sep 17, 2024
  
  76524b70
- [Feature, Hardware] Enable SGLang on AMD GPUs via PyTorch for ROCm (#1420) · 3a6e0418
  HAI authored Sep 17, 2024
  
  3a6e0418
- Simplify sampler and its error handling (#1441) · 2fa5cec7
  Lianmin Zheng authored Sep 16, 2024
  
  2fa5cec7
- Clean up model loader (#1440) · 27b557ae
  Lianmin Zheng authored Sep 16, 2024
  
  27b557ae
16 Sep, 2024 1 commit
- Add constrained_json_whitespace_pattern to ServerArgs (#1438) · 93dffd69
  zifeitong authored Sep 16, 2024
  
  93dffd69
15 Sep, 2024 4 commits
- [Feature] Support LoRA path renaming and add LoRA serving benchmarks (#1433) · 37963394
  Ying Sheng authored Sep 15, 2024
  
  37963394
- Remove deprecated configs (#1431) · 899cf5c4
  Lianmin Zheng authored Sep 15, 2024
  
  899cf5c4
- Release v0.3.1 (#1430) · e79f6cd7
  Lianmin Zheng authored Sep 15, 2024
  
  e79f6cd7
- [Fix] Fix logprob and normalized_logprob (#1428) · 9ba1f097
  Lianmin Zheng authored Sep 15, 2024
  
  9ba1f097
14 Sep, 2024 3 commits
- Enable torch.compile for triton backend (#1422) · 9463bc13
  Lianmin Zheng authored Sep 14, 2024
  
  9463bc13
- Add torchao quant for mixtral and qwen_moe (#1418) · 30b404ce
  Jerry Zhang authored Sep 13, 2024
  
  30b404ce
- Optimize conflicts between CUDA graph and vocab mask tensors (#1392) · 70b68029
  Liangsheng Yin authored Sep 13, 2024
  
  70b68029
13 Sep, 2024 3 commits
- Balance test in CI (#1411) · ad0ff62a
  Lianmin Zheng authored Sep 12, 2024
  
  ad0ff62a
- [CI] Include triton backend and online serving benchmark into CI (#1408) · 68be2f6d
  Lianmin Zheng authored Sep 12, 2024
  
  68be2f6d
- Make stop reason a dict instead of str (#1407) · b912de11
  Lianmin Zheng authored Sep 12, 2024
  
  b912de11
12 Sep, 2024 5 commits
- [Feature] Initial support for multi-LoRA serving (#1307) · 71221692
  Ying Sheng authored Sep 12, 2024
  
  71221692
- Add Support for XVERSE Models (Dense and MoE) to sglang (#1397) · c33d82a2
  hxer7963 authored Sep 12, 2024
```
Co-authored-by: will he <hexin@xverse.cn>
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  c33d82a2
- [Minor Fix] Fix llava modalities issue for single-image (#1402) · 8234e663
  Kaichen Zhang - NTU authored Sep 12, 2024
  
  8234e663
- kernel: use tensor cores for flashinfer gqa kernels (#1403) · debbdb51
  Zihao Ye authored Sep 12, 2024
  
  debbdb51
- Support cuda graph in the triton attention backend (#1401) · 3efa7981
  Lianmin Zheng authored Sep 12, 2024
  
  3efa7981
11 Sep, 2024 7 commits
- Refactor attention backend (#1381) · fec185ce
  Lianmin Zheng authored Sep 11, 2024
  
  fec185ce
- Improve error reporting during server launch (#1390) · c03cece4
  Lianmin Zheng authored Sep 11, 2024
  
  c03cece4
- [Fix] Fix --disable-flashinfer (#1389) · 15c75e41
  Lianmin Zheng authored Sep 11, 2024
  
  15c75e41
- BaiChuan2 Model (#1367) · 224200e3
  Vectory authored Sep 11, 2024
```
Co-authored-by: wanpenghan <wanpenghan@sohu-inc.com>
```
  224200e3
- remove assertion in triton attention and add an unit test (#1385) · 8c0efa51
  Byron Hsu authored Sep 11, 2024
  
  8c0efa51
- Organize flashinfer indices update (#1378) · 144bc70f
  Liangsheng Yin authored Sep 10, 2024
  
  144bc70f
- Deprecate --disable-flashinfer and introduce --attention-backend (#1380) · 46094e0c
  Lianmin Zheng authored Sep 10, 2024
  
  46094e0c
10 Sep, 2024 1 commit
- [Minor] move triton attention kernels into a separate folder (#1379) · 3a6e8b6d
  Lianmin Zheng authored Sep 10, 2024
  
  3a6e8b6d