Commits · e79f6cd73d8e0fd74db29b8c441c2ead0e2fff37 · change / sglang · GitLab

15 Sep, 2024 2 commits
- Release v0.3.1 (#1430) · e79f6cd7
  Lianmin Zheng authored Sep 15, 2024
  
  e79f6cd7
- [Fix] Fix logprob and normalized_logprob (#1428) · 9ba1f097
  Lianmin Zheng authored Sep 15, 2024
  
  9ba1f097
14 Sep, 2024 3 commits
- Enable torch.compile for triton backend (#1422) · 9463bc13
  Lianmin Zheng authored Sep 14, 2024
  
  9463bc13
- Add torchao quant for mixtral and qwen_moe (#1418) · 30b404ce
  Jerry Zhang authored Sep 13, 2024
  
  30b404ce
- Optimize conflicts between CUDA graph and vocab mask tensors (#1392) · 70b68029
  Liangsheng Yin authored Sep 13, 2024
  
  70b68029
13 Sep, 2024 3 commits
- Balance test in CI (#1411) · ad0ff62a
  Lianmin Zheng authored Sep 12, 2024
  
  ad0ff62a
- [CI] Include triton backend and online serving benchmark into CI (#1408) · 68be2f6d
  Lianmin Zheng authored Sep 12, 2024
  
  68be2f6d
- Make stop reason a dict instead of str (#1407) · b912de11
  Lianmin Zheng authored Sep 12, 2024
  
  b912de11
12 Sep, 2024 5 commits
- [Feature] Initial support for multi-LoRA serving (#1307) · 71221692
  Ying Sheng authored Sep 12, 2024
  
  71221692
- Add Support for XVERSE Models (Dense and MoE) to sglang (#1397) · c33d82a2
  hxer7963 authored Sep 12, 2024
```
Co-authored-by: will he <hexin@xverse.cn>
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  c33d82a2
- [Minor Fix] Fix llava modalities issue for single-image (#1402) · 8234e663
  Kaichen Zhang - NTU authored Sep 12, 2024
  
  8234e663
- kernel: use tensor cores for flashinfer gqa kernels (#1403) · debbdb51
  Zihao Ye authored Sep 12, 2024
  
  debbdb51
- Support cuda graph in the triton attention backend (#1401) · 3efa7981
  Lianmin Zheng authored Sep 12, 2024
  
  3efa7981
11 Sep, 2024 7 commits
- Refactor attention backend (#1381) · fec185ce
  Lianmin Zheng authored Sep 11, 2024
  
  fec185ce
- Improve error reporting during server launch (#1390) · c03cece4
  Lianmin Zheng authored Sep 11, 2024
  
  c03cece4
- [Fix] Fix --disable-flashinfer (#1389) · 15c75e41
  Lianmin Zheng authored Sep 11, 2024
  
  15c75e41
- BaiChuan2 Model (#1367) · 224200e3
  Vectory authored Sep 11, 2024
```
Co-authored-by: wanpenghan <wanpenghan@sohu-inc.com>
```
  224200e3
- remove assertion in triton attention and add an unit test (#1385) · 8c0efa51
  Byron Hsu authored Sep 11, 2024
  
  8c0efa51
- Organize flashinfer indices update (#1378) · 144bc70f
  Liangsheng Yin authored Sep 10, 2024
  
  144bc70f
- Deprecate --disable-flashinfer and introduce --attention-backend (#1380) · 46094e0c
  Lianmin Zheng authored Sep 10, 2024
  
  46094e0c
10 Sep, 2024 7 commits
- [Minor] move triton attention kernels into a separate folder (#1379) · 3a6e8b6d
  Lianmin Zheng authored Sep 10, 2024
  
  3a6e8b6d
- Fix vocab mask update bug (#1376) · fbb4754c
  Liangsheng Yin authored Sep 10, 2024
  
  fbb4754c
- [Minor] improve kill scripts and torchao import (#1375) · 6c7cb903
  Lianmin Zheng authored Sep 10, 2024
  
  6c7cb903
- Fix CORS compatibility with OpenAI, vLLM, TGI, LMDeploy (#1373) · dff2860a
  josephrocca authored Sep 11, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  dff2860a
- Support MiniCPM3 (#1371) · e72275cf
  William authored Sep 10, 2024
  
  e72275cf
- [Fix] fix bug of `undefined is_single` in meth `create_abort_task` (#1370) · fec2d122
  wangchao authored Sep 10, 2024
  
  fec2d122
- Support OpenAI API json_schema response format (#1363) · 9144ed10
  zifeitong authored Sep 09, 2024
  
  9144ed10
09 Sep, 2024 7 commits
- Unify forward mode (#1360) · 69b3bb9a
  Liangsheng Yin authored Sep 09, 2024
  
  69b3bb9a
- [CI] Return output logprobs in unit test (#1361) · 689ff588
  Ying Sheng authored Sep 09, 2024
  
  689ff588
- Add torchao quant (int4/int8/fp8) to llama models (#1341) · a7c47e0f
  Jerry Zhang authored Sep 09, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  a7c47e0f
- [Minor] Many cleanup (#1357) · e4d68afc
  Lianmin Zheng authored Sep 09, 2024
  
  e4d68afc
- [server] Passing `model_override_args` to `launch_server` via the CLI. (#1298) · c9b75917
  Kai-Hsun Chen authored Sep 09, 2024
```
Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com>
```
  c9b75917
- [Feat] Add modalities for vision server when handling pixel values for llava (#1346) · 662ecd93
  Kaichen Zhang - NTU authored Sep 09, 2024
  
  662ecd93
- [triton] Support head_dim not 2^n in triton extend and decode attention (#1281) · 8e6bdf85
  Byron Hsu authored Sep 09, 2024
  
  8e6bdf85
08 Sep, 2024 1 commit
- Fix some online scheduling delay (#1345) · 05bea688
  Liangsheng Yin authored Sep 07, 2024
  
  05bea688
05 Sep, 2024 1 commit
- Optimize schedule (#1339) · ab4a83b2
  Liangsheng Yin authored Sep 05, 2024
  
  ab4a83b2
04 Sep, 2024 1 commit
- Remove useless fields in global_config.py (#1328) · eda7c090
  Lianmin Zheng authored Sep 04, 2024
  
  eda7c090
03 Sep, 2024 3 commits
- chore: bump v0.3.0 (#1320) · a63c8275
  Yineng Zhang authored Sep 04, 2024
  
  a63c8275
- misc: speedup load safetensors (#1319) · dc67d976
  Yineng Zhang authored Sep 04, 2024
```
Co-authored-by: ispobock <ISPObaoke@163.com>
```
  dc67d976
- [Fix] Fix select by ensuring each request has at least one token (#1318) · 1e495e08
  Lianmin Zheng authored Sep 03, 2024
  
  1e495e08