Commits · 70b6802982198a739b233a1c72a8fa9871aabec8 · zhaoyu6 / sglang

"examples/modelfiles/recipemaker" did not exist on "1f45f7bb524d098c5636d410ae42e938863c99f8"

14 Sep, 2024 1 commit
- Optimize conflicts between CUDA graph and vocab mask tensors (#1392) · 70b68029
  Liangsheng Yin authored Sep 13, 2024
  
  70b68029
13 Sep, 2024 6 commits
- ci: fix finish (#1414) · f3d32f88
  Yineng Zhang authored Sep 14, 2024
  
  f3d32f88
- Update pr-test.yml (#1412) · 8779da95
  Lianmin Zheng authored Sep 13, 2024
  
  8779da95
- Balance test in CI (#1411) · ad0ff62a
  Lianmin Zheng authored Sep 12, 2024
  
  ad0ff62a
- [Minor] Raise exception for wrong import (#1409) · 9a903a87
  Ying Sheng authored Sep 12, 2024
  
  9a903a87
- [CI] Include triton backend and online serving benchmark into CI (#1408) · 68be2f6d
  Lianmin Zheng authored Sep 12, 2024
  
  68be2f6d
- Make stop reason a dict instead of str (#1407) · b912de11
  Lianmin Zheng authored Sep 12, 2024
  
  b912de11
12 Sep, 2024 7 commits
- [Minor, CI] remove lora test from minimal suite (#1406) · eb02c161
  Ying Sheng authored Sep 12, 2024
  
  eb02c161
- [Feature] Initial support for multi-LoRA serving (#1307) · 71221692
  Ying Sheng authored Sep 12, 2024
  
  71221692
- Add Support for XVERSE Models (Dense and MoE) to sglang (#1397) · c33d82a2
  hxer7963 authored Sep 12, 2024
```
Co-authored-by: will he <hexin@xverse.cn>
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  c33d82a2
- [Minor Fix] Fix llava modalities issue for single-image (#1402) · 8234e663
  Kaichen Zhang - NTU authored Sep 12, 2024
  
  8234e663
- kernel: use tensor cores for flashinfer gqa kernels (#1403) · debbdb51
  Zihao Ye authored Sep 12, 2024
  
  debbdb51
- Support cuda graph in the triton attention backend (#1401) · 3efa7981
  Lianmin Zheng authored Sep 12, 2024
  
  3efa7981
- Fix README format (#1399) · 2a71be5e
  William authored Sep 12, 2024
  
  2a71be5e
11 Sep, 2024 8 commits
- Add no commit to main rule (#1393) · 44621377
  Liangsheng Yin authored Sep 11, 2024
  
  44621377
- Refactor attention backend (#1381) · fec185ce
  Lianmin Zheng authored Sep 11, 2024
  
  fec185ce
- Improve error reporting during server launch (#1390) · c03cece4
  Lianmin Zheng authored Sep 11, 2024
  
  c03cece4
- [Fix] Fix --disable-flashinfer (#1389) · 15c75e41
  Lianmin Zheng authored Sep 11, 2024
  
  15c75e41
- BaiChuan2 Model (#1367) · 224200e3
  Vectory authored Sep 11, 2024
```
Co-authored-by: wanpenghan <wanpenghan@sohu-inc.com>
```
  224200e3
- remove assertion in triton attention and add an unit test (#1385) · 8c0efa51
  Byron Hsu authored Sep 11, 2024
  
  8c0efa51
- Organize flashinfer indices update (#1378) · 144bc70f
  Liangsheng Yin authored Sep 10, 2024
  
  144bc70f
- Deprecate --disable-flashinfer and introduce --attention-backend (#1380) · 46094e0c
  Lianmin Zheng authored Sep 10, 2024
  
  46094e0c
10 Sep, 2024 9 commits
- [Minor] move triton attention kernels into a separate folder (#1379) · 3a6e8b6d
  Lianmin Zheng authored Sep 10, 2024
  
  3a6e8b6d
- Fix vocab mask update bug (#1376) · fbb4754c
  Liangsheng Yin authored Sep 10, 2024
  
  fbb4754c
- [Minor] improve kill scripts and torchao import (#1375) · 6c7cb903
  Lianmin Zheng authored Sep 10, 2024
  
  6c7cb903
- Fix CORS compatibility with OpenAI, vLLM, TGI, LMDeploy (#1373) · dff2860a
  josephrocca authored Sep 11, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  dff2860a
- Support MiniCPM3 (#1371) · e72275cf
  William authored Sep 10, 2024
  
  e72275cf
- [Fix] fix bug of `undefined is_single` in meth `create_abort_task` (#1370) · fec2d122
  wangchao authored Sep 10, 2024
  
  fec2d122
- [Docs] Improve documentations (#1368) · 8d1095db
  Lianmin Zheng authored Sep 09, 2024
  
  8d1095db
- Adding Documentation for installation (#1300) · 743007e1
  Chayenne authored Sep 10, 2024
```
Co-authored-by: zhaochen20 <zhaochenyang20@gmail.com>
```
  743007e1
- Support OpenAI API json_schema response format (#1363) · 9144ed10
  zifeitong authored Sep 09, 2024
  
  9144ed10
09 Sep, 2024 7 commits
- Unify forward mode (#1360) · 69b3bb9a
  Liangsheng Yin authored Sep 09, 2024
  
  69b3bb9a
- [CI] Return output logprobs in unit test (#1361) · 689ff588
  Ying Sheng authored Sep 09, 2024
  
  689ff588
- Add torchao quant (int4/int8/fp8) to llama models (#1341) · a7c47e0f
  Jerry Zhang authored Sep 09, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  a7c47e0f
- [Minor] Many cleanup (#1357) · e4d68afc
  Lianmin Zheng authored Sep 09, 2024
  
  e4d68afc
- [server] Passing `model_override_args` to `launch_server` via the CLI. (#1298) · c9b75917
  Kai-Hsun Chen authored Sep 09, 2024
```
Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com>
```
  c9b75917
- [Feat] Add modalities for vision server when handling pixel values for llava (#1346) · 662ecd93
  Kaichen Zhang - NTU authored Sep 09, 2024
  
  662ecd93
- [triton] Support head_dim not 2^n in triton extend and decode attention (#1281) · 8e6bdf85
  Byron Hsu authored Sep 09, 2024
  
  8e6bdf85
08 Sep, 2024 1 commit
- Fix some online scheduling delay (#1345) · 05bea688
  Liangsheng Yin authored Sep 07, 2024
  
  05bea688
05 Sep, 2024 1 commit
- Optimize schedule (#1339) · ab4a83b2
  Liangsheng Yin authored Sep 05, 2024
  
  ab4a83b2