Commits · 97589a60a2cf2ef75d26ca0de9a78f30e2b63c4e · change / sglang

26 Aug, 2024 8 commits
- [CI] Parallelize unit tests in CI (#1219) · 97589a60
  Mingyi authored Aug 25, 2024
  
  97589a60
- minor: improve CI and dependencies (#1212) · 632d506d
  Liangsheng Yin authored Aug 25, 2024
  
  632d506d
- [Fix] Multi-images loading error (#1218) · 3579162a
  Kaichen Zhang - NTU authored Aug 26, 2024
  
  3579162a
- [CI] Fix CI (#1217) · 7514b9f8
  Mingyi authored Aug 25, 2024
  
  7514b9f8
- improve the threshold and ports in tests (#1215) · 158e8f1e
  Mingyi authored Aug 25, 2024
  
  158e8f1e
- Update workflow files (#1214) · d3efcb39
  Lianmin Zheng authored Aug 25, 2024
  
  d3efcb39
- [Feature] Support fp8 e5m2 kv cache with flashinfer (#1204) · 2c615d12
  Ke Bao authored Aug 26, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  2c615d12
- Update CI runner docs (#1213) · 61bb223e
  Lianmin Zheng authored Aug 25, 2024
  
  61bb223e
25 Aug, 2024 9 commits
- Update CI workflows (#1210) · 15f1a49d
  Lianmin Zheng authored Aug 25, 2024
  
  15f1a49d
- [CI] Fix the issue of unit test hanging (#1211) · 308d0240
  Ying Sheng authored Aug 25, 2024
  
  308d0240
- [Minor] Temporarily skip flaky test (#1209) · ab4990e4
  Ying Sheng authored Aug 25, 2024
  
  ab4990e4
- [Minor] Improve the function organization in TokenizerManager & improve loggers (#1208) · 90227800
  Lianmin Zheng authored Aug 25, 2024
  
  90227800
- Support Alibaba-NLP/gte-Qwen2-7B-instruct embedding Model (#1186) · 30b4f771
  Chayenne authored Aug 26, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  30b4f771
- [Fix] Fixing the multi-images error for llava-onevision (#1205) · 66e7dcaf
  Kaichen Zhang - NTU authored Aug 26, 2024
  
  66e7dcaf
- Relax the assert in moe throughput test to fix the flaky CI (#1207) · bc4c7a35
  Lianmin Zheng authored Aug 25, 2024
  
  bc4c7a35
- [Fix] the issue of random order when input is a list (#1199) · 1cb4da5c
  Ying Sheng authored Aug 24, 2024
  
  1cb4da5c
- [CI] Fix the problem of hf runner too slow (#1202) · e61d13ac
  Ying Sheng authored Aug 24, 2024
  
  e61d13ac
24 Aug, 2024 3 commits
- Update README.md (#1198) · b20daf98
  Lianmin Zheng authored Aug 24, 2024
  
  b20daf98
- Cleanup readme, llava examples, usage examples and nccl init (#1194) · f6af3a65
  Lianmin Zheng authored Aug 24, 2024
  
  f6af3a65
- feat: use gelu_tanh_and_mul (#1193) · c9064e6f
  Yineng Zhang authored Aug 24, 2024
  
  c9064e6f
23 Aug, 2024 1 commit

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2... · a5b14ad0

Kaichen Zhang - NTU authored Aug 24, 2024


[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)
Co-authored-by: Bo Li <drluodian@gmail.com>

a5b14ad0

22 Aug, 2024 3 commits
- Fix benchmark script (#1185) · 5fafcac0
  Ying Sheng authored Aug 22, 2024
  
  5fafcac0
- Fix broken penalty (#1184) · 364d3d72
  Liangsheng Yin authored Aug 22, 2024
  
  364d3d72
- [Minor] Improve logging and rename the health check endpoint name (#1180) · 5623826f
  Lianmin Zheng authored Aug 21, 2024
  
  5623826f
21 Aug, 2024 6 commits
- Improve code style of sampler (#1168) · 83e23c69
  Liangsheng Yin authored Aug 21, 2024
  
  83e23c69
- [Docs] Fix rendering of details in README (#1179) · ac1b74fa
  Zhanghao Wu authored Aug 21, 2024
  
  ac1b74fa
- Support min-p sampling (#1167) · 068e9eae
  intervitens authored Aug 22, 2024
  
  068e9eae
- [Feature] Add a function to convert sampling_params to kwargs (#1170) · d6aeb9fa
  rainred authored Aug 22, 2024
```
Co-authored-by: lzhang <zhanglei@modelbest.cn>
```
  d6aeb9fa
- fix: custom op fallback forward native when lower sm80 (#1177) · 1fb94599
  Yineng Zhang authored Aug 22, 2024
  
  1fb94599
- Improve multi-node stability (#1171) · bea2bb9e
  Lianmin Zheng authored Aug 20, 2024
  
  bea2bb9e
20 Aug, 2024 7 commits
- [Feat] Support update weights without restart server (#1157) · cd10654e
  Shan Yu authored Aug 20, 2024
  
  cd10654e
- fix: resolve README render (#1166) · 350a8160
  Yineng Zhang authored Aug 21, 2024
  
  350a8160
- Generate 1 token to verify the health of the inference service in /health (#1154) · 6242c399
  Lucien authored Aug 21, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  6242c399
- misc: add hypervisor vendor (#1165) · 04707b09
  Yineng Zhang authored Aug 21, 2024
  
  04707b09
- [Feature] add disable-custom-all-reduce (#1148) · ff2cfdb1
  Xu-Chen authored Aug 20, 2024
```
Co-authored-by: chenxu02 <chenxu02@zhihu.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  ff2cfdb1
- Improve docs and warnings (#1164) · a8ae6403
  Lianmin Zheng authored Aug 20, 2024
  
  a8ae6403
- feat: allow streaming for multi-prompt and/or parallel sampling (#1134) · d8476818
  Juwan Yoo authored Aug 20, 2024
  
  d8476818
19 Aug, 2024 3 commits
- Optimize MLA/GQA/MQA Triton decoding (#1138) · df191254
  Ke Bao authored Aug 19, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  df191254
- [Feat]Add support for optional start len of logprobs (#1035) · b997a18d
  yichuan~ authored Aug 18, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
```
  b997a18d
- [Docs] Add instruction for running on clouds and kubernetes with SkyPilot (#1144) · d8627ed1
  Zhanghao Wu authored Aug 18, 2024
```
Co-authored-by: Zongheng Yang <zongheng.y@gmail.com>
```
  d8627ed1