Commits · a5b14ad04337a3371ca2513ef95a5add28b3f34d · zhaoyu6 / sglang

"docs/vscode:/vscode.git/clone" did not exist on "82fd69c7274434678473365678d3ae2aa2ed3ae2"

23 Aug, 2024 1 commit

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2... · a5b14ad0

Kaichen Zhang - NTU authored Aug 24, 2024


[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)
Co-authored-by: Bo Li <drluodian@gmail.com>

a5b14ad0

22 Aug, 2024 3 commits
- Fix benchmark script (#1185) · 5fafcac0
  Ying Sheng authored Aug 22, 2024
  
  5fafcac0
- Fix broken penalty (#1184) · 364d3d72
  Liangsheng Yin authored Aug 22, 2024
  
  364d3d72
- [Minor] Improve logging and rename the health check endpoint name (#1180) · 5623826f
  Lianmin Zheng authored Aug 21, 2024
  
  5623826f
21 Aug, 2024 5 commits
- Improve code style of sampler (#1168) · 83e23c69
  Liangsheng Yin authored Aug 21, 2024
  
  83e23c69
- Support min-p sampling (#1167) · 068e9eae
  intervitens authored Aug 22, 2024
  
  068e9eae
- [Feature] Add a function to convert sampling_params to kwargs (#1170) · d6aeb9fa
  rainred authored Aug 22, 2024
```
Co-authored-by: lzhang <zhanglei@modelbest.cn>
```
  d6aeb9fa
- fix: custom op fallback forward native when lower sm80 (#1177) · 1fb94599
  Yineng Zhang authored Aug 22, 2024
  
  1fb94599
- Improve multi-node stability (#1171) · bea2bb9e
  Lianmin Zheng authored Aug 20, 2024
  
  bea2bb9e
20 Aug, 2024 6 commits
- [Feat] Support update weights without restart server (#1157) · cd10654e
  Shan Yu authored Aug 20, 2024
  
  cd10654e
- Generate 1 token to verify the health of the inference service in /health (#1154) · 6242c399
  Lucien authored Aug 21, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  6242c399
- misc: add hypervisor vendor (#1165) · 04707b09
  Yineng Zhang authored Aug 21, 2024
  
  04707b09
- [Feature] add disable-custom-all-reduce (#1148) · ff2cfdb1
  Xu-Chen authored Aug 20, 2024
```
Co-authored-by: chenxu02 <chenxu02@zhihu.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  ff2cfdb1
- Improve docs and warnings (#1164) · a8ae6403
  Lianmin Zheng authored Aug 20, 2024
  
  a8ae6403
- feat: allow streaming for multi-prompt and/or parallel sampling (#1134) · d8476818
  Juwan Yoo authored Aug 20, 2024
  
  d8476818
19 Aug, 2024 2 commits

Optimize MLA/GQA/MQA Triton decoding (#1138) · df191254
Ke Bao authored Aug 19, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
df191254

[Feat]Add support for optional start len of logprobs (#1035) · b997a18d

yichuan~ authored Aug 18, 2024


Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>

b997a18d

18 Aug, 2024 3 commits
- fixed a typo (#1143) · fa13b95d
  min-xu-et authored Aug 18, 2024
  
  fa13b95d
- Fix duplicated imports in hf_transformers_utils.py (#1141) · 3c1f5a92
  Lianmin Zheng authored Aug 17, 2024
  
  3c1f5a92
- Improve benchmark (#1140) · 57d0bd91
  Lianmin Zheng authored Aug 17, 2024
  
  57d0bd91
17 Aug, 2024 2 commits
- Improve the code style: more comments and remove useless packages (#1139) · cdc8d607
  Lianmin Zheng authored Aug 17, 2024
  
  cdc8d607
- fix: use fp16 dtype for sm75 (#1136) · 9208591f
  Yineng Zhang authored Aug 17, 2024
  
  9208591f
16 Aug, 2024 3 commits
- Fix port conflicts between local CI and runner CI. (#1131) · f624f6a6
  Liangsheng Yin authored Aug 16, 2024
  
  f624f6a6
- Mixed style of chunked prefill (#1013) · 3694f8f9
  Liangsheng Yin authored Aug 16, 2024
  
  3694f8f9
- Fix the deadlock in multi-node tp (#1122) · 5a261bd0
  Lianmin Zheng authored Aug 16, 2024
  
  5a261bd0
15 Aug, 2024 10 commits
- chore: bump v0.2.13 (#1111) · 5bd95374
  Yineng Zhang authored Aug 16, 2024
  
  5bd95374
- set CUDA_DEVICE_MAX_CONNECTIONS=1 (#1113) · 0cb099e2
  Lianmin Zheng authored Aug 15, 2024
  
  0cb099e2
- [Fix] Window attention compatible with RadixAttention and chunked prefill (#1112) · 93d4e354
  Ying Sheng authored Aug 15, 2024
  
  93d4e354
- misc: rm unused model_loader (#1110) · 9195d136
  Yineng Zhang authored Aug 15, 2024
  
  9195d136
- [Fix] fix flashinfer usage for window attention (#1107) · 14cb544d
  Ying Sheng authored Aug 15, 2024
  
  14cb544d
- Enable chunked prefill by default (#1040) · e86b1ccb
  Lianmin Zheng authored Aug 14, 2024
  
  e86b1ccb
- [Fix] fix the typo bug for window attention (#1106) · 8d2d876f
  Ying Sheng authored Aug 14, 2024
  
  8d2d876f
- Use a single workspace for flashinfer (#1077) · 326df4ba
  Lianmin Zheng authored Aug 14, 2024
  
  326df4ba
- Support jinja as chat template file (#1104) · 6767e222
  Ying Sheng authored Aug 14, 2024
  
  6767e222
- Support `stop_token_ids` in sglang API (#1092) · 73cf6834
  Liangsheng Yin authored Aug 14, 2024
  
  73cf6834
14 Aug, 2024 5 commits
- [Fix] Compatibility of window attention and cuda graph (#1090) · 96a2093e
  Ying Sheng authored Aug 14, 2024
  
  96a2093e
- Use `dtype` to control generate (#1082) · a34dd86a
  Liangsheng Yin authored Aug 14, 2024
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  a34dd86a
- Update grok 1 model (#1095) · a59636bb
  Lianmin Zheng authored Aug 14, 2024
  
  a59636bb
- Fix a bug in cuda graph runner (#1094) · 8f790ac1
  Lianmin Zheng authored Aug 14, 2024
  
  8f790ac1
- [Feature] modify Runtime to support skip_tokenizer_init (#1088) · 616b59f3
  rainred authored Aug 14, 2024
```
Co-authored-by: lzhang <zhanglei@modelbest.cn>
```
  616b59f3