Commits · 689ff588eca5b6d401b6bfd736cf98cd2b776144 · change / sglang

09 Sep, 2024 5 commits
- [CI] Return output logprobs in unit test (#1361) · 689ff588
  Ying Sheng authored Sep 09, 2024
  
  689ff588
- Add torchao quant (int4/int8/fp8) to llama models (#1341) · a7c47e0f
  Jerry Zhang authored Sep 09, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  a7c47e0f
- [Minor] Many cleanup (#1357) · e4d68afc
  Lianmin Zheng authored Sep 09, 2024
  
  e4d68afc
- [server] Passing `model_override_args` to `launch_server` via the CLI. (#1298) · c9b75917
  Kai-Hsun Chen authored Sep 09, 2024
```
Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com>
```
  c9b75917
- [Feat] Add modalities for vision server when handling pixel values for llava (#1346) · 662ecd93
  Kaichen Zhang - NTU authored Sep 09, 2024
  
  662ecd93
04 Sep, 2024 1 commit
- Fix the flaky test test_moe_eval_accuracy_large.py (#1326) · 843e63d8
  Lianmin Zheng authored Sep 04, 2024
  
  843e63d8
03 Sep, 2024 1 commit
- [Fix] Fix select by ensuring each request has at least one token (#1318) · 1e495e08
  Lianmin Zheng authored Sep 03, 2024
  
  1e495e08
02 Sep, 2024 1 commit
- feat: update nightly gsm8k eval (#1304) · 2561ed01
  Yineng Zhang authored Sep 03, 2024
  
  2561ed01
01 Sep, 2024 4 commits
- Fix the flaky tests in test_moe_eval_accuracy_large.py (#1293) · 58fa6076
  Lianmin Zheng authored Sep 01, 2024
  
  58fa6076
- [CI] merge all ci tests into one file (#1289) · 761b2ceb
  Lianmin Zheng authored Sep 01, 2024
  
  761b2ceb
- [CI] Add more multi-gpu tests (#1280) · 1b5d56f7
  Lianmin Zheng authored Sep 01, 2024
  
  1b5d56f7
- Optimize the update flashinfer indices (#1262) · d134c139
  xiaobochen authored Aug 31, 2024
  
  d134c139
30 Aug, 2024 1 commit
- Allow more flexible assistant and system response (#1256) · 51c554d8
  Christopher Chou authored Aug 30, 2024
  
  51c554d8
28 Aug, 2024 3 commits
- feat: replace GeluAndMul (#1234) · c411f32e
  Yineng Zhang authored Aug 29, 2024
  
  c411f32e
- [Fix] Fix llava on multi images (#1247) · bf53bf51
  Lianmin Zheng authored Aug 28, 2024
  
  bf53bf51
- fix: increase max_new_tokens when testing generation models (#1244) · 66975360
  Yineng Zhang authored Aug 28, 2024
  
  66975360
27 Aug, 2024 1 commit
- [Minor] add delete test and delete tmp file on ci server (#1227) · 5ff25cdf
  yichuan~ authored Aug 26, 2024
  
  5ff25cdf
26 Aug, 2024 7 commits
- [FEAT] Support batches cancel (#1222) · 2f1d9283
  caiyueliang authored Aug 27, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  2f1d9283
- Torch compile CI throughput test (#1223) · c61a1b6f
  Liangsheng Yin authored Aug 26, 2024
  
  c61a1b6f
- [FEAT] JSON constrained support (#1125) · 9935f97b
  havetc authored Aug 26, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  9935f97b
- [CI] Parallelize unit tests in CI (#1219) · 97589a60
  Mingyi authored Aug 25, 2024
  
  97589a60
- [Fix] Multi-images loading error (#1218) · 3579162a
  Kaichen Zhang - NTU authored Aug 26, 2024
  
  3579162a
- [CI] Fix CI (#1217) · 7514b9f8
  Mingyi authored Aug 25, 2024
  
  7514b9f8
- improve the threshold and ports in tests (#1215) · 158e8f1e
  Mingyi authored Aug 25, 2024
  
  158e8f1e
25 Aug, 2024 7 commits
- Update CI workflows (#1210) · 15f1a49d
  Lianmin Zheng authored Aug 25, 2024
  
  15f1a49d
- [CI] Fix the issue of unit test hanging (#1211) · 308d0240
  Ying Sheng authored Aug 25, 2024
  
  308d0240
- [Minor] Temporarily skip flaky test (#1209) · ab4990e4
  Ying Sheng authored Aug 25, 2024
  
  ab4990e4
- Support Alibaba-NLP/gte-Qwen2-7B-instruct embedding Model (#1186) · 30b4f771
  Chayenne authored Aug 26, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  30b4f771
- [Fix] Fixing the multi-images error for llava-onevision (#1205) · 66e7dcaf
  Kaichen Zhang - NTU authored Aug 26, 2024
  
  66e7dcaf
- Relax the assert in moe throughput test to fix the flaky CI (#1207) · bc4c7a35
  Lianmin Zheng authored Aug 25, 2024
  
  bc4c7a35
- [Fix] the issue of random order when input is a list (#1199) · 1cb4da5c
  Ying Sheng authored Aug 24, 2024
  
  1cb4da5c
24 Aug, 2024 1 commit
- Cleanup readme, llava examples, usage examples and nccl init (#1194) · f6af3a65
  Lianmin Zheng authored Aug 24, 2024
  
  f6af3a65
23 Aug, 2024 1 commit

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2... · a5b14ad0

Kaichen Zhang - NTU authored Aug 24, 2024


[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)
Co-authored-by: Bo Li <drluodian@gmail.com>

a5b14ad0

20 Aug, 2024 2 commits
- [Feat] Support update weights without restart server (#1157) · cd10654e
  Shan Yu authored Aug 20, 2024
  
  cd10654e
- feat: allow streaming for multi-prompt and/or parallel sampling (#1134) · d8476818
  Juwan Yoo authored Aug 20, 2024
  
  d8476818
19 Aug, 2024 1 commit

[Feat]Add support for optional start len of logprobs (#1035) · b997a18d

yichuan~ authored Aug 18, 2024


Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>

b997a18d

17 Aug, 2024 1 commit
- Fix CI accuracy && time out limit (#1133) · 5d0d40d0
  Liangsheng Yin authored Aug 16, 2024
  
  5d0d40d0
16 Aug, 2024 1 commit
- Mixed style of chunked prefill (#1013) · 3694f8f9
  Liangsheng Yin authored Aug 16, 2024
  
  3694f8f9
15 Aug, 2024 2 commits
- Enable chunked prefill by default (#1040) · e86b1ccb
  Lianmin Zheng authored Aug 14, 2024
  
  e86b1ccb
- Support `stop_token_ids` in sglang API (#1092) · 73cf6834
  Liangsheng Yin authored Aug 14, 2024
  
  73cf6834