Commits · 4e4459b91fae13c6d75252ffc41ecacc03950372 · change / sglang · GitLab

28 Sep, 2024 2 commits
- Multiple minor fixes (#1530) · 4e4459b9
  Lianmin Zheng authored Sep 28, 2024
  
  4e4459b9
- [Feature] Support reward model LxzGordon/URM-LLaMa-3.1-8B (#1525) · 9aa6553d
  Ying Sheng authored Sep 27, 2024
  
  9aa6553d
17 Sep, 2024 2 commits
- [Feature, Hardware] Enable SGLang on AMD GPUs via PyTorch for ROCm (#1420) · 3a6e0418
  HAI authored Sep 17, 2024
  
  3a6e0418
- Clean up model loader (#1440) · 27b557ae
  Lianmin Zheng authored Sep 16, 2024
  
  27b557ae
12 Sep, 2024 1 commit
- [Feature] Initial support for multi-LoRA serving (#1307) · 71221692
  Ying Sheng authored Sep 12, 2024
  
  71221692
11 Sep, 2024 3 commits
- Refactor attention backend (#1381) · fec185ce
  Lianmin Zheng authored Sep 11, 2024
  
  fec185ce
- Improve error reporting during server launch (#1390) · c03cece4
  Lianmin Zheng authored Sep 11, 2024
  
  c03cece4
- Deprecate --disable-flashinfer and introduce --attention-backend (#1380) · 46094e0c
  Lianmin Zheng authored Sep 10, 2024
  
  46094e0c
10 Sep, 2024 1 commit
- Fix CORS compatibility with OpenAI, vLLM, TGI, LMDeploy (#1373) · dff2860a
  josephrocca authored Sep 11, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  dff2860a
09 Sep, 2024 1 commit
- [Minor] Many cleanup (#1357) · e4d68afc
  Lianmin Zheng authored Sep 09, 2024
  
  e4d68afc
01 Sep, 2024 1 commit
- [Chore] Rename model_overide_args to model_override_args (#1284) · 08360553
  Kai-Hsun Chen authored Sep 01, 2024
```
Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  08360553
28 Aug, 2024 3 commits
- [Fix] Fix OOM in llava base class (#1249) · 0a97d796
  Lianmin Zheng authored Aug 28, 2024
  
  0a97d796
- [Fix] Fix llava on multi images (#1247) · bf53bf51
  Lianmin Zheng authored Aug 28, 2024
  
  bf53bf51
- feat: support sm75 with FlashInfer v0.1.6 (#1233) · 198974cd
  Yineng Zhang authored Aug 28, 2024
  
  198974cd
26 Aug, 2024 1 commit
- [FEAT] Support batches cancel (#1222) · 2f1d9283
  caiyueliang authored Aug 27, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  2f1d9283
25 Aug, 2024 3 commits
- [Minor] Improve the function organization in TokenizerManager & improve loggers (#1208) · 90227800
  Lianmin Zheng authored Aug 25, 2024
  
  90227800
- Support Alibaba-NLP/gte-Qwen2-7B-instruct embedding Model (#1186) · 30b4f771
  Chayenne authored Aug 26, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  30b4f771
- [Fix] the issue of random order when input is a list (#1199) · 1cb4da5c
  Ying Sheng authored Aug 24, 2024
  
  1cb4da5c
22 Aug, 2024 1 commit
- [Minor] Improve logging and rename the health check endpoint name (#1180) · 5623826f
  Lianmin Zheng authored Aug 21, 2024
  
  5623826f
21 Aug, 2024 1 commit
- Improve multi-node stability (#1171) · bea2bb9e
  Lianmin Zheng authored Aug 20, 2024
  
  bea2bb9e
20 Aug, 2024 2 commits
- [Feat] Support update weights without restart server (#1157) · cd10654e
  Shan Yu authored Aug 20, 2024
  
  cd10654e
- Generate 1 token to verify the health of the inference service in /health (#1154) · 6242c399
  Lucien authored Aug 21, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  6242c399
19 Aug, 2024 1 commit

[Feat]Add support for optional start len of logprobs (#1035) · b997a18d

yichuan~ authored Aug 18, 2024


Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>

b997a18d

17 Aug, 2024 1 commit
- Improve the code style: more comments and remove useless packages (#1139) · cdc8d607
  Lianmin Zheng authored Aug 17, 2024
  
  cdc8d607
16 Aug, 2024 1 commit
- Mixed style of chunked prefill (#1013) · 3694f8f9
  Liangsheng Yin authored Aug 16, 2024
  
  3694f8f9
15 Aug, 2024 3 commits
- set CUDA_DEVICE_MAX_CONNECTIONS=1 (#1113) · 0cb099e2
  Lianmin Zheng authored Aug 15, 2024
  
  0cb099e2
- Use a single workspace for flashinfer (#1077) · 326df4ba
  Lianmin Zheng authored Aug 14, 2024
  
  326df4ba
- Support jinja as chat template file (#1104) · 6767e222
  Ying Sheng authored Aug 14, 2024
  
  6767e222
14 Aug, 2024 1 commit
- [Feature] modify Runtime to support skip_tokenizer_init (#1088) · 616b59f3
  rainred authored Aug 14, 2024
```
Co-authored-by: lzhang <zhanglei@modelbest.cn>
```
  616b59f3
12 Aug, 2024 1 commit
- Re-organize CI tests (#1052) · c877292c
  Lianmin Zheng authored Aug 12, 2024
  
  c877292c
11 Aug, 2024 1 commit
- feat: use FlashInfer rmsnorm and silu (#907) · 94752ac8
  Yineng Zhang authored Aug 11, 2024
  
  94752ac8
09 Aug, 2024 3 commits
- Add skip_tokenizer_init args. (#959) · 9cf0a5ba
  gryffindor-rr authored Aug 10, 2024
```
Co-authored-by: lzhang <zhanglei@modelbest.cn>
```
  9cf0a5ba
- Add openai embedding API (#997) · b16e856f
  Ying Sheng authored Aug 09, 2024
  
  b16e856f
- support models from www.modelscope.cn (#994) · b91a4cb1
  liuyhwangyh authored Aug 09, 2024
```
Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
```
  b91a4cb1
08 Aug, 2024 2 commits
- Add e5-mistral embedding model - step 3/3 (#988) · e040a245
  Ying Sheng authored Aug 08, 2024
  
  e040a245
- Move torch.compile configs into cuda_graph_runner.py (#993) · 9f662501
  Ying Sheng authored Aug 08, 2024
  
  9f662501
07 Aug, 2024 2 commits
- Show more error messages for warmup errors (#932) · ff68ae85
  Ying Sheng authored Aug 06, 2024
  
  ff68ae85
- Add support for Batch API test (#936) · 795eab6d
  yichuan~ authored Aug 07, 2024
  
  795eab6d
04 Aug, 2024 2 commits
- Make API Key OpenAI-compatible (#917) · 0d4f3a9f
  Ying Sheng authored Aug 04, 2024
  
  0d4f3a9f
- Add model accuracy test - step 1 (#866) · 70cc0749
  Ying Sheng authored Aug 03, 2024
  
  70cc0749