Commits · f7102fbd2b5d1e6bc0373e54b5bead7370dab160 · change / sglang · GitLab

30 Oct, 2024 1 commit
- [Production] Drain requests before exit when receive SIGTERM (#1838) · 4e2af03c
  Ying Sheng authored Oct 30, 2024
  
  4e2af03c
29 Oct, 2024 1 commit
- fix get_memory_pool_size deadlock for DP (#1830) · 680cad20
  Byron Hsu authored Oct 28, 2024
  
  680cad20
28 Oct, 2024 1 commit
- Fix update_weights deadlock for DP (#1825) · 0a24eb85
  Byron Hsu authored Oct 28, 2024
  
  0a24eb85
26 Oct, 2024 2 commits
- Fix possible ZMQ hanging (#1800) · 1e890341
  Liangsheng Yin authored Oct 25, 2024
  
  1e890341
- [Fix] Fix --skip-tokenizer-init (#1798) · fb99aaa5
  Lianmin Zheng authored Oct 25, 2024
  
  fb99aaa5
23 Oct, 2024 1 commit
- [API] add get memory pool size (#1760) · 2fce449b
  Ying Sheng authored Oct 23, 2024
```
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  2fce449b
21 Oct, 2024 1 commit
- Llama3.2 vision model support (#1551) · 94cde109
  Liangsheng Yin authored Oct 21, 2024
  
  94cde109
17 Oct, 2024 1 commit
- Fix failed ci tests on long prompts; Better error messages for embedding models (#1700) · 7feba415
  Lianmin Zheng authored Oct 17, 2024
  
  7feba415
11 Oct, 2024 1 commit
- Add the ability to enable and disable the Profiler via HTTP API. (#1626) · bbd72bfc
  科英 authored Oct 11, 2024
  
  bbd72bfc
07 Oct, 2024 1 commit
- [Fix] Fix the case where prompt_len = 0 (#1593) · b6aad70a
  Lianmin Zheng authored Oct 06, 2024
  
  b6aad70a
04 Oct, 2024 2 commits
- Use ipc instead of tcp in zmq (#1566) · 114bbc86
  Lianmin Zheng authored Oct 04, 2024
  
  114bbc86
- Organize sampling batch info better (#1562) · 32eb6e96
  Lianmin Zheng authored Oct 03, 2024
  
  32eb6e96
01 Oct, 2024 1 commit
- [Refactor] Simplify io_struct and tokenizer_manager (#1549) · f202ed97
  Ying Sheng authored Oct 01, 2024
  
  f202ed97
30 Sep, 2024 1 commit
- Process image in parallel (#1539) · 55b974f9
  Liangsheng Yin authored Sep 29, 2024
  
  55b974f9
29 Sep, 2024 2 commits
- Improve process creation (#1534) · 04868543
  Lianmin Zheng authored Sep 29, 2024
  
  04868543
- Organize image inputs (#1531) · fd9ad817
  Liangsheng Yin authored Sep 28, 2024
  
  fd9ad817
28 Sep, 2024 1 commit
- [Feature] Support reward model LxzGordon/URM-LLaMa-3.1-8B (#1525) · 9aa6553d
  Ying Sheng authored Sep 27, 2024
  
  9aa6553d
22 Sep, 2024 1 commit

Pr fix max workers (#1456) · 2a99993c

wellhowtosay authored Sep 22, 2024


Co-authored-by: baolujia <baolujia@shizhuang-inc.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>

2a99993c

15 Sep, 2024 1 commit
- [Fix] Fix logprob and normalized_logprob (#1428) · 9ba1f097
  Lianmin Zheng authored Sep 15, 2024
  
  9ba1f097
12 Sep, 2024 1 commit
- [Feature] Initial support for multi-LoRA serving (#1307) · 71221692
  Ying Sheng authored Sep 12, 2024
  
  71221692
09 Sep, 2024 2 commits
- [Minor] Many cleanup (#1357) · e4d68afc
  Lianmin Zheng authored Sep 09, 2024
  
  e4d68afc
- [Feat] Add modalities for vision server when handling pixel values for llava (#1346) · 662ecd93
  Kaichen Zhang - NTU authored Sep 09, 2024
  
  662ecd93
03 Sep, 2024 1 commit
- [Fix] Reduce memory usage for loading llava model & Remove EntryClassRemapping (#1308) · f64eae3a
  Lianmin Zheng authored Sep 02, 2024
  
  f64eae3a
01 Sep, 2024 1 commit
- [Chore] Rename model_overide_args to model_override_args (#1284) · 08360553
  Kai-Hsun Chen authored Sep 01, 2024
```
Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  08360553
28 Aug, 2024 1 commit
- [Fix] Fix llava on multi images (#1247) · bf53bf51
  Lianmin Zheng authored Aug 28, 2024
  
  bf53bf51
26 Aug, 2024 2 commits
- minor: improve CI and dependencies (#1212) · 632d506d
  Liangsheng Yin authored Aug 25, 2024
  
  632d506d
- [Fix] Multi-images loading error (#1218) · 3579162a
  Kaichen Zhang - NTU authored Aug 26, 2024
  
  3579162a
25 Aug, 2024 4 commits
- [Minor] Improve the function organization in TokenizerManager & improve loggers (#1208) · 90227800
  Lianmin Zheng authored Aug 25, 2024
  
  90227800
- Support Alibaba-NLP/gte-Qwen2-7B-instruct embedding Model (#1186) · 30b4f771
  Chayenne authored Aug 26, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  30b4f771
- [Fix] Fixing the multi-images error for llava-onevision (#1205) · 66e7dcaf
  Kaichen Zhang - NTU authored Aug 26, 2024
  
  66e7dcaf
- [Fix] the issue of random order when input is a list (#1199) · 1cb4da5c
  Ying Sheng authored Aug 24, 2024
  
  1cb4da5c
24 Aug, 2024 1 commit
- Cleanup readme, llava examples, usage examples and nccl init (#1194) · f6af3a65
  Lianmin Zheng authored Aug 24, 2024
  
  f6af3a65
23 Aug, 2024 1 commit

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2... · a5b14ad0

Kaichen Zhang - NTU authored Aug 24, 2024


[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)
Co-authored-by: Bo Li <drluodian@gmail.com>

a5b14ad0

22 Aug, 2024 1 commit
- [Minor] Improve logging and rename the health check endpoint name (#1180) · 5623826f
  Lianmin Zheng authored Aug 21, 2024
  
  5623826f
21 Aug, 2024 1 commit
- Improve code style of sampler (#1168) · 83e23c69
  Liangsheng Yin authored Aug 21, 2024
  
  83e23c69
20 Aug, 2024 2 commits
- [Feat] Support update weights without restart server (#1157) · cd10654e
  Shan Yu authored Aug 20, 2024
  
  cd10654e
- feat: allow streaming for multi-prompt and/or parallel sampling (#1134) · d8476818
  Juwan Yoo authored Aug 20, 2024
  
  d8476818
19 Aug, 2024 1 commit

[Feat]Add support for optional start len of logprobs (#1035) · b997a18d

yichuan~ authored Aug 18, 2024


Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>

b997a18d

13 Aug, 2024 1 commit
- Fix create_abort_task, GenerateReqInput does not have rids. (#1079) · 95f5fbf1
  rainred authored Aug 13, 2024
```
Co-authored-by: lzhang <zhanglei@modelbest.cn>
```
  95f5fbf1
10 Aug, 2024 1 commit
- Support embedding input as a list (#1014) · 7599bade
  Ying Sheng authored Aug 10, 2024
  
  7599bade