Commits · 9610fcd46964507a7c5418ea35e94935c7881815 · change / sglang

13 Oct, 2024 1 commit
- Add an option to disable penalizer (#1651) · 9da5a60b
  Lianmin Zheng authored Oct 12, 2024
  
  9da5a60b
12 Oct, 2024 1 commit
- [Feature, Hardware] Enable SGLang on XPU GPUs via PyTorch (#1480) · 5d638c92
  Zhang, Liangang authored Oct 13, 2024
  
  5d638c92
11 Oct, 2024 3 commits
- Add back data parallelism (#1635) · 23cc66f7
  Lianmin Zheng authored Oct 11, 2024
  
  23cc66f7
- Nit about the decorator of `PortArgs.init_new` (#1611) · 58093b86
  glen-amd authored Oct 11, 2024
  
  58093b86
- Add device support (#1607) · 8275049c
  Zhang, Liangang authored Oct 11, 2024
  
  8275049c
07 Oct, 2024 2 commits
- Remove references to squeezellm (#1603) · 3ff64113
  Jani Monoses authored Oct 07, 2024
  
  3ff64113
- Use is_flashinfer_available to replace is_hip for flashinfer check (#1596) · 6a5b352a
  Lianmin Zheng authored Oct 06, 2024
```
Co-authored-by: Zhang Liangang <liangang.zhang@intel.com>
```
  6a5b352a
04 Oct, 2024 2 commits
- Use ipc instead of tcp in zmq (#1566) · 114bbc86
  Lianmin Zheng authored Oct 04, 2024
  
  114bbc86
- Organize sampling batch info better (#1562) · 32eb6e96
  Lianmin Zheng authored Oct 03, 2024
  
  32eb6e96
30 Sep, 2024 1 commit
- [Fix, LoRA] fix LoRA with updates in main (#1545) · 0f4fb19b
  Ying Sheng authored Sep 30, 2024
  
  0f4fb19b
29 Sep, 2024 2 commits
- [Fix] fix ipv6 url when warm up model (#1537) · acaffd23
  Xinyu Yang authored Sep 30, 2024
  
  acaffd23
- Improve process creation (#1534) · 04868543
  Lianmin Zheng authored Sep 29, 2024
  
  04868543
18 Sep, 2024 1 commit
- Add bench_server_latency.py (#1452) · 5e62a6b7
  Lianmin Zheng authored Sep 18, 2024
  
  5e62a6b7
17 Sep, 2024 4 commits
- Fix attention backend (#1448) · b3710d2c
  Ke Bao authored Sep 17, 2024
  
  b3710d2c
- Enable MLA by default (#1447) · c6b6d2e7
  Ke Bao authored Sep 17, 2024
  
  c6b6d2e7
- Fix torch compile for deepseek-v2 (#1442) · 76524b70
  Ke Bao authored Sep 17, 2024
  
  76524b70
- [Feature, Hardware] Enable SGLang on AMD GPUs via PyTorch for ROCm (#1420) · 3a6e0418
  HAI authored Sep 17, 2024
  
  3a6e0418
16 Sep, 2024 1 commit
- Add constrained_json_whitespace_pattern to ServerArgs (#1438) · 93dffd69
  zifeitong authored Sep 16, 2024
  
  93dffd69
15 Sep, 2024 1 commit
- [Feature] Support LoRA path renaming and add LoRA serving benchmarks (#1433) · 37963394
  Ying Sheng authored Sep 15, 2024
  
  37963394
12 Sep, 2024 1 commit
- [Feature] Initial support for multi-LoRA serving (#1307) · 71221692
  Ying Sheng authored Sep 12, 2024
  
  71221692
11 Sep, 2024 3 commits
- Refactor attention backend (#1381) · fec185ce
  Lianmin Zheng authored Sep 11, 2024
  
  fec185ce
- [Fix] Fix --disable-flashinfer (#1389) · 15c75e41
  Lianmin Zheng authored Sep 11, 2024
  
  15c75e41
- Deprecate --disable-flashinfer and introduce --attention-backend (#1380) · 46094e0c
  Lianmin Zheng authored Sep 10, 2024
  
  46094e0c
09 Sep, 2024 3 commits
- Add torchao quant (int4/int8/fp8) to llama models (#1341) · a7c47e0f
  Jerry Zhang authored Sep 09, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  a7c47e0f
- [Minor] Many cleanup (#1357) · e4d68afc
  Lianmin Zheng authored Sep 09, 2024
  
  e4d68afc
- [server] Passing `model_override_args` to `launch_server` via the CLI. (#1298) · c9b75917
  Kai-Hsun Chen authored Sep 09, 2024
```
Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com>
```
  c9b75917
26 Aug, 2024 1 commit
- [Feature] Support fp8 e5m2 kv cache with flashinfer (#1204) · 2c615d12
  Ke Bao authored Aug 26, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  2c615d12
25 Aug, 2024 2 commits
- [Minor] Improve the function organization in TokenizerManager & improve loggers (#1208) · 90227800
  Lianmin Zheng authored Aug 25, 2024
  
  90227800
- Support Alibaba-NLP/gte-Qwen2-7B-instruct embedding Model (#1186) · 30b4f771
  Chayenne authored Aug 26, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  30b4f771
24 Aug, 2024 1 commit
- Cleanup readme, llava examples, usage examples and nccl init (#1194) · f6af3a65
  Lianmin Zheng authored Aug 24, 2024
  
  f6af3a65
22 Aug, 2024 1 commit
- [Minor] Improve logging and rename the health check endpoint name (#1180) · 5623826f
  Lianmin Zheng authored Aug 21, 2024
  
  5623826f
21 Aug, 2024 1 commit
- Improve multi-node stability (#1171) · bea2bb9e
  Lianmin Zheng authored Aug 20, 2024
  
  bea2bb9e
20 Aug, 2024 1 commit

[Feature] add disable-custom-all-reduce (#1148) · ff2cfdb1

Xu-Chen authored Aug 20, 2024


Co-authored-by: chenxu02 <chenxu02@zhihu.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>

ff2cfdb1

16 Aug, 2024 1 commit
- Mixed style of chunked prefill (#1013) · 3694f8f9
  Liangsheng Yin authored Aug 16, 2024
  
  3694f8f9
15 Aug, 2024 2 commits
- [Fix] Window attention compatible with RadixAttention and chunked prefill (#1112) · 93d4e354
  Ying Sheng authored Aug 15, 2024
  
  93d4e354
- Enable chunked prefill by default (#1040) · e86b1ccb
  Lianmin Zheng authored Aug 14, 2024
  
  e86b1ccb
14 Aug, 2024 2 commits
- [Fix] Compatibility of window attention and cuda graph (#1090) · 96a2093e
  Ying Sheng authored Aug 14, 2024
  
  96a2093e
- [Feat] Add window attention for gemma-2 (#1056) · 0909bb0d
  Ying Sheng authored Aug 13, 2024
  
  0909bb0d
11 Aug, 2024 2 commits
- Test the case when max_new_tokens is very large (#1038) · d84c5e70
  Lianmin Zheng authored Aug 11, 2024
  
  d84c5e70
- Clean up readme and arguments of chunked prefill (#1022) · a97df791
  Lianmin Zheng authored Aug 11, 2024
  
  a97df791