Commits · a0e58740a8307b7edd2a8868d520c9371a532e92 · change / sglang

27 Nov, 2024 1 commit
- Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default (#2217) · a0e58740
  Lianmin Zheng authored Nov 27, 2024
  
  a0e58740
25 Nov, 2024 2 commits
- [Performance]: Process affinity to CPU cores with multiple sockets support (#2171) · 10189d08
  HAI authored Nov 25, 2024
  
  10189d08
- Allow overwrite flashinfer use_tensorcore (#2169) · 8e1adb84
  Lianmin Zheng authored Nov 24, 2024
  
  8e1adb84
24 Nov, 2024 2 commits
- feat: update other MoE models deps (#2156) · e3938b2f
  Yineng Zhang authored Nov 24, 2024
  
  e3938b2f
- feat: remove the dependency on FusedMoE (#2153) · b509db58
  Yineng Zhang authored Nov 24, 2024
  
  b509db58
23 Nov, 2024 1 commit
- Add simple CPU offloading support. (#2081) · d98fa1e9
  Jani Monoses authored Nov 23, 2024
  
  d98fa1e9
22 Nov, 2024 1 commit
- docs: fix module docstrings and copyright headers (#2077) · 62a4a339
  Xuehai Pan authored Nov 22, 2024
  
  62a4a339
18 Nov, 2024 3 commits
- feat: update torch 2.5.1 (#2069) · 76619261
  Yineng Zhang authored Nov 18, 2024
  
  76619261
- Crash the CI jobs on model import errors (#2072) · df7fe452
  Lianmin Zheng authored Nov 17, 2024
  
  df7fe452
- Deprecate --disable-flashinfer and --disable-flashinfer-sampling (#2065) · 11f881d1
  Lianmin Zheng authored Nov 17, 2024
  
  11f881d1
17 Nov, 2024 4 commits
- Remove monkey_patch_vllm_dummy_weight_loader (#2064) · 38625e21
  Lianmin Zheng authored Nov 17, 2024
  
  38625e21
- Revert "chore: update torch v2.5.1" (#2063) · c1f401fc
  Lianmin Zheng authored Nov 17, 2024
  
  c1f401fc
- chore: update torch v2.5.1 (#1849) · 3b878863
  Yineng Zhang authored Nov 18, 2024
  
  3b878863
- Launch dp ranks in parallel (#2053) · f719d9ae
  Lianmin Zheng authored Nov 16, 2024
```
Co-authored-by: Haotian Liu <6631389+haotian-liu@users.noreply.github.com>
```
  f719d9ae
16 Nov, 2024 1 commit
- Add get_amdgpu_memory_capacity() (#2049) · 2ffe0a73
  HAI authored Nov 15, 2024
  
  2ffe0a73
15 Nov, 2024 1 commit
- [Fix] Adjust default chunked prefill size and cuda graph max bs according to... · b01df48c
  Lianmin Zheng authored Nov 15, 2024
```
[Fix] Adjust default chunked prefill size and cuda graph max bs according to GPU memory capacity (#2044)
```
  b01df48c
10 Nov, 2024 1 commit
- Simplify prometheus metrics (#1981) · 1929c067
  Lianmin Zheng authored Nov 10, 2024
```
Co-authored-by: Mohit Reddy <mohitreddy1996@users.noreply.github.com>
```
  1929c067
09 Nov, 2024 1 commit
- Clean up metrics code (#1972) · 9c939a3d
  Lianmin Zheng authored Nov 09, 2024
  
  9c939a3d
08 Nov, 2024 1 commit
- [minor] Improve code style and compatibility (#1961) · a5095520
  Lianmin Zheng authored Nov 08, 2024
  
  a5095520
03 Nov, 2024 1 commit
- Unify the model type checking (#1905) · 0abbf289
  Lianmin Zheng authored Nov 03, 2024
  
  0abbf289
27 Oct, 2024 1 commit
- Add a watch dog thread (#1816) · 86fc0d79
  Lianmin Zheng authored Oct 27, 2024
  
  86fc0d79
26 Oct, 2024 2 commits
- Set `ZMQ` buffer size heuristic (#1801) · a628dd8e
  Liangsheng Yin authored Oct 25, 2024
  
  a628dd8e
- Fix possible ZMQ hanging (#1800) · 1e890341
  Liangsheng Yin authored Oct 25, 2024
  
  1e890341
21 Oct, 2024 1 commit
- Llama3.2 vision model support (#1551) · 94cde109
  Liangsheng Yin authored Oct 21, 2024
  
  94cde109
20 Oct, 2024 2 commits
- Support qwen2 vl model (#1721) · cbbc82b7
  Yineng Zhang authored Oct 19, 2024
```
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: ispobock <ISPObaoke@163.com>
```
  cbbc82b7
- Update vllm to 0.6.3 (#1711) (#1720) · 8bee20f8
  Yineng Zhang authored Oct 19, 2024
```
Co-authored-by: Ke Bao <ISPObaoke@163.com>
```
  8bee20f8
17 Oct, 2024 2 commits
- Add date to logging messages (#1623) (#1679) · 2782132b
  Zeng Zhongchao authored Oct 17, 2024
  
  2782132b
- add orjson for jsonresponse (#1688) · b0facb33
  Michael Feil authored Oct 16, 2024
  
  b0facb33
16 Oct, 2024 1 commit
- Add a new event loop (#1677) · 9116b289
  Lianmin Zheng authored Oct 16, 2024
  
  9116b289
13 Oct, 2024 1 commit
- [Fix] fix eos trim inconsistency (#1650) · 48761171
  Ying Sheng authored Oct 13, 2024
  
  48761171
11 Oct, 2024 1 commit
- Add device support (#1607) · 8275049c
  Zhang, Liangang authored Oct 11, 2024
  
  8275049c
07 Oct, 2024 4 commits
- [Profile] Add pytorch profiler (#1604) · c5325aba
  Ying Sheng authored Oct 07, 2024
  
  c5325aba
- Optimize broadcast & Reorg code (#1598) · ebbc42d9
  Lianmin Zheng authored Oct 07, 2024
  
  ebbc42d9
- Use is_flashinfer_available to replace is_hip for flashinfer check (#1596) · 6a5b352a
  Lianmin Zheng authored Oct 06, 2024
```
Co-authored-by: Zhang Liangang <liangang.zhang@intel.com>
```
  6a5b352a
- [Fix] Fix the case where prompt_len = 0 (#1593) · b6aad70a
  Lianmin Zheng authored Oct 06, 2024
  
  b6aad70a
06 Oct, 2024 1 commit
- [Minor] Improve the style and fix flaky tests (#1584) · 9244f27f
  Lianmin Zheng authored Oct 06, 2024
  
  9244f27f
04 Oct, 2024 1 commit
- Use ipc instead of tcp in zmq (#1566) · 114bbc86
  Lianmin Zheng authored Oct 04, 2024
  
  114bbc86
29 Sep, 2024 2 commits
- [Fix] fix ipv6 url when warm up model (#1537) · acaffd23
  Xinyu Yang authored Sep 30, 2024
  
  acaffd23
- Improve process creation (#1534) · 04868543
  Lianmin Zheng authored Sep 29, 2024
  
  04868543
28 Sep, 2024 1 commit
- [Feature] Support reward model LxzGordon/URM-LLaMa-3.1-8B (#1525) · 9aa6553d
  Ying Sheng authored Sep 27, 2024
  
  9aa6553d