Commits · 7d671e4ad2977d8090f44be5e94f351a15f4c9bf · change / sglang

"vscode:/vscode.git/clone" did not exist on "ce0c95d0972476f1e5a0064edbceb33d3ceed6ba"

20 Nov, 2024 2 commits
- Enable overlap by default (#2067) · 7d671e4a
  Lianmin Zheng authored Nov 19, 2024
  
  7d671e4a
- Set schedule policy more conservative for DP attention (#2096) · 699384cb
  Ke Bao authored Nov 20, 2024
  
  699384cb
19 Nov, 2024 7 commits
- Make constrained decoding work for overlap scheduler (#2095) · ffd20fcd
  Lianmin Zheng authored Nov 19, 2024
  
  ffd20fcd
- minor: add dataset dump and questions shuffle (#2093) · 55bd97f3
  Yineng Zhang authored Nov 20, 2024
  
  55bd97f3
- Use native fp8 format on MI300X (#2094) · e57c3e12
  HAI authored Nov 19, 2024
  
  e57c3e12
- minor: update gsm8k eval (#2091) · f239268f
  Yineng Zhang authored Nov 19, 2024
  
  f239268f
- Fix: incorrect top_logprobs in chat completion (#2088) · 929c7621
  Alexander Waitz authored Nov 19, 2024
  
  929c7621
- Use cuda event wait and synchronization instead of busy waiting (#2089) · b7a065ea
  Lianmin Zheng authored Nov 19, 2024
  
  b7a065ea
- Simplify logits penalizer (#2086) · b1104538
  Lianmin Zheng authored Nov 18, 2024
  
  b1104538
18 Nov, 2024 15 commits
- Allow passing extra request body to bench_offline_throughput.py (#2085) · 3b44bbee
  Lianmin Zheng authored Nov 18, 2024
  
  3b44bbee
- Fix chunked prefill with output logprob (#2083) · 80e2c4a8
  Lianmin Zheng authored Nov 18, 2024
  
  80e2c4a8
- Rename layer_idx to layer_id for consistency (#2078) · 66318ffe
  Jani Monoses authored Nov 18, 2024
  
  66318ffe
- feat: update torch 2.5.1 (#2069) · 76619261
  Yineng Zhang authored Nov 18, 2024
  
  76619261
- support set role as 'tool' (#2075) · 2a3992b6
  yukavio authored Nov 18, 2024
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  2a3992b6
- Simplify flashinfer indices update for prefill (#2074) · 4af3f889
  Lianmin Zheng authored Nov 18, 2024
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
Co-authored-by: kavioyu <kavioyu@gmail.com>
```
  4af3f889
- Crash the CI jobs on model import errors (#2072) · df7fe452
  Lianmin Zheng authored Nov 17, 2024
  
  df7fe452
- Tune the threshold for accuracy tests in CI (#2071) · a7164b62
  Lianmin Zheng authored Nov 17, 2024
  
  a7164b62
- Fix cuda illegal memory access in overlap mode (#2070) · 11668533
  Lianmin Zheng authored Nov 17, 2024
  
  11668533
- [Minor] Fix styles for overlap mode (#2068) · a9e90b4b
  Lianmin Zheng authored Nov 17, 2024
  
  a9e90b4b
- add phi-3 small support (#2062) · 8c280cee
  Tanjiro authored Nov 17, 2024
```
Co-authored-by: Tushar Goel <114812108+AI-Tushar@users.noreply.github.com>
```
  8c280cee
- [Performance] Update xgrammar-related constrained decoding (#2056) · 9c745d07
  DarkSharpness authored Nov 18, 2024
  
  9c745d07
- Rename arguments `--disable-nan-detection` to `--enable-nan-detection` (#2066) · ebaa2f31
  Lianmin Zheng authored Nov 17, 2024
  
  ebaa2f31
- Support cuda graph for DP attention (#2061) · 62832bb2
  Ke Bao authored Nov 18, 2024
  
  62832bb2
- Deprecate --disable-flashinfer and --disable-flashinfer-sampling (#2065) · 11f881d1
  Lianmin Zheng authored Nov 17, 2024
  
  11f881d1
17 Nov, 2024 5 commits
- Remove monkey_patch_vllm_dummy_weight_loader (#2064) · 38625e21
  Lianmin Zheng authored Nov 17, 2024
  
  38625e21
- Revert "chore: update torch v2.5.1" (#2063) · c1f401fc
  Lianmin Zheng authored Nov 17, 2024
  
  c1f401fc
- chore: update torch v2.5.1 (#1849) · 3b878863
  Yineng Zhang authored Nov 18, 2024
  
  3b878863
- Launch dp ranks in parallel (#2053) · f719d9ae
  Lianmin Zheng authored Nov 16, 2024
```
Co-authored-by: Haotian Liu <6631389+haotian-liu@users.noreply.github.com>
```
  f719d9ae
- Fix illegal memory access in overlap mode & Use more fused triton kernels for... · edad3731
  Lianmin Zheng authored Nov 16, 2024
```
Fix illegal memory access in overlap mode & Use more fused triton kernels for building meta data (#2051)
```
  edad3731
16 Nov, 2024 5 commits
- Support DP MLA (#1970) · 976bc302
  Ke Bao authored Nov 16, 2024
  
  976bc302
- Fix weight update for data parallelism (#2050) · 2f2e0743
  Lianmin Zheng authored Nov 16, 2024
  
  2f2e0743
- Add get_amdgpu_memory_capacity() (#2049) · 2ffe0a73
  HAI authored Nov 15, 2024
  
  2ffe0a73
- Add Tensor Parallel to torch_native_llama (#1876) · cf248976
  Ke Wen authored Nov 15, 2024
  
  cf248976
- Fix core (MI300X) with --enable-overlap (#2048) · e5c67150
  HAI authored Nov 15, 2024
  
  e5c67150
15 Nov, 2024 6 commits
- fix small typos in docs (#2047) · 023d0a73
  Xiaoyu Zhang authored Nov 16, 2024
  
  023d0a73
- Release v0.3.5.post2 (#2046) · 32c9a7ec
  Lianmin Zheng authored Nov 15, 2024
  
  32c9a7ec
- [Fix] Adjust default chunked prefill size and cuda graph max bs according to... · b01df48c
  Lianmin Zheng authored Nov 15, 2024
```
[Fix] Adjust default chunked prefill size and cuda graph max bs according to GPU memory capacity (#2044)
```
  b01df48c
- Fix json benchmark (#2043) · c29b98e0
  Lianmin Zheng authored Nov 15, 2024
  
  c29b98e0
- benchmark json schema (#2030) · 954f4e6b
  DarkSharpness authored Nov 15, 2024
  
  954f4e6b
- Fix the default arguments of bench_offline_throughput.py & simplify detokenizer manager (#2042) · 2558d6a6
  Lianmin Zheng authored Nov 15, 2024
  
  2558d6a6