Commits · d98fa1e93dc9af557c2dd0aa80f8ba80a2fe65e5 · change / sglang · GitLab

23 Nov, 2024 2 commits
- Add simple CPU offloading support. (#2081) · d98fa1e9
  Jani Monoses authored Nov 23, 2024
  
  d98fa1e9
- Add initial support for intel Gaudi accelerators (#2121) · 865233e2
  Ankur Neog authored Nov 23, 2024
  
  865233e2
22 Nov, 2024 13 commits
- Revert "Only stream output on tp rank 0" (#2130) · 66d4859a
  Lianmin Zheng authored Nov 22, 2024
  
  66d4859a
- Only stream output on tp rank 0 (#2124) · e1b63624
  Lianmin Zheng authored Nov 22, 2024
  
  e1b63624
- Expose max total num tokens from Runtime & Engine API (#2092) · c35cd1f8
  Henry Hyeonmok Ko authored Nov 22, 2024
  
  c35cd1f8
- feat(pre-commit): trim unnecessary notebook metadata from git history (#2127) · 72f87b72
  Xuehai Pan authored Nov 23, 2024
  
  72f87b72
- docs: fix module docstrings and copyright headers (#2077) · 62a4a339
  Xuehai Pan authored Nov 22, 2024
  
  62a4a339
- fix: add xgrammar dependency (#2126) · 2797bc34
  Yineng Zhang authored Nov 22, 2024
  
  2797bc34
- chore: bump v0.3.6 (#2120) · 9a00e6f4
  Yineng Zhang authored Nov 22, 2024
  
  9a00e6f4
- minor: update gsm8k threshold (#2125) · 4f8c3aea
  Yineng Zhang authored Nov 22, 2024
  
  4f8c3aea
- [minor] Clean up unused imports (#2122) · 2369e882
  Lianmin Zheng authored Nov 22, 2024
```
Co-authored-by: rinrin32 <rinrin.int@gmail.com>
```
  2369e882
- Benchmark with Pytorch Profiler easily (#2110) · ad30d5cf
  bjmsong authored Nov 22, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  ad30d5cf
- Rename sglang.bench_latency to sglang.bench_one_batch (#2118) · dfec7fca
  Lianmin Zheng authored Nov 21, 2024
  
  dfec7fca
- Fix #2037 - Context length check does not take into out pad tokens for visual models (#2106) · 8048c28c
  Jake Poznanski authored Nov 21, 2024
  
  8048c28c
- [router] add base_gpu_id server args & merged radix tree python reference (#2115) · 30af7dfb
  Byron Hsu authored Nov 21, 2024
  
  30af7dfb
21 Nov, 2024 4 commits
- Add support for Qwen2-VL-based embedding models (#2055) · f6f71379
  James Xu authored Nov 21, 2024
  
  f6f71379
- ROCm: Fix MoE padding for none FP8 cases (#2111) · f35cb46c
  HAI authored Nov 21, 2024
  
  f35cb46c
- Turn off autotune for scaled mm for fp8 dynamic quant in torchao (#2116) · 7f8fcd39
  Jerry Zhang authored Nov 21, 2024
  
  7f8fcd39
- Error out when torchao-config option is not recognized (#2107) · 5c6a41fa
  Jerry Zhang authored Nov 20, 2024
  
  5c6a41fa
20 Nov, 2024 7 commits
- Enable overlap scheduler by default for the triton attention backend (#2105) · 722530fa
  Lianmin Zheng authored Nov 20, 2024
  
  722530fa
- Move test_session_id.py to playground (#2104) · 56a347f7
  Lianmin Zheng authored Nov 20, 2024
  
  56a347f7
- Allow skipping warmup in bench_offline_throughput.py (#2103) · 3295cd8a
  Lianmin Zheng authored Nov 20, 2024
  
  3295cd8a
- [feat] Add session control (#2073) · 5942dfc0
  Ying Sheng authored Nov 20, 2024
  
  5942dfc0
- Update nightly-eval.yml (#2100) · 63a395b9
  Lianmin Zheng authored Nov 19, 2024
  
  63a395b9
- Enable overlap by default (#2067) · 7d671e4a
  Lianmin Zheng authored Nov 19, 2024
  
  7d671e4a
- Set schedule policy more conservative for DP attention (#2096) · 699384cb
  Ke Bao authored Nov 20, 2024
  
  699384cb
19 Nov, 2024 7 commits
- Make constrained decoding work for overlap scheduler (#2095) · ffd20fcd
  Lianmin Zheng authored Nov 19, 2024
  
  ffd20fcd
- minor: add dataset dump and questions shuffle (#2093) · 55bd97f3
  Yineng Zhang authored Nov 20, 2024
  
  55bd97f3
- Use native fp8 format on MI300X (#2094) · e57c3e12
  HAI authored Nov 19, 2024
  
  e57c3e12
- minor: update gsm8k eval (#2091) · f239268f
  Yineng Zhang authored Nov 19, 2024
  
  f239268f
- Fix: incorrect top_logprobs in chat completion (#2088) · 929c7621
  Alexander Waitz authored Nov 19, 2024
  
  929c7621
- Use cuda event wait and synchronization instead of busy waiting (#2089) · b7a065ea
  Lianmin Zheng authored Nov 19, 2024
  
  b7a065ea
- Simplify logits penalizer (#2086) · b1104538
  Lianmin Zheng authored Nov 18, 2024
  
  b1104538
18 Nov, 2024 7 commits
- Allow passing extra request body to bench_offline_throughput.py (#2085) · 3b44bbee
  Lianmin Zheng authored Nov 18, 2024
  
  3b44bbee
- Fix chunked prefill with output logprob (#2083) · 80e2c4a8
  Lianmin Zheng authored Nov 18, 2024
  
  80e2c4a8
- Rename layer_idx to layer_id for consistency (#2078) · 66318ffe
  Jani Monoses authored Nov 18, 2024
  
  66318ffe
- feat: update torch 2.5.1 (#2069) · 76619261
  Yineng Zhang authored Nov 18, 2024
  
  76619261
- support set role as 'tool' (#2075) · 2a3992b6
  yukavio authored Nov 18, 2024
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  2a3992b6
- Simplify flashinfer indices update for prefill (#2074) · 4af3f889
  Lianmin Zheng authored Nov 18, 2024
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
Co-authored-by: kavioyu <kavioyu@gmail.com>
```
  4af3f889
- Crash the CI jobs on model import errors (#2072) · df7fe452
  Lianmin Zheng authored Nov 17, 2024
  
  df7fe452