Commits · 32293a299c7aa4c5d985d97bbb1885e5f32e4862 · change / sglang

"model/vscode:/vscode.git/clone" did not exist on "30fb7e19f806c1b0f0fce19d088dbb126b36acaa"

24 Nov, 2024 1 commit
- Improve sglang router (#2148) · 32293a29
  Byron Hsu authored Nov 23, 2024
  
  32293a29
23 Nov, 2024 13 commits
- add prefix match for certain tenant (#2147) · 79216908
  Byron Hsu authored Nov 23, 2024
  
  79216908
- Add more api routes (completion, health, etc) to the router (#2146) · bbb81c24
  Byron Hsu authored Nov 23, 2024
  
  bbb81c24
- fix dp_rank env (#2144) · 52f58fc4
  Byron Hsu authored Nov 23, 2024
  
  52f58fc4
- update router doc (#2143) · 145c0ddc
  Byron Hsu authored Nov 23, 2024
  
  145c0ddc
- Bump sglang-router to 0.0.5 (#2142) · 505d7f71
  Byron Hsu authored Nov 23, 2024
  
  505d7f71
- [router] cache-aware load-balancing router v1 (#2114) · cbedd1db
  Byron Hsu authored Nov 23, 2024
  
  cbedd1db
- fix: resolve bench_serving args (#2139) · ad47749b
  Yineng Zhang authored Nov 23, 2024
  
  ad47749b
- Fix dp print message (#2138) · 751c3a03
  Lianmin Zheng authored Nov 23, 2024
  
  751c3a03
- Add concurrency option for benchmark (#2136) · 60769be1
  Yunmeng authored Nov 23, 2024
  
  60769be1
- [CI] Fix test cases (#2137) · a78d8f8d
  Lianmin Zheng authored Nov 23, 2024
  
  a78d8f8d
- Fix grid size in Triton decoding kernel (#2134) · c5f86501
  Ke Bao authored Nov 23, 2024
  
  c5f86501
- Add simple CPU offloading support. (#2081) · d98fa1e9
  Jani Monoses authored Nov 23, 2024
  
  d98fa1e9
- Add initial support for intel Gaudi accelerators (#2121) · 865233e2
  Ankur Neog authored Nov 23, 2024
  
  865233e2
22 Nov, 2024 13 commits
- Revert "Only stream output on tp rank 0" (#2130) · 66d4859a
  Lianmin Zheng authored Nov 22, 2024
  
  66d4859a
- Only stream output on tp rank 0 (#2124) · e1b63624
  Lianmin Zheng authored Nov 22, 2024
  
  e1b63624
- Expose max total num tokens from Runtime & Engine API (#2092) · c35cd1f8
  Henry Hyeonmok Ko authored Nov 22, 2024
  
  c35cd1f8
- feat(pre-commit): trim unnecessary notebook metadata from git history (#2127) · 72f87b72
  Xuehai Pan authored Nov 23, 2024
  
  72f87b72
- docs: fix module docstrings and copyright headers (#2077) · 62a4a339
  Xuehai Pan authored Nov 22, 2024
  
  62a4a339
- fix: add xgrammar dependency (#2126) · 2797bc34
  Yineng Zhang authored Nov 22, 2024
  
  2797bc34
- chore: bump v0.3.6 (#2120) · 9a00e6f4
  Yineng Zhang authored Nov 22, 2024
  
  9a00e6f4
- minor: update gsm8k threshold (#2125) · 4f8c3aea
  Yineng Zhang authored Nov 22, 2024
  
  4f8c3aea
- [minor] Clean up unused imports (#2122) · 2369e882
  Lianmin Zheng authored Nov 22, 2024
```
Co-authored-by: rinrin32 <rinrin.int@gmail.com>
```
  2369e882
- Benchmark with Pytorch Profiler easily (#2110) · ad30d5cf
  bjmsong authored Nov 22, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  ad30d5cf
- Rename sglang.bench_latency to sglang.bench_one_batch (#2118) · dfec7fca
  Lianmin Zheng authored Nov 21, 2024
  
  dfec7fca
- Fix #2037 - Context length check does not take into out pad tokens for visual models (#2106) · 8048c28c
  Jake Poznanski authored Nov 21, 2024
  
  8048c28c
- [router] add base_gpu_id server args & merged radix tree python reference (#2115) · 30af7dfb
  Byron Hsu authored Nov 21, 2024
  
  30af7dfb
21 Nov, 2024 4 commits
- Add support for Qwen2-VL-based embedding models (#2055) · f6f71379
  James Xu authored Nov 21, 2024
  
  f6f71379
- ROCm: Fix MoE padding for none FP8 cases (#2111) · f35cb46c
  HAI authored Nov 21, 2024
  
  f35cb46c
- Turn off autotune for scaled mm for fp8 dynamic quant in torchao (#2116) · 7f8fcd39
  Jerry Zhang authored Nov 21, 2024
  
  7f8fcd39
- Error out when torchao-config option is not recognized (#2107) · 5c6a41fa
  Jerry Zhang authored Nov 20, 2024
  
  5c6a41fa
20 Nov, 2024 7 commits
- Enable overlap scheduler by default for the triton attention backend (#2105) · 722530fa
  Lianmin Zheng authored Nov 20, 2024
  
  722530fa
- Move test_session_id.py to playground (#2104) · 56a347f7
  Lianmin Zheng authored Nov 20, 2024
  
  56a347f7
- Allow skipping warmup in bench_offline_throughput.py (#2103) · 3295cd8a
  Lianmin Zheng authored Nov 20, 2024
  
  3295cd8a
- [feat] Add session control (#2073) · 5942dfc0
  Ying Sheng authored Nov 20, 2024
  
  5942dfc0
- Update nightly-eval.yml (#2100) · 63a395b9
  Lianmin Zheng authored Nov 19, 2024
  
  63a395b9
- Enable overlap by default (#2067) · 7d671e4a
  Lianmin Zheng authored Nov 19, 2024
  
  7d671e4a
- Set schedule policy more conservative for DP attention (#2096) · 699384cb
  Ke Bao authored Nov 20, 2024
  
  699384cb
19 Nov, 2024 2 commits
- Make constrained decoding work for overlap scheduler (#2095) · ffd20fcd
  Lianmin Zheng authored Nov 19, 2024
  
  ffd20fcd
- minor: add dataset dump and questions shuffle (#2093) · 55bd97f3
  Yineng Zhang authored Nov 20, 2024
  
  55bd97f3