Commits · be0124bda09dc10267f6cbbcb097bf14dd4fd8b6 · change / sglang · GitLab

24 Nov, 2024 13 commits
- Rename triton_fused_moe -> fused_moe_triton (#2163) · be0124bd
  Lianmin Zheng authored Nov 24, 2024
  
  be0124bd
- Balance CI tests (#2162) · fe5d3e81
  Lianmin Zheng authored Nov 24, 2024
  
  fe5d3e81
- Fix mixed chunked prefill in overlap mode (#2158) · 731146f6
  Lianmin Zheng authored Nov 24, 2024
  
  731146f6
- fix: use torch.sum for compatible (#2161) · fa271613
  Yineng Zhang authored Nov 24, 2024
  
  fa271613
- Update CI threshold & Improve code style (#2159) · 5652c565
  Lianmin Zheng authored Nov 24, 2024
  
  5652c565
- feat: update other MoE models deps (#2156) · e3938b2f
  Yineng Zhang authored Nov 24, 2024
  
  e3938b2f
- Simplify batch update (#2154) · c211e7b6
  Lianmin Zheng authored Nov 24, 2024
  
  c211e7b6
- fix: resolve end-of-file-fixer (#2157) · d90c3d6b
  Yineng Zhang authored Nov 24, 2024
  
  d90c3d6b
- feat: update gitignore and add tuning config for FusedMoE (#2155) · 9e8f8fbf
  Yineng Zhang authored Nov 24, 2024
  
  9e8f8fbf
- feat: remove the dependency on FusedMoE (#2153) · b509db58
  Yineng Zhang authored Nov 24, 2024
  
  b509db58
- Merged three native APIs into one: get_server_info (#2152) · dbe17293
  Henry Hyeonmok Ko authored Nov 24, 2024
  
  dbe17293
- Update release-pypi-router.yml · 84a1698d
  Byron Hsu authored Nov 23, 2024
  
  84a1698d
- Improve sglang router (#2148) · 32293a29
  Byron Hsu authored Nov 23, 2024
  
  32293a29
23 Nov, 2024 13 commits
- add prefix match for certain tenant (#2147) · 79216908
  Byron Hsu authored Nov 23, 2024
  
  79216908
- Add more api routes (completion, health, etc) to the router (#2146) · bbb81c24
  Byron Hsu authored Nov 23, 2024
  
  bbb81c24
- fix dp_rank env (#2144) · 52f58fc4
  Byron Hsu authored Nov 23, 2024
  
  52f58fc4
- update router doc (#2143) · 145c0ddc
  Byron Hsu authored Nov 23, 2024
  
  145c0ddc
- Bump sglang-router to 0.0.5 (#2142) · 505d7f71
  Byron Hsu authored Nov 23, 2024
  
  505d7f71
- [router] cache-aware load-balancing router v1 (#2114) · cbedd1db
  Byron Hsu authored Nov 23, 2024
  
  cbedd1db
- fix: resolve bench_serving args (#2139) · ad47749b
  Yineng Zhang authored Nov 23, 2024
  
  ad47749b
- Fix dp print message (#2138) · 751c3a03
  Lianmin Zheng authored Nov 23, 2024
  
  751c3a03
- Add concurrency option for benchmark (#2136) · 60769be1
  Yunmeng authored Nov 23, 2024
  
  60769be1
- [CI] Fix test cases (#2137) · a78d8f8d
  Lianmin Zheng authored Nov 23, 2024
  
  a78d8f8d
- Fix grid size in Triton decoding kernel (#2134) · c5f86501
  Ke Bao authored Nov 23, 2024
  
  c5f86501
- Add simple CPU offloading support. (#2081) · d98fa1e9
  Jani Monoses authored Nov 23, 2024
  
  d98fa1e9
- Add initial support for intel Gaudi accelerators (#2121) · 865233e2
  Ankur Neog authored Nov 23, 2024
  
  865233e2
22 Nov, 2024 13 commits
- Revert "Only stream output on tp rank 0" (#2130) · 66d4859a
  Lianmin Zheng authored Nov 22, 2024
  
  66d4859a
- Only stream output on tp rank 0 (#2124) · e1b63624
  Lianmin Zheng authored Nov 22, 2024
  
  e1b63624
- Expose max total num tokens from Runtime & Engine API (#2092) · c35cd1f8
  Henry Hyeonmok Ko authored Nov 22, 2024
  
  c35cd1f8
- feat(pre-commit): trim unnecessary notebook metadata from git history (#2127) · 72f87b72
  Xuehai Pan authored Nov 23, 2024
  
  72f87b72
- docs: fix module docstrings and copyright headers (#2077) · 62a4a339
  Xuehai Pan authored Nov 22, 2024
  
  62a4a339
- fix: add xgrammar dependency (#2126) · 2797bc34
  Yineng Zhang authored Nov 22, 2024
  
  2797bc34
- chore: bump v0.3.6 (#2120) · 9a00e6f4
  Yineng Zhang authored Nov 22, 2024
  
  9a00e6f4
- minor: update gsm8k threshold (#2125) · 4f8c3aea
  Yineng Zhang authored Nov 22, 2024
  
  4f8c3aea
- [minor] Clean up unused imports (#2122) · 2369e882
  Lianmin Zheng authored Nov 22, 2024
```
Co-authored-by: rinrin32 <rinrin.int@gmail.com>
```
  2369e882
- Benchmark with Pytorch Profiler easily (#2110) · ad30d5cf
  bjmsong authored Nov 22, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  ad30d5cf
- Rename sglang.bench_latency to sglang.bench_one_batch (#2118) · dfec7fca
  Lianmin Zheng authored Nov 21, 2024
  
  dfec7fca
- Fix #2037 - Context length check does not take into out pad tokens for visual models (#2106) · 8048c28c
  Jake Poznanski authored Nov 21, 2024
  
  8048c28c
- [router] add base_gpu_id server args & merged radix tree python reference (#2115) · 30af7dfb
  Byron Hsu authored Nov 21, 2024
  
  30af7dfb
21 Nov, 2024 1 commit
- Add support for Qwen2-VL-based embedding models (#2055) · f6f71379
  James Xu authored Nov 21, 2024
  
  f6f71379