Commits · 954f4e6bd607ae8ed08cc60dab7c8117e1ff1776 · change / sglang · GitLab

15 Nov, 2024 5 commits
- benchmark json schema (#2030) · 954f4e6b
  DarkSharpness authored Nov 15, 2024
  
  954f4e6b
- Fix the default arguments of bench_offline_throughput.py & simplify detokenizer manager (#2042) · 2558d6a6
  Lianmin Zheng authored Nov 15, 2024
  
  2558d6a6
- fix: align enable_overlap_scheduler naming between code and docs (#2038) · 29ebe3df
  ws authored Nov 15, 2024
  
  29ebe3df
- Offline LLM Engine Benchmark Throughput (#1968) · f6dd6486
  zolinthecow authored Nov 14, 2024
```
Co-authored-by: ByronHsu <byronhsu1230@gmail.com>
```
  f6dd6486
- Expose no_stop_trim and skip_special_tokens in openai api (#2039) · ea53c63b
  Lianmin Zheng authored Nov 14, 2024
  
  ea53c63b
14 Nov, 2024 7 commits
- Fix outlines version (#2036) · a10d5309
  Lianmin Zheng authored Nov 14, 2024
  
  a10d5309
- Fix unit tests (#2034) · aae5434b
  Lianmin Zheng authored Nov 14, 2024
  
  aae5434b
- Fix torch.compile for MoE (#2033) · c3eac1b0
  Lianmin Zheng authored Nov 14, 2024
  
  c3eac1b0
- Github runner instructions for AMD (#2031) · b275ce00
  HAI authored Nov 13, 2024
  
  b275ce00
- Add download_dir ServerArgs property (#2027) · 13ce3e4b
  Patrick Yi authored Nov 14, 2024
  
  13ce3e4b
- chore: open lto and optimization in release profile (#2028) · df246e69
  Tzu Gwo authored Nov 14, 2024
  
  df246e69
- set content to empty string (#2026) · fb9fb351
  chottolabs authored Nov 13, 2024
  
  fb9fb351
13 Nov, 2024 5 commits
- Fix dependency and error message for xgrammar (#2024) · c722d9bd
  Lianmin Zheng authored Nov 13, 2024
  
  c722d9bd
- Do not let invalid grammar crash the server (#2023) · 218ab361
  Lianmin Zheng authored Nov 13, 2024
  
  218ab361
- Release v0.3.5.post1 (#2022) · f407fcf9
  Lianmin Zheng authored Nov 13, 2024
  
  f407fcf9
- Fix grammar backend for tensor parallelism (#2020) · 54479d6f
  Lianmin Zheng authored Nov 13, 2024
  
  54479d6f
- Fix grammar backend (#2018) · ba069a24
  Lianmin Zheng authored Nov 12, 2024
  
  ba069a24
12 Nov, 2024 9 commits
- support parallel grammar preprocessing (#1996) · 125b1199
  DarkSharpness authored Nov 13, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  125b1199
- fix test_embedding_models prompt length too long's bug (#2015) · eff468dd
  Xiaoyu Zhang authored Nov 12, 2024
  
  eff468dd
- fix a bug in v1_embeeding_request (#2014) · a1bd7190
  Xiaoyu Zhang authored Nov 12, 2024
  
  a1bd7190
- Fix finish reason (#2013) · 78c1d644
  Lianmin Zheng authored Nov 11, 2024
  
  78c1d644
- support echo=true and logprobs in openai api when logprobs=1 in lm-evaluation-harness (#1998) · 027e6524
  Xiaoyu Zhang authored Nov 12, 2024
  
  027e6524
- Filter empty prompt in random bench serving (#2011) · b808a383
  Ke Bao authored Nov 12, 2024
  
  b808a383
- remove sglang folder in rust (#2010) · 602ebc66
  Byron Hsu authored Nov 11, 2024
  
  602ebc66
- Fix weight loading for tied word embedding when TP > 1 (#2009) · 530ae1bd
  Lianmin Zheng authored Nov 11, 2024
  
  530ae1bd
- Fix a typo in io_struct.py (#2008) · befc6beb
  Lianmin Zheng authored Nov 11, 2024
  
  befc6beb
11 Nov, 2024 14 commits
- [Minor] Remove unused imports (#2006) · 59a5ba9b
  Lianmin Zheng authored Nov 11, 2024
  
  59a5ba9b
- fix sglang_router not found (#2005) · 86c37d01
  Byron Hsu authored Nov 11, 2024
  
  86c37d01
- support internlm2-reward (#1994) · f18b9c72
  RangiLyu authored Nov 12, 2024
  
  f18b9c72
- run rust test on ubuntu instead of 1-gpu-runner (#2003) · 3e335743
  Byron Hsu authored Nov 11, 2024
  
  3e335743
- Bump router to 0.0.3 (#2004) · 0d94f1dd
  Byron Hsu authored Nov 11, 2024
  
  0d94f1dd
- release router from py38 to py312 (#2002) · e728258d
  Byron Hsu authored Nov 11, 2024
  
  e728258d
- Fix rust unit test and pypi token (#2001) · 239eafbd
  Byron Hsu authored Nov 11, 2024
  
  239eafbd
- Add Engine::encode example (#2000) · 9d427265
  James Xu authored Nov 11, 2024
  
  9d427265
- setup router python binding ci (#1999) · 00ffde20
  Byron Hsu authored Nov 11, 2024
  
  00ffde20
- Add engine encode (#1995) · ddeb9d42
  James Xu authored Nov 11, 2024
```
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  ddeb9d42
- docs: add slides link in README (#1997) · aaf0a315
  Yineng Zhang authored Nov 11, 2024
  
  aaf0a315
- [rust] cache-aware DP - approx tree (#1934) · f9633fa9
  Byron Hsu authored Nov 10, 2024
  
  f9633fa9
- [Performance, Triton] Optimize over mask compute to tl.load in fused_moe_kernel (#1980) · 087ab832
  HAI authored Nov 10, 2024
  
  087ab832
- Add gen-shared-prefix dataset in bench_serving (#1990) · 8169c6f4
  Byron Hsu authored Nov 10, 2024
  
  8169c6f4