Commits · fa7ccb3316dccdf0326913222c337da20b436251 · change / sglang

23 Jul, 2024 4 commits
- feat: add e2e latency (#704) · fa7ccb33
  zhyncs authored Jul 24, 2024
  
  fa7ccb33
- Use min new token ratio at start (#701) · 26868443
  Liangsheng Yin authored Jul 23, 2024
  
  26868443
- Fix hf config loading (#702) · 824a77d0
  Ke Bao authored Jul 24, 2024
  
  824a77d0
- Fix flashinfer (#700) · cf99eab7
  Ying Sheng authored Jul 23, 2024
  
  cf99eab7
22 Jul, 2024 5 commits
- misc: fix typo (#698) · 9fdea29d
  zhyncs authored Jul 23, 2024
  
  9fdea29d
- Fix trt benchmark (#697) · df7c4c19
  Ying Sheng authored Jul 22, 2024
  
  df7c4c19
- Tune params (#696) · c3f1aac8
  Ying Sheng authored Jul 22, 2024
  
  c3f1aac8
- misc: update output token logic (#695) · d198791f
  zhyncs authored Jul 22, 2024
  
  d198791f
- fix: update bench serving (#694) · c07526e4
  zhyncs authored Jul 22, 2024
  
  c07526e4
21 Jul, 2024 8 commits
- Support Mistral-Nemo (#691) · 5303c1ed
  Ke Bao authored Jul 22, 2024
  
  5303c1ed
- misc: recommend to use chat model for benchmark (#690) · 65bd1338
  zhyncs authored Jul 22, 2024
  
  65bd1338
- Support Deepseek MoE Model (#689) · eedc12e1
  Liangsheng Yin authored Jul 21, 2024
  
  eedc12e1
- misc: update output file logic (#686) · 6a846bb1
  zhyncs authored Jul 21, 2024
  
  6a846bb1
- feat: update bench serving (#685) · 0fdb3127
  zhyncs authored Jul 21, 2024
  
  0fdb3127
- Fix StreamExecutor.fork() losing the current role start index. (#684) · 5ad033a0
  Max Shawabkeh authored Jul 20, 2024
  
  5ad033a0
- support non-streaming benchmark (#682) · 77e592e8
  Lianmin Zheng authored Jul 20, 2024
  
  77e592e8
- Support gpt-bigcode model class (#681) · caaad53b
  Liangsheng Yin authored Jul 20, 2024
  
  caaad53b
20 Jul, 2024 14 commits
- Decouple kv (#679) · 69d19188
  Liangsheng Yin authored Jul 20, 2024
  
  69d19188
- feat: support TRT LLM benchmark and multiple benchmarks (#670) · 4b4a67f8
  zhyncs authored Jul 21, 2024
  
  4b4a67f8
- Fallback when sampling failed (#678) · 0ac94c36
  Ke Bao authored Jul 21, 2024
  
  0ac94c36
- Update version to 0.1.22 (#677) · 2b4c6462
  Ying Sheng authored Jul 20, 2024
  
  2b4c6462
- Fix illegal tokens during sampling (#676) · f424e76d
  Liangsheng Yin authored Jul 20, 2024
  
  f424e76d
- Fix cuda graph with flashinfer (#675) · 490a1f39
  Lianmin Zheng authored Jul 20, 2024
  
  490a1f39
- refactor model loader: initial refactor (#664) · 06487f12
  Ying Sheng authored Jul 20, 2024
  
  06487f12
- Revert "Temporary fix invalid sample results" (#673) · 39c57317
  Liangsheng Yin authored Jul 20, 2024
  
  39c57317
- Fix random dataset (#671) · 9592a1f3
  Lianmin Zheng authored Jul 20, 2024
  
  9592a1f3
- Support random dataset in bench_serving.py (#669) · 35759efa
  Lianmin Zheng authored Jul 20, 2024
  
  35759efa
- Temporary fix invalid sample results (#668) · 8f4b1559
  Liangsheng Yin authored Jul 20, 2024
  
  8f4b1559
- Update OpenAI API (#667) · e3046ea3
  Mingyi authored Jul 19, 2024
  
  e3046ea3
- Add support for OpenAI API parallel sampling (#640) · 49c5e0ec
  yichuan~ authored Jul 20, 2024
  
  49c5e0ec
- Fix kill process util (#666) · ec2150b2
  Ke Bao authored Jul 20, 2024
  
  ec2150b2
19 Jul, 2024 9 commits
- Fix jump forward when streaming (#665) · 7620cd37
  Liangsheng Yin authored Jul 19, 2024
  
  7620cd37
- Add benchmark instructions (#663) · 11c8efff
  Ying Sheng authored Jul 19, 2024
  
  11c8efff
- Improve docs (#662) · e87c7fd5
  Ying Sheng authored Jul 19, 2024
  
  e87c7fd5
- feat: update check env (#661) · 630479c3
  zhyncs authored Jul 20, 2024
  
  630479c3
- Update Readme (#660) · 51fda143
  Ying Sheng authored Jul 19, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  51fda143
- misc: update SGLang package description (#659) · dc4e4a6a
  zhyncs authored Jul 20, 2024
  
  dc4e4a6a
- refactor model loader [unreachable code]: initial refactor (#655) · 2d96da81
  Ying Sheng authored Jul 19, 2024
  
  2d96da81
- feat: add benchmark serving (#657) · c126a6cc
  zhyncs authored Jul 20, 2024
  
  c126a6cc
- perf: reduce ttft and itl with stream_interval 1 (#658) · ac971ff6
  zhyncs authored Jul 20, 2024
  
  ac971ff6