Commits · 6e4539405167da85107fd46fce1db6d874067fc4 · change / sglang

25 Jul, 2024 2 commits
- chore: add close inactive issues workflow (#722) · 6e453940
  Yineng Zhang authored Jul 25, 2024
  
  6e453940
- docs: update comment (#721) · 97e0f7d2
  Yineng Zhang authored Jul 25, 2024
  
  97e0f7d2
24 Jul, 2024 13 commits
- docs: update supported models (#719) · d5146bae
  Yineng Zhang authored Jul 25, 2024
  
  d5146bae
- Bump version to 0.1.24 (#718) · 459abad2
  Ying Sheng authored Jul 24, 2024
  
  459abad2
- Improve benchmark scripts (#717) · 30d8e130
  Ying Sheng authored Jul 24, 2024
  
  30d8e130
- docs: update doc (#716) · 08a3bd19
  Ying Sheng authored Jul 24, 2024
  
  08a3bd19
- misc: update doc (#715) · 321a963b
  Yineng Zhang authored Jul 25, 2024
  
  321a963b
- fix: llama 3.1 405b fp8 (#714) · e17deb27
  Yineng Zhang authored Jul 25, 2024
  
  e17deb27
- docs: update doc (#713) · 2d3ae4e1
  Yineng Zhang authored Jul 25, 2024
  
  2d3ae4e1
- docs: update README (#712) · 75f4ccb7
  Yineng Zhang authored Jul 24, 2024
  
  75f4ccb7
- format · 83d2b30d
  Ying Sheng authored Jul 24, 2024
  
  83d2b30d
- Fix prefill size (#711) · 4367f4bb
  Ying Sheng authored Jul 24, 2024
  
  4367f4bb
- Update schedule_heuristic.py · 00e4baa7
  Lianmin Zheng authored Jul 24, 2024
  
  00e4baa7
- Auto adjust new ratio (#708) · 4cd64b8e
  Liangsheng Yin authored Jul 23, 2024
  
  4cd64b8e
- Fix multi-node deadlock (#709) · 01d66ae2
  Lianmin Zheng authored Jul 23, 2024
  
  01d66ae2
23 Jul, 2024 7 commits
- Reduce hardcoded logic of kernel usage (#707) · a523a3c1
  Mingyi authored Jul 23, 2024
  
  a523a3c1
- bump version to 0.1.23 (#706) · 9f94728f
  Ying Sheng authored Jul 23, 2024
  
  9f94728f
- Update vllm version to support llama3.1 (#705) · 444a0244
  Ying Sheng authored Jul 23, 2024
  
  444a0244
- feat: add e2e latency (#704) · fa7ccb33
  zhyncs authored Jul 24, 2024
  
  fa7ccb33
- Use min new token ratio at start (#701) · 26868443
  Liangsheng Yin authored Jul 23, 2024
  
  26868443
- Fix hf config loading (#702) · 824a77d0
  Ke Bao authored Jul 24, 2024
  
  824a77d0
- Fix flashinfer (#700) · cf99eab7
  Ying Sheng authored Jul 23, 2024
  
  cf99eab7
22 Jul, 2024 5 commits
- misc: fix typo (#698) · 9fdea29d
  zhyncs authored Jul 23, 2024
  
  9fdea29d
- Fix trt benchmark (#697) · df7c4c19
  Ying Sheng authored Jul 22, 2024
  
  df7c4c19
- Tune params (#696) · c3f1aac8
  Ying Sheng authored Jul 22, 2024
  
  c3f1aac8
- misc: update output token logic (#695) · d198791f
  zhyncs authored Jul 22, 2024
  
  d198791f
- fix: update bench serving (#694) · c07526e4
  zhyncs authored Jul 22, 2024
  
  c07526e4
21 Jul, 2024 12 commits
- docs: update README (#692) · 7b597475
  zhyncs authored Jul 22, 2024
  
  7b597475
- Support Mistral-Nemo (#691) · 5303c1ed
  Ke Bao authored Jul 22, 2024
  
  5303c1ed
- misc: recommend to use chat model for benchmark (#690) · 65bd1338
  zhyncs authored Jul 22, 2024
  
  65bd1338
- Support Deepseek MoE Model (#689) · eedc12e1
  Liangsheng Yin authored Jul 21, 2024
  
  eedc12e1
- update readme · 5a4ef2b5
  Lianmin Zheng authored Jul 21, 2024
  
  5a4ef2b5
- docs: update README (#688) · 9dab947d
  zhyncs authored Jul 21, 2024
  
  9dab947d
- Allow disabling streaming in bench (#687) · 33ee97b0
  Lianmin Zheng authored Jul 21, 2024
  
  33ee97b0
- misc: update output file logic (#686) · 6a846bb1
  zhyncs authored Jul 21, 2024
  
  6a846bb1
- feat: update bench serving (#685) · 0fdb3127
  zhyncs authored Jul 21, 2024
  
  0fdb3127
- Fix StreamExecutor.fork() losing the current role start index. (#684) · 5ad033a0
  Max Shawabkeh authored Jul 20, 2024
  
  5ad033a0
- support non-streaming benchmark (#682) · 77e592e8
  Lianmin Zheng authored Jul 20, 2024
  
  77e592e8
- Support gpt-bigcode model class (#681) · caaad53b
  Liangsheng Yin authored Jul 20, 2024
  
  caaad53b
20 Jul, 2024 1 commit
- Decouple kv (#679) · 69d19188
  Liangsheng Yin authored Jul 20, 2024
  
  69d19188