Commits · 679ebcbbdc37e129b16d305660514dfafda0f7ca · change / sglang

27 Jul, 2024 1 commit
- Deepseek v2 support (#693) · 679ebcbb
  Liangsheng Yin authored Jul 26, 2024
  
  679ebcbb
26 Jul, 2024 4 commits
- misc: format (#741) · 6b32bb1c
  Yineng Zhang authored Jul 26, 2024
  
  6b32bb1c
- feat: support token ids in /v1/completions (#736) · 40facad5
  Toshiki Kataoka authored Jul 26, 2024
  
  40facad5
- fix /generate without sampling_params (#734) · da504445
  Toshiki Kataoka authored Jul 26, 2024
  
  da504445
- fix: small bug for llama-405b fp16 (#733) · 252e0f7b
  Ying Sheng authored Jul 25, 2024
  
  252e0f7b
25 Jul, 2024 4 commits
- Fix bugs (fp8 checkpoints, triton cache manager) (#729) · 8fbba3de
  Ying Sheng authored Jul 25, 2024
  
  8fbba3de
- Revert "fix: fp8 config" (#728) · ae0f6130
  Ying Sheng authored Jul 25, 2024
  
  ae0f6130
- Fix dockerfile and triton cache manager (#720) · 04ec6ba2
  Liangsheng Yin authored Jul 25, 2024
  
  04ec6ba2
- fix: fp8 config (#723) · d63f13c1
  Ying Sheng authored Jul 25, 2024
  
  d63f13c1
24 Jul, 2024 7 commits
- Improve benchmark scripts (#717) · 30d8e130
  Ying Sheng authored Jul 24, 2024
  
  30d8e130
- fix: llama 3.1 405b fp8 (#714) · e17deb27
  Yineng Zhang authored Jul 25, 2024
  
  e17deb27
- format · 83d2b30d
  Ying Sheng authored Jul 24, 2024
  
  83d2b30d
- Fix prefill size (#711) · 4367f4bb
  Ying Sheng authored Jul 24, 2024
  
  4367f4bb
- Update schedule_heuristic.py · 00e4baa7
  Lianmin Zheng authored Jul 24, 2024
  
  00e4baa7
- Auto adjust new ratio (#708) · 4cd64b8e
  Liangsheng Yin authored Jul 23, 2024
  
  4cd64b8e
- Fix multi-node deadlock (#709) · 01d66ae2
  Lianmin Zheng authored Jul 23, 2024
  
  01d66ae2
23 Jul, 2024 5 commits
- Reduce hardcoded logic of kernel usage (#707) · a523a3c1
  Mingyi authored Jul 23, 2024
  
  a523a3c1
- Update vllm version to support llama3.1 (#705) · 444a0244
  Ying Sheng authored Jul 23, 2024
  
  444a0244
- Use min new token ratio at start (#701) · 26868443
  Liangsheng Yin authored Jul 23, 2024
  
  26868443
- Fix hf config loading (#702) · 824a77d0
  Ke Bao authored Jul 24, 2024
  
  824a77d0
- Fix flashinfer (#700) · cf99eab7
  Ying Sheng authored Jul 23, 2024
  
  cf99eab7
22 Jul, 2024 1 commit
- Tune params (#696) · c3f1aac8
  Ying Sheng authored Jul 22, 2024
  
  c3f1aac8
21 Jul, 2024 4 commits
- Support Mistral-Nemo (#691) · 5303c1ed
  Ke Bao authored Jul 22, 2024
  
  5303c1ed
- Support Deepseek MoE Model (#689) · eedc12e1
  Liangsheng Yin authored Jul 21, 2024
  
  eedc12e1
- support non-streaming benchmark (#682) · 77e592e8
  Lianmin Zheng authored Jul 20, 2024
  
  77e592e8
- Support gpt-bigcode model class (#681) · caaad53b
  Liangsheng Yin authored Jul 20, 2024
  
  caaad53b
20 Jul, 2024 11 commits
- Decouple kv (#679) · 69d19188
  Liangsheng Yin authored Jul 20, 2024
  
  69d19188
- Fallback when sampling failed (#678) · 0ac94c36
  Ke Bao authored Jul 21, 2024
  
  0ac94c36
- Fix illegal tokens during sampling (#676) · f424e76d
  Liangsheng Yin authored Jul 20, 2024
  
  f424e76d
- Fix cuda graph with flashinfer (#675) · 490a1f39
  Lianmin Zheng authored Jul 20, 2024
  
  490a1f39
- refactor model loader: initial refactor (#664) · 06487f12
  Ying Sheng authored Jul 20, 2024
  
  06487f12
- Revert "Temporary fix invalid sample results" (#673) · 39c57317
  Liangsheng Yin authored Jul 20, 2024
  
  39c57317
- Support random dataset in bench_serving.py (#669) · 35759efa
  Lianmin Zheng authored Jul 20, 2024
  
  35759efa
- Temporary fix invalid sample results (#668) · 8f4b1559
  Liangsheng Yin authored Jul 20, 2024
  
  8f4b1559
- Update OpenAI API (#667) · e3046ea3
  Mingyi authored Jul 19, 2024
  
  e3046ea3
- Add support for OpenAI API parallel sampling (#640) · 49c5e0ec
  yichuan~ authored Jul 20, 2024
  
  49c5e0ec
- Fix kill process util (#666) · ec2150b2
  Ke Bao authored Jul 20, 2024
  
  ec2150b2
19 Jul, 2024 3 commits
- Fix jump forward when streaming (#665) · 7620cd37
  Liangsheng Yin authored Jul 19, 2024
  
  7620cd37
- Improve docs (#662) · e87c7fd5
  Ying Sheng authored Jul 19, 2024
  
  e87c7fd5
- Update Readme (#660) · 51fda143
  Ying Sheng authored Jul 19, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  51fda143