Commits · 2b4c64627727eaf8c14c56d3695c53fd9832a084 · zhaoyu6 / sglang

"vscode:/vscode.git/clone" did not exist on "123c7319d2e3e7dd9e1b22a4b1c562bd3a82e3b9"

20 Jul, 2024 11 commits
- Update version to 0.1.22 (#677) · 2b4c6462
  Ying Sheng authored Jul 20, 2024
  
  2b4c6462
- Fix illegal tokens during sampling (#676) · f424e76d
  Liangsheng Yin authored Jul 20, 2024
  
  f424e76d
- Fix cuda graph with flashinfer (#675) · 490a1f39
  Lianmin Zheng authored Jul 20, 2024
  
  490a1f39
- refactor model loader: initial refactor (#664) · 06487f12
  Ying Sheng authored Jul 20, 2024
  
  06487f12
- Revert "Temporary fix invalid sample results" (#673) · 39c57317
  Liangsheng Yin authored Jul 20, 2024
  
  39c57317
- Fix random dataset (#671) · 9592a1f3
  Lianmin Zheng authored Jul 20, 2024
  
  9592a1f3
- Support random dataset in bench_serving.py (#669) · 35759efa
  Lianmin Zheng authored Jul 20, 2024
  
  35759efa
- Temporary fix invalid sample results (#668) · 8f4b1559
  Liangsheng Yin authored Jul 20, 2024
  
  8f4b1559
- Update OpenAI API (#667) · e3046ea3
  Mingyi authored Jul 19, 2024
  
  e3046ea3
- Add support for OpenAI API parallel sampling (#640) · 49c5e0ec
  yichuan~ authored Jul 20, 2024
  
  49c5e0ec
- Fix kill process util (#666) · ec2150b2
  Ke Bao authored Jul 20, 2024
  
  ec2150b2
19 Jul, 2024 12 commits
- Fix jump forward when streaming (#665) · 7620cd37
  Liangsheng Yin authored Jul 19, 2024
  
  7620cd37
- Add benchmark instructions (#663) · 11c8efff
  Ying Sheng authored Jul 19, 2024
  
  11c8efff
- Improve docs (#662) · e87c7fd5
  Ying Sheng authored Jul 19, 2024
  
  e87c7fd5
- feat: update check env (#661) · 630479c3
  zhyncs authored Jul 20, 2024
  
  630479c3
- Update Readme (#660) · 51fda143
  Ying Sheng authored Jul 19, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  51fda143
- misc: update SGLang package description (#659) · dc4e4a6a
  zhyncs authored Jul 20, 2024
  
  dc4e4a6a
- refactor model loader [unreachable code]: initial refactor (#655) · 2d96da81
  Ying Sheng authored Jul 19, 2024
  
  2d96da81
- feat: add benchmark serving (#657) · c126a6cc
  zhyncs authored Jul 20, 2024
  
  c126a6cc
- perf: reduce ttft and itl with stream_interval 1 (#658) · ac971ff6
  zhyncs authored Jul 20, 2024
  
  ac971ff6
- Remove cached triton launcher (#656) · e1792cca
  Lianmin Zheng authored Jul 18, 2024
  
  e1792cca
- `TokenizerManager.context_len` should inherit from `server_args.conte… (#654) · 1b7adbb5
  shrirajh authored Jul 19, 2024
  
  1b7adbb5
- Detokenize incrementally when streaming (#653) · a9ef49c1
  Liangsheng Yin authored Jul 18, 2024
  
  a9ef49c1
18 Jul, 2024 6 commits
- Remove useless variables in infer_batch.py (#651) · 21ba3a88
  Ying Sheng authored Jul 18, 2024
  
  21ba3a88
- fix: resolve lint error (#650) · 9c5cac24
  zhyncs authored Jul 18, 2024
  
  9c5cac24
- fix: set ulimit -n 65535 (#647) · b050d928
  zhyncs authored Jul 18, 2024
  
  b050d928
- misc: rm rpyc from PACKAGE_LIST (#649) · 6a4dc996
  zhyncs authored Jul 18, 2024
  
  6a4dc996
- Remove the dependency of rpyc (#646) · d774acad
  Mingyi authored Jul 18, 2024
  
  d774acad
- feat: add check_env (#645) · d93388da
  zhyncs authored Jul 18, 2024
  
  d93388da
17 Jul, 2024 6 commits
- Increase the capacity of the memory pool (#643) · 476584cb
  Ying Sheng authored Jul 17, 2024
  
  476584cb
- Move `global_server_args_dict` (#642) · abd5385a
  Liangsheng Yin authored Jul 17, 2024
  
  abd5385a
- Flashinfer sample kernel (#617) · 3de2f30a
  Liangsheng Yin authored Jul 17, 2024
  
  3de2f30a
- misc: add pre-commit config (#637) · 2e341cd4
  zhyncs authored Jul 18, 2024
  
  2e341cd4
- feat: support internlm2 (#636) · a8552cb1
  zhyncs authored Jul 17, 2024
  
  a8552cb1
- clean up step function (#635) · a470e60c
  Ying Sheng authored Jul 16, 2024
  
  a470e60c
16 Jul, 2024 5 commits
- Fix vertexai (#633) · 5ff60eda
  Liangsheng Yin authored Jul 16, 2024
  
  5ff60eda
- Add support for VertexAI safety settings (#624) · c1930022
  Aidan Cooper authored Jul 16, 2024
  
  c1930022
- Add qwen2 tie word embedding (#630) · fe3be159
  ylying authored Jul 17, 2024
  
  fe3be159
- Disable NCCL_NVLS by default (#631) · 0aa189f1
  Ying Sheng authored Jul 16, 2024
  
  0aa189f1
- Fix model forward grad (#628) · c9ee3d35
  Liangsheng Yin authored Jul 15, 2024
  
  c9ee3d35