Commits · 66ded030677c7a0ca696f8d64e41637f4a358c00 · OpenDAS / vllm_cscc

17 Apr, 2024 1 commit
- [Bugfix] fix output parsing error for trtllm backend (#4137) · fe3b5bbc
  Elinx authored Apr 17, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  fe3b5bbc
12 Apr, 2024 1 commit
- [Doc] Add typing hints / mypy types cleanup (#3816) · c2b4a1bc
  Michael Feil authored Apr 11, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  c2b4a1bc
11 Apr, 2024 2 commits
- [Misc] Add indirection layer for custom ops (#3913) · e9da5a40
  Kunshang Ji authored Apr 11, 2024
  
  e9da5a40
- [Core][5/N] Fully working chunked prefill e2e (#3884) · 67b4221a
  SangBin Cho authored Apr 11, 2024
  
  67b4221a
10 Apr, 2024 1 commit
- [Benchmark] Add cpu options to bench scripts (#3915) · c013d32c
  Zedong Peng authored Apr 10, 2024
  
  c013d32c
06 Apr, 2024 1 commit
- [CI/Benchmark] add more iteration and use median for robust latency benchmark (#3889) · e4be7d70
  youkaichao authored Apr 06, 2024
  
  e4be7d70
04 Apr, 2024 2 commits
- [Benchmark] Refactor sample_requests in benchmark_throughput (#3613) · b7782002
  TianYu GUO authored Apr 04, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  b7782002
- [Bugfix] Fix args in benchmark_serving (#3836) · 819a309c
  Chang Su authored Apr 04, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  819a309c
03 Apr, 2024 1 commit

Enable scaled FP8 (e4m3fn) KV cache on ROCm (AMD GPU) (#3290) · 2ff767b5

Adrian Abeyta authored Apr 03, 2024


Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: HaiShaw <hixiao@gmail.com>
Co-authored-by: AdrianAbeyta <Adrian.Abeyta@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: root <root@gt-pla-u18-08.pla.dcgpu>
Co-authored-by: mawong-amd <156021403+mawong-amd@users.noreply.github.com>
Co-authored-by: ttbachyinsda <ttbachyinsda@outlook.com>
Co-authored-by: guofangze <guofangze@kuaishou.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: jacobthebanana <50071502+jacobthebanana@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

2ff767b5

01 Apr, 2024 1 commit
- [Misc] Fix Benchmark TTFT Calculation for Chat Completions (#3768) · ccb58b23
  Roger Wang authored Apr 01, 2024
  
  ccb58b23
29 Mar, 2024 1 commit
- [Benchmark] Change mii to use persistent deployment and support tensor parallel (#3628) · 98a42e70
  Yile (Michael) Gu authored Mar 28, 2024
  
  98a42e70
28 Mar, 2024 1 commit
- [2/N] Chunked prefill data update (#3538) · b51c1cc9
  SangBin Cho authored Mar 29, 2024
  
  b51c1cc9
27 Mar, 2024 2 commits
- feat(benchmarks): Add Prefix Caching Benchmark to Serving Benchmark (#3277) · 45b6ef65
  Roger Wang authored Mar 27, 2024
  
  45b6ef65
- [Misc] add the "download-dir" option to the latency/throughput benchmarks (#3621) · 19569314
  AmadeusChan authored Mar 27, 2024
  
  19569314
25 Mar, 2024 1 commit
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
16 Mar, 2024 2 commits
- [Misc] fix line length for entire codebase (#3444) · 8e67598a
  Simon Mo authored Mar 16, 2024
  
  8e67598a
- Replace `lstrip()` with `removeprefix()` to fix Ruff linter warning (#2958) · 14e3f9a1
  Ronen Schaffer authored Mar 16, 2024
  
  14e3f9a1
14 Mar, 2024 1 commit
- [Kernel] change benchmark script so that result can be directly used; tune moe... · 8fe83865
  youkaichao authored Mar 14, 2024
```
[Kernel] change benchmark script so that result can be directly used; tune moe kernel in A100/H100 with tp=2,4,8 (#3389)
```
  8fe83865
13 Mar, 2024 1 commit
- Add batched RoPE kernel (#3095) · 7e9bd08f
  Terry authored Mar 13, 2024
  
  7e9bd08f
08 Mar, 2024 1 commit
- [Minor Fix] Fix comments in benchmark_serving (#3252) · 1ece1ae8
  TianYu GUO authored Mar 08, 2024
  
  1ece1ae8
04 Mar, 2024 4 commits
- Fix the openai benchmarking requests to work with latest OpenAI apis (#2992) · 9a4548ba
  Chen Wang authored Mar 04, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  9a4548ba
- enable --gpu-memory-utilization in benchmark_throughput.py (#3175) · 9cbc7e5f
  Allen.Dou authored Mar 05, 2024
```
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
```
  9cbc7e5f
- [Minor Fix] Remove unused code in benchmark_prefix_caching.py (#3171) · 901cf4c5
  TianYu GUO authored Mar 04, 2024
  
  901cf4c5
- Make it easy to profile workers with nsight (#3162) · 17c3103c
  Philipp Moritz authored Mar 03, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  17c3103c
03 Mar, 2024 1 commit
- [FIX] Fix styles in automatic prefix caching & add a automatic prefix caching benchmark (#3158) · 996d095c
  Zhuohan Li authored Mar 03, 2024
  
  996d095c
02 Mar, 2024 1 commit

Add Automatic Prefix Caching (#2762) · ce4f5a29

Sage Moore authored Mar 02, 2024


Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

ce4f5a29

26 Feb, 2024 1 commit
- Optimize Triton MoE Kernel (#2979) · cfc15a10
  Philipp Moritz authored Feb 26, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
```
  cfc15a10
22 Feb, 2024 2 commits
- chore(vllm): codespell for spell checking (#2820) · 93dc5a28
  Massimiliano Pronesti authored Feb 22, 2024
  
  93dc5a28
- Update comment (#2934) · d7f39648
  Ronen Schaffer authored Feb 22, 2024
  
  d7f39648
13 Feb, 2024 1 commit
- Serving Benchmark Refactoring (#2433) · a4211a4d
  Roger Wang authored Feb 12, 2024
  
  a4211a4d
05 Feb, 2024 1 commit
- [Minor] Fix benchmark_latency script (#2765) · 72d3a30c
  Woosuk Kwon authored Feb 05, 2024
  
  72d3a30c
01 Feb, 2024 1 commit
- Remove hardcoded `device="cuda" ` to support more devices (#2503) · 96b6f475
  Kunshang Ji authored Feb 02, 2024
```
Co-authored-by: Jiang Li <jiang1.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  96b6f475
29 Jan, 2024 1 commit

Support FP8-E5M2 KV Cache (#2279) · 9090bf02

zhaoyang-star authored Jan 29, 2024


Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

9090bf02

23 Jan, 2024 2 commits
- lint: format all python file instead of just source code (#2567) · 1e4277d2
  Simon Mo authored Jan 23, 2024
  
  1e4277d2
- [Experimental] Add multi-LoRA support (#1804) · 9b945daa
  Antoni Baum authored Jan 24, 2024
```
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  9b945daa
22 Jan, 2024 1 commit
- Fix progress bar and allow HTTPS in `benchmark_serving.py` (#2552) · 63e835cb
  Harry Mellor authored Jan 22, 2024
  
  63e835cb
19 Jan, 2024 1 commit
- Support OpenAI API server in `benchmark_serving.py` (#2172) · 2709c000
  Harry Mellor authored Jan 19, 2024
  
  2709c000
17 Dec, 2023 1 commit

Optimize model execution with CUDA graph (#1926) · 37ca5581

Woosuk Kwon authored Dec 16, 2023


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

37ca5581

15 Dec, 2023 1 commit
- Add GPTQ support (#916) · 0fbfc4b8
  CHU Tianxiang authored Dec 15, 2023
  
  0fbfc4b8
11 Dec, 2023 1 commit
- Fix latency benchmark script (#2035) · 5dd80d37
  Woosuk Kwon authored Dec 11, 2023
  
  5dd80d37