Commits · 98a42e70782ec38c1f0f20d0226d0f71147d1ac2 · OpenDAS / vllm_cscc · GitLab

29 Mar, 2024 1 commit
- [Benchmark] Change mii to use persistent deployment and support tensor parallel (#3628) · 98a42e70
  Yile (Michael) Gu authored Mar 28, 2024
  
  98a42e70
28 Mar, 2024 1 commit
- [2/N] Chunked prefill data update (#3538) · b51c1cc9
  SangBin Cho authored Mar 29, 2024
  
  b51c1cc9
27 Mar, 2024 2 commits
- feat(benchmarks): Add Prefix Caching Benchmark to Serving Benchmark (#3277) · 45b6ef65
  Roger Wang authored Mar 27, 2024
  
  45b6ef65
- [Misc] add the "download-dir" option to the latency/throughput benchmarks (#3621) · 19569314
  AmadeusChan authored Mar 27, 2024
  
  19569314
25 Mar, 2024 1 commit
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
16 Mar, 2024 2 commits
- [Misc] fix line length for entire codebase (#3444) · 8e67598a
  Simon Mo authored Mar 16, 2024
  
  8e67598a
- Replace `lstrip()` with `removeprefix()` to fix Ruff linter warning (#2958) · 14e3f9a1
  Ronen Schaffer authored Mar 16, 2024
  
  14e3f9a1
14 Mar, 2024 1 commit
- [Kernel] change benchmark script so that result can be directly used; tune moe... · 8fe83865
  youkaichao authored Mar 14, 2024
```
[Kernel] change benchmark script so that result can be directly used; tune moe kernel in A100/H100 with tp=2,4,8 (#3389)
```
  8fe83865
13 Mar, 2024 1 commit
- Add batched RoPE kernel (#3095) · 7e9bd08f
  Terry authored Mar 13, 2024
  
  7e9bd08f
08 Mar, 2024 1 commit
- [Minor Fix] Fix comments in benchmark_serving (#3252) · 1ece1ae8
  TianYu GUO authored Mar 08, 2024
  
  1ece1ae8
04 Mar, 2024 4 commits
- Fix the openai benchmarking requests to work with latest OpenAI apis (#2992) · 9a4548ba
  Chen Wang authored Mar 04, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  9a4548ba
- enable --gpu-memory-utilization in benchmark_throughput.py (#3175) · 9cbc7e5f
  Allen.Dou authored Mar 05, 2024
```
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
```
  9cbc7e5f
- [Minor Fix] Remove unused code in benchmark_prefix_caching.py (#3171) · 901cf4c5
  TianYu GUO authored Mar 04, 2024
  
  901cf4c5
- Make it easy to profile workers with nsight (#3162) · 17c3103c
  Philipp Moritz authored Mar 03, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  17c3103c
03 Mar, 2024 1 commit
- [FIX] Fix styles in automatic prefix caching & add a automatic prefix caching benchmark (#3158) · 996d095c
  Zhuohan Li authored Mar 03, 2024
  
  996d095c
02 Mar, 2024 1 commit

Add Automatic Prefix Caching (#2762) · ce4f5a29

Sage Moore authored Mar 02, 2024


Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

ce4f5a29

26 Feb, 2024 1 commit
- Optimize Triton MoE Kernel (#2979) · cfc15a10
  Philipp Moritz authored Feb 26, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
```
  cfc15a10
22 Feb, 2024 2 commits
- chore(vllm): codespell for spell checking (#2820) · 93dc5a28
  Massimiliano Pronesti authored Feb 22, 2024
  
  93dc5a28
- Update comment (#2934) · d7f39648
  Ronen Schaffer authored Feb 22, 2024
  
  d7f39648
13 Feb, 2024 1 commit
- Serving Benchmark Refactoring (#2433) · a4211a4d
  Roger Wang authored Feb 12, 2024
  
  a4211a4d
05 Feb, 2024 1 commit
- [Minor] Fix benchmark_latency script (#2765) · 72d3a30c
  Woosuk Kwon authored Feb 05, 2024
  
  72d3a30c
01 Feb, 2024 1 commit
- Remove hardcoded `device="cuda" ` to support more devices (#2503) · 96b6f475
  Kunshang Ji authored Feb 02, 2024
```
Co-authored-by: Jiang Li <jiang1.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  96b6f475
29 Jan, 2024 1 commit

Support FP8-E5M2 KV Cache (#2279) · 9090bf02

zhaoyang-star authored Jan 29, 2024


Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

9090bf02

23 Jan, 2024 2 commits
- lint: format all python file instead of just source code (#2567) · 1e4277d2
  Simon Mo authored Jan 23, 2024
  
  1e4277d2
- [Experimental] Add multi-LoRA support (#1804) · 9b945daa
  Antoni Baum authored Jan 24, 2024
```
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  9b945daa
22 Jan, 2024 1 commit
- Fix progress bar and allow HTTPS in `benchmark_serving.py` (#2552) · 63e835cb
  Harry Mellor authored Jan 22, 2024
  
  63e835cb
19 Jan, 2024 1 commit
- Support OpenAI API server in `benchmark_serving.py` (#2172) · 2709c000
  Harry Mellor authored Jan 19, 2024
  
  2709c000
17 Dec, 2023 1 commit

Optimize model execution with CUDA graph (#1926) · 37ca5581

Woosuk Kwon authored Dec 16, 2023


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

37ca5581

15 Dec, 2023 1 commit
- Add GPTQ support (#916) · 0fbfc4b8
  CHU Tianxiang authored Dec 15, 2023
  
  0fbfc4b8
11 Dec, 2023 1 commit
- Fix latency benchmark script (#2035) · 5dd80d37
  Woosuk Kwon authored Dec 11, 2023
  
  5dd80d37
10 Dec, 2023 1 commit
- Replace head_mapping params with num_kv_heads to attention kernel. (#1997) · dacaf5a4
  wbn authored Dec 11, 2023
```
Co-authored-by: wangguoya <wangguoya@baidu.com>
Co-authored-by: Yang Zhao <zhaoyangstar@foxmail.com>
```
  dacaf5a4
06 Dec, 2023 1 commit
- Save pytorch profiler output for latency benchmark (#1871) · 05ff90b6
  Antoni Baum authored Dec 05, 2023
```
* Save profiler output

* Apply feedback from code review
```
  05ff90b6
30 Nov, 2023 3 commits
- Support max-model-len argument for throughput benchmark (#1858) · 8d8c2f6f
  aisensiy authored Dec 01, 2023
  
  8d8c2f6f
- Remove max_num_seqs in latency benchmark script (#1855) · 51d3cb95
  Woosuk Kwon authored Nov 30, 2023
  
  51d3cb95
- Add profile option to latency benchmark script (#1839) · e74b1736
  Woosuk Kwon authored Nov 29, 2023
  
  e74b1736
24 Nov, 2023 1 commit
- [Build] Avoid building too many extensions (#1624) · e0c6f556
  Yanming W authored Nov 23, 2023
  
  e0c6f556
20 Nov, 2023 1 commit
- Migrate linter from `pylint` to `ruff` (#1665) · 5ffc0d13
  Simon Mo authored Nov 20, 2023
  
  5ffc0d13
17 Nov, 2023 1 commit
- [Minor] Fix comment (#1704) · dcc543a2
  Zhuofan authored Nov 18, 2023
  
  dcc543a2
14 Nov, 2023 1 commit
- Add DeepSpeed MII backend to benchmark script (#1649) · 660a7fcf
  Woosuk Kwon authored Nov 14, 2023
  
  660a7fcf
22 Oct, 2023 1 commit

Support SqueezeLLM (#1326) · 1f24755b

chooper1 authored Oct 22, 2023


Co-authored-by: squeeze-ai-lab <squeezeailab.bair@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

1f24755b