Commits · e150cf119b2e20f81775f36fd2cdf55321b7e5a8 · OpenDAS / vllm_cscc · GitLab

27 Nov, 2024 1 commit
- 1.优化medusa推理，节省cpu耗时 · 28375803
  王敏 authored Nov 27, 2024
```
2.更新medusa readme
3.解决benchmark_moe报错问题
```
  28375803
22 Nov, 2024 1 commit
- update pa tc and gc benchmark · 53910677
  zhuwenwen authored Nov 22, 2024
  
  53910677
30 Oct, 2024 1 commit
- fix the problem of poor second inference performance in triton · 510401e2
  zhuwenwen authored Oct 30, 2024
  
  510401e2
10 Oct, 2024 1 commit
- update benchmarks · 00d3d196
  zhuwenwen authored Oct 10, 2024
  
  00d3d196
26 Sep, 2024 1 commit
- support head_dim 160 and update benchmark_throughput.py · 93872128
  zhuwenwen authored Sep 26, 2024
  
  93872128
25 Sep, 2024 3 commits
- Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) · 4f1ba084
  Simon Mo authored Sep 25, 2024
  
  4f1ba084
- rename PromptInputs and inputs with backward compatibility (#8760) · 28e1299e
  Cyrus Leung authored Sep 26, 2024
  
  28e1299e
- [Core] Adding Priority Scheduling (#5958) · 6da1ab6b
  Archit Patke authored Sep 24, 2024
  
  6da1ab6b
24 Sep, 2024 2 commits
- Revert "[Core] Rename `PromptInputs` to `PromptType`, and `inputs` to `prompt`" (#8750) · 3185fb0c
  Simon Mo authored Sep 23, 2024
  
  3185fb0c
- re-implement beam search on top of vllm core (#8726) · 0250dd68
  youkaichao authored Sep 23, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  0250dd68
23 Sep, 2024 1 commit

[Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (#7701) · 86e9c8df

Lucas Wilkinson authored Sep 23, 2024

Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

86e9c8df

21 Sep, 2024 1 commit
- [Core] Rename `PromptInputs` and `inputs`(#8673) · 0057894e
  Cyrus Leung authored Sep 21, 2024
  
  0057894e
19 Sep, 2024 2 commits
- [MISC] remove engine_use_ray in benchmark_throughput.py (#8615) · 855c8ae2
  Kunshang Ji authored Sep 19, 2024
  
  855c8ae2
- [Bugfix] fixing sonnet benchmark bug in benchmark_serving.py (#8616) · c52ec5f0
  Kuntai Du authored Sep 18, 2024
  
  c52ec5f0
18 Sep, 2024 2 commits
- [CI/Build] Update Ruff version (#8469) · 9d104b5b
  Aaron Pham authored Sep 18, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  9d104b5b
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
17 Sep, 2024 1 commit
- [Benchmark] Support sample from HF datasets and image input for benchmark_serving (#8495) · 1b6de835
  Isotr0py authored Sep 17, 2024
  
  1b6de835
11 Sep, 2024 1 commit
- [Misc] Move device options to a single place (#8322) · 8baa4549
  Aarni Koskela authored Sep 11, 2024
  
  8baa4549
09 Sep, 2024 1 commit
- update version · 50ed6d0a
  zhuwenwen authored Sep 09, 2024
  
  50ed6d0a
07 Sep, 2024 1 commit
- Enable Random Prefix Caching in Serving Profiling Tool (benchmark_serving.py) (#8241) · 795b662c
  Wei-Sheng Chin authored Sep 06, 2024
  
  795b662c
06 Sep, 2024 1 commit
- [Frontend] Add --logprobs argument to `benchmark_serving.py` (#8191) · e5cab715
  afeldman-nm authored Sep 06, 2024
  
  e5cab715
05 Sep, 2024 1 commit
- update benchmark_throughput.py · ae3524c6
  zhuwenwen authored Sep 05, 2024
  
  ae3524c6
04 Sep, 2024 2 commits
- [MISC] Replace input token throughput with total token throughput (#8164) · 77d9e514
  Cody Yu authored Sep 04, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  77d9e514
- [Benchmark] Add `--async-engine` option to benchmark_throughput.py (#7964) · d4db9f53
  Nick Hill authored Sep 03, 2024
  
  d4db9f53
29 Aug, 2024 1 commit
- Add more percentiles and latencies (#7759) · 0c785d34
  Wei-Sheng Chin authored Aug 29, 2024
  
  0c785d34
27 Aug, 2024 2 commits
- [benchmark] Update TGI version (#7917) · 345be0e2
  Philipp Schmid authored Aug 28, 2024
  
  345be0e2
- [Core] Asynchronous Output Processor (#7049) · 2eedede8
  Megha Agarwal authored Aug 26, 2024
```
Co-authored-by: Alexander Matveev <alexm@neuralmagic.com>
```
  2eedede8
23 Aug, 2024 1 commit
- [Core] Add multi-step support to LLMEngine (#7789) · 9db93de2
  Alexander Matveev authored Aug 23, 2024
  
  9db93de2
22 Aug, 2024 2 commits
- [Misc] Enhance prefix-caching benchmark tool (#6568) · d3b5b980
  Jiaxin Shan authored Aug 23, 2024
  
  d3b5b980
- [Kernel] Replaced `blockReduce[...]` functions with `cub::BlockReduce` (#7233) · 7937009a
  Luka Govedič authored Aug 21, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7937009a
21 Aug, 2024 1 commit
- [misc] Add Torch profiler support (#7451) · dd53c4b0
  William Lin authored Aug 21, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  dd53c4b0
20 Aug, 2024 1 commit
- [Kernel] (1/N) Machete - Hopper Optimized Mixed Precision Linear Kernel (#7174) · 5288c06a
  Lucas Wilkinson authored Aug 20, 2024
  
  5288c06a
16 Aug, 2024 1 commit
- [Kernel] W8A16 Int8 inside FusedMoE (#7415) · 7fc23be8
  Mor Zusman authored Aug 16, 2024
  
  7fc23be8
15 Aug, 2024 1 commit
- update benchmark_throughput.py · 17928589
  zhuwenwen authored Aug 15, 2024
  
  17928589
10 Aug, 2024 1 commit
- add benchmarks to vllm and add env of VLLM_USE_FLASH_ATTN_AUTO · efb2f75f
  zhuwenwen authored Aug 10, 2024
  
  efb2f75f
09 Aug, 2024 1 commit
- [Bugfix] Fix ITL recording in serving benchmark (#7372) · 70d268a3
  Roger Wang authored Aug 09, 2024
  
  70d268a3
06 Aug, 2024 1 commit
- [Kernel] Add per-tensor and per-token AZP epilogues (#5941) · 8d59dbb0
  Luka Govedič authored Aug 06, 2024
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  8d59dbb0
03 Aug, 2024 1 commit
- update benchmraks and tests · bdac8f06
  zhuwenwen authored Aug 03, 2024
  
  bdac8f06
02 Aug, 2024 1 commit
- [Misc] Disambiguate quantized types via a new ScalarType (#6396) · a8d604ca
  Lucas Wilkinson authored Aug 02, 2024
  
  a8d604ca
01 Aug, 2024 1 commit
- update num_prompts · 75011627
  zhuwenwen authored Aug 01, 2024
  
  75011627