Commits · bcc6a09b63aeb3efce964b54a756d431e580aebc · OpenDAS / vllm_cscc · GitLab

29 Jun, 2024 7 commits
- [Bugfix][CI/Build][Hardware][AMD] Install matching torchvision to fix AMD tests (#5949) · 9def1066
  Matt Wong authored Jun 29, 2024
  
  9def1066
- [CI/Build] Add TP test for vision models (#5892) · 99397da5
  Cyrus Leung authored Jun 29, 2024
  
  99397da5
- [ CI/Build ] Added E2E Test For Compressed Tensors (#5839) · 8dbfcd35
  Robert Shaw authored Jun 29, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  8dbfcd35
- [Bugfix] Support `eos_token_id` from `config.json` (#5954) · 51e971d3
  Cyrus Leung authored Jun 29, 2024
  
  51e971d3
- [Bugfix] Fix precisions in Gemma 1 (#5913) · 580353da
  Woosuk Kwon authored Jun 28, 2024
  
  580353da
- [Kernel] Add punica dimensions for Granite 3b and 8b (#5930) · ba499444
  Joe Runde authored Jun 28, 2024
```
Signed-off-by: Joe Runde <joe@joerun.de>
```
  ba499444
- [Misc] Extend vLLM Metrics logging API (#5925) · 906a19cd
  William Lin authored Jun 28, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  906a19cd
28 Jun, 2024 7 commits
- [Kernel] Flashinfer for prefill & decode, with Cudagraph support for decode (#4628) · 7041de43
  Lily Liu authored Jun 28, 2024
```
Co-authored-by: LiuXiaoxuanPKU &lt;llilyliupku@gmail.com&gt;, bong-furiosa <bongwon.jang@furiosa.ai>
```
  7041de43
- [Bugfix] Fix compute datatype for cutlass 3.x epilogues (#5931) · 6a2d659d
  Tyler Michael Smith authored Jun 28, 2024
  
  6a2d659d
- [Spec Decode] Introduce DraftModelRunner (#5799) · b2c62023
  Cody Yu authored Jun 28, 2024
  
  b2c62023
- [Distributed] Make it clear that % should not be in tensor dict keys. (#5927) · b90d8cd8
  xwjiang2010 authored Jun 28, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
```
  b90d8cd8
- [CI/Build] [2/3] Reorganize entrypoints tests (#5904) · 3b752a65
  Cyrus Leung authored Jun 28, 2024
  
  3b752a65
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
- [Core] Registry for processing model inputs (#5214) · 5cbe8d15
  Cyrus Leung authored Jun 28, 2024
```
Co-authored-by: ywang96 <ywang@roblox.com>
```
  5cbe8d15
27 Jun, 2024 3 commits
- [CI/Build] Fix Args for `_get_logits_warper` in Sampler Test (#5922) · 736ed388
  Roger Wang authored Jun 27, 2024
  
  736ed388
- [CI/Build] [1/3] Reorganize entrypoints tests (#5526) · e9d32d07
  Cyrus Leung authored Jun 27, 2024
  
  e9d32d07
- [VLM][Bugfix] Make sure that `multi_modal_kwargs` is broadcasted properly (#5880) · d12af207
  xwjiang2010 authored Jun 27, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
```
  d12af207
26 Jun, 2024 6 commits
- [Frontend] Add tokenize/detokenize endpoints (#5054) · c54269d9
  sasha0552 authored Jun 26, 2024
  
  c54269d9
- [Kernel] Adding bias epilogue support for `cutlass_scaled_mm` (#5560) · 5bfd1bbc
  Luka Govedič authored Jun 26, 2024
```
Co-authored-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  5bfd1bbc
- [CI/Build] Refactor image test assets (#5821) · 6984c02a
  Cyrus Leung authored Jun 26, 2024
  
  6984c02a
- [bugfix][distributed] fix shm broadcast when the queue size is full (#5801) · 515080ad
  youkaichao authored Jun 25, 2024
  
  515080ad
- [Core] Refactor Worker and ModelRunner to consolidate control plane communication (#5408) · dda48115
  Stephanie Wang authored Jun 25, 2024
```
Signed-off-by: Stephanie Wang <swang@cs.berkeley.edu>
Signed-off-by: Stephanie <swang@anyscale.com>
Co-authored-by: Stephanie <swang@anyscale.com>
```
  dda48115
- [CI/Build] Add E2E tests for MLPSpeculator (#5791) · c2a8ac75
  Thomas Parnell authored Jun 26, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  c2a8ac75
25 Jun, 2024 5 commits
- [Hardware][AMD][CI/Build][Doc] Upgrade to ROCm 6.1, Dockerfile improvements, test fixes (#5422) · dd793d1d
  Matt Wong authored Jun 25, 2024
  
  dd793d1d
- [Misc] Update `w4a16` `compressed-tensors` support to include `w8a16` (#5794) · dd248f76
  Dipika Sikka authored Jun 25, 2024
  
  dd248f76
- [CI/Build] Add unit testing for FlexibleArgumentParser (#5798) · d9b34bae
  Michael Goin authored Jun 25, 2024
  
  d9b34bae
- [Core] Add fault tolerance for `RayTokenizerGroupPool` (#5748) · 67882dbb
  Antoni Baum authored Jun 25, 2024
  
  67882dbb
- [Speculative Decoding] Support draft model on different tensor-parallel size... · 2ce5d668
  Woo-Yeon Lee authored Jun 25, 2024
```
 [Speculative Decoding] Support draft model on different tensor-parallel size than target model (#5414)
```
  2ce5d668
24 Jun, 2024 1 commit
- [Bugfix] Add phi3v resize for dynamic shape and fix torchvision requirement (#5772) · edd5fe5f
  Isotr0py authored Jun 24, 2024
  
  edd5fe5f
23 Jun, 2024 1 commit
- [Distributed] Add send and recv helpers (#5719) · 5d4d9053
  Murali Andoorveedu authored Jun 23, 2024
  
  5d4d9053
21 Jun, 2024 6 commits
- [LoRA] Add support for pinning lora adapters in the LRU cache (#5603) · f5dda63e
  rohithkrn authored Jun 21, 2024
  
  f5dda63e
- [Core][Distributed] add shm broadcast (#5399) · d9a252bc
  youkaichao authored Jun 20, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  d9a252bc
- [Bugfix] Add fully sharded layer for QKVParallelLinearWithLora (#5665) · 67005a07
  Jee Li authored Jun 21, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  67005a07
- [BugFix] Fix test_phi3v.py (#5725) · c35e4a3d
  Chang Su authored Jun 20, 2024
  
  c35e4a3d
- [Kernel] Add punica dimension for Qwen2 LoRA (#5441) · 1f567421
  Jinzhen Lin authored Jun 21, 2024
  
  1f567421
- [Model] MLPSpeculator speculative decoding support (#4947) · b12518d3
  Joshua Rosenkranz authored Jun 20, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Davis Wertheimer <Davis.Wertheimer@ibm.com>
```
  b12518d3
20 Jun, 2024 2 commits
- [Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718) · 8065a7e2
  Michael Goin authored Jun 20, 2024
  
  8065a7e2
- [Misc] Improve conftest (#5681) · 3730a1c8
  Cyrus Leung authored Jun 20, 2024
  
  3730a1c8
19 Jun, 2024 2 commits
- [Misc] Add per channel support for static activation quantization; update w8a8... · 4a30d7e3
  Dipika Sikka authored Jun 19, 2024
```
[Misc] Add per channel support for static activation quantization; update w8a8 schemes to share base classes (#5650)
```
  4a30d7e3
- [Bugfix] AsyncLLMEngine hangs with asyncio.run (#5654) · 78687504
  zifeitong authored Jun 19, 2024
  
  78687504