Commits · 614aa5120303ab09be78fb1db669da198cc43b02 · OpenDAS / vllm_cscc

01 Jul, 2024 1 commit
- [misc][cuda] use nvml to avoid accidentally cuda initialization (#6007) · 614aa512
  youkaichao authored Jun 30, 2024
  
  614aa512
30 Jun, 2024 2 commits
- [ Misc ] Refactor w8a8 to use `process_weights_after_load` (Simplify Weight Loading) (#5940) · af9ad46f
  Robert Shaw authored Jun 30, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  af9ad46f
- [Misc] Fix `get_min_capability` (#5971) · 7836fdcc
  Dipika Sikka authored Jun 30, 2024
  
  7836fdcc
29 Jun, 2024 4 commits
- [CI/Build] Add TP test for vision models (#5892) · 99397da5
  Cyrus Leung authored Jun 29, 2024
  
  99397da5
- [ CI/Build ] Added E2E Test For Compressed Tensors (#5839) · 8dbfcd35
  Robert Shaw authored Jun 29, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  8dbfcd35
- [Kernel] Raise an exception in MoE kernel if the batch size is larger then 65k (#5939) · f7dac83d
  Cody Yu authored Jun 29, 2024
  
  f7dac83d
- [Bugfix] Fix precisions in Gemma 1 (#5913) · 580353da
  Woosuk Kwon authored Jun 28, 2024
  
  580353da
28 Jun, 2024 7 commits
- Unmark more files as executable (#5962) · 5d2a1a9c
  Tyler Michael Smith authored Jun 28, 2024
  
  5d2a1a9c
- Support Deepseek-V2 (#4650) · be0b3af9
  wangding zeng authored Jun 29, 2024
```
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
```
  be0b3af9
- [ Bugfix ] Enabling Loading Models With Fused QKV/MLP on Disk with FP8 (#5921) · 2cd402e1
  Robert Shaw authored Jun 28, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  2cd402e1
- [ Misc ] Remove `fp8_shard_indexer` from Col/Row Parallel Linear (Simplify Weight Loading) (#5928) · b1852307
  Robert Shaw authored Jun 28, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  b1852307
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
- Unmark fused_moe config json file as executable (#5960) · 59326344
  Tyler Michael Smith authored Jun 28, 2024
  
  59326344
- [Core] Registry for processing model inputs (#5214) · 5cbe8d15
  Cyrus Leung authored Jun 28, 2024
```
Co-authored-by: ywang96 <ywang@roblox.com>
```
  5cbe8d15
27 Jun, 2024 6 commits
- [Kernel][ROCm][AMD] fused_moe Triton configs v2 for mi300X (#5932) · c3dde367
  Divakar Verma authored Jun 27, 2024
  
  c3dde367
- [Model] Add Gemma 2 (#5908) · 79c92c7c
  Woosuk Kwon authored Jun 27, 2024
  
  79c92c7c
- [BugFix] Fix `MLPSpeculator` handling of `num_speculative_tokens` (#5876) · 691e29ec
  Nick Hill authored Jun 27, 2024
  
  691e29ec
- [Model][Bugfix] Implicit model flags and reenable Phi-3-Vision (#5896) · 98cf2ed6
  Cyrus Leung authored Jun 28, 2024
  
  98cf2ed6
- [Bugfix] Fix img_sizes Parsing in Phi3-Vision (#5888) · 2061f0b8
  Roger Wang authored Jun 27, 2024
  
  2061f0b8
- [Model] Add base class for LoRA-supported models (#5018) · 96354d6a
  Cyrus Leung authored Jun 27, 2024
  
  96354d6a
26 Jun, 2024 1 commit
- [Bugfix] Fix embedding to support 2D inputs (#5829) · 6806998b
  Woosuk Kwon authored Jun 26, 2024
  
  6806998b
25 Jun, 2024 1 commit
- [Misc] Update `w4a16` `compressed-tensors` support to include `w8a16` (#5794) · dd248f76
  Dipika Sikka authored Jun 25, 2024
  
  dd248f76
24 Jun, 2024 1 commit
- [Bugfix] Add phi3v resize for dynamic shape and fix torchvision requirement (#5772) · edd5fe5f
  Isotr0py authored Jun 24, 2024
  
  edd5fe5f
22 Jun, 2024 1 commit
- [Model] Support Qwen-VL and Qwen-VL-Chat models with text-only inputs (#5710) · 9c62db07
  Jie Fu (傅杰) authored Jun 22, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  9c62db07
21 Jun, 2024 2 commits

[Kernel][CPU] Add Quick `gelu` to CPU (#5717) · bd620b01
Roger Wang authored Jun 20, 2024

bd620b01

[Model] MLPSpeculator speculative decoding support (#4947) · b12518d3

Joshua Rosenkranz authored Jun 20, 2024


Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Davis Wertheimer <Davis.Wertheimer@ibm.com>

b12518d3

20 Jun, 2024 3 commits
- [Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718) · 8065a7e2
  Michael Goin authored Jun 20, 2024
  
  8065a7e2
- [Bugfix] Fix the CUDA version check for FP8 support in the CUTLASS kernels (#5715) · 3f3b6b21
  Tyler Michael Smith authored Jun 20, 2024
  
  3f3b6b21
- [Model] Port over CLIPVisionModel for VLMs (#5591) · ad137cd1
  Roger Wang authored Jun 20, 2024
  
  ad137cd1
19 Jun, 2024 3 commits
- [Misc] Add per channel support for static activation quantization; update w8a8... · 4a30d7e3
  Dipika Sikka authored Jun 19, 2024
```
[Misc] Add per channel support for static activation quantization; update w8a8 schemes to share base classes (#5650)
```
  4a30d7e3
- [Model] Add FP8 kv cache for Qwen2 (#5656) · da971ec7
  Michael Goin authored Jun 19, 2024
  
  da971ec7
- [Bugfix] Fix Phi-3 Long RoPE scaling implementation (#5628) · 59a1eb59
  Shukant Pal authored Jun 18, 2024
  
  59a1eb59
18 Jun, 2024 6 commits
- [Bugfix] Fix for inconsistent behaviour related to sampling and repetition penalties (#5639) · 8a173382
  Thomas Parnell authored Jun 18, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  8a173382
- [Model] LoRA support added for command-r (#5178) · 07feecde
  sergey-tinkoff authored Jun 18, 2024
  
  07feecde
- [Misc] Add channel-wise quantization support for w8a8 dynamic per token... · 95db455e
  Dipika Sikka authored Jun 18, 2024
```
[Misc] Add channel-wise quantization support for w8a8 dynamic per token activation quantization (#5542)
```
  95db455e
- [Misc] Remove import from transformers logging (#5625) · f0cc0e68
  Chang Su authored Jun 18, 2024
  
  f0cc0e68
- [Model] Initialize Phi-3-vision support (#4986) · daef218b
  Isotr0py authored Jun 18, 2024
  
  daef218b
- [Speculative Decoding 1/2 ] Add typical acceptance sampling as one of the... · fa9e3852
  sroy745 authored Jun 17, 2024
```
[Speculative Decoding 1/2 ] Add typical acceptance sampling as one of the sampling techniques in the verifier (#5131)
```
  fa9e3852
17 Jun, 2024 2 commits
- [Hardware][Intel GPU] Add Intel GPU(XPU) inference backend (#3814) · 728c4c8a
  Kunshang Ji authored Jun 18, 2024
```
Co-authored-by: Jiang Li <jiang1.li@intel.com>
Co-authored-by: Abhilash Majumder <abhilash.majumder@intel.com>
Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com>
```
  728c4c8a
- [Kernel] `compressed-tensors` marlin 24 support (#5435) · 890d8d96
  Dipika Sikka authored Jun 17, 2024
  
  890d8d96