Commits · 4851c202bbfb5dadc5a3b3b4409d3f083f4e8e90 · OpenDAS / vllm_cscc · GitLab

11 Sep, 2024 3 commits
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4
- [Model][VLM] Add Qwen2-VL model support (#7905) · 3b7fea77
  Yang Fan authored Sep 12, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b7fea77
- [Hardware][NV] Add support for ModelOpt static scaling checkpoints. (#6112) · efcf946a
  Pavani Majety authored Sep 10, 2024
  
  efcf946a
10 Sep, 2024 1 commit
- [Misc] Fused MoE Marlin support for GPTQ (#8217) · 6cd5e5b0
  Dipika Sikka authored Sep 09, 2024
  
  6cd5e5b0
09 Sep, 2024 1 commit
- [Misc] GPTQ Activation Ordering (#8135) · c7cb5c33
  Kyle Sayers authored Sep 09, 2024
  
  c7cb5c33
06 Sep, 2024 2 commits
- [Misc] Remove `SqueezeLLM` (#8220) · 23f32229
  Dipika Sikka authored Sep 06, 2024
  
  23f32229
- [Kernel] [Triton] Memory optimization for awq_gemm and awq_dequantize, 2x throughput (#8248) · 9db52eab
  rasmith authored Sep 06, 2024
  
  9db52eab
05 Sep, 2024 4 commits
- Move verify_marlin_supported to GPTQMarlinLinearMethod (#8165) · 2ee45281
  Michael Goin authored Sep 05, 2024
  
  2ee45281
- [MODEL] Qwen Multimodal Support (Qwen-VL / Qwen-VL-Chat) (#8029) · 9da25a88
  Alex Brooks authored Sep 05, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9da25a88
- update benchmark_throughput.py · ae3524c6
  zhuwenwen authored Sep 05, 2024
  
  ae3524c6
- [Misc] Clean up RoPE forward_native (#8076) · 4624d98d
  Woosuk Kwon authored Sep 04, 2024
  
  4624d98d
04 Sep, 2024 2 commits
- [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) · 008cf886
  Harsha vardhan manoj Bikki authored Sep 04, 2024
```
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
```
  008cf886
- [Misc] Update fbgemmfp8 to use `vLLMParameters` (#7972) · e16fa99a
  Dipika Sikka authored Sep 03, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  e16fa99a
03 Sep, 2024 1 commit
- [Misc] Update `GPTQ` to use `vLLMParameters` (#7976) · 2188a60c
  Dipika Sikka authored Sep 03, 2024
  
  2188a60c
02 Sep, 2024 1 commit
- [SpecDecode][Kernel] Flashinfer Rejection Sampling (#7244) · e6a26ed0
  Lily Liu authored Sep 01, 2024
  
  e6a26ed0
30 Aug, 2024 2 commits
- [Model] Adding support for MSFT Phi-3.5-MoE (#7729) · 1248e850
  Wenxiang authored Aug 31, 2024
```
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Zeqi Lin <zelin@microsoft.com>
Co-authored-by: Zeqi Lin <Zeqi.Lin@microsoft.com>
```
  1248e850
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
29 Aug, 2024 3 commits
- support bitsandbytes 8-bit and FP4 quantized models (#7445) · 4664ceaa
  chenqianfzh authored Aug 29, 2024
  
  4664ceaa
- [misc] update tpu int8 to use new vLLM Parameters (#7973) · 86a677de
  Dipika Sikka authored Aug 29, 2024
  
  86a677de
- [Bugfix] Fix incorrect vocal embedding shards for GGUF model in tensor parallelism (#7954) · d78789ac
  Isotr0py authored Aug 30, 2024
  
  d78789ac
28 Aug, 2024 2 commits
- [Kernel/Model] Migrate mamba_ssm and causal_conv1d kernels to vLLM (#7651) · fdd9daaf
  Mor Zusman authored Aug 29, 2024
  
  fdd9daaf
- [Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and... · e5697d16
  rasmith authored Aug 28, 2024
```
[Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and awq_gemm to support AWQ (#7386)
```
  e5697d16
27 Aug, 2024 2 commits
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7766) · fc911880
  Dipika Sikka authored Aug 27, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  fc911880
- [Misc] Update compressed tensors lifecycle to remove `prefix` from `create_weights` (#7825) · 015e6cc2
  Dipika Sikka authored Aug 26, 2024
  
  015e6cc2
26 Aug, 2024 2 commits
- [Misc] Update `gptq_marlin_24` to use vLLMParameters (#7762) · dd9857f5
  Dipika Sikka authored Aug 26, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  dd9857f5
- [Misc] Update `qqq` to use vLLMParameters (#7805) · 66530409
  Dipika Sikka authored Aug 26, 2024
  
  66530409
23 Aug, 2024 1 commit
- [Misc] Update `marlin` to use vLLMParameters (#7803) · f1df5dbf
  Dipika Sikka authored Aug 23, 2024
  
  f1df5dbf
22 Aug, 2024 3 commits
- [Misc] update fp8 to use `vLLMParameter` (#7437) · 955b5191
  Dipika Sikka authored Aug 22, 2024
  
  955b5191
- [Misc] Use torch.compile for GemmaRMSNorm (#7642) · b3856bef
  Woosuk Kwon authored Aug 22, 2024
  
  b3856bef
- Revert "[Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527)" (#7764) · aae74ef9
  Michael Goin authored Aug 21, 2024
  
  aae74ef9
21 Aug, 2024 4 commits
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527) · 8678a69a
  Dipika Sikka authored Aug 21, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  8678a69a
- Refactoring the optimized kernel · 0aa2480f
  zhuwenwen authored Aug 21, 2024
  
  0aa2480f
- [Model] Add AWQ quantization support for InternVL2 model (#7187) · 12e1c65b
  Isotr0py authored Aug 21, 2024
  
  12e1c65b
- fix tests and update the usage of fa · 1c18cce0
  zhuwenwen authored Aug 21, 2024
  
  1c18cce0
20 Aug, 2024 1 commit
- [Kernel] (1/N) Machete - Hopper Optimized Mixed Precision Linear Kernel (#7174) · 5288c06a
  Lucas Wilkinson authored Aug 20, 2024
  
  5288c06a
19 Aug, 2024 4 commits
- [Core] Support tensor parallelism for GGUF quantization (#7520) · 7601cb04
  Isotr0py authored Aug 20, 2024
  
  7601cb04
- [Misc] Remove Gemma RoPE (#7638) · df845b2b
  Woosuk Kwon authored Aug 19, 2024
  
  df845b2b
- [Core] Use flashinfer sampling kernel when available (#7137) · f710fb52
  Peng Guanwen authored Aug 19, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  f710fb52
- [Misc] Refactor Llama3 RoPE initialization (#7637) · 200a2ffa
  Woosuk Kwon authored Aug 18, 2024
  
  200a2ffa
18 Aug, 2024 1 commit
- [TPU] Optimize RoPE forward_native2 (#7636) · ab7165f2
  Woosuk Kwon authored Aug 18, 2024
  
  ab7165f2