Commits · 9db52eab3dc0b7b2cf30fa4399d569131e90c2d4 · OpenDAS / vllm_cscc

"vllm/benchmark_throughput.py" did not exist on "b5a10eb0ef68f45c7dbdef2917e02bebca780d1a"

06 Sep, 2024 1 commit
- [Kernel] [Triton] Memory optimization for awq_gemm and awq_dequantize, 2x throughput (#8248) · 9db52eab
  rasmith authored Sep 06, 2024
  
  9db52eab
05 Sep, 2024 3 commits
- Move verify_marlin_supported to GPTQMarlinLinearMethod (#8165) · 2ee45281
  Michael Goin authored Sep 05, 2024
  
  2ee45281
- [MODEL] Qwen Multimodal Support (Qwen-VL / Qwen-VL-Chat) (#8029) · 9da25a88
  Alex Brooks authored Sep 05, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9da25a88
- [Misc] Clean up RoPE forward_native (#8076) · 4624d98d
  Woosuk Kwon authored Sep 04, 2024
  
  4624d98d
04 Sep, 2024 2 commits
- [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) · 008cf886
  Harsha vardhan manoj Bikki authored Sep 04, 2024
```
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
```
  008cf886
- [Misc] Update fbgemmfp8 to use `vLLMParameters` (#7972) · e16fa99a
  Dipika Sikka authored Sep 03, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  e16fa99a
03 Sep, 2024 1 commit
- [Misc] Update `GPTQ` to use `vLLMParameters` (#7976) · 2188a60c
  Dipika Sikka authored Sep 03, 2024
  
  2188a60c
02 Sep, 2024 1 commit
- [SpecDecode][Kernel] Flashinfer Rejection Sampling (#7244) · e6a26ed0
  Lily Liu authored Sep 01, 2024
  
  e6a26ed0
30 Aug, 2024 2 commits
- [Model] Adding support for MSFT Phi-3.5-MoE (#7729) · 1248e850
  Wenxiang authored Aug 31, 2024
```
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Zeqi Lin <zelin@microsoft.com>
Co-authored-by: Zeqi Lin <Zeqi.Lin@microsoft.com>
```
  1248e850
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
29 Aug, 2024 3 commits
- support bitsandbytes 8-bit and FP4 quantized models (#7445) · 4664ceaa
  chenqianfzh authored Aug 29, 2024
  
  4664ceaa
- [misc] update tpu int8 to use new vLLM Parameters (#7973) · 86a677de
  Dipika Sikka authored Aug 29, 2024
  
  86a677de
- [Bugfix] Fix incorrect vocal embedding shards for GGUF model in tensor parallelism (#7954) · d78789ac
  Isotr0py authored Aug 30, 2024
  
  d78789ac
28 Aug, 2024 2 commits
- [Kernel/Model] Migrate mamba_ssm and causal_conv1d kernels to vLLM (#7651) · fdd9daaf
  Mor Zusman authored Aug 29, 2024
  
  fdd9daaf
- [Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and... · e5697d16
  rasmith authored Aug 28, 2024
```
[Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and awq_gemm to support AWQ (#7386)
```
  e5697d16
27 Aug, 2024 2 commits
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7766) · fc911880
  Dipika Sikka authored Aug 27, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  fc911880
- [Misc] Update compressed tensors lifecycle to remove `prefix` from `create_weights` (#7825) · 015e6cc2
  Dipika Sikka authored Aug 26, 2024
  
  015e6cc2
26 Aug, 2024 2 commits
- [Misc] Update `gptq_marlin_24` to use vLLMParameters (#7762) · dd9857f5
  Dipika Sikka authored Aug 26, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  dd9857f5
- [Misc] Update `qqq` to use vLLMParameters (#7805) · 66530409
  Dipika Sikka authored Aug 26, 2024
  
  66530409
23 Aug, 2024 1 commit
- [Misc] Update `marlin` to use vLLMParameters (#7803) · f1df5dbf
  Dipika Sikka authored Aug 23, 2024
  
  f1df5dbf
22 Aug, 2024 3 commits
- [Misc] update fp8 to use `vLLMParameter` (#7437) · 955b5191
  Dipika Sikka authored Aug 22, 2024
  
  955b5191
- [Misc] Use torch.compile for GemmaRMSNorm (#7642) · b3856bef
  Woosuk Kwon authored Aug 22, 2024
  
  b3856bef
- Revert "[Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527)" (#7764) · aae74ef9
  Michael Goin authored Aug 21, 2024
  
  aae74ef9
21 Aug, 2024 2 commits
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527) · 8678a69a
  Dipika Sikka authored Aug 21, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  8678a69a
- [Model] Add AWQ quantization support for InternVL2 model (#7187) · 12e1c65b
  Isotr0py authored Aug 21, 2024
  
  12e1c65b
20 Aug, 2024 1 commit
- [Kernel] (1/N) Machete - Hopper Optimized Mixed Precision Linear Kernel (#7174) · 5288c06a
  Lucas Wilkinson authored Aug 20, 2024
  
  5288c06a
19 Aug, 2024 4 commits
- [Core] Support tensor parallelism for GGUF quantization (#7520) · 7601cb04
  Isotr0py authored Aug 20, 2024
  
  7601cb04
- [Misc] Remove Gemma RoPE (#7638) · df845b2b
  Woosuk Kwon authored Aug 19, 2024
  
  df845b2b
- [Core] Use flashinfer sampling kernel when available (#7137) · f710fb52
  Peng Guanwen authored Aug 19, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  f710fb52
- [Misc] Refactor Llama3 RoPE initialization (#7637) · 200a2ffa
  Woosuk Kwon authored Aug 18, 2024
  
  200a2ffa
18 Aug, 2024 1 commit
- [TPU] Optimize RoPE forward_native2 (#7636) · ab7165f2
  Woosuk Kwon authored Aug 18, 2024
  
  ab7165f2
17 Aug, 2024 1 commit
- [misc] use nvml to get consistent device name (#7582) · eed020f6
  youkaichao authored Aug 16, 2024
  
  eed020f6
16 Aug, 2024 5 commits
- [Model] Align nemotron config with final HF state and fix lm-eval-small (#7611) · 44f26a94
  Michael Goin authored Aug 16, 2024
  
  44f26a94
- [Kernel] fix types used in aqlm and ggml kernels to support dynamo (#7596) · 37fd47e7
  bnellnm authored Aug 16, 2024
  
  37fd47e7
- [Kernel] Add tuned triton configs for ExpertsInt8 (#7601) · 855866ca
  Michael Goin authored Aug 16, 2024
  
  855866ca
- [Kernel] W8A16 Int8 inside FusedMoE (#7415) · 7fc23be8
  Mor Zusman authored Aug 16, 2024
  
  7fc23be8
- [Feature][Hardware][Amd] Add fp8 Linear Layer for Rocm (#7210) · e837b624
  Charlie Fu authored Aug 16, 2024
  
  e837b624
14 Aug, 2024 2 commits
- [Misc] Revert `compressed-tensors` code reuse (#7521) · f55a9aea
  Kyle Sayers authored Aug 14, 2024
  
  f55a9aea
- Fix empty output when temp is too low (#2937) · c134a464
  Chang Su authored Aug 13, 2024
```
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  c134a464
13 Aug, 2024 1 commit
- [Misc] `compressed-tensors` code reuse (#7277) · 373538f9
  Kyle Sayers authored Aug 13, 2024
  
  373538f9