Commits · 6ffa3f314c59e42238f1c5f923ff2839e0af9698 · OpenDAS / vllm_cscc · GitLab

18 Sep, 2024 1 commit
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
17 Sep, 2024 3 commits
- [Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012) · 8110e445
  Tyler Michael Smith authored Sep 17, 2024
  
  8110e445
- [Feature][kernel] tensor parallelism with bitsandbytes quantization (#8434) · 9855b995
  chenqianfzh authored Sep 17, 2024
  
  9855b995
- [refactor] remove triton based sampler (#8524) · 546034b4
  Simon Mo authored Sep 16, 2024
  
  546034b4
16 Sep, 2024 2 commits
- [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) · 5d73ae49
  Luka Govedič authored Sep 16, 2024
  
  5d73ae49
- [Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032) · a091e2da
  ElizaWszola authored Sep 16, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  a091e2da
15 Sep, 2024 1 commit
- [Bugfix][Kernel] Add `IQ1_M` quantization implementation to GGUF kernel (#8357) · fc990f97
  Isotr0py authored Sep 16, 2024
  
  fc990f97
13 Sep, 2024 2 commits
- [Hardware][intel GPU] bump up ipex version to 2.3 (#8365) · 85172520
  Kunshang Ji authored Sep 14, 2024
```
Co-authored-by: Yan Ma <yan.ma@intel.com>
```
  85172520
- [BugFix] fix group_topk (#8430) · 8f44a92d
  Dipika Sikka authored Sep 12, 2024
  
  8f44a92d
12 Sep, 2024 1 commit
- [Misc] Use RoPE cache for MRoPE (#8396) · 42ffba11
  Woosuk Kwon authored Sep 11, 2024
  
  42ffba11
11 Sep, 2024 3 commits
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4
- [Model][VLM] Add Qwen2-VL model support (#7905) · 3b7fea77
  Yang Fan authored Sep 12, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b7fea77
- [Hardware][NV] Add support for ModelOpt static scaling checkpoints. (#6112) · efcf946a
  Pavani Majety authored Sep 10, 2024
  
  efcf946a
10 Sep, 2024 1 commit
- [Misc] Fused MoE Marlin support for GPTQ (#8217) · 6cd5e5b0
  Dipika Sikka authored Sep 09, 2024
  
  6cd5e5b0
09 Sep, 2024 1 commit
- [Misc] GPTQ Activation Ordering (#8135) · c7cb5c33
  Kyle Sayers authored Sep 09, 2024
  
  c7cb5c33
06 Sep, 2024 2 commits
- [Misc] Remove `SqueezeLLM` (#8220) · 23f32229
  Dipika Sikka authored Sep 06, 2024
  
  23f32229
- [Kernel] [Triton] Memory optimization for awq_gemm and awq_dequantize, 2x throughput (#8248) · 9db52eab
  rasmith authored Sep 06, 2024
  
  9db52eab
05 Sep, 2024 3 commits
- Move verify_marlin_supported to GPTQMarlinLinearMethod (#8165) · 2ee45281
  Michael Goin authored Sep 05, 2024
  
  2ee45281
- [MODEL] Qwen Multimodal Support (Qwen-VL / Qwen-VL-Chat) (#8029) · 9da25a88
  Alex Brooks authored Sep 05, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9da25a88
- [Misc] Clean up RoPE forward_native (#8076) · 4624d98d
  Woosuk Kwon authored Sep 04, 2024
  
  4624d98d
04 Sep, 2024 2 commits
- [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) · 008cf886
  Harsha vardhan manoj Bikki authored Sep 04, 2024
```
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
```
  008cf886
- [Misc] Update fbgemmfp8 to use `vLLMParameters` (#7972) · e16fa99a
  Dipika Sikka authored Sep 03, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  e16fa99a
03 Sep, 2024 1 commit
- [Misc] Update `GPTQ` to use `vLLMParameters` (#7976) · 2188a60c
  Dipika Sikka authored Sep 03, 2024
  
  2188a60c
02 Sep, 2024 1 commit
- [SpecDecode][Kernel] Flashinfer Rejection Sampling (#7244) · e6a26ed0
  Lily Liu authored Sep 01, 2024
  
  e6a26ed0
30 Aug, 2024 2 commits
- [Model] Adding support for MSFT Phi-3.5-MoE (#7729) · 1248e850
  Wenxiang authored Aug 31, 2024
```
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Zeqi Lin <zelin@microsoft.com>
Co-authored-by: Zeqi Lin <Zeqi.Lin@microsoft.com>
```
  1248e850
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
29 Aug, 2024 3 commits
- support bitsandbytes 8-bit and FP4 quantized models (#7445) · 4664ceaa
  chenqianfzh authored Aug 29, 2024
  
  4664ceaa
- [misc] update tpu int8 to use new vLLM Parameters (#7973) · 86a677de
  Dipika Sikka authored Aug 29, 2024
  
  86a677de
- [Bugfix] Fix incorrect vocal embedding shards for GGUF model in tensor parallelism (#7954) · d78789ac
  Isotr0py authored Aug 30, 2024
  
  d78789ac
28 Aug, 2024 2 commits
- [Kernel/Model] Migrate mamba_ssm and causal_conv1d kernels to vLLM (#7651) · fdd9daaf
  Mor Zusman authored Aug 29, 2024
  
  fdd9daaf
- [Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and... · e5697d16
  rasmith authored Aug 28, 2024
```
[Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and awq_gemm to support AWQ (#7386)
```
  e5697d16
27 Aug, 2024 2 commits
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7766) · fc911880
  Dipika Sikka authored Aug 27, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  fc911880
- [Misc] Update compressed tensors lifecycle to remove `prefix` from `create_weights` (#7825) · 015e6cc2
  Dipika Sikka authored Aug 26, 2024
  
  015e6cc2
26 Aug, 2024 2 commits
- [Misc] Update `gptq_marlin_24` to use vLLMParameters (#7762) · dd9857f5
  Dipika Sikka authored Aug 26, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  dd9857f5
- [Misc] Update `qqq` to use vLLMParameters (#7805) · 66530409
  Dipika Sikka authored Aug 26, 2024
  
  66530409
23 Aug, 2024 1 commit
- [Misc] Update `marlin` to use vLLMParameters (#7803) · f1df5dbf
  Dipika Sikka authored Aug 23, 2024
  
  f1df5dbf
22 Aug, 2024 3 commits
- [Misc] update fp8 to use `vLLMParameter` (#7437) · 955b5191
  Dipika Sikka authored Aug 22, 2024
  
  955b5191
- [Misc] Use torch.compile for GemmaRMSNorm (#7642) · b3856bef
  Woosuk Kwon authored Aug 22, 2024
  
  b3856bef
- Revert "[Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527)" (#7764) · aae74ef9
  Michael Goin authored Aug 21, 2024
  
  aae74ef9
21 Aug, 2024 1 commit
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527) · 8678a69a
  Dipika Sikka authored Aug 21, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  8678a69a