Commits · 9855b99502c7537db5ef018129e603650800ac46 · OpenDAS / vllm_cscc · GitLab

17 Sep, 2024 4 commits
- [Feature][kernel] tensor parallelism with bitsandbytes quantization (#8434) · 9855b995
  chenqianfzh authored Sep 17, 2024
  
  9855b995
- [Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631) · 1009e93c
  sroy745 authored Sep 17, 2024
  
  1009e93c
- [Misc][Bugfix] Disable guided decoding for mistral tokenizer (#8521) · ee2bceaa
  Roger Wang authored Sep 16, 2024
  
  ee2bceaa
- [refactor] remove triton based sampler (#8524) · 546034b4
  Simon Mo authored Sep 16, 2024
  
  546034b4
16 Sep, 2024 2 commits
- [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) · 5d73ae49
  Luka Govedič authored Sep 16, 2024
  
  5d73ae49
- [Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032) · a091e2da
  ElizaWszola authored Sep 16, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  a091e2da
15 Sep, 2024 2 commits
- [Bugfix][Kernel] Add `IQ1_M` quantization implementation to GGUF kernel (#8357) · fc990f97
  Isotr0py authored Sep 16, 2024
  
  fc990f97
- [Bugfix][Model] Fix Python 3.8 compatibility in Pixtral model by updating type annotations (#8490) · 3724d5f6
  Chris authored Sep 15, 2024
  
  3724d5f6
14 Sep, 2024 2 commits
- [torch.compile] add a flag to disable custom op (#8488) · 47790f3e
  youkaichao authored Sep 14, 2024
  
  47790f3e
- [Model] support minicpm3 (#8297) · 8a0cf1dd
  ywfang authored Sep 14, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8a0cf1dd
13 Sep, 2024 3 commits
- [Hardware][intel GPU] bump up ipex version to 2.3 (#8365) · 85172520
  Kunshang Ji authored Sep 14, 2024
```
Co-authored-by: Yan Ma <yan.ma@intel.com>
```
  85172520
- [Misc] Skip loading extra bias for Qwen2-VL GPTQ-Int8 (#8442) · 06311e29
  Jee Jee Li authored Sep 13, 2024
  
  06311e29
- [BugFix] fix group_topk (#8430) · 8f44a92d
  Dipika Sikka authored Sep 12, 2024
  
  8f44a92d
12 Sep, 2024 7 commits
- [Bugfix] Fix weight loading issue by rename variable. (#8293) · a480939e
  Wenxiang authored Sep 13, 2024
  
  a480939e
- [Hotfix][Pixtral] Fix multiple images bugs (#8415) · d31174a4
  Patrick von Platen authored Sep 13, 2024
  
  d31174a4
- [Hotfix][Core][VLM] Disable chunked prefill by default and prefix caching for... · c1636945
  Roger Wang authored Sep 12, 2024
```
[Hotfix][Core][VLM] Disable chunked prefill by default and prefix caching for multimodal models (#8425)
```
  c1636945
- [Model] Support multiple images for qwen-vl (#8247) · c6202dae
  Alex Brooks authored Sep 12, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c6202dae
- [Bugfix] Fix InternVL2 inference with various num_patches (#8375) · e56bf277
  Isotr0py authored Sep 13, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e56bf277
- [Misc] Use RoPE cache for MRoPE (#8396) · 42ffba11
  Woosuk Kwon authored Sep 11, 2024
  
  42ffba11
- [Gemma2] add bitsandbytes support for Gemma2 (#8338) · 1bf2dd9d
  Blueyo0 authored Sep 12, 2024
  
  1bf2dd9d
11 Sep, 2024 8 commits
- Pixtral (#8377) · d394787e
  Patrick von Platen authored Sep 11, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d394787e
- [Kernel][Misc] register ops to prevent graph breaks (#6917) · 73202dbe
  bnellnm authored Sep 11, 2024
```
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  73202dbe
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4
- [Model][VLM] Add Qwen2-VL model support (#7905) · 3b7fea77
  Yang Fan authored Sep 12, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b7fea77
- [model] Support for Llava-Next-Video model (#7559) · 6a512a00
  Yangshen⚡Deng authored Sep 11, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  6a512a00
- [Hardware][NV] Add support for ModelOpt static scaling checkpoints. (#6112) · efcf946a
  Pavani Majety authored Sep 10, 2024
  
  efcf946a
- [Bugfix] Fix InternVL2 vision embeddings process with pipeline parallel (#8299) · 1230263e
  Isotr0py authored Sep 11, 2024
  
  1230263e
- [Misc] Skip loading extra bias for Qwen2-MOE GPTQ models (#8329) · e497b8ae
  Jee Jee Li authored Sep 11, 2024
  
  e497b8ae
10 Sep, 2024 2 commits
- [Bugfix] Fix missing `post_layernorm` in CLIP (#8155) · da1a844e
  Cyrus Leung authored Sep 10, 2024
  
  da1a844e
- [Misc] Fused MoE Marlin support for GPTQ (#8217) · 6cd5e5b0
  Dipika Sikka authored Sep 09, 2024
  
  6cd5e5b0
09 Sep, 2024 2 commits
- [Misc] GPTQ Activation Ordering (#8135) · c7cb5c33
  Kyle Sayers authored Sep 09, 2024
  
  c7cb5c33
- [Bugfix] Correct adapter usage for cohere and jamba (#8292) · f9b4a2d4
  Vladislav Kruglikov authored Sep 09, 2024
  
  f9b4a2d4
07 Sep, 2024 4 commits
- [Model][VLM] Decouple weight loading logic for `Paligemma` (#8269) · 36bf8150
  Isotr0py authored Sep 08, 2024
  
  36bf8150
- [Model][VLM] Support multi-images inputs for InternVL2 models (#8201) · e8071259
  Isotr0py authored Sep 07, 2024
  
  e8071259
- [Bugfix] Fix broken OpenAI tensorizer test (#8258) · 9f68e00d
  Cyrus Leung authored Sep 07, 2024
  
  9f68e00d
- [Model] Multi-input support for LLaVA (#8238) · 2f707fcb
  Cyrus Leung authored Sep 07, 2024
  
  2f707fcb
06 Sep, 2024 3 commits
- [Model] Allow loading from original Mistral format (#8168) · 29f49cd6
  Patrick von Platen authored Sep 07, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  29f49cd6
- [Misc] Remove `SqueezeLLM` (#8220) · 23f32229
  Dipika Sikka authored Sep 06, 2024
  
  23f32229
- [Kernel] [Triton] Memory optimization for awq_gemm and awq_dequantize, 2x throughput (#8248) · 9db52eab
  rasmith authored Sep 06, 2024
  
  9db52eab
05 Sep, 2024 1 commit
- Move verify_marlin_supported to GPTQMarlinLinearMethod (#8165) · 2ee45281
  Michael Goin authored Sep 05, 2024
  
  2ee45281