Commits · 962d2c63495e930cdd3b59479dce1de48be57ecd · OpenDAS / vllm_cscc · GitLab

20 Oct, 2024 1 commit
- [Model][Pixtral] Use memory_efficient_attention for PixtralHFVision (#9520) · 962d2c63
  Michael Goin authored Oct 20, 2024
  
  962d2c63
19 Oct, 2024 1 commit
- [Model][Pixtral] Optimizations for input_processor_for_pixtral_hf (#9514) · 8e3e7f27
  Michael Goin authored Oct 19, 2024
  
  8e3e7f27
18 Oct, 2024 3 commits
- [Model] Support Pixtral models in the HF Transformers format (#9036) · 3921a2f2
  Michael Goin authored Oct 18, 2024
  
  3921a2f2
- [Qwen2.5] Support bnb quant for Qwen2.5 (#9467) · 154a8ae8
  Haoyu Wang authored Oct 18, 2024
  
  154a8ae8
- [BugFix] Stop silent failures on compressed-tensors parsing (#9381) · 48138a84
  Dipika Sikka authored Oct 17, 2024
  
  48138a84
17 Oct, 2024 7 commits
- Support `BERTModel` (first `encoder-only` embedding model) (#9056) · 343f8e09
  Robert Shaw authored Oct 17, 2024
```
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: laishzh <laishengzhang@gmail.com>
Co-authored-by: Max de Bayser <maxdebayser@gmail.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  343f8e09
- [Hardwware][Neuron] Simplify model load for transformers-neuronx library (#9380) · bb76538b
  Shashwat Srijan authored Oct 17, 2024
  
  bb76538b
- [Bugfix] Fix support for dimension like integers and ScalarType (#9299) · eca2c5f7
  bnellnm authored Oct 17, 2024
  
  eca2c5f7
- [torch.compile] Fine-grained CustomOp enabling mechanism (#9300) · 0f41fbe5
  Luka Govedič authored Oct 17, 2024
  
  0f41fbe5
- [TPU] Call torch._sync(param) during weight loading (#9437) · 8e1cddcd
  Woosuk Kwon authored Oct 17, 2024
  
  8e1cddcd
- [Misc] Print stack trace using `logger.exception` (#9461) · 390be746
  Cyrus Leung authored Oct 17, 2024
  
  390be746
- [Kernel] Add Exllama as a backend for compressed-tensors (#9395) · e312e52b
  Lucas Wilkinson authored Oct 17, 2024
  
  e312e52b
16 Oct, 2024 8 commits
- [CI/Build] mypy: Resolve some errors from checking vllm/engine (#9267) · 776dbd74
  Russell Bryant authored Oct 16, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  776dbd74
- [Model][Bugfix] Add FATReLU activation and support for openbmb/MiniCPM-S-1B-sft (#9396) · 5b8a1fde
  Junhao Li authored Oct 16, 2024
  
  5b8a1fde
- [Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189) · fb60ae9b
  Mor Zusman authored Oct 17, 2024
  
  fb60ae9b
- [Model] Support SDPA attention for Molmo vision backbone (#9410) · cf1d62a6
  Isotr0py authored Oct 16, 2024
  
  cf1d62a6
- [Core] Rename input data types (#8688) · cee711fd
  Cyrus Leung authored Oct 16, 2024
  
  cee711fd
- [Model] VLM2Vec, the first multimodal embedding model in vLLM (#9303) · 7abba39e
  Cyrus Leung authored Oct 16, 2024
  
  7abba39e
- [Misc] Standardize RoPE handling for Qwen2-VL (#9250) · 7e7eae33
  Cyrus Leung authored Oct 16, 2024
  
  7e7eae33
- [Bugfix] Molmo text-only input bug fix (#9397) · ed920135
  Reza Salehi authored Oct 15, 2024
```
Co-authored-by: sanghol <sanghol@allenai.org>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  ed920135
15 Oct, 2024 3 commits
- [Misc] Directly use compressed-tensors for checkpoint definitions (#8909) · 22f8a695
  Michael Goin authored Oct 15, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  22f8a695
- [Bugfix] Update InternVL input mapper to support image embeds (#9351) · 55e081fb
  hhzhang16 authored Oct 14, 2024
  
  55e081fb
- [Bugfix] Clean up some cruft in mamba.py (#9343) · 169b5306
  Tyler Michael Smith authored Oct 14, 2024
  
  169b5306
14 Oct, 2024 2 commits
- [Model] Make llama3.2 support multiple and interleaved images (#9095) · f0fe4fe8
  Xiang Xu authored Oct 14, 2024
  
  f0fe4fe8
- [Model] Molmo vLLM Integration (#9016) · dfe43a20
  Reza Salehi authored Oct 14, 2024
```
Co-authored-by: sanghol <sanghol@allenai.org>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  dfe43a20
12 Oct, 2024 1 commit
- [Bugfix]Fix MiniCPM's LoRA bug (#9286) · 250e26a6
  Jee Jee Li authored Oct 13, 2024
  
  250e26a6
11 Oct, 2024 5 commits
- [Model] Add GLM-4v support and meet vllm==0.6.2 (#9242) · 6cf1167c
  sixgod authored Oct 12, 2024
  
  6cf1167c
- [Kernel] adding fused moe kernel config for L40S TP4 (#9245) · f710090d
  Burkhard Ringlein authored Oct 11, 2024
  
  f710090d
- [Model] Support Mamba (#6484) · 7342a7d7
  Tyler Michael Smith authored Oct 11, 2024
  
  7342a7d7
- [Misc] Collect model support info in a single process per model (#9233) · e808156f
  Cyrus Leung authored Oct 11, 2024
  
  e808156f
- [misc] hide best_of from engine (#9261) · cbc2ef55
  youkaichao authored Oct 10, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  cbc2ef55
10 Oct, 2024 5 commits
- [torch.compile] generic decorators (#9258) · e00c094f
  youkaichao authored Oct 10, 2024
  
  e00c094f
- [torch.compile] integration with compilation control (#9058) · e4d652ea
  youkaichao authored Oct 10, 2024
  
  e4d652ea
- [Model] support input image embedding for minicpmv (#9237) · 04de9057
  whyiug authored Oct 10, 2024
  
  04de9057
- [Bugfix] Fix lm_head weights tying with lora for llama (#9227) · 07c11cf4
  Isotr0py authored Oct 10, 2024
  
  07c11cf4
- [misc] improve model support check in another process (#9208) · de895f16
  youkaichao authored Oct 09, 2024
  
  de895f16
09 Oct, 2024 4 commits
- [Hardware][CPU] Support AWQ for CPU backend (#7515) · ca77dd7a
  Li, Jiang authored Oct 10, 2024
  
  ca77dd7a
- [Bugfix] fix composite weight loading and EAGLE weight loading (#9160) · 8bfaa4e3
  Cyrus Leung authored Oct 09, 2024
  
  8bfaa4e3
- [Model] Remap FP8 kv_scale in CommandR and DBRX (#9174) · cdc72e3c
  Hui Liu authored Oct 08, 2024
  
  cdc72e3c
- support bitsandbytes quantization with more models (#9148) · 2f4117c3
  chenqianfzh authored Oct 08, 2024
  
  2f4117c3