Commits · 9d30a056e7a1c81382a53ac63dc476c5fbe0091d · OpenDAS / vllm_cscc · GitLab

17 Oct, 2024 5 commits
- [misc] CUDA Time Layerwise Profiler (#8337) · 9d30a056
  Lucas Wilkinson authored Oct 17, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  9d30a056
- [Misc] Print stack trace using `logger.exception` (#9461) · 390be746
  Cyrus Leung authored Oct 17, 2024
  
  390be746
- [Kernel] Add Exllama as a backend for compressed-tensors (#9395) · e312e52b
  Lucas Wilkinson authored Oct 17, 2024
  
  e312e52b
- Add notes on the use of Slack (#9442) · dbfa8d31
  Yuan Tang authored Oct 17, 2024
  
  dbfa8d31
- [BugFix] [Kernel] Fix GPU SEGV occurring in int8 kernels (#9391) · 92d86da2
  rasmith authored Oct 16, 2024
  
  92d86da2
16 Oct, 2024 14 commits
- [Bugfix][Kernel] Prevent integer overflow in fp8 dynamic per-token quantize kernel (#9425) · c3fab5f7
  Tyler Michael Smith authored Oct 16, 2024
  
  c3fab5f7
- [CI/Build] mypy: Resolve some errors from checking vllm/engine (#9267) · 776dbd74
  Russell Bryant authored Oct 16, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  776dbd74
- [Performance][Spec Decode] Optimize ngram lookup performance (#9333) · 83450458
  Lily Liu authored Oct 16, 2024
  
  83450458
- [Model][Bugfix] Add FATReLU activation and support for openbmb/MiniCPM-S-1B-sft (#9396) · 5b8a1fde
  Junhao Li authored Oct 16, 2024
  
  5b8a1fde
- [Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189) · fb60ae9b
  Mor Zusman authored Oct 17, 2024
  
  fb60ae9b
- Support mistral interleaved attn (#9414) · 415f76a9
  Patrick von Platen authored Oct 16, 2024
  
  415f76a9
- [Model] Support SDPA attention for Molmo vision backbone (#9410) · cf1d62a6
  Isotr0py authored Oct 16, 2024
  
  cf1d62a6
- [Misc] Consolidate example usage of OpenAI client for multimodal models (#9412) · 59230ef3
  Roger Wang authored Oct 16, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  59230ef3
- [Core] Rename input data types (#8688) · cee711fd
  Cyrus Leung authored Oct 16, 2024
  
  cee711fd
- [CI/Build] Test VLM embeddings (#9406) · 1de76a0e
  Cyrus Leung authored Oct 16, 2024
  
  1de76a0e
- [Model] VLM2Vec, the first multimodal embedding model in vLLM (#9303) · 7abba39e
  Cyrus Leung authored Oct 16, 2024
  
  7abba39e
- [Misc] Standardize RoPE handling for Qwen2-VL (#9250) · 7e7eae33
  Cyrus Leung authored Oct 16, 2024
  
  7e7eae33
- [Bugfix] Molmo text-only input bug fix (#9397) · ed920135
  Reza Salehi authored Oct 15, 2024
```
Co-authored-by: sanghol <sanghol@allenai.org>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  ed920135
- [Bugfix][CI/Build] Fix CUDA 11.8 Build (#9386) · 717a5f82
  Lucas Wilkinson authored Oct 15, 2024
  
  717a5f82
15 Oct, 2024 8 commits
- [Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids (#9034) · ba309422
  Chang Su authored Oct 15, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ba309422
- [Misc] Directly use compressed-tensors for checkpoint definitions (#8909) · 22f8a695
  Michael Goin authored Oct 15, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  22f8a695
- pass ignore_eos parameter to all benchmark_serving calls (#9349) · 5d264f4a
  Grace Ho authored Oct 15, 2024
  
  5d264f4a
- [BugFix] Fix chat API continuous usage stats (#9357) · e9d517f2
  Nick Hill authored Oct 15, 2024
  
  e9d517f2
- [Bugfix] Update InternVL input mapper to support image embeds (#9351) · 55e081fb
  hhzhang16 authored Oct 14, 2024
  
  55e081fb
- [Doc] Fix code formatting in spec_decode.rst (#9348) · 8e836d98
  Michael Goin authored Oct 15, 2024
  
  8e836d98
- [Frontend] Clarify model_type error messages (#9345) · 44eaa5a5
  Steve Grubb authored Oct 15, 2024
  
  44eaa5a5
- [Bugfix] Clean up some cruft in mamba.py (#9343) · 169b5306
  Tyler Michael Smith authored Oct 14, 2024
  
  169b5306
14 Oct, 2024 7 commits
- [Model] Make llama3.2 support multiple and interleaved images (#9095) · f0fe4fe8
  Xiang Xu authored Oct 14, 2024
  
  f0fe4fe8
- [Frontend] merge beam search implementations (#9296) · 4d31cd42
  Brendan Wong authored Oct 14, 2024
  
  4d31cd42
- [TPU] Fix TPU SMEM OOM by Pallas paged attention kernel (#9350) · 473e7b36
  Woosuk Kwon authored Oct 14, 2024
  
  473e7b36
- [Docs] Remove PDF build from Readtehdocs (#9347) · fd47e57f
  Simon Mo authored Oct 14, 2024
  
  fd47e57f
- [CI/Build] setuptools-scm fixes (#8900) · 203ab8f8
  Daniele authored Oct 14, 2024
  
  203ab8f8
- [Hardware][intel GPU] add async output process for xpu (#8897) · 4141608c
  Kunshang Ji authored Oct 15, 2024
  
  4141608c
- [Model] Molmo vLLM Integration (#9016) · dfe43a20
  Reza Salehi authored Oct 14, 2024
```
Co-authored-by: sanghol <sanghol@allenai.org>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  dfe43a20
13 Oct, 2024 2 commits
- [Bugfix] Bandaid fix for speculative decoding tests (#9327) · 16b24e7d
  Tyler Michael Smith authored Oct 13, 2024
  
  16b24e7d
- [CI] Fix merge conflict (#9317) · f519902c
  Lily Liu authored Oct 12, 2024
  
  f519902c
12 Oct, 2024 4 commits
- [Bugfix]Fix MiniCPM's LoRA bug (#9286) · 250e26a6
  Jee Jee Li authored Oct 13, 2024
  
  250e26a6
- [Misc][Installation] Improve source installation script and doc (#9309) · 2b184ddd
  Yunmeng authored Oct 13, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  2b184ddd
- [Bugfix] Fix bug of xformer prefill for encoder-decoder (#9026) · 00298e09
  Xiang Xu authored Oct 12, 2024
  
  00298e09
- [SpecDec] Remove Batch Expansion (2/3) (#9298) · 89feb4c8
  Lily Liu authored Oct 11, 2024
  
  89feb4c8