Commits · c83919c7a6bd47bb452321f08017ef5a5cdd553a · OpenDAS / vllm_cscc · GitLab

28 Nov, 2024 11 commits
- [Model] Add Internlm2 LoRA support (#5064) · c83919c7
  Isotr0py authored Nov 29, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c83919c7
- [V1] Optimize the CPU overheads in FlashAttention custom op (#10733) · 98f47f2a
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  98f47f2a
- [Kernel] Update vllm-flash-attn version to reduce CPU overheads (#10742) · 8c1e77fb
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8c1e77fb
- [Model] Added GLM-4 series hf format model support vllm==0.6.4 (#10561) · 5fc5ce0f
  sixgod authored Nov 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  5fc5ce0f
- [TPU] Update requirements-tpu (#10726) · 3ed5e731
  Richard Liu authored Nov 28, 2024
```
Signed-off-by: Richard Liu <ricliu@google.com>
```
  3ed5e731
- [Kernel] Update vllm-flash-attn version (#10736) · 9a8bff02
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  9a8bff02
- [V1] Do not allocate beyond the max_model_len (#10730) · a79b1224
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  a79b1224
- [Bug][CLI] Allow users to disable prefix caching explicitly (#10724) · d9b4b3f0
  Ricky Xu authored Nov 27, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  d9b4b3f0
- [Doc] Update model in arch_overview.rst to match comment (#10701) · 278be671
  罗泽轩 authored Nov 28, 2024
```
Signed-off-by: spacewander <spacewanderlzx@gmail.com>
```
  278be671
- [Model] support bitsandbytes quantization with minicpm3 model (#10682) · 70dc14fb
  zixuanzhang226 authored Nov 27, 2024
```
Signed-off-by: Ubuntu <zixuanzhang@bytedance.com>
```
  70dc14fb
- [misc] upgrade filelock version (#10731) · cb4e1c3f
  youkaichao authored Nov 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cb4e1c3f
27 Nov, 2024 16 commits
- [Frontend] don't block event loop in tokenization (preprocess) in OpenAI compatible server (#10635) · 395b1c74
  tomeras91 authored Nov 27, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  395b1c74
- [Bugfix] Ignore `lm_head` when loading embedding models (#10719) · 9b4b1503
  Cyrus Leung authored Nov 28, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9b4b1503
- [Bugfix][Mamba] Fix Multistep on Mamba-like models (#10705) · 197b4484
  Mor Zusman authored Nov 27, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  197b4484
- [Bugfix] Fix GGUF inference with FP16 unquantized checkpoint (#10675) · b98c62ba
  Isotr0py authored Nov 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b98c62ba
- [torch.compile] fix shape specialization (#10722) · c411def2
  youkaichao authored Nov 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c411def2
- [ci] fix slow tests (#10698) · 308cc5e2
  youkaichao authored Nov 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  308cc5e2
- [V1] Update interface for mistral-format Pixtral (#10703) · 9e0a147d
  Roger Wang authored Nov 27, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  9e0a147d
- [Bugfix][Hardware][CPU] Fix intel-omp version to avoid segfault (#10700) · 418cb3b9
  Li, Jiang authored Nov 27, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  418cb3b9
- [Model] Support telechat2 (#10311) · 1209261e
  shunxing12345 authored Nov 27, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: xiangw2 <xiangw2@chinatelecom.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  1209261e
- [Kernel] Remove if-else with identical branches in marlin 2:4 (#10687) · e2251109
  Tyler Michael Smith authored Nov 27, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  e2251109
- [Misc]Further reduce BNB static variable (#10597) · 15cc2a9f
  Jee Jee Li authored Nov 27, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  15cc2a9f
- [Hardware][Gaudi]add get_name method for HPUAttentionBackend (#10667) · e85250b1
  Kunshang Ji authored Nov 27, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  e85250b1
- [bugfix] fix the default value of llm_int8_threshold in BitsAndBytesConfig (#10657) · cfb3bf25
  yansh97 authored Nov 27, 2024
  
  cfb3bf25
- [Bugfix][SpecDecode] apply sampling parameters to target probabilities for... · 1bf905dd
  jeongin601 authored Nov 27, 2024
```
[Bugfix][SpecDecode] apply sampling parameters to target probabilities for consistency in rejection sampling. (#10198)
Signed-off-by: jeongin601 <0200angela@gmail.com>
Signed-off-by: jeong_in.bae <jeong_in.bae@navercorp.com>
```
  1bf905dd
- [V1] Update interface for idefics3 (#10680) · 0a4d9685
  Roger Wang authored Nov 26, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  0a4d9685
- Remove hard-dependencies of Speculative decode to CUDA workers (#10587) · 0a71900b
  Chendi.Xue authored Nov 26, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
  0a71900b
26 Nov, 2024 13 commits
- [V1] Refactor model executable interface for multimodal models (#10570) · 2f0a0a17
  Roger Wang authored Nov 26, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  2f0a0a17
- [Bugfix] Check bnb_4bit_quant_storage for bitsandbytes (#10642) · 7576cd38
  Michael Goin authored Nov 26, 2024
  
  7576cd38
- [Bugfix] Fix using `-O[0,3]` with LLM entrypoint (#10677) · 9a99273b
  Michael Goin authored Nov 26, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  9a99273b
- [Hardware][NVIDIA] Add non-NVML CUDA mode for Jetson (#9735) · f5792c7c
  Conroy Cheers authored Nov 27, 2024
```
Signed-off-by: Conroy Cheers <conroy@corncheese.org>
```
  f5792c7c
- [Bugfix] Fix for Spec model TP + Chunked Prefill (#10232) · db66e018
  Murali Andoorveedu authored Nov 26, 2024
```
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
Signed-off-by: Sourashis Roy <sroy@roblox.com>
Co-authored-by: Sourashis Roy <sroy@roblox.com>
```
  db66e018
- [V1] Enable profile for LLMEngine (#10665) · 1f6584ee
  Kunshang Ji authored Nov 26, 2024
  
  1f6584ee
- [ci] add vllm_test_utils (#10659) · 334d64d1
  youkaichao authored Nov 26, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  334d64d1
- [Misc] Remove outdated init protocols (#10655) · 94063534
  Cyrus Leung authored Nov 26, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  94063534
- custom allreduce + torch.compile (#10121) · 9a88f897
  Sage Moore authored Nov 26, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  9a88f897
- [v1] EngineArgs for better config handling for v1 (#10382) · 519e8e41
  Ricky Xu authored Nov 25, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  519e8e41
- [Feature] vLLM ARM Enablement for AARCH64 CPUs (#9228) · a6760f64
  Sanket Kale authored Nov 26, 2024
```
Signed-off-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  a6760f64
- [bugfix] fix aria model and add torch.compile (#10645) · 45ac4ff2
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  45ac4ff2
- [misc] do not read HOST_IP (#10644) · 6e9ff050
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6e9ff050