Commits · 7a3a83e3b87f50fe9c0985a5c5bcc1d4cf2e95cd · OpenDAS / vllm_cscc · GitLab

11 Jan, 2025 2 commits
- [CI/Build] Move model-specific multi-modal processing tests (#11934) · 7a3a83e3
  Cyrus Leung authored Jan 11, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7a3a83e3
- [ci] fix broken distributed-tests-4-gpus (#11937) · 899136b8
  youkaichao authored Jan 11, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  899136b8
10 Jan, 2025 7 commits
- [Hardware][CPU] Support MOE models on x86 CPU (#11831) · aa1e77a1
  Li, Jiang authored Jan 11, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  aa1e77a1
- [Doc] Rename offline inference examples (#11927) · 482cdc49
  Harry Mellor authored Jan 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  482cdc49
- [ci] Fix sampler tests (#11922) · 241ad7b3
  youkaichao authored Jan 10, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  241ad7b3
- Replace "online inference" with "online serving" (#11923) · d85c47d6
  Harry Mellor authored Jan 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d85c47d6
- [Bugfix] Validate lora adapters to avoid crashing server (#11727) · ac2f3f7f
  Joe Runde authored Jan 10, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ac2f3f7f
- [torch.compile] Hide KV cache behind torch.compile boundary (#11677) · cf5f000d
  Chen Zhang authored Jan 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cf5f000d
- [VLM] Enable tokenized inputs for merged multi-modal processor (#11900) · b844b99a
  Cyrus Leung authored Jan 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b844b99a
09 Jan, 2025 6 commits
- [Misc] Provide correct Pixtral-HF chat template (#11891) · 9a228348
  Cyrus Leung authored Jan 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9a228348
- [ci]try to fix flaky multi-step tests (#11894) · bd828722
  youkaichao authored Jan 09, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  bd828722
- [platform] Allow platform specify attention backend (#11609) · 405eb8e3
  wangxiyuan authored Jan 09, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
```
  405eb8e3
- [Bugfix] Override dunder methods of placeholder modules (#11882) · 0bd1ff43
  Cyrus Leung authored Jan 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0bd1ff43
- treat do_lower_case in the same way as the sentence-transformers library (#11815) · 1fe554ba
  Maximilien de Bayser authored Jan 09, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  1fe554ba
- [CI] Turn on basic correctness tests for V1 (#10864) · 615e4a54
  Tyler Michael Smith authored Jan 08, 2025
  
  615e4a54
08 Jan, 2025 4 commits
- [TPU][Quantization] TPU `W8A8` (#11785) · 56fe4c29
  Robert Shaw authored Jan 08, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  56fe4c29
- [Doc] Move examples into categories (#11840) · aba8d6ee
  Harry Mellor authored Jan 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  aba8d6ee
- [VLM] Reorganize profiling/processing-related code (#11812) · 2a0596bc
  Cyrus Leung authored Jan 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2a0596bc
- [misc] improve memory profiling (#11809) · 889e662e
  youkaichao authored Jan 08, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  889e662e
07 Jan, 2025 3 commits
- [Bugfix] Comprehensively test and fix LLaVA-NeXT feature size calculation (#11800) · 8f37be38
  Cyrus Leung authored Jan 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f37be38
- [Kernel][LoRA]Punica prefill kernels fusion (#11234) · b2785579
  Jee Jee Li authored Jan 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Abatom <abzhonghua@gmail.com>
Co-authored-by: Zhonghua Deng <abatom@163.com>
```
  b2785579
- [Bugfix] Fix LLaVA-NeXT feature size precision error (for real) (#11772) · 08fb75c7
  Cyrus Leung authored Jan 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  08fb75c7
06 Jan, 2025 6 commits
- [V1] Extend beyond image modality and support mixed-modality inference with... · 91b361ae
  Roger Wang authored Jan 06, 2025
```
[V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (#11685)
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  91b361ae
- [Kernel] Move attn_type to Attention.__init__() (#11690) · e20c92bb
  Chen Zhang authored Jan 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e20c92bb
- [Bugfix][V1] Fix molmo text-only inputs (#11676) · 32c9eff2
  Jee Jee Li authored Jan 06, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  32c9eff2
- [VLM] Separate out profiling-related logic (#11746) · 996357e4
  Cyrus Leung authored Jan 06, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  996357e4
- [V1] Refactor get_executor_cls (#11754) · 022c5c69
  Rui Qiao authored Jan 05, 2025
  
  022c5c69
- [distributed] remove pynccl's redundant change_state (#11749) · 9e764e7b
  cennn authored Jan 06, 2025
  
  9e764e7b
05 Jan, 2025 1 commit
- [distributed] remove pynccl's redundant stream (#11744) · 635b8972
  cennn authored Jan 05, 2025
  
  635b8972
04 Jan, 2025 5 commits
- [Bugfix][V1] Fix test_kv_cache_utils.py (#11738) · 47831430
  Jee Jee Li authored Jan 05, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  47831430
- [Bugfix] Fix precision error in LLaVA-NeXT (#11735) · ba214dff
  Cyrus Leung authored Jan 04, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ba214dff
- [VLM] Merged multi-modal processors for LLaVA-NeXT-Video and LLaVA-OneVision (#11717) · eed11ebe
  Cyrus Leung authored Jan 04, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eed11ebe
- [Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (#11233) · 300acb83
  Yan Burman authored Jan 04, 2025
```
Signed-off-by: Yan Burman <yanburman@users.noreply.github.com>
Signed-off-by: Ido Asraff <idoa@atero.ai>
```
  300acb83
- [V1] Add kv cache utils tests. (#11513) · d91457d5
  xcnick authored Jan 04, 2025
```
Signed-off-by: xcnick <xcnick0412@gmail.com>
```
  d91457d5
03 Jan, 2025 2 commits
- [V1] Simplify Shutdown (#11659) · 80c751e7
  Robert Shaw authored Jan 03, 2025
  
  80c751e7
- [Model] Whisper model implementation (#11280) · e1a5c2f0
  Aurick Qiao authored Jan 03, 2025
```
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
```
  e1a5c2f0
02 Jan, 2025 1 commit
- [VLM] Merged multi-modal processor for LLaVA-NeXT (#11682) · 8c38ee70
  Cyrus Leung authored Jan 03, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8c38ee70
01 Jan, 2025 3 commits
- [VLM] Move supported limits and max tokens to merged multi-modal processor (#11669) · a115ac46
  Cyrus Leung authored Jan 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  a115ac46
- [V1] Implement Cascade Attention (#11635) · 73001445
  Woosuk Kwon authored Jan 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  73001445
- [Misc] Optimize Qwen2-VL LoRA test (#11663) · 11d8a091
  Jee Jee Li authored Jan 01, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  11d8a091