Commits · ccede2b264668d854cba4fce7f8fbbf203908f60 · OpenDAS / vllm_cscc

12 Dec, 2024 1 commit

[V1] VLM preprocessor hashing (#11020) · 4e116833

Alexander Matveev authored Dec 11, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

4e116833

11 Dec, 2024 5 commits
- [CI/Build] Split up VLM tests (#11083) · d1e21a97
  Cyrus Leung authored Dec 12, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d1e21a97
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
- [CI/Build] Check transformers v4.47 (#10991) · 2e33fe41
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2e33fe41
- [Model] PP support for Mamba-like models (#10992) · ffa48c91
  Mor Zusman authored Dec 11, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  ffa48c91
- [Misc] LoRA + Chunked Prefill (#9057) · d5c5154f
  Aurick Qiao authored Dec 10, 2024
  
  d5c5154f
10 Dec, 2024 3 commits
- [Misc][LoRA] Add PEFTHelper for LoRA (#11003) · d05f8867
  Jee Jee Li authored Dec 10, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  d05f8867
- monitor metrics of tokens per step using cudagraph batchsizes (#11031) · ebf77806
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ebf77806
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
09 Dec, 2024 2 commits
- [Model] merged input processor for Phi-3-Vision models (#10977) · a811dd66
  Isotr0py authored Dec 10, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a811dd66
- [Misc][LoRA] Abstract PunicaWrapper (#10955) · ca871491
  Jee Jee Li authored Dec 10, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ca871491
08 Dec, 2024 1 commit
- [torch.compile] allow candidate compile sizes (#10984) · fd57d2b5
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  fd57d2b5
07 Dec, 2024 4 commits
- [BugFix][Kernel]: fix illegal memory access in causal_conv1d when conv_states is None (#10928) · 78029b34
  zhou fan authored Dec 08, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
```
  78029b34
- [Doc] Explicitly state that PP isn't compatible with speculative decoding yet (#10975) · c889d588
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c889d588
- [Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) · 39e227c7
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  39e227c7
- [3/N] Support and implement merged input processor for LLaVA model (#10676) · 955fa953
  Cyrus Leung authored Dec 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  955fa953
06 Dec, 2024 1 commit
- [CI/Build] Fix broken multimodal test (#10950) · 222f5b08
  Cyrus Leung authored Dec 06, 2024
  
  222f5b08
05 Dec, 2024 4 commits
- [ci][build] add tests for python only compilation (#10915) · 9743d64e
  youkaichao authored Dec 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  9743d64e
- [CI/Build] Bump test transformers version (#10106) · 998eeafe
  Isotr0py authored Dec 06, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  998eeafe
- [Misc][LoRA] Clean up the function interface of Punica (#10917) · 571da8fc
  Jee Jee Li authored Dec 05, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  571da8fc
- [Bugfix] Fallback to outlines for complex json schemas (#10899) · 8d370e91
  Michael Goin authored Dec 04, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  8d370e91
04 Dec, 2024 2 commits
- [LoRA] Change lora_tokenizers capacity (#10796) · 01d079fd
  Xin Yang authored Dec 04, 2024
```
Signed-off-by: Xin Yang <xyang19@gmail.com>
```
  01d079fd
- [CI/Build] Replace mean with torch.all in test_pynccl.py (#10876) · d2bd88b1
  Tyler Michael Smith authored Dec 03, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  d2bd88b1
03 Dec, 2024 4 commits
- [V1] VLM - Run the mm_mapper preprocessor in the frontend process (#10640) · 3bc94cab
  Alexander Matveev authored Dec 03, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  3bc94cab
- [Core][Performance] Add XGrammar support for guided decoding and set it as default (#10785) · 9323a315
  Aaron Pham authored Dec 03, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  9323a315
- [torch.compile] remove compilation_context and simplify code (#10838) · dc5ce861
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dc5ce861
- [core][distributed] add pynccl broadcast (#10843) · 21fe7b48
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  21fe7b48
02 Dec, 2024 4 commits

[Misc][LoRA] Move the implementation of lora bias to punica.py (#10829) · b45f0d79
Jee Jee Li authored Dec 03, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
b45f0d79

[Model]: add some tests for aria model (#10770) · ef31eabc

zhou fan authored Dec 02, 2024


Signed-off-by: xffxff <1247714429@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

ef31eabc

[Kernel] Use `out` arg in flash_attn_varlen_func (#10811) · 073a4bd1
Woosuk Kwon authored Dec 01, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
073a4bd1

[Core] Implement disagg prefill by StatelessProcessGroup (#10502) · 0590ec3f

Kuntai Du authored Dec 01, 2024



This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>

0590ec3f

01 Dec, 2024 2 commits
- [Misc] Rename embedding classes to pooling (#10801) · d2f058e7
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2f058e7
- [Model] Replace embedding models with pooling adapter (#10769) · 13370712
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  13370712
29 Nov, 2024 1 commit
- [Model] Clean up MiniCPMV (#10751) · fa6ecb9a
  Cyrus Leung authored Nov 29, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  fa6ecb9a
28 Nov, 2024 3 commits
- [Model] Added GLM-4 series hf format model support vllm==0.6.4 (#10561) · 5fc5ce0f
  sixgod authored Nov 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  5fc5ce0f
- [V1] Do not allocate beyond the max_model_len (#10730) · a79b1224
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  a79b1224
- [Bug][CLI] Allow users to disable prefix caching explicitly (#10724) · d9b4b3f0
  Ricky Xu authored Nov 27, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  d9b4b3f0
27 Nov, 2024 3 commits
- [Frontend] don't block event loop in tokenization (preprocess) in OpenAI compatible server (#10635) · 395b1c74
  tomeras91 authored Nov 27, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  395b1c74
- [Bugfix][Mamba] Fix Multistep on Mamba-like models (#10705) · 197b4484
  Mor Zusman authored Nov 27, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  197b4484
- [ci] fix slow tests (#10698) · 308cc5e2
  youkaichao authored Nov 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  308cc5e2