Commits · eebad39f265606cfe35af4d1e0bea678516648a3 · OpenDAS / vllm_cscc · GitLab

22 Nov, 2024 3 commits
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
- [bugfix] fix full graph tests (#10581) · db100c5c
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  db100c5c
- [9/N] torch.compile LLM usage (#10552) · 33e0a254
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  33e0a254
21 Nov, 2024 7 commits
- [8/N] enable cli flag without a space (#10529) · 7560ae5c
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7560ae5c
- [Kernel] Register punica ops directly (#10522) · 2385b60d
  Jee Jee Li authored Nov 22, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  2385b60d
- [Bug]: When apply continue_final_message for OpenAI server, the "echo":false is ignored (#10180) · da7e702c
  Chauncey authored Nov 22, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  da7e702c
- [Model] Expose `dynamic_image_size` as mm_processor_kwargs for InternVL2 models (#10518) · d5ec121f
  Isotr0py authored Nov 21, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d5ec121f
- [torch.compile] Inductor code caching fix (#10273) · 8b0fe06c
  Luka Govedič authored Nov 21, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
```
  8b0fe06c
- [Core] Add Sliding Window Support with Flashinfer (#10462) · 6c1208d0
  Pavani Majety authored Nov 20, 2024
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  6c1208d0
- [torch.compile] limit inductor threads and lazy import quant (#10482) · 388ee3de
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  388ee3de
20 Nov, 2024 4 commits
- [Bugfix]: allow extra fields in requests to openai compatible server (#10463) · c68f7ede
  Guillaume Calmettes authored Nov 20, 2024
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  c68f7ede
- [7/N] torch.compile, reduce compilation time (#10460) · 0cd3d971
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0cd3d971
- [Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) · 63f1fde2
  Li, Jiang authored Nov 20, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  63f1fde2
- [Bugfix] Marlin 2:4 temp fix for large M dim (>256) (#10464) · d200972e
  Lucas Wilkinson authored Nov 19, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d200972e
19 Nov, 2024 3 commits
- [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
  ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  b00b33d7
- [6/N] torch.compile rollout to users (#10437) · 803f37ea
  youkaichao authored Nov 19, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  803f37ea
- [Platform][Refactor] Extract func `get_default_attn_backend` to `Platform` (#10358) · 8c1fb507
  Mengqing Cao authored Nov 19, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  8c1fb507
18 Nov, 2024 3 commits
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
- [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) · 6b2d25ef
  Yan Ma authored Nov 19, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  6b2d25ef
- [VLM] Report multi_modal_placeholders in output (#10407) · c7dec926
  lkchen authored Nov 18, 2024
```
Signed-off-by: Linkun Chen <lkchen+anyscale@github.com>
```
  c7dec926
17 Nov, 2024 1 commit
- [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) · 4fd93750
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4fd93750
16 Nov, 2024 2 commits
- [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) · 361c29e1
  电脑星人 authored Nov 17, 2024
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  361c29e1
- [Frontend] Automatic detection of chat content format from AST (#9919) · 32e46e00
  Cyrus Leung authored Nov 16, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  32e46e00
15 Nov, 2024 7 commits
- [Misc] Bump up test_fused_moe tolerance (#10364) · 79ee45b4
  ElizaWszola authored Nov 15, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  79ee45b4
- [Misc] Fix import error in tensorizer tests and cleanup some code (#10349) · b311efd0
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b311efd0
- [Misc] Consolidate pooler config overrides (#10351) · 2ac6d0e7
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2ac6d0e7
- [Model] Support Qwen2 embeddings and use tags to select model tests (#10184) · b40cf640
  Cyrus Leung authored Nov 15, 2024
  
  b40cf640
- [bugfix] Fix static asymmetric quantization case (#10334) · bf2ddc66
  Luka Govedič authored Nov 14, 2024
```
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
```
  bf2ddc66
- [Bugfix] Fix unable to load some models (#10312) · 972112d8
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  972112d8
- [Tool parsing] Improve / correct mistral tool parsing (#10333) · 11cd1ae6
  Patrick von Platen authored Nov 15, 2024
  
  11cd1ae6
14 Nov, 2024 4 commits
- Support Roberta embedding models (#9387) · 4a18fd14
  Maximilien de Bayser authored Nov 14, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
```
  4a18fd14
- [ci][distributed] disable hanging tests (#10317) · 29f3ef26
  youkaichao authored Nov 14, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  29f3ef26
- [Frontend] Pythonic tool parser (#9859) · f67ce05d
  Mike Depinet authored Nov 13, 2024
```
Signed-off-by: Mike Depinet <mike@fixie.ai>
```
  f67ce05d
- [Bugfix] Fix tensor parallel for qwen2 classification model (#10297) · 15bb8330
  Isotr0py authored Nov 14, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  15bb8330
13 Nov, 2024 6 commits
- [Bugfix] bitsandbytes models fail to run pipeline parallel (#10200) · ac49b59d
  HoangCongDuc authored Nov 14, 2024
```
Signed-off-by: Hoang Cong Duc <hoangcongducltt@gmail.com>
```
  ac49b59d
- [1/N] Initial prototype for multi-modal processor (#10044) · 0b8bb86b
  Cyrus Leung authored Nov 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0b8bb86b
- [Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944) · 1b886aa1
  Austin Veselka authored Nov 13, 2024
```
Signed-off-by: FurtherAI <austin.veselka@lighton.ai>
Co-authored-by: FurtherAI <austin.veselka@lighton.ai>
```
  1b886aa1
- [Model] Add support for Qwen2-VL video embeddings input & multiple image... · 3945c823
  电脑星人 authored Nov 13, 2024
```
[Model] Add support for Qwen2-VL video embeddings input & multiple image embeddings input with varied resolutions (#10221)
Signed-off-by: imkero <kerorek@outlook.com>
```
  3945c823
- [core][distributed] use tcp store directly (#10275) · 0d4ea3fb
  youkaichao authored Nov 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0d4ea3fb
- [V1] Fix CI tests on V1 engine (#10272) · 112fa0bb
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  112fa0bb