Commits · c2ec430ab5713d0626c1a7809718ef6c4eebf389 · OpenDAS / vllm_cscc · GitLab

27 Sep, 2024 10 commits
- [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378) · c2ec430a
  Varun Sundar Rabindranath authored Sep 27, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  c2ec430a
- [Bugfix] fix for deepseek w4a16 (#8906) · c5d55356
  Lucas Wilkinson authored Sep 27, 2024
```
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  c5d55356
- [Kernel] AQ AZP 4/4: Integrate asymmetric quantization to linear method (#7271) · 172d1cd2
  Luka Govedič authored Sep 27, 2024
  
  172d1cd2
- [torch.compile] use empty tensor instead of None for profiling (#8875) · a9b15c60
  youkaichao authored Sep 27, 2024
  
  a9b15c60
- [TPU] Update pallas.py to support trillium (#8871) · 8df2dc3c
  Brittany authored Sep 27, 2024
  
  8df2dc3c
- [Bugfix][VLM] Fix Fuyu batching inference with `max_num_seqs>1` (#8892) · 6d792d2f
  Isotr0py authored Sep 27, 2024
  
  6d792d2f
- [MISC] Fix invalid escape sequence '\' (#8830) · 0e088750
  Peter Pan authored Sep 27, 2024
```
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
```
  0e088750
- [misc] fix collect env (#8894) · dc4e3df5
  youkaichao authored Sep 27, 2024
  
  dc4e3df5
- [Core] rename`PromptInputs` and `inputs` (#8876) · 3b00b9c2
  Cyrus Leung authored Sep 27, 2024
  
  3b00b9c2
- [Feature] Add support for Llama 3.1 and 3.2 tool use (#8343) · 344cd2b6
  Maximilien de Bayser authored Sep 26, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  344cd2b6
26 Sep, 2024 13 commits
- [Installation] Allow lower versions of FastAPI to maintain Ray 2.9 compatibility (#8764) · 1b49148e
  Cyrus Leung authored Sep 27, 2024
  
  1b49148e
- [BugFix] Fix test breakages from transformers 4.45 upgrade (#8829) · 4b377d6f
  Nick Hill authored Sep 27, 2024
  
  4b377d6f
- [Bugfix] Fixup advance_step.cu warning (#8815) · 71d21c73
  Tyler Michael Smith authored Sep 26, 2024
  
  71d21c73
- fix validation: Only set tool_choice `auto` if at least one tool is provided (#8568) · ee2da3e9
  Chirag Jain authored Sep 27, 2024
  
  ee2da3e9
- [Bugfix] Fix print_warning_once's line info (#8867) · e2f6f26e
  Tyler Michael Smith authored Sep 26, 2024
  
  e2f6f26e
- [Misc] Change dummy profiling and BOS fallback warns to log once (#8820) · b28d2104
  Michael Goin authored Sep 26, 2024
  
  b28d2104
- [Bugfix] Include encoder prompts len to non-stream api usage response (#8861) · 93d364da
  Pernekhan Utemuratov authored Sep 26, 2024
  
  93d364da
- [ci] Soft fail Entrypoints, Samplers, LoRA, Decoder-only VLM (#8872) · d9cfbc89
  Kevin H. Luu authored Sep 26, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  d9cfbc89
- [misc][installation] build from source without compilation (#8818) · 70de39f6
  youkaichao authored Sep 26, 2024
  
  70de39f6
- [CI/Build] Fix missing ci dependencies (#8834) · 68988d4e
  fyuan1316 authored Sep 27, 2024
  
  68988d4e
- [Docs] Add README to the build docker image (#8825) · 520db4db
  Michael Goin authored Sep 26, 2024
  
  520db4db
- [Build/CI] Upgrade to gcc 10 in the base build Docker image (#8814) · f70bccac
  Tyler Michael Smith authored Sep 26, 2024
  
  f70bccac
- [Misc] Update config loading for Qwen2-VL and remove Granite (#8837) · 4bb98f21
  Roger Wang authored Sep 26, 2024
  
  4bb98f21
25 Sep, 2024 17 commits
- [Misc] Support quantization of MllamaForCausalLM (#8822) · 7193774b
  Michael Goin authored Sep 25, 2024
  
  7193774b
- [Doc] Update doc for Transformers 4.45 (#8817) · e2c6e0a8
  Roger Wang authored Sep 25, 2024
  
  e2c6e0a8
- [Model] Add support for the multi-modal Llama 3.2 model (#8811) · 770ec602
  Chen Zhang authored Sep 25, 2024
```
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  770ec602
- Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) · 4f1ba084
  Simon Mo authored Sep 25, 2024
  
  4f1ba084
- [Misc] Support FP8 MoE for compressed-tensors (#8588) · 873edda6
  Michael Goin authored Sep 25, 2024
  
  873edda6
- [Frontend] MQLLMEngine supports profiling. (#8761) · 64840dfa
  科英 authored Sep 26, 2024
  
  64840dfa
- rename PromptInputs and inputs with backward compatibility (#8760) · 28e1299e
  Cyrus Leung authored Sep 26, 2024
  
  28e1299e
- [VLM][Bugfix] internvl with num_scheduler_steps > 1 (#8614) · 0c4d2ad5
  DefTruth authored Sep 26, 2024
  
  0c4d2ad5
- [[Misc]] Add extra deps for openai server image (#8792) · c6f2485c
  Jee Jee Li authored Sep 26, 2024
  
  c6f2485c
- [Kernel] Fullgraph and opcheck tests (#8479) · 300da091
  bnellnm authored Sep 25, 2024
  
  300da091
- [CI/Build][Bugfix][Doc][ROCm] CI fix and doc update after ROCm 6.2 upgrade (#8777) · 1c046447
  Hongxia Yang authored Sep 25, 2024
  
  1c046447
- [Misc] Fix minor typo in scheduler (#8765) · 8fae5ed7
  Woo-Yeon Lee authored Sep 25, 2024
  
  8fae5ed7
- [Bugfix] Ray 2.9.x doesn't expose available_resources_per_node (#8767) · 3368c3ab
  David Newman authored Sep 25, 2024
```
Signed-off-by: darthhexx <darthhexx@gmail.com>
```
  3368c3ab
- [Frontend] OpenAI server: propagate usage accounting to FastAPI middleware layer (#8672) · 1ac3de09
  Adam Tilghman authored Sep 25, 2024
  
  1ac3de09
- [Bugfix] load fc bias from config for eagle (#8790) · 3e073e66
  sohamparikh authored Sep 25, 2024
  
  3e073e66
- [Hardware][CPU] Enable mrope and support Qwen2-VL on CPU backend (#8770) · c2395367
  Isotr0py authored Sep 25, 2024
  
  c2395367
- [BugFix] Propagate 'trust_remote_code' setting in internvl and minicpmv (#8250) · e3dd0692
  zifeitong authored Sep 24, 2024
  
  e3dd0692