Commits · c8f26bb63694adb4202ab275efb0759c13edcaa8 · OpenDAS / vllm_cscc

06 Oct, 2024 3 commits
- [Bugfix] Fix incorrect updates to num_computed_tokens in multi-step scheduling (#9038) · cb3b2b9b
  Varun Sundar Rabindranath authored Oct 06, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  cb3b2b9b
- [Model] PP support for embedding models and update docs (#9090) · b22b7984
  Cyrus Leung authored Oct 06, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  b22b7984
- [Frontend] API support for beam search (#9087) · 168cab6b
  Brendan Wong authored Oct 05, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  168cab6b
05 Oct, 2024 3 commits
- [Bugfix] Fix order of arguments matters in config.yaml (#8960) · 5df18348
  Andy Dai authored Oct 05, 2024
  
  5df18348
- [Bugfix] Deprecate registration of custom configs to huggingface (#9083) · cfadb9c6
  Chen Zhang authored Oct 05, 2024
  
  cfadb9c6
- [Model] Support Gemma2 embedding model (#9004) · 15986f59
  Xin Yang authored Oct 04, 2024
  
  15986f59
04 Oct, 2024 7 commits
- [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (#8973) · 05d68643
  ElizaWszola authored Oct 04, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika Sikka <ds3822@columbia.edu>
```
  05d68643
- Adds truncate_prompt_tokens param for embeddings creation (#8999) · 0dcc8cbe
  Flávia Béo authored Oct 04, 2024
```
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
```
  0dcc8cbe
- [Core][VLM] Test registration for OOT multimodal models (#8717) · 26aa325f
  Roger Wang authored Oct 04, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  26aa325f
- [Model] add a bunch of supported lora modules for mixtral (#9008) · 9ade8bbc
  Prashant Gupta authored Oct 04, 2024
```
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
```
  9ade8bbc
- [Misc] Move registry to its own file (#9064) · 0e36fd49
  Cyrus Leung authored Oct 04, 2024
  
  0e36fd49
- [Models] Add remaining model PP support (#7168) · 0f6d7a9a
  Murali Andoorveedu authored Oct 03, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Signed-off-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0f6d7a9a
- [Frontend][Feature] support tool calling for internlm/internlm2_5-7b-chat model (#8405) · 3dbb215b
  代君 authored Oct 04, 2024
  
  3dbb215b
03 Oct, 2024 5 commits
- Fix failing spec decode test (#9054) · 91add85e
  sroy745 authored Oct 03, 2024
  
  91add85e
- [misc] add forward context for attention (#9029) · 9aaf14c6
  youkaichao authored Oct 03, 2024
  
  9aaf14c6
- [Frontend] [Neuron] Parse literals out of override-neuron-config (#8959) · 63e39937
  xendo authored Oct 03, 2024
```
Co-authored-by: Jerzy Zagorski <jzagorsk@amazon.com>
```
  63e39937
- [BugFix] Enforce Mistral ToolCall id constraint when using the Mistral tool call parser (#9020) · 83caf35e
  Guillaume Calmettes authored Oct 03, 2024
  
  83caf35e
- [Model] Adding Granite MoE. (#8206) · 19f0d257
  Shawn Tan authored Oct 02, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  19f0d257
02 Oct, 2024 1 commit
- [Core] Combined support for multi-step scheduling, chunked prefill & prefix caching (#8804) · 563649aa
  afeldman-nm authored Oct 02, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
```
  563649aa
01 Oct, 2024 4 commits
- [Spec Decode] (1/2) Remove batch expansion (#8839) · 15702038
  Lily Liu authored Oct 01, 2024
  
  15702038
- [Bugfix] Fix Fuyu tensor parallel inference (#8986) · bc4eb65b
  Isotr0py authored Oct 01, 2024
  
  bc4eb65b
- [Frontend][Core] Move guided decoding params into sampling params (#8252) · 062c89e7
  Joe Runde authored Sep 30, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  062c89e7
- [CI][SpecDecode] Fix spec decode tests, use flash attention backend for spec... · bce32448
  Lily Liu authored Sep 30, 2024
```
[CI][SpecDecode] Fix spec decode tests, use flash attention backend for spec decode CI tests. (#8975)
```
  bce32448
29 Sep, 2024 6 commits
- [Kernel][Model] Varlen prefill + Prefill chunking support for mamba kernels and Jamba model (#8533) · f13a07b1
  Mor Zusman authored Sep 30, 2024
  
  f13a07b1
- [Frontend] Added support for HF's new `continue_final_message` parameter (#8942) · 6c9ba48f
  danieljannai21 authored Sep 29, 2024
  
  6c9ba48f
- [Model][LoRA]LoRA support added for MiniCPMV2.5 (#7199) · 3d49776b
  Jee Jee Li authored Sep 29, 2024
  
  3d49776b
- [CI/Build] Add test decorator for minimum GPU memory (#8925) · 26a68d5d
  Cyrus Leung authored Sep 29, 2024
  
  26a68d5d
- [Bugfix] Fix Marlin MoE act order when is_k_full == False (#8741) · d081da00
  ElizaWszola authored Sep 29, 2024
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  d081da00
- [Bugfix] Block manager v2 with preemption and lookahead slots (#8824) · 5bf8789b
  sroy745 authored Sep 28, 2024
  
  5bf8789b
28 Sep, 2024 2 commits
- [CI/Build] Update models tests & examples (#8874) · e1a3f5e8
  Cyrus Leung authored Sep 29, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  e1a3f5e8
- [Bugfix] Fix PP for Multi-Step (#8887) · 19d02ff9
  Varun Sundar Rabindranath authored Sep 28, 2024
  
  19d02ff9
27 Sep, 2024 6 commits
- [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378) · c2ec430a
  Varun Sundar Rabindranath authored Sep 27, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  c2ec430a
- [Kernel] AQ AZP 4/4: Integrate asymmetric quantization to linear method (#7271) · 172d1cd2
  Luka Govedič authored Sep 27, 2024
  
  172d1cd2
- [torch.compile] use empty tensor instead of None for profiling (#8875) · a9b15c60
  youkaichao authored Sep 27, 2024
  
  a9b15c60
- [Bugfix][VLM] Fix Fuyu batching inference with `max_num_seqs>1` (#8892) · 6d792d2f
  Isotr0py authored Sep 27, 2024
  
  6d792d2f
- [Core] rename`PromptInputs` and `inputs` (#8876) · 3b00b9c2
  Cyrus Leung authored Sep 27, 2024
  
  3b00b9c2
- [Feature] Add support for Llama 3.1 and 3.2 tool use (#8343) · 344cd2b6
  Maximilien de Bayser authored Sep 26, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  344cd2b6
26 Sep, 2024 2 commits
- [BugFix] Fix test breakages from transformers 4.45 upgrade (#8829) · 4b377d6f
  Nick Hill authored Sep 27, 2024
  
  4b377d6f
- fix validation: Only set tool_choice `auto` if at least one tool is provided (#8568) · ee2da3e9
  Chirag Jain authored Sep 27, 2024
  
  ee2da3e9
25 Sep, 2024 1 commit

[Model] Add support for the multi-modal Llama 3.2 model (#8811) · 770ec602

Chen Zhang authored Sep 25, 2024


Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

770ec602