Commits · c0d9a98d0c7182b73c2e7f88508e690a186bf0e3 · OpenDAS / vllm_cscc · GitLab

07 Oct, 2024 10 commits
- [Doc] Include performance benchmark in README (#9135) · c0d9a98d
  Kuntai Du authored Oct 07, 2024
  
  c0d9a98d
- [CI/Build] Add linting for github actions workflows (#7876) · e0dbdb01
  Russell Bryant authored Oct 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  e0dbdb01
- [Doc]: Add deploying_with_k8s guide (#8451) · 93cf74a8
  TimWang authored Oct 08, 2024
  
  93cf74a8
- [Model] Support NVLM-D and fix QK Norm in InternViT (#9045) · 151ef4ef
  Cyrus Leung authored Oct 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  151ef4ef
- [Core] Refactor GGUF parameters packing and forwarding (#8859) · f19da648
  Isotr0py authored Oct 07, 2024
  
  f19da648
- [Hardware][CPU] Cross-attention and Encoder-Decoder models support on CPU backend (#9089) · 4f95ffee
  Isotr0py authored Oct 07, 2024
  
  4f95ffee
- [Model] Explicit interface for vLLM models and support OOT embedding models (#9108) · 8c6de96e
  Cyrus Leung authored Oct 07, 2024
  
  8c6de96e
- [core] remove beam search from the core (#9105) · 18b296fd
  youkaichao authored Oct 06, 2024
  
  18b296fd
- [BugFix][Core] Fix BlockManagerV2 when Encoder Input is None (#9103) · c8f26bb6
  sroy745 authored Oct 06, 2024
  
  c8f26bb6
- [Bugfix][Hardware][CPU] Fix CPU model input for decode (#9044) · 487678d0
  Isotr0py authored Oct 07, 2024
  
  487678d0
06 Oct, 2024 7 commits
- [Bugfix] Fix incorrect updates to num_computed_tokens in multi-step scheduling (#9038) · cb3b2b9b
  Varun Sundar Rabindranath authored Oct 06, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  cb3b2b9b
- [Bugfix] fix tool_parser error handling when serve a model not support it (#8709) · fdf59d30
  Yanyi Liu authored Oct 06, 2024
  
  fdf59d30
- [Model] PP support for embedding models and update docs (#9090) · b22b7984
  Cyrus Leung authored Oct 06, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  b22b7984
- [Misc] Remove user-facing error for removed VLM args (#9104) · f22619fe
  Cyrus Leung authored Oct 06, 2024
  
  f22619fe
- [Frontend] API support for beam search (#9087) · 168cab6b
  Brendan Wong authored Oct 05, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  168cab6b
- [Bugfix] Fix try-catch conditions to import correct Flash Attention Backend in Draft Model (#9101) · 23fea871
  TJian authored Oct 05, 2024
  
  23fea871
- [core] use forward context for flash infer (#9097) · f4dd830e
  youkaichao authored Oct 05, 2024
  
  f4dd830e
05 Oct, 2024 6 commits
- [Bugfix] Fix order of arguments matters in config.yaml (#8960) · 5df18348
  Andy Dai authored Oct 05, 2024
  
  5df18348
- [Bugfix] Deprecate registration of custom configs to huggingface (#9083) · cfadb9c6
  Chen Zhang authored Oct 05, 2024
  
  cfadb9c6
- [Model] Support Gemma2 embedding model (#9004) · 15986f59
  Xin Yang authored Oct 04, 2024
  
  15986f59
- [Bugfix] Fixes Phi3v & Ultravox Multimodal EmbeddingInputs (#8979) · 53b3a330
  hhzhang16 authored Oct 04, 2024
  
  53b3a330
- [Bugfix] use blockmanagerv1 for encoder-decoder (#9084) · dac914b0
  Chen Zhang authored Oct 04, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  dac914b0
- [Doc] Update README.md with Ray summit slides (#9088) · a95354a3
  Zhuohan Li authored Oct 04, 2024
  
  a95354a3
04 Oct, 2024 17 commits
- [torch.compile] improve allreduce registration (#9061) · 663874e0
  youkaichao authored Oct 04, 2024
  
  663874e0
- [Hardware][Neuron] Add on-device sampling support for Neuron (#8746) · cc90419e
  Chongming Ni authored Oct 04, 2024
```
Co-authored-by: Ashraf Mahgoub <ashymahg@amazon.com>
```
  cc90419e
- [Misc] Fix CI lint (#9085) · 27302dd5
  Cody Yu authored Oct 04, 2024
  
  27302dd5
- [Misc] Add random seed for prefix cache benchmark (#9081) · 0cc566ca
  Andy Dai authored Oct 04, 2024
  
  0cc566ca
- [Misc] Improved prefix cache example (#9077) · 05c531be
  Andy Dai authored Oct 04, 2024
  
  05c531be
- [CI] Update performance benchmark: upgrade trt-llm to r24.07, and add SGLang (#7412) · fbb74420
  Kuntai Du authored Oct 04, 2024
  
  fbb74420
- [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (#8973) · 05d68643
  ElizaWszola authored Oct 04, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika Sikka <ds3822@columbia.edu>
```
  05d68643
- Adds truncate_prompt_tokens param for embeddings creation (#8999) · 0dcc8cbe
  Flávia Béo authored Oct 04, 2024
```
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
```
  0dcc8cbe
- [Core][VLM] Test registration for OOT multimodal models (#8717) · 26aa325f
  Roger Wang authored Oct 04, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  26aa325f
- [Hardware][PowerPC] Make oneDNN dependency optional for Power (#9039) · e5dc713c
  Varad Ahirwadkar authored Oct 04, 2024
```
Signed-off-by: Varad Ahirwadkar <varad.ahirwadkar1@ibm.com>
```
  e5dc713c
- Remove AMD Ray Summit Banner (#9075) · 36eecfbd
  Simon Mo authored Oct 04, 2024
  
  36eecfbd
- [Model] add a bunch of supported lora modules for mixtral (#9008) · 9ade8bbc
  Prashant Gupta authored Oct 04, 2024
```
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
```
  9ade8bbc
- [Bugfix] Flash attention arches not getting set properly (#9062) · 22482e49
  Lucas Wilkinson authored Oct 04, 2024
  
  22482e49
- [Bugfix] Reshape the dimensions of the input image embeddings in Qwen2VL (#9071) · 3d826d2c
  whyiug authored Oct 04, 2024
  
  3d826d2c
- [Misc] Move registry to its own file (#9064) · 0e36fd49
  Cyrus Leung authored Oct 04, 2024
  
  0e36fd49
- [Models] Add remaining model PP support (#7168) · 0f6d7a9a
  Murali Andoorveedu authored Oct 03, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Signed-off-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0f6d7a9a
- [Misc] Enable multi-step output streaming by default (#9047) · 303d4479
  Michael Goin authored Oct 03, 2024
  
  303d4479