Commits · e808156f305ce2ecfbe87eefa19ce2ae11c83d00 · OpenDAS / vllm_cscc · GitLab

11 Oct, 2024 2 commits
- [Misc] Collect model support info in a single process per model (#9233) · e808156f
  Cyrus Leung authored Oct 11, 2024
  
  e808156f
- [misc] hide best_of from engine (#9261) · cbc2ef55
  youkaichao authored Oct 10, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  cbc2ef55
10 Oct, 2024 7 commits
- [torch.compile] generic decorators (#9258) · e00c094f
  youkaichao authored Oct 10, 2024
  
  e00c094f
- [torch.compile] integration with compilation control (#9058) · e4d652ea
  youkaichao authored Oct 10, 2024
  
  e4d652ea
- [Model] support input image embedding for minicpmv (#9237) · 04de9057
  whyiug authored Oct 10, 2024
  
  04de9057
- [Bugfix] Fix lm_head weights tying with lora for llama (#9227) · 07c11cf4
  Isotr0py authored Oct 10, 2024
  
  07c11cf4
- [Core] Add an environment variable which needs to be set explicitly to allow... · f3a507f1
  sroy745 authored Oct 09, 2024
```
[Core] Add an environment variable which needs to be set explicitly to allow BlockSpaceManagerV1 (#9149)
```
  f3a507f1
- [misc] improve model support check in another process (#9208) · de895f16
  youkaichao authored Oct 09, 2024
  
  de895f16
- [Core] Fix invalid args to _process_request (#9201) · cf25b93b
  Russell Bryant authored Oct 10, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  cf25b93b
09 Oct, 2024 10 commits
- [CI/Build] mypy: check vllm/entrypoints (#9194) · cdca8994
  Russell Bryant authored Oct 09, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  cdca8994
- [Hardware][CPU] Support AWQ for CPU backend (#7515) · ca77dd7a
  Li, Jiang authored Oct 10, 2024
  
  ca77dd7a
- [Bugfix] Access `get_vocab` instead of `vocab` in tool parsers (#9188) · cfaa6008
  Cyrus Leung authored Oct 09, 2024
  
  cfaa6008
- [Bugfix] Fix lora loading for Compressed Tensors in #9120 (#9179) · 21906a6f
  Ahmad Fahadh Ilyas authored Oct 09, 2024
  
  21906a6f
- [ci][test] use load dummy for testing (#9165) · c8627cd4
  youkaichao authored Oct 09, 2024
  
  c8627cd4
- [Bugfix] fix composite weight loading and EAGLE weight loading (#9160) · 8bfaa4e3
  Cyrus Leung authored Oct 09, 2024
  
  8bfaa4e3
- [Frontend] Log the maximum supported concurrency (#8831) · 0b5b5d76
  AlpinDale authored Oct 09, 2024
  
  0b5b5d76
- [Model] Remap FP8 kv_scale in CommandR and DBRX (#9174) · cdc72e3c
  Hui Liu authored Oct 08, 2024
  
  cdc72e3c
- [Misc] Improve validation errors around best_of and n (#9167) · 480b7f40
  Travis Johnson authored Oct 08, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  480b7f40
- support bitsandbytes quantization with more models (#9148) · 2f4117c3
  chenqianfzh authored Oct 08, 2024
  
  2f4117c3
08 Oct, 2024 7 commits
- mypy: check additional directories (#9162) · 2a131965
  Russell Bryant authored Oct 08, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  2a131965
- [Bugfix] Try to handle older versions of pytorch (#9086) · bd37b9fb
  bnellnm authored Oct 08, 2024
  
  bd37b9fb
- [Bugfix] fix OpenAI API server startup with --disable-frontend-multiprocessing (#8537) · 9a94ca4a
  Daniele authored Oct 08, 2024
  
  9a94ca4a
- [Frontend] Add Early Validation For Chat Template / Tool Call Parser (#9151) · 069d3bd8
  Alex Brooks authored Oct 08, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  069d3bd8
- [Core][Frontend] Add Support for Inference Time mm_processor_kwargs (#9131) · a3691b6b
  Alex Brooks authored Oct 08, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  a3691b6b
- [Frontend] API support for beam search for MQLLMEngine (#9117) · 8c746226
  Brendan Wong authored Oct 07, 2024
  
  8c746226
- [Intel GPU] Fix xpu decode input (#9145) · 80b57f00
  Kunshang Ji authored Oct 08, 2024
  
  80b57f00
07 Oct, 2024 8 commits
- [misc] fix comment and variable name (#9139) · fa45513a
  youkaichao authored Oct 07, 2024
  
  fa45513a
- [Model] Support NVLM-D and fix QK Norm in InternViT (#9045) · 151ef4ef
  Cyrus Leung authored Oct 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  151ef4ef
- [Core] Refactor GGUF parameters packing and forwarding (#8859) · f19da648
  Isotr0py authored Oct 07, 2024
  
  f19da648
- [Hardware][CPU] Cross-attention and Encoder-Decoder models support on CPU backend (#9089) · 4f95ffee
  Isotr0py authored Oct 07, 2024
  
  4f95ffee
- [Model] Explicit interface for vLLM models and support OOT embedding models (#9108) · 8c6de96e
  Cyrus Leung authored Oct 07, 2024
  
  8c6de96e
- [core] remove beam search from the core (#9105) · 18b296fd
  youkaichao authored Oct 06, 2024
  
  18b296fd
- [BugFix][Core] Fix BlockManagerV2 when Encoder Input is None (#9103) · c8f26bb6
  sroy745 authored Oct 06, 2024
  
  c8f26bb6
- [Bugfix][Hardware][CPU] Fix CPU model input for decode (#9044) · 487678d0
  Isotr0py authored Oct 07, 2024
  
  487678d0
06 Oct, 2024 6 commits
- [Bugfix] Fix incorrect updates to num_computed_tokens in multi-step scheduling (#9038) · cb3b2b9b
  Varun Sundar Rabindranath authored Oct 06, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  cb3b2b9b
- [Bugfix] fix tool_parser error handling when serve a model not support it (#8709) · fdf59d30
  Yanyi Liu authored Oct 06, 2024
  
  fdf59d30
- [Model] PP support for embedding models and update docs (#9090) · b22b7984
  Cyrus Leung authored Oct 06, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  b22b7984
- [Misc] Remove user-facing error for removed VLM args (#9104) · f22619fe
  Cyrus Leung authored Oct 06, 2024
  
  f22619fe
- [Frontend] API support for beam search (#9087) · 168cab6b
  Brendan Wong authored Oct 05, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  168cab6b
- [Bugfix] Fix try-catch conditions to import correct Flash Attention Backend in Draft Model (#9101) · 23fea871
  TJian authored Oct 05, 2024
  
  23fea871