Commits · 571841b7fcc67f8b1d171522f6249ed4224033e1 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "512f52219240b0aa1be687955ab52fcdd0c5a40e"

25 Nov, 2024 4 commits
- [torch.compile] support encoder based models (#10613) · 571841b7
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  571841b7
- [Refactor][MISC] del redundant code in ParallelConfig.postinit (#10614) · 7ea3cd7c
  Mengqing Cao authored Nov 25, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  7ea3cd7c
- Support Cross encoder models (#10400) · 214efc2c
  Maximilien de Bayser authored Nov 24, 2024
```
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
```
  214efc2c
- [Doc] Update README.md with Ray Summit talk links (#10610) · 49628fe1
  Zhuohan Li authored Nov 24, 2024
  
  49628fe1
24 Nov, 2024 6 commits
- [doc] update the code to add models (#10603) · e4fbb144
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  e4fbb144
- [model][utils] add extract_layer_index utility function (#10599) · c0557478
  youkaichao authored Nov 23, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c0557478
- Revert "Print running script to enhance CI log readability" (#10601) · eda2b358
  youkaichao authored Nov 23, 2024
  
  eda2b358
- [CI/Build] Print running script to enhance CI log readability (#10594) · 1c445dca
  Jee Jee Li authored Nov 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1c445dca
- [Bugfix] Fix LoRA weight sharding (#10450) · 1700c543
  Jee Jee Li authored Nov 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  1700c543
- [bugfix] Fix example/tensorize_vllm_model tests (#10595) · 17d8fc18
  Jee Jee Li authored Nov 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  17d8fc18
23 Nov, 2024 18 commits
- [Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593) · 04668ebe
  Isotr0py authored Nov 24, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  04668ebe
- For ppc64le, disabled tests for now and addressed space issues (#10538) · 651f6c31
  Nishidha authored Nov 23, 2024
  
  651f6c31
- [Platforms] Refactor openvino code (#10573) · 86a44fb8
  JiHuazhong authored Nov 23, 2024
```
Signed-off-by: statelesshz <hzji210@gmail.com>
```
  86a44fb8
- [Bugfix] `multi_modal_kwargs` broadcast for CPU tensor parallel (#10541) · 4cfe5d2b
  Isotr0py authored Nov 23, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  4cfe5d2b
- [2/N] handling placeholders in merged multi-modal processor (#10485) · c8acd805
  Cyrus Leung authored Nov 23, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c8acd805
- Prefix Cache Aware Scheduling [1/n] (#10128) · 4634a89d
  Ricky Xu authored Nov 22, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  4634a89d
- [AMD] Add support for GGUF quantization on ROCm (#10254) · 7c25fe45
  kliuae authored Nov 23, 2024
  
  7c25fe45
- Update default max_num_batch_tokens for chunked prefill to 2048 (#10544) · 02a43f82
  Michael Goin authored Nov 23, 2024
  
  02a43f82
- [Model] Fix Baichuan BNB online quantization (#10572) · cfea9c04
  Chen Wu authored Nov 23, 2024
```
Signed-off-by: Chen Wu <cntryroa@gmail.com>
```
  cfea9c04
- [Bugfix] Internal Server Error when tool_choice is incorrect. (#10567) · 7d8ffb34
  Varun Vinayak Shenoy authored Nov 22, 2024
```
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
```
  7d8ffb34
- [core] gemma2 full context length support (#10584) · 4aba6e3d
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4aba6e3d
- [Misc] Add pynccl wrappers for all_gather and reduce_scatter (#9432) · 978b3974
  Tyler Michael Smith authored Nov 22, 2024
  
  978b3974
- [Core] Fix broken log configuration (#10458) · ebda5196
  Russell Bryant authored Nov 22, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ebda5196
- [Bugfix][Frontend] Update Llama Chat Templates to also support Non-Tool use (#10164) · 9195dbdb
  Travis Johnson authored Nov 22, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  9195dbdb
- [bugfix] fix cpu tests (#10585) · d559979c
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d559979c
- [V1] EngineCore supports profiling (#10564) · d345f409
  Zhonghua Deng authored Nov 23, 2024
```
Signed-off-by: Abatom <abzhonghua@gmail.com>
```
  d345f409
- [Core] remove temporary local variables in LLMEngine.__init__ (#10577) · 28598f39
  Russell Bryant authored Nov 22, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  28598f39
- support bitsandbytes quantization with qwen model (#10549) · 948c8595
  zixuanzhang226 authored Nov 22, 2024
```
Signed-off-by: Ubuntu <zixuanzhang@bytedance.com>
```
  948c8595
22 Nov, 2024 9 commits
- [v1] Refactor KVCacheManager for more hash input than token ids (#10507) · 97814fbf
  Ricky Xu authored Nov 22, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  97814fbf
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
- [bugfix] fix full graph tests (#10581) · db100c5c
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  db100c5c
- Remove token-adding chat embedding params (#10551) · 11fcf0e0
  Noam Gat authored Nov 22, 2024
```
Signed-off-by: Noam Gat <noamgat@gmail.com>
```
  11fcf0e0
- [Bugfix] Fix Phi-3 BNB quantization with tensor parallel (#9948) · b6374e09
  Isotr0py authored Nov 22, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b6374e09
- [platforms] absorb worker cls difference into platforms folder (#10555) · a111d015
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  a111d015
- [Minor] Fix line-too-long (#10563) · 446c7806
  Woosuk Kwon authored Nov 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  446c7806
- [9/N] torch.compile LLM usage (#10552) · 33e0a254
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  33e0a254
- [Benchmark] Add new H100 machine (#10547) · aed07486
  Simon Mo authored Nov 21, 2024
  
  aed07486
21 Nov, 2024 3 commits
- Add small example to metrics.rst (#10550) · 9afa0145
  Michael Goin authored Nov 21, 2024
  
  9afa0145
- [Minor] Revert change in offline inference example (#10545) · 46fe9b46
  Woosuk Kwon authored Nov 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  46fe9b46
- [misc] improve error message (#10553) · cf656f5a
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cf656f5a