Commits · 11fcf0e0661365f24bfff9591434a0cec640df6c · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "0f2f24c8b205b5bf2dadacf1f95f1ad9f7de73e0"

22 Nov, 2024 6 commits
- Remove token-adding chat embedding params (#10551) · 11fcf0e0
  Noam Gat authored Nov 22, 2024
```
Signed-off-by: Noam Gat <noamgat@gmail.com>
```
  11fcf0e0
- [Bugfix] Fix Phi-3 BNB quantization with tensor parallel (#9948) · b6374e09
  Isotr0py authored Nov 22, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b6374e09
- [platforms] absorb worker cls difference into platforms folder (#10555) · a111d015
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  a111d015
- [Minor] Fix line-too-long (#10563) · 446c7806
  Woosuk Kwon authored Nov 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  446c7806
- [9/N] torch.compile LLM usage (#10552) · 33e0a254
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  33e0a254
- [Benchmark] Add new H100 machine (#10547) · aed07486
  Simon Mo authored Nov 21, 2024
  
  aed07486
21 Nov, 2024 21 commits
- Add small example to metrics.rst (#10550) · 9afa0145
  Michael Goin authored Nov 21, 2024
  
  9afa0145
- [Minor] Revert change in offline inference example (#10545) · 46fe9b46
  Woosuk Kwon authored Nov 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  46fe9b46
- [misc] improve error message (#10553) · cf656f5a
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cf656f5a
- [CI][Installation] Avoid uploading CUDA 11.8 wheel (#10535) · edec3385
  Yunmeng authored Nov 22, 2024
```
Signed-off-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  edec3385
- [V1] Fix Compilation config & Enable CUDA graph by default (#10528) · f9310cbd
  Woosuk Kwon authored Nov 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f9310cbd
- [8/N] enable cli flag without a space (#10529) · 7560ae5c
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7560ae5c
- [Bugfix] Allow token ID-only inputs in Qwen2-Audio (#10536) · e7a8341c
  Cyrus Leung authored Nov 22, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e7a8341c
- [Misc] Suppress duplicated logging regarding multimodal input pipeline (#10530) · c51e397f
  Roger Wang authored Nov 21, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  c51e397f
- [Kernel] Register punica ops directly (#10522) · 2385b60d
  Jee Jee Li authored Nov 22, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  2385b60d
- [Bug]: When apply continue_final_message for OpenAI server, the "echo":false is ignored (#10180) · da7e702c
  Chauncey authored Nov 22, 2024
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  da7e702c
- [Bugfix] Embedding model pooling_type equals ALL and multi input's bug (#10494) · 4d676f08
  Xiaoyu Zhang authored Nov 21, 2024
  
  4d676f08
- [Model] Expose `dynamic_image_size` as mm_processor_kwargs for InternVL2 models (#10518) · d5ec121f
  Isotr0py authored Nov 21, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d5ec121f
- fix the issue that len(tokenizer(prompt)["input_ids"]) > prompt_len (#10524) · 8a93a598
  Wang, Yi authored Nov 21, 2024
```
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
```
  8a93a598
- [Model] Add Support for Multimodal Granite Models (#10291) · 1cfde82f
  Alex Brooks authored Nov 21, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  1cfde82f
- [Doc] fix a small typo in docstring of llama_tool_parser (#10513) · f0e02380
  Zhong Qishuai authored Nov 21, 2024
  
  f0e02380
- [platforms] improve error message for unspecified platforms (#10520) · aaddce5d
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  aaddce5d
- [Misc] Increase default video fetch timeout (#10495) · 3430857b
  Cyrus Leung authored Nov 21, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3430857b
- [torch.compile] Inductor code caching fix (#10273) · 8b0fe06c
  Luka Govedič authored Nov 21, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
```
  8b0fe06c
- [Platforms] Add `device_type` in `Platform` (#10508) · 9d827170
  Mengqing Cao authored Nov 21, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  9d827170
- [Core] Add Sliding Window Support with Flashinfer (#10462) · 6c1208d0
  Pavani Majety authored Nov 20, 2024
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  6c1208d0
- [torch.compile] limit inductor threads and lazy import quant (#10482) · 388ee3de
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  388ee3de
20 Nov, 2024 13 commits
- [TPU] Implement prefix caching for TPUs (#10307) · 2f77b6cf
  Woosuk Kwon authored Nov 20, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2f77b6cf
- [Bugfix]: allow extra fields in requests to openai compatible server (#10463) · c68f7ede
  Guillaume Calmettes authored Nov 20, 2024
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  c68f7ede
- [7/N] torch.compile, reduce compilation time (#10460) · 0cd3d971
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0cd3d971
- [perf bench] H200 development (#9768) · 5f1d6af2
  Simon Mo authored Nov 20, 2024
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  5f1d6af2
- [platforms] restore xpu check for parallel config (#10479) · 772a6673
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  772a6673
- [Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) · 63f1fde2
  Li, Jiang authored Nov 20, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  63f1fde2
- [Platforms] Refactor xpu code (#10468) · d5b28447
  Mengqing Cao authored Nov 20, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  d5b28447
- [Bugfix] Handle conflicts between modern and legacy fields (#10471) · 09dbf9ff
  Cyrus Leung authored Nov 20, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  09dbf9ff
- [model] Reduce medusa weight (#10454) · 343041c4
  Sky Lee authored Nov 20, 2024
```
Signed-off-by: skylee-01 <497627264@qq.com>
```
  343041c4
- [ci/build] Combine nightly and optional (#10465) · ed701ca9
  Kevin H. Luu authored Nov 19, 2024
  
  ed701ca9
- [CI/Build] Support compilation with local cutlass path (#10423) (#10424) · 7629a9c6
  wchen61 authored Nov 20, 2024
  
  7629a9c6
- [CI/Build] Add sphinx/rst linter for docs (#10366) · 709c9f1f
  Rafael Vasquez authored Nov 20, 2024
  
  709c9f1f
- [Bugfix] Enforce no chunked prefill for embedding models (#10470) · b4be5a8a
  Cyrus Leung authored Nov 20, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b4be5a8a