Commits · 539aa9926007d4896397ca3fa83d48da492b4831 · OpenDAS / vllm_cscc · GitLab

27 Sep, 2024 1 commit
- Merge tag 'v0.6.2' into v0.6.2-dev · 539aa992
  zhuwenwen authored Sep 27, 2024
  
  539aa992
26 Sep, 2024 1 commit
- support head_dim 160 and update benchmark_throughput.py · 93872128
  zhuwenwen authored Sep 26, 2024
  
  93872128
25 Sep, 2024 25 commits
- [Misc] Support quantization of MllamaForCausalLM (#8822) · 7193774b
  Michael Goin authored Sep 25, 2024
  
  7193774b
- [Doc] Update doc for Transformers 4.45 (#8817) · e2c6e0a8
  Roger Wang authored Sep 25, 2024
  
  e2c6e0a8
- [Model] Add support for the multi-modal Llama 3.2 model (#8811) · 770ec602
  Chen Zhang authored Sep 25, 2024
```
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  770ec602
- Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) · 4f1ba084
  Simon Mo authored Sep 25, 2024
  
  4f1ba084
- [Misc] Support FP8 MoE for compressed-tensors (#8588) · 873edda6
  Michael Goin authored Sep 25, 2024
  
  873edda6
- [Frontend] MQLLMEngine supports profiling. (#8761) · 64840dfa
  科英 authored Sep 26, 2024
  
  64840dfa
- rename PromptInputs and inputs with backward compatibility (#8760) · 28e1299e
  Cyrus Leung authored Sep 26, 2024
  
  28e1299e
- [VLM][Bugfix] internvl with num_scheduler_steps > 1 (#8614) · 0c4d2ad5
  DefTruth authored Sep 26, 2024
  
  0c4d2ad5
- [[Misc]] Add extra deps for openai server image (#8792) · c6f2485c
  Jee Jee Li authored Sep 26, 2024
  
  c6f2485c
- [Kernel] Fullgraph and opcheck tests (#8479) · 300da091
  bnellnm authored Sep 25, 2024
  
  300da091
- [CI/Build][Bugfix][Doc][ROCm] CI fix and doc update after ROCm 6.2 upgrade (#8777) · 1c046447
  Hongxia Yang authored Sep 25, 2024
  
  1c046447
- [Misc] Fix minor typo in scheduler (#8765) · 8fae5ed7
  Woo-Yeon Lee authored Sep 25, 2024
  
  8fae5ed7
- [Bugfix] Ray 2.9.x doesn't expose available_resources_per_node (#8767) · 3368c3ab
  David Newman authored Sep 25, 2024
```
Signed-off-by: darthhexx <darthhexx@gmail.com>
```
  3368c3ab
- [Frontend] OpenAI server: propagate usage accounting to FastAPI middleware layer (#8672) · 1ac3de09
  Adam Tilghman authored Sep 25, 2024
  
  1ac3de09
- [Bugfix] load fc bias from config for eagle (#8790) · 3e073e66
  sohamparikh authored Sep 25, 2024
  
  3e073e66
- [Hardware][CPU] Enable mrope and support Qwen2-VL on CPU backend (#8770) · c2395367
  Isotr0py authored Sep 25, 2024
  
  c2395367
- [BugFix] Propagate 'trust_remote_code' setting in internvl and minicpmv (#8250) · e3dd0692
  zifeitong authored Sep 24, 2024
  
  e3dd0692
- Fix tests in test_chunked_prefill_scheduler which fail with BlockManager V2 (#8752) · fc3afc20
  sroy745 authored Sep 24, 2024
  
  fc3afc20
- [Bugfix][Kernel] Implement acquire/release polyfill for Pascal (#8776) · b4522474
  sasha0552 authored Sep 25, 2024
  
  b4522474
- Fix test_schedule_swapped_simple in test_scheduler.py (#8780) · ee777d9c
  sroy745 authored Sep 24, 2024
  
  ee777d9c
- [Bugfix] Use heartbeats instead of health checks (#8583) · 6e0c9d6b
  Joe Runde authored Sep 24, 2024
  
  6e0c9d6b
- [Core] Adding Priority Scheduling (#5958) · 6da1ab6b
  Archit Patke authored Sep 24, 2024
  
  6da1ab6b
- remove reason of unsupported · a087cda8
  zhuwenwen authored Sep 25, 2024
  
  a087cda8
- [Core][Bugfix] Support prompt_logprobs returned with speculative decoding (#8047) · 01b6f9e1
  Travis Johnson authored Sep 24, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  01b6f9e1
- [[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (#8768) · 13f9f7a3
  Jee Jee Li authored Sep 25, 2024
  
  13f9f7a3
24 Sep, 2024 13 commits
- [misc] soft drop beam search (#8763) · 1e7d5c01
  youkaichao authored Sep 24, 2024
  
  1e7d5c01
- [CI/Build] fix setuptools-scm usage (#8771) · 2467b642
  Daniele authored Sep 24, 2024
  
  2467b642
- [Bugfix] Fix torch dynamo fixes caused by `replace_parameters` (#8748) · 72fc97a0
  Lucas Wilkinson authored Sep 24, 2024
  
  72fc97a0
- [Frontend] Batch inference for llm.chat() API (#8648) · 2529d09b
  Andy authored Sep 24, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  2529d09b
- [Kernel] Split Marlin MoE kernels into multiple files (#8661) · a928ded9
  ElizaWszola authored Sep 24, 2024
```
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  a928ded9
- [Bugfix] Fix potentially unsafe custom allreduce synchronization (#8558) · cc4325b6
  Hanzhi Zhou authored Sep 24, 2024
  
  cc4325b6
- [Model] Expose Phi3v num_crops as a mm_processor_kwarg (#8658) · 8ff7ced9
  Alex Brooks authored Sep 24, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8ff7ced9
- [Core][Model] Support loading weights by ID within models (#7931) · 3f06bae9
  Peter Salas authored Sep 24, 2024
  
  3f06bae9
- [MISC] Skip dumping inputs when unpicklable (#8744) · b8747e8a
  Cody Yu authored Sep 23, 2024
  
  b8747e8a
- Revert "[Core] Rename `PromptInputs` to `PromptType`, and `inputs` to `prompt`" (#8750) · 3185fb0c
  Simon Mo authored Sep 23, 2024
  
  3185fb0c
- re-implement beam search on top of vllm core (#8726) · 0250dd68
  youkaichao authored Sep 23, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  0250dd68
- Fix tests in test_scheduler.py that fail with BlockManager V2 (#8728) · 88577ac9
  sroy745 authored Sep 23, 2024
  
  88577ac9
- 优化pa小batch性能（pa_v2），优化pa小seq性能(pa_v1)，reusekv=16优化 · 2b91ac93
  zhuwenwen authored Sep 24, 2024
  
  2b91ac93