Commits · 43735bf5e19eaf243b6edaa5af4c7561a14fc2f6 · OpenDAS / vllm_cscc · GitLab

19 Aug, 2024 10 commits
- [TPU] Remove redundant input tensor cloning (#7660) · 43735bf5
  Woosuk Kwon authored Aug 19, 2024
  
  43735bf5
- [Bugfix] Don't disable existing loggers (#7664) · da115230
  Andrew Song authored Aug 19, 2024
  
  da115230
- [Core] Support tensor parallelism for GGUF quantization (#7520) · 7601cb04
  Isotr0py authored Aug 20, 2024
  
  7601cb04
- [core] Multi Step Scheduling (#7000) · 47b65a55
  William Lin authored Aug 19, 2024
```
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
```
  47b65a55
- [Bugfix] fix lora_dtype value type in arg_utils.py - part 2 (#5428) · dad961ef
  Ali Panahi authored Aug 19, 2024
  
  dad961ef
- [MISC] Add prefix cache hit rate to metrics (#7606) · 3ac50b47
  Cody Yu authored Aug 19, 2024
  
  3ac50b47
- [Misc] Remove Gemma RoPE (#7638) · df845b2b
  Woosuk Kwon authored Aug 19, 2024
  
  df845b2b
- [Core] Use flashinfer sampling kernel when available (#7137) · f710fb52
  Peng Guanwen authored Aug 19, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  f710fb52
- [Core] Optimize SPMD architecture with delta + serialization optimization (#7109) · ff7ec82c
  SangBin Cho authored Aug 18, 2024
  
  ff7ec82c
- [Misc] Refactor Llama3 RoPE initialization (#7637) · 200a2ffa
  Woosuk Kwon authored Aug 18, 2024
  
  200a2ffa
18 Aug, 2024 3 commits
- [ Bugfix ] Fix Prometheus Metrics With `zeromq` Frontend (#7279) · e3b31821
  Robert Shaw authored Aug 18, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  e3b31821
- [TPU] Optimize RoPE forward_native2 (#7636) · ab7165f2
  Woosuk Kwon authored Aug 18, 2024
  
  ab7165f2
- [TPU] Use mark_dynamic only for dummy run (#7634) · 0c2fa50b
  Woosuk Kwon authored Aug 18, 2024
  
  0c2fa50b
17 Aug, 2024 8 commits
- [TPU] Skip creating empty tensor (#7630) · ce143353
  Woosuk Kwon authored Aug 17, 2024
  
  ce143353
- [VLM] Refactor `MultiModalConfig` initialization and profiling (#7530) · bbf55c48
  Roger Wang authored Aug 17, 2024
  
  bbf55c48
- [Misc]Fix BitAndBytes exception messages (#7626) · 1ef13cf9
  Jee Jee Li authored Aug 18, 2024
  
  1ef13cf9
- [Model] Pipeline parallel support for JAIS (#7603) · e73f76ee
  Besher Alkurdi authored Aug 17, 2024
  
  e73f76ee
- [core][misc] update libcudart finding (#7620) · d95cc0a5
  youkaichao authored Aug 16, 2024
```
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  d95cc0a5
- [misc] use nvml to get consistent device name (#7582) · eed020f6
  youkaichao authored Aug 16, 2024
  
  eed020f6
- [Bugfix] Clear engine reference in AsyncEngineRPCServer (#7618) · bae888cb
  Rui Qiao authored Aug 16, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  bae888cb
- [Bugfix] Fix custom_ar support check (#7617) · e6803499
  bnellnm authored Aug 16, 2024
  
  e6803499
16 Aug, 2024 17 commits
- [Model] Align nemotron config with final HF state and fix lm-eval-small (#7611) · 44f26a94
  Michael Goin authored Aug 16, 2024
  
  44f26a94
- [Kernel] fix types used in aqlm and ggml kernels to support dynamo (#7596) · 37fd47e7
  bnellnm authored Aug 16, 2024
  
  37fd47e7
- [Kernel][Misc] dynamo support for ScalarType (#7594) · 7759ae95
  bnellnm authored Aug 16, 2024
  
  7759ae95
- [Kernel] register punica functions as torch ops (#7591) · 9f698563
  bnellnm authored Aug 16, 2024
  
  9f698563
- [Core] Fix tracking of model forward time in case of PP>1 (#7440) · 93478b63
  Mahesh Keralapura authored Aug 16, 2024
```
[Core] Fix tracking of model forward time to the span traces in case of PP>1 (#7440)
```
  93478b63
- [spec decode] [4/N] Move update_flash_attn_metadata to attn backend (#7571) · f366f633
  William Lin authored Aug 16, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  f366f633
- [Kernel] Add tuned triton configs for ExpertsInt8 (#7601) · 855866ca
  Michael Goin authored Aug 16, 2024
  
  855866ca
- [Kernel] W8A16 Int8 inside FusedMoE (#7415) · 7fc23be8
  Mor Zusman authored Aug 16, 2024
  
  7fc23be8
- [Feature][Hardware][Amd] Add fp8 Linear Layer for Rocm (#7210) · e837b624
  Charlie Fu authored Aug 16, 2024
  
  e837b624
- support tqdm in notebooks (#7510) · ec724a72
  fzyzcjy authored Aug 17, 2024
  
  ec724a72
- [Bugfix][Hardware][AMD][Frontend] add quantization param to embedding checking method (#7513) · 0e39a33c
  Gordon Wong authored Aug 17, 2024
  
  0e39a33c
- [Core] Use uvloop with zmq-decoupled front-end (#7570) · 9587b050
  Nick Hill authored Aug 15, 2024
  
  9587b050
- register custom op for flash attn and use from torch.ops (#7536) · 54bd9a03
  youkaichao authored Aug 15, 2024
  
  54bd9a03
- Chat method for offline llm (#5049) · 3b19e39d
  nunjunj authored Aug 16, 2024
```
Co-authored-by: nunjunj <ray@g-3ff9f30f2ed650001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-1df6075697c3f0001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-c5a2c23abc49e0001.c.vllm-405802.internal>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b19e39d
- [Feature]: Add OpenAI server prompt_logprobs support #6508 (#7453) · f878c8fe
  Grant Pinkert authored Aug 16, 2024
  
  f878c8fe
- [Misc] Add quantization config support for speculative model. (#7343) · b67ae00c
  shangmingc authored Aug 16, 2024
  
  b67ae00c
- [Bugfix][Harmless] Fix float16 dtype for model_is_embedding (#7566) · 9c8e2d11
  Michael Goin authored Aug 15, 2024
  
  9c8e2d11
15 Aug, 2024 2 commits
- [Bugfix] Fix default weight loading for scalars (#7534) · 21313e09
  Michael Goin authored Aug 15, 2024
  
  21313e09
- [Bugfix] update neuron for version > 0.5.0 (#7175) · 9c1f78d5
  omrishiv authored Aug 15, 2024
```
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  9c1f78d5