Commits · 42c7f66a386b2243dcd313feed7dec4e1d508167 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "7940d8a6a7841113c89b168080b51785946f0cdb"

22 Jul, 2024 3 commits
- [Core] Support dynamically loading Lora adapter from HuggingFace (#6234) · 42c7f66a
  Jiaxin Shan authored Jul 22, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  42c7f66a
- [Bugfix][Kernel] Use int64_t for indices in fp8 quant kernels (#6649) · fea59c77
  Tyler Michael Smith authored Jul 22, 2024
  
  fea59c77
- [Frontend] Refactor prompt processing (#4028) · 739b61a3
  Cyrus Leung authored Jul 23, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  739b61a3
21 Jul, 2024 3 commits
- [Kernel][Core] Add AWQ support to the Marlin kernel (#6612) · 396d92d5
  Alexander Matveev authored Jul 21, 2024
  
  396d92d5
- [Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both... · 14f91fe6
  sroy745 authored Jul 20, 2024
```
[Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both draft and target models. (#6485)
```
  14f91fe6
- [Frontend] Move chat utils (#6602) · d7f4178d
  Cyrus Leung authored Jul 21, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d7f4178d
20 Jul, 2024 4 commits
- [Bugfix][CI/Build][Hardware][AMD] Fix AMD tests, add HF cache, update CK FA,... · 06d6c5fe
  Matt Wong authored Jul 20, 2024
```
[Bugfix][CI/Build][Hardware][AMD] Fix AMD tests, add HF cache, update CK FA, add partially supported model notes (#6543)
```
  06d6c5fe
- [Misc] Consolidate and optimize logic for building padded tensors (#6541) · 9042d683
  Cyrus Leung authored Jul 20, 2024
  
  9042d683
- [Core] Allow specifying custom Executor (#6557) · 7bd82002
  Antoni Baum authored Jul 19, 2024
  
  7bd82002
- [ Kernel ] FP8 Dynamic Per Token Quant - Add scale_ub (#6593) · 2e265642
  Varun Sundar Rabindranath authored Jul 19, 2024
```
Co-authored-by: Varun Sundar Rabindranth <varun@neuralmagic.com>
```
  2e265642
19 Jul, 2024 8 commits
- [ Kernel ] Enable Dynamic Per Token `fp8` (#6547) · 4cc24f01
  Robert Shaw authored Jul 19, 2024
  
  4cc24f01
- [Bugfix] [SpecDecode] AsyncMetricsCollector: update time since last collection (#6578) · f0bbfaf9
  Thomas Parnell authored Jul 19, 2024
  
  f0bbfaf9
- [Misc] Small perf improvements (#6520) · 9ed82e70
  Antoni Baum authored Jul 19, 2024
  
  9ed82e70
- [Model] RowParallelLinear: pass bias to quant_method.apply (#6327) · a5314e86
  Thomas Parnell authored Jul 19, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  a5314e86
- [BUGFIX] Raise an error for no draft token case when draft_tp>1 (#6369) · a921e863
  Woo-Yeon Lee authored Jul 19, 2024
  
  a921e863
- [Bugfix][Frontend] Fix missing `/metrics` endpoint (#6463) · 6366efc6
  Cyrus Leung authored Jul 19, 2024
  
  6366efc6
- [Bugfix] Make spec. decode respect per-request seed. (#6034) · d4201e06
  Thomas Parnell authored Jul 19, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  d4201e06
- [Core] Multiprocessing Pipeline Parallel support (#6130) · b5672a11
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  b5672a11
18 Jul, 2024 4 commits
- [ci][test] add correctness test for cpu offloading (#6549) · f53b8f0d
  youkaichao authored Jul 18, 2024
  
  f53b8f0d
- [BugFix][Frontend] Use LoRA tokenizer in OpenAI APIs (#6227) · e2fbaee7
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  e2fbaee7
- [Model] Pipeline parallel support for Mixtral (#6516) · b5af8c22
  Cody Yu authored Jul 17, 2024
  
  b5af8c22
- [ Kernel ] FP8 Dynamic-Per-Token Quant Kernel (#6511) · b5241e41
  Varun Sundar Rabindranath authored Jul 17, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  b5241e41
17 Jul, 2024 6 commits
- [Core] draft_model_runner: Implement prepare_inputs on GPU for advance_step (#6338) · e76466dd
  Alexander Matveev authored Jul 17, 2024
  
  e76466dd
- [Bugfix] Fix Ray Metrics API usage (#6354) · 5f0b9933
  Antoni Baum authored Jul 17, 2024
  
  5f0b9933
- [Core] Refactor _prepare_model_input_tensors - take 2 (#6164) · 2fa4623d
  Cody Yu authored Jul 17, 2024
  
  2fa4623d
- [Bugfix] Fix for multinode crash on 4 PP (#6495) · 5fa6e987
  Murali Andoorveedu authored Jul 17, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  5fa6e987
- [Doc][CI/Build] Update docs and tests to use `vllm serve` (#6431) · 5bf35a91
  Cyrus Leung authored Jul 17, 2024
  
  5bf35a91
- [misc][distributed] improve tests (#6488) · 7f62077a
  youkaichao authored Jul 16, 2024
  
  7f62077a
16 Jul, 2024 8 commits
- [ci][distributed] add pipeline parallel correctness test (#6410) · 09c2eb85
  youkaichao authored Jul 16, 2024
  
  09c2eb85
- [Kernel][Attention] Separate `Attention.kv_scale` into `k_scale` and `v_scale` (#6081) · 978aed53
  Michael Goin authored Jul 16, 2024
  
  978aed53
- [Misc] Log spec decode metrics (#6454) · 160e1d8c
  Cody Yu authored Jul 16, 2024
  
  160e1d8c
- [CI/Build] Remove "boardwalk" image asset (#6460) · 38ef9488
  Cyrus Leung authored Jul 16, 2024
  
  38ef9488
- [Frontend] Support for chat completions input in the tokenize endpoint (#5923) · 7a3d2a5b
  sasha0552 authored Jul 16, 2024
  
  7a3d2a5b
- [CI/Build] vLLM cache directory for images (#6444) · d9701151
  Cyrus Leung authored Jul 16, 2024
  
  d9701151
- [Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (#6419) · d92b3c5c
  Joe authored Jul 15, 2024
  
  d92b3c5c
- [BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cleanup bug (#6425) · 9ad32dac
  Mor Zusman authored Jul 16, 2024
```
Co-authored-by: Mor Zusman <morz@ai21.com>
```
  9ad32dac
15 Jul, 2024 3 commits
- [Bugfix] use float32 precision in samplers/test_logprobs.py for comparing with HF (#6409) · 4ef95b0f
  Thomas Parnell authored Jul 15, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  4ef95b0f
- [core][distributed] simplify code to support pipeline parallel (#6406) · 69672f11
  youkaichao authored Jul 14, 2024
  
  69672f11
- [BugFix] BatchResponseData body should be optional (#6345) · b47008b4
  zifeitong authored Jul 14, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b47008b4
14 Jul, 2024 1 commit
- [Feature] vLLM CLI (#5090) · dbfe254e
  Ethan Xu authored Jul 14, 2024
```
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  dbfe254e