Commits · 25e778aa169ec871ad65524c793bf9bd43f6885a · OpenDAS / vllm_cscc

21 Jul, 2024 1 commit
- [Model] Refactor and decouple phi3v image embedding (#6621) · 25e778aa
  Isotr0py authored Jul 22, 2024
  
  25e778aa
20 Jul, 2024 1 commit
- [Bugfix][CI/Build][Hardware][AMD] Fix AMD tests, add HF cache, update CK FA,... · 06d6c5fe
  Matt Wong authored Jul 20, 2024
```
[Bugfix][CI/Build][Hardware][AMD] Fix AMD tests, add HF cache, update CK FA, add partially supported model notes (#6543)
```
  06d6c5fe
19 Jul, 2024 2 commits
- [Misc] Small perf improvements (#6520) · 9ed82e70
  Antoni Baum authored Jul 19, 2024
  
  9ed82e70
- [ Misc ] non-uniform quantization via `compressed-tensors` for `Llama` (#6515) · dbe55885
  Robert Shaw authored Jul 18, 2024
  
  dbe55885
18 Jul, 2024 3 commits
- [Model] Support Mistral-Nemo (#6548) · 15c6a079
  Michael Goin authored Jul 18, 2024
  
  15c6a079
- [core][model] yet another cpu offload implementation (#6496) · 1c27d25f
  youkaichao authored Jul 17, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  1c27d25f
- [Model] Pipeline parallel support for Mixtral (#6516) · b5af8c22
  Cody Yu authored Jul 17, 2024
  
  b5af8c22
17 Jul, 2024 1 commit
- [Distributed][PP] only create embedding & lm head when necessary (#6455) · 1d094fd7
  Wushi Dong authored Jul 16, 2024
```
original title: [Distributed][Model] Rank-based Component Creation for Pipeline Parallelism Memory Optimization
```
  1d094fd7
16 Jul, 2024 2 commits
- [Kernel][Attention] Separate `Attention.kv_scale` into `k_scale` and `v_scale` (#6081) · 978aed53
  Michael Goin authored Jul 16, 2024
  
  978aed53
- [BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cleanup bug (#6425) · 9ad32dac
  Mor Zusman authored Jul 16, 2024
```
Co-authored-by: Mor Zusman <morz@ai21.com>
```
  9ad32dac
15 Jul, 2024 3 commits
- [misc][distributed] fix pp missing layer condition (#6446) · 4cf256ae
  youkaichao authored Jul 15, 2024
  
  4cf256ae
- [VLM] Minor space optimization for `ClipVisionModel` (#6436) · 6ae1597d
  Roger Wang authored Jul 15, 2024
  
  6ae1597d
- [core][distributed] simplify code to support pipeline parallel (#6406) · 69672f11
  youkaichao authored Jul 14, 2024
  
  69672f11
14 Jul, 2024 2 commits
- [Model] Initialize Fuyu-8B support (#3924) · 540c0368
  Isotr0py authored Jul 14, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  540c0368
- [ Misc ] Apply MoE Refactor to Deepseekv2 To Support Fp8 (#6417) · fb6af8bc
  Robert Shaw authored Jul 13, 2024
  
  fb6af8bc
12 Jul, 2024 1 commit
- [Bugfix] Fix dtype mismatch in PaliGemma (#6367) · 024ad87c
  Cyrus Leung authored Jul 12, 2024
  
  024ad87c
11 Jul, 2024 2 commits
- [bug fix] Fix llava next feature size calculation. (#6339) · 1df43de9
  xwjiang2010 authored Jul 11, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
```
  1df43de9
- [Bugfix] GPTBigCodeForCausalLM: Remove lm_head from supported_lora_modules. (#6326) · 8a1415cf
  Thomas Parnell authored Jul 11, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  8a1415cf
10 Jul, 2024 3 commits
- [Bugfix] MLPSpeculator: Use ParallelLMHead in tie_weights=False case. (#6303) · c38eba30
  Thomas Parnell authored Jul 10, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  c38eba30
- [Bugfix] Support 2D input shape in MoE layer (#6287) · e72ae80b
  Woosuk Kwon authored Jul 10, 2024
  
  e72ae80b
- [Speculative Decoding] Medusa Implementation with Top-1 proposer (#4978) · 2416b26e
  Abhinav Goyal authored Jul 10, 2024
  
  2416b26e
08 Jul, 2024 1 commit
- [Bugfix] Mamba cache Cuda Graph padding (#6214) · ddc369fb
  tomeras91 authored Jul 08, 2024
  
  ddc369fb
07 Jul, 2024 1 commit
- [Model] Add PaliGemma (#5189) · 6206dcb2
  Roger Wang authored Jul 06, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6206dcb2
05 Jul, 2024 2 commits
- [VLM] Cleanup validation and update docs (#6149) · ea4b5704
  Cyrus Leung authored Jul 05, 2024
  
  ea4b5704
- [VLM] Improve consistency between feature size calculation and dummy data for profiling (#6146) · a41357e9
  Roger Wang authored Jul 04, 2024
  
  a41357e9
04 Jul, 2024 2 commits
- [VLM] Calculate maximum number of multi-modal tokens by model (#6121) · ae96ef8f
  Cyrus Leung authored Jul 05, 2024
  
  ae96ef8f
- [Kernel][Model] logits_soft_cap for Gemma2 with flashinfer (#6051) · 69ec3ca1
  Lily Liu authored Jul 04, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  69ec3ca1
03 Jul, 2024 3 commits

[vlm] Remove vision language config. (#6089) · d9e98f42

xwjiang2010 authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

d9e98f42

[Bugfix] Fix `compute_logits` in Jamba (#6093) · 7cd2ebb0
Roger Wang authored Jul 03, 2024

7cd2ebb0

[Core] Dynamic image size support for VLMs (#5276) · 9831aec4

Cyrus Leung authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

9831aec4

02 Jul, 2024 5 commits

[Model] Jamba support (#4115) · 9d6a8daa

Mor Zusman authored Jul 03, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

9d6a8daa

[CORE] Quantized lm-head Framework (#4442) · ee93f4f9
Qubitium-ModelCloud authored Jul 03, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: ZX <zx@lbx.dev>
```
ee93f4f9

[ Misc ] Refactor MoE to isolate Fp8 From Mixtral (#5970) · 7c008c51

Robert Shaw authored Jul 02, 2024


Co-authored-by: Robert Shaw <rshaw@neuralmagic>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

7c008c51

[Core] Pipeline Parallel Support (#4412) · c5832d2a
Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
c5832d2a

[VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c

xwjiang2010 authored Jul 02, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

98d6682c

01 Jul, 2024 1 commit
- [Model] Changes to MLPSpeculator to support tie_weights and input_scale (#5965) · 54600709
  Thomas Parnell authored Jul 02, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Joshua Rosenkranz <jmrosenk@us.ibm.com>
```
  54600709
29 Jun, 2024 2 commits
- [CI/Build] Add TP test for vision models (#5892) · 99397da5
  Cyrus Leung authored Jun 29, 2024
  
  99397da5
- [Bugfix] Fix precisions in Gemma 1 (#5913) · 580353da
  Woosuk Kwon authored Jun 28, 2024
  
  580353da
28 Jun, 2024 2 commits
- Unmark more files as executable (#5962) · 5d2a1a9c
  Tyler Michael Smith authored Jun 28, 2024
  
  5d2a1a9c
- Support Deepseek-V2 (#4650) · be0b3af9
  wangding zeng authored Jun 29, 2024
```
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
```
  be0b3af9