Commits · bbf55c4805efba5f1d7094f5e2888b3ef26c0fd7 · OpenDAS / vllm_cscc

17 Aug, 2024 4 commits
- [VLM] Refactor `MultiModalConfig` initialization and profiling (#7530) · bbf55c48
  Roger Wang authored Aug 17, 2024
  
  bbf55c48
- [Misc]Fix BitAndBytes exception messages (#7626) · 1ef13cf9
  Jee Jee Li authored Aug 18, 2024
  
  1ef13cf9
- [Model] Pipeline parallel support for JAIS (#7603) · e73f76ee
  Besher Alkurdi authored Aug 17, 2024
  
  e73f76ee
- [misc] use nvml to get consistent device name (#7582) · eed020f6
  youkaichao authored Aug 16, 2024
  
  eed020f6
16 Aug, 2024 5 commits
- [Model] Align nemotron config with final HF state and fix lm-eval-small (#7611) · 44f26a94
  Michael Goin authored Aug 16, 2024
  
  44f26a94
- [Kernel] fix types used in aqlm and ggml kernels to support dynamo (#7596) · 37fd47e7
  bnellnm authored Aug 16, 2024
  
  37fd47e7
- [Kernel] Add tuned triton configs for ExpertsInt8 (#7601) · 855866ca
  Michael Goin authored Aug 16, 2024
  
  855866ca
- [Kernel] W8A16 Int8 inside FusedMoE (#7415) · 7fc23be8
  Mor Zusman authored Aug 16, 2024
  
  7fc23be8
- [Feature][Hardware][Amd] Add fp8 Linear Layer for Rocm (#7210) · e837b624
  Charlie Fu authored Aug 16, 2024
  
  e837b624
15 Aug, 2024 1 commit
- [Bugfix] Fix default weight loading for scalars (#7534) · 21313e09
  Michael Goin authored Aug 15, 2024
  
  21313e09
14 Aug, 2024 3 commits
- [Misc] Revert `compressed-tensors` code reuse (#7521) · f55a9aea
  Kyle Sayers authored Aug 14, 2024
  
  f55a9aea
- [VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126) · 3f674a49
  Cyrus Leung authored Aug 15, 2024
  
  3f674a49
- Fix empty output when temp is too low (#2937) · c134a464
  Chang Su authored Aug 13, 2024
```
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  c134a464
13 Aug, 2024 9 commits
- [misc][plugin] add plugin system implementation (#7426) · 16422ea7
  youkaichao authored Aug 13, 2024
  
  16422ea7
- [Misc] `compressed-tensors` code reuse (#7277) · 373538f9
  Kyle Sayers authored Aug 13, 2024
  
  373538f9
- [Misc] Update `awq` and `awq_marlin` to use `vLLMParameters` (#7422) · b1e5afc3
  Dipika Sikka authored Aug 13, 2024
  
  b1e5afc3
- [Misc] Update Fused MoE weight loading (#7334) · d3bdfd3a
  Dipika Sikka authored Aug 13, 2024
  
  d3bdfd3a
- [Misc] Update `gptq_marlin` to use new vLLMParameters (#7281) · fb377d7e
  Dipika Sikka authored Aug 13, 2024
  
  fb377d7e
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
- [hardware] unify usage of is_tpu to current_platform.is_tpu() (#7102) · 4d2dc507
  youkaichao authored Aug 13, 2024
  
  4d2dc507
- [Bugfix] Fix weight loading for Chameleon when TP>1 (#7410) · 7025b11d
  Cyrus Leung authored Aug 13, 2024
  
  7025b11d
- [mypy] Misc. typing improvements (#7417) · 9ba85bc1
  Cyrus Leung authored Aug 13, 2024
  
  9ba85bc1
12 Aug, 2024 1 commit
- [Core][VLM] Support image embeddings as input (#6613) · e6e42e4b
  Roger Wang authored Aug 12, 2024
  
  e6e42e4b
10 Aug, 2024 1 commit
- [Bugfix] Fix phi3v batch inference when images have different aspect ratio (#7392) · 4c5d8e8e
  Isotr0py authored Aug 11, 2024
  
  4c5d8e8e
09 Aug, 2024 7 commits
- [Bugfix] Fix `PerTensorScaleParameter` weight loading for fused models (#7376) · 5c6c54d6
  Dipika Sikka authored Aug 09, 2024
  
  5c6c54d6
- [Model][Jamba] Mamba cache single buffer (#6739) · 07ab1607
  Mor Zusman authored Aug 09, 2024
```
Co-authored-by: Mor Zusman <morz@ai21.com>
```
  07ab1607
- [Speculative decoding] [Multi-Step] decouple should_modify_greedy_probs_inplace (#6971) · 57b7be0e
  William Lin authored Aug 08, 2024
  
  57b7be0e
- [Bugfix] Fix speculative decoding with MLPSpeculator with padded vocabulary (#7218) · 99b4cf5f
  Travis Johnson authored Aug 08, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  99b4cf5f
- [Performance] Optimize e2e overheads: Reduce python allocations (#7162) · e02ac556
  Alexander Matveev authored Aug 09, 2024
  
  e02ac556
- [Core] Support serving encoder/decoder models (#7258) · 7eb4a51c
  Cyrus Leung authored Aug 09, 2024
  
  7eb4a51c
- [TPU] Add Load-time W8A16 quantization for TPU Backend (#7005) · 0fa14907
  Siyuan Liu authored Aug 08, 2024
  
  0fa14907
08 Aug, 2024 2 commits
- [Bugfix] Fix new Llama3.1 GGUF model loading (#7269) · 8334c39f
  Isotr0py authored Aug 09, 2024
  
  8334c39f
- [Model] Rename MiniCPMVQwen2 to MiniCPMV2.6 (#7273) · 757ac70a
  Jee Jee Li authored Aug 08, 2024
  
  757ac70a
07 Aug, 2024 5 commits
- [Bugfix] Fix gptq failure on T4s (#7264) · 311f7438
  Lucas Wilkinson authored Aug 07, 2024
  
  311f7438
- [Bugfix][FP8] Fix dynamic FP8 Marlin quantization (#7219) · 5223199e
  Michael Goin authored Aug 07, 2024
  
  5223199e
- [Bugfix] Fix input processor for InternVL2 model (#7164) · b7645476
  Isotr0py authored Aug 08, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b7645476
- [Misc] Refactor linear layer weight loading; introduce `BasevLLMParameter` and... · 0f7052bc
  Dipika Sikka authored Aug 07, 2024
```
[Misc] Refactor linear layer weight loading; introduce `BasevLLMParameter` and `weight_loader_v2` (#5874)
```
  0f7052bc
- [Bugfix] Fix GPTQ and GPTQ Marlin CPU Offloading (#7225) · f9a56006
  Michael Goin authored Aug 06, 2024
  
  f9a56006
06 Aug, 2024 2 commits

[Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026

afeldman-nm authored Aug 06, 2024


[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

fd95e026

[SpecDecode] [Minor] Fix spec decode sampler tests (#7183) · 5c60c8c4
Lily Liu authored Aug 06, 2024

5c60c8c4