Commits · 5923532e15b12832febe91ef5e9a4cef2786cefc · OpenDAS / vllm_cscc

08 Aug, 2024 2 commits
- [Bugfix] Fix new Llama3.1 GGUF model loading (#7269) · 8334c39f
  Isotr0py authored Aug 09, 2024
  
  8334c39f
- [Model] Rename MiniCPMVQwen2 to MiniCPMV2.6 (#7273) · 757ac70a
  Jee Jee Li authored Aug 08, 2024
  
  757ac70a
07 Aug, 2024 5 commits
- [Bugfix] Fix gptq failure on T4s (#7264) · 311f7438
  Lucas Wilkinson authored Aug 07, 2024
  
  311f7438
- [Bugfix][FP8] Fix dynamic FP8 Marlin quantization (#7219) · 5223199e
  Michael Goin authored Aug 07, 2024
  
  5223199e
- [Bugfix] Fix input processor for InternVL2 model (#7164) · b7645476
  Isotr0py authored Aug 08, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b7645476
- [Misc] Refactor linear layer weight loading; introduce `BasevLLMParameter` and... · 0f7052bc
  Dipika Sikka authored Aug 07, 2024
```
[Misc] Refactor linear layer weight loading; introduce `BasevLLMParameter` and `weight_loader_v2` (#5874)
```
  0f7052bc
- [Bugfix] Fix GPTQ and GPTQ Marlin CPU Offloading (#7225) · f9a56006
  Michael Goin authored Aug 06, 2024
  
  f9a56006
06 Aug, 2024 3 commits
- [Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026
  afeldman-nm authored Aug 06, 2024
```
[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  fd95e026
- [SpecDecode] [Minor] Fix spec decode sampler tests (#7183) · 5c60c8c4
  Lily Liu authored Aug 06, 2024
  
  5c60c8c4
- [Model] Support SigLIP encoder and alternative decoders for LLaVA models (#7153) · 1f26efbb
  Cyrus Leung authored Aug 06, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  1f26efbb
05 Aug, 2024 4 commits
- [Core] Support loading GGUF model (#5191) · 360bd67c
  Isotr0py authored Aug 06, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  360bd67c
- [Doc] [SpecDecode] Update MLPSpeculator documentation (#7100) · 789937af
  Thomas Parnell authored Aug 06, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  789937af
- [Model] SiglipVisionModel ported from transformers (#6942) · c0d8f163
  Jungho Christopher Cho authored Aug 05, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  c0d8f163
- [Model] Add multi-image support for minicpmv (#7122) · 7b86e7c9
  Alphi authored Aug 05, 2024
```
Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  7b86e7c9
04 Aug, 2024 2 commits
- [Model]Refactor MiniCPMV (#7020) · 179a6a36
  Jee Jee Li authored Aug 04, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  179a6a36
- Support for guided decoding for offline LLM (#6878) · 654bc5ca
  Yihuan Bu authored Aug 03, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  654bc5ca
03 Aug, 2024 2 commits

[Model] Refactor and decouple weight loading logic for InternVL2 model (#7067) · 0c25435d
Isotr0py authored Aug 03, 2024

0c25435d

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

02 Aug, 2024 2 commits
- [Misc] Disambiguate quantized types via a new ScalarType (#6396) · a8d604ca
  Lucas Wilkinson authored Aug 02, 2024
  
  a8d604ca
- [Core] Comment out unused code in sampler (#7023) · db351863
  Peng Guanwen authored Aug 02, 2024
  
  db351863
01 Aug, 2024 8 commits
- [Misc] Support attention logits soft-capping with flash-attn (#7022) · 805a8a75
  Woosuk Kwon authored Aug 01, 2024
  
  805a8a75
- [Models] Support Qwen model with PP (#6974) · fc912e08
  Murali Andoorveedu authored Aug 01, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  fc912e08
- [Bugfix] Lower gemma's unloaded_params exception to warning (#7002) · f4fd390f
  Michael Goin authored Aug 01, 2024
  
  f4fd390f
- [Bugfix] Fix RMSNorm forward in InternViT attention qk_layernorm (#6992) · 2dd34371
  Isotr0py authored Aug 02, 2024
  
  2dd34371
- [CI/Build] Update PyTorch to 2.4.0 (#6951) · 7e0861bd
  Sage Moore authored Aug 01, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7e0861bd
- [Bugfix][Model] Skip loading lm_head weights if using tie_word_embeddings (#6758) · 630dd9e0
  Travis Johnson authored Jul 31, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  630dd9e0
- [Bugfix][TPU] Do not use torch.Generator for TPUs (#6981) · 23993a79
  Woosuk Kwon authored Jul 31, 2024
  
  23993a79
- [Model] Pipeline parallel support for Qwen2 (#6924) · 1d2e7fb7
  xuyi authored Aug 01, 2024
  
  1d2e7fb7
31 Jul, 2024 5 commits
- [Bugfix] Support cpu offloading with fp8 quantization (#6960) · 460c1884
  Michael Goin authored Jul 31, 2024
  
  460c1884
- [Model] use FusedMoE layer in Jamba (#6935) · 2ee8d3ba
  Avshalom Manevich authored Jul 31, 2024
  
  2ee8d3ba
- [Bugfix] Fix feature size calculation for LLaVA-NeXT (#6982) · daed30c4
  Cyrus Leung authored Jul 31, 2024
  
  daed30c4
- [Bugfix] Clean up MiniCPM-V (#6939) · 2f4e108f
  Alphi authored Jul 31, 2024
```
Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  2f4e108f
- Support W4A8 quantization for vllm (#5218) · 6512937d
  HandH1998 authored Jul 31, 2024
  
  6512937d
30 Jul, 2024 3 commits
- [Kernel] Remove scaled_fp8_quant kernel padding footgun (#6842) · d7a299ed
  Tyler Michael Smith authored Jul 30, 2024
  
  d7a299ed
- [BugFix] Fix use of per-request seed with pipeline parallel (#6698) · 5cf9254a
  Nick Hill authored Jul 30, 2024
  
  5cf9254a
- [Bugfix] Fix PaliGemma MMP (#6930) · c66c7f86
  Roger Wang authored Jul 30, 2024
  
  c66c7f86
29 Jul, 2024 3 commits
- [Bugfix] Allow vllm to still work if triton is not installed. (#6786) · 9a7e2d05
  Thomas Parnell authored Jul 29, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  9a7e2d05
- [Core] Reduce unnecessary compute when logprobs=None (#6532) · db9e5708
  Peng Guanwen authored Jul 30, 2024
  
  db9e5708
- [Model] Initialize support for InternVL2 series models (#6514) · 7cbd9ec7
  Isotr0py authored Jul 29, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  7cbd9ec7
28 Jul, 2024 1 commit
- [Misc] Pass cutlass_fp8_supported correctly in fbgemm_fp8 (#6871) · 3eeb148f
  Elsa Granger authored Jul 28, 2024
  
  3eeb148f