Commits · ac49b59d8b01ffb9979e18e67b252d45410bc1e6 · OpenDAS / vllm_cscc · GitLab

13 Nov, 2024 1 commit
- [Bugfix] bitsandbytes models fail to run pipeline parallel (#10200) · ac49b59d
  HoangCongDuc authored Nov 14, 2024
```
Signed-off-by: Hoang Cong Duc <hoangcongducltt@gmail.com>
```
  ac49b59d
09 Nov, 2024 1 commit
- [5/N] pass the whole config to model (#9983) · 1a95f10e
  youkaichao authored Nov 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1a95f10e
06 Nov, 2024 1 commit
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
05 Nov, 2024 1 commit
- [Misc] Modify BNB parameter name (#9997) · b9c64c0c
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b9c64c0c
04 Nov, 2024 3 commits
- [Misc]Reduce BNB static variable (#9987) · fb2716d6
  Jee Jee Li authored Nov 05, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  fb2716d6
- [4/N] make quant config first-class citizen (#9978) · 8d72bb20
  youkaichao authored Nov 04, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8d72bb20
- [Bugfix] Fix MiniCPMV and Mllama BNB bug (#9917) · c49f0407
  Jee Jee Li authored Nov 04, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c49f0407
02 Nov, 2024 2 commits
- [bugfix] fix tsts (#9959) · 3bb4befe
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3bb4befe
- [3/N] model runner pass the whole config to model (#9958) · cea808f3
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cea808f3
30 Oct, 2024 1 commit
- [Model] Support math-shepherd-mistral-7b-prm model (#9697) · 81f09cfd
  Went-Liang authored Oct 31, 2024
```
Signed-off-by: Went-Liang <wenteng_liang@163.com>
```
  81f09cfd
29 Oct, 2024 2 commits
- [Bugfix] Fix prefix strings for quantized VLMs (#9772) · bc73e982
  Michael Goin authored Oct 29, 2024
  
  bc73e982
- [Model] Add BNB quantization support for Mllama (#9720) · 09500f7d
  Isotr0py authored Oct 29, 2024
  
  09500f7d
09 Oct, 2024 1 commit
- support bitsandbytes quantization with more models (#9148) · 2f4117c3
  chenqianfzh authored Oct 08, 2024
  
  2f4117c3
04 Oct, 2024 1 commit
- [Misc] Move registry to its own file (#9064) · 0e36fd49
  Cyrus Leung authored Oct 04, 2024
  
  0e36fd49
01 Oct, 2024 1 commit
- [Misc] add process_weights_after_loading for DummyLoader (#8969) · 82f3937e
  Divakar Verma authored Sep 30, 2024
  
  82f3937e
25 Sep, 2024 1 commit
- [[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (#8768) · 13f9f7a3
  Jee Jee Li authored Sep 25, 2024
  
  13f9f7a3
24 Sep, 2024 1 commit
- [Core][Model] Support loading weights by ID within models (#7931) · 3f06bae9
  Peter Salas authored Sep 24, 2024
  
  3f06bae9
18 Sep, 2024 1 commit
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
17 Sep, 2024 1 commit
- [Feature][kernel] tensor parallelism with bitsandbytes quantization (#8434) · 9855b995
  chenqianfzh authored Sep 17, 2024
  
  9855b995
11 Sep, 2024 1 commit
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4
07 Sep, 2024 1 commit
- [Bugfix] Fix broken OpenAI tensorizer test (#8258) · 9f68e00d
  Cyrus Leung authored Sep 07, 2024
  
  9f68e00d
06 Sep, 2024 1 commit
- [Model] Allow loading from original Mistral format (#8168) · 29f49cd6
  Patrick von Platen authored Sep 07, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  29f49cd6
29 Aug, 2024 1 commit
- support bitsandbytes 8-bit and FP4 quantized models (#7445) · 4664ceaa
  chenqianfzh authored Aug 29, 2024
  
  4664ceaa
22 Aug, 2024 1 commit
- Fix ShardedStateLoader for vllm fp8 quantization (#7708) · 4f419c00
  Flex Wang authored Aug 22, 2024
  
  4f419c00
17 Aug, 2024 2 commits
- [VLM] Refactor `MultiModalConfig` initialization and profiling (#7530) · bbf55c48
  Roger Wang authored Aug 17, 2024
  
  bbf55c48
- [Misc]Fix BitAndBytes exception messages (#7626) · 1ef13cf9
  Jee Jee Li authored Aug 18, 2024
  
  1ef13cf9
14 Aug, 2024 1 commit
- [VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126) · 3f674a49
  Cyrus Leung authored Aug 15, 2024
  
  3f674a49
13 Aug, 2024 2 commits
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
- [hardware] unify usage of is_tpu to current_platform.is_tpu() (#7102) · 4d2dc507
  youkaichao authored Aug 13, 2024
  
  4d2dc507
09 Aug, 2024 1 commit
- [TPU] Add Load-time W8A16 quantization for TPU Backend (#7005) · 0fa14907
  Siyuan Liu authored Aug 08, 2024
  
  0fa14907
06 Aug, 2024 1 commit
- [Model] Support SigLIP encoder and alternative decoders for LLaVA models (#7153) · 1f26efbb
  Cyrus Leung authored Aug 06, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  1f26efbb
05 Aug, 2024 1 commit
- [Core] Support loading GGUF model (#5191) · 360bd67c
  Isotr0py authored Aug 06, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  360bd67c
31 Jul, 2024 1 commit
- [Bugfix] Support cpu offloading with fp8 quantization (#6960) · 460c1884
  Michael Goin authored Jul 31, 2024
  
  460c1884
24 Jul, 2024 1 commit
- [Bugfix]fix modelscope compatible issue (#6730) · f4f8a9d8
  liuyhwangyh authored Jul 24, 2024
  
  f4f8a9d8
23 Jul, 2024 2 commits
- [bitsandbytes]: support read bnb pre-quantized model (#5753) · 87525fab
  dongmao zhang authored Jul 23, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  87525fab
- support ignore patterns in model loader (#6673) · 3eda4ec7
  Simon Mo authored Jul 22, 2024
  
  3eda4ec7
16 Jul, 2024 1 commit
- [BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cleanup bug (#6425) · 9ad32dac
  Mor Zusman authored Jul 16, 2024
```
Co-authored-by: Mor Zusman <morz@ai21.com>
```
  9ad32dac
15 Jul, 2024 1 commit
- [Misc] Add CustomOp Interface to UnquantizedFusedMoEMethod (#6289) · ec9933f4
  Woosuk Kwon authored Jul 15, 2024
  
  ec9933f4
03 Jul, 2024 2 commits
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [hardware][misc] introduce platform abstraction (#6080) · 482045ee
  youkaichao authored Jul 02, 2024
  
  482045ee