Commits · 3f674a49b5033a6ed778ab960e86e03cfa64aa1f · OpenDAS / vllm_cscc · GitLab

14 Aug, 2024 1 commit
- [VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126) · 3f674a49
  Cyrus Leung authored Aug 15, 2024
  
  3f674a49
13 Aug, 2024 3 commits
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
- [hardware] unify usage of is_tpu to current_platform.is_tpu() (#7102) · 4d2dc507
  youkaichao authored Aug 13, 2024
  
  4d2dc507
- [Bugfix] Fix weight loading for Chameleon when TP>1 (#7410) · 7025b11d
  Cyrus Leung authored Aug 13, 2024
  
  7025b11d
09 Aug, 2024 1 commit
- [TPU] Add Load-time W8A16 quantization for TPU Backend (#7005) · 0fa14907
  Siyuan Liu authored Aug 08, 2024
  
  0fa14907
08 Aug, 2024 1 commit
- [Bugfix] Fix new Llama3.1 GGUF model loading (#7269) · 8334c39f
  Isotr0py authored Aug 09, 2024
  
  8334c39f
06 Aug, 2024 1 commit
- [Model] Support SigLIP encoder and alternative decoders for LLaVA models (#7153) · 1f26efbb
  Cyrus Leung authored Aug 06, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  1f26efbb
05 Aug, 2024 1 commit
- [Core] Support loading GGUF model (#5191) · 360bd67c
  Isotr0py authored Aug 06, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  360bd67c
01 Aug, 2024 1 commit
- [Bugfix][TPU] Do not use torch.Generator for TPUs (#6981) · 23993a79
  Woosuk Kwon authored Jul 31, 2024
  
  23993a79
31 Jul, 2024 1 commit
- [Bugfix] Support cpu offloading with fp8 quantization (#6960) · 460c1884
  Michael Goin authored Jul 31, 2024
  
  460c1884
24 Jul, 2024 1 commit
- [Bugfix]fix modelscope compatible issue (#6730) · f4f8a9d8
  liuyhwangyh authored Jul 24, 2024
  
  f4f8a9d8
23 Jul, 2024 4 commits
- [bitsandbytes]: support read bnb pre-quantized model (#5753) · 87525fab
  dongmao zhang authored Jul 23, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  87525fab
- support ignore patterns in model loader (#6673) · 3eda4ec7
  Simon Mo authored Jul 22, 2024
  
  3eda4ec7
- [misc] only tqdm for first rank (#6672) · c5201240
  youkaichao authored Jul 22, 2024
  
  c5201240
- add tqdm when loading checkpoint shards (#6569) · e519ae09
  zhaotyer authored Jul 23, 2024
```
Co-authored-by: tianyi.zhao <tianyi.zhao@transwarp.io>
Co-authored-by: youkaichao <youkaichao@126.com>
```
  e519ae09
17 Jul, 2024 1 commit
- [misc][distributed] add seed to dummy weights (#6491) · ce37be7b
  youkaichao authored Jul 16, 2024
  
  ce37be7b
16 Jul, 2024 2 commits
- [Kernel][Attention] Separate `Attention.kv_scale` into `k_scale` and `v_scale` (#6081) · 978aed53
  Michael Goin authored Jul 16, 2024
  
  978aed53
- [BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cleanup bug (#6425) · 9ad32dac
  Mor Zusman authored Jul 16, 2024
```
Co-authored-by: Mor Zusman <morz@ai21.com>
```
  9ad32dac
15 Jul, 2024 1 commit
- [Misc] Add CustomOp Interface to UnquantizedFusedMoEMethod (#6289) · ec9933f4
  Woosuk Kwon authored Jul 15, 2024
  
  ec9933f4
14 Jul, 2024 1 commit
- [ Misc ] Enable Quantizing All Layers of DeekSeekv2 (#6423) · 73030b7d
  Robert Shaw authored Jul 14, 2024
  
  73030b7d
03 Jul, 2024 2 commits
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [hardware][misc] introduce platform abstraction (#6080) · 482045ee
  youkaichao authored Jul 02, 2024
  
  482045ee
02 Jul, 2024 1 commit

[VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c

xwjiang2010 authored Jul 02, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

98d6682c

01 Jul, 2024 1 commit
- [misc][cuda] use nvml to avoid accidentally cuda initialization (#6007) · 614aa512
  youkaichao authored Jun 30, 2024
  
  614aa512
28 Jun, 2024 1 commit
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
27 Jun, 2024 1 commit
- [Model] Add base class for LoRA-supported models (#5018) · 96354d6a
  Cyrus Leung authored Jun 27, 2024
  
  96354d6a
20 Jun, 2024 1 commit
- [Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718) · 8065a7e2
  Michael Goin authored Jun 20, 2024
  
  8065a7e2
15 Jun, 2024 1 commit
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
12 Jun, 2024 2 commits
- [Frontend] [Core] Support for sharded tensorized models (#4990) · 51602eef
  Travis Johnson authored Jun 12, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Sanger Steel <sangersteel@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  51602eef
- [Hardware] Initial TPU integration (#5292) · 1a8bfd92
  Woosuk Kwon authored Jun 12, 2024
  
  1a8bfd92
10 Jun, 2024 1 commit
- [Misc] Update to comply with the new `compressed-tensors` config (#5350) · 5884c2b4
  Dipika Sikka authored Jun 09, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  5884c2b4
01 Jun, 2024 2 commits
- [Feature][Kernel] Support bitsandbytes quantization and QLoRA (#4776) · b9c0605a
  chenqianfzh authored Jun 01, 2024
  
  b9c0605a
- [Minor] Fix the path typo in loader.py: save_sharded_states.py -> save_sharded_state.py (#5151) · c3540728
  Ye Cao authored Jun 02, 2024
```
Signed-off-by: Ye Cao <caoye.cao@alibaba-inc.com>
```
  c3540728
24 May, 2024 1 commit
- [Bugfix] Fix Mistral v0.3 Weight Loading (#5005) · 91977095
  Robert Shaw authored May 24, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  91977095
23 May, 2024 1 commit

[Kernel] Initial Activation Quantization Support (#4525) · a1242324

Dipika Sikka authored May 23, 2024


Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

a1242324

20 May, 2024 2 commits
- [Core] Sharded State Loader download from HF (#4889) · 1937e298
  Aurick Qiao authored May 20, 2024
  
  1937e298
- [Bugfix] Fix dummy weight for fp8 (#4916) · f0eecee6
  Mor Zusman authored May 20, 2024
```
Allow dummy load format for fp8,
torch.uniform_ doesn't support FP8 at the moment
Co-authored-by: Mor Zusman <morz@ai21.com>
```
  f0eecee6
19 May, 2024 1 commit
- [Bugfix][Model] Add base class for vision-language models (#4809) · f68470e8
  Cyrus Leung authored May 19, 2024
  
  f68470e8
16 May, 2024 1 commit
- [Core] Implement sharded state loader (#4690) · 30e75439
  Aurick Qiao authored May 16, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  30e75439
13 May, 2024 1 commit
- [Frontend] [Core] perf: Automatically detect vLLM-tensorized model, update... · 8bc68e19
  Sanger Steel authored May 13, 2024
```
[Frontend] [Core] perf: Automatically detect vLLM-tensorized model, update `tensorizer` to version 2.9.0 (#4208)
```
  8bc68e19