Commits · e7c1b7f341c11309ed7ef09939dbcaa163969123 · OpenDAS / vllm_cscc

21 Aug, 2024 1 commit
- fix tests and update the usage of fa · 1c18cce0
  zhuwenwen authored Aug 21, 2024
  
  1c18cce0
20 Aug, 2024 1 commit
- Refactoring the optimized kernel · 4405f82c
  zhuwenwen authored Aug 20, 2024
  
  4405f82c
09 Aug, 2024 1 commit
- 增添lmslim gptq支持 · dcaabcf7
  gaoqiong authored Aug 09, 2024
  
  dcaabcf7
04 Aug, 2024 2 commits
- add qwen awq support · 982c1545
  gaoqiong authored Aug 05, 2024
  
  982c1545
- add llama model awq support · 5f5ddc3d
  gaoqiong authored Aug 04, 2024
  
  5f5ddc3d
01 Aug, 2024 1 commit
- [Bugfix][TPU] Do not use torch.Generator for TPUs (#6981) · 23993a79
  Woosuk Kwon authored Jul 31, 2024
  
  23993a79
31 Jul, 2024 1 commit
- [Bugfix] Support cpu offloading with fp8 quantization (#6960) · 460c1884
  Michael Goin authored Jul 31, 2024
  
  460c1884
24 Jul, 2024 3 commits
- [Bugfix]fix modelscope compatible issue (#6730) · f4f8a9d8
  liuyhwangyh authored Jul 24, 2024
  
  f4f8a9d8
- add gemm pad and fa pad for 7b model · 0b5e4e11
  zhuwenwen authored Jul 24, 2024
  
  0b5e4e11
- add a list that supports nn model architectures · 0e7df36b
  zhuwenwen authored Jul 24, 2024
  
  0e7df36b
23 Jul, 2024 4 commits
- [bitsandbytes]: support read bnb pre-quantized model (#5753) · 87525fab
  dongmao zhang authored Jul 23, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  87525fab
- support ignore patterns in model loader (#6673) · 3eda4ec7
  Simon Mo authored Jul 22, 2024
  
  3eda4ec7
- [misc] only tqdm for first rank (#6672) · c5201240
  youkaichao authored Jul 22, 2024
  
  c5201240
- add tqdm when loading checkpoint shards (#6569) · e519ae09
  zhaotyer authored Jul 23, 2024
```
Co-authored-by: tianyi.zhao <tianyi.zhao@transwarp.io>
Co-authored-by: youkaichao <youkaichao@126.com>
```
  e519ae09
20 Jul, 2024 2 commits
- support nn layout · 1e0cb1f4
  zhuwenwen authored Jul 20, 2024
  
  1e0cb1f4
- modify gemm pad strategy · 7fe40ced
  zhuwenwen authored Jul 20, 2024
  
  7fe40ced
17 Jul, 2024 2 commits
- set default layout · 22839191
  zhuwenwen authored Jul 17, 2024
  
  22839191
- [misc][distributed] add seed to dummy weights (#6491) · ce37be7b
  youkaichao authored Jul 16, 2024
  
  ce37be7b
16 Jul, 2024 2 commits
- [Kernel][Attention] Separate `Attention.kv_scale` into `k_scale` and `v_scale` (#6081) · 978aed53
  Michael Goin authored Jul 16, 2024
  
  978aed53
- [BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cleanup bug (#6425) · 9ad32dac
  Mor Zusman authored Jul 16, 2024
```
Co-authored-by: Mor Zusman <morz@ai21.com>
```
  9ad32dac
15 Jul, 2024 1 commit
- [Misc] Add CustomOp Interface to UnquantizedFusedMoEMethod (#6289) · ec9933f4
  Woosuk Kwon authored Jul 15, 2024
  
  ec9933f4
14 Jul, 2024 1 commit
- [ Misc ] Enable Quantizing All Layers of DeekSeekv2 (#6423) · 73030b7d
  Robert Shaw authored Jul 14, 2024
  
  73030b7d
06 Jul, 2024 1 commit
- add fa pad · 371b1251
  zhuwenwen authored Jul 06, 2024
  
  371b1251
03 Jul, 2024 2 commits
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [hardware][misc] introduce platform abstraction (#6080) · 482045ee
  youkaichao authored Jul 02, 2024
  
  482045ee
02 Jul, 2024 1 commit

[VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c

xwjiang2010 authored Jul 02, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

98d6682c

01 Jul, 2024 2 commits
- [misc][cuda] use nvml to avoid accidentally cuda initialization (#6007) · 614aa512
  youkaichao authored Jun 30, 2024
  
  614aa512
- add qwen2 arch · e9aa4ff0
  zhuwenwen authored Jul 01, 2024
  
  e9aa4ff0
28 Jun, 2024 2 commits
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
- add gemm paddig · e58014d7
  zhuwenwen authored Jun 28, 2024
  
  e58014d7
27 Jun, 2024 1 commit
- [Model] Add base class for LoRA-supported models (#5018) · 96354d6a
  Cyrus Leung authored Jun 27, 2024
  
  96354d6a
20 Jun, 2024 1 commit
- [Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718) · 8065a7e2
  Michael Goin authored Jun 20, 2024
  
  8065a7e2
15 Jun, 2024 1 commit
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
12 Jun, 2024 2 commits
- [Frontend] [Core] Support for sharded tensorized models (#4990) · 51602eef
  Travis Johnson authored Jun 12, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Sanger Steel <sangersteel@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  51602eef
- [Hardware] Initial TPU integration (#5292) · 1a8bfd92
  Woosuk Kwon authored Jun 12, 2024
  
  1a8bfd92
10 Jun, 2024 1 commit
- [Misc] Update to comply with the new `compressed-tensors` config (#5350) · 5884c2b4
  Dipika Sikka authored Jun 09, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  5884c2b4
01 Jun, 2024 2 commits
- [Feature][Kernel] Support bitsandbytes quantization and QLoRA (#4776) · b9c0605a
  chenqianfzh authored Jun 01, 2024
  
  b9c0605a
- [Minor] Fix the path typo in loader.py: save_sharded_states.py -> save_sharded_state.py (#5151) · c3540728
  Ye Cao authored Jun 02, 2024
```
Signed-off-by: Ye Cao <caoye.cao@alibaba-inc.com>
```
  c3540728
30 May, 2024 1 commit
- support tn/nn · e5d707db
  zhuwenwen authored May 30, 2024
  
  e5d707db
25 May, 2024 1 commit
- skip fp8 · f09d77ac
  zhuwenwen authored May 25, 2024
  
  f09d77ac