Commits · 4b4eeb26d5e8f1000787973818b08e896dc27834 · OpenDAS / vllm_cscc

23 Oct, 2024 2 commits
- [torch.compile] Adding torch compile annotations to some models (#9614) · 9013e24f
  Yongzao authored Oct 24, 2024
  
  9013e24f
- [misc] comment to avoid future confusion about baichuan (#9620) · dbdd3b5e
  youkaichao authored Oct 23, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dbdd3b5e
16 Oct, 2024 1 commit
- update paged_attn.py · 85e8224c
  zhuwenwen authored Oct 16, 2024
  
  85e8224c
15 Oct, 2024 1 commit
- support baichuan awq and skip _rocm_C · b3ab1cdc
  zhuwenwen authored Oct 15, 2024
  
  b3ab1cdc
04 Oct, 2024 1 commit

[Models] Add remaining model PP support (#7168) · 0f6d7a9a

Murali Andoorveedu authored Oct 03, 2024

Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Signed-off-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

0f6d7a9a

30 Aug, 2024 1 commit
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
20 Aug, 2024 1 commit
- [Bugfix] support `tie_word_embeddings` for all models (#5724) · f4fc7337
  Zijian Hu authored Aug 19, 2024
  
  f4fc7337
19 Aug, 2024 2 commits
- Add fa pad conditions and automatic switching strategy · b40f2ffc
  zhuwenwen authored Aug 19, 2024
  
  b40f2ffc
- Add fa pad conditions and automatic switching strategy · fa973559
  zhuwenwen authored Aug 19, 2024
  
  fa973559
17 Aug, 2024 2 commits
- update fp16 model layout conversion conditions · 4d821524
  zhuwenwen authored Aug 17, 2024
  
  4d821524
- update fp16 model layout conversion conditions · a528f350
  zhuwenwen authored Aug 17, 2024
  
  a528f350
13 Aug, 2024 1 commit
- [Bugfix] Fix weight loading for Chameleon when TP>1 (#7410) · 7025b11d
  Cyrus Leung authored Aug 13, 2024
  
  7025b11d
01 Aug, 2024 1 commit
- support fa pad · ffbef65c
  zhuwenwen authored Aug 01, 2024
  
  ffbef65c
24 Jul, 2024 2 commits
- support transpose lm_head weight to nn layout · 99426767
  zhuwenwen authored Jul 24, 2024
  
  99426767
- add gemm pad and fa pad for 7b model · 0b5e4e11
  zhuwenwen authored Jul 24, 2024
  
  0b5e4e11
22 Jul, 2024 1 commit
- fix index error of baichuan2 · 795ce518
  zhuwenwen authored Jul 22, 2024
  
  795ce518
20 Jul, 2024 1 commit
- support nn layout · 1e0cb1f4
  zhuwenwen authored Jul 20, 2024
  
  1e0cb1f4
08 Jul, 2024 1 commit
- add 7b pad dim · 5cdabd7b
  zhuwenwen authored Jul 08, 2024
  
  5cdabd7b
06 Jul, 2024 1 commit
- add fa pad · 371b1251
  zhuwenwen authored Jul 06, 2024
  
  371b1251
02 Jul, 2024 2 commits
- [CORE] Quantized lm-head Framework (#4442) · ee93f4f9
  Qubitium-ModelCloud authored Jul 03, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: ZX <zx@lbx.dev>
```
  ee93f4f9
- [Core] Pipeline Parallel Support (#4412) · c5832d2a
  Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  c5832d2a
27 Jun, 2024 2 commits
- [Model][Bugfix] Implicit model flags and reenable Phi-3-Vision (#5896) · 98cf2ed6
  Cyrus Leung authored Jun 28, 2024
  
  98cf2ed6
- [Model] Add base class for LoRA-supported models (#5018) · 96354d6a
  Cyrus Leung authored Jun 27, 2024
  
  96354d6a
22 May, 2024 1 commit

[Misc] Load FP8 kv-cache scaling factors from checkpoints (#4893) · a3a73ab0

Cody Yu authored May 22, 2024

The 2nd PR for #4532.

This PR supports loading FP8 kv-cache scaling factors from a FP8 checkpoint (with .kv_scale parameter).

a3a73ab0

13 May, 2024 1 commit
- [Misc] Enhance attention selector (#4751) · 0fca3cdc
  Woosuk Kwon authored May 13, 2024
  
  0fca3cdc
26 Apr, 2024 1 commit
- [Misc][Refactor] Generalize linear_method to be quant_method (#4373) · a62aaf1d
  Cody Yu authored Apr 26, 2024
  
  a62aaf1d
16 Apr, 2024 1 commit
- [Core] Refactor model loading code (#4097) · 69e1d2fb
  Antoni Baum authored Apr 16, 2024
  
  69e1d2fb
10 Apr, 2024 1 commit

[Core][Refactor] move parallel_utils into vllm/distributed (#3950) · 63e7176f

youkaichao authored Apr 10, 2024

[WIP][Core][Refactor] move vllm/model_executor/parallel_utils into vllm/distributed and vllm/device_communicators (#3950)

63e7176f

26 Mar, 2024 1 commit
- Enable more models to inference based on LoRA (#3382) · 8af890a8
  Jee Li authored Mar 26, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  8af890a8
25 Mar, 2024 2 commits
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
- [Core] Refactor Attention Take 2 (#3462) · 925f3332
  Woosuk Kwon authored Mar 24, 2024
  
  925f3332
20 Mar, 2024 1 commit
- Migrate `logits` computation and gather to `model_runner` (#3233) · f1c0fc39
  Roy authored Mar 21, 2024
  
  f1c0fc39
11 Mar, 2024 1 commit
- Re-enable the 80 char line width limit (#3305) · 2f8844ba
  Zhuohan Li authored Mar 10, 2024
  
  2f8844ba
07 Mar, 2024 1 commit
- Separate attention backends (#3005) · 2daf23ab
  Woosuk Kwon authored Mar 07, 2024
  
  2daf23ab
27 Feb, 2024 1 commit
- [Minor] Remove unused config files (#3039) · d9f726c4
  Roy authored Feb 27, 2024
  
  d9f726c4
13 Feb, 2024 2 commits
- Revert "Refactor llama family models (#2637)" (#2851) · ea356004
  Philipp Moritz authored Feb 13, 2024
```
This reverts commit 5c976a7e.
```
  ea356004
- Refactor llama family models (#2637) · 5c976a7e
  Roy authored Feb 13, 2024
  
  5c976a7e
03 Jan, 2024 1 commit
- Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221) · fd4ea8ef
  Zhuohan Li authored Jan 04, 2024
  
  fd4ea8ef
17 Dec, 2023 1 commit

Optimize model execution with CUDA graph (#1926) · 37ca5581

Woosuk Kwon authored Dec 16, 2023


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

37ca5581

15 Dec, 2023 1 commit
- Add GPTQ support (#916) · 0fbfc4b8
  CHU Tianxiang authored Dec 15, 2023
  
  0fbfc4b8