Commits · f0d4e145575bf6fb96c141d776ce92c9bfc79c49 · kecinstone / 2024pra-vllm

06 Feb, 2024 1 commit
- Add fused top-K softmax kernel for MoE (#2769) · f0d4e145
  Woosuk Kwon authored Feb 05, 2024
  
  f0d4e145
01 Feb, 2024 1 commit
- Add Internlm2 (#2666) · cd9e60c7
  Fengzhe Zhou authored Feb 02, 2024
  
  cd9e60c7
31 Jan, 2024 2 commits
- Add unit test for Mixtral MoE layer (#2677) · d0d93b92
  Philipp Moritz authored Jan 31, 2024
  
  d0d93b92
- Add quantized mixtral support (#2673) · 3dad9444
  Woosuk Kwon authored Jan 30, 2024
  
  3dad9444
30 Jan, 2024 2 commits
- Fused MOE for Mixtral (#2542) · ab406446
  Philipp Moritz authored Jan 29, 2024
```
Co-authored-by: chen shen <scv119@gmail.com>
```
  ab406446
- DeepseekMoE support with Fused MoE kernel (#2453) · 5d60def0
  wangding zeng authored Jan 30, 2024
```
Co-authored-by: roy <jasonailu87@gmail.com>
```
  5d60def0
26 Jan, 2024 1 commit
- Support for Stable LM 2 (#2598) · 3a0e1fc0
  dakotamahan-stability authored Jan 26, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  3a0e1fc0
25 Jan, 2024 1 commit
- fix names and license for Qwen2 (#2589) · 2832e7b9
  Junyang Lin authored Jan 25, 2024
  
  2832e7b9
23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa

22 Jan, 2024 1 commit
- Add qwen2 (#2495) · 94b5edeb
  Junyang Lin authored Jan 23, 2024
  
  94b5edeb
18 Jan, 2024 1 commit
- fix stablelm.py tensor-parallel-size bug (#2482) · 8a25d3a7
  YingchaoX authored Jan 19, 2024
  
  8a25d3a7
17 Jan, 2024 1 commit
- Add StableLM3B model (#2372) · e1957c6e
  Hyunsung Lee authored Jan 17, 2024
  
  e1957c6e
12 Jan, 2024 2 commits
- Address Phi modeling update 2 (#2428) · 7878958c
  Gary Hui authored Jan 13, 2024
  
  7878958c
- Rename phi_1_5 -> phi (#2385) · 50376faa
  Woosuk Kwon authored Jan 11, 2024
  
  50376faa
03 Jan, 2024 1 commit
- Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221) · fd4ea8ef
  Zhuohan Li authored Jan 04, 2024
  
  fd4ea8ef
30 Dec, 2023 1 commit
- Support GPT-NeoX Models without attention biases (#2301) · 4934d492
  Jong-hun Shin authored Dec 31, 2023
  
  4934d492
20 Dec, 2023 1 commit
- [BugFix] Fix weight loading for Mixtral with TP (#2208) · ba4f8267
  Woosuk Kwon authored Dec 19, 2023
  
  ba4f8267
19 Dec, 2023 1 commit
- Added DeciLM-7b and DeciLM-7b-instruct (#2062) · de60a3fb
  avideci authored Dec 19, 2023
  
  de60a3fb
17 Dec, 2023 1 commit

Optimize model execution with CUDA graph (#1926) · 37ca5581

Woosuk Kwon authored Dec 16, 2023


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

37ca5581

16 Dec, 2023 1 commit
- Simplify weight loading logic (#2133) · eed74a55
  Roy authored Dec 17, 2023
  
  eed74a55
15 Dec, 2023 1 commit
- Add GPTQ support (#916) · 0fbfc4b8
  CHU Tianxiang authored Dec 15, 2023
  
  0fbfc4b8
14 Dec, 2023 1 commit
- Optimize Mixtral with expert parallelism (#2090) · 21d93c14
  Antoni Baum authored Dec 13, 2023
  
  21d93c14
13 Dec, 2023 1 commit
- Implement lazy model loader (#2044) · 518369d7
  Woosuk Kwon authored Dec 12, 2023
  
  518369d7
12 Dec, 2023 2 commits
- Support MPT with GQA (#1938) · 6428f1d0
  Megha Agarwal authored Dec 12, 2023
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6428f1d0
- Upgrade transformers version to 4.36.0 (#2046) · cb3f30c6
  Woosuk Kwon authored Dec 11, 2023
  
  cb3f30c6
11 Dec, 2023 5 commits
- Remove python 3.10 requirement (#2040) · 31d2ab4a
  Woosuk Kwon authored Dec 11, 2023
  
  31d2ab4a
- Fix import error msg for megablocks (#2038) · 6120e5aa
  Woosuk Kwon authored Dec 11, 2023
  
  6120e5aa
- [Minor] Fix type annotation in Mixtral (#2036) · 81ce2a4b
  Woosuk Kwon authored Dec 11, 2023
  
  81ce2a4b
- Minor fixes for Mixtral (#2015) · 4ff02039
  Woosuk Kwon authored Dec 11, 2023
  
  4ff02039
- Mixtral 8x7B support (#2011) · b5f882cc
  Pierre Stock authored Dec 11, 2023
```
Co-authored-by: Pierre Stock <p@mistral.ai>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  b5f882cc
10 Dec, 2023 2 commits
- [Minor] Add comment on skipping rope caches (#2004) · 24cde76a
  Woosuk Kwon authored Dec 10, 2023
  
  24cde76a
- [Minor] Fix code style for baichuan (#2003) · fe470ae5
  Woosuk Kwon authored Dec 09, 2023
  
  fe470ae5
09 Dec, 2023 1 commit
- Fix for KeyError on Loading LLaMA (#1978) · 3a8c2381
  Jun Gao authored Dec 10, 2023
  
  3a8c2381
08 Dec, 2023 1 commit
- Fix Baichuan2-7B-Chat (#1987) · 2b981012
  firebook authored Dec 09, 2023
  
  2b981012
07 Dec, 2023 1 commit
- Make InternLM follow `rope_scaling` in `config.json` (#1956) · ebede26e
  Jie Li authored Dec 08, 2023
```
Co-authored-by: lijie8 <lijie8@sensetime.com>
```
  ebede26e
01 Dec, 2023 1 commit
- Normalize head weights for Baichuan 2 (#1876) · e5452ddf
  Woosuk Kwon authored Nov 30, 2023
  
  e5452ddf
30 Nov, 2023 1 commit
- Refactor Worker & InputMetadata (#1843) · 27feead2
  Woosuk Kwon authored Nov 29, 2023
  
  27feead2
29 Nov, 2023 2 commits
- Refactor Attention (#1840) · a9e45742
  Woosuk Kwon authored Nov 29, 2023
  
  a9e45742
- [Fix] Fix RoPE in ChatGLM-32K (#1841) · a7b3e330
  Woosuk Kwon authored Nov 29, 2023
  
  a7b3e330
28 Nov, 2023 1 commit
- Fix OPT param names (#1819) · b9438904
  Woosuk Kwon authored Nov 28, 2023
  
  b9438904