Commits · 93b38bea5dd03e1b140ca997dfaadef86f8f1855 · kecinstone / 2024pra-vllm

31 Jan, 2024 2 commits
- Add unit test for Mixtral MoE layer (#2677) · d0d93b92
  Philipp Moritz authored Jan 31, 2024
  
  d0d93b92
- Add quantized mixtral support (#2673) · 3dad9444
  Woosuk Kwon authored Jan 30, 2024
  
  3dad9444
30 Jan, 2024 4 commits
- [Minor] Fix false warning when TP=1 (#2674) · 105a40f5
  Woosuk Kwon authored Jan 30, 2024
  
  105a40f5
- [Minor] Fix a small typo (#2672) · bbe9bd96
  Philipp Moritz authored Jan 30, 2024
  
  bbe9bd96
- Fused MOE for Mixtral (#2542) · ab406446
  Philipp Moritz authored Jan 29, 2024
```
Co-authored-by: chen shen <scv119@gmail.com>
```
  ab406446
- DeepseekMoE support with Fused MoE kernel (#2453) · 5d60def0
  wangding zeng authored Jan 30, 2024
```
Co-authored-by: roy <jasonailu87@gmail.com>
```
  5d60def0
29 Jan, 2024 1 commit

Support FP8-E5M2 KV Cache (#2279) · 9090bf02

zhaoyang-star authored Jan 29, 2024


Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

9090bf02

27 Jan, 2024 2 commits
- Implement custom all reduce kernels (#2192) · 38017003
  Hanzhi Zhou authored Jan 28, 2024
  
  38017003
- AWQ: Up to 2.66x higher throughput (#2566) · beb89f68
  Casper authored Jan 27, 2024
  
  beb89f68
26 Jan, 2024 1 commit
- Support for Stable LM 2 (#2598) · 3a0e1fc0
  dakotamahan-stability authored Jan 26, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  3a0e1fc0
25 Jan, 2024 1 commit
- fix names and license for Qwen2 (#2589) · 2832e7b9
  Junyang Lin authored Jan 25, 2024
  
  2832e7b9
23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa

22 Jan, 2024 2 commits
- Add qwen2 (#2495) · 94b5edeb
  Junyang Lin authored Jan 23, 2024
  
  94b5edeb
- [Speculative decoding 2/9] Multi-step worker for draft model (#2424) · 18bfcdd0
  Cade Daniel authored Jan 21, 2024
  
  18bfcdd0
21 Jan, 2024 1 commit
- Add `group` as an argument in broadcast ops (#2522) · 5b23c3f2
  Junda Chen authored Jan 20, 2024
  
  5b23c3f2
20 Jan, 2024 1 commit
- [Bugfix] fix load local safetensors model (#2512) · 91a61da9
  Roy authored Jan 20, 2024
  
  91a61da9
19 Jan, 2024 2 commits
- Simplify broadcast logic for control messages (#2501) · ef9b636e
  Zhuohan Li authored Jan 19, 2024
  
  ef9b636e
- refactor complemention api for readability (#2499) · dd7e8f5f
  Simon Mo authored Jan 18, 2024
  
  dd7e8f5f
18 Jan, 2024 3 commits
- Don't download both safetensor and bin files. (#2480) · 7e108113
  Nikola Borisov authored Jan 18, 2024
  
  7e108113
- fix stablelm.py tensor-parallel-size bug (#2482) · 8a25d3a7
  YingchaoX authored Jan 19, 2024
  
  8a25d3a7
- [Experimental] Prefix Caching Support (#1669) · d10f8e1d
  shiyi.c_98 authored Jan 17, 2024
```
Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  d10f8e1d
17 Jan, 2024 1 commit
- Add StableLM3B model (#2372) · e1957c6e
  Hyunsung Lee authored Jan 17, 2024
  
  e1957c6e
15 Jan, 2024 1 commit
- fix weigit loading for GQA with TP (#2379) · f780504d
  Chenhui Zhang authored Jan 16, 2024
  
  f780504d
12 Jan, 2024 3 commits
- Aligning `top_p` and `top_k` Sampling (#1885) · 218dc2cc
  陈序 authored Jan 13, 2024
```
* Align top_p and top_k with huggingface

* remove _get_prompt_and_output_tokens

* rename _apply_top_p_top_k

* compare top_p top_k with hf

* fix test errors
```
  218dc2cc
- Address Phi modeling update 2 (#2428) · 7878958c
  Gary Hui authored Jan 13, 2024
  
  7878958c
- Rename phi_1_5 -> phi (#2385) · 50376faa
  Woosuk Kwon authored Jan 11, 2024
  
  50376faa
09 Jan, 2024 1 commit
- [Speculative decoding 1/9] Optimized rejection sampler (#2336) · 79d64c49
  Cade Daniel authored Jan 09, 2024
  
  79d64c49
08 Jan, 2024 1 commit
- [Minor] Remove unused code in attention (#2384) · 28c3f121
  Woosuk Kwon authored Jan 08, 2024
  
  28c3f121
03 Jan, 2024 2 commits
- Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221) · fd4ea8ef
  Zhuohan Li authored Jan 04, 2024
  
  fd4ea8ef
- [Minor] Fix typo and remove unused code (#2305) · 91405610
  Roy authored Jan 03, 2024
  
  91405610
30 Dec, 2023 1 commit
- Support GPT-NeoX Models without attention biases (#2301) · 4934d492
  Jong-hun Shin authored Dec 31, 2023
  
  4934d492
20 Dec, 2023 2 commits
- Remove Sampler copy stream (#2209) · bd29cf3d
  Antoni Baum authored Dec 20, 2023
  
  bd29cf3d
- [BugFix] Fix weight loading for Mixtral with TP (#2208) · ba4f8267
  Woosuk Kwon authored Dec 19, 2023
  
  ba4f8267
19 Dec, 2023 1 commit
- Added DeciLM-7b and DeciLM-7b-instruct (#2062) · de60a3fb
  avideci authored Dec 19, 2023
  
  de60a3fb
17 Dec, 2023 4 commits
- [Minor] Fix a typo in .pt weight support (#2160) · 2c9b6380
  Woosuk Kwon authored Dec 17, 2023
  
  2c9b6380
- Make sampler less blocking (#1889) · a7347d9a
  Antoni Baum authored Dec 17, 2023
  
  a7347d9a
- Remove dependency on CuPy (#2152) · c3372e87
  Woosuk Kwon authored Dec 17, 2023
  
  c3372e87
- Optimize model execution with CUDA graph (#1926) · 37ca5581
  Woosuk Kwon authored Dec 16, 2023
```
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  37ca5581
16 Dec, 2023 1 commit
- Simplify weight loading logic (#2133) · eed74a55
  Roy authored Dec 17, 2023
  
  eed74a55
15 Dec, 2023 1 commit
- Add GPTQ support (#916) · 0fbfc4b8
  CHU Tianxiang authored Dec 15, 2023
  
  0fbfc4b8