Commits · 65b89d16ee54063d0737e3da1b15e2177916118d · kecinstone / 2024pra-vllm · GitLab

08 Feb, 2024 1 commit
- [Ray] Integration compiled DAG off by default (#2471) · 65b89d16
  SangBin Cho authored Feb 09, 2024
  
  65b89d16
06 Feb, 2024 3 commits
- [Minor] More fix of test_cache.py CI test failure (#2750) · fe6d09ae
  Lily Liu authored Feb 06, 2024
  
  fe6d09ae
- modelscope: fix issue when model parameter is not a model id but path of the model. (#2489) · ed70c70e
  liuyhwangyh authored Feb 07, 2024
  
  ed70c70e
- Add fused top-K softmax kernel for MoE (#2769) · f0d4e145
  Woosuk Kwon authored Feb 05, 2024
  
  f0d4e145
05 Feb, 2024 1 commit
- Set local logging level via env variable (#2774) · b92adec8
  Lukas authored Feb 05, 2024
  
  b92adec8
04 Feb, 2024 2 commits
- Remove eos tokens from output by default (#2611) · 5a6c81b0
  Rex authored Feb 04, 2024
  
  5a6c81b0
- set&get llm internal tokenizer instead of the TokenizerGroup (#2741) · 51cd22ce
  dancingpipi authored Feb 05, 2024
```
Co-authored-by: shujunhua1 <shujunhua1@jd.com>
```
  51cd22ce
01 Feb, 2024 5 commits
- Fix default length_penalty to 1.0 (#2667) · 0e163fce
  zspo authored Feb 02, 2024
  
  0e163fce
- Remove hardcoded `device="cuda" ` to support more devices (#2503) · 96b6f475
  Kunshang Ji authored Feb 02, 2024
```
Co-authored-by: Jiang Li <jiang1.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  96b6f475
- Use revision when downloading the quantization config file (#2697) · c410f5d0
  Pernekhan Utemuratov authored Feb 01, 2024
```
Co-authored-by: Pernekhan Utemuratov <pernekhan@deepinfra.com>
```
  c410f5d0
- fix python 3.8 syntax (#2716) · b9e96b17
  Simon Mo authored Feb 01, 2024
  
  b9e96b17
- Add Internlm2 (#2666) · cd9e60c7
  Fengzhe Zhou authored Feb 02, 2024
  
  cd9e60c7
31 Jan, 2024 6 commits
- Refactor Prometheus and Add Request Level Metrics (#2316) · 93b38bea
  Robert Shaw authored Jan 31, 2024
  
  93b38bea
- Add unit test for Mixtral MoE layer (#2677) · d0d93b92
  Philipp Moritz authored Jan 31, 2024
  
  d0d93b92
- fix some bugs (#2689) · c664b0e6
  zspo authored Feb 01, 2024
  
  c664b0e6
- Fixes assertion failure in prefix caching: the lora index mapping should respect prefix_len (#2688) · d69ff0cb
  Tao He authored Feb 01, 2024
```
Signed-off-by: Tao He <sighingnow@gmail.com>
```
  d69ff0cb
- Bump up version to v0.3.0 (#2656) · 1af090b5
  Zhuohan Li authored Jan 31, 2024
  
  1af090b5
- Add quantized mixtral support (#2673) · 3dad9444
  Woosuk Kwon authored Jan 30, 2024
  
  3dad9444
30 Jan, 2024 5 commits
- [Minor] Fix false warning when TP=1 (#2674) · 105a40f5
  Woosuk Kwon authored Jan 30, 2024
  
  105a40f5
- [Minor] Fix a small typo (#2672) · bbe9bd96
  Philipp Moritz authored Jan 30, 2024
  
  bbe9bd96
- Fix 'Actor methods cannot be called directly' when using `--engine-use-ray` (#2664) · d79ced32
  Wen Sun authored Jan 31, 2024
```
* fix: engine-useray complain

* fix: typo
```
  d79ced32
- Fused MOE for Mixtral (#2542) · ab406446
  Philipp Moritz authored Jan 29, 2024
```
Co-authored-by: chen shen <scv119@gmail.com>
```
  ab406446
- DeepseekMoE support with Fused MoE kernel (#2453) · 5d60def0
  wangding zeng authored Jan 30, 2024
```
Co-authored-by: roy <jasonailu87@gmail.com>
```
  5d60def0
29 Jan, 2024 2 commits
- Fix error when tp > 1 (#2644) · b72af8f1
  zhaoyang-star authored Jan 29, 2024
```
Co-authored-by: zhaoyang-star <zhao.yang16@zte.com.cn>
```
  b72af8f1
- Support FP8-E5M2 KV Cache (#2279) · 9090bf02
  zhaoyang-star authored Jan 29, 2024
```
Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  9090bf02
28 Jan, 2024 1 commit
- Small async_llm_engine refactor (#2618) · 89be30fa
  Murali Andoorveedu authored Jan 27, 2024
  
  89be30fa
27 Jan, 2024 4 commits
- [Minor] Fix warning on Ray dependencies (#2630) · 5f036d2b
  Woosuk Kwon authored Jan 27, 2024
  
  5f036d2b
- Implement custom all reduce kernels (#2192) · 38017003
  Hanzhi Zhou authored Jan 28, 2024
  
  38017003
- Use head_dim in config if exists (#2622) · 220a4762
  Xiang Xu authored Jan 27, 2024
  
  220a4762
- AWQ: Up to 2.66x higher throughput (#2566) · beb89f68
  Casper authored Jan 27, 2024
  
  beb89f68
26 Jan, 2024 3 commits
- Don't build punica kernels by default (#2605) · 390b495f
  Philipp Moritz authored Jan 26, 2024
  
  390b495f
- Support for Stable LM 2 (#2598) · 3a0e1fc0
  dakotamahan-stability authored Jan 26, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  3a0e1fc0
- [ROCm] add support to ROCm 6.0 and MI300 (#2274) · 6b7de1a0
  Hongxia Yang authored Jan 26, 2024
  
  6b7de1a0
25 Jan, 2024 2 commits
- fix names and license for Qwen2 (#2589) · 2832e7b9
  Junyang Lin authored Jan 25, 2024
  
  2832e7b9
- Support Batch Completion in Server (#2529) · 3a7dd7e3
  Simon Mo authored Jan 24, 2024
  
  3a7dd7e3
24 Jan, 2024 2 commits
- Added `include_stop_str_in_output` and `length_penalty` parameters to OpenAI API (#2562) · f1f6cc10
  Federico Galatolo authored Jan 24, 2024
  
  f1f6cc10
- [Bugfix] fix crash if max_tokens=None (#2570) · 3209b490
  Nikola Borisov authored Jan 23, 2024
  
  3209b490
23 Jan, 2024 2 commits
- [Experimental] Add multi-LoRA support (#1804) · 9b945daa
  Antoni Baum authored Jan 24, 2024
```
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  9b945daa
- [Feature] Simple API token authentication and pluggable middlewares (#1106) · 9c1352eb
  Erfan Al-Hossami authored Jan 23, 2024
  
  9c1352eb
22 Jan, 2024 1 commit
- Add qwen2 (#2495) · 94b5edeb
  Junyang Lin authored Jan 23, 2024
  
  94b5edeb