Commits · b9e96b17de4c555e2249db9f6149b346232c000e · xdb4_94051 / vllm

"vscode:/vscode.git/clone" did not exist on "4f8b99f68cdc8ee368f9de5f236b2ad33b70ca02"

01 Feb, 2024 3 commits
- fix python 3.8 syntax (#2716) · b9e96b17
  Simon Mo authored Feb 01, 2024
  
  b9e96b17
- Fix compile error when using rocm (#2648) · 923797fe
  zhaoyang-star authored Feb 02, 2024
  
  923797fe
- Add Internlm2 (#2666) · cd9e60c7
  Fengzhe Zhou authored Feb 02, 2024
  
  cd9e60c7
31 Jan, 2024 7 commits
- Refactor Prometheus and Add Request Level Metrics (#2316) · 93b38bea
  Robert Shaw authored Jan 31, 2024
  
  93b38bea
- Add unit test for Mixtral MoE layer (#2677) · d0d93b92
  Philipp Moritz authored Jan 31, 2024
  
  d0d93b92
- [Minor] Fix test_cache.py CI test failure (#2684) · 89efcf1c
  Philipp Moritz authored Jan 31, 2024
  
  89efcf1c
- fix some bugs (#2689) · c664b0e6
  zspo authored Feb 01, 2024
  
  c664b0e6
- Fixes assertion failure in prefix caching: the lora index mapping should respect prefix_len (#2688) · d69ff0cb
  Tao He authored Feb 01, 2024
```
Signed-off-by: Tao He <sighingnow@gmail.com>
```
  d69ff0cb
- Bump up version to v0.3.0 (#2656) · 1af090b5
  Zhuohan Li authored Jan 31, 2024
  
  1af090b5
- Add quantized mixtral support (#2673) · 3dad9444
  Woosuk Kwon authored Jan 30, 2024
  
  3dad9444
30 Jan, 2024 6 commits
- [Minor] Fix false warning when TP=1 (#2674) · 105a40f5
  Woosuk Kwon authored Jan 30, 2024
  
  105a40f5
- [Minor] Fix a small typo (#2672) · bbe9bd96
  Philipp Moritz authored Jan 30, 2024
  
  bbe9bd96
- Add swap_blocks unit tests (#2616) · 4f65af0e
  Vladimir authored Jan 30, 2024
  
  4f65af0e
- Fix 'Actor methods cannot be called directly' when using `--engine-use-ray` (#2664) · d79ced32
  Wen Sun authored Jan 31, 2024
```
* fix: engine-useray complain

* fix: typo
```
  d79ced32
- Fused MOE for Mixtral (#2542) · ab406446
  Philipp Moritz authored Jan 29, 2024
```
Co-authored-by: chen shen <scv119@gmail.com>
```
  ab406446
- DeepseekMoE support with Fused MoE kernel (#2453) · 5d60def0
  wangding zeng authored Jan 30, 2024
```
Co-authored-by: roy <jasonailu87@gmail.com>
```
  5d60def0
29 Jan, 2024 4 commits
- ROCm: Allow setting compilation target (#2581) · ea8489fc
  Rasmus Larsen authored Jan 29, 2024
  
  ea8489fc
- No repeated IPC open (#2642) · 1b20639a
  Hanzhi Zhou authored Jan 30, 2024
  
  1b20639a
- Fix error when tp > 1 (#2644) · b72af8f1
  zhaoyang-star authored Jan 29, 2024
```
Co-authored-by: zhaoyang-star <zhao.yang16@zte.com.cn>
```
  b72af8f1
- Support FP8-E5M2 KV Cache (#2279) · 9090bf02
  zhaoyang-star authored Jan 29, 2024
```
Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  9090bf02
28 Jan, 2024 3 commits
- Update Ray version requirements (#2636) · 7d648418
  Simon Mo authored Jan 28, 2024
  
  7d648418
- Small async_llm_engine refactor (#2618) · 89be30fa
  Murali Andoorveedu authored Jan 27, 2024
  
  89be30fa
- Speed up Punica compilation (#2632) · f8ecb84c
  Woosuk Kwon authored Jan 27, 2024
  
  f8ecb84c
27 Jan, 2024 4 commits
- [Minor] Fix warning on Ray dependencies (#2630) · 5f036d2b
  Woosuk Kwon authored Jan 27, 2024
  
  5f036d2b
- Implement custom all reduce kernels (#2192) · 38017003
  Hanzhi Zhou authored Jan 28, 2024
  
  38017003
- Use head_dim in config if exists (#2622) · 220a4762
  Xiang Xu authored Jan 27, 2024
  
  220a4762
- AWQ: Up to 2.66x higher throughput (#2566) · beb89f68
  Casper authored Jan 27, 2024
  
  beb89f68
26 Jan, 2024 4 commits
- Don't build punica kernels by default (#2605) · 390b495f
  Philipp Moritz authored Jan 26, 2024
  
  390b495f
- Support for Stable LM 2 (#2598) · 3a0e1fc0
  dakotamahan-stability authored Jan 26, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  3a0e1fc0
- [ROCm] add support to ROCm 6.0 and MI300 (#2274) · 6b7de1a0
  Hongxia Yang authored Jan 26, 2024
  
  6b7de1a0
- use a correct device when creating OptionalCUDAGuard (#2583) · 5265631d
  Vladimir authored Jan 26, 2024
  
  5265631d
25 Jan, 2024 2 commits
- fix names and license for Qwen2 (#2589) · 2832e7b9
  Junyang Lin authored Jan 25, 2024
  
  2832e7b9
- Support Batch Completion in Server (#2529) · 3a7dd7e3
  Simon Mo authored Jan 24, 2024
  
  3a7dd7e3
24 Jan, 2024 3 commits
- Fix the syntax error in the doc of supported_models (#2584) · 223c1922
  LastWhisper authored Jan 25, 2024
  
  223c1922
- Added `include_stop_str_in_output` and `length_penalty` parameters to OpenAI API (#2562) · f1f6cc10
  Federico Galatolo authored Jan 24, 2024
  
  f1f6cc10
- [Bugfix] fix crash if max_tokens=None (#2570) · 3209b490
  Nikola Borisov authored Jan 23, 2024
  
  3209b490
23 Jan, 2024 3 commits
- lint: format all python file instead of just source code (#2567) · 1e4277d2
  Simon Mo authored Jan 23, 2024
  
  1e4277d2
- [Experimental] Add multi-LoRA support (#1804) · 9b945daa
  Antoni Baum authored Jan 24, 2024
```
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  9b945daa
- [Feature] Simple API token authentication and pluggable middlewares (#1106) · 9c1352eb
  Erfan Al-Hossami authored Jan 23, 2024
  
  9c1352eb
22 Jan, 2024 1 commit
- Add a 1-line docstring to explain why calling context_attention_fwd twice in... · 7a0b011d
  Jason Zhu authored Jan 22, 2024
```
Add a 1-line docstring to explain why calling context_attention_fwd twice in test_prefix_prefill.py (#2553)
```
  7a0b011d