Commits · 5d60def02cb5a43fa5864fcb123909b101df9ec5 · xdb4_94051 / vllm

"vscode:/vscode.git/clone" did not exist on "f0f5c4e805f6408328fc988e9601c923eff134a7"

30 Jan, 2024 1 commit
- DeepseekMoE support with Fused MoE kernel (#2453) · 5d60def0
  wangding zeng authored Jan 30, 2024
```
Co-authored-by: roy <jasonailu87@gmail.com>
```
  5d60def0
29 Jan, 2024 4 commits
- ROCm: Allow setting compilation target (#2581) · ea8489fc
  Rasmus Larsen authored Jan 29, 2024
  
  ea8489fc
- No repeated IPC open (#2642) · 1b20639a
  Hanzhi Zhou authored Jan 30, 2024
  
  1b20639a
- Fix error when tp > 1 (#2644) · b72af8f1
  zhaoyang-star authored Jan 29, 2024
```
Co-authored-by: zhaoyang-star <zhao.yang16@zte.com.cn>
```
  b72af8f1
- Support FP8-E5M2 KV Cache (#2279) · 9090bf02
  zhaoyang-star authored Jan 29, 2024
```
Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  9090bf02
28 Jan, 2024 3 commits
- Update Ray version requirements (#2636) · 7d648418
  Simon Mo authored Jan 28, 2024
  
  7d648418
- Small async_llm_engine refactor (#2618) · 89be30fa
  Murali Andoorveedu authored Jan 27, 2024
  
  89be30fa
- Speed up Punica compilation (#2632) · f8ecb84c
  Woosuk Kwon authored Jan 27, 2024
  
  f8ecb84c
27 Jan, 2024 4 commits
- [Minor] Fix warning on Ray dependencies (#2630) · 5f036d2b
  Woosuk Kwon authored Jan 27, 2024
  
  5f036d2b
- Implement custom all reduce kernels (#2192) · 38017003
  Hanzhi Zhou authored Jan 28, 2024
  
  38017003
- Use head_dim in config if exists (#2622) · 220a4762
  Xiang Xu authored Jan 27, 2024
  
  220a4762
- AWQ: Up to 2.66x higher throughput (#2566) · beb89f68
  Casper authored Jan 27, 2024
  
  beb89f68
26 Jan, 2024 4 commits
- Don't build punica kernels by default (#2605) · 390b495f
  Philipp Moritz authored Jan 26, 2024
  
  390b495f
- Support for Stable LM 2 (#2598) · 3a0e1fc0
  dakotamahan-stability authored Jan 26, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  3a0e1fc0
- [ROCm] add support to ROCm 6.0 and MI300 (#2274) · 6b7de1a0
  Hongxia Yang authored Jan 26, 2024
  
  6b7de1a0
- use a correct device when creating OptionalCUDAGuard (#2583) · 5265631d
  Vladimir authored Jan 26, 2024
  
  5265631d
25 Jan, 2024 2 commits
- fix names and license for Qwen2 (#2589) · 2832e7b9
  Junyang Lin authored Jan 25, 2024
  
  2832e7b9
- Support Batch Completion in Server (#2529) · 3a7dd7e3
  Simon Mo authored Jan 24, 2024
  
  3a7dd7e3
24 Jan, 2024 3 commits
- Fix the syntax error in the doc of supported_models (#2584) · 223c1922
  LastWhisper authored Jan 25, 2024
  
  223c1922
- Added `include_stop_str_in_output` and `length_penalty` parameters to OpenAI API (#2562) · f1f6cc10
  Federico Galatolo authored Jan 24, 2024
  
  f1f6cc10
- [Bugfix] fix crash if max_tokens=None (#2570) · 3209b490
  Nikola Borisov authored Jan 23, 2024
  
  3209b490
23 Jan, 2024 3 commits
- lint: format all python file instead of just source code (#2567) · 1e4277d2
  Simon Mo authored Jan 23, 2024
  
  1e4277d2
- [Experimental] Add multi-LoRA support (#1804) · 9b945daa
  Antoni Baum authored Jan 24, 2024
```
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  9b945daa
- [Feature] Simple API token authentication and pluggable middlewares (#1106) · 9c1352eb
  Erfan Al-Hossami authored Jan 23, 2024
  
  9c1352eb
22 Jan, 2024 6 commits
- Add a 1-line docstring to explain why calling context_attention_fwd twice in... · 7a0b011d
  Jason Zhu authored Jan 22, 2024
```
Add a 1-line docstring to explain why calling context_attention_fwd twice in test_prefix_prefill.py (#2553)
```
  7a0b011d
- Fix progress bar and allow HTTPS in `benchmark_serving.py` (#2552) · 63e835cb
  Harry Mellor authored Jan 22, 2024
  
  63e835cb
- Add qwen2 (#2495) · 94b5edeb
  Junyang Lin authored Jan 23, 2024
  
  94b5edeb
- Fix https://github.com/vllm-project/vllm/issues/2540 (#2545) · ab7e6006
  Philipp Moritz authored Jan 22, 2024
  
  ab7e6006
- [Speculative decoding 2/9] Multi-step worker for draft model (#2424) · 18bfcdd0
  Cade Daniel authored Jan 21, 2024
  
  18bfcdd0
- migrate pydantic from v1 to v2 (#2531) · 71d63ed7
  Jannis Schönleber authored Jan 22, 2024
  
  71d63ed7
21 Jan, 2024 2 commits
- [Fix] Keep `scheduler.running` as deque (#2523) · d75c4073
  Nick Hill authored Jan 20, 2024
  
  d75c4073
- Add `group` as an argument in broadcast ops (#2522) · 5b23c3f2
  Junda Chen authored Jan 20, 2024
  
  5b23c3f2
20 Jan, 2024 2 commits
- Add benchmark serving to CI (#2505) · 00efdc84
  Simon Mo authored Jan 19, 2024
  
  00efdc84
- [Bugfix] fix load local safetensors model (#2512) · 91a61da9
  Roy authored Jan 20, 2024
  
  91a61da9
19 Jan, 2024 3 commits
- Simplify broadcast logic for control messages (#2501) · ef9b636e
  Zhuohan Li authored Jan 19, 2024
  
  ef9b636e
- Support OpenAI API server in `benchmark_serving.py` (#2172) · 2709c000
  Harry Mellor authored Jan 19, 2024
  
  2709c000
- refactor complemention api for readability (#2499) · dd7e8f5f
  Simon Mo authored Jan 18, 2024
  
  dd7e8f5f
18 Jan, 2024 3 commits
- [BugFix] Fix abort_seq_group (#2463) · d2a68364
  ljss authored Jan 19, 2024
  
  d2a68364
- Don't download both safetensor and bin files. (#2480) · 7e108113
  Nikola Borisov authored Jan 18, 2024
  
  7e108113
- [Neuron] Add an option to build with neuron (#2065) · 18473cf4
  Liangfu Chen authored Jan 18, 2024
  
  18473cf4