Commits · 703e42ee4b3efed3c71e7ae7d15f0f96e05722d4 · kecinstone / 2024pra-vllm

29 Feb, 2024 3 commits
- Add guided decoding for OpenAI API server (#2819) · 703e42ee
  felixzhu555 authored Feb 29, 2024
```
Co-authored-by: br3no <breno@veltefaria.de>
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  703e42ee
- [Fix] Don't deep-copy LogitsProcessors when copying SamplingParams (#3099) · 29a8d6a5
  Nick Hill authored Feb 29, 2024
  
  29a8d6a5
- add cache_config's info to prometheus metrics. (#3100) · 9289e577
  Allen.Dou authored Feb 29, 2024
  
  9289e577
28 Feb, 2024 1 commit
- [Neuron] Support inference with transformers-neuronx (#2569) · 3b7178cf
  Liangfu Chen authored Feb 28, 2024
  
  3b7178cf
27 Feb, 2024 1 commit
- Don't use cupy when `enforce_eager=True` (#3037) · c1c0d00b
  Roy authored Feb 27, 2024
  
  c1c0d00b
25 Feb, 2024 1 commit
- Port metrics from `aioprometheus` to `prometheus_client` (#2730) · ef978fe4
  Harry Mellor authored Feb 25, 2024
  
  ef978fe4
22 Feb, 2024 1 commit
- Include tokens from prompt phase in `counter_generation_tokens` (#2802) · 4caf7044
  Ronen Schaffer authored Feb 23, 2024
  
  4caf7044
21 Feb, 2024 2 commits
- Support per-request seed (#2514) · 7d2dcce1
  Nick Hill authored Feb 21, 2024
  
  7d2dcce1
- Add metrics to RequestOutput (#2876) · 017d9f15
  Antoni Baum authored Feb 20, 2024
  
  017d9f15
19 Feb, 2024 1 commit
- Fix `vllm:prompt_tokens_total` metric calculation (#2869) · e433c115
  Ronen Schaffer authored Feb 19, 2024
  
  e433c115
18 Feb, 2024 1 commit
- Add code-revision config argument for Hugging Face Hub (#2892) · 786b7f18
  Mark Mozolewski authored Feb 17, 2024
  
  786b7f18
17 Feb, 2024 1 commit

Defensively copy `sampling_params` (#2881) · 185b2c29

Nick Hill authored Feb 17, 2024

If the SamplingParams object passed to LLMEngine.add_request() is mutated after it returns, it could affect the async sampling process for that request.

Suggested by @Yard1 https://github.com/vllm-project/vllm/pull/2514#discussion_r1490106059

185b2c29

13 Feb, 2024 1 commit
- Use CuPy for CUDA graphs (#2811) · a463c333
  Woosuk Kwon authored Feb 13, 2024
  
  a463c333
08 Feb, 2024 1 commit
- [Ray] Integration compiled DAG off by default (#2471) · 65b89d16
  SangBin Cho authored Feb 09, 2024
  
  65b89d16
04 Feb, 2024 1 commit
- Remove eos tokens from output by default (#2611) · 5a6c81b0
  Rex authored Feb 04, 2024
  
  5a6c81b0
01 Feb, 2024 1 commit
- Remove hardcoded `device="cuda" ` to support more devices (#2503) · 96b6f475
  Kunshang Ji authored Feb 02, 2024
```
Co-authored-by: Jiang Li <jiang1.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  96b6f475
31 Jan, 2024 2 commits
- Refactor Prometheus and Add Request Level Metrics (#2316) · 93b38bea
  Robert Shaw authored Jan 31, 2024
  
  93b38bea
- fix some bugs (#2689) · c664b0e6
  zspo authored Feb 01, 2024
  
  c664b0e6
30 Jan, 2024 1 commit
- Fix 'Actor methods cannot be called directly' when using `--engine-use-ray` (#2664) · d79ced32
  Wen Sun authored Jan 31, 2024
```
* fix: engine-useray complain

* fix: typo
```
  d79ced32
29 Jan, 2024 2 commits
- Fix error when tp > 1 (#2644) · b72af8f1
  zhaoyang-star authored Jan 29, 2024
```
Co-authored-by: zhaoyang-star <zhao.yang16@zte.com.cn>
```
  b72af8f1
- Support FP8-E5M2 KV Cache (#2279) · 9090bf02
  zhaoyang-star authored Jan 29, 2024
```
Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  9090bf02
28 Jan, 2024 1 commit
- Small async_llm_engine refactor (#2618) · 89be30fa
  Murali Andoorveedu authored Jan 27, 2024
  
  89be30fa
27 Jan, 2024 2 commits
- [Minor] Fix warning on Ray dependencies (#2630) · 5f036d2b
  Woosuk Kwon authored Jan 27, 2024
  
  5f036d2b
- Implement custom all reduce kernels (#2192) · 38017003
  Hanzhi Zhou authored Jan 28, 2024
  
  38017003
23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa

22 Jan, 2024 2 commits
- Fix https://github.com/vllm-project/vllm/issues/2540 (#2545) · ab7e6006
  Philipp Moritz authored Jan 22, 2024
  
  ab7e6006
- [Speculative decoding 2/9] Multi-step worker for draft model (#2424) · 18bfcdd0
  Cade Daniel authored Jan 21, 2024
  
  18bfcdd0
18 Jan, 2024 1 commit

[Experimental] Prefix Caching Support (#1669) · d10f8e1d

shiyi.c_98 authored Jan 17, 2024


Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

d10f8e1d

12 Jan, 2024 1 commit
- [DOC] Add additional comments for LLMEngine and AsyncLLMEngine (#1011) · 6549aef2
  Jiaxiang authored Jan 12, 2024
  
  6549aef2
07 Jan, 2024 1 commit
- Changed scheduler to use deques instead of lists (#2290) · 05921a9a
  Nadav Shmayovits authored Jan 07, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  05921a9a
05 Jan, 2024 1 commit
- Ensure metrics are logged regardless of requests (#2347) · d0215a58
  Iskren Ivov Chernev authored Jan 05, 2024
  
  d0215a58
04 Jan, 2024 1 commit
- Miner fix of type hint (#2340) · aee8ef66
  ljss authored Jan 04, 2024
  
  aee8ef66
03 Jan, 2024 1 commit
- Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221) · fd4ea8ef
  Zhuohan Li authored Jan 04, 2024
  
  fd4ea8ef
26 Dec, 2023 1 commit
- [BUGFIX] Do not return ignored sentences twice in async llm engine (#2258) · e0ff9200
  Zhuohan Li authored Dec 26, 2023
  
  e0ff9200
21 Dec, 2023 1 commit
- Disable Ray usage stats collection (#2206) · 3a4fd5ca
  Woosuk Kwon authored Dec 20, 2023
  
  3a4fd5ca
18 Dec, 2023 3 commits
- Update Help Text for --gpu-memory-utilization Argument (#2183) · 290e015c
  Suhong Moon authored Dec 18, 2023
  
  290e015c
- [Minor] Fix typo (#2166) · bbe4466f
  JohnSaxon authored Dec 18, 2023
```
Co-authored-by: John-Saxon <zhang.xiangxuan@oushu.com>
```
  bbe4466f
- [BugFix] Raise error when max_model_len is larger than KV cache (#2163) · 8041b730
  Woosuk Kwon authored Dec 17, 2023
  
  8041b730
17 Dec, 2023 2 commits
- [Minor] Add more detailed explanation on `quantization` argument (#2145) · 30fb0956
  Woosuk Kwon authored Dec 17, 2023
  
  30fb0956
- Remove dependency on CuPy (#2152) · c3372e87
  Woosuk Kwon authored Dec 17, 2023
  
  c3372e87