Commits · c2c5e0909ad4457ad542117939c2629ebe2db609 · OpenDAS / vllm_cscc · GitLab

08 Mar, 2024 7 commits
- Move model filelocks from `/tmp/` to `~/.cache/vllm/locks/` dir (#3241) · c2c5e090
  Michael Goin authored Mar 08, 2024
  
  c2c5e090
- [FIX] Make `flash_attn` optional (#3269) · 1cb0cc29
  Woosuk Kwon authored Mar 08, 2024
  
  1cb0cc29
- [Docs] Fix Unmocked Imports (#3275) · 99c3cfb8
  Roger Wang authored Mar 08, 2024
  
  99c3cfb8
- [Minor Fix] Fix comments in benchmark_serving (#3252) · 1ece1ae8
  TianYu GUO authored Mar 08, 2024
  
  1ece1ae8
- Feature add lora support for Qwen2 (#3177) · c59e120c
  whyiug authored Mar 08, 2024
  
  c59e120c
- Connect engine healthcheck to openai server (#3260) · d2339d68
  Nick Hill authored Mar 07, 2024
  
  d2339d68
- Fix auto prefix bug (#3239) · b35cc934
  ElizaWszola authored Mar 08, 2024
  
  b35cc934
07 Mar, 2024 5 commits
- Possible fix for conflict between Automated Prefix Caching (#2762) and... · 8cbba462
  jacobthebanana authored Mar 07, 2024
```
Possible fix for conflict between Automated Prefix Caching (#2762) and multi-LoRA support (#1804) (#3263)
```
  8cbba462
- Measure model memory usage (#3120) · 385da2da
  Michael Goin authored Mar 07, 2024
  
  385da2da
- Separate attention backends (#3005) · 2daf23ab
  Woosuk Kwon authored Mar 07, 2024
  
  2daf23ab
- Update requirements-dev.txt to include package for benchmarking scripts. (#3181) · cbf4c05b
  Chen Wang authored Mar 07, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  cbf4c05b
- Add GPTQ support for Gemma (#3200) · d3c04b6a
  TechxGenus authored Mar 07, 2024
  
  d3c04b6a
06 Mar, 2024 4 commits
- Add tqdm `dynamic_ncols=True` (#3242) · 4cb3b924
  Chujie Zheng authored Mar 06, 2024
  
  4cb3b924
- [Testing] Fix core tests (#3224) · a33ce60c
  Cade Daniel authored Mar 06, 2024
  
  a33ce60c
- [Tests] Add block manager and scheduler tests (#3108) · 24aecf42
  SangBin Cho authored Mar 06, 2024
  
  24aecf42
- [Fix] Avoid pickling entire LLMEngine for Ray workers (#3207) · 2efce05d
  Nick Hill authored Mar 05, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  2efce05d
05 Mar, 2024 2 commits
- Store `eos_token_id` in `Sequence` for easy access (#3166) · 8999ec3c
  Nick Hill authored Mar 05, 2024
  
  8999ec3c
- [ROCm] enable cupy in order to enable cudagraph mode for AMD GPUs (#3123) · 05af6da8
  Hongxia Yang authored Mar 04, 2024
```
Co-authored-by: lcskrishna <lollachaitanya@gmail.com>
```
  05af6da8
04 Mar, 2024 9 commits
- Fix the openai benchmarking requests to work with latest OpenAI apis (#2992) · 9a4548ba
  Chen Wang authored Mar 04, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  9a4548ba
- Add health check, make async Engine more robust (#3015) · ff578cae
  Antoni Baum authored Mar 04, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  ff578cae
- Push logprob generation to LLMEngine (#3065) · 22de4523
  Antoni Baum authored Mar 04, 2024
```
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  22de4523
- [Minor fix] The domain dns.google may cause a socket.gaierror exception (#3176) · 76e8a704
  ttbachyinsda authored Mar 05, 2024
```
Co-authored-by: guofangze <guofangze@kuaishou.com>
```
  76e8a704
- enable --gpu-memory-utilization in benchmark_throughput.py (#3175) · 9cbc7e5f
  Allen.Dou authored Mar 05, 2024
```
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
```
  9cbc7e5f
- Add document for vllm paged attention kernel. (#2978) · 27a7b070
  Jialun Lyu authored Mar 04, 2024
  
  27a7b070
- [Minor Fix] Remove unused code in benchmark_prefix_caching.py (#3171) · 901cf4c5
  TianYu GUO authored Mar 04, 2024
  
  901cf4c5
- [DOC] add setup document to support neuron backend (#2777) · d0fae881
  Liangfu Chen authored Mar 03, 2024
  
  d0fae881
- Make it easy to profile workers with nsight (#3162) · 17c3103c
  Philipp Moritz authored Mar 03, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  17c3103c
03 Mar, 2024 2 commits
- [FIX] Fix styles in automatic prefix caching & add a automatic prefix caching benchmark (#3158) · 996d095c
  Zhuohan Li authored Mar 03, 2024
  
  996d095c
- Add vLLM version info to logs and openai API server (#3161) · d65fac27
  Jason Cox authored Mar 03, 2024
  
  d65fac27
02 Mar, 2024 2 commits
- Add Automatic Prefix Caching (#2762) · ce4f5a29
  Sage Moore authored Mar 02, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  ce4f5a29
- Reorder kv dtype check to avoid nvcc not found error on AMD platform (#3104) · baee28c4
  cloudhan authored Mar 02, 2024
  
  baee28c4
01 Mar, 2024 7 commits
- allow user chose log level by --log-level instead of fixed 'info'. (#3109) · 29e70e3e
  Allen.Dou authored Mar 02, 2024
```
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  29e70e3e
- Bump up to v0.3.3 (#3129) · 82091b86
  Woosuk Kwon authored Mar 01, 2024
  
  82091b86
- Integrate Marlin Kernels for Int4 GPTQ inference (#2497) · c0c2335c
  Robert Shaw authored Mar 01, 2024
```
Co-authored-by: Robert Shaw <114415538+rib-2@users.noreply.github.com>
Co-authored-by: alexm <alexm@neuralmagic.com>
```
  c0c2335c
- fix relative import path of protocol.py (#3134) · 90fbf125
  Huarong authored Mar 02, 2024
```
Co-authored-by: huohuarong <huohuarong@zuoshouyisheng.com>
```
  90fbf125
- docs: Add tutorial on deploying vLLM model with KServe (#2586) · 49d849b3
  Yuan Tang authored Mar 01, 2024
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  49d849b3
- Remove exclude_unset in streaming response (#3143) · 27ca23dc
  Seonghyeon authored Mar 02, 2024
  
  27ca23dc
- Fix: Output text is always truncated in some models (#3016) · 54d35447
  Sherry authored Mar 01, 2024
  
  54d35447
29 Feb, 2024 2 commits
- Add guided decoding for OpenAI API server (#2819) · 703e42ee
  felixzhu555 authored Feb 29, 2024
```
Co-authored-by: br3no <breno@veltefaria.de>
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  703e42ee
- [Fix] Don't deep-copy LogitsProcessors when copying SamplingParams (#3099) · 29a8d6a5
  Nick Hill authored Feb 29, 2024
  
  29a8d6a5