Commits · 05af6da8d927f70d15ab1ed25b01df3c967ad961 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "b7215de2c5fcdf8af96cf941556d63934ea8f353"

05 Mar, 2024 1 commit
- [ROCm] enable cupy in order to enable cudagraph mode for AMD GPUs (#3123) · 05af6da8
  Hongxia Yang authored Mar 04, 2024
```
Co-authored-by: lcskrishna <lollachaitanya@gmail.com>
```
  05af6da8
04 Mar, 2024 9 commits
- Fix the openai benchmarking requests to work with latest OpenAI apis (#2992) · 9a4548ba
  Chen Wang authored Mar 04, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  9a4548ba
- Add health check, make async Engine more robust (#3015) · ff578cae
  Antoni Baum authored Mar 04, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  ff578cae
- Push logprob generation to LLMEngine (#3065) · 22de4523
  Antoni Baum authored Mar 04, 2024
```
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  22de4523
- [Minor fix] The domain dns.google may cause a socket.gaierror exception (#3176) · 76e8a704
  ttbachyinsda authored Mar 05, 2024
```
Co-authored-by: guofangze <guofangze@kuaishou.com>
```
  76e8a704
- enable --gpu-memory-utilization in benchmark_throughput.py (#3175) · 9cbc7e5f
  Allen.Dou authored Mar 05, 2024
```
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
```
  9cbc7e5f
- Add document for vllm paged attention kernel. (#2978) · 27a7b070
  Jialun Lyu authored Mar 04, 2024
  
  27a7b070
- [Minor Fix] Remove unused code in benchmark_prefix_caching.py (#3171) · 901cf4c5
  TianYu GUO authored Mar 04, 2024
  
  901cf4c5
- [DOC] add setup document to support neuron backend (#2777) · d0fae881
  Liangfu Chen authored Mar 03, 2024
  
  d0fae881
- Make it easy to profile workers with nsight (#3162) · 17c3103c
  Philipp Moritz authored Mar 03, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  17c3103c
03 Mar, 2024 2 commits
- [FIX] Fix styles in automatic prefix caching & add a automatic prefix caching benchmark (#3158) · 996d095c
  Zhuohan Li authored Mar 03, 2024
  
  996d095c
- Add vLLM version info to logs and openai API server (#3161) · d65fac27
  Jason Cox authored Mar 03, 2024
  
  d65fac27
02 Mar, 2024 2 commits
- Add Automatic Prefix Caching (#2762) · ce4f5a29
  Sage Moore authored Mar 02, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  ce4f5a29
- Reorder kv dtype check to avoid nvcc not found error on AMD platform (#3104) · baee28c4
  cloudhan authored Mar 02, 2024
  
  baee28c4
01 Mar, 2024 7 commits
- allow user chose log level by --log-level instead of fixed 'info'. (#3109) · 29e70e3e
  Allen.Dou authored Mar 02, 2024
```
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  29e70e3e
- Bump up to v0.3.3 (#3129) · 82091b86
  Woosuk Kwon authored Mar 01, 2024
  
  82091b86
- Integrate Marlin Kernels for Int4 GPTQ inference (#2497) · c0c2335c
  Robert Shaw authored Mar 01, 2024
```
Co-authored-by: Robert Shaw <114415538+rib-2@users.noreply.github.com>
Co-authored-by: alexm <alexm@neuralmagic.com>
```
  c0c2335c
- fix relative import path of protocol.py (#3134) · 90fbf125
  Huarong authored Mar 02, 2024
```
Co-authored-by: huohuarong <huohuarong@zuoshouyisheng.com>
```
  90fbf125
- docs: Add tutorial on deploying vLLM model with KServe (#2586) · 49d849b3
  Yuan Tang authored Mar 01, 2024
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  49d849b3
- Remove exclude_unset in streaming response (#3143) · 27ca23dc
  Seonghyeon authored Mar 02, 2024
  
  27ca23dc
- Fix: Output text is always truncated in some models (#3016) · 54d35447
  Sherry authored Mar 01, 2024
  
  54d35447
29 Feb, 2024 7 commits
- Add guided decoding for OpenAI API server (#2819) · 703e42ee
  felixzhu555 authored Feb 29, 2024
```
Co-authored-by: br3no <breno@veltefaria.de>
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  703e42ee
- [Fix] Don't deep-copy LogitsProcessors when copying SamplingParams (#3099) · 29a8d6a5
  Nick Hill authored Feb 29, 2024
  
  29a8d6a5
- Fix building from source on WSL (#3112) · 2c08ff23
  aliencaocao authored Mar 01, 2024
  
  2c08ff23
- Support starcoder2 architecture (#3089) · bfdcfa6a
  Seonghyeon authored Feb 29, 2024
  
  bfdcfa6a
- add cache_config's info to prometheus metrics. (#3100) · 9289e577
  Allen.Dou authored Feb 29, 2024
  
  9289e577
- Fix: `AttributeError` in OpenAI-compatible server (#3018) · a6d471c7
  Jae-Won Chung authored Feb 29, 2024
  
  a6d471c7
- Add Support for 2/3/8-bit GPTQ Quantization Models (#2330) · 01a5d18a
  CHU Tianxiang authored Feb 29, 2024
  
  01a5d18a
28 Feb, 2024 4 commits
- Add LoRA support for Gemma (#3050) · 929b4f29
  Woosuk Kwon authored Feb 28, 2024
  
  929b4f29
- [Neuron] Support inference with transformers-neuronx (#2569) · 3b7178cf
  Liangfu Chen authored Feb 28, 2024
  
  3b7178cf
- Restrict prometheus_client >= 0.18.0 to prevent errors when importing pkgs (#3070) · e46fa5d5
  Allen.Dou authored Feb 28, 2024
  
  e46fa5d5
- multi-lora documentation fix (#3064) · a8683102
  Ganesh Jagadeesan authored Feb 28, 2024
  
  a8683102
27 Feb, 2024 8 commits
- Enable GQA support in the prefix prefill kernels (#3007) · 71bcaf99
  Tao He authored Feb 27, 2024
```
Signed-off-by: Tao He <sighingnow@gmail.com>
```
  71bcaf99
- [Minor] Fix StableLMEpochForCausalLM -> StableLmForCausalLM (#3046) · 8b430d7d
  Woosuk Kwon authored Feb 26, 2024
  
  8b430d7d
- Support logit bias for OpenAI API (#3027) · e0ade06d
  Dylan Hawk authored Feb 26, 2024
  
  e0ade06d
- [Minor] Fix type annotation in fused moe (#3045) · 4bd18ec0
  Woosuk Kwon authored Feb 26, 2024
  
  4bd18ec0
- fix `get_ip` error in pure ipv6 environment (#2931) · 2410e320
  Jingru authored Feb 27, 2024
  
  2410e320
- Support Orion model (#2539) · 48a8f4a7
  张大成 authored Feb 27, 2024
```
Co-authored-by: zhangdacheng <zhangdacheng@ainirobot.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  48a8f4a7
- Fix stablelm (#3038) · 4dd6416f
  Roy authored Feb 27, 2024
  
  4dd6416f
- Don't use cupy when `enforce_eager=True` (#3037) · c1c0d00b
  Roy authored Feb 27, 2024
  
  c1c0d00b