Commits · 385da2dae2b90e5273da8dfce881727bd9c574a1 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "cc2dca96d2d289222d2c382b155bc0b6da69356e"

07 Mar, 2024 3 commits
- Measure model memory usage (#3120) · 385da2da
  Michael Goin authored Mar 07, 2024
  
  385da2da
- Separate attention backends (#3005) · 2daf23ab
  Woosuk Kwon authored Mar 07, 2024
  
  2daf23ab
- Add GPTQ support for Gemma (#3200) · d3c04b6a
  TechxGenus authored Mar 07, 2024
  
  d3c04b6a
06 Mar, 2024 3 commits
- Add tqdm `dynamic_ncols=True` (#3242) · 4cb3b924
  Chujie Zheng authored Mar 06, 2024
  
  4cb3b924
- [Testing] Fix core tests (#3224) · a33ce60c
  Cade Daniel authored Mar 06, 2024
  
  a33ce60c
- [Fix] Avoid pickling entire LLMEngine for Ray workers (#3207) · 2efce05d
  Nick Hill authored Mar 05, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  2efce05d
05 Mar, 2024 2 commits
- Store `eos_token_id` in `Sequence` for easy access (#3166) · 8999ec3c
  Nick Hill authored Mar 05, 2024
  
  8999ec3c
- [ROCm] enable cupy in order to enable cudagraph mode for AMD GPUs (#3123) · 05af6da8
  Hongxia Yang authored Mar 04, 2024
```
Co-authored-by: lcskrishna <lollachaitanya@gmail.com>
```
  05af6da8
04 Mar, 2024 4 commits
- Add health check, make async Engine more robust (#3015) · ff578cae
  Antoni Baum authored Mar 04, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  ff578cae
- Push logprob generation to LLMEngine (#3065) · 22de4523
  Antoni Baum authored Mar 04, 2024
```
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  22de4523
- [Minor fix] The domain dns.google may cause a socket.gaierror exception (#3176) · 76e8a704
  ttbachyinsda authored Mar 05, 2024
```
Co-authored-by: guofangze <guofangze@kuaishou.com>
```
  76e8a704
- Make it easy to profile workers with nsight (#3162) · 17c3103c
  Philipp Moritz authored Mar 03, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  17c3103c
03 Mar, 2024 2 commits
- [FIX] Fix styles in automatic prefix caching & add a automatic prefix caching benchmark (#3158) · 996d095c
  Zhuohan Li authored Mar 03, 2024
  
  996d095c
- Add vLLM version info to logs and openai API server (#3161) · d65fac27
  Jason Cox authored Mar 03, 2024
  
  d65fac27
02 Mar, 2024 2 commits
- Add Automatic Prefix Caching (#2762) · ce4f5a29
  Sage Moore authored Mar 02, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  ce4f5a29
- Reorder kv dtype check to avoid nvcc not found error on AMD platform (#3104) · baee28c4
  cloudhan authored Mar 02, 2024
  
  baee28c4
01 Mar, 2024 6 commits
- allow user chose log level by --log-level instead of fixed 'info'. (#3109) · 29e70e3e
  Allen.Dou authored Mar 02, 2024
```
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  29e70e3e
- Bump up to v0.3.3 (#3129) · 82091b86
  Woosuk Kwon authored Mar 01, 2024
  
  82091b86
- Integrate Marlin Kernels for Int4 GPTQ inference (#2497) · c0c2335c
  Robert Shaw authored Mar 01, 2024
```
Co-authored-by: Robert Shaw <114415538+rib-2@users.noreply.github.com>
Co-authored-by: alexm <alexm@neuralmagic.com>
```
  c0c2335c
- fix relative import path of protocol.py (#3134) · 90fbf125
  Huarong authored Mar 02, 2024
```
Co-authored-by: huohuarong <huohuarong@zuoshouyisheng.com>
```
  90fbf125
- Remove exclude_unset in streaming response (#3143) · 27ca23dc
  Seonghyeon authored Mar 02, 2024
  
  27ca23dc
- Fix: Output text is always truncated in some models (#3016) · 54d35447
  Sherry authored Mar 01, 2024
  
  54d35447
29 Feb, 2024 6 commits
- Add guided decoding for OpenAI API server (#2819) · 703e42ee
  felixzhu555 authored Feb 29, 2024
```
Co-authored-by: br3no <breno@veltefaria.de>
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  703e42ee
- [Fix] Don't deep-copy LogitsProcessors when copying SamplingParams (#3099) · 29a8d6a5
  Nick Hill authored Feb 29, 2024
  
  29a8d6a5
- Support starcoder2 architecture (#3089) · bfdcfa6a
  Seonghyeon authored Feb 29, 2024
  
  bfdcfa6a
- add cache_config's info to prometheus metrics. (#3100) · 9289e577
  Allen.Dou authored Feb 29, 2024
  
  9289e577
- Fix: `AttributeError` in OpenAI-compatible server (#3018) · a6d471c7
  Jae-Won Chung authored Feb 29, 2024
  
  a6d471c7
- Add Support for 2/3/8-bit GPTQ Quantization Models (#2330) · 01a5d18a
  CHU Tianxiang authored Feb 29, 2024
  
  01a5d18a
28 Feb, 2024 2 commits
- Add LoRA support for Gemma (#3050) · 929b4f29
  Woosuk Kwon authored Feb 28, 2024
  
  929b4f29
- [Neuron] Support inference with transformers-neuronx (#2569) · 3b7178cf
  Liangfu Chen authored Feb 28, 2024
  
  3b7178cf
27 Feb, 2024 8 commits
- Enable GQA support in the prefix prefill kernels (#3007) · 71bcaf99
  Tao He authored Feb 27, 2024
```
Signed-off-by: Tao He <sighingnow@gmail.com>
```
  71bcaf99
- Support logit bias for OpenAI API (#3027) · e0ade06d
  Dylan Hawk authored Feb 26, 2024
  
  e0ade06d
- [Minor] Fix type annotation in fused moe (#3045) · 4bd18ec0
  Woosuk Kwon authored Feb 26, 2024
  
  4bd18ec0
- fix `get_ip` error in pure ipv6 environment (#2931) · 2410e320
  Jingru authored Feb 27, 2024
  
  2410e320
- Support Orion model (#2539) · 48a8f4a7
  张大成 authored Feb 27, 2024
```
Co-authored-by: zhangdacheng <zhangdacheng@ainirobot.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  48a8f4a7
- Fix stablelm (#3038) · 4dd6416f
  Roy authored Feb 27, 2024
  
  4dd6416f
- Don't use cupy when `enforce_eager=True` (#3037) · c1c0d00b
  Roy authored Feb 27, 2024
  
  c1c0d00b
- [Minor] Remove unused config files (#3039) · d9f726c4
  Roy authored Feb 27, 2024
  
  d9f726c4
26 Feb, 2024 2 commits
- Optimize Triton MoE Kernel (#2979) · cfc15a10
  Philipp Moritz authored Feb 26, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
```
  cfc15a10
- Add LogProbs for Chat Completions in OpenAI (#2918) · 70f3e8e3
  Jared Moore authored Feb 25, 2024
  
  70f3e8e3