Commits · 2e8fc0d4c3bf8374f1f55569069e59ef45d4bc98 · xdb4_94051 / vllm

10 Dec, 2023 5 commits
- Fix completion API echo and logprob combo (#1992) · 2e8fc0d4
  Simon Mo authored Dec 10, 2023
  
  2e8fc0d4
- Replace head_mapping params with num_kv_heads to attention kernel. (#1997) · dacaf5a4
  wbn authored Dec 11, 2023
```
Co-authored-by: wangguoya <wangguoya@baidu.com>
Co-authored-by: Yang Zhao <zhaoyangstar@foxmail.com>
```
  dacaf5a4
- [Minor] Add comment on skipping rope caches (#2004) · 24cde76a
  Woosuk Kwon authored Dec 10, 2023
  
  24cde76a
- Fix OpenAI server completion_tokens referenced before assignment (#1996) · 1aa13615
  Jin Shang authored Dec 10, 2023
  
  1aa13615
- [Minor] Fix code style for baichuan (#2003) · fe470ae5
  Woosuk Kwon authored Dec 09, 2023
  
  fe470ae5
09 Dec, 2023 1 commit
- Fix for KeyError on Loading LLaMA (#1978) · 3a8c2381
  Jun Gao authored Dec 10, 2023
  
  3a8c2381
08 Dec, 2023 2 commits

Fix Baichuan2-7B-Chat (#1987) · 2b981012
firebook authored Dec 09, 2023

2b981012

Merge EmbeddedLLM/vllm-rocm into vLLM main (#1836) · 6ccc0bff

TJian authored Dec 08, 2023


Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Amir Balwel <amoooori04@gmail.com>
Co-authored-by: root <kuanfu.liu@akirakan.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kuanfu <kuanfu.liu@embeddedllm.com>
Co-authored-by: miloice <17350011+kliuae@users.noreply.github.com>

6ccc0bff

07 Dec, 2023 1 commit
- Make InternLM follow `rope_scaling` in `config.json` (#1956) · ebede26e
  Jie Li authored Dec 08, 2023
```
Co-authored-by: lijie8 <lijie8@sensetime.com>
```
  ebede26e
05 Dec, 2023 1 commit
- Support ChatGLMForConditionalGeneration (#1932) · 1d9b737e
  dancingpipi authored Dec 06, 2023
```
Co-authored-by: shujunhua1 <shujunhua1@jd.com>
```
  1d9b737e
03 Dec, 2023 6 commits
- add custom server params (#1868) · 60dc62dc
  Roy authored Dec 04, 2023
  
  60dc62dc
- Bump up to v0.2.3 (#1903) · 0f90effc
  Woosuk Kwon authored Dec 03, 2023
  
  0f90effc
- Fix num_gpus when TP > 1 (#1852) · 464dd985
  Woosuk Kwon authored Dec 03, 2023
  
  464dd985
- Add PyTorch-native implementation of custom layers (#1898) · 9b294976
  Woosuk Kwon authored Dec 02, 2023
  
  9b294976
- Add Production Metrics in Prometheus format (#1890) · 5313c2cb
  Simon Mo authored Dec 02, 2023
  
  5313c2cb
- Fix broken sampler tests (#1896) · 5f09cbdb
  Woosuk Kwon authored Dec 02, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  5f09cbdb
01 Dec, 2023 4 commits
- Fix the typo in SamplingParams' docstring (#1886) · f86bd619
  Jerry authored Dec 01, 2023
  
  f86bd619
- Normalize head weights for Baichuan 2 (#1876) · e5452ddf
  Woosuk Kwon authored Nov 30, 2023
  
  e5452ddf
- Fix Baichuan tokenizer error (#1874) · d06980df
  Woosuk Kwon authored Nov 30, 2023
  
  d06980df
- Support chat template and `echo` for chat API (#1756) · 66785cc0
  Adam Brusselback authored Nov 30, 2023
  
  66785cc0
30 Nov, 2023 4 commits
- Fix rope cache key error (#1867) · d27f4bae
  Roy authored Dec 01, 2023
  
  d27f4bae
- Avoid multiple instantiations of the RoPE class (#1828) · 63b2206a
  Jee Li authored Nov 30, 2023
  
  63b2206a
- Refactor Worker & InputMetadata (#1843) · 27feead2
  Woosuk Kwon authored Nov 29, 2023
  
  27feead2
- Disable Logs Requests should Disable Logging of requests. (#1779) · c7821956
  Michael McCulloch authored Nov 29, 2023
```
Co-authored-by: Michael McCulloch <mjm.gitlab@fastmail.com>
```
  c7821956
29 Nov, 2023 5 commits
- Refactor Attention (#1840) · a9e45742
  Woosuk Kwon authored Nov 29, 2023
  
  a9e45742
- Better integration with Ray Serve (#1821) · 0229c386
  FlorianJoncour authored Nov 29, 2023
```
Co-authored-by: FlorianJoncour <florian@zetta-sys.com>
```
  0229c386
- [Fix] Fix RoPE in ChatGLM-32K (#1841) · a7b3e330
  Woosuk Kwon authored Nov 29, 2023
  
  a7b3e330
- [FIX] Fix formatting error · 1cb4ad8d
  Zhuohan Li authored Nov 29, 2023
  
  1cb4ad8d
- Use the type BlockTable (#1791) · 6ed068a7
  explainerauthors authored Nov 28, 2023
  
  6ed068a7
28 Nov, 2023 3 commits
- [FIX] Fix class naming (#1803) · 708e6c18
  Zhuohan Li authored Nov 28, 2023
  
  708e6c18
- Fix OPT param names (#1819) · b9438904
  Woosuk Kwon authored Nov 28, 2023
  
  b9438904
- Correct comments in parallel_state.py (#1818) · a1125ad4
  explainerauthors authored Nov 28, 2023
  
  a1125ad4
27 Nov, 2023 2 commits
- Init model on GPU to reduce CPU memory footprint (#1796) · a8b150c5
  ljss authored Nov 28, 2023
  
  a8b150c5
- Added echo function to OpenAI API server. (#1504) · 665cbcec
  Yunmo Chen authored Nov 27, 2023
  
  665cbcec
24 Nov, 2023 2 commits
- Fix model docstrings (#1764) · 7c600440
  Woosuk Kwon authored Nov 23, 2023
  
  7c600440
- [Build] Avoid building too many extensions (#1624) · e0c6f556
  Yanming W authored Nov 23, 2023
  
  e0c6f556
22 Nov, 2023 4 commits
- Fix repetition penalty aligned with huggingface (#1577) · de23687d
  ljss authored Nov 23, 2023
  
  de23687d
- Set top_p=0 and top_k=-1 in greedy sampling (#1748) · 4cea74c7
  ljss authored Nov 23, 2023
  
  4cea74c7
- [DOCS] Add engine args documentation (#1741) · a921d8be
  Casper authored Nov 22, 2023
  
  a921d8be
- Add stop_token_ids in SamplingParams.__repr__ (#1745) · 094f716b
  陈序 authored Nov 22, 2023
  
  094f716b