Commits · 21d93c140d0a97af5f0c59e660cf04bd417fd424 · kecinstone / 2024pra-vllm

14 Dec, 2023 1 commit
- Optimize Mixtral with expert parallelism (#2090) · 21d93c14
  Antoni Baum authored Dec 13, 2023
  
  21d93c14
13 Dec, 2023 3 commits
- [BugFix] Fix input positions for long context with sliding window (#2088) · f1c85201
  Woosuk Kwon authored Dec 13, 2023
  
  f1c85201
- Implement lazy model loader (#2044) · 518369d7
  Woosuk Kwon authored Dec 12, 2023
  
  518369d7
- Fix peak memory profiling (#2031) · 30bad5c4
  Woosuk Kwon authored Dec 12, 2023
  
  30bad5c4
12 Dec, 2023 2 commits
- Support MPT with GQA (#1938) · 6428f1d0
  Megha Agarwal authored Dec 12, 2023
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6428f1d0
- Upgrade transformers version to 4.36.0 (#2046) · cb3f30c6
  Woosuk Kwon authored Dec 11, 2023
  
  cb3f30c6
11 Dec, 2023 7 commits
- Remove python 3.10 requirement (#2040) · 31d2ab4a
  Woosuk Kwon authored Dec 11, 2023
  
  31d2ab4a
- Bump up to v0.2.4 (#2034) · 4dd4b5c5
  Woosuk Kwon authored Dec 11, 2023
  
  4dd4b5c5
- Fix import error msg for megablocks (#2038) · 6120e5aa
  Woosuk Kwon authored Dec 11, 2023
  
  6120e5aa
- [Minor] Fix type annotation in Mixtral (#2036) · 81ce2a4b
  Woosuk Kwon authored Dec 11, 2023
  
  81ce2a4b
- Change the load format to pt for Mixtral (#2028) · b9bcdc71
  Woosuk Kwon authored Dec 11, 2023
  
  b9bcdc71
- Minor fixes for Mixtral (#2015) · 4ff02039
  Woosuk Kwon authored Dec 11, 2023
  
  4ff02039
- Mixtral 8x7B support (#2011) · b5f882cc
  Pierre Stock authored Dec 11, 2023
```
Co-authored-by: Pierre Stock <p@mistral.ai>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  b5f882cc
10 Dec, 2023 5 commits
- Fix completion API echo and logprob combo (#1992) · 2e8fc0d4
  Simon Mo authored Dec 10, 2023
  
  2e8fc0d4
- Replace head_mapping params with num_kv_heads to attention kernel. (#1997) · dacaf5a4
  wbn authored Dec 11, 2023
```
Co-authored-by: wangguoya <wangguoya@baidu.com>
Co-authored-by: Yang Zhao <zhaoyangstar@foxmail.com>
```
  dacaf5a4
- [Minor] Add comment on skipping rope caches (#2004) · 24cde76a
  Woosuk Kwon authored Dec 10, 2023
  
  24cde76a
- Fix OpenAI server completion_tokens referenced before assignment (#1996) · 1aa13615
  Jin Shang authored Dec 10, 2023
  
  1aa13615
- [Minor] Fix code style for baichuan (#2003) · fe470ae5
  Woosuk Kwon authored Dec 09, 2023
  
  fe470ae5
09 Dec, 2023 1 commit
- Fix for KeyError on Loading LLaMA (#1978) · 3a8c2381
  Jun Gao authored Dec 10, 2023
  
  3a8c2381
08 Dec, 2023 2 commits

Fix Baichuan2-7B-Chat (#1987) · 2b981012
firebook authored Dec 09, 2023

2b981012

Merge EmbeddedLLM/vllm-rocm into vLLM main (#1836) · 6ccc0bff

TJian authored Dec 08, 2023


Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Amir Balwel <amoooori04@gmail.com>
Co-authored-by: root <kuanfu.liu@akirakan.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kuanfu <kuanfu.liu@embeddedllm.com>
Co-authored-by: miloice <17350011+kliuae@users.noreply.github.com>

6ccc0bff

07 Dec, 2023 1 commit
- Make InternLM follow `rope_scaling` in `config.json` (#1956) · ebede26e
  Jie Li authored Dec 08, 2023
```
Co-authored-by: lijie8 <lijie8@sensetime.com>
```
  ebede26e
05 Dec, 2023 1 commit
- Support ChatGLMForConditionalGeneration (#1932) · 1d9b737e
  dancingpipi authored Dec 06, 2023
```
Co-authored-by: shujunhua1 <shujunhua1@jd.com>
```
  1d9b737e
03 Dec, 2023 6 commits
- add custom server params (#1868) · 60dc62dc
  Roy authored Dec 04, 2023
  
  60dc62dc
- Bump up to v0.2.3 (#1903) · 0f90effc
  Woosuk Kwon authored Dec 03, 2023
  
  0f90effc
- Fix num_gpus when TP > 1 (#1852) · 464dd985
  Woosuk Kwon authored Dec 03, 2023
  
  464dd985
- Add PyTorch-native implementation of custom layers (#1898) · 9b294976
  Woosuk Kwon authored Dec 02, 2023
  
  9b294976
- Add Production Metrics in Prometheus format (#1890) · 5313c2cb
  Simon Mo authored Dec 02, 2023
  
  5313c2cb
- Fix broken sampler tests (#1896) · 5f09cbdb
  Woosuk Kwon authored Dec 02, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  5f09cbdb
01 Dec, 2023 4 commits
- Fix the typo in SamplingParams' docstring (#1886) · f86bd619
  Jerry authored Dec 01, 2023
  
  f86bd619
- Normalize head weights for Baichuan 2 (#1876) · e5452ddf
  Woosuk Kwon authored Nov 30, 2023
  
  e5452ddf
- Fix Baichuan tokenizer error (#1874) · d06980df
  Woosuk Kwon authored Nov 30, 2023
  
  d06980df
- Support chat template and `echo` for chat API (#1756) · 66785cc0
  Adam Brusselback authored Nov 30, 2023
  
  66785cc0
30 Nov, 2023 4 commits
- Fix rope cache key error (#1867) · d27f4bae
  Roy authored Dec 01, 2023
  
  d27f4bae
- Avoid multiple instantiations of the RoPE class (#1828) · 63b2206a
  Jee Li authored Nov 30, 2023
  
  63b2206a
- Refactor Worker & InputMetadata (#1843) · 27feead2
  Woosuk Kwon authored Nov 29, 2023
  
  27feead2
- Disable Logs Requests should Disable Logging of requests. (#1779) · c7821956
  Michael McCulloch authored Nov 29, 2023
```
Co-authored-by: Michael McCulloch <mjm.gitlab@fastmail.com>
```
  c7821956
29 Nov, 2023 3 commits
- Refactor Attention (#1840) · a9e45742
  Woosuk Kwon authored Nov 29, 2023
  
  a9e45742
- Better integration with Ray Serve (#1821) · 0229c386
  FlorianJoncour authored Nov 29, 2023
```
Co-authored-by: FlorianJoncour <florian@zetta-sys.com>
```
  0229c386
- [Fix] Fix RoPE in ChatGLM-32K (#1841) · a7b3e330
  Woosuk Kwon authored Nov 29, 2023
  
  a7b3e330