Commits · 6774bd50b0c19b3f33717d15763022df385a1b70 · norm / vllm

"ios/vscode:/vscode.git/clone" did not exist on "88358528422cf0dc1b6f752f8a58e7330bb87cae"

14 Dec, 2023 3 commits
- Fix typing in AsyncLLMEngine & add toml to requirements-dev (#2100) · 6774bd50
  mezuzza authored Dec 14, 2023
  
  6774bd50
- Bump up to v0.2.5 (#2095) · 31c1f325
  Woosuk Kwon authored Dec 13, 2023
  
  31c1f325
- Optimize Mixtral with expert parallelism (#2090) · 21d93c14
  Antoni Baum authored Dec 13, 2023
  
  21d93c14
13 Dec, 2023 7 commits
- [BugFix] Fix input positions for long context with sliding window (#2088) · f1c85201
  Woosuk Kwon authored Dec 13, 2023
  
  f1c85201
- [Docs] Add notes on ROCm-supported models (#2087) · 096827c2
  Woosuk Kwon authored Dec 13, 2023
  
  096827c2
- Update installation instruction for vLLM + CUDA 11.8 (#2086) · 6565d9e3
  Woosuk Kwon authored Dec 13, 2023
  
  6565d9e3
- [ROCm] Upgrade xformers version for ROCm & update doc (#2079) · f375ec84
  TJian authored Dec 13, 2023
```
Co-authored-by: miloice <jeffaw99@hotmail.com>
```
  f375ec84
- Implement lazy model loader (#2044) · 518369d7
  Woosuk Kwon authored Dec 12, 2023
  
  518369d7
- Fix peak memory profiling (#2031) · 30bad5c4
  Woosuk Kwon authored Dec 12, 2023
  
  30bad5c4
- Update Dockerfile to build Megablocks (#2042) · 3fefe271
  Simon Mo authored Dec 12, 2023
  
  3fefe271
12 Dec, 2023 4 commits
- Support MPT with GQA (#1938) · 6428f1d0
  Megha Agarwal authored Dec 12, 2023
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6428f1d0
- Remove einops from requirements (#2049) · 7e1b21da
  Woosuk Kwon authored Dec 12, 2023
  
  7e1b21da
- Upgrade transformers version to 4.36.0 (#2046) · cb3f30c6
  Woosuk Kwon authored Dec 11, 2023
  
  cb3f30c6
- [CI/CD] Upgrade PyTorch version to v2.1.1 (#2045) · f3e024be
  Woosuk Kwon authored Dec 11, 2023
  
  f3e024be
11 Dec, 2023 13 commits
- Remove python 3.10 requirement (#2040) · 31d2ab4a
  Woosuk Kwon authored Dec 11, 2023
  
  31d2ab4a
- Update Dockerfile to support Mixtral (#2027) · eb172128
  Simon Mo authored Dec 11, 2023
  
  eb172128
- Bump up to v0.2.4 (#2034) · 4dd4b5c5
  Woosuk Kwon authored Dec 11, 2023
  
  4dd4b5c5
- Fix import error msg for megablocks (#2038) · 6120e5aa
  Woosuk Kwon authored Dec 11, 2023
  
  6120e5aa
- Update README.md to add megablocks requirement for mixtral (#2033) · 2eaa81b2
  Ram authored Dec 12, 2023
  
  2eaa81b2
- [Minor] Fix type annotation in Mixtral (#2036) · 81ce2a4b
  Woosuk Kwon authored Dec 11, 2023
  
  81ce2a4b
- Fix latency benchmark script (#2035) · 5dd80d37
  Woosuk Kwon authored Dec 11, 2023
  
  5dd80d37
- Revert adding Megablocks (#2030) · beeee69b
  Woosuk Kwon authored Dec 11, 2023
  
  beeee69b
- Update requirements.txt for mixtral (#2029) · 9bf28d0b
  Ram authored Dec 12, 2023
  
  9bf28d0b
- Update run_on_sky.rst (#2025) · c0ce15df
  Ikko Eltociear Ashimine authored Dec 12, 2023
```
sharable -> shareable
```
  c0ce15df
- Change the load format to pt for Mixtral (#2028) · b9bcdc71
  Woosuk Kwon authored Dec 11, 2023
  
  b9bcdc71
- Minor fixes for Mixtral (#2015) · 4ff02039
  Woosuk Kwon authored Dec 11, 2023
  
  4ff02039
- Mixtral 8x7B support (#2011) · b5f882cc
  Pierre Stock authored Dec 11, 2023
```
Co-authored-by: Pierre Stock <p@mistral.ai>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  b5f882cc
10 Dec, 2023 5 commits
- Fix completion API echo and logprob combo (#1992) · 2e8fc0d4
  Simon Mo authored Dec 10, 2023
  
  2e8fc0d4
- Replace head_mapping params with num_kv_heads to attention kernel. (#1997) · dacaf5a4
  wbn authored Dec 11, 2023
```
Co-authored-by: wangguoya <wangguoya@baidu.com>
Co-authored-by: Yang Zhao <zhaoyangstar@foxmail.com>
```
  dacaf5a4
- [Minor] Add comment on skipping rope caches (#2004) · 24cde76a
  Woosuk Kwon authored Dec 10, 2023
  
  24cde76a
- Fix OpenAI server completion_tokens referenced before assignment (#1996) · 1aa13615
  Jin Shang authored Dec 10, 2023
  
  1aa13615
- [Minor] Fix code style for baichuan (#2003) · fe470ae5
  Woosuk Kwon authored Dec 09, 2023
  
  fe470ae5
09 Dec, 2023 1 commit
- Fix for KeyError on Loading LLaMA (#1978) · 3a8c2381
  Jun Gao authored Dec 10, 2023
  
  3a8c2381
08 Dec, 2023 4 commits
- [Docker] Add cuda arch list as build option (#1950) · c85b80c2
  Simon Mo authored Dec 08, 2023
  
  c85b80c2
- Fix Baichuan2-7B-Chat (#1987) · 2b981012
  firebook authored Dec 09, 2023
  
  2b981012
- Merge EmbeddedLLM/vllm-rocm into vLLM main (#1836) · 6ccc0bff
  TJian authored Dec 08, 2023
```
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Amir Balwel <amoooori04@gmail.com>
Co-authored-by: root <kuanfu.liu@akirakan.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kuanfu <kuanfu.liu@embeddedllm.com>
Co-authored-by: miloice <17350011+kliuae@users.noreply.github.com>
```
  6ccc0bff
- fix typo in getenv call (#1972) · c8e7eb1e
  Daya Khudia authored Dec 07, 2023
  
  c8e7eb1e
07 Dec, 2023 3 commits
- [Docker] Adding number of nvcc_threads during build as envar (#1893) · 24f60a54
  AguirreNicolas authored Dec 07, 2023
  
  24f60a54
- Fix quickstart.rst typo jinja (#1964) · 42c02f58
  gottlike authored Dec 07, 2023
  
  42c02f58
- Make InternLM follow `rope_scaling` in `config.json` (#1956) · ebede26e
  Jie Li authored Dec 08, 2023
```
Co-authored-by: lijie8 <lijie8@sensetime.com>
```
  ebede26e