Commits · d27f4bae393214b4e7715fc3cb5754d4bf801bce · kecinstone / 2024pra-vllm

30 Nov, 2023 3 commits
- Fix rope cache key error (#1867) · d27f4bae
  Roy authored Dec 01, 2023
  
  d27f4bae
- Avoid multiple instantiations of the RoPE class (#1828) · 63b2206a
  Jee Li authored Nov 30, 2023
  
  63b2206a
- Refactor Worker & InputMetadata (#1843) · 27feead2
  Woosuk Kwon authored Nov 29, 2023
  
  27feead2
29 Nov, 2023 2 commits
- Refactor Attention (#1840) · a9e45742
  Woosuk Kwon authored Nov 29, 2023
  
  a9e45742
- [Fix] Fix RoPE in ChatGLM-32K (#1841) · a7b3e330
  Woosuk Kwon authored Nov 29, 2023
  
  a7b3e330
28 Nov, 2023 3 commits
- [FIX] Fix class naming (#1803) · 708e6c18
  Zhuohan Li authored Nov 28, 2023
  
  708e6c18
- Fix OPT param names (#1819) · b9438904
  Woosuk Kwon authored Nov 28, 2023
  
  b9438904
- Correct comments in parallel_state.py (#1818) · a1125ad4
  explainerauthors authored Nov 28, 2023
  
  a1125ad4
27 Nov, 2023 1 commit
- Init model on GPU to reduce CPU memory footprint (#1796) · a8b150c5
  ljss authored Nov 28, 2023
  
  a8b150c5
24 Nov, 2023 2 commits
- Fix model docstrings (#1764) · 7c600440
  Woosuk Kwon authored Nov 23, 2023
  
  7c600440
- [Build] Avoid building too many extensions (#1624) · e0c6f556
  Yanming W authored Nov 23, 2023
  
  e0c6f556
22 Nov, 2023 1 commit
- Fix repetition penalty aligned with huggingface (#1577) · de23687d
  ljss authored Nov 23, 2023
  
  de23687d
21 Nov, 2023 3 commits
- [FIX] Fix the case when `input_is_parallel=False` for `ScaledActivation` (#1737) · 7d761fe3
  Zhuohan Li authored Nov 20, 2023
  
  7d761fe3
- [BugFix] Fix TP support for AWQ (#1731) · cf35d8f3
  Woosuk Kwon authored Nov 20, 2023
  
  cf35d8f3
- Rewrite torch.repeat_interleave to remove cpu synchronization (#1599) · 819b18e7
  ljss authored Nov 21, 2023
  
  819b18e7
20 Nov, 2023 2 commits
- [BugFix] Fix a bug in loading safetensors (#1732) · f5a37c6c
  Woosuk Kwon authored Nov 20, 2023
  
  f5a37c6c
- Migrate linter from `pylint` to `ruff` (#1665) · 5ffc0d13
  Simon Mo authored Nov 20, 2023
  
  5ffc0d13
19 Nov, 2023 3 commits
- [Optimization] Implement fused add rmsnorm (#1667) · e1054247
  ljss authored Nov 19, 2023
  
  e1054247
- Add AWQ support for all models (#1714) · 8d17774f
  Woosuk Kwon authored Nov 18, 2023
  
  8d17774f
- use get_tensor in safe_open (#1696) · e946260c
  twaka authored Nov 19, 2023
  
  e946260c
18 Nov, 2023 2 commits
- Use `quantization_config` in hf config (#1695) · bb00f66e
  Woosuk Kwon authored Nov 17, 2023
  
  bb00f66e
- Support Min P Sampler (#1642) · e87557b0
  Roy authored Nov 18, 2023
  
  e87557b0
16 Nov, 2023 4 commits

Support Microsoft Phi 1.5 (#1664) · 521b35f7
maximzubkov authored Nov 16, 2023

521b35f7
Fix loading error when safetensors contains empty tensor (#1687) · 2a2c135b
twaka authored Nov 17, 2023

2a2c135b
Revert `MptConfig` to `MPTConfig` (#1668) · b514d3c4
Megha Agarwal authored Nov 16, 2023

b514d3c4

TP/quantization/weight loading refactor part 2 - Refactor quantized linear... · 7076fa1c

Zhuohan Li authored Nov 15, 2023

TP/quantization/weight loading refactor part 2 - Refactor quantized linear logic and extend quantization support to all models (#1622)

Refactor the tensor parallelism, quantization, and weight-loading codes.

Summary of the new features enabled by this PR:
- **All models** are able to be quantized with AWQ and SqueezeLLM, and [soon GPTQ](https://github.com/vllm-project/vllm/pull/1580).
- Model loading code became much simpler.
- Support model parallelism for all MQA/GQA models when the number of key/value heads is smaller than the tensor parallel size.

7076fa1c

13 Nov, 2023 1 commit
- [Minor] Move RoPE selection logic to `get_rope` (#1633) · 054072be
  Woosuk Kwon authored Nov 12, 2023
  
  054072be
12 Nov, 2023 1 commit
- Fix #1474 - AssertionError:assert param_slice.shape == loaded_weight.shape (#1631) · eb825c1e
  lirui authored Nov 13, 2023
  
  eb825c1e
09 Nov, 2023 1 commit
- Add Yi model to quantization support (#1600) · ab9e8488
  forpanyang authored Nov 10, 2023
  
  ab9e8488
07 Nov, 2023 1 commit
- ChatGLM Support (#1261) · 1a2bbc93
  GoHomeToMacDonal authored Nov 07, 2023
  
  1a2bbc93
06 Nov, 2023 1 commit
- Support Yi model (#1567) · e7f579eb
  Roy authored Nov 07, 2023
  
  e7f579eb
03 Nov, 2023 2 commits
- Support YaRN models (#1264) · 9f669a9a
  Antoni Baum authored Nov 03, 2023
```
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
Co-authored-by: Viktor Ferenczi <viktor@ferenczi.eu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  9f669a9a
- Added logits processor API to sampling params (#1469) · 555bdcc5
  Noam Gat authored Nov 03, 2023
  
  555bdcc5
01 Nov, 2023 2 commits
- Force paged attention v2 for long contexts (#1510) · 9738b84a
  Antoni Baum authored Nov 01, 2023
  
  9738b84a
- Remove `MPTConfig` (#1529) · 1fe09900
  Woosuk Kwon authored Nov 01, 2023
  
  1fe09900
31 Oct, 2023 1 commit
- Add `MptForCausalLM` key in model_loader (#1526) · cf8849f2
  Wenfei Yan authored Oct 31, 2023
  
  cf8849f2
30 Oct, 2023 1 commit
- Delay GPU->CPU sync in sampling (#1337) · 15f56323
  Antoni Baum authored Oct 30, 2023
  
  15f56323
29 Oct, 2023 3 commits
- Fix bias in InternLM (#1501) · aa9af07c
  Woosuk Kwon authored Oct 30, 2023
  
  aa9af07c
- Support repetition_penalty (#1424) · 69be658b
  ljss authored Oct 30, 2023
  
  69be658b
- Add rope_scaling to Aquila model (#1457) · 28b47d1e
  Qing authored Oct 29, 2023
  
  28b47d1e