Commits · 64da65b3225b7a2e6c2b161b726dc9751f973f33 · norm / vllm · GitLab

16 Feb, 2024 1 commit
- Prefix Caching- fix t4 triton error (#2517) · 64da65b3
  shiyi.c_98 authored Feb 16, 2024
  
  64da65b3
15 Feb, 2024 4 commits
- [ROCm] Dockerfile fix for flash-attention build (#2885) · 5255d99d
  Hongxia Yang authored Feb 15, 2024
  
  5255d99d
- Fix DeciLM (#2883) · 4f2ad111
  Philipp Moritz authored Feb 14, 2024
  
  4f2ad111
- [BugFix] Fix GC bug for `LLM` class (#2882) · d7afab6d
  Woosuk Kwon authored Feb 14, 2024
  
  d7afab6d
- Align LoRA code between Mistral and Mixtral (fixes #2875) (#2880) · 31348dff
  Philipp Moritz authored Feb 14, 2024
```
* Fix AttributeError: MixtralModel object has no attribute org_vocab_size.

* Make LoRA logic for Mistral and Mixtral the same

---------
Co-authored-by: Pernekhan Utemuratov <pernekhan@deepinfra.com>
```
  31348dff
14 Feb, 2024 6 commits
- Don't use cupy NCCL for AMD backends (#2855) · 25e86b6a
  Woosuk Kwon authored Feb 14, 2024
  
  25e86b6a
- Migrate AquilaForCausalLM to LlamaForCausalLM (#2867) · 4efbac6d
  Roy authored Feb 15, 2024
  
  4efbac6d
- Fix docker python version (#2845) · 87069ccf
  Nikola Borisov authored Feb 14, 2024
  
  87069ccf
- [Fix] Fix memory profiling when GPU is used by multiple processes (#2863) · 7e45107f
  Woosuk Kwon authored Feb 13, 2024
  
  7e45107f
- Fix internlm after https://github.com/vllm-project/vllm/pull/2860 (#2861) · 0c48b37c
  Philipp Moritz authored Feb 13, 2024
  
  0c48b37c
- Migrate InternLMForCausalLM to LlamaForCausalLM (#2860) · 7eacffd9
  Philipp Moritz authored Feb 13, 2024
```
Co-authored-by: Roy <jasonailu87@gmail.com>
```
  7eacffd9
13 Feb, 2024 7 commits
- Add LoRA support for Mixtral (#2831) · 2a543d6e
  Terry authored Feb 13, 2024
```
* add mixtral lora support

* formatting

* fix incorrectly ported logic

* polish tests

* minor fixes and refactoring

* minor fixes

* formatting

* rename and remove redundant logic

* refactoring

* refactoring

* minor fix

* minor refactoring

* fix code smell
```
  2a543d6e
- Remove Yi model definition, please use `LlamaForCausalLM` instead (#2854) · 317b29de
  Philipp Moritz authored Feb 13, 2024
```
Co-authored-by: Roy <jasonailu87@gmail.com>
```
  317b29de
- Use CuPy for CUDA graphs (#2811) · a463c333
  Woosuk Kwon authored Feb 13, 2024
  
  a463c333
- Revert "Refactor llama family models (#2637)" (#2851) · ea356004
  Philipp Moritz authored Feb 13, 2024
```
This reverts commit 5c976a7e.
```
  ea356004
- Refactor llama family models (#2637) · 5c976a7e
  Roy authored Feb 13, 2024
  
  5c976a7e
- [CI] Ensure documentation build is checked in CI (#2842) · f9644932
  Simon Mo authored Feb 12, 2024
  
  f9644932
- Serving Benchmark Refactoring (#2433) · a4211a4d
  Roger Wang authored Feb 12, 2024
  
  a4211a4d
12 Feb, 2024 2 commits
- Refactor 2 awq gemm kernels into m16nXk32 (#2723) · 56383649
  Rex authored Feb 12, 2024
```
Co-authored-by: Chunan Zeng <chunanzeng@Chunans-Air.attlocal.net>
```
  56383649
- Add documentation section about LoRA (#2834) · 4ca2c358
  Philipp Moritz authored Feb 12, 2024
  
  4ca2c358
11 Feb, 2024 1 commit
- [ROCm] support Radeon™ 7900 series (gfx1100) without using flash-attention (#2768) · 0580aab0
  Hongxia Yang authored Feb 11, 2024
  
  0580aab0
08 Feb, 2024 2 commits
- Disable custom all reduce by default (#2808) · 3711811b
  Woosuk Kwon authored Feb 08, 2024
  
  3711811b
- [Ray] Integration compiled DAG off by default (#2471) · 65b89d16
  SangBin Cho authored Feb 09, 2024
  
  65b89d16
07 Feb, 2024 2 commits
- Add documentation on how to do incremental builds (#2796) · 931746bc
  Philipp Moritz authored Feb 07, 2024
  
  931746bc
- [ROCm] Fix build problem resulted from previous commit related to FP8 kv-cache support (#2790) · c81dddb4
  Hongxia Yang authored Feb 07, 2024
  
  c81dddb4
06 Feb, 2024 3 commits
- [Minor] More fix of test_cache.py CI test failure (#2750) · fe6d09ae
  Lily Liu authored Feb 06, 2024
  
  fe6d09ae
- modelscope: fix issue when model parameter is not a model id but path of the model. (#2489) · ed70c70e
  liuyhwangyh authored Feb 07, 2024
  
  ed70c70e
- Add fused top-K softmax kernel for MoE (#2769) · f0d4e145
  Woosuk Kwon authored Feb 05, 2024
  
  f0d4e145
05 Feb, 2024 5 commits
- [ROCm] Fixup arch checks for ROCM (#2627) · 2ccee3de
  Douglas Lehr authored Feb 05, 2024
  
  2ccee3de
- Set local logging level via env variable (#2774) · b92adec8
  Lukas authored Feb 05, 2024
  
  b92adec8
- [ROCm] Fix some kernels failed unit tests (#2498) · 56f738ae
  Hongxia Yang authored Feb 05, 2024
  
  56f738ae
- [Minor] Fix benchmark_latency script (#2765) · 72d3a30c
  Woosuk Kwon authored Feb 05, 2024
  
  72d3a30c
- Require triton >= 2.1.0 (#2746) · c9b45ade
  whyiug authored Feb 05, 2024
```
Co-authored-by: yangrui1 <yangrui@lanjingren.com>
```
  c9b45ade
04 Feb, 2024 3 commits
- Remove eos tokens from output by default (#2611) · 5a6c81b0
  Rex authored Feb 04, 2024
  
  5a6c81b0
- set&get llm internal tokenizer instead of the TokenizerGroup (#2741) · 51cd22ce
  dancingpipi authored Feb 05, 2024
```
Co-authored-by: shujunhua1 <shujunhua1@jd.com>
```
  51cd22ce
- docs: fix langchain (#2736) · 5ed704ec
  Massimiliano Pronesti authored Feb 04, 2024
  
  5ed704ec
02 Feb, 2024 1 commit
- Add one example to run batch inference distributed on Ray (#2696) · 4abf6336
  Cheng Su authored Feb 02, 2024
  
  4abf6336
01 Feb, 2024 3 commits
- Fix default length_penalty to 1.0 (#2667) · 0e163fce
  zspo authored Feb 02, 2024
  
  0e163fce
- Remove hardcoded `device="cuda" ` to support more devices (#2503) · 96b6f475
  Kunshang Ji authored Feb 02, 2024
```
Co-authored-by: Jiang Li <jiang1.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  96b6f475
- Use revision when downloading the quantization config file (#2697) · c410f5d0
  Pernekhan Utemuratov authored Feb 01, 2024
```
Co-authored-by: Pernekhan Utemuratov <pernekhan@deepinfra.com>
```
  c410f5d0