Commits · 87fa80c91f5b24a2ee1805b80c3eca8fd6600cd5 · OpenDAS / vllm_cscc

14 Apr, 2024 2 commits
- [BugFix] Fix tensorizer extra in setup.py (#4072) · 563c54f7
  Nick Hill authored Apr 14, 2024
  
  563c54f7
- [Frontend] [Core] feat: Add model loading using `tensorizer` (#3476) · 711a0002
  Sanger Steel authored Apr 13, 2024
  
  711a0002
12 Apr, 2024 1 commit
- [Doc] Add typing hints / mypy types cleanup (#3816) · c2b4a1bc
  Michael Feil authored Apr 11, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  c2b4a1bc
05 Apr, 2024 1 commit
- [Misc] Define common requirements (#3841) · cfaf49a1
  Woosuk Kwon authored Apr 05, 2024
  
  cfaf49a1
04 Apr, 2024 1 commit
- [Core] manage nccl via a pypi package & upgrade to pt 2.2.1 (#3805) · ca81ff51
  youkaichao authored Apr 04, 2024
  
  ca81ff51
02 Apr, 2024 1 commit

[Hardware][Intel] Add CPU inference backend (#3634) · 0e3f06fe

bigPYJ1151 authored Apr 02, 2024


Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Yuan Zhou <yuan.zhou@intel.com>

0e3f06fe

28 Mar, 2024 1 commit
- [CI/Build] update default number of jobs and nvcc threads to avoid overloading the system (#3675) · 3492859b
  youkaichao authored Mar 27, 2024
  
  3492859b
27 Mar, 2024 1 commit
- [Core] remove cupy dependency (#3625) · 8f44facd
  youkaichao authored Mar 27, 2024
  
  8f44facd
25 Mar, 2024 2 commits
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
- [CI/Build] respect the common environment variable MAX_JOBS (#3600) · 42bc3861
  youkaichao authored Mar 24, 2024
  
  42bc3861
21 Mar, 2024 1 commit
- [BugFix] Hot fix in setup.py for neuron build (#3537) · 523e30ea
  Zhuohan Li authored Mar 20, 2024
  
  523e30ea
20 Mar, 2024 1 commit
- Check for _is_cuda() in compute_num_jobs (#3481) · ba8ae1d8
  bnellnm authored Mar 20, 2024
  
  ba8ae1d8
18 Mar, 2024 1 commit
- Cmake based build system (#2830) · 9fdf3de3
  bnellnm authored Mar 18, 2024
  
  9fdf3de3
17 Mar, 2024 1 commit
- [Misc] Use dataclass for InputMetadata (#3452) · abfc4f33
  Woosuk Kwon authored Mar 17, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  abfc4f33
16 Mar, 2024 2 commits
- Fix setup.py neuron-ls issue (#2671) · 6b78837b
  Simon Mo authored Mar 16, 2024
  
  6b78837b
- [Misc] fix line length for entire codebase (#3444) · 8e67598a
  Simon Mo authored Mar 16, 2024
  
  8e67598a
15 Mar, 2024 1 commit
- [Misc] add error message in non linux platform (#3438) · 604f2359
  youkaichao authored Mar 15, 2024
  
  604f2359
13 Mar, 2024 1 commit
- [Minor Fix] Use cupy-cuda11x in CUDA 11.8 build (#3256) · 739c350c
  陈序 authored Mar 14, 2024
  
  739c350c
11 Mar, 2024 1 commit
- Re-enable the 80 char line width limit (#3305) · 2f8844ba
  Zhuohan Li authored Mar 10, 2024
  
  2f8844ba
08 Mar, 2024 1 commit
- [FIX] Make `flash_attn` optional (#3269) · 1cb0cc29
  Woosuk Kwon authored Mar 08, 2024
  
  1cb0cc29
07 Mar, 2024 1 commit
- Separate attention backends (#3005) · 2daf23ab
  Woosuk Kwon authored Mar 07, 2024
  
  2daf23ab
01 Mar, 2024 1 commit

Integrate Marlin Kernels for Int4 GPTQ inference (#2497) · c0c2335c

Robert Shaw authored Mar 01, 2024


Co-authored-by: Robert Shaw <114415538+rib-2@users.noreply.github.com>
Co-authored-by: alexm <alexm@neuralmagic.com>

c0c2335c

29 Feb, 2024 1 commit
- Fix building from source on WSL (#3112) · 2c08ff23
  aliencaocao authored Mar 01, 2024
  
  2c08ff23
26 Feb, 2024 1 commit
- Optimize Triton MoE Kernel (#2979) · cfc15a10
  Philipp Moritz authored Feb 26, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
```
  cfc15a10
20 Feb, 2024 1 commit
- [ROCm] include gfx908 as supported (#2792) · 264017a2
  James Whedbee authored Feb 19, 2024
  
  264017a2
11 Feb, 2024 1 commit
- [ROCm] support Radeon™ 7900 series (gfx1100) without using flash-attention (#2768) · 0580aab0
  Hongxia Yang authored Feb 11, 2024
  
  0580aab0
07 Feb, 2024 1 commit
- Add documentation on how to do incremental builds (#2796) · 931746bc
  Philipp Moritz authored Feb 07, 2024
  
  931746bc
06 Feb, 2024 1 commit
- Add fused top-K softmax kernel for MoE (#2769) · f0d4e145
  Woosuk Kwon authored Feb 05, 2024
  
  f0d4e145
05 Feb, 2024 1 commit
- [ROCm] Fixup arch checks for ROCM (#2627) · 2ccee3de
  Douglas Lehr authored Feb 05, 2024
  
  2ccee3de
30 Jan, 2024 1 commit
- DeepseekMoE support with Fused MoE kernel (#2453) · 5d60def0
  wangding zeng authored Jan 30, 2024
```
Co-authored-by: roy <jasonailu87@gmail.com>
```
  5d60def0
29 Jan, 2024 2 commits
- ROCm: Allow setting compilation target (#2581) · ea8489fc
  Rasmus Larsen authored Jan 29, 2024
  
  ea8489fc
- Support FP8-E5M2 KV Cache (#2279) · 9090bf02
  zhaoyang-star authored Jan 29, 2024
```
Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  9090bf02
27 Jan, 2024 1 commit
- Implement custom all reduce kernels (#2192) · 38017003
  Hanzhi Zhou authored Jan 28, 2024
  
  38017003
26 Jan, 2024 2 commits
- Don't build punica kernels by default (#2605) · 390b495f
  Philipp Moritz authored Jan 26, 2024
  
  390b495f
- [ROCm] add support to ROCm 6.0 and MI300 (#2274) · 6b7de1a0
  Hongxia Yang authored Jan 26, 2024
  
  6b7de1a0
23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa

18 Jan, 2024 1 commit
- [Neuron] Add an option to build with neuron (#2065) · 18473cf4
  Liangfu Chen authored Jan 18, 2024
  
  18473cf4
14 Jan, 2024 1 commit
- [CI] Add Buildkite (#2355) · 6e01e8c1
  Simon Mo authored Jan 14, 2024
  
  6e01e8c1
18 Dec, 2023 1 commit
- [ROCm] Fixes for GPTQ on ROCm (#2180) · 1b7c791d
  kliuae authored Dec 19, 2023
  
  1b7c791d
16 Dec, 2023 1 commit
- [ROCm] Temporarily remove GPTQ ROCm support (#2138) · 2acd76f3
  Woosuk Kwon authored Dec 15, 2023
  
  2acd76f3