Commits · 06bea8960fa5a85ae9ee5a5eacb8863af977472e · OpenDAS / AutoAWQ

09 Sep, 2023 7 commits
- Add xformers requirement · 06bea896
  Casper Hansen authored Sep 09, 2023
  
  06bea896
- Fuse MPT · f3695d60
  Casper Hansen authored Sep 09, 2023
  
  f3695d60
- Update module name · 5bd6fbc7
  Casper Hansen authored Sep 09, 2023
  
  5bd6fbc7
- Implement ALiBi. · adc5304b
  Casper Hansen authored Sep 09, 2023
  
  adc5304b
- Default to GEMM · 48be2ee2
  Casper Hansen authored Sep 09, 2023
  
  48be2ee2
- Remove warnings about GEMM · c58ec73b
  Casper Hansen authored Sep 09, 2023
  
  c58ec73b
- Create torch SDPA implementation · ebbbc3a5
  Casper Hansen authored Sep 09, 2023
  
  ebbbc3a5
08 Sep, 2023 14 commits
- GEMM + GEMV compatibility · 890b6aa7
  Casper Hansen authored Sep 08, 2023
  
  890b6aa7
- More detailed benchmark · 5297eccc
  Casper Hansen authored Sep 08, 2023
  
  5297eccc
- New benchmark script. · 341c8868
  Casper Hansen authored Sep 08, 2023
  
  341c8868
- Use apply_rotary_emb · 64e6b3e1
  Casper Hansen authored Sep 08, 2023
  
  64e6b3e1
- Update kernel, remove unused code · a11c313a
  Casper Hansen authored Sep 08, 2023
  
  a11c313a
- Attention works · fbfa9d82
  Casper Hansen authored Sep 08, 2023
  
  fbfa9d82
- Fix MLP module · 75710806
  Casper Hansen authored Sep 08, 2023
  
  75710806
- Fix variables and no contiguous memory for GEMV · e197a733
  Casper Hansen authored Sep 08, 2023
  
  e197a733
- Default to GEMV · 331ff953
  Casper Hansen authored Sep 08, 2023
  
  331ff953
- Fix variables · 6534f5e6
  Casper Hansen authored Sep 08, 2023
  
  6534f5e6
- Implement GEMM/GEMV in quantize function and fused modules · 5db43a7f
  Casper Hansen authored Sep 08, 2023
  
  5db43a7f
- Add deprecation warning · 9b2946b6
  Casper Hansen authored Sep 08, 2023
  
  9b2946b6
- Refactor modules, create separate GEMM and GEMV · fe314160
  Casper Hansen authored Sep 08, 2023
  
  fe314160
- New kernels · ef6b60e2
  Casper Hansen authored Sep 08, 2023
  
  ef6b60e2
07 Sep, 2023 2 commits
- Merge pull request #35 from casper-hansen/falcon-180b · 84fb7e98
  Casper authored Sep 07, 2023
```
Support Falcon 180B
```
  84fb7e98
- Support Falcon 180B · 475481b4
  Casper authored Sep 07, 2023
  
  475481b4
06 Sep, 2023 15 commits
- Remove threads · 9304af95
  Casper authored Sep 06, 2023
  
  9304af95
- Bump version · a0fb91d9
  Casper authored Sep 06, 2023
  
  a0fb91d9
- Latest news · d374a2dd
  Casper authored Sep 06, 2023
  
  d374a2dd
- Merge pull request #23 from casper-hansen/yarn · bcf78947
  Casper authored Sep 06, 2023
```
YaRN support for LLaMa models
```
  bcf78947
- Merge pull request #28 from casper-hansen/fix_rotary_emb · 198ba2fb
  Casper authored Sep 06, 2023
```
[BUG] Fix illegal memory access + Quantized Multi-GPU support
```
  198ba2fb
- Remove simple dispatch · 562c0d52
  Casper authored Sep 06, 2023
  
  562c0d52
- Multi-GPU support for quantized models · f8273a0c
  Casper Hansen authored Sep 06, 2023
  
  f8273a0c
- Get correct devices · 7cf0d987
  Casper Hansen authored Sep 06, 2023
  
  7cf0d987
- Correct input sizes · ab7d68e7
  Casper Hansen authored Sep 06, 2023
  
  ab7d68e7
- Switch to model.generate() · a4626828
  Casper Hansen authored Sep 06, 2023
  
  a4626828
- Update shapes for cuda kernel · e71181bd
  Casper Hansen authored Sep 06, 2023
  
  e71181bd
- Update kernel to original · 88964968
  Casper Hansen authored Sep 06, 2023
  
  88964968
- Update rotary embedding kernel · 97af18e4
  Casper Hansen authored Sep 06, 2023
  
  97af18e4
- Add threaded build · 54b63712
  Casper Hansen authored Sep 06, 2023
  
  54b63712
- Use hf_rotary per default · b7290868
  Casper Hansen authored Sep 06, 2023
  
  b7290868
05 Sep, 2023 2 commits
- Merge pull request #26 from casper-hansen/speed_batch_size · 85430ddc
  Casper authored Sep 05, 2023
```
Implement batch size for speed test
```
  85430ddc
- Remove CustomQuantAttention class · 42655843
  Casper Hansen authored Sep 05, 2023
  
  42655843