Commits · 18712d00d7a420c0c8e774cff96131d151e6878c · OpenDAS / AutoAWQ

28 Oct, 2023 1 commit
- [`core`] Support fp32 / bf16 inference (#121) · 18712d00
  Younes Belkada authored Oct 28, 2023
  
  18712d00
19 Oct, 2023 1 commit
- add comments · bcfdeb38
  twaka authored Oct 17, 2023
  
  bcfdeb38
17 Oct, 2023 1 commit
- gpt_neox · ad45716f
  twaka authored Oct 17, 2023
  
  ad45716f
07 Oct, 2023 1 commit
- Only apply attention mask if seqlen is greater than 1 · e94b7f40
  Casper Hansen authored Oct 07, 2023
  
  e94b7f40
06 Oct, 2023 8 commits
- Bump to v0.1.4 · 0baf5e18
  Casper authored Oct 06, 2023
  
  0baf5e18
- Remove attention sinks · b13e2a85
  Casper Hansen authored Oct 06, 2023
  
  b13e2a85
- Create RoPE module · 69733d2c
  Casper Hansen authored Oct 06, 2023
  
  69733d2c
- Move attention shapes to fused_utils · 306de683
  Casper Hansen authored Oct 06, 2023
  
  306de683
- Create ALiBi module · 428504e4
  Casper Hansen authored Oct 06, 2023
  
  428504e4
- Update comment · 7a3d06d6
  Casper Hansen authored Oct 06, 2023
  
  7a3d06d6
- Use new WindowedCache · e46703d8
  Casper Hansen authored Oct 06, 2023
  
  e46703d8
- Create cache for fused modules · 66b2e233
  Casper Hansen authored Oct 06, 2023
  
  66b2e233
05 Oct, 2023 3 commits
- Remove warning · 661047f1
  Casper authored Oct 05, 2023
  
  661047f1
- Bump to v0.1.3 · a6ecd0db
  Casper authored Oct 05, 2023
  
  a6ecd0db
- Remove past_key_value (save 2GB VRAM) · 40e6952a
  Casper Hansen authored Oct 05, 2023
  
  40e6952a
02 Oct, 2023 4 commits
- Bump to v0.1.2 · eccb8f9c
  Casper Hansen authored Oct 02, 2023
  
  eccb8f9c
- Mistral fused modules · d5bb4ec8
  Casper Hansen authored Oct 02, 2023
  
  d5bb4ec8
- Fix Falcon n_kv_heads parameter · 80eea43e
  Casper Hansen authored Oct 02, 2023
  
  80eea43e
- Fix unexpected keyword · d016c513
  Casper Hansen authored Oct 02, 2023
  
  d016c513
27 Sep, 2023 9 commits
- Bump to v0.1.1 · 6bdcd786
  Casper authored Sep 27, 2023
  
  6bdcd786
- Add exception handling · 1ac3f976
  Casper Hansen authored Sep 27, 2023
  
  1ac3f976
- Mistral support · bd56c301
  Casper Hansen authored Sep 27, 2023
  
  bd56c301
- Fix apply_rotary_emb() to have both tensors on the same device · 841a2313
  s4rduk4r authored Sep 27, 2023
  
  841a2313
- Rely on accelerate.dispatch_model() only · ffaaa259
  s4rduk4r authored Sep 27, 2023
  
  ffaaa259
- Clean code · cba9a28c
  Casper Hansen authored Sep 27, 2023
  
  cba9a28c
- Remove unused imports · 707315a6
  Casper Hansen authored Sep 27, 2023
  
  707315a6
- Move cache initialization back to init · 8c80b3e0
  Casper Hansen authored Sep 27, 2023
  
  8c80b3e0
- Roll cache efficiently and correctly · c2cd6535
  Casper Hansen authored Sep 27, 2023
  
  c2cd6535
26 Sep, 2023 5 commits
- Clean · 91de3bfa
  s4rduk4r authored Sep 27, 2023
  
  91de3bfa
- Offload to cpu · 4f76ecb2
  s4rduk4r authored Sep 27, 2023
  
  4f76ecb2
- Fix edge case · 0392a823
  Casper Hansen authored Sep 26, 2023
  
  0392a823
- Add compute_memory_used_pct function · 3bb4a9f6
  Casper Hansen authored Sep 26, 2023
  
  3bb4a9f6
- Automatically reset/increase cache · 202c6d6a
  Casper Hansen authored Sep 26, 2023
  
  202c6d6a
24 Sep, 2023 1 commit
- Fix typing classes · 4abfefc9
  Vik Paruchuri authored Sep 23, 2023
  
  4abfefc9
21 Sep, 2023 6 commits
- Remove imports · 9060b980
  Casper Hansen authored Sep 21, 2023
  
  9060b980
- Remove fused modules (later) · 37db5759
  Casper Hansen authored Sep 21, 2023
  
  37db5759
- Fix usage of engine · 133dd7a7
  Casper authored Sep 21, 2023
  
  133dd7a7
- Fix module import · fbeea40b
  Casper Hansen authored Sep 21, 2023
  
  fbeea40b
- Bump to v0.1.0 · 872387fc
  Casper authored Sep 21, 2023
  
  872387fc
- Fix argument · 7c48033a
  Casper Hansen authored Sep 21, 2023
  
  7c48033a