Commits · 68e9d6ab333715008c542467c8d5202cf4692253 · OpenDAS / text-generation-inference · GitLab

10 May, 2023 1 commit
- feat(server): shard token decode (#303) · 68e9d6ab
  OlivierDehaene authored May 10, 2023
  
  68e9d6ab
09 May, 2023 1 commit
- feat(server): optim flash causal lm decode_token (#285) · ad66f6ef
  OlivierDehaene authored May 09, 2023
  
  ad66f6ef
03 May, 2023 1 commit
- fix(server): fix typo in tokenizers decode (#269) · 4096000e
  OlivierDehaene authored May 03, 2023
```
closes #268
```
  4096000e
24 Apr, 2023 1 commit
- feat(router): use number of tokens in batch as input for dynamic batching (#226) · ebc74d56
  OlivierDehaene authored Apr 24, 2023
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ebc74d56
21 Apr, 2023 7 commits
- fix(server): fix flash batch filtering (#220) · 4b460e72
  OlivierDehaene authored Apr 21, 2023
  
  4b460e72
- fix(server): fix flash causal (#219) · 1ffea36e
  OlivierDehaene authored Apr 21, 2023
  
  1ffea36e
- fix(server): fix flash causal (#218) · 86bca365
  OlivierDehaene authored Apr 21, 2023
  
  86bca365
- fix(server): cleanup new flash past_key_values logic (#217) · afc5b999
  OlivierDehaene authored Apr 21, 2023
  
  afc5b999
- fix(server): fix past key values logic (#216) · db4cb5e4
  OlivierDehaene authored Apr 21, 2023
```
@njhill fyi
```
  db4cb5e4
- feat(router): add device and dtype info (#215) · 343437c7
  OlivierDehaene authored Apr 21, 2023
  
  343437c7
- feat(server): flash attention past key value optimizations (#213) · ac8c0f6f
  Nick Hill authored Apr 21, 2023
  
  ac8c0f6f
20 Apr, 2023 1 commit
- feat(router): drop requests when client closes the channel (#202) · 709d8936
  OlivierDehaene authored Apr 20, 2023
  
  709d8936
19 Apr, 2023 1 commit
- feat(server): support quantization for flash models (#200) · e14ae3b5
  OlivierDehaene authored Apr 19, 2023
```
closes #197
```
  e14ae3b5
12 Apr, 2023 1 commit
- feat(server): optimize decode for sane tokenizers (#170) · 5fa8ae04
  OlivierDehaene authored Apr 12, 2023
  
  5fa8ae04
11 Apr, 2023 1 commit
- feat(server): add flash attention llama (#144) · 299217c9
  OlivierDehaene authored Apr 11, 2023
  
  299217c9
09 Apr, 2023 1 commit
- feat(router): make router input validation optional (#164) · 99879600
  OlivierDehaene authored Apr 09, 2023
  
  99879600
03 Apr, 2023 1 commit
- feat(server): flash santacoder (#153) · c0aeb325
  OlivierDehaene authored Apr 03, 2023
  
  c0aeb325
26 Mar, 2023 1 commit
- feat(server): cleanup flash neox loading (#139) · 678b2f39
  OlivierDehaene authored Mar 26, 2023
  
  678b2f39
24 Mar, 2023 2 commits
- fix(server): fix flash-neox scores warping (#137) · d6a93fe9
  OlivierDehaene authored Mar 24, 2023
  
  d6a93fe9
- feat(server): flash neoX (#133) · 05e9a796
  OlivierDehaene authored Mar 24, 2023
  
  05e9a796