Commits · c4fb09f2aef0730d88ce7e4ce54eacd7839b48cc · OpenDAS / text-generation-inference

25 Apr, 2023 1 commit
- chore(server): update safetensors version (#235) · 37b64a5c
  OlivierDehaene authored Apr 25, 2023
  
  37b64a5c
24 Apr, 2023 3 commits
- feat(router): use number of tokens in batch as input for dynamic batching (#226) · ebc74d56
  OlivierDehaene authored Apr 24, 2023
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ebc74d56
- chore(server): update huggingface-hub (#227) · 98a3e0d1
  OlivierDehaene authored Apr 24, 2023
  
  98a3e0d1
- feat(server): reduce memory requirement (#214) · 4a7dd408
  Nick Hill authored Apr 24, 2023
  
  4a7dd408
21 Apr, 2023 8 commits
- v0.6.0 (#222) · 6ded76a4
  OlivierDehaene authored Apr 21, 2023
  
  6ded76a4
- fix(server): fix flash batch filtering (#220) · 4b460e72
  OlivierDehaene authored Apr 21, 2023
  
  4b460e72
- fix(server): fix flash causal (#219) · 1ffea36e
  OlivierDehaene authored Apr 21, 2023
  
  1ffea36e
- fix(server): fix flash causal (#218) · 86bca365
  OlivierDehaene authored Apr 21, 2023
  
  86bca365
- fix(server): cleanup new flash past_key_values logic (#217) · afc5b999
  OlivierDehaene authored Apr 21, 2023
  
  afc5b999
- fix(server): fix past key values logic (#216) · db4cb5e4
  OlivierDehaene authored Apr 21, 2023
```
@njhill fyi
```
  db4cb5e4
- feat(router): add device and dtype info (#215) · 343437c7
  OlivierDehaene authored Apr 21, 2023
  
  343437c7
- feat(server): flash attention past key value optimizations (#213) · ac8c0f6f
  Nick Hill authored Apr 21, 2023
  
  ac8c0f6f
20 Apr, 2023 1 commit
- feat(router): drop requests when client closes the channel (#202) · 709d8936
  OlivierDehaene authored Apr 20, 2023
  
  709d8936
19 Apr, 2023 5 commits
- feat(router): add git sha to info route (#208) · b6ee0ec7
  OlivierDehaene authored Apr 19, 2023
  
  b6ee0ec7
- fix(docker): remove unused dependencies (#205) · 6837b2eb
  OlivierDehaene authored Apr 19, 2023
  
  6837b2eb
- fix(server): fix hf_transfer issue with private repos (#203) · 5d27f525
  OlivierDehaene authored Apr 19, 2023
  
  5d27f525
- feat(server): check cuda capability when importing flash models (#201) · a88c54bb
  OlivierDehaene authored Apr 19, 2023
```
close #198
```
  a88c54bb
- feat(server): support quantization for flash models (#200) · e14ae3b5
  OlivierDehaene authored Apr 19, 2023
```
closes #197
```
  e14ae3b5
16 Apr, 2023 1 commit
- fix(docker): fix docker image dependencies (#187) · 7a1ba585
  OlivierDehaene authored Apr 17, 2023
  
  7a1ba585
14 Apr, 2023 1 commit
- fea(dockerfile): better layer caching (#159) · 53ee09c0
  OlivierDehaene authored Apr 14, 2023
  
  53ee09c0
13 Apr, 2023 1 commit
- fix(ci): fix CVE in github-slug-action (#174) · 64347b05
  OlivierDehaene authored Apr 13, 2023
  
  64347b05
12 Apr, 2023 2 commits
- feat(server): support sharded santacoder (#167) · 880a76ee
  OlivierDehaene authored Apr 12, 2023
  
  880a76ee
- feat(server): optimize decode for sane tokenizers (#170) · 5fa8ae04
  OlivierDehaene authored Apr 12, 2023
  
  5fa8ae04
11 Apr, 2023 3 commits
- v0.5.0 (#168) · 6f0f1d70
  OlivierDehaene authored Apr 11, 2023
  
  6f0f1d70
- feat(server): support OPT models (#55) · f26dfd0d
  OlivierDehaene authored Apr 11, 2023
```
OPT models do not all have a `tokenizer.json` file on the hub at the
moment. Can't merge for now.
```
  f26dfd0d
- feat(server): add flash attention llama (#144) · 299217c9
  OlivierDehaene authored Apr 11, 2023
  
  299217c9
09 Apr, 2023 2 commits
- feat(router): make router input validation optional (#164) · 99879600
  OlivierDehaene authored Apr 09, 2023
  
  99879600
- feat(docker): improve flash_attention caching (#160) · 1883d8ec
  OlivierDehaene authored Apr 09, 2023
  
  1883d8ec
05 Apr, 2023 1 commit
- fix(server): fix escape characters in stop sequence (#155) · 3f2542bb
  OlivierDehaene authored Apr 05, 2023
  
  3f2542bb
03 Apr, 2023 1 commit
- feat(server): flash santacoder (#153) · c0aeb325
  OlivierDehaene authored Apr 03, 2023
  
  c0aeb325
30 Mar, 2023 4 commits
- v0.4.3 (#152) · fef1a1c3
  OlivierDehaene authored Mar 30, 2023
  
  fef1a1c3
- v0.4.2 (#151) · 84722f3e
  OlivierDehaene authored Mar 30, 2023
  
  84722f3e
- fix(server): fix flash neox rotary embeddings (#150) · 08b7e4a2
  OlivierDehaene authored Mar 30, 2023
  
  08b7e4a2
- feat(benchmark): tui based benchmarking tool (#149) · 610bb1f9
  OlivierDehaene authored Mar 30, 2023
  
  610bb1f9
28 Mar, 2023 2 commits
- feat(server): reduce mlp and attn in one op for flash neox (#145) · c9bdaa8b
  OlivierDehaene authored Mar 28, 2023
  
  c9bdaa8b
- feat(server): clear cache on error (#143) · f0000689
  OlivierDehaene authored Mar 28, 2023
  
  f0000689
27 Mar, 2023 2 commits
- feat(server): Add mypy-protobuf (#141) · 8e8dd984
  Nick Hill authored Mar 27, 2023
```
Generates .pyi files for protobuf stubs which provide strong typing
information. Very helpful for IDE auto-completion, etc.
```
  8e8dd984
- fix(server): Avoid using try/except to determine kind of AutoModel (#142) · 462530c2
  Nick Hill authored Mar 27, 2023
  
  462530c2
26 Mar, 2023 2 commits
- v0.4.1 (#140) · ab5fd8cf
  OlivierDehaene authored Mar 26, 2023
  
  ab5fd8cf
- feat(server): cleanup flash neox loading (#139) · 678b2f39
  OlivierDehaene authored Mar 26, 2023
  
  678b2f39