Commits · b9ae7e5da19573de3d9e727884f6609df04f935d · OpenDAS / text-generation-inference · GitLab

27 Apr, 2023 3 commits
- chore(server): update transformers (#250) · b9ae7e5d
  OlivierDehaene authored Apr 27, 2023
  
  b9ae7e5d
- fix(server): Small tidy of code from recent changes (#251) · 34bca0b8
  Nick Hill authored Apr 27, 2023
```
remaining_decode_tokens was calculated twice in Seq2SeqLMBatch.filter()
```
  34bca0b8
- fix(server): fix reshaping of bloom past_key_values in concatenate() (#252) · b4cf832c
  Nick Hill authored Apr 27, 2023
```
Introduced in #214 

Fixes #249
```
  b4cf832c
26 Apr, 2023 3 commits
- feat(router): new healthcheck that skips the queue (#244) · db2b4e07
  Nicolas Patry authored Apr 26, 2023
```
Co-authored-by: OlivierDehaene <23298448+OlivierDehaene@users.noreply.github.com>
Co-authored-by: OlivierDehaene <olivier@huggingface.co>
```
  db2b4e07
- feat(router): add tests to validation (#237) · c4fb09f2
  Nicolas Patry authored Apr 26, 2023
  
  c4fb09f2
- chore(launcher): refactor logic (#242) · 77758f60
  Nicolas Patry authored Apr 26, 2023
```
Hopefully it's cleaner
```
  77758f60
25 Apr, 2023 5 commits
- fix(benchmarking): fix benchmarking tool · 7de8a377
  OlivierDehaene authored Apr 26, 2023
  
  7de8a377
- Starting some routing tests. (#233) · 45344244
  Nicolas Patry authored Apr 25, 2023
  
  45344244
- fix(python-client): add auth headers to is supported requests (#234) · 323546df
  OlivierDehaene authored Apr 25, 2023
  
  323546df
- chore(server): update safetensors version (#235) · 37b64a5c
  OlivierDehaene authored Apr 25, 2023
  
  37b64a5c
- feat(router): add endpoint info to /info route (#228) · 8b182eb9
  OlivierDehaene authored Apr 25, 2023
  
  8b182eb9
24 Apr, 2023 3 commits
- feat(router): use number of tokens in batch as input for dynamic batching (#226) · ebc74d56
  OlivierDehaene authored Apr 24, 2023
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ebc74d56
- chore(server): update huggingface-hub (#227) · 98a3e0d1
  OlivierDehaene authored Apr 24, 2023
  
  98a3e0d1
- feat(server): reduce memory requirement (#214) · 4a7dd408
  Nick Hill authored Apr 24, 2023
  
  4a7dd408
21 Apr, 2023 9 commits
- v0.6.0 (#222) · 6ded76a4
  OlivierDehaene authored Apr 21, 2023
  
  6ded76a4
- misc: update to rust 1.69 (#221) · 97df0c7b
  OlivierDehaene authored Apr 21, 2023
  
  97df0c7b
- fix(server): fix flash batch filtering (#220) · 4b460e72
  OlivierDehaene authored Apr 21, 2023
  
  4b460e72
- fix(server): fix flash causal (#219) · 1ffea36e
  OlivierDehaene authored Apr 21, 2023
  
  1ffea36e
- fix(server): fix flash causal (#218) · 86bca365
  OlivierDehaene authored Apr 21, 2023
  
  86bca365
- fix(server): cleanup new flash past_key_values logic (#217) · afc5b999
  OlivierDehaene authored Apr 21, 2023
  
  afc5b999
- fix(server): fix past key values logic (#216) · db4cb5e4
  OlivierDehaene authored Apr 21, 2023
```
@njhill fyi
```
  db4cb5e4
- feat(router): add device and dtype info (#215) · 343437c7
  OlivierDehaene authored Apr 21, 2023
  
  343437c7
- feat(server): flash attention past key value optimizations (#213) · ac8c0f6f
  Nick Hill authored Apr 21, 2023
  
  ac8c0f6f
20 Apr, 2023 2 commits
- fix(ci): fix sha in docker image (#212) · 274513e6
  OlivierDehaene authored Apr 20, 2023
  
  274513e6
- feat(router): drop requests when client closes the channel (#202) · 709d8936
  OlivierDehaene authored Apr 20, 2023
  
  709d8936
19 Apr, 2023 6 commits
- feat(router): add git sha to info route (#208) · b6ee0ec7
  OlivierDehaene authored Apr 19, 2023
  
  b6ee0ec7
- fix(router): add auth token to get model info (#207) · 252f42c1
  OlivierDehaene authored Apr 19, 2023
  
  252f42c1
- fix(docker): remove unused dependencies (#205) · 6837b2eb
  OlivierDehaene authored Apr 19, 2023
  
  6837b2eb
- fix(server): fix hf_transfer issue with private repos (#203) · 5d27f525
  OlivierDehaene authored Apr 19, 2023
  
  5d27f525
- feat(server): check cuda capability when importing flash models (#201) · a88c54bb
  OlivierDehaene authored Apr 19, 2023
```
close #198
```
  a88c54bb
- feat(server): support quantization for flash models (#200) · e14ae3b5
  OlivierDehaene authored Apr 19, 2023
```
closes #197
```
  e14ae3b5
18 Apr, 2023 1 commit
- feat(router): add info route (#196) · 2475aede
  OlivierDehaene authored Apr 18, 2023
```
close #125
```
  2475aede
17 Apr, 2023 2 commits
- feat(python-client): get list of currently deployed tgi models using the inference API (#191) · b927244e
  OlivierDehaene authored Apr 17, 2023
  
  b927244e
- fix(router): fix truncation (#190) · c13b9d87
  OlivierDehaene authored Apr 17, 2023
```
closes #189
```
  c13b9d87
16 Apr, 2023 1 commit
- fix(docker): fix docker image dependencies (#187) · 7a1ba585
  OlivierDehaene authored Apr 17, 2023
  
  7a1ba585
14 Apr, 2023 5 commits
- fix(docker): revert dockerfile changes (#186) · 379c5c4d
  OlivierDehaene authored Apr 14, 2023
  
  379c5c4d
- fix(docker): fix image (#185) · f9047562
  OlivierDehaene authored Apr 14, 2023
  
  f9047562
- fix(docker): fix docker image (#184) · 1bb39463
  OlivierDehaene authored Apr 14, 2023
  
  1bb39463
- fix(ci): fix cosign error (#183) · 01c0e368
  OlivierDehaene authored Apr 14, 2023
  
  01c0e368
- fea(dockerfile): better layer caching (#159) · 53ee09c0
  OlivierDehaene authored Apr 14, 2023
  
  53ee09c0