Commits · e2502822138dfddefccde6e6a39bce22720c8cfc · OpenDAS / text-generation-inference

09 May, 2023 1 commit
- feat(docker): add benchmarking tool to docker image (#298) · e2502822
  OlivierDehaene authored May 09, 2023
  
  e2502822
04 May, 2023 1 commit
- feat(launcher): Improve error message when download process fails. (#276) · e68509ad
  Nicolas Patry authored May 04, 2023
  
  e68509ad
03 May, 2023 2 commits
- fix(launcher): pass weights cache override to the download process (#274) · b67908e0
  OlivierDehaene authored May 03, 2023
```
closes #273
```
  b67908e0
- feat(server): support hf endpoint weight layout (#266) · 85aa7e2e
  OlivierDehaene authored May 03, 2023
  
  85aa7e2e
02 May, 2023 1 commit
- chore(github): add templates (#264) · 411b0d4e
  Nicolas Patry authored May 02, 2023
  
  411b0d4e
29 Apr, 2023 1 commit
- doc(launcher): add more docs to the `launcher` itself and link in the README (#257) · b0b97fd9
  Nicolas Patry authored Apr 29, 2023
  
  b0b97fd9
26 Apr, 2023 2 commits
- feat(router): new healthcheck that skips the queue (#244) · db2b4e07
  Nicolas Patry authored Apr 26, 2023
```
Co-authored-by: OlivierDehaene <23298448+OlivierDehaene@users.noreply.github.com>
Co-authored-by: OlivierDehaene <olivier@huggingface.co>
```
  db2b4e07
- chore(launcher): refactor logic (#242) · 77758f60
  Nicolas Patry authored Apr 26, 2023
```
Hopefully it's cleaner
```
  77758f60
24 Apr, 2023 1 commit
- feat(router): use number of tokens in batch as input for dynamic batching (#226) · ebc74d56
  OlivierDehaene authored Apr 24, 2023
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ebc74d56
21 Apr, 2023 1 commit
- v0.6.0 (#222) · 6ded76a4
  OlivierDehaene authored Apr 21, 2023
  
  6ded76a4
19 Apr, 2023 1 commit
- fix(router): add auth token to get model info (#207) · 252f42c1
  OlivierDehaene authored Apr 19, 2023
  
  252f42c1
18 Apr, 2023 1 commit
- feat(router): add info route (#196) · 2475aede
  OlivierDehaene authored Apr 18, 2023
```
close #125
```
  2475aede
16 Apr, 2023 1 commit
- fix(docker): fix docker image dependencies (#187) · 7a1ba585
  OlivierDehaene authored Apr 17, 2023
  
  7a1ba585
13 Apr, 2023 1 commit
- fix(launcher): revert change on shard errors (#173) · e3a63b6f
  OlivierDehaene authored Apr 13, 2023
  
  e3a63b6f
11 Apr, 2023 3 commits
- v0.5.0 (#168) · 6f0f1d70
  OlivierDehaene authored Apr 11, 2023
  
  6f0f1d70
- feat(server): support OPT models (#55) · f26dfd0d
  OlivierDehaene authored Apr 11, 2023
```
OPT models do not all have a `tokenizer.json` file on the hub at the
moment. Can't merge for now.
```
  f26dfd0d
- feat(server): add flash attention llama (#144) · 299217c9
  OlivierDehaene authored Apr 11, 2023
  
  299217c9
09 Apr, 2023 1 commit
- feat(launcher): allow disabling hf_transfer (#161) · e63a21eb
  OlivierDehaene authored Apr 09, 2023
  
  e63a21eb
30 Mar, 2023 2 commits
- v0.4.3 (#152) · fef1a1c3
  OlivierDehaene authored Mar 30, 2023
  
  fef1a1c3
- v0.4.2 (#151) · 84722f3e
  OlivierDehaene authored Mar 30, 2023
  
  84722f3e
26 Mar, 2023 1 commit
- v0.4.1 (#140) · ab5fd8cf
  OlivierDehaene authored Mar 26, 2023
  
  ab5fd8cf
09 Mar, 2023 2 commits
- v0.4.0 (#119) · 411d6247
  OlivierDehaene authored Mar 09, 2023
  
  411d6247
- feat(router): add best_of parameter (#117) · 55bd4fed
  OlivierDehaene authored Mar 09, 2023
  
  55bd4fed
08 Mar, 2023 2 commits
- feat(launcher): default num_shard to CUDA_VISIBLE_DEVICES if possible (#108) · 5fd2dcb5
  OlivierDehaene authored Mar 08, 2023
  
  5fd2dcb5
- feat(launcher): allow parsing num_shard from CUDA_VISIBLE_DEVICES (#107) · 0ac38d33
  OlivierDehaene authored Mar 08, 2023
  
  0ac38d33
06 Mar, 2023 2 commits
- feat: allow local models (#101) · cd5961b5
  OlivierDehaene authored Mar 06, 2023
```
closes #99
```
  cd5961b5
- fix(server): fix generate_stream by forcing tokens to be decoded correctly (#100) · 9b205d33
  OlivierDehaene authored Mar 06, 2023
  
  9b205d33
03 Mar, 2023 2 commits
- v0.3.2 (#97) · 1c19b093
  OlivierDehaene authored Mar 03, 2023
  
  1c19b093
- fix(launcher): add router parameters to launcher (#95) · 240c4187
  OlivierDehaene authored Mar 03, 2023
  
  240c4187
02 Mar, 2023 1 commit
- feat(server): add logits watermark (#90) · 9b8ea6a6
  OlivierDehaene authored Mar 02, 2023
  
  9b8ea6a6
24 Feb, 2023 2 commits
- feat(server): add special token bool (#85) · 0ac184ce
  OlivierDehaene authored Feb 24, 2023
  
  0ac184ce
- v0.3.1 (#84) · 4b1c9720
  OlivierDehaene authored Feb 24, 2023
  
  4b1c9720
18 Feb, 2023 1 commit
- feat(server): enable hf-transfer (#76) · 17bc841b
  OlivierDehaene authored Feb 18, 2023
  
  17bc841b
17 Feb, 2023 1 commit
- feat(router): add cors allow origin options (#73) · 6796d38c
  OlivierDehaene authored Feb 17, 2023
  
  6796d38c
16 Feb, 2023 2 commits
- v0.3.0 (#72) · c720555a
  OlivierDehaene authored Feb 16, 2023
  
  c720555a
- fix(launcher): copy current env vars to subprocesses (#70) · 7b3d460d
  OlivierDehaene authored Feb 16, 2023
```
closes #69
```
  7b3d460d
15 Feb, 2023 2 commits
- feat(launcher): add disable_custom_kernels arg (#67) · 68455353
  OlivierDehaene authored Feb 15, 2023
  
  68455353
- feat(server): improve download logging (#66) · c5a4a1fa
  OlivierDehaene authored Feb 15, 2023
  
  c5a4a1fa
14 Feb, 2023 1 commit
- feat: add safetensors conversion (#63) · 0fbc6919
  OlivierDehaene authored Feb 14, 2023
  
  0fbc6919
13 Feb, 2023 1 commit
- feat: add distributed tracing (#62) · 9af45414
  OlivierDehaene authored Feb 13, 2023
  
  9af45414