Commits · ebc74d5666bf2a304181a5fe8cc7dfc6ddbe1d95 · OpenDAS / text-generation-inference

24 Apr, 2023 1 commit
- feat(router): use number of tokens in batch as input for dynamic batching (#226) · ebc74d56
  OlivierDehaene authored Apr 24, 2023
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ebc74d56
21 Apr, 2023 1 commit
- v0.6.0 (#222) · 6ded76a4
  OlivierDehaene authored Apr 21, 2023
  
  6ded76a4
19 Apr, 2023 1 commit
- fix(router): add auth token to get model info (#207) · 252f42c1
  OlivierDehaene authored Apr 19, 2023
  
  252f42c1
18 Apr, 2023 1 commit
- feat(router): add info route (#196) · 2475aede
  OlivierDehaene authored Apr 18, 2023
```
close #125
```
  2475aede
16 Apr, 2023 1 commit
- fix(docker): fix docker image dependencies (#187) · 7a1ba585
  OlivierDehaene authored Apr 17, 2023
  
  7a1ba585
13 Apr, 2023 1 commit
- fix(launcher): revert change on shard errors (#173) · e3a63b6f
  OlivierDehaene authored Apr 13, 2023
  
  e3a63b6f
11 Apr, 2023 3 commits
- v0.5.0 (#168) · 6f0f1d70
  OlivierDehaene authored Apr 11, 2023
  
  6f0f1d70
- feat(server): support OPT models (#55) · f26dfd0d
  OlivierDehaene authored Apr 11, 2023
```
OPT models do not all have a `tokenizer.json` file on the hub at the
moment. Can't merge for now.
```
  f26dfd0d
- feat(server): add flash attention llama (#144) · 299217c9
  OlivierDehaene authored Apr 11, 2023
  
  299217c9
09 Apr, 2023 1 commit
- feat(launcher): allow disabling hf_transfer (#161) · e63a21eb
  OlivierDehaene authored Apr 09, 2023
  
  e63a21eb
30 Mar, 2023 2 commits
- v0.4.3 (#152) · fef1a1c3
  OlivierDehaene authored Mar 30, 2023
  
  fef1a1c3
- v0.4.2 (#151) · 84722f3e
  OlivierDehaene authored Mar 30, 2023
  
  84722f3e
26 Mar, 2023 1 commit
- v0.4.1 (#140) · ab5fd8cf
  OlivierDehaene authored Mar 26, 2023
  
  ab5fd8cf
09 Mar, 2023 2 commits
- v0.4.0 (#119) · 411d6247
  OlivierDehaene authored Mar 09, 2023
  
  411d6247
- feat(router): add best_of parameter (#117) · 55bd4fed
  OlivierDehaene authored Mar 09, 2023
  
  55bd4fed
08 Mar, 2023 2 commits
- feat(launcher): default num_shard to CUDA_VISIBLE_DEVICES if possible (#108) · 5fd2dcb5
  OlivierDehaene authored Mar 08, 2023
  
  5fd2dcb5
- feat(launcher): allow parsing num_shard from CUDA_VISIBLE_DEVICES (#107) · 0ac38d33
  OlivierDehaene authored Mar 08, 2023
  
  0ac38d33
06 Mar, 2023 2 commits
- feat: allow local models (#101) · cd5961b5
  OlivierDehaene authored Mar 06, 2023
```
closes #99
```
  cd5961b5
- fix(server): fix generate_stream by forcing tokens to be decoded correctly (#100) · 9b205d33
  OlivierDehaene authored Mar 06, 2023
  
  9b205d33
03 Mar, 2023 2 commits
- v0.3.2 (#97) · 1c19b093
  OlivierDehaene authored Mar 03, 2023
  
  1c19b093
- fix(launcher): add router parameters to launcher (#95) · 240c4187
  OlivierDehaene authored Mar 03, 2023
  
  240c4187
02 Mar, 2023 1 commit
- feat(server): add logits watermark (#90) · 9b8ea6a6
  OlivierDehaene authored Mar 02, 2023
  
  9b8ea6a6
24 Feb, 2023 2 commits
- feat(server): add special token bool (#85) · 0ac184ce
  OlivierDehaene authored Feb 24, 2023
  
  0ac184ce
- v0.3.1 (#84) · 4b1c9720
  OlivierDehaene authored Feb 24, 2023
  
  4b1c9720
18 Feb, 2023 1 commit
- feat(server): enable hf-transfer (#76) · 17bc841b
  OlivierDehaene authored Feb 18, 2023
  
  17bc841b
17 Feb, 2023 1 commit
- feat(router): add cors allow origin options (#73) · 6796d38c
  OlivierDehaene authored Feb 17, 2023
  
  6796d38c
16 Feb, 2023 2 commits
- v0.3.0 (#72) · c720555a
  OlivierDehaene authored Feb 16, 2023
  
  c720555a
- fix(launcher): copy current env vars to subprocesses (#70) · 7b3d460d
  OlivierDehaene authored Feb 16, 2023
```
closes #69
```
  7b3d460d
15 Feb, 2023 2 commits
- feat(launcher): add disable_custom_kernels arg (#67) · 68455353
  OlivierDehaene authored Feb 15, 2023
  
  68455353
- feat(server): improve download logging (#66) · c5a4a1fa
  OlivierDehaene authored Feb 15, 2023
  
  c5a4a1fa
14 Feb, 2023 1 commit
- feat: add safetensors conversion (#63) · 0fbc6919
  OlivierDehaene authored Feb 14, 2023
  
  0fbc6919
13 Feb, 2023 1 commit
- feat: add distributed tracing (#62) · 9af45414
  OlivierDehaene authored Feb 13, 2023
  
  9af45414
08 Feb, 2023 1 commit
- fix(docker): increase shm size (#60) · 1ad3250b
  OlivierDehaene authored Feb 08, 2023
  
  1ad3250b
07 Feb, 2023 2 commits
- V0.2.1 (#58) · 2fe5e1b3
  OlivierDehaene authored Feb 07, 2023
  
  2fe5e1b3
- fix(server): better handling of inference mode (#57) · 4acc42a6
  OlivierDehaene authored Feb 07, 2023
  
  4acc42a6
03 Feb, 2023 1 commit
- feat(router): refactor API and add openAPI schemas (#53) · 20c3c594
  OlivierDehaene authored Feb 03, 2023
  
  20c3c594
02 Feb, 2023 2 commits

breaking(router): modify /generate API to only return generated text (#50) · b1482d90

OlivierDehaene authored Feb 02, 2023

@njhill, @yk FYI

generated_text was concatenated to the user prompt for legacy reason. We
want to remove this behaviour as we don't think it is useful and even
detrimonial to usability.

We also remove the unused Vec.

b1482d90

feat(router): use background task to manage request queue (#52) · 7b870e1e
OlivierDehaene authored Feb 02, 2023
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
7b870e1e

01 Feb, 2023 1 commit
- feat(server): allow the server to use a local weight cache (#49) · 775115e3
  OlivierDehaene authored Feb 01, 2023
  
  775115e3
31 Jan, 2023 1 commit
- feat(server): Support GPT-Neox (#39) · f830706b
  OlivierDehaene authored Jan 31, 2023
  
  f830706b