Commits · ebc74d5666bf2a304181a5fe8cc7dfc6ddbe1d95 · OpenDAS / text-generation-inference

24 Apr, 2023 1 commit
- feat(router): use number of tokens in batch as input for dynamic batching (#226) · ebc74d56
  OlivierDehaene authored Apr 24, 2023
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ebc74d56
19 Apr, 2023 1 commit
- fix(router): add auth token to get model info (#207) · 252f42c1
  OlivierDehaene authored Apr 19, 2023
  
  252f42c1
18 Apr, 2023 1 commit
- feat(router): add info route (#196) · 2475aede
  OlivierDehaene authored Apr 18, 2023
```
close #125
```
  2475aede
16 Apr, 2023 1 commit
- fix(docker): fix docker image dependencies (#187) · 7a1ba585
  OlivierDehaene authored Apr 17, 2023
  
  7a1ba585
13 Apr, 2023 1 commit
- fix(launcher): revert change on shard errors (#173) · e3a63b6f
  OlivierDehaene authored Apr 13, 2023
  
  e3a63b6f
11 Apr, 2023 1 commit

feat(server): support OPT models (#55) · f26dfd0d

OlivierDehaene authored Apr 11, 2023

OPT models do not all have a `tokenizer.json` file on the hub at the
moment. Can't merge for now.

f26dfd0d

09 Apr, 2023 1 commit
- feat(launcher): allow disabling hf_transfer (#161) · e63a21eb
  OlivierDehaene authored Apr 09, 2023
  
  e63a21eb
09 Mar, 2023 1 commit
- feat(router): add best_of parameter (#117) · 55bd4fed
  OlivierDehaene authored Mar 09, 2023
  
  55bd4fed
08 Mar, 2023 2 commits
- feat(launcher): default num_shard to CUDA_VISIBLE_DEVICES if possible (#108) · 5fd2dcb5
  OlivierDehaene authored Mar 08, 2023
  
  5fd2dcb5
- feat(launcher): allow parsing num_shard from CUDA_VISIBLE_DEVICES (#107) · 0ac38d33
  OlivierDehaene authored Mar 08, 2023
  
  0ac38d33
06 Mar, 2023 1 commit
- feat: allow local models (#101) · cd5961b5
  OlivierDehaene authored Mar 06, 2023
```
closes #99
```
  cd5961b5
03 Mar, 2023 1 commit
- fix(launcher): add router parameters to launcher (#95) · 240c4187
  OlivierDehaene authored Mar 03, 2023
  
  240c4187
02 Mar, 2023 1 commit
- feat(server): add logits watermark (#90) · 9b8ea6a6
  OlivierDehaene authored Mar 02, 2023
  
  9b8ea6a6
18 Feb, 2023 1 commit
- feat(server): enable hf-transfer (#76) · 17bc841b
  OlivierDehaene authored Feb 18, 2023
  
  17bc841b
17 Feb, 2023 1 commit
- feat(router): add cors allow origin options (#73) · 6796d38c
  OlivierDehaene authored Feb 17, 2023
  
  6796d38c
16 Feb, 2023 1 commit
- fix(launcher): copy current env vars to subprocesses (#70) · 7b3d460d
  OlivierDehaene authored Feb 16, 2023
```
closes #69
```
  7b3d460d
15 Feb, 2023 2 commits
- feat(launcher): add disable_custom_kernels arg (#67) · 68455353
  OlivierDehaene authored Feb 15, 2023
  
  68455353
- feat(server): improve download logging (#66) · c5a4a1fa
  OlivierDehaene authored Feb 15, 2023
  
  c5a4a1fa
14 Feb, 2023 1 commit
- feat: add safetensors conversion (#63) · 0fbc6919
  OlivierDehaene authored Feb 14, 2023
  
  0fbc6919
13 Feb, 2023 1 commit
- feat: add distributed tracing (#62) · 9af45414
  OlivierDehaene authored Feb 13, 2023
  
  9af45414
08 Feb, 2023 1 commit
- fix(docker): increase shm size (#60) · 1ad3250b
  OlivierDehaene authored Feb 08, 2023
  
  1ad3250b
07 Feb, 2023 1 commit
- fix(server): better handling of inference mode (#57) · 4acc42a6
  OlivierDehaene authored Feb 07, 2023
  
  4acc42a6
03 Feb, 2023 1 commit
- feat(router): refactor API and add openAPI schemas (#53) · 20c3c594
  OlivierDehaene authored Feb 03, 2023
  
  20c3c594
02 Feb, 2023 1 commit
- feat(router): use background task to manage request queue (#52) · 7b870e1e
  OlivierDehaene authored Feb 02, 2023
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  7b870e1e
01 Feb, 2023 1 commit
- feat(server): allow the server to use a local weight cache (#49) · 775115e3
  OlivierDehaene authored Feb 01, 2023
  
  775115e3
31 Jan, 2023 1 commit
- feat(server): Support GPT-Neox (#39) · f830706b
  OlivierDehaene authored Jan 31, 2023
  
  f830706b
20 Jan, 2023 1 commit
- feat(server): Support SantaCoder (#26) · 15511edc
  OlivierDehaene authored Jan 20, 2023
  
  15511edc
17 Jan, 2023 1 commit

fix(server): Minor refactorization using new_zeros (#24) · e6d3eb5d

Nick Hill authored Jan 17, 2023

- Fix some type hints, in particular base tokenizer class
- Make use of `tensor.new_zero/empty` methods
- Simplify env var string parsing in launcher

e6d3eb5d

05 Jan, 2023 1 commit
- feat(launcher): Log server stdout (#19) · fcc2c5fc
  OlivierDehaene authored Jan 05, 2023
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  fcc2c5fc
07 Nov, 2022 1 commit
- feat(server): Improved doc · 4236e41b
  OlivierDehaene authored Nov 07, 2022
  
  4236e41b
04 Nov, 2022 1 commit
- feat(launcher): Pass CUDA_VISIBLE_DEVICES to the shard · cea6051e
  OlivierDehaene authored Nov 04, 2022
  
  cea6051e
02 Nov, 2022 1 commit
- feat: Use json formatter by default in docker image · b3b7ea0d
  OlivierDehaene authored Nov 02, 2022
  
  b3b7ea0d
28 Oct, 2022 1 commit
- feat(server): Support all AutoModelForCausalLM on a best effort basis · 3cf6368c
  OlivierDehaene authored Oct 28, 2022
  
  3cf6368c
27 Oct, 2022 1 commit
- feat(server): Support bitsandbytes · 09674e6d
  OlivierDehaene authored Oct 27, 2022
  
  09674e6d
22 Oct, 2022 1 commit

feat(server): Use safetensors · c8ce9b25

Nicolas Patry authored Oct 22, 2022


Co-authored-by: OlivierDehaene <23298448+OlivierDehaene@users.noreply.github.com>

c8ce9b25

21 Oct, 2022 1 commit
- feat(router): Add max_waiting_tokens · c8378933
  OlivierDehaene authored Oct 21, 2022
  
  c8378933
20 Oct, 2022 1 commit
- v0.1.0 · f16f2f5a
  Olivier Dehaene authored Oct 18, 2022
  
  f16f2f5a