Commits · fe80f5360ca885af4c63a3f1db9f6786df7ebc76 · OpenDAS / text-generation-inference

19 Jul, 2023 1 commit
- feat(server): auto max_batch_total_tokens for flash att models (#630) · fe80f536
  OlivierDehaene authored Jul 19, 2023
  
  fe80f536
30 Jun, 2023 1 commit
- feat(server): add paged attention to flash models (#516) · e74bd41e
  OlivierDehaene authored Jun 30, 2023
```
Closes #478
```
  e74bd41e
23 Jun, 2023 1 commit
- fix(router): add timeout on flume sends (#488) · bd3a9d8e
  OlivierDehaene authored Jun 23, 2023
  
  bd3a9d8e
16 Jun, 2023 1 commit
- feat(router): add ngrok integration (#453) · f59fb8b6
  OlivierDehaene authored Jun 16, 2023
  
  f59fb8b6
02 Jun, 2023 1 commit
- feat(server): only compute prefill logprobs when asked (#406) · 895c5f15
  OlivierDehaene authored Jun 02, 2023
```
Close #288
```
  895c5f15
26 Apr, 2023 2 commits
- feat(router): new healthcheck that skips the queue (#244) · db2b4e07
  Nicolas Patry authored Apr 26, 2023
```
Co-authored-by: OlivierDehaene <23298448+OlivierDehaene@users.noreply.github.com>
Co-authored-by: OlivierDehaene <olivier@huggingface.co>
```
  db2b4e07
- feat(router): add tests to validation (#237) · c4fb09f2
  Nicolas Patry authored Apr 26, 2023
  
  c4fb09f2
24 Apr, 2023 1 commit
- feat(router): use number of tokens in batch as input for dynamic batching (#226) · ebc74d56
  OlivierDehaene authored Apr 24, 2023
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ebc74d56
20 Apr, 2023 1 commit
- feat(router): drop requests when client closes the channel (#202) · 709d8936
  OlivierDehaene authored Apr 20, 2023
  
  709d8936
09 Apr, 2023 1 commit
- feat(router): make router input validation optional (#164) · 99879600
  OlivierDehaene authored Apr 09, 2023
  
  99879600
30 Mar, 2023 1 commit
- feat(benchmark): tui based benchmarking tool (#149) · 610bb1f9
  OlivierDehaene authored Mar 30, 2023
  
  610bb1f9
16 Mar, 2023 1 commit
- fix(server): use server tokenizer as gt (#128) · b49dbf2d
  OlivierDehaene authored Mar 16, 2023
  
  b49dbf2d
09 Mar, 2023 1 commit
- feat: support typical sampling (#114) · 1a2d6825
  OlivierDehaene authored Mar 09, 2023
```
closes #112
```
  1a2d6825
06 Mar, 2023 1 commit
- feat: allow local models (#101) · cd5961b5
  OlivierDehaene authored Mar 06, 2023
```
closes #99
```
  cd5961b5
02 Mar, 2023 1 commit
- feat(server): add logits watermark (#90) · 9b8ea6a6
  OlivierDehaene authored Mar 02, 2023
  
  9b8ea6a6
16 Feb, 2023 1 commit
- feat(router): add prometheus metrics scrape endpoint (#71) · 439fcaf8
  OlivierDehaene authored Feb 16, 2023
  
  439fcaf8
13 Feb, 2023 1 commit
- feat: add distributed tracing (#62) · 9af45414
  OlivierDehaene authored Feb 13, 2023
  
  9af45414
02 Feb, 2023 1 commit
- feat(router): use background task to manage request queue (#52) · 7b870e1e
  OlivierDehaene authored Feb 02, 2023
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  7b870e1e