GitLab

closes #268

Co-authored-by: Nick Hill <nickhill@us.ibm.com>

OPT models do not all have a `tokenizer.json` file on the hub at the
moment. Can't merge for now.

closes #105

closes #99

closes #80

Co-authored-by: Yannic Kilcher <yk@users.noreply.github.com>

- Fix some type hints, in particular base tokenizer class
- Make use of `tensor.new_zero/empty` methods
- Simplify env var string parsing in launcher

- Avoid theoretical hang in batcher loop
- Avoid a couple of clones in the router generate method
- Keep attention mask tensors as integers
- Remove num_heads attribute
Co-authored-by: OlivierDehaene <Olivier.dehaene@gmail.com>