GitLab

closes #268

remaining_decode_tokens was calculated twice in Seq2SeqLMBatch.filter()

Introduced in #214 

Fixes #249

Co-authored-by: Nick Hill <nickhill@us.ibm.com>

@njhill, @yk FYI

generated_text was concatenated to the user prompt for legacy reason. We
want to remove this behaviour as we don't think it is useful and even
detrimonial to usability.

We also remove the unused Vec.

Reverts huggingface/text-generation-inference#36

Add token streaming using ServerSideEvents (SSE).

The signature of the SSE events is: 

```rust
struct Details {
    finish_reason: String,
    generated_tokens: u32,
    seed: Option<u64>,
}

struct StreamResponse {
    token: Token,
    generated_text: Option<String>,
    details: Option<Details>,
}

struct ErrorResponse {
    error: String,
}
```

Co-authored-by: Yannic Kilcher <yk@users.noreply.github.com>

- Fix some type hints, in particular base tokenizer class
- Make use of `tensor.new_zero/empty` methods
- Simplify env var string parsing in launcher

Fixes #12 in the easiest way I could think of.

AFAIK there is no torch device type called "gpu".