Skip to content

GitLab

Switch branch/tag
  1. 03 Jun, 2024 5 commits
    • Pavithra Devi M's avatar
      Fix Cannot convert [array()] to EagerTensor of dtype int64 (#31109) · f4f69625
      Pavithra Devi M authored 
      While running the model.prepare_tf_dataset() method,
it raises the error below:
```
TypeError: Cannot convert [array([322.,   1.])] to EagerTensor of dtype int64
```

This happens, in  "DataCollatorForSeq2Seq" function when we are try
to convert the labels to tensors. While converting the labels to tensors,
the labels can be in the format of list of list or list of ndarrays.
There is no problem converting the list of list lables. There is a problem
when the list of ndarrays are float values(like below).

```
[array([322.,   1.])]
```

so the exception raises while trying to convert this label to tensors using
below code.

```
batch["labels"] = tf.constant(batch["labels"], dtype=tf.int64)
```

The labels are always integer values, so this got converted to float
values in the label padding operation below.
```
batch["labels"] = [
                    call(label)
                    if padding_side == "right"
    ...
      f4f69625
    • Arthur's avatar
      [`GemmaModel`] fix small typo (#31202) · 1749841a
      Arthur authored 
      * fixes

* fix-copies
      1749841a
    • Ahmed Moubtahij's avatar
      Token healing (#30081) · 39b2ff69
      Ahmed Moubtahij authored 
      

* token healing impl + trie with extensions

* make fixup

* prefix-robust space tokenization

* examples readme and requirements

* make fixup

* allow input prompt and model

* redundant defaults

* Specialized Trie

* make fixup

* updated tests with new inherited Tree

* input ids to auto device_map

* rm unused import

* Update src/transformers/generation/utils.py
Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>

* naming convention

* Revert "naming convention"

This reverts commit dd39d9c5b7a969e2d8a8d2a8e54f121b82dc44f0.

* naming convention

* last -hopefully- changes

---------
Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      39b2ff69
    • amyeroberts's avatar
      Remove copied froms for deprecated models (#31153) · 5b5b48b1
      amyeroberts authored 
      * Remove copied froms for deprecated models

* Remove automatically in script
      5b5b48b1
    • CharlesCNorton's avatar
      Fix typo: use_safetenstors to use_safetensors (#31184) · 97e5a707
      CharlesCNorton authored 
      Corrected a typo in security.md. Changed `use_safetenstors` to `use_safetensors` in the section discussing the usage of safe formats for loading models to prevent arbitrary code execution.
      97e5a707
  3. 31 May, 2024 10 commits
  5. 30 May, 2024 4 commits
  7. 29 May, 2024 12 commits
  9. 28 May, 2024 9 commits