Skip to content

GitLab

"...git@developer.sourcefind.cn:chenpangpang/transformers.git" did not exist on "2418c64a1cfe317bb8d238d3670693799276d16d"
Unverified Commit c96aca3a authored by Ankur Singh's avatar Ankur Singh Committed by GitHub
Browse files

Added the necessay import of module (#30804)

parent ccdabc56
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
docs/source/en/training.md
View file @ c96aca3a
...@@ -186,6 +186,7 @@ so we can just convert that directly to a NumPy array without tokenization! ...@@ -186,6 +186,7 @@ so we can just convert that directly to a NumPy array without tokenization!
```py ```py
from transformers import AutoTokenizer from transformers import AutoTokenizer
import numpy as np
tokenizer = AutoTokenizer.from_pretrained("google-bert/bert-base-cased") tokenizer = AutoTokenizer.from_pretrained("google-bert/bert-base-cased")
tokenized_data = tokenizer(dataset["sentence"], return_tensors="np", padding=True) tokenized_data = tokenizer(dataset["sentence"], return_tensors="np", padding=True)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment