Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
cdf19c50
Unverified
Commit
cdf19c50
authored
Feb 15, 2022
by
Nicolas Patry
Committed by
GitHub
Feb 15, 2022
Browse files
Re-export `KeyDataset`. (#15645)
* Re-export `KeyDataset`. * Update the docs locations.
parent
28e6155d
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
7 additions
and
4 deletions
+7
-4
docs/source/main_classes/pipelines.mdx
docs/source/main_classes/pipelines.mdx
+2
-2
docs/source/quicktour.mdx
docs/source/quicktour.mdx
+2
-2
src/transformers/pipelines/base.py
src/transformers/pipelines/base.py
+3
-0
No files found.
docs/source/main_classes/pipelines.mdx
View file @
cdf19c50
...
@@ -78,7 +78,7 @@ GPU. If it doesn't don't hesitate to create an issue.
...
@@ -78,7 +78,7 @@ GPU. If it doesn't don't hesitate to create an issue.
```python
```python
import datasets
import datasets
from transformers import pipeline
from transformers import pipeline
from transformers.pipelines.
base
import KeyDataset
from transformers.pipelines.
pt_utils
import KeyDataset
from tqdm.auto import tqdm
from tqdm.auto import tqdm
pipe = pipeline("automatic-speech-recognition", model="facebook/wav2vec2-base-960h", device=0)
pipe = pipeline("automatic-speech-recognition", model="facebook/wav2vec2-base-960h", device=0)
...
@@ -128,7 +128,7 @@ whenever the pipeline uses its streaming ability (so when passing lists or `Data
...
@@ -128,7 +128,7 @@ whenever the pipeline uses its streaming ability (so when passing lists or `Data
```python
```python
from transformers import pipeline
from transformers import pipeline
from transformers.pipelines.
base
import KeyDataset
from transformers.pipelines.
pt_utils
import KeyDataset
import datasets
import datasets
dataset = datasets.load_dataset("imdb", name="plain_text", split="unsupervised")
dataset = datasets.load_dataset("imdb", name="plain_text", split="unsupervised")
...
...
docs/source/quicktour.mdx
View file @
cdf19c50
...
@@ -118,7 +118,7 @@ Next, load a dataset (see the 🤗 Datasets [Quick Start](https://huggingface.co
...
@@ -118,7 +118,7 @@ Next, load a dataset (see the 🤗 Datasets [Quick Start](https://huggingface.co
Now you can iterate over the dataset with the pipeline. `KeyDataset` retrieves the item in the dictionary returned by the dataset:
Now you can iterate over the dataset with the pipeline. `KeyDataset` retrieves the item in the dictionary returned by the dataset:
```py
```py
>>> from transformers.pipelines.
base
import KeyDataset
>>> from transformers.pipelines.
pt_utils
import KeyDataset
>>> from tqdm.auto import tqdm
>>> from tqdm.auto import tqdm
>>> for out in tqdm(speech_recognizer(KeyDataset(dataset, "file"))):
>>> for out in tqdm(speech_recognizer(KeyDataset(dataset, "file"))):
...
@@ -316,4 +316,4 @@ One particularly cool 🤗 Transformers feature is the ability to save a model a
...
@@ -316,4 +316,4 @@ One particularly cool 🤗 Transformers feature is the ability to save a model a
>>> tokenizer = AutoTokenizer.from_pretrained(pt_save_directory)
>>> tokenizer = AutoTokenizer.from_pretrained(pt_save_directory)
>>> tf_model = TFAutoModelForSequenceClassification.from_pretrained(pt_save_directory, from_pt=True)
>>> tf_model = TFAutoModelForSequenceClassification.from_pretrained(pt_save_directory, from_pt=True)
```
```
\ No newline at end of file
src/transformers/pipelines/base.py
View file @
cdf19c50
...
@@ -49,6 +49,9 @@ if is_torch_available():
...
@@ -49,6 +49,9 @@ if is_torch_available():
from
torch.utils.data
import
DataLoader
,
Dataset
from
torch.utils.data
import
DataLoader
,
Dataset
from
..models.auto.modeling_auto
import
AutoModel
from
..models.auto.modeling_auto
import
AutoModel
# Re-export for backward compatibility
from
.pt_utils
import
KeyDataset
else
:
else
:
Dataset
=
None
Dataset
=
None
KeyDataset
=
None
KeyDataset
=
None
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment