training.mdx 14.6 KB
Newer Older
Steven Liu's avatar
Steven Liu committed
1
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
Sylvain Gugger's avatar
Sylvain Gugger committed
2
3
4
5
6
7
8
9
10
11
12

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

Steven Liu's avatar
Steven Liu committed
13
# Fine-tune a pretrained model
Sylvain Gugger's avatar
Sylvain Gugger committed
14

15
16
[[open-in-colab]]

Steven Liu's avatar
Steven Liu committed
17
There are significant benefits to using a pretrained model. It reduces computation costs, your carbon footprint, and allows you to use state-of-the-art models without having to train one from scratch. 馃 Transformers provides access to thousands of pretrained models for a wide range of tasks. When you use a pretrained model, you train it on a dataset specific to your task. This is known as fine-tuning, an incredibly powerful training technique. In this tutorial, you will fine-tune a pretrained model with a deep learning framework of your choice:
Sylvain Gugger's avatar
Sylvain Gugger committed
18

Steven Liu's avatar
Steven Liu committed
19
20
21
* Fine-tune a pretrained model with 馃 Transformers [`Trainer`].
* Fine-tune a pretrained model in TensorFlow with Keras.
* Fine-tune a pretrained model in native PyTorch.
Sylvain Gugger's avatar
Sylvain Gugger committed
22
23
24

<a id='data-processing'></a>

Steven Liu's avatar
Steven Liu committed
25
## Prepare a dataset
Sylvain Gugger's avatar
Sylvain Gugger committed
26
27
28

<Youtube id="_BZearw7f0w"/>

Steven Liu's avatar
Steven Liu committed
29
Before you can fine-tune a pretrained model, download a dataset and prepare it for training. The previous tutorial showed you how to process data for training, and now you get an opportunity to put those skills to the test!
Sylvain Gugger's avatar
Sylvain Gugger committed
30

Steven Liu's avatar
Steven Liu committed
31
Begin by loading the [Yelp Reviews](https://huggingface.co/datasets/yelp_review_full) dataset:
Sylvain Gugger's avatar
Sylvain Gugger committed
32

Steven Liu's avatar
Steven Liu committed
33
34
```py
>>> from datasets import load_dataset
Sylvain Gugger's avatar
Sylvain Gugger committed
35

Steven Liu's avatar
Steven Liu committed
36
>>> dataset = load_dataset("yelp_review_full")
37
>>> dataset["train"][100]
Steven Liu's avatar
Steven Liu committed
38
39
{'label': 0,
 'text': 'My expectations for McDonalds are t rarely high. But for one to still fail so spectacularly...that takes something special!\\nThe cashier took my friends\'s order, then promptly ignored me. I had to force myself in front of a cashier who opened his register to wait on the person BEHIND me. I waited over five minutes for a gigantic order that included precisely one kid\'s meal. After watching two people who ordered after me be handed their food, I asked where mine was. The manager started yelling at the cashiers for \\"serving off their orders\\" when they didn\'t have their food. But neither cashier was anywhere near those controls, and the manager was the one serving food to customers and clearing the boards.\\nThe manager was rude when giving me my order. She didn\'t make sure that I had everything ON MY RECEIPT, and never even had the decency to apologize that I felt I was getting poor service.\\nI\'ve eaten at various McDonalds restaurants for over 30 years. I\'ve worked at more than one location. I expect bad days, bad moods, and the occasional mistake. But I have yet to have a decent experience at this store. It will remain a place I avoid unless someone in my party needs to avoid illness from low blood sugar. Perhaps I should go back to the racially biased service of Steak n Shake instead!'}
Sylvain Gugger's avatar
Sylvain Gugger committed
40
41
```

Steven Liu's avatar
Steven Liu committed
42
As you now know, you need a tokenizer to process the text and include a padding and truncation strategy to handle any variable sequence lengths. To process your dataset in one step, use 馃 Datasets [`map`](https://huggingface.co/docs/datasets/process.html#map) method to apply a preprocessing function over the entire dataset:
Sylvain Gugger's avatar
Sylvain Gugger committed
43

Steven Liu's avatar
Steven Liu committed
44
45
```py
>>> from transformers import AutoTokenizer
Sylvain Gugger's avatar
Sylvain Gugger committed
46

Steven Liu's avatar
Steven Liu committed
47
>>> tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
Sylvain Gugger's avatar
Sylvain Gugger committed
48
49


Steven Liu's avatar
Steven Liu committed
50
51
>>> def tokenize_function(examples):
...     return tokenizer(examples["text"], padding="max_length", truncation=True)
Sylvain Gugger's avatar
Sylvain Gugger committed
52
53


Steven Liu's avatar
Steven Liu committed
54
>>> tokenized_datasets = dataset.map(tokenize_function, batched=True)
Sylvain Gugger's avatar
Sylvain Gugger committed
55
56
```

Steven Liu's avatar
Steven Liu committed
57
If you like, you can create a smaller subset of the full dataset to fine-tune on to reduce the time it takes:
Sylvain Gugger's avatar
Sylvain Gugger committed
58

Steven Liu's avatar
Steven Liu committed
59
60
61
```py
>>> small_train_dataset = tokenized_datasets["train"].shuffle(seed=42).select(range(1000))
>>> small_eval_dataset = tokenized_datasets["test"].shuffle(seed=42).select(range(1000))
Sylvain Gugger's avatar
Sylvain Gugger committed
62
63
64
65
```

<a id='trainer'></a>

Steven Liu's avatar
Steven Liu committed
66
## Fine-tune with `Trainer`
Sylvain Gugger's avatar
Sylvain Gugger committed
67
68
69

<Youtube id="nvBXf7s7vTI"/>

Steven Liu's avatar
Steven Liu committed
70
馃 Transformers provides a [`Trainer`] class optimized for training 馃 Transformers models, making it easier to start training without manually writing your own training loop. The [`Trainer`] API supports a wide range of training options and features such as logging, gradient accumulation, and mixed precision.
Sylvain Gugger's avatar
Sylvain Gugger committed
71

Steven Liu's avatar
Steven Liu committed
72
Start by loading your model and specify the number of expected labels. From the Yelp Review [dataset card](https://huggingface.co/datasets/yelp_review_full#data-fields), you know there are five labels:
Sylvain Gugger's avatar
Sylvain Gugger committed
73

Steven Liu's avatar
Steven Liu committed
74
75
```py
>>> from transformers import AutoModelForSequenceClassification
Sylvain Gugger's avatar
Sylvain Gugger committed
76

Steven Liu's avatar
Steven Liu committed
77
>>> model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased", num_labels=5)
Sylvain Gugger's avatar
Sylvain Gugger committed
78
79
```

Steven Liu's avatar
Steven Liu committed
80
<Tip>
Sylvain Gugger's avatar
Sylvain Gugger committed
81

Steven Liu's avatar
Steven Liu committed
82
83
You will see a warning about some of the pretrained weights not being used and some weights being randomly
initialized. Don't worry, this is completely normal! The pretrained head of the BERT model is discarded, and replaced with a randomly initialized classification head. You will fine-tune this new model head on your sequence classification task, transferring the knowledge of the pretrained model to it.
Sylvain Gugger's avatar
Sylvain Gugger committed
84

Steven Liu's avatar
Steven Liu committed
85
</Tip>
Sylvain Gugger's avatar
Sylvain Gugger committed
86

Steven Liu's avatar
Steven Liu committed
87
### Training hyperparameters
Sylvain Gugger's avatar
Sylvain Gugger committed
88

Steven Liu's avatar
Steven Liu committed
89
Next, create a [`TrainingArguments`] class which contains all the hyperparameters you can tune as well as flags for activating different training options. For this tutorial you can start with the default training [hyperparameters](https://huggingface.co/docs/transformers/main_classes/trainer#transformers.TrainingArguments), but feel free to experiment with these to find your optimal settings.
Sylvain Gugger's avatar
Sylvain Gugger committed
90

Steven Liu's avatar
Steven Liu committed
91
Specify where to save the checkpoints from your training:
Sylvain Gugger's avatar
Sylvain Gugger committed
92

Steven Liu's avatar
Steven Liu committed
93
94
```py
>>> from transformers import TrainingArguments
Sylvain Gugger's avatar
Sylvain Gugger committed
95

Steven Liu's avatar
Steven Liu committed
96
>>> training_args = TrainingArguments(output_dir="test_trainer")
Sylvain Gugger's avatar
Sylvain Gugger committed
97
98
```

Steven Liu's avatar
Steven Liu committed
99
### Metrics
Sylvain Gugger's avatar
Sylvain Gugger committed
100

Steven Liu's avatar
Steven Liu committed
101
[`Trainer`] does not automatically evaluate model performance during training. You will need to pass [`Trainer`] a function to compute and report metrics. The 馃 Datasets library provides a simple [`accuracy`](https://huggingface.co/metrics/accuracy) function you can load with the `load_metric` (see this [tutorial](https://huggingface.co/docs/datasets/metrics.html) for more information) function:
Sylvain Gugger's avatar
Sylvain Gugger committed
102

Steven Liu's avatar
Steven Liu committed
103
104
105
```py
>>> import numpy as np
>>> from datasets import load_metric
Sylvain Gugger's avatar
Sylvain Gugger committed
106

Steven Liu's avatar
Steven Liu committed
107
108
>>> metric = load_metric("accuracy")
```
Sylvain Gugger's avatar
Sylvain Gugger committed
109

Steven Liu's avatar
Steven Liu committed
110
Call `compute` on `metric` to calculate the accuracy of your predictions. Before passing your predictions to `compute`, you need to convert the predictions to logits (remember all 馃 Transformers models return logits):
Sylvain Gugger's avatar
Sylvain Gugger committed
111

Steven Liu's avatar
Steven Liu committed
112
113
114
115
116
```py
>>> def compute_metrics(eval_pred):
...     logits, labels = eval_pred
...     predictions = np.argmax(logits, axis=-1)
...     return metric.compute(predictions=predictions, references=labels)
Sylvain Gugger's avatar
Sylvain Gugger committed
117
118
```

Steven Liu's avatar
Steven Liu committed
119
120
121
122
If you'd like to monitor your evaluation metrics during fine-tuning, specify the `evaluation_strategy` parameter in your training arguments to report the evaluation metric at the end of each epoch:

```py
>>> from transformers import TrainingArguments
Sylvain Gugger's avatar
Sylvain Gugger committed
123

Steven Liu's avatar
Steven Liu committed
124
125
>>> training_args = TrainingArguments(output_dir="test_trainer", evaluation_strategy="epoch")
```
Sylvain Gugger's avatar
Sylvain Gugger committed
126

Steven Liu's avatar
Steven Liu committed
127
### Trainer
Sylvain Gugger's avatar
Sylvain Gugger committed
128

Steven Liu's avatar
Steven Liu committed
129
Create a [`Trainer`] object with your model, training arguments, training and test datasets, and evaluation function:
Sylvain Gugger's avatar
Sylvain Gugger committed
130

Steven Liu's avatar
Steven Liu committed
131
132
133
134
135
136
137
138
```py
>>> trainer = Trainer(
...     model=model,
...     args=training_args,
...     train_dataset=small_train_dataset,
...     eval_dataset=small_eval_dataset,
...     compute_metrics=compute_metrics,
... )
Sylvain Gugger's avatar
Sylvain Gugger committed
139
140
```

Steven Liu's avatar
Steven Liu committed
141
Then fine-tune your model by calling [`~transformers.Trainer.train`]:
Sylvain Gugger's avatar
Sylvain Gugger committed
142

Steven Liu's avatar
Steven Liu committed
143
144
145
```py
>>> trainer.train()
```
Sylvain Gugger's avatar
Sylvain Gugger committed
146
147
148

<a id='keras'></a>

Steven Liu's avatar
Steven Liu committed
149
## Fine-tune with Keras
Sylvain Gugger's avatar
Sylvain Gugger committed
150
151
152

<Youtube id="rnTGBy2ax1c"/>

Steven Liu's avatar
Steven Liu committed
153
馃 Transformers models also supports training in TensorFlow with the Keras API. You only need to make a few changes before you can fine-tune.
Sylvain Gugger's avatar
Sylvain Gugger committed
154

Steven Liu's avatar
Steven Liu committed
155
### Convert dataset to TensorFlow format
Sylvain Gugger's avatar
Sylvain Gugger committed
156

Steven Liu's avatar
Steven Liu committed
157
The [`DefaultDataCollator`] assembles tensors into a batch for the model to train on. Make sure you specify `return_tensors` to return TensorFlow tensors:
Sylvain Gugger's avatar
Sylvain Gugger committed
158

Steven Liu's avatar
Steven Liu committed
159
160
```py
>>> from transformers import DefaultDataCollator
Sylvain Gugger's avatar
Sylvain Gugger committed
161

Steven Liu's avatar
Steven Liu committed
162
>>> data_collator = DefaultDataCollator(return_tensors="tf")
Sylvain Gugger's avatar
Sylvain Gugger committed
163
164
```

Steven Liu's avatar
Steven Liu committed
165
166
167
168
169
<Tip>

[`Trainer`] uses [`DataCollatorWithPadding`] by default so you don't need to explicitly specify a data collator.

</Tip>
Sylvain Gugger's avatar
Sylvain Gugger committed
170

Steven Liu's avatar
Steven Liu committed
171
Next, convert the tokenized datasets to TensorFlow datasets with the [`to_tf_dataset`](https://huggingface.co/docs/datasets/package_reference/main_classes.html#datasets.Dataset.to_tf_dataset) method. Specify your inputs in `columns`, and your label in `label_cols`:
Sylvain Gugger's avatar
Sylvain Gugger committed
172

Steven Liu's avatar
Steven Liu committed
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
```py
>>> tf_train_dataset = small_train_dataset.to_tf_dataset(
...     columns=["attention_mask", "input_ids", "token_type_ids"],
...     label_cols=["labels"],
...     shuffle=True,
...     collate_fn=data_collator,
...     batch_size=8,
... )

>>> tf_validation_dataset = small_eval_dataset.to_tf_dataset(
...     columns=["attention_mask", "input_ids", "token_type_ids"],
...     label_cols=["labels"],
...     shuffle=False,
...     collate_fn=data_collator,
...     batch_size=8,
... )
Sylvain Gugger's avatar
Sylvain Gugger committed
189
190
```

Steven Liu's avatar
Steven Liu committed
191
192
193
### Compile and fit

Load a TensorFlow model with the expected number of labels:
Sylvain Gugger's avatar
Sylvain Gugger committed
194

Steven Liu's avatar
Steven Liu committed
195
196
197
```py
>>> import tensorflow as tf
>>> from transformers import TFAutoModelForSequenceClassification
Sylvain Gugger's avatar
Sylvain Gugger committed
198

Steven Liu's avatar
Steven Liu committed
199
>>> model = TFAutoModelForSequenceClassification.from_pretrained("bert-base-cased", num_labels=5)
Sylvain Gugger's avatar
Sylvain Gugger committed
200
201
```

Steven Liu's avatar
Steven Liu committed
202
Then compile and fine-tune your model with [`fit`](https://keras.io/api/models/model_training_apis/) as you would with any other Keras model:
Sylvain Gugger's avatar
Sylvain Gugger committed
203

Steven Liu's avatar
Steven Liu committed
204
205
206
207
208
209
```py
>>> model.compile(
...     optimizer=tf.keras.optimizers.Adam(learning_rate=5e-5),
...     loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
...     metrics=tf.metrics.SparseCategoricalAccuracy(),
... )
Sylvain Gugger's avatar
Sylvain Gugger committed
210

Steven Liu's avatar
Steven Liu committed
211
>>> model.fit(tf_train_dataset, validation_data=tf_validation_dataset, epochs=3)
Sylvain Gugger's avatar
Sylvain Gugger committed
212
213
214
215
```

<a id='pytorch_native'></a>

Steven Liu's avatar
Steven Liu committed
216
## Fine-tune in native PyTorch
Sylvain Gugger's avatar
Sylvain Gugger committed
217
218
219

<Youtube id="Dh9CL8fyG80"/>

Steven Liu's avatar
Steven Liu committed
220
[`Trainer`] takes care of the training loop and allows you to fine-tune a model in a single line of code. For users who prefer to write their own training loop, you can also fine-tune a 馃 Transformers model in native PyTorch.
Sylvain Gugger's avatar
Sylvain Gugger committed
221

Steven Liu's avatar
Steven Liu committed
222
223
224
At this point, you may need to restart your notebook or execute the following code to free some memory:

```py
Sylvain Gugger's avatar
Sylvain Gugger committed
225
226
227
228
229
230
del model
del pytorch_model
del trainer
torch.cuda.empty_cache()
```

Steven Liu's avatar
Steven Liu committed
231
232
233
234
235
236
237
238
239
Next, manually postprocess `tokenized_dataset` to prepare it for training.

1. Remove the `text` column because the model does not accept raw text as an input:

    ```py
    >>> tokenized_datasets = tokenized_datasets.remove_columns(["text"])
    ```

2. Rename the `label` column to `labels` because the model expects the argument to be named `labels`:
Sylvain Gugger's avatar
Sylvain Gugger committed
240

Steven Liu's avatar
Steven Liu committed
241
242
243
    ```py
    >>> tokenized_datasets = tokenized_datasets.rename_column("label", "labels")
    ```
Sylvain Gugger's avatar
Sylvain Gugger committed
244

Steven Liu's avatar
Steven Liu committed
245
3. Set the format of the dataset to return PyTorch tensors instead of lists:
Sylvain Gugger's avatar
Sylvain Gugger committed
246

Steven Liu's avatar
Steven Liu committed
247
248
249
    ```py
    >>> tokenized_datasets.set_format("torch")
    ```
Sylvain Gugger's avatar
Sylvain Gugger committed
250

Steven Liu's avatar
Steven Liu committed
251
252
253
254
255
Then create a smaller subset of the dataset as previously shown to speed up the fine-tuning:

```py
>>> small_train_dataset = tokenized_datasets["train"].shuffle(seed=42).select(range(1000))
>>> small_eval_dataset = tokenized_datasets["test"].shuffle(seed=42).select(range(1000))
Sylvain Gugger's avatar
Sylvain Gugger committed
256
257
```

Steven Liu's avatar
Steven Liu committed
258
259
260
### DataLoader

Create a `DataLoader` for your training and test datasets so you can iterate over batches of data:
Sylvain Gugger's avatar
Sylvain Gugger committed
261

Steven Liu's avatar
Steven Liu committed
262
263
```py
>>> from torch.utils.data import DataLoader
Sylvain Gugger's avatar
Sylvain Gugger committed
264

Steven Liu's avatar
Steven Liu committed
265
266
>>> train_dataloader = DataLoader(small_train_dataset, shuffle=True, batch_size=8)
>>> eval_dataloader = DataLoader(small_eval_dataset, batch_size=8)
Sylvain Gugger's avatar
Sylvain Gugger committed
267
268
```

Steven Liu's avatar
Steven Liu committed
269
Load your model with the number of expected labels:
Sylvain Gugger's avatar
Sylvain Gugger committed
270

Steven Liu's avatar
Steven Liu committed
271
272
```py
>>> from transformers import AutoModelForSequenceClassification
Sylvain Gugger's avatar
Sylvain Gugger committed
273

Steven Liu's avatar
Steven Liu committed
274
>>> model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased", num_labels=5)
Sylvain Gugger's avatar
Sylvain Gugger committed
275
276
```

Steven Liu's avatar
Steven Liu committed
277
### Optimizer and learning rate scheduler
Sylvain Gugger's avatar
Sylvain Gugger committed
278

Steven Liu's avatar
Steven Liu committed
279
Create an optimizer and learning rate scheduler to fine-tune the model. Let's use the [`AdamW`](https://pytorch.org/docs/stable/generated/torch.optim.AdamW.html) optimizer from PyTorch:
Sylvain Gugger's avatar
Sylvain Gugger committed
280

Steven Liu's avatar
Steven Liu committed
281
282
283
284
```py
>>> from torch.optim import AdamW

>>> optimizer = AdamW(model.parameters(), lr=5e-5)
Sylvain Gugger's avatar
Sylvain Gugger committed
285
286
```

Steven Liu's avatar
Steven Liu committed
287
Create the default learning rate scheduler from [`Trainer`]:
Sylvain Gugger's avatar
Sylvain Gugger committed
288

Steven Liu's avatar
Steven Liu committed
289
290
```py
>>> from transformers import get_scheduler
Sylvain Gugger's avatar
Sylvain Gugger committed
291

Steven Liu's avatar
Steven Liu committed
292
293
294
295
296
>>> num_epochs = 3
>>> num_training_steps = num_epochs * len(train_dataloader)
>>> lr_scheduler = get_scheduler(
...     name="linear", optimizer=optimizer, num_warmup_steps=0, num_training_steps=num_training_steps
... )
Sylvain Gugger's avatar
Sylvain Gugger committed
297
298
```

Steven Liu's avatar
Steven Liu committed
299
Lastly, specify `device` to use a GPU if you have access to one. Otherwise, training on a CPU may take several hours instead of a couple of minutes.
Sylvain Gugger's avatar
Sylvain Gugger committed
300

Steven Liu's avatar
Steven Liu committed
301
302
```py
>>> import torch
Sylvain Gugger's avatar
Sylvain Gugger committed
303

Steven Liu's avatar
Steven Liu committed
304
305
>>> device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
>>> model.to(device)
Sylvain Gugger's avatar
Sylvain Gugger committed
306
307
```

Steven Liu's avatar
Steven Liu committed
308
309
310
311
312
313
314
315
316
317
318
<Tip>

Get free access to a cloud GPU if you don't have one with a hosted notebook like [Colaboratory](https://colab.research.google.com/) or [SageMaker StudioLab](https://studiolab.sagemaker.aws/).

</Tip>

Great, now you are ready to train! 馃コ 

### Training loop

To keep track of your training progress, use the [tqdm](https://tqdm.github.io/) library to add a progress bar over the number of training steps:
Sylvain Gugger's avatar
Sylvain Gugger committed
319

Steven Liu's avatar
Steven Liu committed
320
321
```py
>>> from tqdm.auto import tqdm
Sylvain Gugger's avatar
Sylvain Gugger committed
322

Steven Liu's avatar
Steven Liu committed
323
>>> progress_bar = tqdm(range(num_training_steps))
Sylvain Gugger's avatar
Sylvain Gugger committed
324

Steven Liu's avatar
Steven Liu committed
325
326
327
328
329
330
331
>>> model.train()
>>> for epoch in range(num_epochs):
...     for batch in train_dataloader:
...         batch = {k: v.to(device) for k, v in batch.items()}
...         outputs = model(**batch)
...         loss = outputs.loss
...         loss.backward()
Sylvain Gugger's avatar
Sylvain Gugger committed
332

Steven Liu's avatar
Steven Liu committed
333
334
335
336
...         optimizer.step()
...         lr_scheduler.step()
...         optimizer.zero_grad()
...         progress_bar.update(1)
Sylvain Gugger's avatar
Sylvain Gugger committed
337
338
```

Steven Liu's avatar
Steven Liu committed
339
### Metrics
Sylvain Gugger's avatar
Sylvain Gugger committed
340

Steven Liu's avatar
Steven Liu committed
341
Just like how you need to add an evaluation function to [`Trainer`], you need to do the same when you write your own training loop. But instead of calculating and reporting the metric at the end of each epoch, this time you will accumulate all the batches with [`add_batch`](https://huggingface.co/docs/datasets/package_reference/main_classes.html?highlight=add_batch#datasets.Metric.add_batch) and calculate the metric at the very end.
Sylvain Gugger's avatar
Sylvain Gugger committed
342

Steven Liu's avatar
Steven Liu committed
343
344
345
346
347
348
349
```py
>>> metric = load_metric("accuracy")
>>> model.eval()
>>> for batch in eval_dataloader:
...     batch = {k: v.to(device) for k, v in batch.items()}
...     with torch.no_grad():
...         outputs = model(**batch)
Sylvain Gugger's avatar
Sylvain Gugger committed
350

Steven Liu's avatar
Steven Liu committed
351
352
353
...     logits = outputs.logits
...     predictions = torch.argmax(logits, dim=-1)
...     metric.add_batch(predictions=predictions, references=batch["labels"])
Sylvain Gugger's avatar
Sylvain Gugger committed
354

Steven Liu's avatar
Steven Liu committed
355
>>> metric.compute()
Sylvain Gugger's avatar
Sylvain Gugger committed
356
357
358
359
360
361
```

<a id='additional-resources'></a>

## Additional resources

Steven Liu's avatar
Steven Liu committed
362
For more fine-tuning examples, refer to:
Sylvain Gugger's avatar
Sylvain Gugger committed
363

Steven Liu's avatar
Steven Liu committed
364
365
- [馃 Transformers Examples](https://github.com/huggingface/transformers/tree/master/examples) includes scripts
  to train common NLP tasks in PyTorch and TensorFlow.
Sylvain Gugger's avatar
Sylvain Gugger committed
366

Steven Liu's avatar
Steven Liu committed
367
- [馃 Transformers Notebooks](notebooks) contains various notebooks on how to fine-tune a model for specific tasks in PyTorch and TensorFlow.