Unverified Commit 9709ab11 authored by Maria Khalusova's avatar Maria Khalusova Committed by GitHub
Browse files

[docs] last hidden state vs hidden_states[-1] (#26142)

* last hidden state clarification

* feedback addressed
parent e52f1cb6
...@@ -40,6 +40,13 @@ an optional `attentions` attribute. Here we have the `loss` since we passed alon ...@@ -40,6 +40,13 @@ an optional `attentions` attribute. Here we have the `loss` since we passed alon
`hidden_states` and `attentions` because we didn't pass `output_hidden_states=True` or `hidden_states` and `attentions` because we didn't pass `output_hidden_states=True` or
`output_attentions=True`. `output_attentions=True`.
<Tip>
When passing `output_hidden_states=True` you may expect the `outputs.hidden_states[-1]` to match `outputs.last_hidden_states` exactly.
However, this is not always the case. Some models apply normalization or subsequent process to the last hidden state when it's returned.
</Tip>
You can access each attribute as you would usually do, and if that attribute has not been returned by the model, you You can access each attribute as you would usually do, and if that attribute has not been returned by the model, you
will get `None`. Here for instance `outputs.loss` is the loss computed by the model, and `outputs.attentions` is will get `None`. Here for instance `outputs.loss` is the loss computed by the model, and `outputs.attentions` is
`None`. `None`.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment