This Online Contrastive loss is similar to :class:`ConstrativeLoss`, but it selects hard positive (positives that
are far apart) and hard negative pairs (negatives that are close) and computes the loss only for these pairs.
This loss often yields better performances than ContrastiveLoss.
:param model: SentenceTransformer model
:param distance_metric: Function that returns a distance between two embeddings. The class SiameseDistanceMetric contains pre-defined metrics that can be used
:param margin: Negative samples (label == 0) should have a distance of at least the margin value.
"{MODEL_DESCRIPTION}":"<!--- Describe your model here -->",
"{TRAINING_SECTION}":"",
"{USAGE_TRANSFORMERS_SECTION}":"",
"{EVALUATION}":"<!--- Describe how your model was evaluated -->",
"{CITING}":"<!--- Describe where people can find more information -->",
}
__MODEL_CARD__="""
---
library_name: sentence-transformers
pipeline_tag: {PIPELINE_TAG}
tags:
{TAGS}
{DATASETS}
---
# {MODEL_NAME}
This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a {NUM_DIMENSIONS} dimensional dense vector space and can be used for tasks like clustering or semantic search.
{MODEL_DESCRIPTION}
## Usage (Sentence-Transformers)
Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
```
pip install -U sentence-transformers
```
Then you can use the model like this:
```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]
model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
```
{USAGE_TRANSFORMERS_SECTION}
## Evaluation Results
{EVALUATION}
For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})
{TRAINING_SECTION}
## Full Model Architecture
```
{FULL_MODEL_STR}
```
## Citing & Authors
{CITING}
"""
__TRAINING_SECTION__="""
## Training
The model was trained with the parameters:
{LOSS_FUNCTIONS}
Parameters of the fit()-Method:
```
{FIT_PARAMETERS}
```
"""
__USAGE_TRANSFORMERS__="""\n
## Usage (HuggingFace Transformers)
Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
```python
from transformers import AutoTokenizer, AutoModel
import torch
{POOLING_FUNCTION}
# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']
"""Performs pooling (max or mean) on the token embeddings.
Using pooling, it generates from a variable sized sentence a fixed sized sentence embedding. This layer also allows
to use the CLS token if it is returned by the underlying word embedding model. You can concatenate multiple poolings
together.
:param word_embedding_dimension: Dimensions for the word embeddings
:param pooling_mode: Either "cls", "lasttoken", "max", "mean", "mean_sqrt_len_tokens", or "weightedmean". If set, overwrites the other pooling_mode_* settings
:param pooling_mode_cls_token: Use the first token (CLS token) as text representations
:param pooling_mode_max_tokens: Use max in each dimension over all tokens.
:param pooling_mode_mean_sqrt_len_tokens: Perform mean-pooling, but divide by sqrt(input_length).
:param pooling_mode_weightedmean_tokens: Perform (position) weighted mean pooling. See `SGPT: GPT Sentence Embeddings for Semantic Search <https://arxiv.org/abs/2202.08904>`_.
:param pooling_mode_lasttoken: Perform last token pooling. See `SGPT: GPT Sentence Embeddings for Semantic Search <https://arxiv.org/abs/2202.08904>`_ and `Text and Code Embeddings by Contrastive Pre-Training <https://arxiv.org/abs/2201.10005>`_.
"""
POOLING_MODES=(
"cls",
"lasttoken",
"max",
"mean",
"mean_sqrt_len_tokens",
"weightedmean",
)
def__init__(
self,
word_embedding_dimension:int,
pooling_mode:str=None,
pooling_mode_cls_token:bool=False,
pooling_mode_max_tokens:bool=False,
pooling_mode_mean_tokens:bool=True,
pooling_mode_mean_sqrt_len_tokens:bool=False,
pooling_mode_weightedmean_tokens:bool=False,
pooling_mode_lasttoken:bool=False,
include_prompt=True,
)->None:
super(Pooling,self).__init__()
self.config_keys=[
"word_embedding_dimension",
"pooling_mode_cls_token",
"pooling_mode_mean_tokens",
"pooling_mode_max_tokens",
"pooling_mode_mean_sqrt_len_tokens",
"pooling_mode_weightedmean_tokens",
"pooling_mode_lasttoken",
"include_prompt",
]
ifpooling_modeisnotNone:# Set pooling mode by string
