"{MODEL_DESCRIPTION}":"<!--- Describe your model here -->",
"{TRAINING_SECTION}":"",
"{USAGE_TRANSFORMERS_SECTION}":"",
"{EVALUATION}":"<!--- Describe how your model was evaluated -->",
"{CITING}":"<!--- Describe where people can find more information -->",
}
__MODEL_CARD__="""
---
library_name: sentence-transformers
pipeline_tag: {PIPELINE_TAG}
tags:
{TAGS}
{DATASETS}
---
# {MODEL_NAME}
This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a {NUM_DIMENSIONS} dimensional dense vector space and can be used for tasks like clustering or semantic search.
{MODEL_DESCRIPTION}
## Usage (Sentence-Transformers)
Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
```
pip install -U sentence-transformers
```
Then you can use the model like this:
```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]
model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)
```
{USAGE_TRANSFORMERS_SECTION}
## Evaluation Results
{EVALUATION}
For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})
{TRAINING_SECTION}
## Full Model Architecture
```
{FULL_MODEL_STR}
```
## Citing & Authors
{CITING}
"""
__TRAINING_SECTION__="""
## Training
The model was trained with the parameters:
{LOSS_FUNCTIONS}
Parameters of the fit()-Method:
```
{FIT_PARAMETERS}
```
"""
__USAGE_TRANSFORMERS__="""\n
## Usage (HuggingFace Transformers)
Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
```python
from transformers import AutoTokenizer, AutoModel
import torch
{POOLING_FUNCTION}
# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']
"""Performs pooling (max or mean) on the token embeddings.
Using pooling, it generates from a variable sized sentence a fixed sized sentence embedding. This layer also allows
to use the CLS token if it is returned by the underlying word embedding model. You can concatenate multiple poolings
together.
:param word_embedding_dimension: Dimensions for the word embeddings
:param pooling_mode: Either "cls", "lasttoken", "max", "mean", "mean_sqrt_len_tokens", or "weightedmean". If set, overwrites the other pooling_mode_* settings
:param pooling_mode_cls_token: Use the first token (CLS token) as text representations
:param pooling_mode_max_tokens: Use max in each dimension over all tokens.
:param pooling_mode_mean_sqrt_len_tokens: Perform mean-pooling, but divide by sqrt(input_length).
:param pooling_mode_weightedmean_tokens: Perform (position) weighted mean pooling. See `SGPT: GPT Sentence Embeddings for Semantic Search <https://arxiv.org/abs/2202.08904>`_.
:param pooling_mode_lasttoken: Perform last token pooling. See `SGPT: GPT Sentence Embeddings for Semantic Search <https://arxiv.org/abs/2202.08904>`_ and `Text and Code Embeddings by Contrastive Pre-Training <https://arxiv.org/abs/2201.10005>`_.
"""
POOLING_MODES=(
"cls",
"lasttoken",
"max",
"mean",
"mean_sqrt_len_tokens",
"weightedmean",
)
def__init__(
self,
word_embedding_dimension:int,
pooling_mode:str=None,
pooling_mode_cls_token:bool=False,
pooling_mode_max_tokens:bool=False,
pooling_mode_mean_tokens:bool=True,
pooling_mode_mean_sqrt_len_tokens:bool=False,
pooling_mode_weightedmean_tokens:bool=False,
pooling_mode_lasttoken:bool=False,
include_prompt=True,
)->None:
super(Pooling,self).__init__()
self.config_keys=[
"word_embedding_dimension",
"pooling_mode_cls_token",
"pooling_mode_mean_tokens",
"pooling_mode_max_tokens",
"pooling_mode_mean_sqrt_len_tokens",
"pooling_mode_weightedmean_tokens",
"pooling_mode_lasttoken",
"include_prompt",
]
ifpooling_modeisnotNone:# Set pooling mode by string
Mapping of tokens to a float weight value. Words embeddings are multiplied by this float value. Tokens in word_weights must not be equal to the vocab (can contain more or less values)
:param unknown_word_weight:
Weight for words in vocab, that do not appear in the word_weights lookup. These can be for example rare words in the vocab, where no weight exists.
"""Tokenizes the text with respect to existent phrases in the vocab.
This tokenizers respects phrases that are in the vocab. Phrases are separated with 'ngram_separator', for example,
in Google News word2vec file, ngrams are separated with a _ like New_York. These phrases are detected in text and merged as one special token. (New York is the ... => [New_York, is, the])
