first add

docs/img/logo.svg

+<svg id="Capa_1" enable-background="new 0 0 511.819 511.819" height="512" viewBox="0 0 511.819 511.819" width="512" xmlns="http://www.w3.org/2000/svg"><g><g><circle cx="456.893" cy="31.167" fill="#8ac9fe" r="23.673"/></g><g><circle cx="377.415" cy="451.511" fill="#8ac9fe" r="23.673"/></g><g><circle cx="456.893" cy="126.691" fill="#fdb441" r="23.673"/></g><g><circle cx="265.304" cy="480.651" fill="#fdb441" r="23.673"/></g><g><circle cx="446.786" cy="383.551" fill="#fdb441" r="23.673"/></g><g><circle cx="54.926" cy="314.426" fill="#8ac9fe" r="23.673"/></g><g><circle cx="150.45" cy="190.108" fill="#ee6161" r="23.673"/></g><g><circle cx="150.45" cy="314.426" fill="#ee6161" r="23.673"/></g><g><circle cx="381.248" cy="190.108" fill="#ee6161" r="23.673"/></g><g><circle cx="381.248" cy="314.426" fill="#ee6161" r="23.673"/></g><g><circle cx="265.304" cy="117.437" fill="#ee6161" r="23.673"/></g><g><circle cx="265.304" cy="385.127" fill="#ee6161" r="23.673"/></g><g><path d="m480.565 31.164c0 13.08-10.602 23.672-23.672 23.672-10.032 0-18.616-6.245-22.053-15.068 1.239.2 2.508.3 3.797.3 13.08 0 23.682-10.602 23.682-23.672 0-3.028-.57-5.935-1.619-8.593 11.262 1.818 19.865 11.59 19.865 23.361z" fill="#60b7ff"/></g><g><path d="m394.152 468.254c-9.243 9.243-24.231 9.243-33.474 0-2.398-2.398-4.177-5.186-5.326-8.154 7.254 1.179 14.958-1.029 20.554-6.625 6.845-6.845 8.623-16.847 5.326-25.33 4.726.769 9.273 2.978 12.92 6.625 9.243 9.243 9.243 24.231 0 33.484z" fill="#60b7ff"/></g><g><path d="m480.565 126.689c0 13.08-10.602 23.672-23.672 23.672-10.032 0-18.616-6.245-22.053-15.068 1.239.2 2.508.3 3.797.3 13.08 0 23.682-10.602 23.682-23.682 0-3.028-.57-5.925-1.619-8.583 11.262 1.818 19.865 11.591 19.865 23.361z" fill="#fea613"/></g><g><path d="m288.974 480.655c0 13.07-10.592 23.672-23.672 23.672-10.042 0-18.626-6.255-22.053-15.088 1.239.21 2.508.31 3.807.31 13.07 0 23.672-10.602 23.672-23.672 0-3.028-.57-5.925-1.619-8.583 11.271 1.817 19.865 11.58 19.865 23.361z" fill="#fea613"/></g><g><path d="m463.528 400.287c-9.243 9.253-24.241 9.253-33.484 0-2.398-2.398-4.167-5.176-5.316-8.144 7.254 1.179 14.948-1.029 20.544-6.625 6.845-6.845 8.623-16.847 5.326-25.33 4.736.769 9.283 2.978 12.93 6.625 9.243 9.243 9.243 24.231 0 33.474z" fill="#fea613"/></g><g><path d="m78.597 314.424c0 13.08-10.602 23.672-23.672 23.672-10.032 0-18.616-6.245-22.053-15.068 1.239.2 2.508.3 3.797.3 13.08 0 23.682-10.602 23.682-23.672 0-3.028-.57-5.935-1.619-8.593 11.262 1.817 19.865 11.59 19.865 23.361z" fill="#60b7ff"/></g><g><path d="m174.123 190.11c0 13.07-10.602 23.672-23.672 23.672-10.042 0-18.626-6.255-22.053-15.078 1.239.2 2.508.3 3.797.3 13.08 0 23.672-10.592 23.672-23.672 0-3.038-.57-5.935-1.619-8.593 11.262 1.818 19.875 11.58 19.875 23.371z" fill="#e94444"/></g><g><path d="m174.123 314.424c0 13.08-10.602 23.672-23.672 23.672-10.042 0-18.616-6.245-22.053-15.068 1.239.2 2.508.3 3.797.3 13.08 0 23.672-10.602 23.672-23.672 0-3.038-.57-5.935-1.619-8.603 11.262 1.817 19.875 11.59 19.875 23.371z" fill="#e94444"/></g><g><path d="m404.924 190.11c0 13.07-10.602 23.672-23.672 23.672-10.052 0-18.636-6.255-22.063-15.088 1.239.21 2.508.31 3.807.31 13.07 0 23.672-10.592 23.672-23.672 0-3.038-.57-5.935-1.619-8.593 11.261 1.818 19.875 11.58 19.875 23.371z" fill="#e94444"/></g><g><path d="m404.924 314.424c0 13.08-10.602 23.672-23.672 23.672-10.042 0-18.636-6.245-22.063-15.078 1.239.21 2.508.31 3.807.31 13.07 0 23.672-10.602 23.672-23.672 0-3.038-.57-5.935-1.619-8.603 11.261 1.817 19.875 11.59 19.875 23.371z" fill="#e94444"/></g><g><path d="m288.974 117.437c0 13.07-10.592 23.672-23.672 23.672-10.042 0-18.626-6.255-22.053-15.088 1.239.21 2.508.31 3.807.31 13.07 0 23.672-10.592 23.672-23.672 0-3.028-.57-5.925-1.619-8.583 11.271 1.817 19.865 11.59 19.865 23.361z" fill="#e94444"/></g><g><path d="m288.974 385.129c0 13.07-10.592 23.672-23.672 23.672-10.042 0-18.626-6.255-22.053-15.088 1.239.21 2.508.31 3.807.31 13.07 0 23.672-10.592 23.672-23.672 0-3.028-.57-5.925-1.619-8.583 11.271 1.818 19.865 11.58 19.865 23.361z" fill="#e94444"/></g><path d="m464.387 96.44v-35.023c13.577-3.365 23.673-15.648 23.673-30.25 0-17.186-13.981-31.167-31.167-31.167s-31.167 13.981-31.167 31.167c0 14.603 10.096 26.886 23.673 30.25v35.023c-13.577 3.365-23.673 15.648-23.673 30.251 0 4.757 1.075 9.265 2.989 13.302l-30.541 23.964c-4.878-3.168-10.688-5.017-16.926-5.017-5.059 0-9.836 1.219-14.065 3.368l-71.063-40.189c.231-1.528.353-3.092.353-4.684 0-17.186-13.981-31.167-31.167-31.167s-31.167 13.981-31.167 31.167c0 1.592.121 3.156.353 4.684l-23.168 13.102c-3.603 2.037-4.871 6.61-2.834 10.212 1.378 2.438 3.917 3.807 6.53 3.807 1.25 0 2.517-.313 3.683-.972l21.616-12.224c5.688 7.618 14.771 12.559 24.988 12.559s19.3-4.941 24.988-12.559l65.035 36.78c-3.312 4.95-5.247 10.895-5.247 17.284 0 14.603 10.096 26.886 23.673 30.25v63.817c-13.577 3.365-23.673 15.648-23.673 30.25 0 5.776 1.585 11.187 4.335 15.83l-64.123 36.264c-5.688-7.618-14.771-12.559-24.988-12.559s-19.3 4.941-24.988 12.559l-63.309-35.803c2.92-4.744 4.609-10.323 4.609-16.291 0-14.603-10.096-26.886-23.673-30.25v-63.817c13.577-3.365 23.673-15.648 23.673-30.25 0-6.582-2.056-12.69-5.553-17.727l12.502-7.071c3.603-2.038 4.871-6.61 2.834-10.212-2.038-3.604-6.608-4.872-10.213-2.834l-17.243 9.752c-4.085-1.97-8.663-3.075-13.494-3.075-17.186 0-31.167 13.981-31.167 31.167 0 14.603 10.096 26.886 23.673 30.25v63.817c-11.175 2.769-19.987 11.581-22.756 22.756h-35.026c-3.365-13.577-15.648-23.673-30.25-23.673-17.186 0-31.167 13.981-31.167 31.167s13.981 31.167 31.167 31.167c14.602 0 26.886-10.096 30.25-23.673h35.024c3.365 13.577 15.648 23.673 30.25 23.673 5.556 0 10.773-1.468 15.295-4.027l68.745 38.878c-.231 1.528-.353 3.092-.353 4.684 0 14.602 10.096 26.886 23.673 30.25v35.023c-13.577 3.365-23.673 15.648-23.673 30.251 0 17.186 13.981 31.167 31.167 31.167s31.167-13.981 31.167-31.167c0-14.603-10.096-26.886-23.673-30.251v-35.023c13.577-3.365 23.673-15.648 23.673-30.25 0-1.592-.121-3.156-.353-4.684l69.294-39.188c4.644 2.751 10.056 4.337 15.834 4.337 4.94 0 9.613-1.16 13.767-3.215l25.082 25.082c-5.943 9.86-5.944 22.322 0 32.182l-25.721 25.721c-12.073-7.835-28.421-6.469-38.999 4.109-12.152 12.152-12.152 31.924 0 44.077 6.076 6.076 14.057 9.114 22.039 9.114s15.963-3.038 22.039-9.114c10.07-10.071 11.786-25.371 5.167-37.234l26.066-26.066c4.809 2.912 10.334 4.469 16.1 4.469 8.324 0 16.151-3.242 22.038-9.129 2.392-2.392 4.365-5.13 5.864-8.137 1.846-3.704.34-8.204-3.365-10.05-3.709-1.848-8.205-.34-10.051 3.365-.775 1.556-1.8 2.977-3.047 4.224-3.055 3.056-7.118 4.739-11.439 4.739-4.322 0-8.385-1.683-11.44-4.739-6.307-6.306-6.308-16.567-.005-22.876.002-.002.004-.003.005-.005s.003-.004.005-.005c5.29-5.282 13.437-6.27 19.815-2.4 3.537 2.147 8.147 1.019 10.295-2.52 2.147-3.538 1.019-8.147-2.52-10.294-10.104-6.13-22.496-5.931-32.246-.029l-24.197-24.197c3.717-5.132 5.917-11.431 5.917-18.238 0-14.603-10.096-26.886-23.673-30.25v-63.817c13.577-3.365 23.673-15.648 23.673-30.25 0-5.505-1.44-10.677-3.955-15.17l29.733-23.33c5.213 3.922 11.689 6.25 18.7 6.25 17.186 0 31.167-13.981 31.167-31.167 0-14.603-10.096-26.887-23.673-30.252zm-199.083 37.175c-8.921 0-16.179-7.258-16.179-16.179s7.258-16.179 16.179-16.179 16.179 7.258 16.179 16.179-7.258 16.179-16.179 16.179zm-131.033 56.493c0-8.921 7.258-16.179 16.179-16.179s16.179 7.258 16.179 16.179-7.258 16.179-16.179 16.179-16.179-7.258-16.179-16.179zm-79.345 140.496c-8.921 0-16.179-7.258-16.179-16.179s7.258-16.179 16.179-16.179 16.179 7.258 16.179 16.179-7.258 16.179-16.179 16.179zm79.345-16.178c0-8.921 7.258-16.179 16.179-16.179s16.179 7.258 16.179 16.179c0 8.765-7.409 16.179-16.179 16.179-8.921-.001-16.179-7.259-16.179-16.179zm147.212 166.226c0 8.921-7.258 16.179-16.179 16.179s-16.179-7.258-16.179-16.179 7.258-16.179 16.179-16.179 16.179 7.257 16.179 16.179zm-16.179-79.346c-8.921 0-16.179-7.258-16.179-16.179s7.258-16.179 16.179-16.179 16.179 7.258 16.179 16.179-7.258 16.179-16.179 16.179zm123.551 61.645c-6.306 6.307-16.57 6.31-22.88 0-6.309-6.309-6.309-16.573 0-22.881 3.154-3.154 7.297-4.731 11.439-4.731 4.143 0 8.286 1.577 11.44 4.731 6.31 6.309 6.31 16.573.001 22.881zm51.859-431.784c0-8.921 7.258-16.179 16.179-16.179s16.179 7.258 16.179 16.179-7.258 16.179-16.179 16.179-16.179-7.258-16.179-16.179zm-43.288 283.259c0 8.694-7.456 16.179-16.179 16.179-8.921 0-16.178-7.258-16.178-16.179s7.258-16.179 16.179-16.179c8.92 0 16.178 7.258 16.178 16.179zm-16.179-108.139c-8.921 0-16.179-7.258-16.179-16.179s7.258-16.179 16.179-16.179 16.179 7.258 16.179 16.179-7.258 16.179-16.179 16.179zm75.646-63.417c-8.921 0-16.179-7.258-16.179-16.179s7.258-16.179 16.179-16.179 16.179 7.258 16.179 16.179-7.258 16.179-16.179 16.179z"/></g></svg>
\ No newline at end of file

docs/installation.md

+# Installation
+
+We recommend **Python 3.8** or higher, **[PyTorch 1.11.0](https://pytorch.org/get-started/locally/)** or higher and **[transformers v4.32.0](https://github.com/huggingface/transformers)** or higher.
+
+## Install SentenceTransformers
+
+**Install with pip**
+
+Install the *sentence-transformers* with `pip`:
+```
+pip install -U sentence-transformers
+```
+
+**Install with conda**
+
+Apple silicon Installation of *sentence-transformers*
+```
+conda install -c conda-forge sentence-transformers
+```
+
+**Install from source**
+
+Alternatively, you can also clone the latest version from the [repository](https://github.com/UKPLab/sentence-transformers) and install it directly from the source code:
+````
+pip install -e .
+```` 
+
+## Install PyTorch with CUDA support
+
+If you want to use a GPU / CUDA, you must install PyTorch with the matching CUDA Version. Follow
+[PyTorch - Get Started](https://pytorch.org/get-started/locally/) for further details how to install PyTorch.

docs/package_reference/SentenceTransformer.md

+# SentenceTransformer
+
+This page documents the properties and methods when you load a SentenceTransformer model:
+```python
+from sentence_transformers import SentenceTransformer
+
+model = SentenceTransformer("model-name")
+```
+
+```eval_rst
+.. autoclass:: sentence_transformers.SentenceTransformer
+   :members:
+   :exclude-members: save_to_hub
+
+```
\ No newline at end of file

docs/package_reference/cross_encoder.md

+# cross_encoder
+For an introduction to Cross-Encoders, see [Cross-Encoders](../../examples/applications/cross-encoder/README.md).
+```eval_rst
+.. autoclass:: sentence_transformers.cross_encoder.CrossEncoder
+   :members:
+```
+
+
+## Evaluation
+CrossEncoder have their own evaluation classes, that are in `sentence_transformers.cross_encoder.evaluation`.
+
+```eval_rst
+.. autoclass:: sentence_transformers.cross_encoder.evaluation.CEBinaryAccuracyEvaluator
+.. autoclass:: sentence_transformers.cross_encoder.evaluation.CEBinaryClassificationEvaluator
+.. autoclass:: sentence_transformers.cross_encoder.evaluation.CECorrelationEvaluator
+.. autoclass:: sentence_transformers.cross_encoder.evaluation.CEF1Evaluator
+.. autoclass:: sentence_transformers.cross_encoder.evaluation.CESoftmaxAccuracyEvaluator
+.. autoclass:: sentence_transformers.cross_encoder.evaluation.CERerankingEvaluator
+```
\ No newline at end of file

docs/package_reference/datasets.md

+# Datasets
+`sentence_transformers.datasets` contains classes to organize your training input examples.
+
+
+
+## ParallelSentencesDataset
+`ParallelSentencesDataset` is used for multilingual training. For details, see [multilingual training](../../examples/training/multilingual/README.md).
+```eval_rst
+.. autoclass:: sentence_transformers.datasets.ParallelSentencesDataset
+```
+
+
+## SentenceLabelDataset
+`SentenceLabelDataset` can be used if you have labeled sentences and want to train with triplet loss.
+```eval_rst
+.. autoclass:: sentence_transformers.datasets.SentenceLabelDataset
+```
+
+## DenoisingAutoEncoderDataset
+`DenoisingAutoEncoderDataset` is used for unsupervised training with the TSDAE method.
+```eval_rst
+.. autoclass:: sentence_transformers.datasets.DenoisingAutoEncoderDataset
+```
+
+## NoDuplicatesDataLoader
+`NoDuplicatesDataLoader`can be used together with MultipleNegativeRankingLoss to ensure that no duplicates are within the same batch.
+```eval_rst
+.. autoclass:: sentence_transformers.datasets.NoDuplicatesDataLoader
+```
+
+

docs/package_reference/evaluation.md

+# Evaluation
+`sentence_transformers.evaluation` defines different classes, that can be used to evaluate the model during training.
+
+```eval_rst
+.. autoclass:: sentence_transformers.evaluation.BinaryClassificationEvaluator
+.. autoclass:: sentence_transformers.evaluation.EmbeddingSimilarityEvaluator
+.. autoclass:: sentence_transformers.evaluation.InformationRetrievalEvaluator
+.. autoclass:: sentence_transformers.evaluation.LabelAccuracyEvaluator
+.. autoclass:: sentence_transformers.evaluation.MSEEvaluator
+.. autoclass:: sentence_transformers.evaluation.MSEEvaluatorFromDataFrame
+.. autoclass:: sentence_transformers.evaluation.ParaphraseMiningEvaluator
+.. autoclass:: sentence_transformers.evaluation.RerankingEvaluator
+.. autoclass:: sentence_transformers.evaluation.SentenceEvaluator
+.. autoclass:: sentence_transformers.evaluation.SequentialEvaluator
+.. autoclass:: sentence_transformers.evaluation.TranslationEvaluator
+.. autoclass:: sentence_transformers.evaluation.TripletEvaluator
+```

docs/package_reference/losses.md

+# Losses
+`sentence_transformers.losses` defines different loss functions that can be used to fine-tune embedding models on training data. The choice of loss function plays a critical role when fine-tuning the model. It determines how well our embedding model will work for the specific downstream task.
+
+Sadly, there is no "one size fits all" loss function. Which loss function is suitable depends on the available training data and on the target task. Consider checking out the [Loss Overview](../training/loss_overview.html) to help narrow down your choice of loss function(s).
+
+## BatchAllTripletLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.BatchAllTripletLoss
+```
+
+## BatchHardSoftMarginTripletLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.BatchHardSoftMarginTripletLoss
+```
+
+## BatchHardTripletLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.BatchHardTripletLoss
+```
+
+## BatchSemiHardTripletLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.BatchSemiHardTripletLoss
+```
+
+## ContrastiveLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.ContrastiveLoss
+```
+
+## OnlineContrastiveLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.OnlineContrastiveLoss
+```
+
+## ContrastiveTensionLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.ContrastiveTensionLoss
+```
+
+## ContrastiveTensionLossInBatchNegatives
+```eval_rst
+.. autoclass:: sentence_transformers.losses.ContrastiveTensionLossInBatchNegatives
+```
+
+## CoSENTLoss
+
+```eval_rst
+.. autoclass:: sentence_transformers.losses.CoSENTLoss
+```
+
+## AnglELoss
+
+```eval_rst
+.. autoclass:: sentence_transformers.losses.AnglELoss
+```
+
+## CosineSimilarityLoss
+
+![SBERT Siamese Network Architecture](../img/SBERT_Siamese_Network.png "SBERT Siamese Architecture")
+
+
+For each sentence pair, we pass sentence A and sentence B through our network which yields the embeddings *u* und *v*. The similarity of these embeddings is computed using cosine similarity and the result is compared to the gold similarity score. 
+
+This allows our network to be fine-tuned to recognize the similarity of sentences.
+
+
+```eval_rst
+.. autoclass:: sentence_transformers.losses.CosineSimilarityLoss
+```
+
+## DenoisingAutoEncoderLoss
+
+```eval_rst
+.. autoclass:: sentence_transformers.losses.DenoisingAutoEncoderLoss
+```
+
+## GISTEmbedLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.GISTEmbedLoss
+```
+
+## MSELoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.MSELoss
+```
+
+## MarginMSELoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.MarginMSELoss
+```
+
+## MatryoshkaLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.MatryoshkaLoss
+```
+
+## Matryoshka2dLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.Matryoshka2dLoss
+```
+
+## AdaptiveLayerLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.AdaptiveLayerLoss
+```
+
+## MegaBatchMarginLoss
+
+```eval_rst
+.. autoclass:: sentence_transformers.losses.MegaBatchMarginLoss
+```
+
+## MultipleNegativesRankingLoss
+
+*MultipleNegativesRankingLoss* is a great loss function if you only have positive pairs, for example, only pairs of similar texts like pairs of paraphrases, pairs of duplicate questions, pairs of (query, response), or pairs of (source_language, target_language).
+
+```eval_rst
+.. autoclass:: sentence_transformers.losses.MultipleNegativesRankingLoss
+```
+
+## CachedMultipleNegativesRankingLoss
+
+```eval_rst
+.. autoclass:: sentence_transformers.losses.CachedMultipleNegativesRankingLoss
+```
+
+## MultipleNegativesSymmetricRankingLoss
+
+```eval_rst
+.. autoclass:: sentence_transformers.losses.MultipleNegativesSymmetricRankingLoss
+```
+
+## SoftmaxLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.SoftmaxLoss
+```
+
+## TripletLoss
+```eval_rst
+.. autoclass:: sentence_transformers.losses.TripletLoss
+```

docs/package_reference/models.md

+# Models
+`sentence_transformers.models` defines different building blocks, that can be used to create SentenceTransformer networks from scratch. For more details, see [Training Overview](../training/overview.md).
+
+## Main Classes
+```eval_rst
+.. autoclass:: sentence_transformers.models.Transformer
+.. autoclass:: sentence_transformers.models.Pooling
+.. autoclass:: sentence_transformers.models.Dense
+```
+
+## Further Classes
+```eval_rst
+.. autoclass:: sentence_transformers.models.Asym
+.. autoclass:: sentence_transformers.models.BoW
+.. autoclass:: sentence_transformers.models.CNN
+.. autoclass:: sentence_transformers.models.LSTM
+.. autoclass:: sentence_transformers.models.Normalize
+.. autoclass:: sentence_transformers.models.WeightedLayerPooling
+.. autoclass:: sentence_transformers.models.WordEmbeddings
+.. autoclass:: sentence_transformers.models.WordWeights
+```

docs/package_reference/quantization.md

+# quantization
+`sentence_transformers.quantization` defines different helpful functions to quantize.
+
+```eval_rst
+.. automodule:: sentence_transformers.quantization
+   :members: quantize_embeddings, semantic_search_faiss, semantic_search_usearch
+```

docs/package_reference/util.md

+# util
+`sentence_transformers.util` defines different helpful functions to work with text embeddings.
+
+```eval_rst
+.. automodule:: sentence_transformers.util
+   :members: cos_sim, dot_score, paraphrase_mining, semantic_search, community_detection, http_get, truncate_embeddings
+```

docs/pretrained-models/ce-msmarco.md

+# MS MARCO Cross-Encoders
+
+[MS MARCO](https://microsoft.github.io/msmarco/) is a large scale information retrieval corpus that was created based on real user search queries using Bing search engine. The provided models can be used for semantic search, i.e., given keywords / a search phrase / a question, the model will find passages that are relevant for the search query.
+
+The training data consists of over 500k examples, while the complete corpus consists of over 8.8 million passages.
+
+## Usage with SentenceTransformers
+Pre-trained models can be used like this:
+```python
+from sentence_transformers import CrossEncoder
+
+model = CrossEncoder("model_name", max_length=512)
+scores = model.predict(
+    [("Query", "Paragraph1"), ("Query", "Paragraph2"), ("Query", "Paragraph3")]
+)
+```
+
+## Usage with Transformers
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+
+model = AutoModelForSequenceClassification.from_pretrained("model_name")
+tokenizer = AutoTokenizer.from_pretrained("model_name")
+
+features = tokenizer(["Query", "Query"], ["Paragraph1", "Paragraph2"], padding=True, truncation=True, return_tensors="pt")
+
+model.eval()
+with torch.no_grad():
+    scores = model(**features).logits
+    print(scores)
+```
+
+
+## Models & Performance
+
+In the following table, we provide various pre-trained Cross-Encoders together with their performance on the [TREC Deep Learning 2019](https://microsoft.github.io/TREC-2019-Deep-Learning/) and the [MS Marco Passage Reranking](https://github.com/microsoft/MSMARCO-Passage-Ranking/) dataset. 
+
+
+| Model-Name        | NDCG@10 (TREC DL 19) | MRR@10 (MS Marco Dev)  | Docs / Sec |
+| ------------- | :-------------: | :-----: | ---: | 
+| **Version 2 models** | | | 
+| cross-encoder/ms-marco-TinyBERT-L-2-v2 | 69.84 | 32.56 | 9000
+| cross-encoder/ms-marco-MiniLM-L-2-v2 | 71.01 | 34.85 | 4100
+| cross-encoder/ms-marco-MiniLM-L-4-v2 | 73.04 | 37.70 | 2500
+| cross-encoder/ms-marco-MiniLM-L-6-v2 | 74.30 | 39.01 | 1800
+| cross-encoder/ms-marco-MiniLM-L-12-v2 | 74.31 | 39.02 | 960
+| **Version 1 models** | | | 
+| cross-encoder/ms-marco-TinyBERT-L-2  | 67.43 | 30.15  | 9000 | 
+| cross-encoder/ms-marco-TinyBERT-L-4  | 68.09 | 34.50  | 2900 | 
+| cross-encoder/ms-marco-TinyBERT-L-6 |  69.57 | 36.13  | 680 | 
+| cross-encoder/ms-marco-electra-base | 71.99 | 36.41 | 340 | 
+| **Other models** | | | |
+| nboost/pt-tinybert-msmarco | 63.63 | 28.80 | 2900 | 
+| nboost/pt-bert-base-uncased-msmarco | 70.94 | 34.75 | 340 | 
+| nboost/pt-bert-large-msmarco | 73.36 | 36.48 | 100 |  
+| Capreolus/electra-base-msmarco | 71.23 | 36.89 | 340 | 
+| amberoad/bert-multilingual-passage-reranking-msmarco | 68.40 | 35.54 | 330 |  
+| sebastian-hofstaetter/distilbert-cat-margin_mse-T2-msmarco | 72.82 | 37.88 | 720
+ 
+ Note: Runtime was computed on a V100 GPU with Huggingface Transformers v4. 

docs/pretrained-models/dpr.md

+# DPR-Models
+In [Dense Passage Retrieval  for Open-Domain Question Answering](https://arxiv.org/abs/2004.04906)  Karpukhin et al. trained models based on [Google's Natural Questions dataset](https://ai.google.com/research/NaturalQuestions):
+- **facebook-dpr-ctx_encoder-single-nq-base** 
+- **facebook-dpr-question_encoder-single-nq-base**
+
+They also trained models on the combination of Natural Questions, TriviaQA, WebQuestions, and CuratedTREC.
+- **facebook-dpr-ctx_encoder-multiset-base** 
+- **facebook-dpr-question_encoder-multiset-base**
+
+
+There is one model to encode passages and one model to encode question / queries.
+
+## Usage
+
+To encode paragraphs, you need to provide a title (e.g. the Wikipedia article title) and the text passage. These must be separated with a `[SEP]` token.  For encoding paragraphs, we use the **ctx_encoder**.
+
+Queries are encoded with **question_encoder**:
+```python
+from sentence_transformers import SentenceTransformer, util
+
+passage_encoder = SentenceTransformer("facebook-dpr-ctx_encoder-single-nq-base")
+
+passages = [
+    "London [SEP] London is the capital and largest city of England and the United Kingdom.",
+    "Paris [SEP] Paris is the capital and most populous city of France.",
+    "Berlin [SEP] Berlin is the capital and largest city of Germany by both area and population.",
+]
+
+passage_embeddings = passage_encoder.encode(passages)
+
+query_encoder = SentenceTransformer("facebook-dpr-question_encoder-single-nq-base")
+query = "What is the capital of England?"
+query_embedding = query_encoder.encode(query)
+
+# Important: You must use dot-product, not cosine_similarity
+scores = util.dot_score(query_embedding, passage_embeddings)
+print("Scores:", scores)
+```
+
+**Important note:** When you use these models, you have to use them with dot-product (e.g. as implemented in `util.dot_score`) and not with cosine similarity.

docs/pretrained-models/msmarco-v1.md

+# MSMARCO Models
+[MS MARCO](https://microsoft.github.io/msmarco/) is a large scale information retrieval corpus that was created based on real user search queries using Bing search engine. The provided models can be used for semantic search, i.e., given keywords / a search phrase / a question, the model will find passages that are relevant for the search query.
+
+The training data constist of over 500k examples, while the complete  corpus consist of over 8.8 Million passages.
+ 
+
+
+## Version Histroy 
+As we work on the topic, we will publish updated (and improved) models.
+
+### v1
+Version 1 models were trained on the training set of MS Marco Passage retrieval task. The models were trained using in-batch negative sampling via the MultipleNegativesRankingLoss with a scaling factor of 20 and a batch size of 128.
+
+They can be used like this:
+```python
+from sentence_transformers import SentenceTransformer, util
+
+model = SentenceTransformer("distilroberta-base-msmarco-v1")
+
+query_embedding = model.encode("[QRY] " + "How big is London")
+passage_embedding = model.encode("[DOC] " + "London has 9,787,426 inhabitants at the 2011 census")
+
+print("Similarity:", util.pytorch_cos_sim(query_embedding, passage_embedding))
+```
+
+**Models**:
+- **distilroberta-base-msmarco-v1** - Performance MSMARCO dev dataset (queries.dev.small.tsv) MRR@10: 23.28
\ No newline at end of file

docs/pretrained-models/msmarco-v2.md

+# MSMARCO Models (Version 2)
+[MS MARCO](https://microsoft.github.io/msmarco/) is a large scale information retrieval corpus that was created based on real user search queries using Bing search engine. The provided models can be used for semantic search, i.e., given keywords / a search phrase / a question, the model will find passages that are relevant for the search query.
+
+The training data consists of over 500k examples, while the complete  corpus consist of over 8.8 Million passages.
+ 
+## Usage
+```python
+from sentence_transformers import SentenceTransformer, util
+
+model = SentenceTransformer("msmarco-distilroberta-base-v2")
+
+query_embedding = model.encode("How big is London")
+passage_embedding = model.encode("London has 9,787,426 inhabitants at the 2011 census")
+
+print("Similarity:", util.pytorch_cos_sim(query_embedding, passage_embedding))
+```
+
+
+For more details on the usage, see [Applications - Information Retrieval](../../examples/applications/retrieve_rerank/README.md)
+
+
+## Performance
+Performance is evaluated on [TREC-DL 2019](https://microsoft.github.io/TREC-2019-Deep-Learning/), which is a query-passage retrieval task where multiple queries have been annotated as with their relevance with respect to the given query.  Further, we evaluate on the [MS Marco Passage Retrieval](https://github.com/microsoft/MSMARCO-Passage-Ranking/) dataset. 
+
+As baseline we show the results for lexical search with BM25 using Elasticsearch.
+
+| Approach       | NDCG@10 (TREC DL 19 Reranking) | MRR@10 (MS Marco Dev) |  
+| ------------- |:-------------: | :---: |
+| BM25 (Elasticsearch)   | 45.46 | 17.29  |
+| msmarco-distilroberta-base-v2   | 65.65 |  28.55    |  
+| msmarco-roberta-base-v2 | 67.18 | 29.17 | 
+| msmarco-distilbert-base-v2 | 68.35 | 30.77 |
+
+
+
+## Version Histroy 
+As we work on the topic, we will publish updated (and improved) models.
+
+- [Version 1](msmarco-v1.md)