Q: How can I serve multiple models on a single port using the OpenAI API?
Q: How can I serve multiple models on a single port using the OpenAI API?
...
@@ -9,4 +9,4 @@ A: Assuming that you're referring to using OpenAI compatible server to serve mul
...
@@ -9,4 +9,4 @@ A: Assuming that you're referring to using OpenAI compatible server to serve mul
Q: Which model to use for offline inference embedding?
Q: Which model to use for offline inference embedding?
A: If you want to use an embedding model, try: https://huggingface.co/intfloat/e5-mistral-7b-instruct. Instead models, such as Llama-3-8b, Mistral-7B-Instruct-v0.3, are generation models rather than an embedding model
A: If you want to use an embedding model, try: https://huggingface.co/intfloat/e5-mistral-7b-instruct. Instead models, such as Llama-3-8b, Mistral-7B-Instruct-v0.3, are generation models rather than an embedding model