Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
SIYIXNI
vllm
Commits
0b32a987
"docs/source/en/training/lora.md" did not exist on "022479416f8667c25d71c336fedb9b6a4ed8a89c"
Unverified
Commit
0b32a987
authored
Jun 20, 2023
by
Zhuohan Li
Committed by
GitHub
Jun 20, 2023
Browse files
Add and list supported models in README (#161)
parent
570fb2e9
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
15 additions
and
1 deletion
+15
-1
README.md
README.md
+7
-0
docs/source/conf.py
docs/source/conf.py
+2
-0
docs/source/models/supported_models.rst
docs/source/models/supported_models.rst
+6
-1
No files found.
README.md
View file @
0b32a987
...
...
@@ -39,6 +39,13 @@ vLLM is flexible and easy to use with:
-
Streaming outputs
-
OpenAI-compatible API server
vLLM seamlessly supports many Huggingface models, including the following architectures:
-
GPT-2 (e.g.,
`gpt2`
,
`gpt2-xl`
, etc.)
-
GPTNeoX (e.g.,
`EleutherAI/gpt-neox-20b`
,
`databricks/dolly-v2-12b`
,
`stabilityai/stablelm-tuned-alpha-7b`
, etc.)
-
LLaMA (e.g.,
`lmsys/vicuna-13b-v1.3`
,
`young-geng/koala`
,
`openlm-research/open_llama_13b`
, etc.)
-
OPT (e.g.,
`facebook/opt-66b`
,
`facebook/opt-iml-max-30b`
, etc.)
Install vLLM with pip or
[
from source
](
https://llm-serving-cacheflow.readthedocs-hosted.com/en/latest/getting_started/installation.html#build-from-source
)
:
```
bash
...
...
docs/source/conf.py
View file @
0b32a987
...
...
@@ -53,7 +53,9 @@ copybutton_prompt_is_regexp = True
#
html_title
=
project
html_theme
=
'sphinx_book_theme'
html_logo
=
'assets/logos/vllm-logo-text-light.png'
html_theme_options
=
{
'logo_only'
:
True
,
'path_to_docs'
:
'docs/source'
,
'repository_url'
:
'https://github.com/WoosukKwon/vllm'
,
'use_repository_button'
:
True
,
...
...
docs/source/models/supported_models.rst
View file @
0b32a987
...
...
@@ -8,19 +8,24 @@ The following is the list of model architectures that are currently supported by
Alongside each architecture, we include some popular models that use it.
.. list-table::
:widths: 25
75
:widths: 25
25 50
:header-rows: 1
* - Architecture
- Models
- Example HuggingFace Models
* - :code:`GPT2LMHeadModel`
- GPT-2
- :code:`gpt2`, :code:`gpt2-xl`, etc.
* - :code:`GPTNeoXForCausalLM`
- GPT-NeoX, Pythia, OpenAssistant, Dolly V2, StableLM
- :code:`EleutherAI/gpt-neox-20b`, :code:`EleutherAI/pythia-12b`, :code:`OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5`, :code:`databricks/dolly-v2-12b`, :code:`stabilityai/stablelm-tuned-alpha-7b`, etc.
* - :code:`LlamaForCausalLM`
- LLaMA, Vicuna, Alpaca, Koala, Guanaco
- :code:`openlm-research/open_llama_13b`, :code:`lmsys/vicuna-13b-v1.3`, :code:`young-geng/koala`, :code:`JosephusCheung/Guanaco`, etc.
* - :code:`OPTForCausalLM`
- OPT, OPT-IML
- :code:`facebook/opt-66b`, :code:`facebook/opt-iml-max-30b`, etc.
If your model uses one of the above model architectures, you can seamlessly run your model with vLLM.
Otherwise, please refer to :ref:`Adding a New Model <adding_a_new_model>` for instructions on how to implement support for your model.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment