Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
norm
vllm
Commits
fffa2e1f
Commit
fffa2e1f
authored
Feb 13, 2023
by
Woosuk Kwon
Browse files
Add model_utils
parent
bb59a3e7
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
17 additions
and
2 deletions
+17
-2
cacheflow/worker/models/__init__.py
cacheflow/worker/models/__init__.py
+4
-2
cacheflow/worker/models/model_utils.py
cacheflow/worker/models/model_utils.py
+13
-0
No files found.
cacheflow/worker/models/__init__.py
View file @
fffa2e1f
from
cacheflow.worker.models.opt
import
OPTForCausalLM
from
cacheflow.worker.models.model_utils
import
get_model
__all__
=
[
'OPTForCausalLM'
,
'get_model'
,
]
cacheflow/worker/models/model_utils.py
0 → 100644
View file @
fffa2e1f
import
torch.nn
as
nn
from
cacheflow.worker.models.opt
import
OPTForCausalLM
MODEL_CLASSES
=
{
'opt'
:
OPTForCausalLM
,
}
def
get_model
(
model_name
:
str
)
->
nn
.
Module
:
if
model_name
not
in
MODEL_CLASSES
:
raise
ValueError
(
f
'Invalid model name:
{
model_name
}
'
)
return
MODEL_CLASSES
[
model_name
].
from_pretrained
(
model_name
)
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment