Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
Lmdeploy
Commits
1bbc6e05
Unverified
Commit
1bbc6e05
authored
Nov 03, 2023
by
RunningLeon
Committed by
GitHub
Nov 03, 2023
Browse files
add cli to list the supported model names (#639)
* update * resolve comment
parent
6e91e5ce
Changes
3
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
26 additions
and
2 deletions
+26
-2
README.md
README.md
+1
-1
README_zh-CN.md
README_zh-CN.md
+1
-1
lmdeploy/cli/cli.py
lmdeploy/cli/cli.py
+24
-0
No files found.
README.md
View file @
1bbc6e05
...
@@ -52,7 +52,7 @@ LMDeploy is a toolkit for compressing, deploying, and serving LLM, developed by
...
@@ -52,7 +52,7 @@ LMDeploy is a toolkit for compressing, deploying, and serving LLM, developed by
## Supported Models
## Supported Models
`LMDeploy`
has two inference backends,
`Pytorch`
and
`TurboMind`
.
`LMDeploy`
has two inference backends,
`Pytorch`
and
`TurboMind`
.
You can run
`lmdeploy list`
to check the supported model names.
### TurboMind
### TurboMind
...
...
README_zh-CN.md
View file @
1bbc6e05
...
@@ -53,7 +53,7 @@ LMDeploy 由 [MMDeploy](https://github.com/open-mmlab/mmdeploy) 和 [MMRazor](ht
...
@@ -53,7 +53,7 @@ LMDeploy 由 [MMDeploy](https://github.com/open-mmlab/mmdeploy) 和 [MMRazor](ht
## 支持的模型
## 支持的模型
`LMDeploy`
支持
`TurboMind`
和
`Pytorch`
两种推理后端
`LMDeploy`
支持
`TurboMind`
和
`Pytorch`
两种推理后端
。运行
`lmdeploy list`
可查看支持模型列表
### TurboMind
### TurboMind
...
...
lmdeploy/cli/cli.py
View file @
1bbc6e05
...
@@ -49,6 +49,30 @@ class CLI(object):
...
@@ -49,6 +49,30 @@ class CLI(object):
quant_path
=
quant_path
,
quant_path
=
quant_path
,
group_size
=
group_size
)
group_size
=
group_size
)
def
list
(
self
,
engine
:
str
=
'turbomind'
):
"""List supported model names.
Examples 1:
lmdeploy list
Examples 2:
lmdeploy list --engine pytorch
Args:
engine (str): The backend for the model to run. Choice from
['turbomind', 'pytorch'].
"""
assert
engine
in
[
'turbomind'
,
'pytorch'
]
if
engine
==
'pytorch'
:
model_names
=
[
'llama'
,
'llama2'
,
'internlm-7b'
]
elif
engine
==
'turbomind'
:
from
lmdeploy.model
import
MODELS
model_names
=
list
(
MODELS
.
module_dict
.
keys
())
model_names
=
[
n
for
n
in
model_names
if
n
.
lower
()
not
in
[
'base'
]]
model_names
.
sort
()
print
(
'Supported model names:'
)
print
(
'
\n
'
.
join
(
model_names
))
def
run
():
def
run
():
"""The entry point of running LMDeploy CLI."""
"""The entry point of running LMDeploy CLI."""
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment