Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
Lmdeploy
Commits
93604c3f
Unverified
Commit
93604c3f
authored
Jun 25, 2023
by
tpoisonooo
Committed by
GitHub
Jun 25, 2023
Browse files
style(doc): README.md
parent
e0c7f51b
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
6 deletions
+6
-6
README.md
README.md
+6
-6
No files found.
README.md
View file @
93604c3f
...
...
@@ -80,7 +80,7 @@ Run one of the following commands to serve a LLaMA model on NVIDIA GPU server:
```
shell
python3 llmdeploy/serve/fastertransformer/deploy.py llama-7B /path/to/llama-7b llama
\
--tokenizer_path
/path/to/tokenizer/model
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/
/
backends/fastertransformer
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/backends/fastertransformer
```
</details>
...
...
@@ -91,7 +91,7 @@ bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fas
```
shell
python3 llmdeploy/serve/fastertransformer/deploy.py llama-13B /path/to/llama-13b llama
\
--tokenizer_path
/path/to/tokenizer/model
--tp
2
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/
/
backends/fastertransformer
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/backends/fastertransformer
```
</details>
...
...
@@ -102,7 +102,7 @@ bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fas
```
shell
python3 llmdeploy/serve/fastertransformer/deploy.py llama-33B /path/to/llama-33b llama
\
--tokenizer_path
/path/to/tokenizer/model
--tp
4
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/
/
backends/fastertransformer
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/backends/fastertransformer
```
</details>
...
...
@@ -113,7 +113,7 @@ bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fas
```
shell
python3 llmdeploy/serve/fastertransformer/deploy.py llama-65B /path/to/llama-65b llama
\
--tokenizer_path
/path/to/tokenizer/model
--tp
8
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/
/
backends/fastertransformer
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/backends/fastertransformer
```
</details>
...
...
@@ -131,7 +131,7 @@ python3 -m fastchat.model.apply_delta \
--delta-path
lmsys/vicuna-7b-delta-v1.1
python3 llmdeploy/serve/fastertransformer/deploy.py vicuna-7B /path/to/vicuna-7b hf
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/
/
backends/fastertransformer
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/backends/fastertransformer
```
</details>
...
...
@@ -147,7 +147,7 @@ python3 -m fastchat.model.apply_delta \
--delta-path
lmsys/vicuna-13b-delta-v1.1
python3 llmdeploy/serve/fastertransformer/deploy.py vicuna-13B /path/to/vicuna-13b hf
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/
/
backends/fastertransformer
bash workspace/service_docker_up.sh
--lib-dir
$(
pwd
)
/build/install/backends/fastertransformer
```
</details>
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment