Skip to content

GitLab

Unverified Commit 93604c3f authored by tpoisonooo's avatar tpoisonooo Committed by GitHub
Browse files

style(doc): README.md

parent e0c7f51b
Show whitespace changes
Inline Side-by-side
Showing with 6 additions and 6 deletions
README.md
View file @ 93604c3f
...@@ -80,7 +80,7 @@ Run one of the following commands to serve a LLaMA model on NVIDIA GPU server: ...@@ -80,7 +80,7 @@ Run one of the following commands to serve a LLaMA model on NVIDIA GPU server:
```shell ```shell
python3 llmdeploy/serve/fastertransformer/deploy.py llama-7B /path/to/llama-7b llama \ python3 llmdeploy/serve/fastertransformer/deploy.py llama-7B /path/to/llama-7b llama \
--tokenizer_path /path/to/tokenizer/model --tokenizer_path /path/to/tokenizer/model
bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fastertransformer bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install/backends/fastertransformer
``` ```
</details> </details>
...@@ -91,7 +91,7 @@ bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fas ...@@ -91,7 +91,7 @@ bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fas
```shell ```shell
python3 llmdeploy/serve/fastertransformer/deploy.py llama-13B /path/to/llama-13b llama \ python3 llmdeploy/serve/fastertransformer/deploy.py llama-13B /path/to/llama-13b llama \
--tokenizer_path /path/to/tokenizer/model --tp 2 --tokenizer_path /path/to/tokenizer/model --tp 2
bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fastertransformer bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install/backends/fastertransformer
``` ```
</details> </details>
...@@ -102,7 +102,7 @@ bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fas ...@@ -102,7 +102,7 @@ bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fas
```shell ```shell
python3 llmdeploy/serve/fastertransformer/deploy.py llama-33B /path/to/llama-33b llama \ python3 llmdeploy/serve/fastertransformer/deploy.py llama-33B /path/to/llama-33b llama \
--tokenizer_path /path/to/tokenizer/model --tp 4 --tokenizer_path /path/to/tokenizer/model --tp 4
bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fastertransformer bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install/backends/fastertransformer
``` ```
</details> </details>
...@@ -113,7 +113,7 @@ bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fas ...@@ -113,7 +113,7 @@ bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fas
```shell ```shell
python3 llmdeploy/serve/fastertransformer/deploy.py llama-65B /path/to/llama-65b llama \ python3 llmdeploy/serve/fastertransformer/deploy.py llama-65B /path/to/llama-65b llama \
--tokenizer_path /path/to/tokenizer/model --tp 8 --tokenizer_path /path/to/tokenizer/model --tp 8
bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fastertransformer bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install/backends/fastertransformer
``` ```
</details> </details>
...@@ -131,7 +131,7 @@ python3 -m fastchat.model.apply_delta \ ...@@ -131,7 +131,7 @@ python3 -m fastchat.model.apply_delta \
--delta-path lmsys/vicuna-7b-delta-v1.1 --delta-path lmsys/vicuna-7b-delta-v1.1
python3 llmdeploy/serve/fastertransformer/deploy.py vicuna-7B /path/to/vicuna-7b hf python3 llmdeploy/serve/fastertransformer/deploy.py vicuna-7B /path/to/vicuna-7b hf
bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fastertransformer bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install/backends/fastertransformer
``` ```
</details> </details>
...@@ -147,7 +147,7 @@ python3 -m fastchat.model.apply_delta \ ...@@ -147,7 +147,7 @@ python3 -m fastchat.model.apply_delta \
--delta-path lmsys/vicuna-13b-delta-v1.1 --delta-path lmsys/vicuna-13b-delta-v1.1
python3 llmdeploy/serve/fastertransformer/deploy.py vicuna-13B /path/to/vicuna-13b hf python3 llmdeploy/serve/fastertransformer/deploy.py vicuna-13B /path/to/vicuna-13b hf
bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install//backends/fastertransformer bash workspace/service_docker_up.sh --lib-dir $(pwd)/build/install/backends/fastertransformer
``` ```
</details> </details>
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment