Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
LLaMA_vllm
Commits
dcd126ae
Commit
dcd126ae
authored
Apr 15, 2025
by
chenzk
Browse files
Update url.md
parent
00a55946
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
18 additions
and
9 deletions
+18
-9
README.md
README.md
+18
-9
No files found.
README.md
View file @
dcd126ae
...
...
@@ -79,15 +79,25 @@ export VLLM_RANK7_NUMA=7
### 模型下载
**快速下载通道:**
| 基座模型 | chat模型 | GPTQ模型 | AWQ模型 |
| ------- | ------- | ------- | ------- |
|
[
Llama-2-7b-hf
](
http://113.200.138.88:18080/aimodels/Llama-2-7b-hf
)
|
[
Llama-2-7b-chat-hf
](
http://113.200.138.88:18080/aimodels/Llama-2-7b-chat-hf
)
|
[
Llama-2-7B-Chat-GPTQ
](
http://113.200.138.88:18080/aimodels/Llama-2-7B-Chat-GPTQ
)
|
[
Llama-2-7B-Chat-AWQ
](
http://113.200.138.88:18080/aimodels/thebloke/Llama-2-7B-AWQ
)
|
|
[
Llama-2-13b-hf
](
http://113.200.138.88:18080/aimodels/Llama-2-13b-hf
)
|
[
Llama-2-13b-chat-hf
](
http://113.200.138.88:18080/aimodels/meta-llama/Llama-2-13b-chat-hf
)
|
[
Llama-2-13B-GPTQ
](
http://113.200.138.88:18080/aimodels/Llama-2-13B-chat-GPTQ
)
|
[
Llama-2-13B-AWQ
](
http://113.200.138.88:18080/aimodels/thebloke/Llama-2-13B-AWQ
)
|
|
[
Llama-2-70b-hf
](
http://113.200.138.88:18080/aimodels/Llama-2-70b-hf
)
|
[
Llama-2-70b-chat-hf
](
http://113.200.138.88:18080/aimodels/meta-llama/Llama-2-70b-chat-hf
)
|
[
Llama-2-70B-Chat-GPTQ
](
http://113.200.138.88:18080/aimodels/Llama-2-70B-Chat-GPTQ
)
|
[
Llama-2-70B-Chat-AWQ
](
http://113.200.138.88:18080/aimodels/thebloke/Llama-2-70B-AWQ
)
|
|
[
Meta-Llama-3-8B
](
http://113.200.138.88:18080/aimodels/Meta-Llama-3-8B
)
|
[
Meta-Llama-3-8B-Instruct
](
http://113.200.138.88:18080/aimodels/Meta-Llama-3-8B-Instruct
)
|
[
Meta-Llama-3-8B-Instruct-AWQ
](
http://113.200.138.88:18080/aimodels/solidrust/Meta-Llama-3-8B-Instruct-hf-AWQ
)
|
|
[
Meta-Llama-3-70B
](
http://113.200.138.88:18080/aimodels/Meta-Llama-3-70B
)
|
[
Meta-Llama-3-70B-Instruct
](
http://113.200.138.88:18080/aimodels/Meta-Llama-3-70B-Instruct
)
|
[
Meta-Llama-3-70B-Instruct-AWQ
](
http://113.200.138.88:18080/aimodels/techxgenus/Meta-Llama-3-70B-Instruct-AWQ
)
|
可从HF下载以下模型进行使用:
Llama-2-7b-hf
Llama-2-7b-chat-hf
Llama-2-7B-Chat-GPTQ
Llama-2-7B-AWQ
Llama-2-13b-hf
Llama-2-13b-chat-hf
Llama-2-13B-GPTQ
Llama-2-13B-AWQ
Llama-2-70b-hf
Llama-2-70B-Chat-GPTQ
Llama-2-70B-AWQ
Meta-Llama-3-8B
Meta-Llama-3-8B-Instruct
Meta-Llama-3-8B-Instruct-AWQ
Meta-Llama-3-70B
Meta-Llama-3-70B-Instruct
Meta-Llama-3-70B-Instruct-AWQ
### 离线批量推理
```
bash
...
...
@@ -108,7 +118,6 @@ python benchmarks/benchmark_throughput.py --num-prompts 1 --input-len 32 --outpu
下载数据集:
```
bash
wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json
wget http://113.200.138.88:18080/aidatasets/anon8231489123/ShareGPT_Vicuna_unfiltered.git
```
```
bash
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment