Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
InternLM_vllm
Commits
53543689
Commit
53543689
authored
Apr 15, 2025
by
chenzk
Browse files
Update url.md
parent
e285e523
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
5 deletions
+2
-5
README.md
README.md
+2
-5
No files found.
README.md
View file @
53543689
...
@@ -90,7 +90,7 @@ export VLLM_RANK7_NUMA=7
...
@@ -90,7 +90,7 @@ export VLLM_RANK7_NUMA=7
| 基座模型 | | | |
| 基座模型 | | | |
| ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
|
[
internlm2-7b
](
http://
113.200.138.88:18080/aimodels
/internlm/internlm2-7b
.git
)
|
[
internlm2-20b
](
http://
113.200.138.88:18080/aimodels
/internlm/internlm2-20b
.git
)
|
[
internlm2_5-7b
](
http://
113.200.138.88:18080/aimodels
/internlm/internlm2_5-7b
.git
)
|
[
internlm2_5-20b
](
http://
113.200.138.88:18080/aimodels
/internlm/internlm2_5-20b
.git
)
|
|
[
internlm2-7b
](
http
s
://
huggingface.co
/internlm/internlm2-7b
)
|
[
internlm2-20b
](
http
s
://
huggingface.co
/internlm/internlm2-20b
)
|
[
internlm2_5-7b
](
http
s
://
huggingface.co
/internlm/internlm2_5-7b
)
|
[
internlm2_5-20b
](
http
s
://
huggingface.co
/internlm/internlm2_5-20b
)
|
### 离线批量推理
### 离线批量推理
...
@@ -113,10 +113,7 @@ python benchmarks/benchmark_throughput.py --num-prompts 1 --input-len 32 --outpu
...
@@ -113,10 +113,7 @@ python benchmarks/benchmark_throughput.py --num-prompts 1 --input-len 32 --outpu
2、使用数据集
2、使用数据集
下载数据集:
下载数据集:
[
ShareGPT_Vicuna_unfiltered
](
https://huggingface.co/datasets/Aeala/ShareGPT_Vicuna_unfiltered
)
```
bash
wget http://113.200.138.88:18080/aidatasets/anon8231489123/ShareGPT_Vicuna_unfiltered.git
```
```
bash
```
bash
python benchmarks/benchmark_throughput.py
--num-prompts
1
--model
internlm/internlm2_5-7b
--dataset
ShareGPT_V3_unfiltered_cleaned_split.json
-tp
1
--trust-remote-code
--enforce-eager
--dtype
float16
python benchmarks/benchmark_throughput.py
--num-prompts
1
--model
internlm/internlm2_5-7b
--dataset
ShareGPT_V3_unfiltered_cleaned_split.json
-tp
1
--trust-remote-code
--enforce-eager
--dtype
float16
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment