Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
LLaMA_Fastchat_pytorch
Commits
254b8d55
Commit
254b8d55
authored
Apr 03, 2024
by
dcuai
Browse files
Update README.md
parent
a29ff9c7
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
13 additions
and
13 deletions
+13
-13
README.md
README.md
+13
-13
No files found.
README.md
View file @
254b8d55
...
...
@@ -112,17 +112,6 @@ bash mpi_job.sh
如果单节点运行7B的模型出现oom,可以适当减少batch size。
## result
### 精度
训练数据:
[
./FastChat-main/playground/data/alpaca-data-conversation.json
](
链接
)
使用的GPGPU:16张DCU-Z100L-32G。
模型精度(max_sequence_length: 2048):
| 卡数 | 分布式工具 | 收敛性 |
| :------: | :------: |:------: |
| 16 | deepspeed | total_loss: 0.62/150 steps |
### input
```
plaintext
...
...
@@ -135,11 +124,22 @@ bash mpi_job.sh
>>>回答:避寒,当然是去海南呀!海南的冬天,阳光明媚,温度适宜,而且空气清新,没有雾霾,没有沙尘暴,没有雾霾,没有雾霾!
```
### 精度
训练数据:
[
./FastChat-main/playground/data/alpaca-data-conversation.json
](
链接
)
使用的GPGPU:16张DCU-Z100L-32G。
模型精度(max_sequence_length: 2048):
| 卡数 | 分布式工具 | 收敛性 |
| :------: | :------: |:------: |
| 16 | deepspeed | total_loss: 0.62/150 steps |
## 应用场景
### 算法类别
`
自然语言处理
`
`
对话问答
`
### 热点应用行业
...
...
@@ -152,4 +152,4 @@ bash mpi_job.sh
## 参考
*
https://huggingface.co/decapoda-research/llama-13b-hf
*
https://github.com/lm-sys/FastChat
\ No newline at end of file
*
https://github.com/lm-sys/FastChat
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment