Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
megatron-deepspeed-llama_pytorch
Commits
9594ddfd
Commit
9594ddfd
authored
Oct 19, 2023
by
liangjing
Browse files
Update README.md
parent
ef8d494d
Pipeline
#596
canceled with stages
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
12 deletions
+1
-12
README.md
README.md
+1
-12
No files found.
README.md
View file @
9594ddfd
...
...
@@ -18,17 +18,6 @@ LLaMA,这是一个基础语言模型的集合,参数范围从7B到65B。在

LLaMA模型具体参数:
**| 模型名称 | 隐含层维度 | 层数 | 头数 | **
**|**
**--------**
**|**
**--------**
**|**
**--------**
**|**
**--------**
**|**
**|**
LLaMA
**-**
7B
**|**
4,096
**|**
32
**|**
32
**|**
**|**
LLaMA
**-**
13B
**|**
5,120
**|**
40
**|**
40
**|**
**|**
LLaMA
**-**
65B
**|**
8192
**|**
80
**|**
64
**|**
## 算法原理
以下是与原始 Transformer 架构的主要区别:
...
...
@@ -153,4 +142,4 @@ sbatch run.sh
-
https://www.deepspeed.ai/getting-started/
-
https://deepspeed.readthedocs.io/en/latest/index.html
\ No newline at end of file
-
https://deepspeed.readthedocs.io/en/latest/index.html
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment