Commit 9594ddfd authored by liangjing's avatar liangjing
Browse files

Update README.md

parent ef8d494d
Pipeline #596 canceled with stages
......@@ -18,17 +18,6 @@ LLaMA,这是一个基础语言模型的集合,参数范围从7B到65B。在
![img](./llama模型结构.png)
LLaMA模型具体参数:
**| 模型名称 | 隐含层维度 | 层数 | 头数 | **
**|** **--------** **|** **--------** **|** **--------** **|** **--------** **|**
**|** LLaMA**-**7B **|** 4,096 **|** 32 **|** 32 **|**
**|** LLaMA**-**13B **|** 5,120 **|** 40 **|** 40 **|**
**|** LLaMA**-**65B **|** 8192 **|** 80 **|** 64 **|**
## 算法原理
以下是与原始 Transformer 架构的主要区别:
......@@ -153,4 +142,4 @@ sbatch run.sh
- https://www.deepspeed.ai/getting-started/
- https://deepspeed.readthedocs.io/en/latest/index.html
\ No newline at end of file
- https://deepspeed.readthedocs.io/en/latest/index.html
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment