Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
Qwen-7B_fastllm
Commits
69cac0e1
Commit
69cac0e1
authored
Nov 01, 2023
by
zhouxiang
Browse files
修改格式
parent
caca906d
Changes
5
Hide whitespace changes
Inline
Side-by-side
Showing
5 changed files
with
3 additions
and
3 deletions
+3
-3
README.md
README.md
+3
-3
doc/qwen.png
doc/qwen.png
+0
-0
doc/qwen推理.gif
doc/qwen推理.gif
+0
-0
doc/transformer.jpg
doc/transformer.jpg
+0
-0
qwen.jpg
qwen.jpg
+0
-0
No files found.
README.md
View file @
69cac0e1
...
...
@@ -12,7 +12,7 @@ https://arxiv.org/pdf/2308.12966.pdf
本项目主要针对Qwen-7B-Chat在DCU平台的推理性能优化,达到DCU平台较快的对话效果。


...
...
@@ -20,7 +20,7 @@ https://arxiv.org/pdf/2308.12966.pdf
Qwen-7B的构建采用了类似LLaMA的架构。与标准transformer的主要差异有:1)使用非连接嵌入、2)使用旋转位置嵌入、3)在注意力中除了QKV外不使用偏置、4)使用RMSNorm代替LayerNorm、5)使用SwiGLU代替ReLU、以及6)采用快速注意力来加速训练。该模型共有32层,嵌入维度为4096,注意力头数为32。


## 环境配置
...
...
@@ -104,7 +104,7 @@ chmod +x benchmark
## result


### 精度
...
...
qwen.png
→
doc/
qwen.png
View file @
69cac0e1
File moved
qwen推理.gif
→
doc/
qwen推理.gif
View file @
69cac0e1
File moved
doc/transformer.jpg
0 → 100644
View file @
69cac0e1
87.9 KB
qwen.jpg
deleted
100644 → 0
View file @
caca906d
32.7 KB
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment