Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
codellama_pytorch
Commits
1a4f7e30
Commit
1a4f7e30
authored
Aug 28, 2024
by
dengjb
Browse files
Update README.md
parent
fa5e9fd3
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
20 additions
and
5 deletions
+20
-5
README.md
README.md
+20
-5
No files found.
README.md
View file @
1a4f7e30
...
...
@@ -2,7 +2,7 @@
Codellama模型是一系列7B、13B、34B和70B模型,使用500B-1T的tokens进行训练。
## 论文
`Code Llama: Open Foundation Models for Code`
<br>
[
CodeLlama
](
https://
scontent-lax3-1.xx.fbcdn.net/v/t39.2365-6/440937359_1249838219330505_1104237120116944930_n.pdf?_nc_cat=109&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=nf68R9-64t8Q7kNvgHxIBjM&_nc_ht=scontent-lax3-1.xx&oh=00_AYBcUILDMCTVXhedseMkIiEnVwAkWPP9xejZllnlBiaK0Q&oe=66893420
)
[
CodeLlama
](
https://
ai.meta.com/research/publications/code-llama-open-foundation-models-for-code/
)
## 模型结构
codellama的模型结构主要基于llama2架构进行训练而来,使用了不同的训练方法得到了基于不同任务目的的代码生成模型。
<div
align=
center
>
...
...
@@ -58,7 +58,14 @@ export HF_ENDPOINT=https://hf-mirror.com
```
## 数据集
finetune训练样例数据采用verilog-dataset-v3
[
下载地址
](
https://hf-mirror.com/datasets/emilgoh/verilog-dataset-v3
)
<br>
finetune训练样例数据采用verilog-dataset-v3
<br>
-
数据集快速下载中心:
-
[
SCNet AIDatasets
](
http://113.200.138.88:18080/aidatasets
)
-
数据集快速通道下载地址:
-
[
数据集快速下载地址
](
http://113.200.138.88:18080/aidatasets/verilog-dataset-v3/-/tree/main
)
-
官方下载地址
-
[
官方下载地址
](
https://hf-mirror.com/datasets/emilgoh/verilog-dataset-v3
)
```
angular2html
.
├── verilog-dataset-v3
...
...
@@ -84,7 +91,16 @@ bash ./train.sh
## 推理
基于Huggingface's Transformers进行推理.
<br>
模型下载后
[
模型下载地址
](
https://hf-mirror.com/meta-llama/CodeLlama-7b-Instruct-hf
)
[
快速模型下载地址
]
(http://113.200.138.88:18080/aimodels/CodeLlama-7b-Instruct-hf) 默认需存放至weights文件夹中
<br>
-
可使用官方模型权重进行推理,也可使用自己训练模型权重进行推理
-
模型权重快速下载中心
-
[
SCNet AIModels
](
http://113.200.138.88:18080/aimodels
)
-
模型权重快速通道下载地址
-
[
模型权重快速下载地址
](
http://113.200.138.88:18080/aimodels/CodeLlama-7b-Instruct-hf
)
-
官方下载地址:
-
[
下载地址:yolov5s.pt
](
https://hf-mirror.com/meta-llama/CodeLlama-7b-Instruct-hf
)
-
这里以官方模型推理举例
默认需存放至weights文件夹中
<br>
也可自行更改 inference.py文件中的 model_name 参数
<br>
```
bash
HIP_VISIBLE_DEVICES
=
0 python inference.py
...
...
@@ -141,10 +157,9 @@ NV:绿色 DCU:红色
```
## 源码仓库及问题反馈
-
https://developer.hpccube.com/codes/modelzoo/
starcoder2
_pytorch
-
https://developer.hpccube.com/codes/modelzoo/
codellama
_pytorch
## 参考资料
-
https://developer.hpccube.com/codes/modelzoo/codellama_pytorch
-
https://hf-mirror.com/meta-llama/CodeLlama-7b-Instruct-hf
-
https://github.com/meta-llama/codellama
-
https://hf-mirror.com/datasets/emilgoh/verilog-dataset-v3
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment