Skip to content

GitLab

Commit 12415ddb authored by dongchy920's avatar dongchy920
Browse files

Initial commit

parents
Pipeline #2830 canceled with stages
Hide whitespace changes
Inline Side-by-side
Showing with 1196 additions and 0 deletions
LICENSE 0 → 100644
View file @ 12415ddb
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
\ No newline at end of file
MODEL_LICENSE 0 → 100644
View file @ 12415ddb
The aiXcoder Model License
1. Definitions
“Licensor” means the aiXcoder Model Team that distributes its Software.
“Software” means the aiXcoder model parameters made available under this license.
2. License Grant
Subject to the terms and conditions of this License, the Licensor hereby grants to you a non-exclusive, worldwide, non-transferable, non-sublicensable, revocable, royalty-free copyright license to use the Software solely for your non-commercial research purposes.
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
3. Restriction
You will not use, copy, modify, merge, publish, distribute, reproduce, or create derivative works of the Software, in whole or in part, for any commercial, military, or illegal purposes.
You will not use the Software for any act that may undermine China's national security and national unity, harm the public interest of society, or infringe upon the rights and interests of human beings.
4. Disclaimer
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
5. Limitation of Liability
EXCEPT TO THE EXTENT PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL THEORY, WHETHER BASED IN TORT, NEGLIGENCE, CONTRACT, LIABILITY, OR OTHERWISE WILL ANY LICENSOR BE LIABLE TO YOU FOR ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES, OR ANY OTHER COMMERCIAL LOSSES, EVEN IF THE LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
6. Dispute Resolution
This license shall be governed and construed in accordance with the laws of People’s Republic of China. Any dispute arising from or in connection with this License shall be submitted to Haidian District People's Court in Beijing.
Note that the license is subject to update to a more comprehensive version. For any questions related to the license and copyright, please contact us at license@aixcoder.com.
\ No newline at end of file
README.md 0 → 100644
View file @ 12415ddb
# Qwen2.5-Coder
## 论文
- https://arxiv.org/pdf/2410.13187
## 模型结构
aiXcoder-7B 是一个轻量级且高效的代码大语言模型，专为代码补全、生成和理解设计。支持多种编程语言，包括Python、Java、C/C++、JavaScript、TypeScript、HTML等，适用于多种开发场景。官方文档未完全公开其详细的模型结构超参数。可以参考huggingface上公布的配置参数。
## 算法原理
aiXcoder-7B 的性能归功于以下关键训练策略：
多目标训练（Multi-objective Training）：
- Next-Token Prediction (NTP)：模拟从头编写代码，预测下一个 token。
- Fill-in-the-Middle (FIM)：训练模型基于前后上下文预测中间代码片段，适合修改现有代码。
- Structured Fill-in-the-Middle (SFIM)：创新的训练目标，基于语法树选择完整代码片段（如函数、循环块）作为 middle，提高生成完整代码的能力。
数据采样策略：
- 考虑文件间关系（如文件内容相似性、依赖关系、路径相似性），增强跨文件上下文理解。
数据处理：
- 数据收集流程包括爬取、清洗、去重、代码质量检查和敏感信息移除，处理 2.8TB 自然语言数据和 3.5TB 源代码数据。
- 数据覆盖多种编程语言，确保多语言支持。
## 环境配置
### Docker（方法一）
从[光源](https://www.sourcefind.cn/#/service-list)中拉取docker镜像：
```
docker pull image.sourcefind.cn:5000/dcu/admin/base/custom:vllm0.8.5-ubuntu22.04-dtk25.04-rc7-das1.5-py3.10-20250521-fixpy-rocblas0521-beta2
```
创建容器并挂载目录进行开发：
```
docker run -it --name {name} --shm-size=1024G --device=/dev/kfd --device=/dev/dri/ --privileged --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --ulimit memlock=-1:-1 --ipc=host --network host --group-add video -v /opt/hyhal:/opt/hyhal:ro -v {}:{} {docker_image} /bin/bash
# 修改1 {name} 需要改为自定义名称
# 修改2 {docker_image} 需要需要创建容器的对应镜像名称
# 修改3 -v 挂载路径到容器指定路径
pip install -r requirements.txt
```
### Dockerfile（方法二）
```
cd docker
docker build --no-cache -t qwen2_5_coder:1.0 .
docker run -it --name {name} --shm-size=1024G --device=/dev/kfd --device=/dev/dri/ --privileged --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --ulimit memlock=-1:-1 --ipc=host --network host --group-add video -v /opt/hyhal:/opt/hyhal:ro -v {}:{} {docker_image} /bin/bash
pip install -r requirements.txt
```
### Anaconda（方法三）
线上节点推荐使用conda进行环境配置。
创建python=3.10的conda环境并激活
```
conda create -n qwen2_5_coder python=3.10
conda activate qwen2_5_coder
```
关于本项目DCU显卡所需的特殊深度学习库可从[光合](https://developer.hpccube.com/tool/)开发者社区下载安装。
```
DTK驱动：dtk25.04
python：python3.10
pytorch:2.4.1
torchvision:0.19.1
sentencepiece>=0.2.0
```
安装其他依赖包
```
pip install -r requirements.txt
```
下载预训练权重文件：[HuggingFace下载](https://huggingface.co/aiXcoder/aixcoder-7b-base)
可以通过huggingface-cli下载，下载命令：
```
pip install -U huggingface_hub
export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download --resume-download aiXcoder/aixcoder-7b-base --local-dir aiXcoder/aixcoder-7b-base
```
## 数据集
微调数据集：[bigcode/the-stack-smol](bigcode/the-stack-smol)
可以通过huggingface-cli下载，下载命令：
```
huggingface-cli download --repo-type dataset --resume-download bigcode/the-stack-smol --local-dir bigcode/the-stack-smol
```
## 训练
```
# 注意：
accelerate launch finetune.py \
--model_id "aiXcoder/aixcoder-7b-base" \
--dataset_name "bigcode/the-stack-smol" \
--subset "data/rust" \
--dataset_text_field "content" \
--split "train" \
--max_seq_length 1024 \
--max_steps 100 \
--micro_batch_size 1 \
--gradient_accumulation_steps 8 \
--learning_rate 5e-6 \
--warmup_steps 20 \
--fim_rate 0.5 \
--num_proc "$(nproc)"
```
## 推理
```
# 注意：修改为自己的模型路径
python sess_huggingface.py
```
## result
<div align=center>
<img src="./assets/infer.png"/>
</div>
## 精度
DCU与GPU精度一致，推理框架：pytorch。
## 应用场景
### 算法类别
代码生成
### 热点应用行业
代码生成,代码修改,教育
## 其他预训练权重
Instruct系列模型
[Qwen2.5-Coder-0.5B-Instruct](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-0.5B-Instruct)
[Qwen2.5-Coder-1.5B-Instruct](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-1.5B-Instruct)
[Qwen2.5-Coder-3B-Instruct](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-3B-Instruct)
[Qwen2.5-Coder-7B-Instruct](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-7B-Instruct)
[Qwen2.5-Coder-14B-Instruct](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-14B-Instruct)
Base系列模型
[Qwen2.5-Coder-0.5B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-0.5B)
[Qwen2.5-Coder-1.5B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-1.5B)
[Qwen2.5-Coder-3B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-3B)
[Qwen2.5-Coder-7B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-7B)
[Qwen2.5-Coder-14B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-14B)
[Qwen2.5-Coder-32B](http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-Coder-32B)
## 源码仓库及问题反馈
[https://developer.sourcefind.cn/codes/dongchy920/qwen2.5_coder](https://developer.sourcefind.cn/codes/dongchy920/qwen2.5_coder)
## 参考资料
[https://github.com/QwenLM/Qwen2.5-Coder](https://github.com/QwenLM/Qwen2.5-Coder)
[https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct)
README_CN.md 0 → 100644
View file @ 12415ddb
# aiXcoder-7B Code Large Language Model
<p align="center">
🏠 <a href="https://www.aixcoder.com/" target="_blank">官网</a>｜🛠 <a href="https://marketplace.visualstudio.com/items?itemName=aixcoder-plugin.aixcoder" target="_blank">VS Code 插件</a>｜🛠 <a href="https://plugins.jetbrains.com/plugin/13574-aixcoder-code-completer" target="_blank">Jetbrains 插件</a>｜🤗 <a href="https://huggingface.co/aiXcoder/aixcoder-7b-base" target="_blank">模型下载</a>｜<a href="./assets/wechat_1.jpg" target="_blank">技术交流群</a>｜<a href="./assets/wechat_2.jpg" target="_blank">公众号</a>
</p>
欢迎来到aiXcoder-7B代码大型语言模型的官方仓库。该模型旨在理解和生成多种编程语言中的代码，提供在代码完成、理解、生成以及更多关于编程语言的任务中的最先进性能。
目录
1. [模型简介](#模型简介)
2. [快速上手](#快速上手)
- [运行环境](#运行环境)
- [模型权重](#模型权重)
- [推理示例](#推理示例)
- [Bitsandbytes 量化执行](#bitsandbytes-量化执行)
- [微调示例](#微调示例)
3. [aiXcoder 7B 训练数据](#aixcoder-7b-训练数据)
4. [训练](#训练)
- [训练超参数](#训练超参数)
- [批量数据组织方式](#批量数据组织方式)
- [预训练任务](#预训练任务)
5. [实验结果](#实验结果)
- [NL2Code 基准测试](#nl2code-基准测试)
- [代码补全 (Fill in the Middle)](#代码补全-fill-in-the-middle)
- [跨文件代码生成](#跨文件代码生成)
6. [License](#license)
7. [Acknowledgments](#acknowledgments)
## 模型简介
随着代码大模型的能力逐渐被挖掘出来，aiXcoder 也一直在思考怎样才能令代码大模型在实际开发场景中有更大的帮助。为此，我们开源了 aiXcoder 7B Base，该模型在1.2T Unique Tokens上做了大量的训练，并且该模型的预训练任务及上下文信息都为真实代码生成场景做了独特的设计。
aiXcoder 7B Base 在代码补全场景下是所有同等级参数量模型中效果最好的，主流多语言nl2code benchmark 平均上效果也超过codellama 34B 和StarCoder2 15B。
在我们持续推动代码大模型应用的探索过程中，aiXcoder 7B Base 模型的发布标志着一个重要的里程碑。当前版本的 aiXcoder 7B Base 是一个基础模型，专注于提升代码补全和代码生成的效率与准确性，旨在为开发人员在这些场景下提供强有力的支持。值得注意的是，这个版本尚未经过特别的instruct微调，意味着它在特定的高级任务如测试用例生成和代码调试方面可能还未达到最优表现。
然而，我们已经在规划中包含了对aiXcoder模型系列的进一步发展。在不久的将来，我们计划发布新的模型版本，这些版本将经过精心的Instruct微调，专门针对更广泛的编程任务，包括但不限于测试用例生成和代码调试。通过这些经过Instruct微调的模型，我们期待能够为开发者提供更全面、更深入的编程支持，帮助他们在软件开发的每一个阶段都能发挥出最大的效率。
![table_1](./assets/table_1.png)
> aiXcoder 7B surpasses mainstream models in nl2code benchmark. aiXcoder-7B is an enhancement of aiXcoder-7B-Base, fine-tuned on one hundred thousand data entries similar to Evol-instruct for one epoch.
<br>
<br>
![table_3](./assets/table_3.png)
> aiXcoder 7B Base surpasses mainstream models in code completion scenarios.
<br>
<br>
## 快速上手
### 运行环境
#### 选择一：构建一个运行环境
主要的环境依赖为：
- Python 3.8 or higher
- PyTorch 2.1.0 or higher
- sentencepiece 0.2.0 or higher
- transformers 4.34.1 or higher (if run inference by transformers library)
在支持CUDA环境的宿主机或者容器内，执行以下命令安装环境依赖项：
```bash
conda create -n aixcoder-7b python=3.11
conda activate aixcoder-7b
git clone git@github.com:aixcoder-plugin/aiXcoder-7b.git
cd aiXcoder-7b
pip install -r requirements.txt
```
`requirements.txt` 列举了所有的依赖项及其版本号。
如果想要加快推理速度，我们强烈建议安装 FlashAttention 库（可选）。在确定您的芯片版本与CUDA版本支持FlashAttention 的条件下，可通过以下步骤进行安装：
```bash
git clone git@github.com:Dao-AILab/flash-attention.git
cd flash-attention
MAX_JOBS=8 python setup.py install
```
#### Option 2: Docker
为了更好地隔离开发环境，我们建议您可以在 Docker 容器内运行模型推理。如下是启动准备 docker 环境的步骤：
1. 安装 Docker：如果您的机器还没有安装Docker，您可以参考官方的安装步骤安装。
2. 拉取镜像: 从 Docker Hub 拉取 PyTorch 镜像。
```bash
docker pull pytorch/pytorch:2.1.0-cuda11.8-cudnn8-devel
```
3. 启动容器: 拉取docker 镜像后，可以启动容器，并在容器中运行模型。
```bash
docker run --gpus all -it -v /dev/shm:/dev/shm --name aix_instance pytorch/pytorch:2.1.0-cuda11.8-cudnn8-devel /bin/bash
pip install sentencepiece
git clone git@github.com:aixcoder-plugin/aiXcoder-7b.git
cd aiXcoder-7b
```
如果想要加快推理速度，我们强烈建议安装 FlashAttention 库（可选）。在确定您的芯片版本与CUDA版本支持FlashAttention 的条件下，可通过以下步骤进行安装：
```bash
git clone git@github.com:Dao-AILab/flash-attention.git
cd flash-attention
MAX_JOBS=8 python setup.py install
```
4. 模型推理: 在容器内，您可以安装推理示例代码进行预测。
### 模型权重
您能从以下地址下载模型：
- [aiXcoder Base Download](https://huggingface.co/aiXcoder/aixcoder-7b-base)
- aiXcoder Instruct Download (Comming soon...)
### 推理示例
#### 命令行执行
如果需要快速执行，只需要通过以下命令行即可运行推理样本:
```bash
torchrun --nproc_per_node 1 sess_megatron.py --model_dir "path/to/model_weights_dir"
```
将 "path/to/model_weights_dir" 替换为您下载模型权重后的本地地址。
或者通过 huggingface 的 transformers 库进行推理测试：
```bash
python sess_huggingface.py
```
#### Python 脚本
如果您想嵌入自己的工具流，或者获得更灵活的使用方式，您能通过以下代码直接调用：
```python
from sess_megatron import TestInference
infer = TestInference()
res = infer.run_infer(
# for FIM style input, code_string stands for prefix context
code_string="""# 快速排序算法""",
# for FIM style input, later_code stands for suffix context
later_code="\n",
# file_path should be a path from project to file
file_path="test.py",
# max num for generated tokens
max_new_tokens=256,
)
print(res)
"""output:
def quick_sort(arr):
if len(arr) <= 1:
return arr
pivot = arr[0]
less = [i for i in arr[1:] if i <= pivot]
greater = [i for i in arr[1:] if i > pivot]
return quick_sort(less) + [pivot] + quick_sort(greater)
# 测试
arr = [3, 2, 1, 4, 5]
print(quick_sort(arr)) # [1, 2, 3, 4, 5]
"""
```
```python
import torch
import sys
from hf_mini.utils import input_wrapper
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto
tokenizer = AutoTokenizer.from_pretrained("aiXcoder/aixcoder-7b-base")
model = AutoModelForCausalLM.from_pretrained("aiXcoder/aixcoder-7b-base", torch_dtype=torch.bfloat16)
text = input_wrapper(
# for FIM style input, code_string stands for prefix context
code_string="# 快速排序算法",
# for FIM style input, later_code stands for suffix context
later_code="\n# 测试\narr = [3, 2, 1, 4, 5]\nprint(quick_sort(arr)) # [1, 2, 3, 4, 5]",
# file_path should be a path from project to file
path="test.py"
)
if len(text) == 0:
sys.exit()
inputs = tokenizer(text, return_tensors="pt", return_token_type_ids=False)
inputs = inputs.to(device)
model.to(device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))
"""output:
def quick_sort(arr):
# 如果数组长度小于等于1，直接返回
if len(arr) <= 1:
return arr
# 选择数组的第一个元素作为基准
pivot = arr[0]
# 初始化左右指针
left, right = 1, len(arr) - 1
# 循环直到左指针小于右指针
while left < right:
# 从右到左找到第一个小于基准的元素，与左指针元素交换
if arr[right] < pivot:
arr[left], arr[right] = arr[right], arr[left]
left += 1
# 从左到右找到第一个大于等于基准的元素，与右指针元素交换
if arr[left] >= pivot:
right -= 1
# 将基准元素与左指针元素交换
arr[left], arr[0] = arr[0], arr[left]
# 对左半部分进行递归排序
quick_sort(arr[:left])
# 对右半部分进行递归排序
quick_sort(arr[left + 1:])
return arr</s>
"""
```
### Bitsandbytes 量化执行
我们也能通过 `pip install bitsandbytes accelerate` 安装 Bitsandbytes，并简单加加上配置项执行int8或int4的量化推理（如果需要进一步压缩运行时申请的临时显存，推荐安装 FlashAttention）：
```python
import sys
import torch
from hf_mini.utils import input_wrapper
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
# to use 4bit use `load_in_4bit=True` instead
bnb_config = BitsAndBytesConfig(load_in_8bit=True)
device = "cuda" # the device to load the model onto
tokenizer = AutoTokenizer.from_pretrained("aiXcoder/aixcoder-7b-base")
model = AutoModelForCausalLM.from_pretrained("aiXcoder/aixcoder-7b-base", quantization_config=bnb_config, device_map=device, attn_implementation='flash_attention_2')
text = input_wrapper(
code_string="# 快速排序算法",
later_code="\n",
path="test.py"
)
if len(text) == 0:
sys.exit()
inputs = tokenizer(text, return_tensors="pt", return_token_type_ids=False)
inputs = inputs.to(device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(f"Model memory footprint: {model.get_memory_footprint() / 2**20:.2f} MB")
print(f"Torch max memory allocated: {torch.cuda.max_memory_allocated() / 2**20:.2f} MB")
"""
load_in_4bit=True:
- Model memory footprint: 5656.52 MB
- Torch max memory allocated: 6448.89 MB
load_in_8bit=True:
- Model memory footprint: 9008.52 MB
- Torch max memory allocated: 10061.51 MB
"""
```
### 微调示例
如果希望针对自有代码进行微调，可以借助 Huggingface 的 PEFT 工具快速上手训练。在此之前你需要先安装依赖库 `pip install -r requirements_peft.txt`。
然后执行训练命令：
```bash
accelerate launch finetune.py \
--model_id "aiXcoder/aixcoder-7b-base" \
--dataset_name "bigcode/the-stack-smol" \
--subset "data/rust" \
--dataset_text_field "content" \
--split "train" \
--max_seq_length 1024 \
--max_steps 10000 \
--micro_batch_size 1 \
--gradient_accumulation_steps 8 \
--learning_rate 5e-6 \
--warmup_steps 20 \
--fim_rate 0.5 \
--num_proc "$(nproc)"
```
在微调脚本中，我们构造了简单的随机 FIM 训练任务，可以训练模型在自有数据上的补全与生成能力。需要注意的是，aiXcoder-7b-base 在预训练中采用的是[结构化 FIM](#预训练任务)，即将一个完整代码块构造成 MIDDLE，不过构造这样的训练数据涉及到语法解析，可能需要开发者自行实现。
## aiXcoder 7B 训练数据
aiXcoder 的数据分为核心数据集与扩展数据集，核心数据集由业务上常用的几大编程语言，以及与代码息息相关的自然语言组成。核心数据集的编程语言主要有 C++、Python、Java、JavaScript等近百种主流编程语言，自然语言上主要由 StackOverFlow 问答、技术博客、代码文档、计算机领域论文等组成。扩展数据集主要由过滤后的代码开源数据集，英文自然语言高质量数据集，中文自然语言高质量数据集组成。
<!-- <br>
<br>
![table_0](./assets/table_0.png)
<br>
<br> -->
aiXcoder 核心数据集主要用于强化代码大模型在以上编程语言上的效果，其经过大量的过滤与筛选过程。具体而言主要分为如下几步：1) 原始数据挑选； 2) 对项目进行综合排序，并筛选；3) 基于 MinHashes(Broder, 2000) 等方法进行代码去重、去除自动生成的代码；4) 个人敏感信息识别与处理；5) 清理被注释的代码；6) 语法分析过滤不正确或者异常的代码文件；7）结合静态分析工具，检测并排除 Java\CPP\Python\JS 等主流编程语言中高风险的163 种 bug 和197 种缺陷。
1. 原始数据挑选
- 排除 copyleft licenses 项目
- 对 aiXcoder 在各大代码托管平台上抓取的数据及开源数据做项目去重
2. 项目级的综合排序
- 统计项目的Star量、Git Commit 数量、Test文件数量，并综合评分
- 排除综合评分最低的10%数据
3. 代码文件级筛选
- 删除自动生成代码
- near-deduplication去重
4. 敏感信息去除
- 命名实体模型识别并删除人名、IP、账号密码、网址等敏感信息
5. 被注释的代码
- 按比例随机删除被注释的大段代码
6. 语法分析
- 删除主流数十种语言存在语法解析错误或者语法错误的代码
7. 静态分析
- 结合静态分析工具，扫描并定位影响代码可靠性和可维护性的161种Bug，影响代码安全性的197种漏洞
```python
# "__init__" method should not return a value
# Noncompliant: a TypeError will be raised
class MyClass(object):
def __init__(self):
self.message = 'HelloWorld'
return self
# Compliant solution
class MyClass(object):
def __init__(self):
self.message = 'HelloWorld'
```
上述代码展示了python中的一种bug模式，即 __init__ 方法不应该返回值。
## 训练
### 训练超参数
分词器:
- 基于字节码的 BPE 分词器
- 词表大小：49,152
模型结构:
- RoPE (Rotary Positional Embedding) 相对位置编码
- 中间全连接层采用 SwiGLU
- Grouped Query Attention
训练配置:
- 70% 为结构化 FIM (Fill in the middle)训练任务，30% 为自回归语言模型任务；
- BFloat 16 数据类型
- AdamW 优化器，学习率最大1e-5，最小 1e-6，采用余弦衰减
- 预训练长度为 32,768
### 批量数据组织方式
我们的代码数据在完成预处理之后会以项目为单位组织起来，项目内的文件排序既考虑了规则也考虑了随机性。具体而言，我们会尝试使用CallingGraph、K-Means聚类、文件路径相似性、TF-IDE距离等将相似或相依赖的代码文件聚在一起，让模型更理解代码文件之间的相关性。但与此同时，代码文件的排序也需要考虑随机性，因为在实际编程场景中，项目并不是完整的，相似性代码或者依赖性代码也许还没有完善。
在保证项目代码文件整体具有随机性，局部具有相似关系或依赖关系之后，我们将项目代码文件拉长为一个向量，使用Transformer-XL的方式组织Batch之间的顺序。尽管预训练过程中单个Batch的序列长度已经达到了32768，但通过这种方式仍然能将可见序列长度扩展到更长。
### 预训练任务
与其它自然语言大模型或代码大模型不同，在代码编程场景下， aiXcoder 考虑了代码本身的结构性特点，尽量让模型预测出完整的代码节点。简单而言，aiXcoder 训练任务结合了 fill in the middle(FIM, Bavarian et al., 2022) and parser generator tool，在构造训练数据时我们会将代码解析为抽象语法树，并随机截取一个完整的节点构造 FIM。这样做的原因首先我们需要保证输入数据相对完整，前序与后续都处于同一层级。其次，我们也需要模型预测结果更加完整，生成的代码具有完整层次结构。
```python
for i in range(20):
if i % 5 == 0:
print("Hello World")
```
![table_0](./assets/graphviz.svg)
> 如上简单的代码能解析为抽象语法树，我们会根据抽象语法树的节点构造结构化的 FIM 训练任务。
<br>
<br>
假设我们选中了上述代码中的 IF 结点，那么我们将从 IF 结点及其子树构造出训练样本。如下两条例子是等价的：
```bash
# fill in the middle, SPM mode
"<s>▁<AIX-SPAN-PRE>▁<AIX-SPAN-POST> print(\"Hello World\")\n▁<AIX-SPAN-MIDDLE># the file path is: test.py\n# the code file is written by Python\nfor i in range(20):\n if i % 5 == 0:<\s>"
# fill in the middle, PSM mode
"<s>▁<AIX-SPAN-PRE># the file path is: test.py\n# the code file is written by Python\nfor i in range(20):\n if ▁<AIX-SPAN-POST> print(\"Hello World\")\n▁<AIX-SPAN-MIDDLE>i % 5 == 0:<\s>"
```
## 实验结果
### NL2Code 基准测试
表1 展示了 aiXcoder-7B Base 模型在独立方法生成基准上的表现，我们的模型在各大预训练基础模型中表现很好，在百亿级参数量下拥有当前最好的效果。
![table_1](./assets/table_1.png)
### 代码补全 (Fill in the Middle)
与 Table 1 的 Stand alone nl2code 不同，在实际编程场景中，我们更需要考虑光标上下文的代码补全能力。一般而言，各种开源代码大模型都会考虑在预训练中加入Fill in the middle(FIM) 模式，来强化模型在考虑代码上下文的场景下生成更准确的结果。为此，我们将以FIM为默认代码补全方式，评测各个模型在实际编程场景中的能力。
当前考虑上下文的代码补全，主流评测集是 Santacoder(Ben Allal et al., 2023) 提出来的单行评测方法。该评测集会从HumanEval 或者 MultiPL-E 中抽取的单行代码，然后在给定完整的上文与下文条件下，评估模型生成结果的Exact Match指标。
![table_2](./assets/table_2.png)
为了进一步精细地评测代码大模型在代码补全上的能力，aiXcoder 构建了一个数据量更大，被测代码多样性更高、被测代码上下文长度更长、更接近实际开发项目的评测集，该评测集也将同步在GitHub上开源。在评估过程中，我们保证不同代码大模型之间采用相同的16K最大序列长度，并且评估不同场景下的生成效果，例如生成完整方法块、条件判断块、循环处理块、异常捕捉块等十三种情况。
Table 3 展示了不同模型在不同语言上的平均生成效果，最终的评估结果是所有补全场景与评测样本的均值。aiXcoder 7B Base 模型在各大编程语言，各种评估标准下都是效果最好的，这表明aiXcoder 7B Base 在最基础的代码补全能力上是所有同量级开源模型最好的，最适合用于实际编程场景中提供代码补全能力的基础模型。
![table_3](./assets/table_3.png)
对于 Table 3 中每一条评测结果，其都有更为详细的评测维度。Table 4 到 7 展示了不同模型在不同语言上多维度评估的细节：
- **Method signature**: 表示模型根据上下文生成方法签名；
- **Method body**: 表示模型根据上下文，包括函数签名，生成完整的方法体；
- **Single line**: 表示单行代码补全；
- **Method with comment**: 表示根据上下文，包括函数签名与函数注释，生成对应的函数体；
- **Empty**: 表示在上下文完整的情况下，模型需要预测为空；
- **Method body top, mid, bottom**: 表示分别在函数体上半部分，函数体中间部分，函数体下半部分的代码生成效果；
- **If, for, while, try, switch statement**: 表示生成条件代码块、循环代码块、异常捕捉代码块、条件分支代码块的效果;
![table_4](./assets/table_4.png)
![table_5](./assets/table_5.png)
![table_6](./assets/table_6.png)
![table_7](./assets/table_7.png)
### 跨文件代码生成
代码大模型另一个比较重要的能力是跨文件的代码上下文理解能力，因为开发者在实际编写项目中，经常需要考虑当前项目其它文件内的信息。因此我们采用了CrossCodeEval (Ding et al., 2023)评测集，来评估模型提取跨文件上下文信息的能力。
在 Table 8 中，我们固定所有模型的上下文长度为 16K，并通过 FIM 中的 PSM 模式作为输入构造的格式，模型完成推理后，所有输出结果都以 Greedy Search 的方式解码。首先作为 Baseline，我们在单文件的情况下评测各代码大模型的生成能力。
然后在 BM25 为相似性指标的情况下，通过上文搜索项目内最相似的三个代码块作为prompt，再次评估模型的生成效果。最后，w/Ref. 表示假设我们知道正确的References 代码是什么样的，然后通过 References 搜索项目内最相似的三个代码作为prompt，再次评估模型的生成效果。
最终 aiXcoder-7B 模型在所有语言上的效果都是很好的，这证明了我们模型在提取上下文信息上，尤其是跨文件的上下文信息的能力。
![table_8](./assets/table_8.png)
## License
The source code in this repository is licensed under the [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0) License - see the LICENSE file for details.
The model weights are licensed under the [Model License](./MODEL_LICENSE) for academic research use; for commercial use, please apply by sending an email to support@aixcoder.com.
## Acknowledgments
We would like to thank all contributors to the open-source projects and datasets that made this work possible.
For any questions or issues, please open an issue on this repository.
Thank you for your interest in our Code Large Language Model. We look forward to your contributions and feedback!
\ No newline at end of file
aiXcoder_extended_dataset/aiXcoder_extended_cplus.jsonl 0 → 100644
View file @ 12415ddb
This source diff could not be displayed because it is too large. You can view the blob instead.
aiXcoder_extended_dataset/aiXcoder_extended_java.jsonl 0 → 100644
View file @ 12415ddb
This source diff could not be displayed because it is too large. You can view the blob instead.
aiXcoder_extended_dataset/aiXcoder_extended_javascript.jsonl 0 → 100644
View file @ 12415ddb
This source diff could not be displayed because it is too large. You can view the blob instead.
aiXcoder_extended_dataset/aiXcoder_extended_python.jsonl 0 → 100644
View file @ 12415ddb
This source diff could not be displayed because it is too large. You can view the blob instead.
assets/graphviz.svg 0 → 100644
View file @ 12415ddb
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 2.43.0 (0)
-->
<!-- Title: %3 Pages: 1 -->
<svg width="672pt" height="530pt"
viewBox="0.00 0.00 671.81 529.96" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 525.96)">
<title>%3</title>
<polygon fill="white" stroke="transparent" points="-4,4 -4,-525.96 667.81,-525.96 667.81,4 -4,4"/>
<!-- 139720528084944 -->
<g id="node1" class="node">
<title>139720528084944</title>
<ellipse fill="none" stroke="black" cx="221.4" cy="-503.96" rx="45.49" ry="18"/>
<text text-anchor="middle" x="221.4" y="-500.26" font-family="Times,serif" font-size="14.00">Module</text>
</g>
<!-- 139720528084896 -->
<g id="node2" class="node">
<title>139720528084896</title>
<ellipse fill="none" stroke="black" cx="221.4" cy="-420.96" rx="27" ry="18"/>
<text text-anchor="middle" x="221.4" y="-417.26" font-family="Times,serif" font-size="14.00">For</text>
</g>
<!-- 139720528084944&#45;&gt;139720528084896 -->
<g id="edge1" class="edge">
<title>139720528084944&#45;&gt;139720528084896</title>
<path fill="none" stroke="black" d="M221.4,-485.78C221.4,-475.15 221.4,-461.27 221.4,-449.16"/>
<polygon fill="black" stroke="black" points="224.9,-449.11 221.4,-439.11 217.9,-449.11 224.9,-449.11"/>
<text text-anchor="middle" x="234.4" y="-459.96" font-family="Times,serif" font-size="10.00">body</text>
</g>
<!-- 139720528084848 -->
<g id="node3" class="node">
<title>139720528084848</title>
<ellipse fill="none" stroke="black" cx="64.4" cy="-329.09" rx="41.94" ry="26.74"/>
<text text-anchor="middle" x="64.4" y="-332.89" font-family="Times,serif" font-size="14.00">Name</text>
<text text-anchor="middle" x="64.4" y="-317.89" font-family="Times,serif" font-size="14.00">i</text>
</g>
<!-- 139720528084896&#45;&gt;139720528084848 -->
<g id="edge2" class="edge">
<title>139720528084896&#45;&gt;139720528084848</title>
<path fill="none" stroke="black" d="M201.51,-408.58C177.03,-394.56 134.88,-370.43 103.75,-352.62"/>
<polygon fill="black" stroke="black" points="105.3,-349.47 94.89,-347.54 101.83,-355.55 105.3,-349.47"/>
<text text-anchor="middle" x="172.9" y="-376.96" font-family="Times,serif" font-size="10.00">target</text>
</g>
<!-- 139720528084752 -->
<g id="node5" class="node">
<title>139720528084752</title>
<ellipse fill="none" stroke="black" cx="221.4" cy="-329.09" rx="41.02" ry="26.74"/>
<text text-anchor="middle" x="221.4" y="-332.89" font-family="Times,serif" font-size="14.00">Call</text>
<text text-anchor="middle" x="221.4" y="-317.89" font-family="Times,serif" font-size="14.00">range</text>
</g>
<!-- 139720528084896&#45;&gt;139720528084752 -->
<g id="edge4" class="edge">
<title>139720528084896&#45;&gt;139720528084752</title>
<path fill="none" stroke="black" d="M221.4,-402.63C221.4,-392.35 221.4,-378.92 221.4,-366.42"/>
<polygon fill="black" stroke="black" points="224.9,-366.22 221.4,-356.22 217.9,-366.22 224.9,-366.22"/>
<text text-anchor="middle" x="230.4" y="-376.96" font-family="Times,serif" font-size="10.00">iter</text>
</g>
<!-- 139720528084656 -->
<g id="node8" class="node">
<title>139720528084656</title>
<ellipse fill="none" stroke="black" cx="377.4" cy="-329.09" rx="27" ry="18"/>
<text text-anchor="middle" x="377.4" y="-325.39" font-family="Times,serif" font-size="14.00">If</text>
</g>
<!-- 139720528084896&#45;&gt;139720528084656 -->
<g id="edge7" class="edge">
<title>139720528084896&#45;&gt;139720528084656</title>
<path fill="none" stroke="black" d="M241.15,-408.58C268.32,-392.93 317.37,-364.67 348.61,-346.67"/>
<polygon fill="black" stroke="black" points="350.39,-349.69 357.31,-341.66 346.89,-343.62 350.39,-349.69"/>
<text text-anchor="middle" x="312.4" y="-376.96" font-family="Times,serif" font-size="10.00">body</text>
</g>
<!-- 139720529985008 -->
<g id="node4" class="node">
<title>139720529985008</title>
<ellipse fill="none" stroke="black" cx="36.4" cy="-228.35" rx="36.29" ry="18"/>
<text text-anchor="middle" x="36.4" y="-224.65" font-family="Times,serif" font-size="14.00">Store</text>
</g>
<!-- 139720528084848&#45;&gt;139720529985008 -->
<g id="edge3" class="edge">
<title>139720528084848&#45;&gt;139720529985008</title>
<path fill="none" stroke="black" d="M57.11,-302.41C53.07,-288.15 48.05,-270.45 43.96,-256.03"/>
<polygon fill="black" stroke="black" points="47.28,-254.89 41.18,-246.22 40.54,-256.8 47.28,-254.89"/>
<text text-anchor="middle" x="59.4" y="-276.22" font-family="Times,serif" font-size="10.00">ctx</text>
</g>
<!-- 139720528084704 -->
<g id="node6" class="node">
<title>139720528084704</title>
<ellipse fill="none" stroke="black" cx="132.4" cy="-228.35" rx="41.94" ry="26.74"/>
<text text-anchor="middle" x="132.4" y="-232.15" font-family="Times,serif" font-size="14.00">Name</text>
<text text-anchor="middle" x="132.4" y="-217.15" font-family="Times,serif" font-size="14.00">range</text>
</g>
<!-- 139720528084752&#45;&gt;139720528084704 -->
<g id="edge5" class="edge">
<title>139720528084752&#45;&gt;139720528084704</title>
<path fill="none" stroke="black" d="M201.2,-305.68C188.89,-292.02 172.99,-274.38 159.57,-259.5"/>
<polygon fill="black" stroke="black" points="162.12,-257.09 152.82,-252.01 156.92,-261.78 162.12,-257.09"/>
<text text-anchor="middle" x="191.9" y="-276.22" font-family="Times,serif" font-size="10.00">func</text>
</g>
<!-- 139720528084608 -->
<g id="node7" class="node">
<title>139720528084608</title>
<ellipse fill="none" stroke="black" cx="249.4" cy="-228.35" rx="57.05" ry="26.74"/>
<text text-anchor="middle" x="249.4" y="-232.15" font-family="Times,serif" font-size="14.00">Constant</text>
<text text-anchor="middle" x="249.4" y="-217.15" font-family="Times,serif" font-size="14.00">20</text>
</g>
<!-- 139720528084752&#45;&gt;139720528084608 -->
<g id="edge6" class="edge">
<title>139720528084752&#45;&gt;139720528084608</title>
<path fill="none" stroke="black" d="M228.68,-302.41C231.9,-291.07 235.73,-277.54 239.22,-265.23"/>
<polygon fill="black" stroke="black" points="242.69,-265.83 242.05,-255.25 235.96,-263.92 242.69,-265.83"/>
<text text-anchor="middle" x="247.9" y="-276.22" font-family="Times,serif" font-size="10.00">args</text>
</g>
<!-- 139720528084560 -->
<g id="node9" class="node">
<title>139720528084560</title>
<ellipse fill="none" stroke="black" cx="377.4" cy="-228.35" rx="52.79" ry="18"/>
<text text-anchor="middle" x="377.4" y="-224.65" font-family="Times,serif" font-size="14.00">Compare</text>
</g>
<!-- 139720528084656&#45;&gt;139720528084560 -->
<g id="edge8" class="edge">
<title>139720528084656&#45;&gt;139720528084560</title>
<path fill="none" stroke="black" d="M377.4,-310.89C377.4,-295.88 377.4,-273.85 377.4,-256.47"/>
<polygon fill="black" stroke="black" points="380.9,-256.35 377.4,-246.35 373.9,-256.35 380.9,-256.35"/>
<text text-anchor="middle" x="386.9" y="-276.22" font-family="Times,serif" font-size="10.00">test</text>
</g>
<!-- 139720528084128 -->
<g id="node16" class="node">
<title>139720528084128</title>
<ellipse fill="none" stroke="black" cx="485.4" cy="-228.35" rx="33.29" ry="18"/>
<text text-anchor="middle" x="485.4" y="-224.65" font-family="Times,serif" font-size="14.00">Expr</text>
</g>
<!-- 139720528084656&#45;&gt;139720528084128 -->
<g id="edge15" class="edge">
<title>139720528084656&#45;&gt;139720528084128</title>
<path fill="none" stroke="black" d="M392.87,-313.94C410.87,-297.49 440.78,-270.14 461.71,-251.01"/>
<polygon fill="black" stroke="black" points="464.23,-253.45 469.25,-244.12 459.5,-248.28 464.23,-253.45"/>
<text text-anchor="middle" x="448.4" y="-276.22" font-family="Times,serif" font-size="10.00">body</text>
</g>
<!-- 139720528084512 -->
<g id="node10" class="node">
<title>139720528084512</title>
<ellipse fill="none" stroke="black" cx="200.4" cy="-127.61" rx="39.79" ry="18"/>
<text text-anchor="middle" x="200.4" y="-123.91" font-family="Times,serif" font-size="14.00">BinOp</text>
</g>
<!-- 139720528084560&#45;&gt;139720528084512 -->
<g id="edge9" class="edge">
<title>139720528084560&#45;&gt;139720528084512</title>
<path fill="none" stroke="black" d="M350.9,-212.57C319.6,-195.11 267.53,-166.06 233.58,-147.12"/>
<polygon fill="black" stroke="black" points="234.95,-143.88 224.51,-142.06 231.54,-149.99 234.95,-143.88"/>
<text text-anchor="middle" x="302.9" y="-175.48" font-family="Times,serif" font-size="10.00">left</text>
</g>
<!-- 139720529982752 -->
<g id="node14" class="node">
<title>139720529982752</title>
<ellipse fill="none" stroke="black" cx="285.4" cy="-127.61" rx="27" ry="18"/>
<text text-anchor="middle" x="285.4" y="-123.91" font-family="Times,serif" font-size="14.00">Eq</text>
</g>
<!-- 139720528084560&#45;&gt;139720529982752 -->
<g id="edge13" class="edge">
<title>139720528084560&#45;&gt;139720529982752</title>
<path fill="none" stroke="black" d="M362.2,-211.05C346.84,-194.56 323,-168.97 305.95,-150.67"/>
<polygon fill="black" stroke="black" points="308.49,-148.26 299.11,-143.33 303.36,-153.03 308.49,-148.26"/>
<text text-anchor="middle" x="343.9" y="-175.48" font-family="Times,serif" font-size="10.00">ops</text>
</g>
<!-- 139720528084176 -->
<g id="node15" class="node">
<title>139720528084176</title>
<ellipse fill="none" stroke="black" cx="387.4" cy="-127.61" rx="57.05" ry="26.74"/>
<text text-anchor="middle" x="387.4" y="-131.41" font-family="Times,serif" font-size="14.00">Constant</text>
<text text-anchor="middle" x="387.4" y="-116.41" font-family="Times,serif" font-size="14.00">0</text>
</g>
<!-- 139720528084560&#45;&gt;139720528084176 -->
<g id="edge14" class="edge">
<title>139720528084560&#45;&gt;139720528084176</title>
<path fill="none" stroke="black" d="M379.14,-210.15C380.4,-197.67 382.16,-180.33 383.73,-164.85"/>
<polygon fill="black" stroke="black" points="387.25,-164.81 384.77,-154.51 380.28,-164.11 387.25,-164.81"/>
<text text-anchor="middle" x="415.4" y="-175.48" font-family="Times,serif" font-size="10.00">comparators</text>
</g>
<!-- 139720528084464 -->
<g id="node11" class="node">
<title>139720528084464</title>
<ellipse fill="none" stroke="black" cx="109.4" cy="-26.87" rx="41.94" ry="26.74"/>
<text text-anchor="middle" x="109.4" y="-30.67" font-family="Times,serif" font-size="14.00">Name</text>
<text text-anchor="middle" x="109.4" y="-15.67" font-family="Times,serif" font-size="14.00">i</text>
</g>
<!-- 139720528084512&#45;&gt;139720528084464 -->
<g id="edge10" class="edge">
<title>139720528084512&#45;&gt;139720528084464</title>
<path fill="none" stroke="black" d="M185.78,-110.75C172.71,-96.56 153.2,-75.4 137.17,-58"/>
<polygon fill="black" stroke="black" points="139.44,-55.3 130.09,-50.32 134.29,-60.04 139.44,-55.3"/>
<text text-anchor="middle" x="166.9" y="-74.74" font-family="Times,serif" font-size="10.00">left</text>
</g>
<!-- 139720529984048 -->
<g id="node12" class="node">
<title>139720529984048</title>
<ellipse fill="none" stroke="black" cx="200.4" cy="-26.87" rx="31.4" ry="18"/>
<text text-anchor="middle" x="200.4" y="-23.17" font-family="Times,serif" font-size="14.00">Mod</text>
</g>
<!-- 139720528084512&#45;&gt;139720529984048 -->
<g id="edge11" class="edge">
<title>139720528084512&#45;&gt;139720529984048</title>
<path fill="none" stroke="black" d="M200.4,-109.41C200.4,-94.4 200.4,-72.37 200.4,-54.99"/>
<polygon fill="black" stroke="black" points="203.9,-54.87 200.4,-44.87 196.9,-54.87 203.9,-54.87"/>
<text text-anchor="middle" x="206.9" y="-74.74" font-family="Times,serif" font-size="10.00">op</text>
</g>
<!-- 139720528084224 -->
<g id="node13" class="node">
<title>139720528084224</title>
<ellipse fill="none" stroke="black" cx="306.4" cy="-26.87" rx="57.05" ry="26.74"/>
<text text-anchor="middle" x="306.4" y="-30.67" font-family="Times,serif" font-size="14.00">Constant</text>
<text text-anchor="middle" x="306.4" y="-15.67" font-family="Times,serif" font-size="14.00">5</text>
</g>
<!-- 139720528084512&#45;&gt;139720528084224 -->
<g id="edge12" class="edge">
<title>139720528084512&#45;&gt;139720528084224</title>
<path fill="none" stroke="black" d="M216.96,-111.18C232.04,-97.13 254.75,-75.98 273.51,-58.5"/>
<polygon fill="black" stroke="black" points="276.34,-60.65 281.28,-51.27 271.57,-55.53 276.34,-60.65"/>
<text text-anchor="middle" x="269.9" y="-74.74" font-family="Times,serif" font-size="10.00">right</text>
</g>
<!-- 139720528084080 -->
<g id="node17" class="node">
<title>139720528084080</title>
<ellipse fill="none" stroke="black" cx="499.4" cy="-127.61" rx="36.54" ry="26.74"/>
<text text-anchor="middle" x="499.4" y="-131.41" font-family="Times,serif" font-size="14.00">Call</text>
<text text-anchor="middle" x="499.4" y="-116.41" font-family="Times,serif" font-size="14.00">print</text>
</g>
<!-- 139720528084128&#45;&gt;139720528084080 -->
<g id="edge16" class="edge">
<title>139720528084128&#45;&gt;139720528084080</title>
<path fill="none" stroke="black" d="M487.84,-210.15C489.6,-197.67 492.06,-180.33 494.26,-164.85"/>
<polygon fill="black" stroke="black" points="497.79,-164.9 495.72,-154.51 490.85,-163.92 497.79,-164.9"/>
<text text-anchor="middle" x="507.4" y="-175.48" font-family="Times,serif" font-size="10.00">value</text>
</g>
<!-- 139720528084032 -->
<g id="node18" class="node">
<title>139720528084032</title>
<ellipse fill="none" stroke="black" cx="461.4" cy="-26.87" rx="41.94" ry="26.74"/>
<text text-anchor="middle" x="461.4" y="-30.67" font-family="Times,serif" font-size="14.00">Name</text>
<text text-anchor="middle" x="461.4" y="-15.67" font-family="Times,serif" font-size="14.00">print</text>
</g>
<!-- 139720528084080&#45;&gt;139720528084032 -->
<g id="edge17" class="edge">
<title>139720528084080&#45;&gt;139720528084032</title>
<path fill="none" stroke="black" d="M489.71,-101.44C485.19,-89.69 479.73,-75.52 474.82,-62.76"/>
<polygon fill="black" stroke="black" points="478.06,-61.41 471.2,-53.34 471.52,-63.93 478.06,-61.41"/>
<text text-anchor="middle" x="493.9" y="-74.74" font-family="Times,serif" font-size="10.00">func</text>
</g>
<!-- 139720528083984 -->
<g id="node19" class="node">
<title>139720528083984</title>
<ellipse fill="none" stroke="black" cx="592.4" cy="-26.87" rx="71.34" ry="26.74"/>
<text text-anchor="middle" x="592.4" y="-30.67" font-family="Times,serif" font-size="14.00">Constant</text>
<text text-anchor="middle" x="592.4" y="-15.67" font-family="Times,serif" font-size="14.00">Hello World</text>
</g>
<!-- 139720528084080&#45;&gt;139720528083984 -->
<g id="edge18" class="edge">
<title>139720528084080&#45;&gt;139720528083984</title>
<path fill="none" stroke="black" d="M520.04,-104.7C532.43,-91.54 548.42,-74.56 562.22,-59.91"/>
<polygon fill="black" stroke="black" points="564.89,-62.18 569.2,-52.5 559.79,-57.38 564.89,-62.18"/>
<text text-anchor="middle" x="559.9" y="-74.74" font-family="Times,serif" font-size="10.00">args</text>
</g>
</g>
</svg>
assets/main_supported_lang.txt 0 → 100644
View file @ 12415ddb
Class 1:
----
c
cpp
c-sharp
go
javascript
java
kotlin
python
typescript
----
Class 2:
----
cmake
cuda
julia
markdown
makefile
mathematica
php
powershell
ruby
rust
shell
sql
swift
vue
React
----
Class 3:
----
lean
haskell
protocol-buffer
racket
scheme
dart
erlang
common-lisp
ada
dockerfile
css
clojure
elm
ocaml
perl
glsl
emacs-lisp
zig
json
agda
thrift
pascal
elixir
r
yaml
lua
fortran
restructuredtext
html
vhdl
verilog
scala
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment