Skip to content

GitLab

Commit 1768a324 authored by dengjb's avatar dengjb
Browse files

update codes

parent 18493eef
Pipeline #1372 failed with stages
in 0 seconds
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 2378 additions and 1 deletion
LICENSE 0 → 100644
View file @ 1768a324
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2024 CodeGeeX Team @ Zhipu AI
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
MODEL_LICENSE 0 → 100644
View file @ 1768a324
The CodeGeeX4 License
1. 定义
“许可方”是指分发其软件的 CodeGeeX 团队。
“软件”是指根据本许可提供的 CodeGeeX4 模型参数。
2. 许可授予
根据本许可的条款和条件，许可方特此授予您非排他性、全球性、不可转让、不可再许可、可撤销、免版税的版权许可。
本许可允许您免费使用本仓库中的所有开源模型进行学术研究，对于希望将模型用于商业目的的用户，需在[这里](https://open.bigmodel.cn/mla/form)完成登记。经过登记的用户可以免费使用本模型进行商业活动，但必须遵守本许可的所有条款和条件。
上述版权声明和本许可声明应包含在本软件的所有副本或重要部分中。
如果您分发或提供 THUDM / 智谱AI 关于 CodeGeeX4 开源模型的材料（或其任何衍生作品），或使用其中任何材料（包括 CodeGeeX4 系列的所有开源模型）的产品或服务，您应:
(A) 随任何此类 THUDM / 智谱AI 材料提供本协议的副本；
(B) 在相关网站、用户界面、博客文章、关于页面或产品文档上突出显示 “Built with CodeGeeX4”。
如果您使用 THUDM / 智谱AI的 CodeGeeX4 开源模型的材料来创建、训练、微调或以其他方式改进已分发或可用的 AI 模型，您还应在任何此类 AI 模型名称的开头添加 “CodeGeeX4”。
3. 限制
您不得出于任何军事或非法目的使用、复制、修改、合并、发布、分发、复制或创建本软件的全部或部分衍生作品。
您不得利用本软件从事任何危害国家安全和国家统一，危害社会公共利益及公序良俗，侵犯他人商业秘密、知识产权、名誉权、肖像权、财产权等权益的行为。
您在使用中应遵循使用地所适用的法律法规政策、道德规范等要求。
4. 免责声明
本软件“按原样”提供，不提供任何明示或暗示的保证，包括但不限于对适销性、特定用途的适用性和非侵权性的保证。
在任何情况下，作者或版权持有人均不对任何索赔、损害或其他责任负责，无论是在合同诉讼、侵权行为还是其他方面，由软件或软件的使用或其他交易引起、由软件引起或与之相关
软件。
5. 责任限制
除适用法律禁止的范围外，在任何情况下且根据任何法律理论，无论是基于侵权行为、疏忽、合同、责任或其他原因，任何许可方均不对您承担任何直接、间接、特殊、偶然、示范性、
或间接损害，或任何其他商业损失，即使许可人已被告知此类损害的可能性。
6. 争议解决
本许可受中华人民共和国法律管辖并按其解释。 因本许可引起的或与本许可有关的任何争议应提交北京市海淀区人民法院。
请注意，许可证可能会更新到更全面的版本。 有关许可和版权的任何问题，请通过 license@zhipuai.cn 与我们联系。
1. Definitions
“Licensor” means the CodeGeeX Team that distributes its Software.
“Software” means the CodeGeeX4 model parameters made available under this license.
2. License
Under the terms and conditions of this license, the Licensor hereby grants you a non-exclusive, worldwide, non-transferable, non-sublicensable, revocable, royalty-free copyright license.
This license allows you to use all open source models in this repository for free for academic research. For users who wish to use the models for commercial purposes, please do so [here](https://bigmodel.cn/mla/form?mcode=CodeGeeX4-ALL-9B)
Complete registration. Registered users are free to use this model for commercial activities, but must comply with all terms and conditions of this license.
The copyright notice and this license notice shall be included in all copies or substantial portions of the Software.
If you distribute or provide THUDM / Zhipu AI materials on the CodeGeeX4 open source model (or any derivative works thereof), or products or services that use any materials therein (including all open source models of the CodeGeeX4 series), you should:
(A) Provide a copy of this Agreement with any such THUDM/Zhipu AI Materials;
(B) Prominently display "Built with CodeGeeX4" on the relevant website, user interface, blog post, related page or product documentation.
If you use materials from THUDM/Zhipu AI's CodeGeeX4 model to create, train, operate, or otherwise improve assigned or available AI models, you should also add "CodeGeeX4" to the beginning of any such AI model name.
3. Restrictions
You are not allowed to use, copy, modify, merge, publish, distribute, copy or create all or part of the derivative works of this software for any military or illegal purposes.
You are not allowed to use this software to engage in any behavior that endangers national security and unity, endangers social public interests and public order, infringes on the rights and interests of others such as trade secrets, intellectual property rights, reputation rights, portrait rights, and property rights.
You should comply with the applicable laws, regulations, policies, ethical standards, and other requirements in the place of use during use.
4. Disclaimer
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
5. Limitation of Liability
EXCEPT TO THE EXTENT PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL THEORY, WHETHER BASED IN TORT,
NEGLIGENCE, CONTRACT, LIABILITY, OR OTHERWISE WILL ANY LICENSOR BE LIABLE TO YOU FOR ANY DIRECT, INDIRECT, SPECIAL,
INCIDENTAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES, OR ANY OTHER COMMERCIAL LOSSES, EVEN IF THE LICENSOR HAS BEEN ADVISED
OF THE POSSIBILITY OF SUCH DAMAGES.
6. Dispute Resolution
This license shall be governed and construed in accordance with the laws of People’s Republic of China. Any dispute
arising from or in connection with this License shall be submitted to Haidian District People's Court in Beijing.
Note that the license is subject to update to a more comprehensive version. For any questions related to the license and
copyright, please contact us at license@zhipuai.cn.
README.md
View file @ 1768a324
# codegeex4_pyotch
它是一个在GLM-4-9B上持续训练的多语言代码生成模型，显著增强了其代码生成能力。
## 论文
暂无
Multilingual Code Generation Model
\ No newline at end of file
## 模型结构
基于transformer结构
<div align=center>
<img src="./asserts/transformers.png" witdh=300 height=400/>
</div>
## 算法原理
CodeGeeX4基于第四代模型的实现了128K 长文本能力，并且生成速度全面超越上一代模型，实现3倍速生成，首响速度提高了8倍以上。
<div align=center>
<img src="./asserts/model.png"/>
</div>
## 环境配置
-v 路径、docker_name和imageID根据实际情况修改
### Docker（方法一）
```bash
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-centos7.6-dtk24.04-py310
docker run -it -v /path/your_code_data/:/path/your_code_data/ -v /opt/hyhal/:/opt/hyhal/:ro --shm-size=80G --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name docker_name imageID bash
cd /your_code_path/codegeex4_pytorch
pip install -r requirements.txt
pip install -U huggingface_hub hf_transfer
export HF_ENDPOINT=https://hf-mirror.com
```
### Dockerfile（方法二）
```bash
cd docker
docker build --no-cache -t codegeex4:latest .
docker run -it -v /path/your_code_data/:/path/your_code_data/ -v /opt/hyhal/:/opt/hyhal/:ro --shm-size=80G --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name docker_name imageID bash
cd /your_code_path/codegeex4_pytorch
pip install -r requirements.txt
pip install -U huggingface_hub hf_transfer
export HF_ENDPOINT=https://hf-mirror.com
```
### Anaconda（方法三）
关于本项目DCU显卡所需的特殊深度学习库可从[光合](https://developer.hpccube.com/tool/)开发者社区下载安装。
```
DTK驱动: dtk24.04
python: python3.10
torch: 2.1.0
```
`Tips：以上dtk驱动、python、torch等DCU相关工具版本需要严格一一对应`
其它非深度学习库安装方式如下：
```bash
pip install -r requirements.txt
pip install -U huggingface_hub hf_transfer
export HF_ENDPOINT=https://hf-mirror.com
```
## 数据集
暂无
## 训练
暂无
## 推理
基于Huggingface's Transformers进行推理
模型下载后 默认需存放至weights文件夹中
也可自行更改 inference.py文件中的 model_name 参数
```python
HIP_VISIBLE_DEVICES=0 python inference.py
```
## Result
prompt：write a quick sort.<br>
<div align=center>
<img src="./asserts/result.png"/>
</div>
### 精度
暂无
## 应用场景
### 算法类别
代码生成
### 热点应用行业
制造,能源,教育
## 预训练权重
- 模型权重快速下载中心
- [SCNet AIModels](http://113.200.138.88:18080/aimodels)
- 模型权重快速通道下载地址
- [模型权重快速下载地址](http://113.200.138.88:18080/aimodels/thudm/codegeex4-all-9b)
- 官方下载地址：
- [codegeex4-all-9b](https://huggingface.co/THUDM/codegeex4-all-9b)
模型目录结构如下：
```bash
└── codegeex4-all-9b
├── config.json
├── configuration_chatglm.py
├── configuration.json
├── generation_config.json
├── LICENSE
├── model-00001-of-00004.safetensors
├── model-00002-of-00004.safetensors
├── model-00003-of-00004.safetensors
├── model-00004-of-00004.safetensors
├── modeling_chatglm.py
├── model.safetensors.index.json
├── README.md
├── README_zh.md
├── tokenization_chatglm.py
├── tokenizer_config.json
└── tokenizer.model
```
## 源码仓库及问题反馈
- https://developer.hpccube.com/codes/modelzoo/codegeex4_pyotch
## 参考资料
- https://github.com/THUDM/CodeGeeX4/tree/main
- https://huggingface.co/THUDM/codegeex4-all-9b
\ No newline at end of file
README_ori.md 0 → 100644
View file @ 1768a324
![](resources/logo.jpeg)
<p align="center">
🏠 <a href="https://codegeex.cn" target="_blank">Homepage</a>｜🛠 Extensions <a href="https://marketplace.visualstudio.com/items?itemName=aminer.codegeex" target="_blank">VS Code</a>, <a href="https://plugins.jetbrains.com/plugin/20587-codegeex" target="_blank">Jetbrains</a>｜🤗 <a href="https://huggingface.co/THUDM/codegeex4-all-9b" target="_blank">HF Repo</a> | 🪧 <a href="https://huggingface.co/spaces/THUDM/CodeGeeX" target="_blank">HF DEMO</a>
</p>
[English](./README.md) | [中文](./README_zh.md)
# CodeGeeX4: Open Multilingual Code Generation Model
We introduce CodeGeeX4-ALL-9B, the open-source version of the latest CodeGeeX4 model series. It is a multilingual code generation model continually trained on the [GLM-4-9B](https://github.com/THUDM/GLM-4), significantly enhancing its code generation capabilities. Using a single CodeGeeX4-ALL-9B model, it can support comprehensive functions such as code completion and generation, code interpreter, web search, function call, repository-level code Q&A, covering various scenarios of software development. CodeGeeX4-ALL-9B has achieved highly competitive performance on public benchmarks, such as [BigCodeBench](https://huggingface.co/datasets/bigcode/bigcodebench) and [NaturalCodeBench](https://github.com/THUDM/NaturalCodeBench). It is currently the most powerful code generation model with less than 10B parameters, even surpassing much larger general-purpose models, achieving the best balance in terms of inference speed and model performance.
## Model List
| Model | Type | Seq Length | Download |
|-------------------|------|------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| codegeex4-all-9b | Chat | 128K | [🤗 Huggingface](https://huggingface.co/THUDM/codegeex4-all-9b) [🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/codegeex4-all-9b) [🟣 WiseModel](https://wisemodel.cn/models/ZhipuAI/codegeex4-all-9b) |
## Get Started
### Ollama
CodeGeeX4 is now available on [Ollama](https://ollama.com/library/codegeex4)!
Please install [Ollama 0.2](https://github.com/ollama/ollama/releases/tag/v0.2.0) or later and run the following command:
```bash
ollama run codegeex4
```
To connect the local model to our [VS Code](https://marketplace.visualstudio.com/items?itemName=aminer.codegeex) / [Jetbrains](https://plugins.jetbrains.com/plugin/20587-codegeex) extensions, please check [Local Mode Guideline](./guides/Local_mode_guideline.md).
### Huggingface transformers
Use `4.39.0<=transformers<=4.40.2` to quickly launch [codegeex4-all-9b](https://huggingface.co/THUDM/codegeex4-all-9b)：
```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained("THUDM/codegeex4-all-9b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
"THUDM/codegeex4-all-9b",
torch_dtype=torch.bfloat16,
low_cpu_mem_usage=True,
trust_remote_code=True
).to(device).eval()
inputs = tokenizer.apply_chat_template([{"role": "user", "content": "write a quick sort"}], add_generation_prompt=True, tokenize=True, return_tensors="pt", return_dict=True ).to(device)
with torch.no_grad():
outputs = model.generate(**inputs)
outputs = outputs[:, inputs['input_ids'].shape[1]:]
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
### vLLM
Use `vllm==0.5.1` to quickly launch [codegeex4-all-9b](https://huggingface.co/THUDM/codegeex4-all-9b):
```
from transformers import AutoTokenizer
from vllm import LLM, SamplingParams
# CodeGeeX4-ALL-9B
# max_model_len, tp_size = 1048576, 4
# If OOM，please reduce max_model_len，or increase tp_size
max_model_len, tp_size = 131072, 1
model_name = "codegeex4-all-9b"
prompt = [{"role": "user", "content": "Hello"}]
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
llm = LLM(
model=model_name,
tensor_parallel_size=tp_size,
max_model_len=max_model_len,
trust_remote_code=True,
enforce_eager=True,
# If OOM，try using follong parameters
# enable_chunked_prefill=True,
# max_num_batched_tokens=8192
)
stop_token_ids = [151329, 151336, 151338]
sampling_params = SamplingParams(temperature=0.95, max_tokens=1024, stop_token_ids=stop_token_ids)
inputs = tokenizer.apply_chat_template(prompt, tokenize=False, add_generation_prompt=True)
outputs = llm.generate(prompts=inputs, sampling_params=sampling_params)
print(outputs[0].outputs[0].text)
```
Set up OpenAI Compatible Server via vllm, detailed please check [OpenAI Compatible Server](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html)
```
python -m vllm.entrypoints.openai.api_server \
--model THUDM/codegeex4-all-9b \
--trust_remote_code
```
## Tutorials
CodeGeeX4-ALL-9B provides three user guides to help users quickly understand and use the model:
![ALL Fuctions](./resources/all_functions.jpg)
1. **[System Prompt Guideline](./guides/System_prompt_guideline.md)**: This guide introduces how to use system prompts in CodeGeeX4-ALL-9B, including the VSCode extension official system prompt, customized system prompts, and some tips for maintaining multi-turn dialogue history.
2. **[Infilling Guideline](./guides/Infilling_guideline.md)**: This guide explains the VSCode extension official infilling format, covering general infilling, cross-file infilling, and generating a new file in a repository.
3. **[Repository Tasks Guideline](./guides/Repository_tasks_guideline.md)**: This guide demonstrates how to use repository tasks in CodeGeeX4-ALL-9B, including QA tasks at the repository level and how to trigger the aicommiter capability of CodeGeeX4-ALL-9B to perform deletions, additions, and changes to files at the repository level.
4. **[Local Mode Guideline](./guides/Local_mode_guideline.md)**：This guide introduces how to deploy CodeGeeX4-ALL-9B locally and connect it to Visual Studio Code / Jetbrains extensions.
These guides aim to provide a comprehensive understanding and facilitate efficient use of the model.
## Evaluation
CodeGeeX4-ALL-9B is ranked as the most powerful model under 10 billion parameters, even surpassing general models several times its size, achieving the best balance between inference performance and model effectiveness.
| **Model** | **Seq Length** | **HumanEval** | **MBPP** | **NCB** | **LCB** | **HumanEvalFIM** | **CRUXEval-O** |
|-----------------------------|----------------|---------------|----------|---------|---------|------------------|----------------|
| Llama3-70B-intruct | 8K | 77.4 | 82.3 | 37.0 | 27.4 | - | - |
| DeepSeek Coder 33B Instruct | 16K | 81.1 | 80.4 | 39.3 | 29.3 | 78.2 | 49.9 |
| Codestral-22B | 32K | 81.1 | 78.2 | 46.0 | 35.3 | 91.6 | 51.3 |
| CodeGeeX4-All-9B | 128K | 82.3 | 75.7 | 40.4 | 28.5 | 85.0 | 47.1 |
CodeGeeX4-ALL-9B scored `48.9` and `40.4` for the `complete` and `instruct` tasks of BigCodeBench, which are the highest scores among models with less than 20 billion parameters.
![BigCodeBench Test Results](./metric/pics/Bigcodebench.png)
In CRUXEval, a benchmark for testing code reasoning, understanding, and execution capabilities, CodeGeeX4-ALL-9B presented remarkable results with its COT (chain-of-thought) abilities. From easy code generation tasks in HumanEval and MBPP, to very challenging tasks in NaturalCodeBench, CodeGeeX4-ALL-9B also achieved outstanding performance at its scale. It is currently the only code model that supports Function Call capabilities and even achieves a better execution success rate than GPT-4.
![Function Call Evaluation](./metric/pics/FunctionCall.png)
Furthermore, in the "Code Needle In A Haystack" (NIAH) evaluation, the CodeGeeX4-ALL-9B model demonstrated its ability to retrieve code within contexts up to 128K, achieving a 100% retrieval accuracy in all python scripts.
<p align="center">
<img src=./metric/pics/NIAH_PYTHON.png alt="图片1描述" width="45%">
<img src="./metric/pics/NIAH_ALL.png" alt="图片2描述" width="45%">
</p>
Details of the evaluation results can be found in the **[Evaluation](./metric/README.md)**.
## License
The code in this repository is open source under the [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0) license. The model weights are licensed under the [Model License](MODEL_LICENSE). CodeGeeX4-9B weights are open for academic research. For users who wish to use the models for commercial purposes, please fill in the [registration form](https://bigmodel.cn/mla/form?mcode=CodeGeeX4-ALL-9B).
## Citation
If you find our work helpful, please feel free to cite the following paper:
```bibtex
@inproceedings{zheng2023codegeex,
title={CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X},
author={Qinkai Zheng and Xiao Xia and Xu Zou and Yuxiao Dong and Shan Wang and Yufei Xue and Zihan Wang and Lei Shen and Andi Wang and Yang Li and Teng Su and Zhilin Yang and Jie Tang},
booktitle={Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
pages={5673--5684},
year={2023}
}
```
README_zh.md 0 → 100644
View file @ 1768a324
![](resources/logo.jpeg)
<p align="center">
🏠 <a href="https://codegeex.cn" target="_blank">Homepage</a>｜🛠 Extensions <a href="https://marketplace.visualstudio.com/items?itemName=aminer.codegeex" target="_blank">VS Code</a>, <a href="https://plugins.jetbrains.com/plugin/20587-codegeex" target="_blank">Jetbrains</a>｜🤗 <a href="https://huggingface.co/THUDM/codegeex4-all-9b" target="_blank">HF Repo</a> | 🪧 <a href="https://huggingface.co/spaces/THUDM/CodeGeeX" target="_blank">HF DEMO</a>
</p>
[English](./README.md) | [中文](./README_zh.md)
# CodeGeeX4: 全能的开源多语言代码生成模型
我们推出了 CodeGeeX4-ALL-9B，这是最新的 CodeGeeX4 系列模型的开源版本。该模型是在 [GLM-4-9B](https://github.com/THUDM/GLM-4) 基础上持续训练的多语言代码生成模型，显著提升了代码生成能力。使用单个 CodeGeeX4-ALL-9B 模型，可以支持代码补全与生成、代码解释、联网搜索、函数调用、仓库级代码问答等多种功能，覆盖了软件开发的各个场景。CodeGeeX4-ALL-9B 在 [BigCodeBench](https://huggingface.co/datasets/bigcode/bigcodebench) 和 [NaturalCodeBench](https://github.com/THUDM/NaturalCodeBench) 等公开基准测试中取得了极具竞争力的表现。它是目前参数量少于 100 亿的最强代码生成模型，甚至超越了更大的通用模型，在推理速度和模型性能方面达到了最佳平衡。
## 模型列表
| 模型 | 类型 | 上下文长度 | 下载地址 |
|-------------------|------|------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| codegeex4-all-9b | Chat | 128K | [🤗 Huggingface](https://huggingface.co/THUDM/codegeex4-all-9b) [🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/codegeex4-all-9b) [🟣 WiseModel](https://wisemodel.cn/models/ZhipuAI/codegeex4-all-9b) |
## 快速开始
### Ollama
CodeGeeX4 正式上线[Ollama](https://ollama.com/library/codegeex4)！
请安装[Ollama 0.2](https://github.com/ollama/ollama/releases/tag/v0.2.0)或更高版本，并运行以下命令:
```bash
ollama run codegeex4
```
把本地模型接入[VS Code](https://marketplace.visualstudio.com/items?itemName=aminer.codegeex) / [Jetbrains](https://plugins.jetbrains.com/plugin/20587-codegeex)插件，请参考[本地模式教程](./guides/Local_mode_guideline_zh.md).
### Huggingface transformers
请使用 `4.39.0<=transformers<=4.40.2` 部署 [codegeex4-all-9b](https://huggingface.co/THUDM/codegeex4-all-9b)：
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained("THUDM/codegeex4-all-9b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
"THUDM/codegeex4-all-9b",
torch_dtype=torch.bfloat16,
low_cpu_mem_usage=True,
trust_remote_code=True
).to(device).eval()
inputs = tokenizer.apply_chat_template([{"role": "user", "content": "write a quick sort"}], add_generation_prompt=True, tokenize=True, return_tensors="pt", return_dict=True).to(device)
with torch.no_grad():
outputs = model.generate(**inputs)
outputs = outputs[:, inputs['input_ids'].shape[1]:]
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
### vLLM
使用 `vllm==0.5.1` 快速启动 [codegeex4-all-9b](https://huggingface.co/THUDM/codegeex4-all-9b)：
```python
from transformers import AutoTokenizer
from vllm import LLM, SamplingParams
# CodeGeeX4-ALL-9B
# max_model_len, tp_size = 1048576, 4
# 如果出现内存不足（OOM），减少max_model_len，或增加tp_size
max_model_len, tp_size = 131072, 1
model_name = "codegeex4-all-9b"
prompt = [{"role": "user", "content": "Hello"}]
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
llm = LLM(
model=model_name,
tensor_parallel_size=tp_size,
max_model_len=max_model_len,
trust_remote_code=True,
enforce_eager=True,
# 如果出现OOM，尝试使用以下参数
# enable_chunked_prefill=True,
# max_num_batched_tokens=8192
)
stop_token_ids = [151329, 151336, 151338]
sampling_params = SamplingParams(temperature=0.95, max_tokens=1024, stop_token_ids=stop_token_ids)
inputs = tokenizer.apply_chat_template(prompt, tokenize=False, add_generation_prompt=True)
outputs = llm.generate(prompts=inputs, sampling_params=sampling_params)
print(outputs[0].outputs[0].text)
```
通过 vllm 设置 OpenAI 兼容服务，详细信息请查看 [OpenAI 兼容服务器](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html)：
```bash
python -m vllm.entrypoints.openai.api_server \
--model THUDM/codegeex4-all-9b \
--trust_remote_code
```
## 用户指南
我们为 CodeGeeX4-ALL-9B 提供了用户指南，帮助用户快速了解和使用该模型：
![ALL Fuctions](./resources/all_functions_zh.jpg)
1. **[系统提示指南](./guides/System_prompt_guideline_zh.md)**：本指南介绍了如何在 CodeGeeX4-ALL-9B 中使用系统提示，包括 VSCode 插件的官方系统提示、自定义系统提示以及维护多轮对话历史的一些技巧。
2. **[上下文补全指南](./guides/Infilling_guideline_zh.md)**：本指南解释了 VSCode 插件的官方填充格式，涵盖一般补全、跨文件补全和在仓库中生成新文件。
3. **[项目级代码生成指南](./guides/Repository_tasks_guideline_zh.md)**：本指南展示了如何在 CodeGeeX4-ALL-9B 中使用项目级任务，包括项目级别的问答任务，以及如何触发 CodeGeeX4-ALL-9B 的 aicommiter 功能以执行仓库级别任务中的删除、添加和更改文件操作。
4. **[本地模式指南](./guides/Local_mode_guideline_zh.md)**：本指南展示了如果在本地部署 CodeGeeX4-ALL-9B 并接入 Visual Studio Code / Jetbrains 插件中使用。
这些指南旨在帮助大家全面理解模型的用法并更好发挥模型的能力。
## 评测指标
CodeGeeX4-ALL-9B 被评为参数量100 亿内的最强模型，甚至超越了参数量大几倍的通用模型，在推理性能和模型能力之间达到了最佳效果。
| **模型** | **序列长度** | **HumanEval** | **MBPP** | **NCB** | **LCB** | **HumanEvalFIM** | **CRUXEval-O** |
|-----------------------------|----------------|---------------|----------|---------|---------|------------------|----------------|
| Llama3-70B-intruct | 8K | 77.4 | 82.3 | 37.0 | 27.4 | - | - |
| DeepSeek Coder 33B Instruct | 16K | 81.1 | 80.4 | 39.3 | 29.3 | 78.2 | 49.9 |
| Codestral-22B | 32K | 81.1 | 78.2 | 46.0 | 35.3 | 91.6 | 51.3 |
| CodeGeeX4-All-9B | 128K | 82.3 | 75.7 | 40.4 | 28.5 | 85.0 | 47.1 |
在 BigCodeBench 的 complete 和 instruct 任务中，CodeGeeX4-ALL-9B 分别取得了 `48.9` 和 `40.4` 的高分，这在参数量 200 亿内的模型中是最高的分数。
![BigCodeBench Test Results](./metric/pics/Bigcodebench.png)
Crux-Eval 是测试代码推理、理解和执行能力的基准测试，借助于其强大的 COT 能力，CodeGeeX4-ALL-9B 展现出色的表现。在 HumanEval、MBPP 和 NaturalCodeBench 等代码生成任务中，CodeGeeX4-ALL-9B 也取得了出色的成绩。目前，它是唯一支持 Function Call 功能的代码模型，甚至取得了比 GPT-4 更高的分数。
![Function Call Evaluation](./metric/pics/FunctionCall.png)
此外，在“Code Needle In A Haystack” (NIAH) 评估中，CodeGeeX4-ALL-9B 模型展示了在 128K 范围内检索代码的能力，在python语言环境达到了 100% 的检索准确率，并在跨文件补全任务中表现出色。
<p align="center">
<img src=./metric/pics/NIAH_PYTHON.png alt="图片1描述" width="45%">
<img src="./metric/pics/NIAH_ALL.png" alt="图片2描述" width="45%">
</p>
更详细的评估结果请看 **[评估结果](./metric/README_zh.md)** 。
## 许可证
本仓库中的代码是根据 [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0) 许可证开源的。模型权重根据 [模型许可证](MODEL_LICENSE) 许可。CodeGeeX4-9B 权重对学术研究开放。对于希望将模型用于商业目的的用户，请填写 [登记表](https://bigmodel.cn/mla/form?mcode=CodeGeeX4-ALL-9B)。
## 引用
如果您觉得我们的工作对您有帮助，欢迎引用以下论文：
```bibtex
@inproceedings{zheng2023codegeex,
title={CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X},
author={Qinkai Zheng and Xiao Xia and Xu Zou and Yuxiao Dong and Shan Wang and Yufei Xue and Zihan Wang and Lei Shen and Andi Wang and Yang Li and Teng Su and Zhilin Yang and Jie Tang},
booktitle={Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
pages={5673--5684},
year={2023}
}
```
docker/Dockerfile 0 → 100644
View file @ 1768a324
FROM image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-centos7.6-dtk24.04-py310
\ No newline at end of file
function_call_demo/main.py 0 → 100644
View file @ 1768a324
import json
import re
from json import JSONDecodeError
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
def main():
device = "cuda" if torch.cuda.is_available() else "cpu"
model_name_or_path = "THUDM/codegeex4-all-9b"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_name_or_path,
torch_dtype=torch.bfloat16,
trust_remote_code=True
).to(device).eval()
tool_content = {
"function": [
{
"name": "weather",
"description": "Use for searching weather at a specific location",
"parameters": {
"type": "object",
"properties": {
"location": {
"description": "the location need to check the weather",
"type": "str",
}
},
"required": [
"location"
]
}
}
]
}
response, _ = model.chat(
tokenizer,
query="Tell me about the weather in Beijing",
history=[{"role": "tool", "content": tool_content}],
max_new_tokens=1024,
temperature=0.1
)
# support parallel calls, thus the result is a list
functions = post_process(response)
try:
return [json.loads(func) for func in functions if func]
# get rid of some possible invalid formats
except JSONDecodeError:
try:
return [json.loads(func.replace('(', '[').replace(')', ']')) for func in functions if func]
except JSONDecodeError:
try:
return [json.loads(func.replace("'", '"')) for func in functions if func]
except JSONDecodeError as e:
return [{"answer": response, "errors": e}]
def post_process(text: str) -> list[str]:
"""
Process model's response.
In case there are parallel calls, each call is warpped with ```json```.
"""
pattern = r'```json(.*?)```'
matches = re.findall(pattern, text, re.DOTALL)
return matches
if __name__ == '__main__':
output = main()
print(output) # [{"name": "weather", "arguments": {"location": "Beijing"}}]
guides/Infilling_guideline.md 0 → 100644
View file @ 1768a324
**Code Completion Tutorial: Contextual, Cross-File, and Project-Level Completion**
This tutorial primarily introduces the code completion capabilities of plugin-enabled models. These capabilities include contextual completion, cross-file completion, and project-level file completion.
- **Contextual Completion:** Within the same code file, based on the cursor's position and the surrounding context.
- **Cross-File Completion:** Enhances code completion capabilities by incorporating dependencies or related files of the current code file.
- **Project-Level Completion:** The model can generate complete new files based on your project information and requirements.
You can use the CodeGeeX4-ALL-9B-128k model and control memory requirements by setting different `max_length` values. For example, you can set `max_length` to 16k or 32k to run the model on consumer-grade graphics cards.
<a name="heading_0"></a>**Code Completion Usage Tutorial**
<a name="heading_1"></a>1. **Contextual Completion**
1. **File Path:** `"###PATH:"` + relative file path or file name
2. **Language Tag:** This is very important and must be included. The list of languages is mentioned above, and the format generally starts with the language's initial capital letter, with a few exceptions. If unsure about the language, you can leave it blank after the colon. All language tags start with `"###LANGUAGE:"`.
3. **Two Modes:** `"###MODE:"`, `LINE` generates a single line, and `BLOCK` generates multiple lines. The default is `BLOCK` mode.
4. **Format:**
```
<|user|>
###PATH:{path}
###LANGUAGE:{code_language}
###MODE:{LINE/BLOCK}
<|code_suffix|>{code}<|code_prefix|>{code}<|code_middle|><|assistant|>\n
```
5. Example:
When all information including path, code_language, mode, suffix, and prefix is provided:
```
<|user|>
###PATH:src.py
###LANGUAGE:Python
###MODE:LINE/BLOCK
<|code_suffix|> else:
depth -= 1
return max_depth
return [parse_paren_group(x) for x in paren_string.split(' ') if x]
<|code_prefix|>from typing import List
def parse_nested_parens(paren_string: str) -> List[int]:
""" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.
For each of the group, output the deepest level of nesting of parentheses.
E.g. (()()) has maximum two levels of nesting while ((())) has three.
>>> parse_nested_parens('(()()) ((())) () ((())()())')
[2, 3, 1, 3]
"""
<|code_middle|><|assistant|>\n
```
- Situation with no language and no suffix provided
```
<|user|>
###PATH:src.py
###LANGUAGE:
###MODE:LINE/BLOCK
<|code_suffix|><|code_prefix|>from typing import List
def parse_nested_parens(paren_string: str) -> List[int]:
""" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.
For each of the group, output the deepest level of nesting of parentheses.
E.g. (()()) has maximum two levels of nesting while ((())) has three.
>>> parse_nested_parens('(()()) ((())) () ((())()())')
[2, 3, 1, 3]
"""
<|code_middle|><|assistant|>\n
```
<a name="heading_2"></a>2. **Cross File Infilling**
1. Please format reference code as below:
```
###REFERENCE:
###PATH: relative file path or file name
code snippet
###REFERENCE:
###PATH: relative file path or file name
code snippet
```
2. **File Path:** `"###PATH:"` + relative file path or file name
3. **Language Tag:** This is very important and must be included. The list of languages is mentioned above, and the format generally starts with the language's initial capital letter, with a few exceptions. If unsure about the language, you can leave it blank after the colon. All language tags start with `"###LANGUAGE:"`.
4. **Two Modes:** `"###MODE:"`, `LINE` generates a single line, and `BLOCK` generates multiple lines. The default is `BLOCK` mode.
5. **Format:**
```
<|user|>
###REFERENCE:
###PATH:{path}
{code}
...
...
...
###REFERENCE:
###PATH:{path}
{code}
###PATH:{path}\n\n###LANGUAGE:{code_language}\n###MODE:{LINE/BLOCK}\n<|code_suffix|>{code}<|code_prefix|>{code}<|code_middle|><|assistant|>\n
```
6. Example:
```
|Python
<|user|>
###PATH:./sort/quick_sort.py
def quick_sort(arr):
if len(arr) <= 1:
return arr
pivot = arr[len(arr) // 2]
left = [x for x in arr if x < pivot]
middle = [x for x in arr if x == pivot]
right = [x for x in arr if x > pivot]
return quick_sort(left) + middle + quick_sort(right)
arr = [3,6,8,10,1,2,1]
print(quick_sort(arr))
###PATH:src.py
###LANGUAGE:Python
###MODE:LINE/BLOCK
<|code_suffix|> else:
depth -= 1
return max_depth
return [parse_paren_group(x) for x in paren_string.split(' ') if x]
<|code_prefix|>from typing import List
def parse_nested_parens(paren_string: str) -> List[int]:
""" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.
For each of the group, output the deepest level of nesting of parentheses.
E.g. (()()) has maximum two levels of nesting while ((())) has three.
>>> parse_nested_parens('(()()) ((())) () ((())()())')
[2, 3, 1, 3]
"""
<|code_middle|><|assistant|>\n
```
<a name="heading_3"></a>3. **Repository Level File Generation**
1. You can use the project-level add, delete, and modify format to complete the task of adding a file in the project.
2. Related files: Example format is as follows:
```
###REFERENCE:
###PATH: relative file path or file name
代码
###REFERENCE:
###PATH: relative file path or file name
代码
```
3. **File Path:** `"###PATH:"` + relative file path or file name
4. **Language Tag:** This is very important and must be included. The list of languages is mentioned above, and the format generally starts with the language's initial capital letter, with a few exceptions. If unsure about the language, you can leave it blank after the colon. All language tags start with `"###LANGUAGE:"`.
5. **Two Modes:** `"###MODE:"`, `LINE` generates a single line, and `BLOCK` generates multiple lines. The default is `BLOCK` mode.
6. **Format:**
```
<|user|>
###REFERENCE:
###PATH:{path}
{code}
...
...
...
###REFERENCE:
###PATH:{path}
{code}
###PATH:{path}
###LANGUAGE:{code_language}
###MODE:{LINE/BLOCK}
<|code_suffix|>{code}<|code_prefix|>{code}<|code_middle|><|assistant|>\n
```
7. Example:
```
<|user|>
###PATH:./sort/quick_sort.py
def quick_sort(arr):
if len(arr) <= 1:
return arr
pivot = arr[len(arr) // 2]
left = [x for x in arr if x < pivot]
middle = [x for x in arr if x == pivot]
right = [x for x in arr if x > pivot]
return quick_sort(left) + middle + quick_sort(right)
arr = [3,6,8,10,1,2,1]
print(quick_sort(arr))
###PATH:src.py
###LANGUAGE:Python
###MODE:LINE/BLOCK
<|code_suffix|> else:
depth -= 1
return max_depth
return [parse_paren_group(x) for x in paren_string.split(' ') if x]
<|code_prefix|>from typing import List
def parse_nested_parens(paren_string: str) -> List[int]:
""" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.
For each of the group, output the deepest level of nesting of parentheses.
E.g. (()()) has maximum two levels of nesting while ((())) has three.
>>> parse_nested_parens('(()()) ((())) () ((())()())')
[2, 3, 1, 3]
"""
<|code_middle|><|assistant|>\n
```
guides/Infilling_guideline_zh.md 0 → 100644
View file @ 1768a324
**代码补全使用教程，上下文补全，跨文件补全，项目级别补全**
本篇教程中，主要介绍了插件功能的模型的代码补全能力。代码补全能力包括上下文补全，跨文件补全和项目级文件补全。
- 上下文补全：在同一个代码文件内，根据光标所在位置，上下文信息。
- 跨文件补全：引入当前代码文件的依赖文件或者相关文件，增强代码补全的能力。
- 项目级别补全：模型可以根据您的项目信息，以及您的需求，为您生成完整的新文件。
您可以使用CodeGeeX4-ALL-9B-128k模型，通过设置不同的max_length控制对显存的需求。例如，您可以将max_length设置在16k或者32k以便将模型跑在消费级显卡上
<a name="heading_0"></a>**代码补全使用教程**
<a name="heading_1"></a>1. **上下文填空**
1. 文件路径："###PATH:" + 文件相对路径或文件名
2. 语言标签非常重要，必须要加上，语言列表见上方，格式一般是语言开头大写，有个别特殊；在不确定语言的情况下，冒号后面可以空着），所有语言标签都是以"###LANGUAGE:"开头，不作另外区分。
3. 两个种模式："###MODE:"，LINE生成单行，BLOCK生成多行。默认BLOCK模式。
4. 格式：
```
<|user|>
###PATH:{path}
###LANGUAGE:{code_language}
###MODE:{LINE/BLOCK}
<|code_suffix|>{code}<|code_prefix|>{code}<|code_middle|><|assistant|>\n
```
5. 示例如下:
- 有path,code_language,mode,suffix,prefix全部信息的情况
```
<|user|>
###PATH:src.py
###LANGUAGE:Python
###MODE:LINE/BLOCK
<|code_suffix|> else:
depth -= 1
return max_depth
return [parse_paren_group(x) for x in paren_string.split(' ') if x]
<|code_prefix|>from typing import List
def parse_nested_parens(paren_string: str) -> List[int]:
""" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.
For each of the group, output the deepest level of nesting of parentheses.
E.g. (()()) has maximum two levels of nesting while ((())) has three.
>>> parse_nested_parens('(()()) ((())) () ((())()())')
[2, 3, 1, 3]
"""
<|code_middle|><|assistant|>\n
```
- 没有语言、没有后缀的情况
```
<|user|>
###PATH:src.py
###LANGUAGE:
###MODE:LINE/BLOCK
<|code_suffix|><|code_prefix|>from typing import List
def parse_nested_parens(paren_string: str) -> List[int]:
""" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.
For each of the group, output the deepest level of nesting of parentheses.
E.g. (()()) has maximum two levels of nesting while ((())) has three.
>>> parse_nested_parens('(()()) ((())) () ((())()())')
[2, 3, 1, 3]
"""
<|code_middle|><|assistant|>\n
```
<a name="heading_2"></a>2. **跨文件补全**
1. 相关文件：格式示例如下
```
###REFERENCE:
###PATH: 文件相对路径或文件名
代码
###REFERENCE:
###PATH: 文件相对路径或文件名
代码
```
2. 文件路径："###PATH:" + 文件相对路径或文件名
3. 语言标签非常重要，必须要加上，语言列表见上方，格式一般是语言开头大写，有个别特殊；在不确定语言的情况下，冒号后面可以空着），所有语言标签都是以"###LANGUAGE:"开头，不作另外区分。
4. 两个种模式："###MODE:"，LINE生成单行，BLOCK生成多行。默认BLOCK模式。
5. 格式：
```
<|user|>
###REFERENCE:
###PATH:{path}
{code}
...
...
...
###REFERENCE:
###PATH:{path}
{code}
###PATH:{path}\n\n###LANGUAGE:{code_language}\n###MODE:{LINE/BLOCK}\n<|code_suffix|>{code}<|code_prefix|>{code}<|code_middle|><|assistant|>
6. 示例如下:
|Python
<|user|>
###PATH:./sort/quick_sort.py
def quick_sort(arr):
if len(arr) <= 1:
return arr
pivot = arr[len(arr) // 2]
left = [x for x in arr if x < pivot]
middle = [x for x in arr if x == pivot]
right = [x for x in arr if x > pivot]
return quick_sort(left) + middle + quick_sort(right)
arr = [3,6,8,10,1,2,1]
print(quick_sort(arr))
###PATH:src.py
###LANGUAGE:Python
###MODE:LINE/BLOCK
<|code_suffix|> else:
depth -= 1
return max_depth
return [parse_paren_group(x) for x in paren_string.split(' ') if x]
<|code_prefix|>from typing import List
def parse_nested_parens(paren_string: str) -> List[int]:
""" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.
For each of the group, output the deepest level of nesting of parentheses.
E.g. (()()) has maximum two levels of nesting while ((())) has three.
>>> parse_nested_parens('(()()) ((())) () ((())()())')
[2, 3, 1, 3]
"""
<|code_middle|><|assistant|>\n
```
<a name="heading_3"></a>3. **项目级文件补全**
1. 您可以使用项目级增删改的格式，完成在项目中补一个文件的任务
2. 相关文件：格式示例如下
```
###REFERENCE:
###PATH: 文件相对路径或文件名
代码
###REFERENCE:
###PATH: 文件相对路径或文件名
代码
```
3. 文件路径："###PATH:" + 文件相对路径或文件名
4. 语言标签非常重要，必须要加上，语言列表见上方，格式一般是语言开头大写，有个别特殊；在不确定语言的情况下，冒号后面可以空着），所有语言标签都是以"###LANGUAGE:"开头，不作另外区分。
5. 两个种模式："###MODE:"，LINE生成单行，BLOCK生成多行。默认BLOCK模式。
6. 格式：
```
<|user|>
###REFERENCE:
###PATH:{path}
{code}
...
...
...
###REFERENCE:
###PATH:{path}
{code}
###PATH:{path}
###LANGUAGE:{code_language}
###MODE:{LINE/BLOCK}
<|code_suffix|>{code}<|code_prefix|>{code}<|code_middle|><|assistant|>\n
```
7. 示例如下:
```
<|user|>
###PATH:./sort/quick_sort.py
def quick_sort(arr):
if len(arr) <= 1:
return arr
pivot = arr[len(arr) // 2]
left = [x for x in arr if x < pivot]
middle = [x for x in arr if x == pivot]
right = [x for x in arr if x > pivot]
return quick_sort(left) + middle + quick_sort(right)
arr = [3,6,8,10,1,2,1]
print(quick_sort(arr))
###PATH:src.py
###LANGUAGE:Python
###MODE:LINE/BLOCK
<|code_suffix|> else:
depth -= 1
return max_depth
return [parse_paren_group(x) for x in paren_string.split(' ') if x]
<|code_prefix|>from typing import List
def parse_nested_parens(paren_string: str) -> List[int]:
""" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.
For each of the group, output the deepest level of nesting of parentheses.
E.g. (()()) has maximum two levels of nesting while ((())) has three.
>>> parse_nested_parens('(()()) ((())) () ((())()())')
[2, 3, 1, 3]
"""
<|code_middle|><|assistant|>\n
```
guides/Local_mode_guideline.md 0 → 100644
View file @ 1768a324
# Local Mode Tutorial: Local deployment with the Visual Studio Code / Jetbrains extensions
The steps for two platforms are the same.
1. Click [VS Code](https://marketplace.visualstudio.com/items?itemName=aminer.codegeex) / [Jetbrains](https://plugins.jetbrains.com/plugin/20587-codegeex) to download the extension.
2. Open the local mode in the extension settings (no need to login).
3. Start Ollama server (other OpenAI compatible APIs are also supported) with the following command (keep the server running in background):
```bash
export OLLAMA_ORIGINS="*"
ollama run codegeex4
ollama serve
```
4. Enter the api address and model name in local mode settings. Then enjoy coding with CodeGeeX4!
![local mode](../resources/local_mode.png)
guides/Local_mode_guideline_zh.md 0 → 100644
View file @ 1768a324
# 本地模式教程: 本地部署并接入 Visual Studio Code / Jetbrains 插件
两个平台的步骤相同。
1. 点击 [VS Code](https://marketplace.visualstudio.com/items?itemName=aminer.codegeex) / [Jetbrains](https://plugins.jetbrains.com/plugin/20587-codegeex) 下载插件。
2. 在插件设置中打开本地模式（无需登录）。
3. 使用以下命令启动 Ollama 服务器（支持其他兼容 OpenAI 格式的 API）：
```bash
export OLLAMA_ORIGINS="*"
ollama run codegeex4
ollama serve
```
4. 在本地模式设置中输入 api 地址和模型名称。然后享受 CodeGeeX4 的编码体验！
![local mode](../resources/local_mode_zh.png)
guides/Repository_tasks_guideline.md 0 → 100644
View file @ 1768a324
**Project-Level Usage Tutorial: Project File Add, Delete, Modify, Project Q&A**
For project-level tasks, it is recommended to use the CodeGeeX4-ALL-9B-128k model version. This model supports 128k context, which is approximately 10,000 lines of code.
Due to the inference time cost and the possibility of project content exceeding 128k, we strongly recommend trimming the model output when using the CodeGeeX4-ALL-9B-128k model.
**Model Output Trimming**
- BM25 & embedding recall
- Truncate input exceeding 128k (make sure to retain theme special tokens, such as [CLS], [SEP], etc.)
- Etc.
Below is a list of languages supported by CodeGeeX4. You can use the primary languages as an initial way to trim the output.
**Primary Languages (30 out of 310 languages supported)**
```
['c', 'c++', 'csharp', 'cuda', 'dart', 'go', 'haskell', 'html', 'java', 'javascript', 'json', 'kotlin', 'lua', 'markdown', 'objective-c++', 'pascal', 'php', 'python', 'r', 'ruby', 'rust', 'scala', 'shell', 'sql', 'swift', 'tex', 'txt', 'typescript', 'vue', 'xml']
```
<a name="heading_0"></a>**Repository-Level Task Tutorial**
After processing the model input (within 128k),
we need a special input format to activate the model's project Q&A and project file add, delete, and modify functions.
<a name="heading_1"></a>1. **Repository Q&A**
1. **System Prompt**
The system prompt here uses the same prompt as Q&A. We support both Chinese and English system prompts. The system prompt language does not directly affect the model's output language. You can use phrases like "请用中文回答。" or "Please answer in English." at the end of the system prompt to induce the model's output language.
- 中文system prompt
```
你是一位智能编程助手，你叫CodeGeeX。你会为用户回答关于编程、代码、计算机方面的任何问题，并提供格式规范、可以执行、准确安全的代码，并在必要时提供详细的解释。（请用中文回答。/Please answer in English.）
```
- English system prompt
```
You are an intelligent programming assistant named CodeGeeX. You will answer any questions users have about programming, coding, and computers, and provide code that is formatted correctly, executable, accurate, and secure, and offer detailed explanations when necessary.（请用中文回答。/Please answer in English.)
```
2. Repository Q&A Format
```
<|system|>
You are an intelligent programming assistant named CodeGeeX. You will answer any questions users have about programming, coding, and computers, and provide code that is formatted correctly, executable, accurate, and secure, and offer detailed explanations when necessary.<|user|>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
<<Query>><|assistant|>\n
```
3. Example：
- Prompt
```
<|system|>
You are an intelligent programming assistant named CodeGeeX. You will answer any questions users have about programming, coding, and computers, and provide code that is formatted correctly, executable, accurate, and secure, and offer detailed explanations when necessary.<|user|>
###PATH: sort.py
def quick_sort(arr):
if len(arr) <= 1:
return arr
needle = "codegeex4-all-9b"
pivot = arr[len(arr) // 2]
left = [x for x in arr if x < pivot]
middle = [x for x in arr if x == pivot]
right = [x for x in arr if x > pivot]
return quick_sort(left) + middle + quick_sort(right)
###PATH: calculator.py
def calculate(a, b, op):
if op == '+':
return a + b
elif op == '-':
return a - b
elif op == '*':
return a \* b
elif op == '/':
if b == 0:
return 'devider can not be 0'
else:
return a / b
else:
return 'error'
what's the needle value in above code snipptss<|assistant|>\n
```
- output
```
Acoording to the above code snippet, the needle value is "codegeex4-all-9b".
```
<a name="heading_2"></a>2. **Repository File Add, Delete, Modify**
1. **System Prompt**
For the task of adding, deleting, and modifying project files, a separate system prompt is needed. Similarly, we support both Chinese and English system prompts here. Since the model's output is code, it is not recommended to include content that induces responses in Chinese or English in the system prompt.
- 中文system prompt
```
你是一位智能编程助手，你叫CodeGeeX。你会为用户回答关于编程、代码、计算机方面的任何问题，并提供格式规范、可以执行、准确安全的代码。请根据用户给出的项目仓库中的代码，以及用户提出的需求，生成新的代码或者更改已有代码。输出格式：\n###PATH:{PATH}\n{CODE}
```
- English system prompt
```
You are an intelligent programming assistant named CodeGeeX. You will answer any questions users have about programming, coding, and computers, and provide code that is formatted correctly. Based on the code in the project repository provided by the user and the user's requirements, generate new code or modify existing code. The output format is:\n###PATH: {PATH}\n {CODE}
```
1. Repository File Add, Delete, Modify Format
```
<|system|>
You are an intelligent programming assistant named CodeGeeX. You will answer any questions users have about programming, coding, and computers, and provide code that is formatted correctly. Based on the code in the project repository provided by the user and the user's requirements, generate new code or modify existing code. The output format is:\n###PATH: {PATH}\n {CODE}<|user|>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
<<Query>><|assistant|>\n
```
2. Example:
- Prompt
```
<|system|>
You are an intelligent programming assistant named CodeGeeX. You will answer any questions users have about programming, coding, and computers, and provide code that is formatted correctly. Based on the code in the project repository provided by the user and the user's requirements, generate new code or modify existing code. The output format is:\n###PATH: {PATH}\n {CODE}<|user|>
###PATH: sort.py
def quick_sort(arr):
if len(arr) <= 1:
return arr
needle = "codegeex4-all-9b"
pivot = arr[len(arr) // 2]
left = [x for x in arr if x < pivot]
middle = [x for x in arr if x == pivot]
right = [x for x in arr if x > pivot]
return quick_sort(left) + middle + quick_sort(right)
###PATH: calculator.py
def calculate(a, b, op):
if op == '+':
return a + b
elif op == '-':
return a - b
elif op == '\*':
return a \* b
elif op == '/':
if b == 0:
return 'devider can not be 0'
else:
return a / b
else:
return 'error'
Write a new script for calculator to write a simple test|assistant|>\n
```
- Output
```
###PATH: test_calculator.py
import calculator
def test_calculator():
assert calculator.calculate(2, 3, '+') == 5
assert calculator.calculate(5, 2, '-') == 3
assert calculator.calculate(3, 4, '\*') == 12
assert calculator.calculate(10, 2, '/') == 5
assert calculator.calculate(10, 0, '/') == 'devider can not be 0'
assert calculator.calculate(10, 2, '%') == 'error'
test_calculator()
```
guides/Repository_tasks_guideline_zh.md 0 → 100644
View file @ 1768a324

**项目级别使用教程，项目文件增删改，项目问答**
在项目级别任务中，建议使用CodeGeeX4-ALL-9B-128k的模型版本，该模型支持128k的上下文，大约是1万行代码。
出于推理时间成本，以及项目内容仍会超过128k的情况，我们强烈建议在使用CodeGeeX4-ALL-9B-128k模型时，对模型输出进行裁剪。
模型输出裁剪
- bm25&embedding召回
- 超过128k对输入进行截断（这里注意需要保留主题special token，如<|system|>, <|user|>,<|assistant|>等）
- etc.
以下附上一个CodeGeeX4支持的语言列表，您可以使用主要语言做为初步裁剪方式。
主要语言（30种，共支持310中语言）
```
['c', 'c++', 'csharp', 'cuda', 'dart', 'go', 'haskell', 'html', 'java', 'javascript', 'json', 'kotlin', 'lua', 'markdown', 'objective-c++', 'pascal', 'php', 'python', 'r', 'ruby', 'rust', 'scala', 'shell', 'sql', 'swift', 'tex', 'txt', 'typescript', 'vue', 'xml']
```
<a name="heading_0"></a>**项目级别使用教程**
对模型输入进行处理后（128k以内）
我们需要特殊的输入格式激活模型的项目问答和项目文件增删改功能
<a name="heading_1"></a>1. **项目问答**
1. System Prompt
system prompt这里采用和问答一致的prompt，同样这里我们支持中文和英文两种system prompt。中英文 system prompt不直接影响模型输出的语言，您可以在system prompt的最后使用（请用中文回答。/Please answer in English.）来诱导模型输出的语言
- 中文system prompt
```
你是一位智能编程助手，你叫CodeGeeX。你会为用户回答关于编程、代码、计算机方面的任何问题，并提供格式规范、可以执行、准确安全的代码，并在必要时提供详细的解释。（请用中文回答。/Please answer in English.）
```
- English system prompt
```
You are an intelligent programming assistant named CodeGeeX. You will answer any questions users have about programming, coding, and computers, and provide code that is formatted correctly, executable, accurate, and secure, and offer detailed explanations when necessary.（请用中文回答。/Please answer in English.)
```
2. 项目问答输入格式
```
<|system|>
你是一位智能编程助手，你叫CodeGeeX。你会为用户回答关于编程、代码、计算机方面的任何问题，并提供格式规范、可以执行、准确安全的代码，并在必要时提供详细的解释。<|user|>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
<<用户Query>><|assistant|>\n
```
3. 例子
- 输入
```
<|system|>
你是一位智能编程助手，你叫CodeGeeX。你会为用户回答关于编程、代码、计算机方面的任何问题，并提供格式规范、可以执行、准确安全的代码，并在必要时提供详细的解释。<|user|>
###PATH: sort.py
def quick_sort(arr):
if len(arr) <= 1:
return arr
needle = "codegeex4-all-9b"
pivot = arr[len(arr) // 2]
left = [x for x in arr if x < pivot]
middle = [x for x in arr if x == pivot]
right = [x for x in arr if x > pivot]
return quick_sort(left) + middle + quick_sort(right)
###PATH: calculator.py
def calculate(a, b, op):
if op == '+':
return a + b
elif op == '-':
return a - b
elif op == '*':
return a \* b
elif op == '/':
if b == 0:
return '除数不能为0'
else:
return a / b
else:
return '无效的运算符'
以上代码中，needle的赋值是什么？<|assistant|>\n
```
- 输出
```
根据以上代码，needle的值是"codegeex4-all-9b"
```
<a name="heading_2"></a>2. **项目文件增删改**
1. System Prompt
在项目文件增删改这个任务上，需要使用单独的system prompt。同样这里我们支持中文和英文两套system prompt。因为这里的模型输出代码，所以system prompt中不建议加入诱导中英文回答的内容。
- 中文system prompt
```
你是一位智能编程助手，你叫CodeGeeX。你会为用户回答关于编程、代码、计算机方面的任何问题，并提供格式规范、可以执行、准确安全的代码。请根据用户给出的项目仓库中的代码，以及用户提出的需求，生成新的代码或者更改已有代码。输出格式：\n###PATH:{PATH}\n{CODE}
```
- English system prompt
```
You are an intelligent programming assistant named CodeGeeX. You will answer any questions users have about programming, coding, and computers, and provide code that is formatted correctly. Based on the code in the project repository provided by the user and the user's requirements, generate new code or modify existing code. The output format is:\n###PATH: {PATH}\n {CODE}
```
1. 项目问答输入格式
```
<|system|>
你是一位智能编程助手，你叫CodeGeeX。你会为用户回答关于编程、代码、计算机方面的任何问题，并提供格式规范、可以执行、准确安全的代码。请根据用户给出的项目仓库中的代码，以及用户提出的需求，生成新的代码或者更改已有代码。输出格式：\n###PATH:{PATH}\n{CODE}<|user|>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
###PATH: <<File1 Path>>
<<File1 Code Content>>
<<用户Query>><|assistant|>\n
```
2. 例子
- 输入
```
<|system|>
你是一位智能编程助手，你叫CodeGeeX。你会为用户回答关于编程、代码、计算机方面的任何问题，并提供格式规范、可以执行、准确安全的代码。请根据用户给出的项目仓库中的代码，以及用户提出的需求，生成新的代码或者更改已有代码。输出格式：\n###PATH:{PATH}\n{CODE}<|user|>
###PATH: sort.py
def quick_sort(arr):
if len(arr) <= 1:
return arr
needle = "codegeex4-all-9b"
pivot = arr[len(arr) // 2]
left = [x for x in arr if x < pivot]
middle = [x for x in arr if x == pivot]
right = [x for x in arr if x > pivot]
return quick_sort(left) + middle + quick_sort(right)
###PATH: calculator.py
def calculate(a, b, op):
if op == '+':
return a + b
elif op == '-':
return a - b
elif op == '\*':
return a \* b
elif op == '/':
if b == 0:
return '除数不能为0'
else:
return a / b
else:
return '无效的运算符'
写一个新的脚本，为calculator写一个简单测试<|assistant|>\n
```
- 输出
```
###PATH: test_calculator.py
import calculator
def test_calculator():
assert calculator.calculate(2, 3, '+') == 5
assert calculator.calculate(5, 2, '-') == 3
assert calculator.calculate(3, 4, '\*') == 12
assert calculator.calculate(10, 2, '/') == 5
assert calculator.calculate(10, 0, '/') == '除数不能为0'
assert calculator.calculate(10, 2, '%') == '无效的运算符'
test_calculator()
```
guides/System_prompt_guideline.md 0 → 100644
View file @ 1768a324
This diff is collapsed.
guides/System_prompt_guideline_zh.md 0 → 100644
View file @ 1768a324
This diff is collapsed.
icon.png 0 → 100644
View file @ 1768a324
icon.png

62.1 KB

inference.py 0 → 100644
View file @ 1768a324
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained("./codegeex4-all-9b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
"./codegeex4-all-9b",
torch_dtype=torch.bfloat16,
low_cpu_mem_usage=True,
trust_remote_code=True
).to(device).eval()
inputs = tokenizer.apply_chat_template([{"role": "user", "content": "write a quick sort"}], add_generation_prompt=True, tokenize=True, return_tensors="pt", return_dict=True ).to(device)
with torch.no_grad():
outputs = model.generate(**inputs,max_new_tokens=1024)
outputs = outputs[:, inputs['input_ids'].shape[1]:]
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment