If you have already identified the reason, we strongly appreciate you creating a new PR according to [the tutorial](https://opencompass.readthedocs.io/en/master/community/CONTRIBUTING.html)!
If you need our help, please fill in the following form to help us to identify the bug.
-type:checkboxes
attributes:
label:Prerequisite
description:Please check the following items before creating a new issue.
options:
-label:I have searched [Issues](https://github.com/InternLM/opencompass/issues/) and [Discussions](https://github.com/InternLM/opencompass/discussions) but cannot get the expected help.
required:true
-label:The bug has not been fixed in the [latest version](https://github.com/InternLM/opencompass).
required:true
-type:dropdown
id:task
attributes:
label:Type
description:The problem arises when
options:
-I'm evaluating with the officially supported tasks/models/datasets.
-I have modified the code (config is not considered code), or I'm working on my own tasks/models/datasets.
validations:
required:true
-type:textarea
id:describe
id:environment
validations:
required:true
attributes:
label:Describe the bug
label:Environment
description:|
Please provide a clear and concise description of what the bug is.
Preferably a simple and minimal code snippet that we can reproduce the error by running the code.
Please run `python -c "import opencompass.utils;import pprint;pprint.pprint(dict(opencompass.utils.collect_env()))"` to collect necessary environment information and paste it here.
placeholder:|
A clear and concise description of what the bug is.
```python
# The output the above command
```
-type:textarea
attributes:
label:Reproduces the problem - code/configuration sample
description:|
Please provide a code or configuration sample that reproduces the problem you ran into. It can be a Colab link or just a code snippet.
placeholder:|
```python
# Sample code to reproduce the problem
```
validations:
required:true
-type:textarea
attributes:
label:Reproduces the problem - command or script
description:|
What command or script did you run?
placeholder:|
```shell
The command or script you run.
```
```
The error message or logs you got, with the full traceback.
```
-type:textarea
id:environment
validations:
required:true
-type:textarea
attributes:
label:Environment
label:Reproduces the problem - error message
description:|
Please run `python -c "import opencompass.utils;import pprint;pprint.pprint(dict(opencompass.utils.collect_env()))"` to collect necessary environment information and paste it here.
Please provide the error message or logs you got, with the full traceback.
Tip: You can attach images or log files by dragging them into the text area..
placeholder:|
```python
# The output the above command
```
The error message or logs you got, with the full traceback.
```
validations:
required:true
-type:textarea
id:other
...
...
@@ -55,5 +89,6 @@ body:
description:|
Tell us anything else you think we should know.
1. Did you make any modifications on the code or config?