If you have already identified the reason, we strongly appreciate you creating a new PR according to [the tutorial](https://opencompass.readthedocs.io/en/master/community/CONTRIBUTING.html)!
If you have already identified the reason, we strongly appreciate you creating a new PR according to [the tutorial](https://opencompass.readthedocs.io/en/master/community/CONTRIBUTING.html)!
If you need our help, please fill in the following form to help us to identify the bug.
If you need our help, please fill in the following form to help us to identify the bug.
-type:checkboxes
attributes:
label:Prerequisite
description:Please check the following items before creating a new issue.
options:
-label:I have searched [Issues](https://github.com/InternLM/opencompass/issues/) and [Discussions](https://github.com/InternLM/opencompass/discussions) but cannot get the expected help.
required:true
-label:The bug has not been fixed in the [latest version](https://github.com/InternLM/opencompass).
required:true
-type:dropdown
id:task
attributes:
label:Type
description:The problem arises when
options:
-I'm evaluating with the officially supported tasks/models/datasets.
-I have modified the code (config is not considered code), or I'm working on my own tasks/models/datasets.
validations:
required:true
-type:textarea
-type:textarea
id:describe
id:environment
validations:
validations:
required:true
required:true
attributes:
attributes:
label:Describe the bug
label:Environment
description:|
description:|
Please provide a clear and concise description of what the bug is.
Please run `python -c "import opencompass.utils;import pprint;pprint.pprint(dict(opencompass.utils.collect_env()))"` to collect necessary environment information and paste it here.
Preferably a simple and minimal code snippet that we can reproduce the error by running the code.
placeholder:|
placeholder:|
A clear and concise description of what the bug is.
```python
# The output the above command
```
-type:textarea
attributes:
label:Reproduces the problem - code/configuration sample
description:|
Please provide a code or configuration sample that reproduces the problem you ran into. It can be a Colab link or just a code snippet.
placeholder:|
```python
```python
# Sample code to reproduce the problem
# Sample code to reproduce the problem
```
```
validations:
required:true
-type:textarea
attributes:
label:Reproduces the problem - command or script
description:|
What command or script did you run?
placeholder:|
```shell
```shell
The command or script you run.
The command or script you run.
```
```
```
The error message or logs you got, with the full traceback.
```
-type:textarea
id:environment
validations:
validations:
required:true
required:true
-type:textarea
attributes:
attributes:
label:Environment
label:Reproduces the problem - error message
description:|
description:|
Please run `python -c "import opencompass.utils;import pprint;pprint.pprint(dict(opencompass.utils.collect_env()))"` to collect necessary environment information and paste it here.
Please provide the error message or logs you got, with the full traceback.
Tip: You can attach images or log files by dragging them into the text area..
placeholder:|
placeholder:|
```python
# The output the above command
```
```
The error message or logs you got, with the full traceback.
```
validations:
required:true
-type:textarea
-type:textarea
id:other
id:other
...
@@ -55,5 +89,6 @@ body:
...
@@ -55,5 +89,6 @@ body:
description:|
description:|
Tell us anything else you think we should know.
Tell us anything else you think we should know.
1. Did you make any modifications on the code or config?