Commits · 3a354bd1da9eabad19fa6ff1afe0132e46d55424 · OpenDAS / opencompass

07 Dec, 2023 1 commit
- add qwen and deepseek configs (#672) · 3a354bd1
  Fengzhe Zhou authored Dec 07, 2023
  
  3a354bd1
06 Dec, 2023 1 commit

New subjective judgement (#660) · 1c95790f

bittersweet1999 authored Dec 06, 2023



* TabMWP

* TabMWP

* fixed

* fixed

* fixed

* done

* done

* done

* add new subjective judgement

* add new subjective judgement

* add new subjective judgement

* add new subjective judgement

* add new subjective judgement

* modified to a more general way

* modified to a more general way

* final

* final

* add summarizer

* add new summarize

* fixed

* fixed

* fixed

---------
Co-authored-by: caomaosong <caomaosong@pjlab.org.cn>

1c95790f

01 Dec, 2023 4 commits

added rolebench dataset. (#633) · e10f1c91

rolellm authored Dec 01, 2023

* added rolebench

* 修改了不合理的变量名

* 修改了评论中的变量名

e10f1c91

[Feature] Update MathBench CodeInterpreter & fix MathBench Bug (#657) · f4bbff65

liushz authored Dec 01, 2023



* Update MathBench CodeInterpreter & fix MathBench Bug

* Fix errors

* update

---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: Fengzhe Zhou <zfz-960727@163.com>

f4bbff65

[Feat] update gsm8k and math agent config (#652) · 9eb5cadc
Hubert authored Dec 01, 2023
```
* [Feat] update gsm8k and math agent config

* minor fix
```
9eb5cadc

[Feature] Add wikibench dataset (#655) · a331c9ab

liushz authored Dec 01, 2023



* Add WikiBench

* Add WikiBench

* format

---------
Co-authored-by: Leymore <zfz-960727@163.com>

a331c9ab

30 Nov, 2023 2 commits

[Feature] Add Chinese version: commonsenseqa, crowspairs and nq (#144) · e019c831

liushz authored Nov 30, 2023



* add Chinese version: csqa crowspairs nq

* Update cn_data

* Update cn_data

* update format

---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: Leymore <zfz-960727@163.com>

e019c831

[Feature] Support chat style inferencer. (#643) · 6aaf3b91

Ma Zerun authored Nov 30, 2023



* [Feature] Support chat style inferencer.

* [Fix] use new prompt

* [Fix] use new prompt

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>

6aaf3b91

29 Nov, 2023 1 commit
- fix hellaswag_ppl_47bff9 (#648) · 5933c04f
  Fengzhe Zhou authored Nov 29, 2023
  
  5933c04f
27 Nov, 2023 3 commits

[Feat] support zhipu post process (#642) · d4af31ba

Hubert authored Nov 27, 2023

* [Feat] support zhipu post

* [Feat] support zhipu post

* [Feat] support zhipu post

d4af31ba

[Feature] Add GSM_Hard dataset (#619) · 6d0d7898

liushz authored Nov 27, 2023



* Add SVAMP dataset

* Add SVAMP dataset

* Add SVAMP dataset

* Add gsm_hard dataset

* Add gsm_hard dataset

* format

---------
Co-authored-by: Leymore <zfz-960727@163.com>

6d0d7898

[Sync] some renaming (#641) · 9083dea6
Fengzhe Zhou authored Nov 27, 2023

9083dea6

23 Nov, 2023 3 commits
- [Feature] Add circular eval (#610) · d949e3c0
  Fengzhe Zhou authored Nov 23, 2023
```
* refactor default, add circular summarizer

* add circular

* update impl

* update doc

* minor update

* no more to be added
```
  d949e3c0
- [API] Update API (#624) · 5202456b
  Songyang Zhang authored Nov 23, 2023
```
* update api

* update generation_kwargs impl

* update api

* refactor

---------
Co-authored-by: Leymore <zfz-960727@163.com>
```
  5202456b
- [Sync] Fix cmnli, fix vicuna meta template, fix longbench postprocess and other minor fixes (#625) · d4d1330a
  Fengzhe Zhou authored Nov 23, 2023
  
  d4d1330a
22 Nov, 2023 2 commits

[Feature] support download from modelscope (#534) · c0785e53

Kevin Wang authored Nov 22, 2023



* [Feature] download from modelscope

* [Feature] download from modelscope

* minor fix

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>

c0785e53

[Feature] Add SVAMP dataset (#604) · 04877519
liushz authored Nov 22, 2023
```
* Add SVAMP dataset

* Add SVAMP dataset

* Add SVAMP dataset
```
04877519

21 Nov, 2023 4 commits

Integrate turbomind python api (#484) · eb56fd6d

Lyu Han authored Nov 21, 2023



* integrate turbomind python api

* update

* update user guide

* update

* fix according to reviewer's comments

* fix error

* fix linting

* update user guide

* remove debug log

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

eb56fd6d

[Feature] Support 360API and FixKRetriever for CSQA dataset (#601) · d9257482

Songyang Zhang authored Nov 21, 2023



* [Feature] Support 360API and FixKRetriever for CSQA dataset

* Update API

* Update API

* [Feature] Support 360API and FixKRetriever for CSQA dataset

* Update API

* Update API

* rm mathbench

* fix_lint

* Update opencompass/models/bytedance_api.py
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>

* update

* update

* update

---------
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>

d9257482

[Feature] Support Lightllm API (#613) · d3b0d5c4

Yang Yong authored Nov 21, 2023



* [Feature] Support Lightllm api

* formatting & renaming

---------
Co-authored-by: Leymore <zfz-960727@163.com>

d3b0d5c4

Add support for DataCanvas Alaya LM (#612) · 7199acc2
Yuan Feng authored Nov 21, 2023
```
* Support for Alaya

* Remove useless requirements
```
7199acc2

20 Nov, 2023 3 commits

Add aritch to mathbench (#607) · dbacd363
liushz authored Nov 20, 2023

dbacd363
Mathbench update postprocess (#600) · c9c5c5d9
liushz authored Nov 20, 2023
```
* Update mathbench

* Update mathbench
```
c9c5c5d9

[Feature] Add multi-prompt generation demo (#568) · 5e75e297

Jingming authored Nov 20, 2023



* [Feature] Add multi-prompt generation demo

* [Fix] change form in winogrande_gen_XXX.py

* [Fix] make multi prompt demo more directly

* [Fix] fix bug

* [Fix] minor fix

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>

5e75e297

16 Nov, 2023 2 commits

[Feat] support humaneval and mbpp pass@k (#598) · 91fba2c2

Hubert authored Nov 16, 2023



* [Feat] support pass@ k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k docs

* update naming

---------
Co-authored-by: Leymore <zfz-960727@163.com>

91fba2c2

[Feature] Add FinanceIQ dataset (#596) · c0acd06b
Raymond Zhang authored Nov 16, 2023

c0acd06b

15 Nov, 2023 1 commit
- update word spell (#594) · 8160cb84
  Yu authored Nov 15, 2023
  
  8160cb84
14 Nov, 2023 2 commits
- Fix bugs in subjective evaluation (#589) · 14e6fe6f
  Wei Jueqi authored Nov 14, 2023
```
* rename

* fix sub bugs and update docs

* update

* update
```
  14e6fe6f
- [Feature] Update mathbench (#580) · c8cb38e8
  Songyang Zhang authored Nov 14, 2023
```
* update xunfei api

* fix lint

* update mathbench to avoid incomplete prediction
```
  c8cb38e8
13 Nov, 2023 3 commits

[Sync] Bump version to 0.1.8 (#576) · 1ea88d58
Fengzhe Zhou authored Nov 13, 2023

1ea88d58
[Sync] update model configs (#574) · d3de5c41
Fengzhe Zhou authored Nov 13, 2023

d3de5c41

[Feature] Use dataset in local path (#570) · 689ffe5b

Fengzhe Zhou authored Nov 13, 2023

* update commonsenseqa

* update drop

* update flores_first100

* update gsm8k

* update humaneval

* update lambda

* update obqa

* update piqa

* update race

* update siqa

* update story_cloze

* update strategyqa

* update tydiqa

* update winogrande

* update doc

* update hellaswag

* fix obqa

* update collections

* update .zip name

689ffe5b

12 Nov, 2023 1 commit
- [Feature] Update cmb (#571) · d6aaac22
  Fengzhe Zhou authored Nov 13, 2023
  
  d6aaac22
10 Nov, 2023 2 commits
- [Docs] fix dataset name error (#533) · 7f77e8da
  Kevin Wang authored Nov 10, 2023
  
  7f77e8da
- [Docs] add humanevalx dataset link in config (#559) · 95e0da01
  Hubert authored Nov 10, 2023
```
* [Docs] add humanevalx dataset link in config

* [Docs] add humanevalx dataset link in config

* minor fix
```
  95e0da01
09 Nov, 2023 2 commits

[Feature] Add py150 and maxmin (#562) · b3cbef32

jingmingzhuo authored Nov 09, 2023

* [feat] add clozeTesst_maxmin dataset

* [feat] add py150 datasets

* [feat] change __init__.py in opencompass/datasets

* [fix] pre-commit check

* [fix] rename py150 and masxmin datasets in configs

* [feat] add gen.py of py150 and maxmin in configs/datasets

b3cbef32

[Fix] fix log re-direct (#564) · 889a6b26
Hubert authored Nov 09, 2023

889a6b26

07 Nov, 2023 2 commits

[Feat] Support cibench (#538) · bb2ecf41

Hubert authored Nov 07, 2023

* [Feat] support cidataset

* [Feat] support cidataset

* [Feat] support cidataset

* [Feat] support cidataset

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* rename cibench

* rename cibench

* rename cibench

* rename cibench

* minor fix

* minor fix

* minor fix

bb2ecf41

[Fix] fix filename typo (#549) · 36360bdf
Hubert authored Nov 07, 2023

36360bdf

06 Nov, 2023 1 commit
- 【Feature】Update Mathbench dataset prompt and fix small errors (#546) · 214a34f0
  liushz authored Nov 06, 2023
```
* Update mathbench

* Update mathbench

* Update mathbench
```
  214a34f0