Commits · dd444685bbd7fe2207e21851c778fe92a3d29853 · OpenDAS / opencompass

06 Feb, 2024 4 commits

fix bug of gsm8k_postprocess (#863) · dd444685

hailsham authored Feb 06, 2024



* fix bug of gsm8k_postprocess

* update postprocess

---------
Co-authored-by: Lei Fei <SENSETIME\leifei1@cn3114002087l.domain.sensetime.com>
Co-authored-by: Leymore <zfz-960727@163.com>

dd444685

[feat] support multipl-e (#846) · 444d8d95

Connor-Shen authored Feb 06, 2024



* [feat] support humaneval_multipl-e

* format

---------
Co-authored-by: Leymore <zfz-960727@163.com>

444d8d95

fix lawbench 2-1 f0.5 score calculation bug (#795) · a6c49f15

Yggdrasill7D6 authored Feb 06, 2024



* fix lawbench 2-1 f0.5 score calculation bug

* use path in overall datasets folder

---------
Co-authored-by: Leymore <zfz-960727@163.com>

a6c49f15

[Fix] hotfix for mtbench (#877) · 1c8e193d
bittersweet1999 authored Feb 06, 2024
```
* hotfix for mtbench

* hotfix
```
1c8e193d

05 Feb, 2024 5 commits

[Sync] Merge branch 'dev' into zfz/update-keyset-demo (#876) · d34ba111
Fengzhe Zhou authored Feb 05, 2024

d34ba111

[Fix] add do sample demo for subjective dataset (#873) · 32b5948f

bittersweet1999 authored Feb 05, 2024



* add do sample demo for subjective dataset

* fix strings

* format

---------
Co-authored-by: Leymore <zfz-960727@163.com>

32b5948f

Support NPHardEval (#835) · 7ad11680

Skyfall-xzz authored Feb 05, 2024



* support NPHardEval

* add .md file and fix minor bugs

* refactor and minor fix

---------
Co-authored-by: Leymore <zfz-960727@163.com>

7ad11680

Update daily test (#871) · b4a9acd7

zhulinJulia24 authored Feb 05, 2024



* add daily test case

* Update pr-run-test.yml

* Update daily-run-test.yml

* Update daily-run-test.yml

* Update pr-run-test.yml

* Update daily-run-test.yml

* Update oc_score_assert.py

* Update daily-run-test.yml

* Update daily-run-test.yml

* Update daily-run-test.yml

* update testcase baseline

* fix test case name

* add more models into daily test

---------
Co-authored-by: zhulin1 <zhulin1@pjlab.org.cn>
Co-authored-by: Leymore <zfz-960727@163.com>

b4a9acd7

[CI] Update github workflow cuda image (#874) · fc84aff9
Fengzhe Zhou authored Feb 05, 2024
```
* update workflow

* another trial

* another trial

* another trial
```
fc84aff9

04 Feb, 2024 2 commits

[Fix] Fix error in gsm8k evaluator (#782) · fed7d800
Yuchen Yan authored Feb 04, 2024
```
Co-authored-by: jiangjin1999 <1261842974@qq.com>
```
fed7d800

[Feature] support alpacaeval (#809) · 7806cd0f

bittersweet1999 authored Feb 04, 2024



* support alpacaeval_v1

* Update opencompass/summarizers/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/summarizers/subjective/alpacaeval_v1.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* fix conflict

* support alpacaeval v2

* support alpacav2

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

7806cd0f

02 Feb, 2024 1 commit

[Feature] Add daily test case (#864) · 0919b08e

zhulinJulia24 authored Feb 02, 2024



* add daily test case

* Update pr-run-test.yml

* Update daily-run-test.yml

* Update daily-run-test.yml

* Update pr-run-test.yml

---------
Co-authored-by: zhulin1 <zhulin1@pjlab.org.cn>

0919b08e

01 Feb, 2024 1 commit
- [Feature] Add end_str for turbomind (#859) · 4c87e777
  RunningLeon authored Feb 01, 2024
```
* fix

* update

* fix internlm1

* fix docs

* remove sys
```
  4c87e777
30 Jan, 2024 1 commit
- fix compass arena (#854) · 5c6dc908
  bittersweet1999 authored Jan 30, 2024
  
  5c6dc908
29 Jan, 2024 2 commits

Update runtime.txt to fix rouge_chinese bugs. (#803) · 4f78388c

Guo Qipeng authored Jan 29, 2024



* Update runtime.txt to fix rouge_chinese bugs.

the wheel file of rouge_chinese will overwrite the rouge package, causing bugs. Replacing it to the github code, which is the correct version.

* fix PEP format issues

* fix PEP format issues

* enable pip install

---------
Co-authored-by: 郭琦鹏 <guoqipeng@pjlab.org.cn>
Co-authored-by: Leymore <zfz-960727@163.com>

4f78388c

Create link-check.yml (#853) · e8067ac4
del-zhenwu authored Jan 29, 2024
```
* Create link-check.yml

* Update link-check.yml
```
e8067ac4

28 Jan, 2024 1 commit
- [Fix] Update Zhipu API and Fix issue min_out_len issue of API models (#847) · cdca59ff
  Songyang Zhang authored Jan 28, 2024
```
* Update zhipu api and fix min_out_len issue of API class

* Update example

* Update example
```
  cdca59ff
27 Jan, 2024 1 commit
- [Fix] Fix acc of IFEval (#849) · 28018833
  Jingming authored Jan 27, 2024
```
* [Feature] Add IFEval

* [Fix] Changing the Score Rule.
```
  28018833
26 Jan, 2024 3 commits
- [Fix] Update MedBench (#845) · 35aace77
  Xiaoming Shi authored Jan 26, 2024
  
  35aace77
- Update Sensetime API (#844) · 8ed022b4
  Songyang Zhang authored Jan 26, 2024
  
  8ed022b4
- [Feat] minor update agent related (#839) · 4aa74565
  Hubert authored Jan 26, 2024
```
* [Feat] update cibench

* [Feat] Support CIBench

* [Feat] Support CIBench

* [Feat] Support CIBench

* [Feat] Support CIBench
```
  4aa74565
24 Jan, 2024 6 commits

[Fix] fix corev2 (#838) · 77be07db
bittersweet1999 authored Jan 24, 2024
```
* fix corev2

* fix corev2
```
77be07db
[Sync] Updata dataset cfg for internMath (#837) · 0991dd33
Fengzhe Zhou authored Jan 24, 2024
```
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
```
0991dd33
add fail notify (#836) · f7d7837a
zhulinJulia24 authored Jan 24, 2024

f7d7837a
update doc (#830) · f3675516
Fengzhe Zhou authored Jan 24, 2024

f3675516
[Feature] Update API implementation (#834) · 793e32c9
Songyang Zhang authored Jan 24, 2024

793e32c9

[Feature] add mtbench (#829) · 2ee8e8a1

bittersweet1999 authored Jan 24, 2024



* add mtbench

* add mtbench

* Update configs/datasets/subjective/multiround/mtbench_judgeby_gpt4.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/multiround/mtbench_judgeby_gpt4.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/mtbench.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* fix mtbench

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

2ee8e8a1

23 Jan, 2024 3 commits

[Feature] Add IFEval (#813) · e059a5c2
Jingming authored Jan 23, 2024
```
* [Feature] Add IFEval

* [Doc] add introduction of IFEval
```
e059a5c2

[Fix] fix strings (#833) · 3d9bb4ae

bittersweet1999 authored Jan 23, 2024



* add compass arena

* add compass_arena

* add compass arena

* Update opencompass/summarizers/subjective/compass_arena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/summarizers/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/compass_arena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/eval_subjective_compassarena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/compassarena/compassarena_compare.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/eval_subjective_compassarena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/compassarena/compassarena_compare.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* fix check position bias

* fix string

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

3d9bb4ae

[Feature] Add CompassArena (#828) · 2d4da8dd

bittersweet1999 authored Jan 23, 2024



* add compass arena

* add compass_arena

* add compass arena

* Update opencompass/summarizers/subjective/compass_arena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/summarizers/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/compass_arena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/eval_subjective_compassarena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/compassarena/compassarena_compare.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/eval_subjective_compassarena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/compassarena/compassarena_compare.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* fix check position bias

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

2d4da8dd

19 Jan, 2024 2 commits

Update hf_internlm2_chat template (#823) · 40a2441d
RangiLyu authored Jan 19, 2024
```
* Update hf_internlm2_chat template

* Update 20B
```
40a2441d

Update cdme config and evaluator (#812) · e975a96f

Guo Qipeng authored Jan 19, 2024



* update cdme config and evaluator

* fix cdme prompt

* move CDME trim post-processor as a separate evaluator

---------
Co-authored-by: 郭琦鹏 <guoqipeng@pjlab.org.cn>

e975a96f

18 Jan, 2024 4 commits
- Add LightllmApi KeyError log & Update doc (#816) · f09a2ff4
  Yang Yong authored Jan 18, 2024
```
* Add LightllmApi KeyError log

* Update LightllmApi doc
```
  f09a2ff4
- Test runner update - split step, change schedule time and disable hf cache (#814) · 8b5c467c
  zhulinJulia24 authored Jan 18, 2024
```
* Update pr-run-test.yml

* Update pr-run-test.yml

* Update pr-run-test.yml

* split step and change order, change schedule time and disable hf cache
```
  8b5c467c
- [Fix] Update yi 200k config (#815) · dcc32ed8
  Mo Li authored Jan 18, 2024
  
  dcc32ed8
- [Fix] Fix turbomind and update docs (#808) · 61fe873c
  RunningLeon authored Jan 18, 2024
```
* update

* update docs

* add engine_config and gen_config in eval_config

* update

* fix

* fix

* fix

* fix docstr

* fix url
```
  61fe873c
17 Jan, 2024 4 commits

[Doc] Update News (#810) · 9e5746d3
Fengzhe Zhou authored Jan 17, 2024

9e5746d3
[Sync] Add InternLM2 Keyset Evaluation Demo (#807) · b4afe3e7
Fengzhe Zhou authored Jan 17, 2024
```
Co-authored-by: zhangyifan1 <zhangyifan1@pjlab.org.cn>
```
b4afe3e7

Added support for multi-needle testing in needle-in-a-haystack test (#802) · acae5609

Mo Li authored Jan 17, 2024



* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

* add English version support

* change NeedleInAHaystackDataset to dynamic loading

* change NeedleInAHaystackDataset to dynamic loading

* fix needleinahaystack test eval bug

* fix needleinahaystack config bug

* Added support for multi-needle testing in needle-in-a-haystack test

* Optimize the code for plotting in the needle-in-a-haystack test.

* Correct the typo in the dataset parameters.

* update needleinahaystack test docs

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

acae5609

[Feature] Update evaluate turbomind (#804) · 0836aec6
RunningLeon authored Jan 17, 2024
```
* update

* fix

* fix

* fix
```
0836aec6