Commits · 120bf8b3990e5e3c085683398edd5997b7d31005 · OpenDAS / opencompass

01 Mar, 2024 1 commit
- add vllm model configs (#938) · 120bf8b3
  Mo Li authored Mar 01, 2024
  
  120bf8b3
29 Feb, 2024 1 commit
- [Feature] Support OpenFinData (#896) · 4c45a71b
  Skyfall-xzz authored Feb 29, 2024
```
* [Feature] Support OpenFinData

* add README for OpenFinData

* update README
```
  4c45a71b
28 Feb, 2024 1 commit
- [Feature] add support for gemini (#931) · 001e77fe
  bittersweet1999 authored Feb 28, 2024
```
* add gemini

* add gemini

* add gemini
```
  001e77fe
23 Feb, 2024 1 commit
- [Fix] fix ifeval (#909) · 53fe788d
  Jingming authored Feb 23, 2024
  
  53fe788d
22 Feb, 2024 2 commits
- [Fix] Fix IFEval (#906) · 45c606bc
  bittersweet1999 authored Feb 22, 2024
```
* fix ifeval

* fix ifeval

* fix ifeval

* fix ifeval
```
  45c606bc
- Support lmdeploy pytorch engine (#875) · 32ba0b07
  RunningLeon authored Feb 22, 2024
```
* add lmdeploy pytorch model

* fix

* speed up encoding and decoding

* fix

* change tokenizer
```
  32ba0b07
21 Feb, 2024 1 commit
- [Fix] Fix moss template config (#897) · 6d04deca
  Xu Song authored Feb 21, 2024
  
  6d04deca
19 Feb, 2024 3 commits
- [Fix] Fix chatglm2 config (#893) · 2b7d376e
  Fengzhe Zhou authored Feb 19, 2024
  
  2b7d376e
- [Fix] rename qwen2-beta -> qwen1.5 (#894) · 9119e2ac
  Fengzhe Zhou authored Feb 19, 2024
  
  9119e2ac
- Support LightllmApi input_format (#888) · b6e21ece
  Yang Yong authored Feb 19, 2024
  
  b6e21ece
06 Feb, 2024 4 commits

[Feature] add global retriever config (#842) · e257254b

hailsham authored Feb 07, 2024



* add global retriever config

* give zero shot overwrite example

* give zero shot overwrite example

---------
Co-authored-by: Lei Fei <SENSETIME\leifei1@cn3114002087l.domain.sensetime.com>
Co-authored-by: Leymore <zfz-960727@163.com>

e257254b

fix bug of gsm8k_postprocess (#863) · dd444685

hailsham authored Feb 06, 2024



* fix bug of gsm8k_postprocess

* update postprocess

---------
Co-authored-by: Lei Fei <SENSETIME\leifei1@cn3114002087l.domain.sensetime.com>
Co-authored-by: Leymore <zfz-960727@163.com>

dd444685

[feat] support multipl-e (#846) · 444d8d95

Connor-Shen authored Feb 06, 2024



* [feat] support humaneval_multipl-e

* format

---------
Co-authored-by: Leymore <zfz-960727@163.com>

444d8d95

[Fix] hotfix for mtbench (#877) · 1c8e193d
bittersweet1999 authored Feb 06, 2024
```
* hotfix for mtbench

* hotfix
```
1c8e193d

05 Feb, 2024 3 commits
- [Sync] Merge branch 'dev' into zfz/update-keyset-demo (#876) · d34ba111
  Fengzhe Zhou authored Feb 05, 2024
  
  d34ba111
- [Fix] add do sample demo for subjective dataset (#873) · 32b5948f
  bittersweet1999 authored Feb 05, 2024
```
* add do sample demo for subjective dataset

* fix strings

* format

---------
Co-authored-by: Leymore <zfz-960727@163.com>
```
  32b5948f
- Support NPHardEval (#835) · 7ad11680
  Skyfall-xzz authored Feb 05, 2024
```
* support NPHardEval

* add .md file and fix minor bugs

* refactor and minor fix

---------
Co-authored-by: Leymore <zfz-960727@163.com>
```
  7ad11680
04 Feb, 2024 1 commit

[Feature] support alpacaeval (#809) · 7806cd0f

bittersweet1999 authored Feb 04, 2024



* support alpacaeval_v1

* Update opencompass/summarizers/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/summarizers/subjective/alpacaeval_v1.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* fix conflict

* support alpacaeval v2

* support alpacav2

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

7806cd0f

01 Feb, 2024 1 commit
- [Feature] Add end_str for turbomind (#859) · 4c87e777
  RunningLeon authored Feb 01, 2024
```
* fix

* update

* fix internlm1

* fix docs

* remove sys
```
  4c87e777
30 Jan, 2024 1 commit
- fix compass arena (#854) · 5c6dc908
  bittersweet1999 authored Jan 30, 2024
  
  5c6dc908
28 Jan, 2024 1 commit
- [Fix] Update Zhipu API and Fix issue min_out_len issue of API models (#847) · cdca59ff
  Songyang Zhang authored Jan 28, 2024
```
* Update zhipu api and fix min_out_len issue of API class

* Update example

* Update example
```
  cdca59ff
27 Jan, 2024 1 commit
- [Fix] Fix acc of IFEval (#849) · 28018833
  Jingming authored Jan 27, 2024
```
* [Feature] Add IFEval

* [Fix] Changing the Score Rule.
```
  28018833
26 Jan, 2024 3 commits
- [Fix] Update MedBench (#845) · 35aace77
  Xiaoming Shi authored Jan 26, 2024
  
  35aace77
- Update Sensetime API (#844) · 8ed022b4
  Songyang Zhang authored Jan 26, 2024
  
  8ed022b4
- [Feat] minor update agent related (#839) · 4aa74565
  Hubert authored Jan 26, 2024
```
* [Feat] update cibench

* [Feat] Support CIBench

* [Feat] Support CIBench

* [Feat] Support CIBench

* [Feat] Support CIBench
```
  4aa74565
24 Jan, 2024 3 commits

[Fix] fix corev2 (#838) · 77be07db
bittersweet1999 authored Jan 24, 2024
```
* fix corev2

* fix corev2
```
77be07db
[Sync] Updata dataset cfg for internMath (#837) · 0991dd33
Fengzhe Zhou authored Jan 24, 2024
```
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
```
0991dd33

[Feature] add mtbench (#829) · 2ee8e8a1

bittersweet1999 authored Jan 24, 2024



* add mtbench

* add mtbench

* Update configs/datasets/subjective/multiround/mtbench_judgeby_gpt4.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/multiround/mtbench_judgeby_gpt4.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/mtbench.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* fix mtbench

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

2ee8e8a1

23 Jan, 2024 2 commits

[Feature] Add IFEval (#813) · e059a5c2
Jingming authored Jan 23, 2024
```
* [Feature] Add IFEval

* [Doc] add introduction of IFEval
```
e059a5c2

[Feature] Add CompassArena (#828) · 2d4da8dd

bittersweet1999 authored Jan 23, 2024



* add compass arena

* add compass_arena

* add compass arena

* Update opencompass/summarizers/subjective/compass_arena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/summarizers/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/compass_arena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/eval_subjective_compassarena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/compassarena/compassarena_compare.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/eval_subjective_compassarena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/compassarena/compassarena_compare.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* fix check position bias

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

2d4da8dd

19 Jan, 2024 2 commits

Update hf_internlm2_chat template (#823) · 40a2441d
RangiLyu authored Jan 19, 2024
```
* Update hf_internlm2_chat template

* Update 20B
```
40a2441d

Update cdme config and evaluator (#812) · e975a96f

Guo Qipeng authored Jan 19, 2024



* update cdme config and evaluator

* fix cdme prompt

* move CDME trim post-processor as a separate evaluator

---------
Co-authored-by: 郭琦鹏 <guoqipeng@pjlab.org.cn>

e975a96f

18 Jan, 2024 2 commits
- [Fix] Update yi 200k config (#815) · dcc32ed8
  Mo Li authored Jan 18, 2024
  
  dcc32ed8
- [Fix] Fix turbomind and update docs (#808) · 61fe873c
  RunningLeon authored Jan 18, 2024
```
* update

* update docs

* add engine_config and gen_config in eval_config

* update

* fix

* fix

* fix

* fix docstr

* fix url
```
  61fe873c
17 Jan, 2024 3 commits

[Sync] Add InternLM2 Keyset Evaluation Demo (#807) · b4afe3e7
Fengzhe Zhou authored Jan 17, 2024
```
Co-authored-by: zhangyifan1 <zhangyifan1@pjlab.org.cn>
```
b4afe3e7

Added support for multi-needle testing in needle-in-a-haystack test (#802) · acae5609

Mo Li authored Jan 17, 2024



* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

* add English version support

* change NeedleInAHaystackDataset to dynamic loading

* change NeedleInAHaystackDataset to dynamic loading

* fix needleinahaystack test eval bug

* fix needleinahaystack config bug

* Added support for multi-needle testing in needle-in-a-haystack test

* Optimize the code for plotting in the needle-in-a-haystack test.

* Correct the typo in the dataset parameters.

* update needleinahaystack test docs

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

acae5609

[Feature] Update evaluate turbomind (#804) · 0836aec6
RunningLeon authored Jan 17, 2024
```
* update

* fix

* fix

* fix
```
0836aec6

16 Jan, 2024 1 commit
- reorganize subject files (#801) · 814b3f73
  bittersweet1999 authored Jan 16, 2024
  
  814b3f73
12 Jan, 2024 1 commit

[Feature] Add configs for creationbench (#791) · 83d6c483

bittersweet1999 authored Jan 12, 2024

* add creationv2_zh

* add creationv2_zh

* add eng config for creationbench

* add eng config for creationbench

* add eng config for creationbench

83d6c483

11 Jan, 2024 1 commit
- Update gsm8k agent prompt (#788) · 467ad0ac
  Songyang Zhang authored Jan 11, 2024
  
  467ad0ac