Commits · 4bb92ebc5d290cc89c7e952775673e46d7b64cbe · gaoqiong / lm-evaluation-harness

14 May, 2024 8 commits
- add utility · 5ba791e2
  JessicaOjo authored May 14, 2024
  
  5ba791e2
- remove afrimmlu_translate folder · 3290b2b9
  JessicaOjo authored May 14, 2024
  
  3290b2b9
- spacing · 5518b9fa
  JessicaOjo authored May 14, 2024
  
  5518b9fa
- restructure mmlu folder · 47d8de5a
  JessicaOjo authored May 14, 2024
  
  47d8de5a
- fix exact match valueError · 065d91bb
  JessicaOjo authored May 14, 2024
  
  065d91bb
- fix exact match valueError · b91cabae
  JessicaOjo authored May 14, 2024
  
  b91cabae
- added afrixnli_translate · c1ed7068
  Israel Abebe Azime authored May 14, 2024
  
  c1ed7068
- added translate test · 554846a4
  Israel Abebe Azime authored May 14, 2024
  
  554846a4
13 May, 2024 9 commits
- add squad metric to mgsm, remove whitespace filter · 03ebd203
  JessicaOjo authored May 13, 2024
  
  03ebd203
- change eng to ewe in ewe yaml mmlu · e02732be
  JessicaOjo authored May 13, 2024
  
  e02732be
- add swahili to mmlu · 63d328fe
  JessicaOjo authored May 13, 2024
  
  63d328fe
- remove test configs · 3664097f
  JessicaOjo authored May 13, 2024
  
  3664097f
- remove test limit · 6031d7d7
  JessicaOjo authored May 13, 2024
  
  6031d7d7
- add utils · 2a285714
  JessicaOjo authored May 13, 2024
  
  2a285714
- add lin to utils · dade45c6
  JessicaOjo authored May 13, 2024
  
  dade45c6
- change f1 function · 909b7b33
  JessicaOjo authored May 13, 2024
  
  909b7b33
- update native-direct with lin · 9af86259
  JessicaOjo authored May 13, 2024
  
  9af86259
12 May, 2024 10 commits
- add f1 score to mgsm · 5d662de1
  JessicaOjo authored May 12, 2024
  
  5d662de1
- add f1 score to mgsm · d66d6ca1
  JessicaOjo authored May 12, 2024
  
  d66d6ca1
- remove squad metric · 58692fb5
  JessicaOjo authored May 12, 2024
  
  58692fb5
- remove squad metric · 052c7bde
  JessicaOjo authored May 12, 2024
  
  052c7bde
- filter and metric fix -mgsm · fef54568
  JessicaOjo authored May 12, 2024
  
  fef54568
- prompt translations fix · b691c952
  JessicaOjo authored May 12, 2024
  
  b691c952
- prompt translations fix · 6c8d4053
  JessicaOjo authored May 12, 2024
  
  6c8d4053
- update metrics for afrixnli · a3953e0f
  JessicaOjo authored May 12, 2024
  
  a3953e0f
- update afrixnli tasks · b5067932
  JessicaOjo authored May 12, 2024
  
  b5067932
- update afrixnli tasks · b84c8c9e
  JessicaOjo authored May 12, 2024
  
  b84c8c9e
10 May, 2024 8 commits
- added transate test · 452d2024
  Israel Abebe Azime authored May 10, 2024
  
  452d2024
- fix direct path, add bash script for gpt models · bcee8f2e
  JessicaOjo authored May 10, 2024
  
  bcee8f2e
- add few show. metric fixes · c4f634c6
  JessicaOjo authored May 10, 2024
  
  c4f634c6
- remove print · ae6e5cbd
  JessicaOjo authored May 10, 2024
  
  ae6e5cbd
- remove direct util, update common yaml · 3eab4b9a
  JessicaOjo authored May 10, 2024
  
  3eab4b9a
- fix bash script · 21fb0db7
  JessicaOjo authored May 10, 2024
  
  21fb0db7
- add squad metric · 6ca56eac
  JessicaOjo authored May 10, 2024
  
  6ca56eac
- add afrimgsm -direct · 187ab735
  JessicaOjo authored May 10, 2024
  
  187ab735
09 May, 2024 3 commits
- Copal task (#1803) · 1980a13c
  Edd authored May 10, 2024
```
* add copal

* change name to copal id for clarity and the task name

* remove `copal_id...` to yaml to make it work

* checkmark on README

* change group name to `copal_id`
```
  1980a13c
- remove print · 901ad392
  Israel Abebe Azime authored May 09, 2024
  
  901ad392
- updated prompt · 343880ab
  Israel Abebe Azime authored May 09, 2024
  
  343880ab
08 May, 2024 2 commits

add task for mmlu evaluation in arc multiple choice format (#1745) · 9097ad3e

jonabur authored May 08, 2024



* add mmlu arc style evaluation

* rename arc_style to continuation

---------
Co-authored-by: Jonathan Burdge <jburdge@mahti-login11.mahti.csc.fi>
Co-authored-by: Jonathan Burdge <jburdge@mahti-login12.mahti.csc.fi>

9097ad3e

afrimmlu folder update · f64b943d
Israel Abebe Azime authored May 08, 2024

f64b943d