Commits · b7075cb99f13e613f6d9d3c38f49b509351155dc · gaoqiong / lm-evaluation-harness

23 Jan, 2024 35 commits
- edit test yaml · b7075cb9
  lintangsutawika authored Jan 22, 2024
  
  b7075cb9
- add GroupConfig object · 42619159
  lintangsutawika authored Jan 22, 2024
  
  42619159
- check if config is task update · 318d706b
  lintangsutawika authored Jan 22, 2024
  
  318d706b
- group in a list of group can accept parameter changes like `num_fewshot` · 6584e6d6
  lintangsutawika authored Jan 22, 2024
  
  6584e6d6
- removed comments · db50055e
  lintangsutawika authored Jan 22, 2024
  
  db50055e
- fixed bug · e5e6b286
  lintangsutawika authored Jan 22, 2024
  
  e5e6b286
- remove unused import · 9c8670b7
  lintangsutawika authored Jan 22, 2024
  
  9c8670b7
- removed unused code · b9efb088
  lintangsutawika authored Jan 20, 2024
  
  b9efb088
- removed test lines · 9e98d1a9
  lintangsutawika authored Jan 20, 2024
  
  9e98d1a9
- remove lines for testing · e39ec52d
  lintangsutawika authored Jan 20, 2024
  
  e39ec52d
- further tidy up · 73c8c87c
  lintangsutawika authored Jan 20, 2024
  
  73c8c87c
- simplified code · 52e0148a
  lintangsutawika authored Jan 20, 2024
  
  52e0148a
- fixed typo · 6475a2b5
  lintangsutawika authored Jan 20, 2024
  
  6475a2b5
- fixed typo · 007e5b80
  lintangsutawika authored Jan 20, 2024
  
  007e5b80
- iron out bugs · 8db2ec3e
  lintangsutawika authored Jan 20, 2024
  
  8db2ec3e
- adapted initialize_tasks · 153ea28e
  lintangsutawika authored Jan 20, 2024
  
  153ea28e
- indexing and loading are part of a task_manager object · e487748e
  lintangsutawika authored Jan 20, 2024
  
  e487748e
- temp save · f9c51329
  lintangsutawika authored Jan 20, 2024
  
  f9c51329
- allow groups and tasks to work · 72859bbc
  lintangsutawika authored Jan 19, 2024
  
  72859bbc
- changed variable names · 5ab4142d
  lintangsutawika authored Jan 19, 2024
  
  5ab4142d
- half working for nested groups · fea304e0
  lintangsutawika authored Jan 19, 2024
  
  fea304e0
- pre-commit format · 66926469
  lintangsutawika authored Jan 19, 2024
  
  66926469
- load_task_or_group works but only for tasks · 702593ec
  lintangsutawika authored Jan 19, 2024
  
  702593ec
- no need to use get_task_dict · 0371129a
  lintangsutawika authored Jan 19, 2024
  
  0371129a
- adjust how task objects are called · 93cdc5bf
  lintangsutawika authored Jan 19, 2024
  
  93cdc5bf
- add exit after error · 0d6d1d7c
  lintangsutawika authored Jan 19, 2024
  
  0d6d1d7c
- pre-commit format · 57f158b3
  lintangsutawika authored Jan 19, 2024
  
  57f158b3
- more comprehensive way to index tasks and groups · fe9f0d46
  lintangsutawika authored Jan 19, 2024
  
  fe9f0d46
- moved constructor process to inside load_yaml_config · 711eddcf
  lintangsutawika authored Jan 19, 2024
  
  711eddcf
- remove trust_remote_code for now · 12fc1233
  lintangsutawika authored Jan 19, 2024
  
  12fc1233
- initialize_tasks returns list of tasks and groups · 55ea2888
  lintangsutawika authored Jan 19, 2024
  
  55ea2888
- tasks should only accept TaskObjects · 89c09169
  lintangsutawika authored Jan 19, 2024
  
  89c09169
- changed source of ALL_TASKS · 641ffeba
  lintangsutawika authored Jan 19, 2024
  
  641ffeba
- task for testing recursive · d5071b70
  lintangsutawika authored Jan 19, 2024
  
  d5071b70
- add trust_remote_code as default · 4f69410c
  lintangsutawika authored Jan 19, 2024
  
  4f69410c
19 Jan, 2024 1 commit
- Update polemo2_in.yaml (#1318) · 4477d572
  Lintang Sutawika authored Jan 20, 2024
  
  4477d572
18 Jan, 2024 3 commits
- Fix group register (#1315) · 72ea626e
  Lintang Sutawika authored Jan 19, 2024
```
* tuple should be considered as well

* set option to keep callable as callable
```
  72ea626e
- Fix polemo2_in.yaml config name (#1313) · b8cbc425
  Quentin Lhoest authored Jan 18, 2024
  
  b8cbc425
- Update nq_open.yaml (#1305) · 10488d0d
  Hannibal046 authored Jan 18, 2024
```
* Update nq_open.yaml

change regex

* Bump NQ version

---------
Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>
```
  10488d0d
16 Jan, 2024 1 commit
- Update nq_open.yaml (#1289) · 032e879b
  Hailey Schoelkopf authored Jan 15, 2024
  
  032e879b