Commits · cea4784859779d3aecd4611ff0f7b5b115e45183 · gaoqiong / lm-evaluation-harness

23 Jan, 2024 39 commits
- merged with main · cea47848
  lintangsutawika authored Jan 23, 2024
  
  cea47848
- remove args · af7351ef
  lintangsutawika authored Jan 23, 2024
  
  af7351ef
- edit test yaml · b7075cb9
  lintangsutawika authored Jan 22, 2024
  
  b7075cb9
- add GroupConfig object · 42619159
  lintangsutawika authored Jan 22, 2024
  
  42619159
- check if config is task update · 318d706b
  lintangsutawika authored Jan 22, 2024
  
  318d706b
- group in a list of group can accept parameter changes like `num_fewshot` · 6584e6d6
  lintangsutawika authored Jan 22, 2024
  
  6584e6d6
- removed comments · db50055e
  lintangsutawika authored Jan 22, 2024
  
  db50055e
- fixed bug · e5e6b286
  lintangsutawika authored Jan 22, 2024
  
  e5e6b286
- remove unused import · 9c8670b7
  lintangsutawika authored Jan 22, 2024
  
  9c8670b7
- removed unused code · b9efb088
  lintangsutawika authored Jan 20, 2024
  
  b9efb088
- removed test lines · 9e98d1a9
  lintangsutawika authored Jan 20, 2024
  
  9e98d1a9
- remove lines for testing · e39ec52d
  lintangsutawika authored Jan 20, 2024
  
  e39ec52d
- further tidy up · 73c8c87c
  lintangsutawika authored Jan 20, 2024
  
  73c8c87c
- simplified code · 52e0148a
  lintangsutawika authored Jan 20, 2024
  
  52e0148a
- fixed typo · 6475a2b5
  lintangsutawika authored Jan 20, 2024
  
  6475a2b5
- fixed typo · 007e5b80
  lintangsutawika authored Jan 20, 2024
  
  007e5b80
- iron out bugs · 8db2ec3e
  lintangsutawika authored Jan 20, 2024
  
  8db2ec3e
- adapted initialize_tasks · 153ea28e
  lintangsutawika authored Jan 20, 2024
  
  153ea28e
- indexing and loading are part of a task_manager object · e487748e
  lintangsutawika authored Jan 20, 2024
  
  e487748e
- temp save · f9c51329
  lintangsutawika authored Jan 20, 2024
  
  f9c51329
- allow groups and tasks to work · 72859bbc
  lintangsutawika authored Jan 19, 2024
  
  72859bbc
- changed variable names · 5ab4142d
  lintangsutawika authored Jan 19, 2024
  
  5ab4142d
- half working for nested groups · fea304e0
  lintangsutawika authored Jan 19, 2024
  
  fea304e0
- pre-commit format · 66926469
  lintangsutawika authored Jan 19, 2024
  
  66926469
- load_task_or_group works but only for tasks · 702593ec
  lintangsutawika authored Jan 19, 2024
  
  702593ec
- no need to use get_task_dict · 0371129a
  lintangsutawika authored Jan 19, 2024
  
  0371129a
- adjust how task objects are called · 93cdc5bf
  lintangsutawika authored Jan 19, 2024
  
  93cdc5bf
- add exit after error · 0d6d1d7c
  lintangsutawika authored Jan 19, 2024
  
  0d6d1d7c
- pre-commit format · 57f158b3
  lintangsutawika authored Jan 19, 2024
  
  57f158b3
- more comprehensive way to index tasks and groups · fe9f0d46
  lintangsutawika authored Jan 19, 2024
  
  fe9f0d46
- moved constructor process to inside load_yaml_config · 711eddcf
  lintangsutawika authored Jan 19, 2024
  
  711eddcf
- remove trust_remote_code for now · 12fc1233
  lintangsutawika authored Jan 19, 2024
  
  12fc1233
- initialize_tasks returns list of tasks and groups · 55ea2888
  lintangsutawika authored Jan 19, 2024
  
  55ea2888
- tasks should only accept TaskObjects · 89c09169
  lintangsutawika authored Jan 19, 2024
  
  89c09169
- changed source of ALL_TASKS · 641ffeba
  lintangsutawika authored Jan 19, 2024
  
  641ffeba
- task for testing recursive · d5071b70
  lintangsutawika authored Jan 19, 2024
  
  d5071b70
- add trust_remote_code as default · 4f69410c
  lintangsutawika authored Jan 19, 2024
  
  4f69410c
- Update migrated HF dataset paths (#1332) · 45a8f709
  Hailey Schoelkopf authored Jan 22, 2024
```
* Update arc_easy.yaml

* Update flan_cot.yaml

* update HF dataset path

* Update freeform.yaml

* Update flan_cot.yaml

---------
Co-authored-by: Lintang Sutawika <lintang@eleuther.ai>
```
  45a8f709
- skip "benchmarks" in changed_tasks (#1336) · 6e65ef38
  Baber Abbasi authored Jan 23, 2024
  
  6e65ef38
22 Jan, 2024 1 commit
- fix a trailing whitespace that breaks a lint job (#1335) · 84357a46
  Brian Vaughan authored Jan 22, 2024
  
  84357a46