Commits · 2196490cfe3903b73b1aaa491634d8ff3c028f8f · gaoqiong / lm-evaluation-harness

12 May, 2023 1 commit
- TaskConfig with yaml · 2196490c
  lintangsutawika authored May 12, 2023
  
  2196490c
11 May, 2023 3 commits
- process yaml and registered tasks through args.task · 8299ab3b
  lintangsutawika authored May 11, 2023
  
  8299ab3b
- ignore egg-info · 41ec3bc4
  lintangsutawika authored May 11, 2023
  
  41ec3bc4
- minor fix · fc09c8f7
  lintangsutawika authored May 11, 2023
  
  fc09c8f7
10 May, 2023 9 commits
- update · 7bf74f16
  lintangsutawika authored May 10, 2023
  
  7bf74f16
- adjusted to adapt to modiefied registry process · 52ee80a6
  lintangsutawika authored May 10, 2023
  
  52ee80a6
- modified registry process · 4bc837d4
  lintangsutawika authored May 10, 2023
  
  4bc837d4
- moved METRICS related dicts · 35730ace
  lintangsutawika authored May 10, 2023
  
  35730ace
- moved registry processes here · d9862c58
  lintangsutawika authored May 10, 2023
  
  d9862c58
- simplify · 484fa090
  lintangsutawika authored May 10, 2023
  
  484fa090
- configurable can be overidden with newer configs · cf79b1c6
  lintangsutawika authored May 10, 2023
  
  cf79b1c6
- alternate step so that we can change num_fewshots · cac85281
  lintangsutawika authored May 10, 2023
  
  cac85281
- Merge branch 'big-refactor' of... · 53a4817e
  lintangsutawika authored May 10, 2023
```
Merge branch 'big-refactor' of https://github.com/EleutherAI/lm-evaluation-harness into p3-prompt-task
```
  53a4817e
09 May, 2023 2 commits
- Merge pull request #486 from EleutherAI/yaml-parameterize · f6b76f5d
  Hailey Schoelkopf authored May 09, 2023
```
[Refactor] Add decorator for registering YAMLs as tasks
```
  f6b76f5d
- muliple prompts · 41d16c2b
  lintangsutawika authored May 09, 2023
  
  41d16c2b
08 May, 2023 3 commits
- save git hash to results dump · f1beac00
  haileyschoelkopf authored May 08, 2023
  
  f1beac00
- add yaml registering decorator · 38244e15
  haileyschoelkopf authored May 08, 2023
  
  38244e15
- add mutual info, code cleanup · 95642aa6
  haileyschoelkopf authored May 08, 2023
  
  95642aa6
02 May, 2023 6 commits
- modify layout of yaml vs vanilla tasks · 05e97d84
  haileyschoelkopf authored May 02, 2023
  
  05e97d84
- Merge pull request #438 from EleutherAI/configurable-tasks · 2da74953
  Hailey Schoelkopf authored May 02, 2023
```
Configurable-Tasks
```
  2da74953
- Merge branch 'big-refactor' into configurable-tasks · d2b16757
  Hailey Schoelkopf authored May 02, 2023
  
  d2b16757
- Merge pull request #467 from EleutherAI/configurable-mcqa-ppl · 1f23061b
  Hailey Schoelkopf authored May 02, 2023
```
[Refactor, WIP] Multiple Choice + loglikelihood_rolling support for YAML tasks
```
  1f23061b
- add configurable ppl task · 337419b8
  haileyschoelkopf authored May 02, 2023
  
  337419b8
- prelim. multiple choice support · 487f7811
  haileyschoelkopf authored May 02, 2023
  
  487f7811
01 May, 2023 2 commits
- Merge pull request #465 from EleutherAI/wip-hailey-refactor · fa686d04
  Hailey Schoelkopf authored May 01, 2023
```
[WIP, Refactor] Staging more changes
```
  fa686d04
- change id_ to idx in instance · 9d8e0532
  haileyschoelkopf authored May 01, 2023
  
  9d8e0532
28 Apr, 2023 5 commits
- updated yaml · e7f49cca
  lintangsutawika authored Apr 28, 2023
  
  e7f49cca
- added opt8ion to use prompts, removed typos on metric kwargs · 3ec68d38
  lintangsutawika authored Apr 28, 2023
  
  3ec68d38
- can process looglikelihood requests · cea713dc
  lintangsutawika authored Apr 28, 2023
  
  cea713dc
- registries for metrics · 84191b83
  lintangsutawika authored Apr 28, 2023
  
  84191b83
- yaml for lambada and sglue · 7ad96ded
  lintangsutawika authored Apr 28, 2023
  
  7ad96ded
27 Apr, 2023 2 commits
- metric list easier to define args · 33b01a2b
  lintangsutawika authored Apr 27, 2023
  
  33b01a2b
- ported changes here · f107ae29
  lintangsutawika authored Apr 27, 2023
  
  f107ae29
24 Apr, 2023 3 commits
- add metric + agg registries · 2a9da9fb
  haileyschoelkopf authored Apr 24, 2023
  
  2a9da9fb
- add template_vars + make jinja rendering stricter · 460584ca
  haileyschoelkopf authored Apr 23, 2023
  
  460584ca
- make tasks and models registered by decorators · f275301a
  haileyschoelkopf authored Apr 23, 2023
  
  f275301a
23 Apr, 2023 1 commit
- refactor Instance dataclass · e7c18e53
  haileyschoelkopf authored Apr 23, 2023
  
  e7c18e53
19 Apr, 2023 1 commit
- in-place replace main with lm-eval2, keeping old git history · d2a9b759
  haileyschoelkopf authored Apr 19, 2023
  
  d2a9b759
18 Apr, 2023 2 commits
- Merge pull request #416 from trail-of-forks/fix-setup · 814940e8
  Stella Biderman authored Apr 18, 2023
```
Update setup.py dependencies
```
  814940e8
- Update setup.py dependencies · 83d45912
  Riccardo Schirone authored Apr 18, 2023
  
  83d45912