Commits · e4f1dfb6aca73d3c8efc16ea826f108ce8882aaf · gaoqiong / lm-evaluation-harness

20 May, 2023 5 commits
- improved warning message for processes < total gpus · e4f1dfb6
  Benjamin Fattori authored May 20, 2023
  
  e4f1dfb6
- keep local changes · 80fe11d8
  Benjamin Fattori authored May 20, 2023
  
  80fe11d8
- Merge remote-tracking branch 'upstream/big-refactor' into multigpu-feature · c23dd0fd
  Benjamin Fattori authored May 20, 2023
  
  c23dd0fd
- unwrap model to get pos embs · 65127577
  Benjamin Fattori authored May 20, 2023
  
  65127577
- update README with instructions for multiGPU evals · f1372139
  Benjamin Fattori authored May 20, 2023
  
  f1372139
10 May, 2023 3 commits
- balance documents among hosts for wikitext · 8f859cd2
  Benjamin Fattori authored May 10, 2023
  
  8f859cd2
- drop comments about pytorch DP · 3cfd23a1
  Benjamin Fattori authored May 10, 2023
  
  3cfd23a1
- fix warning issue if single device · 4d3ea67a
  Benjamin Fattori authored May 10, 2023
  
  4d3ea67a
09 May, 2023 1 commit
- Merge pull request #486 from EleutherAI/yaml-parameterize · f6b76f5d
  Hailey Schoelkopf authored May 09, 2023
```
[Refactor] Add decorator for registering YAMLs as tasks
```
  f6b76f5d
08 May, 2023 7 commits
- save git hash to results dump · f1beac00
  haileyschoelkopf authored May 08, 2023
  
  f1beac00
- add yaml registering decorator · 38244e15
  haileyschoelkopf authored May 08, 2023
  
  38244e15
- add mutual info, code cleanup · 95642aa6
  haileyschoelkopf authored May 08, 2023
  
  95642aa6
- warning if launched incorrectly on multiGPU machine · 46f37ec0
  Benjamin Fattori authored May 08, 2023
  
  46f37ec0
- clean up cumul batches for rolling + remove unused print · c4447dab
  Benjamin Fattori authored May 08, 2023
  
  c4447dab
- world_size condition for padding · e6492441
  Benjamin Fattori authored May 08, 2023
  
  e6492441
- clarify comments around padding · 074099f9
  Benjamin Fattori authored May 08, 2023
  
  074099f9
06 May, 2023 4 commits
- partial support for document-level padding · 32d7593d
  Benjamin Fattori authored May 06, 2023
  
  32d7593d
- assert + help message if script called incorrectly with multigpu · a8d9a449
  Benjamin Fattori authored May 06, 2023
  
  a8d9a449
- limit for doc iterator · 159c1a93
  Benjamin Fattori authored May 06, 2023
  
  159c1a93
- record _rank/_world_size for single GPU · a7d6cb2d
  Benjamin Fattori authored May 06, 2023
  
  a7d6cb2d
05 May, 2023 4 commits
- add utility func for slicing iterators, only tqdm on main process · eddd627a
  Benjamin Fattori authored May 05, 2023
  
  eddd627a
- bugfixes missed from local branch · 629bcfba
  Benjamin Fattori authored May 05, 2023
  
  629bcfba
- accelerate added to requirements in setup.py · 09e91431
  Benjamin Fattori authored May 05, 2023
  
  09e91431
- sync working changes with upstream · d4c5315a
  Benjamin Fattori authored May 05, 2023
  
  d4c5315a
02 May, 2023 6 commits
- modify layout of yaml vs vanilla tasks · 05e97d84
  haileyschoelkopf authored May 02, 2023
  
  05e97d84
- Merge pull request #438 from EleutherAI/configurable-tasks · 2da74953
  Hailey Schoelkopf authored May 02, 2023
```
Configurable-Tasks
```
  2da74953
- Merge branch 'big-refactor' into configurable-tasks · d2b16757
  Hailey Schoelkopf authored May 02, 2023
  
  d2b16757
- Merge pull request #467 from EleutherAI/configurable-mcqa-ppl · 1f23061b
  Hailey Schoelkopf authored May 02, 2023
```
[Refactor, WIP] Multiple Choice + loglikelihood_rolling support for YAML tasks
```
  1f23061b
- add configurable ppl task · 337419b8
  haileyschoelkopf authored May 02, 2023
  
  337419b8
- prelim. multiple choice support · 487f7811
  haileyschoelkopf authored May 02, 2023
  
  487f7811
01 May, 2023 2 commits
- Merge pull request #465 from EleutherAI/wip-hailey-refactor · fa686d04
  Hailey Schoelkopf authored May 01, 2023
```
[WIP, Refactor] Staging more changes
```
  fa686d04
- change id_ to idx in instance · 9d8e0532
  haileyschoelkopf authored May 01, 2023
  
  9d8e0532
28 Apr, 2023 5 commits
- updated yaml · e7f49cca
  lintangsutawika authored Apr 28, 2023
  
  e7f49cca
- added opt8ion to use prompts, removed typos on metric kwargs · 3ec68d38
  lintangsutawika authored Apr 28, 2023
  
  3ec68d38
- can process looglikelihood requests · cea713dc
  lintangsutawika authored Apr 28, 2023
  
  cea713dc
- registries for metrics · 84191b83
  lintangsutawika authored Apr 28, 2023
  
  84191b83
- yaml for lambada and sglue · 7ad96ded
  lintangsutawika authored Apr 28, 2023
  
  7ad96ded
27 Apr, 2023 2 commits
- metric list easier to define args · 33b01a2b
  lintangsutawika authored Apr 27, 2023
  
  33b01a2b
- ported changes here · f107ae29
  lintangsutawika authored Apr 27, 2023
  
  f107ae29
24 Apr, 2023 1 commit
- add metric + agg registries · 2a9da9fb
  haileyschoelkopf authored Apr 24, 2023
  
  2a9da9fb