Commits · 6b72d7b76f3d76b5c626c4e26fe65bd2cf902221 · gaoqiong / lm-evaluation-harness

08 Jun, 2023 3 commits

change import origin · 224b0854
lintangsutawika authored Jun 08, 2023

224b0854

[Refactor] Non-greedy generation ; WIP GSM8k yaml (#559) · 232632c6

Hailey Schoelkopf authored Jun 07, 2023

* add wip gsm8k yaml

* cleanup tasks dir

* push gsm8k yaml changes

* rename gpt2.py

* add updated gsm8k , triviaqa baseline

* add new cot yaml

* allow for multiple filter pipelines, new filter types

* updated gsm8k + sampling gen configs

* cleanup self-consistency yaml

232632c6

fixed acc_mutula_info calc position · a2e41158
lintangsutawika authored Jun 08, 2023

a2e41158

07 Jun, 2023 12 commits
- fixed register_model origin and other imports · 424a4280
  lintangsutawika authored Jun 07, 2023
  
  424a4280
- added error message · ddb7c0f3
  lintangsutawika authored Jun 07, 2023
  
  ddb7c0f3
- changed origin of get_model · 06eb4e1b
  lintangsutawika authored Jun 07, 2023
  
  06eb4e1b
- fixed error in aggregation · c75ce009
  lintangsutawika authored Jun 07, 2023
  
  c75ce009
- adjustments on import origin · 829280e8
  lintangsutawika authored Jun 07, 2023
  
  829280e8
- modified import origin · 9c3f7227
  lintangsutawika authored Jun 07, 2023
  
  9c3f7227
- removed · 7b0b42c4
  lintangsutawika authored Jun 07, 2023
  
  7b0b42c4
- modified import orgin · a22d8ffa
  lintangsutawika authored Jun 07, 2023
  
  a22d8ffa
- MODEL_REGISTRY to registry.py · f2166089
  lintangsutawika authored Jun 07, 2023
  
  f2166089
- moved METRIC_REGISTRY to registry.py · 9e75a84d
  lintangsutawika authored Jun 07, 2023
  
  9e75a84d
- removed import metric · 9b39df9b
  lintangsutawika authored Jun 07, 2023
  
  9b39df9b
- renamed register.py to registry.py · 79db4088
  lintangsutawika authored Jun 07, 2023
  
  79db4088
06 Jun, 2023 5 commits
- added more registry · 7d0e3ebf
  lintangsutawika authored Jun 06, 2023
  
  7d0e3ebf
- mejor revision into how metrics are properly loaded for all output_types · ba41ab7c
  lintangsutawika authored Jun 06, 2023
  
  ba41ab7c
- metrics are now in a special folder so that registry can work better · b3591562
  lintangsutawika authored Jun 06, 2023
  
  b3591562
- stderr_for_metric imported from lm_eval.metrics now · 48344fcb
  lintangsutawika authored Jun 06, 2023
  
  48344fcb
- much better way to process all metrics chosen · 0a9ad6ee
  lintangsutawika authored Jun 06, 2023
  
  0a9ad6ee
05 Jun, 2023 7 commits
- added `unwrap_model` for generate · 63eabe79
  lintangsutawika authored Jun 05, 2023
  
  63eabe79
- fixed typo · 6d7a24b2
  lintangsutawika authored Jun 05, 2023
  
  6d7a24b2
- fixed warning and info with proper logging · 82786023
  lintangsutawika authored Jun 05, 2023
  
  82786023
- revert to take_first filter · 47b888bb
  lintangsutawika authored Jun 05, 2023
  
  47b888bb
- fixing metric for each output_type · c746d1fb
  lintangsutawika authored Jun 05, 2023
  
  c746d1fb
- allow to use alternative methods to use hf datasets, allow configuration with dataset_kwargs · a339ffd8
  lintangsutawika authored Jun 05, 2023
  
  a339ffd8
- removed prefix "configurable" from get_task_name_from_config · 36da9c66
  lintangsutawika authored Jun 05, 2023
  
  36da9c66
02 Jun, 2023 6 commits
- push task checklist · 00d33dfe
  haileyschoelkopf authored Jun 02, 2023
  
  00d33dfe
- push many yaml seed tasks · 44eec73b
  haileyschoelkopf authored Jun 02, 2023
  
  44eec73b
- fix default .py task filters · f61974ed
  haileyschoelkopf authored Jun 02, 2023
  
  f61974ed
- Update gpt2.py · e3677848
  Lintang Sutawika authored Jun 02, 2023
  
  e3677848
- adjusted to follow pre-commit conventions · 6f98c780
  lintangsutawika authored Jun 02, 2023
  
  6f98c780
- result_dict process · 6acc5c47
  lintangsutawika authored Jun 02, 2023
  
  6acc5c47
01 Jun, 2023 1 commit
- add promptsource as an extra · 9329fc9b
  haileyschoelkopf authored Jun 01, 2023
  
  9329fc9b
23 May, 2023 4 commits
- better error warning · b698048d
  lintangsutawika authored May 23, 2023
  
  b698048d
- remove device_placement flag and fix rank/world_size assignments · 650d3c76
  Benjamin Fattori authored May 23, 2023
  
  650d3c76
- added more promptsource examples · eb42b01b
  lintangsutawika authored May 23, 2023
  
  eb42b01b
- update · 1cf51552
  lintangsutawika authored May 23, 2023
  
  1cf51552
22 May, 2023 1 commit
- pad out all batches loglikelihood_rolling. confirmed correctness · 71388a7e
  Benjamin Fattori authored May 22, 2023
  
  71388a7e
20 May, 2023 1 commit
- unwrap gpt2 model to get n_ctx · c10c08a2
  Benjamin Fattori authored May 20, 2023
  
  c10c08a2