Commits · eb7b9095ef5afa73f4c9a3643e20c4f50c5225b3 · gaoqiong / lm-evaluation-harness

09 Jun, 2023 1 commit
- batch support for loglikelihood tokens · eb7b9095
  Benjamin Fattori authored Jun 09, 2023
  
  eb7b9095
07 Jun, 2023 2 commits
- accelerator multidevice setup · 226063ce
  Benjamin Fattori authored Jun 07, 2023
  
  226063ce
- commit on clean branch · 28500952
  Benjamin Fattori authored Jun 07, 2023
  
  28500952
02 Jun, 2023 1 commit
- Update gpt2.py · e3677848
  Lintang Sutawika authored Jun 02, 2023
  
  e3677848
23 May, 2023 1 commit
- remove device_placement flag and fix rank/world_size assignments · 650d3c76
  Benjamin Fattori authored May 23, 2023
  
  650d3c76
22 May, 2023 1 commit
- pad out all batches loglikelihood_rolling. confirmed correctness · 71388a7e
  Benjamin Fattori authored May 22, 2023
  
  71388a7e
20 May, 2023 3 commits
- unwrap gpt2 model to get n_ctx · c10c08a2
  Benjamin Fattori authored May 20, 2023
  
  c10c08a2
- improved warning message for processes < total gpus · e4f1dfb6
  Benjamin Fattori authored May 20, 2023
  
  e4f1dfb6
- unwrap model to get pos embs · 65127577
  Benjamin Fattori authored May 20, 2023
  
  65127577
19 May, 2023 1 commit
- pre-commit stuff · c4c20ff5
  lintangsutawika authored May 19, 2023
  
  c4c20ff5
16 May, 2023 1 commit
- added logging process · 8d608117
  lintangsutawika authored May 16, 2023
  
  8d608117
10 May, 2023 3 commits
- balance documents among hosts for wikitext · 8f859cd2
  Benjamin Fattori authored May 10, 2023
  
  8f859cd2
- drop comments about pytorch DP · 3cfd23a1
  Benjamin Fattori authored May 10, 2023
  
  3cfd23a1
- fix warning issue if single device · 4d3ea67a
  Benjamin Fattori authored May 10, 2023
  
  4d3ea67a
08 May, 2023 3 commits
- add mutual info, code cleanup · 95642aa6
  haileyschoelkopf authored May 08, 2023
  
  95642aa6
- warning if launched incorrectly on multiGPU machine · 46f37ec0
  Benjamin Fattori authored May 08, 2023
  
  46f37ec0
- clean up cumul batches for rolling + remove unused print · c4447dab
  Benjamin Fattori authored May 08, 2023
  
  c4447dab
06 May, 2023 3 commits
- partial support for document-level padding · 32d7593d
  Benjamin Fattori authored May 06, 2023
  
  32d7593d
- assert + help message if script called incorrectly with multigpu · a8d9a449
  Benjamin Fattori authored May 06, 2023
  
  a8d9a449
- record _rank/_world_size for single GPU · a7d6cb2d
  Benjamin Fattori authored May 06, 2023
  
  a7d6cb2d
05 May, 2023 3 commits
- add utility func for slicing iterators, only tqdm on main process · eddd627a
  Benjamin Fattori authored May 05, 2023
  
  eddd627a
- bugfixes missed from local branch · 629bcfba
  Benjamin Fattori authored May 05, 2023
  
  629bcfba
- sync working changes with upstream · d4c5315a
  Benjamin Fattori authored May 05, 2023
  
  d4c5315a
02 May, 2023 1 commit
- prelim. multiple choice support · 487f7811
  haileyschoelkopf authored May 02, 2023
  
  487f7811
24 Apr, 2023 3 commits
- add metric + agg registries · 2a9da9fb
  haileyschoelkopf authored Apr 24, 2023
  
  2a9da9fb
- add template_vars + make jinja rendering stricter · 460584ca
  haileyschoelkopf authored Apr 23, 2023
  
  460584ca
- make tasks and models registered by decorators · f275301a
  haileyschoelkopf authored Apr 23, 2023
  
  f275301a
19 Apr, 2023 1 commit
- in-place replace main with lm-eval2, keeping old git history · d2a9b759
  haileyschoelkopf authored Apr 19, 2023
  
  d2a9b759
13 Apr, 2023 1 commit
- feat: peft model evals · 2b9a6e94
  Zach Nussbaum authored Apr 13, 2023
  
  2b9a6e94
07 Mar, 2023 1 commit
- enable "low_cpu_mem_usage" · 859f96fd
  Xingjian Shi authored Mar 07, 2023
  
  859f96fd
04 Feb, 2023 1 commit
- hotfix(gpt2): Remove vocab-size logits slice · 1a545447
  jon-tow authored Feb 04, 2023
  
  1a545447
17 Jan, 2023 1 commit
- first step: add HF+accelerate implementations · db1f55ff
  haileyschoelkopf authored Jan 17, 2023
  
  db1f55ff
04 Nov, 2022 1 commit
- Add `TextSynth` API (#299) · 4f0410a4
  Jonathan Tow authored Nov 03, 2022
```
* Add `TextSynth` API
```
  4f0410a4
15 Sep, 2022 3 commits
- Remove stale tokenizer comment · 3dc27bd2
  jon-tow authored Sep 15, 2022
  
  3dc27bd2
- Move `TODO`s into one source location · 5b1beceb
  jon-tow authored Sep 15, 2022
  
  5b1beceb
- Manually concat tokenizer revision with subfolder · ff83e9b2
  jon-tow authored Sep 15, 2022
  
  ff83e9b2
03 May, 2022 2 commits
- fix codespell · d95a4333
  Fabrizio Milo authored May 02, 2022
  
  d95a4333
- add pre-commit · 121b7096
  Fabrizio Milo authored May 02, 2022
  
  121b7096
29 Apr, 2022 2 commits
- Revert "Merge branch 'master' into master" · 7585ec56
  jon-tow authored Apr 29, 2022
```
This reverts commit 5fe7e2c0, reversing
changes made to 14b8ec7e.
```
  7585ec56
- Fix max generation limit · eda365f6
  jon-tow authored Apr 28, 2022
  
  eda365f6