Commits · 026d2c212cbc26d2c5f744d3c70145e11f2cdad8 · gaoqiong / lm-evaluation-harness

03 Jul, 2023 4 commits
- update task readme with completed race · 026d2c21
  Benjamin Fattori authored Jul 04, 2023
  
  026d2c21
- acc_norm not needed for pubmed_qa · 81bdd5bd
  Benjamin Fattori authored Jul 04, 2023
  
  81bdd5bd
- update race preprocess + rename dataset · 2411c9d3
  Benjamin Fattori authored Jul 03, 2023
  
  2411c9d3
- add race task, with custom HF dataset · 7feab5e3
  Benjamin Fattori authored Jul 03, 2023
  
  7feab5e3
02 Jul, 2023 3 commits
- update task readme for winogrande · 4b8e7ca7
  Benjamin Fattori authored Jul 02, 2023
  
  4b8e7ca7
- add new task type: winograd_schema · a8396b2c
  Benjamin Fattori authored Jul 02, 2023
  
  a8396b2c
- winogrande task implemented · d674c7bd
  Benjamin Fattori authored Jul 02, 2023
  
  d674c7bd
30 Jun, 2023 2 commits
- task readme updated for pubmed_qa · d9802303
  Benjamin Fattori authored Jun 30, 2023
  
  d9802303
- pubmed_qa implemented and tested · eb29b97b
  Benjamin Fattori authored Jun 30, 2023
  
  eb29b97b
29 Jun, 2023 5 commits
- update task readme for arithmetic · e50c8c70
  Benjamin Fattori authored Jun 29, 2023
  
  e50c8c70
- add arithmetic tasks · 470e33b1
  Benjamin Fattori authored Jun 29, 2023
  
  470e33b1
- update task readme for swag · 72c61866
  Benjamin Fattori authored Jun 29, 2023
  
  72c61866
- SWAG completed · a4b6f914
  Benjamin Fattori authored Jun 29, 2023
  
  a4b6f914
- commit blank task yamls · 886190e9
  Benjamin Fattori authored Jun 29, 2023
  
  886190e9
28 Jun, 2023 13 commits
- Merge pull request #613 from EleutherAI/add-back-cache · ef4a26f6
  Lintang Sutawika authored Jun 28, 2023
```
[Refactor] batch generation better for `hf` model ; deprecate `hf-causal` in new release
```
  ef4a26f6
- Merge branch 'big-refactor' into add-back-cache · c5dbf289
  Hailey Schoelkopf authored Jun 28, 2023
  
  c5dbf289
- Merge pull request #619 from EleutherAI/cachinglm-only · 4e0d0e3a
  Hailey Schoelkopf authored Jun 28, 2023
```
[Refactor] CachingLM support via `--use_cache`
```
  4e0d0e3a
- revert mutual info whitespace change · 03b9db6b
  haileyschoelkopf authored Jun 28, 2023
  
  03b9db6b
- clean up batched code and add comments · b250b001
  haileyschoelkopf authored Jun 28, 2023
  
  b250b001
- make OAI completions work with new generation kwargs format · 5a5442ff
  haileyschoelkopf authored Jun 28, 2023
  
  5a5442ff
- add cache hooks to API LMs · d1c5abef
  haileyschoelkopf authored Jun 28, 2023
  
  d1c5abef
- change generation_kwargs -> kwargs · 732f7ed2
  haileyschoelkopf authored Jun 28, 2023
  
  732f7ed2
- make cache-loading msg part of logger · 7ffb74ce
  haileyschoelkopf authored Jun 28, 2023
  
  7ffb74ce
- Update model.py · b2fb53d4
  Hailey Schoelkopf authored Jun 28, 2023
  
  b2fb53d4
- Add tqdm progress bar for cachingLM · f95b160b
  Hailey Schoelkopf authored Jun 28, 2023
  
  f95b160b
- push WIP batched multi-kwarg code · ffc3a456
  haileyschoelkopf authored Jun 28, 2023
  
  ffc3a456
- add Grouper · 83f957bc
  haileyschoelkopf authored Jun 28, 2023
  
  83f957bc
27 Jun, 2023 1 commit
- update lm_eval.models.__init__ · 2fef6bc5
  haileyschoelkopf authored Jun 27, 2023
  
  2fef6bc5
26 Jun, 2023 2 commits
- remove mention of hf-causal from readme · abbbc96f
  haileyschoelkopf authored Jun 26, 2023
  
  abbbc96f
- remove hf-causal model type · 4f002df5
  haileyschoelkopf authored Jun 26, 2023
  
  4f002df5
24 Jun, 2023 3 commits
- squeeze tensors before pad_and_concat · b3aab393
  haileyschoelkopf authored Jun 24, 2023
  
  b3aab393
- remove stale commented code · 513352ae
  haileyschoelkopf authored Jun 24, 2023
  
  513352ae
- add batched generation · dced6e96
  haileyschoelkopf authored Jun 24, 2023
  
  dced6e96
23 Jun, 2023 4 commits
- add cache hooks to HF · e3960fa0
  haileyschoelkopf authored Jun 23, 2023
  
  e3960fa0
- fix mutual info normalization · a68c3fa4
  haileyschoelkopf authored Jun 23, 2023
  
  a68c3fa4
- add use_cache arg · 6449ab1a
  haileyschoelkopf authored Jun 23, 2023
  
  6449ab1a
- add CachingLM back · 1c0ff968
  haileyschoelkopf authored Jun 23, 2023
  
  1c0ff968
22 Jun, 2023 3 commits
- Hotfix samplers.py · 9dea125b
  Hailey Schoelkopf authored Jun 22, 2023
  
  9dea125b
- Merge pull request #565 from fattorib/seq2seq-refactor · 1fa02395
  Lintang Sutawika authored Jun 22, 2023
```
[Refactor] Seq2Seq Models with Multi-Device Support
```
  1fa02395
- Merge pull request #5 from EleutherAI/seq2seq-support · d3cfdcf6
  Hailey Schoelkopf authored Jun 22, 2023
```
fix edge cases for seq2seq
```
  d3cfdcf6