Commits · c1e722d9e33903c70b93e3d18e3f2b9bab256aed · gaoqiong / lm-evaluation-harness

"official/legacy/transformer/README.md" did not exist on "b4b8c723d377caae2f4243cf8fc711eab71736a1"

08 Jul, 2023 2 commits
- implement C-Eval · c1e722d9
  hyz's github authored Jul 08, 2023
  
  c1e722d9
- implement C-Eval · 6345d4eb
  hyz's github authored Jul 08, 2023
  
  6345d4eb
29 Jun, 2023 2 commits
- Merge pull request #630 from EleutherAI/fix-stopseq · bc10a390
  Hailey Schoelkopf authored Jun 29, 2023
```
Add error handling for multi-token stopseq and `hf-causal` model type
```
  bc10a390
- add err handling for multi-tok stopseq · c9c141d2
  haileyschoelkopf authored Jun 29, 2023
  
  c9c141d2
27 Jun, 2023 11 commits
- Merge pull request #601 from svenhendrikx/instantiate-model-from-Automodel · 72b7f0c0
  Hailey Schoelkopf authored Jun 27, 2023
```
Instantiate model from automodel
```
  72b7f0c0
- Merge pull request #1 from EleutherAI/pass-automodel · 72ee34dc
  Sven Hendrikx authored Jun 27, 2023
```
Fixes for passing AutoModel
```
  72ee34dc
- update docstring · 1e98c74e
  haileyschoelkopf authored Jun 27, 2023
  
  1e98c74e
- switch to using get_model() · 4221d30f
  haileyschoelkopf authored Jun 27, 2023
  
  4221d30f
- pass batch size · d362dfee
  haileyschoelkopf authored Jun 27, 2023
  
  d362dfee
- remove tokenizer assert · 5f1d18d4
  haileyschoelkopf authored Jun 27, 2023
  
  5f1d18d4
- fix issue with dumping model_name to results · 2a92cde0
  haileyschoelkopf authored Jun 27, 2023
  
  2a92cde0
- change self.gpt2 -> self.model · e0498dd7
  haileyschoelkopf authored Jun 27, 2023
  
  e0498dd7
- Merge pull request #621 from EleutherAI/revert-610-patch-1 · 9f4862f6
  Hailey Schoelkopf authored Jun 27, 2023
```
Revert "[triviaqa] The ground truth must be a *substring* of the generated an…"
```
  9f4862f6
- Revert "[triviaqa] The ground truth must be a *substring* of the generated an…" · ac0cb7e4
  Hailey Schoelkopf authored Jun 27, 2023
  
  ac0cb7e4
- Merge pull request #616 from EleutherAI/patch-sort-descending · f2ae05a5
  Lintang Sutawika authored Jun 27, 2023
```
Sort descending as intended for `greedy_until`
```
  f2ae05a5
26 Jun, 2023 3 commits
- Merge pull request #610 from Vermeille/patch-1 · 0dd45190
  Stella Biderman authored Jun 26, 2023
```
[triviaqa] The ground truth must be a *substring* of the generated an…
```
  0dd45190
- Merge pull request #517 from jquesnelle/scrolls · 5e56fbf6
  Hailey Schoelkopf authored Jun 26, 2023
```
add SCROLLS
```
  5e56fbf6
- Update README.md · f6b81c6b
  Lintang Sutawika authored Jun 26, 2023
  
  f6b81c6b
25 Jun, 2023 1 commit
- Change collation order for greedy_until · a6952a0b
  Hailey Schoelkopf authored Jun 25, 2023
  
  a6952a0b
21 Jun, 2023 1 commit
- [triviaqa] The ground truth must be a *substring* of the generated answer · 0570991a
  Guillaume "Vermeille" Sanchez authored Jun 21, 2023
  
  0570991a
18 Jun, 2023 1 commit
- Merge branch 'master' into instantiate-model-from-Automodel · e6960b9a
  svenhendrikx authored Jun 18, 2023
  
  e6960b9a
16 Jun, 2023 3 commits
- Add logic to simple_evaluate to instantiate HFLM from transformers.PreTrainedModel instance · ddc634f2
  svenhendrikx authored Jun 16, 2023
  
  ddc634f2
- Turn bigbench_resources into module such that it's included in the build · 42caa660
  svenhendrikx authored Jun 16, 2023
  
  42caa660
- Allow HFLM model to be initialized with transformers.PreTrainedModel instance · a28c0193
  svenhendrikx authored Jun 16, 2023
  
  a28c0193
15 Jun, 2023 5 commits
- Update README.md · b281b092
  Stella Biderman authored Jun 15, 2023
  
  b281b092
- Update README.md · ecba73d7
  Stella Biderman authored Jun 15, 2023
  
  ecba73d7
- Merge pull request #497 from ollmer/mmlu_fix · 3e2e6d82
  Stella Biderman authored Jun 15, 2023
```
MMLU task fix
```
  3e2e6d82
- Update README.md · 2b70a7c9
  Hailey Schoelkopf authored Jun 15, 2023
  
  2b70a7c9
- Make Hailey and Lintang the code owners. · babb83b5
  Stella Biderman authored Jun 15, 2023
  
  babb83b5
14 Jun, 2023 5 commits
- Merge pull request #525 from seopbo/fix-triviaqa · b018a7d5
  Hailey Schoelkopf authored Jun 14, 2023
```
Fix triviaqa task
```
  b018a7d5
- Update triviaqa.py · 50cf0984
  Hailey Schoelkopf authored Jun 14, 2023
  
  50cf0984
- Merge pull request #579 from SONG-WONHO/master · 137d0c43
  Hailey Schoelkopf authored Jun 14, 2023
```
Add - 4bit-related args
```
  137d0c43
- rm triviaqa to use HF filtered dev set · 859e5920
  Hailey Schoelkopf authored Jun 14, 2023
  
  859e5920
- Change triviaqa dataset path · a1160392
  Hailey Schoelkopf authored Jun 14, 2023
  
  a1160392
13 Jun, 2023 6 commits
- Merge remote-tracking branch 'upstream/master' into mmlu_fix · 5ba0c5f9
  Oleh Shliazhko authored Jun 14, 2023
  
  5ba0c5f9
- Merge pull request #585 from EleutherAI/fix-quantized-device · 9d06c953
  Hailey Schoelkopf authored Jun 13, 2023
```
Add error handling for calling `.to(device)`
```
  9d06c953
- add error handling for moving model to device · 2d27f9e1
  haileyschoelkopf authored Jun 13, 2023
  
  2d27f9e1
- Merge pull request #584 from gakada/perf · 51d6951c
  Hailey Schoelkopf authored Jun 13, 2023
```
Fix non-callable attributes in CachingLM
```
  51d6951c
- Fix non-callable attributes in CachingLM · d4889e1a
  gk authored Jun 14, 2023
  
  d4889e1a
- Merge pull request #582 from EleutherAI/fix-max-len · 4c08d72a
  Stella Biderman authored Jun 13, 2023
```
Fix seqlen issues for bloom, remove extraneous OPT tokenizer check
```
  4c08d72a