Commits · 25dfd3f6697edd7fbefc10dcb858337ed742910d · gaoqiong / lm-evaluation-harness

"examples/community/pipeline_stable_diffusion_xl_ipex.py" did not exist on "67d070749ae393a234470b6ef653821bb4f02cc6"

03 Jul, 2023 5 commits
- Merge pull request #389 from Muennighoff/babi · 25dfd3f6
  Hailey Schoelkopf authored Jul 03, 2023
```
Add bAbI
```
  25dfd3f6
- Update babi.py · caadcbc9
  Hailey Schoelkopf authored Jul 03, 2023
  
  caadcbc9
- Merge pull request #623 from Shas3011/master · a8996a2a
  Hailey Schoelkopf authored Jul 03, 2023
```
Dockerfile added
```
  a8996a2a
- fix typo · 44363c33
  Hailey Schoelkopf authored Jul 03, 2023
  
  44363c33
- Merge pull request #643 from gakada/master · d1537059
  Hailey Schoelkopf authored Jul 03, 2023
```
Fix trust_remote_code, bnb_4bit_*, max_batch_size
```
  d1537059
02 Jul, 2023 1 commit
- Fix trust_remote_code, bnb_4bit_*, max_batch_size for gpt2 · a946c6cd
  gk authored Jul 02, 2023
  
  a946c6cd
30 Jun, 2023 3 commits
- Merge branch 'EleutherAI:master' into master · e26cfb0e
  Shas3011 authored Jun 30, 2023
  
  e26cfb0e
- Merge pull request #637 from if001/trust_remote_code · 13014b2c
  Hailey Schoelkopf authored Jun 30, 2023
```
add trust_remote_code to tokenizer.from_pretrained
```
  13014b2c
- add trust_remote_code to tokenizer.from_pretrained · e9f1af36
  windows_on_wsl authored Jun 30, 2023
  
  e9f1af36
29 Jun, 2023 2 commits
- Merge pull request #630 from EleutherAI/fix-stopseq · bc10a390
  Hailey Schoelkopf authored Jun 29, 2023
```
Add error handling for multi-token stopseq and `hf-causal` model type
```
  bc10a390
- add err handling for multi-tok stopseq · c9c141d2
  haileyschoelkopf authored Jun 29, 2023
  
  c9c141d2
28 Jun, 2023 1 commit
- Dockerfile added · 60281778
  Shas S authored Jun 28, 2023
  
  60281778
27 Jun, 2023 11 commits
- Merge pull request #601 from svenhendrikx/instantiate-model-from-Automodel · 72b7f0c0
  Hailey Schoelkopf authored Jun 27, 2023
```
Instantiate model from automodel
```
  72b7f0c0
- Merge pull request #1 from EleutherAI/pass-automodel · 72ee34dc
  Sven Hendrikx authored Jun 27, 2023
```
Fixes for passing AutoModel
```
  72ee34dc
- update docstring · 1e98c74e
  haileyschoelkopf authored Jun 27, 2023
  
  1e98c74e
- switch to using get_model() · 4221d30f
  haileyschoelkopf authored Jun 27, 2023
  
  4221d30f
- pass batch size · d362dfee
  haileyschoelkopf authored Jun 27, 2023
  
  d362dfee
- remove tokenizer assert · 5f1d18d4
  haileyschoelkopf authored Jun 27, 2023
  
  5f1d18d4
- fix issue with dumping model_name to results · 2a92cde0
  haileyschoelkopf authored Jun 27, 2023
  
  2a92cde0
- change self.gpt2 -> self.model · e0498dd7
  haileyschoelkopf authored Jun 27, 2023
  
  e0498dd7
- Merge pull request #621 from EleutherAI/revert-610-patch-1 · 9f4862f6
  Hailey Schoelkopf authored Jun 27, 2023
```
Revert "[triviaqa] The ground truth must be a *substring* of the generated an…"
```
  9f4862f6
- Revert "[triviaqa] The ground truth must be a *substring* of the generated an…" · ac0cb7e4
  Hailey Schoelkopf authored Jun 27, 2023
  
  ac0cb7e4
- Merge pull request #616 from EleutherAI/patch-sort-descending · f2ae05a5
  Lintang Sutawika authored Jun 27, 2023
```
Sort descending as intended for `greedy_until`
```
  f2ae05a5
26 Jun, 2023 3 commits
- Merge pull request #610 from Vermeille/patch-1 · 0dd45190
  Stella Biderman authored Jun 26, 2023
```
[triviaqa] The ground truth must be a *substring* of the generated an…
```
  0dd45190
- Merge pull request #517 from jquesnelle/scrolls · 5e56fbf6
  Hailey Schoelkopf authored Jun 26, 2023
```
add SCROLLS
```
  5e56fbf6
- Update README.md · f6b81c6b
  Lintang Sutawika authored Jun 26, 2023
  
  f6b81c6b
25 Jun, 2023 1 commit
- Change collation order for greedy_until · a6952a0b
  Hailey Schoelkopf authored Jun 25, 2023
  
  a6952a0b
21 Jun, 2023 1 commit
- [triviaqa] The ground truth must be a *substring* of the generated answer · 0570991a
  Guillaume "Vermeille" Sanchez authored Jun 21, 2023
  
  0570991a
18 Jun, 2023 1 commit
- Merge branch 'master' into instantiate-model-from-Automodel · e6960b9a
  svenhendrikx authored Jun 18, 2023
  
  e6960b9a
16 Jun, 2023 3 commits
- Add logic to simple_evaluate to instantiate HFLM from transformers.PreTrainedModel instance · ddc634f2
  svenhendrikx authored Jun 16, 2023
  
  ddc634f2
- Turn bigbench_resources into module such that it's included in the build · 42caa660
  svenhendrikx authored Jun 16, 2023
  
  42caa660
- Allow HFLM model to be initialized with transformers.PreTrainedModel instance · a28c0193
  svenhendrikx authored Jun 16, 2023
  
  a28c0193
15 Jun, 2023 5 commits
- Update README.md · b281b092
  Stella Biderman authored Jun 15, 2023
  
  b281b092
- Update README.md · ecba73d7
  Stella Biderman authored Jun 15, 2023
  
  ecba73d7
- Merge pull request #497 from ollmer/mmlu_fix · 3e2e6d82
  Stella Biderman authored Jun 15, 2023
```
MMLU task fix
```
  3e2e6d82
- Update README.md · 2b70a7c9
  Hailey Schoelkopf authored Jun 15, 2023
  
  2b70a7c9
- Make Hailey and Lintang the code owners. · babb83b5
  Stella Biderman authored Jun 15, 2023
  
  babb83b5
14 Jun, 2023 3 commits
- Merge pull request #525 from seopbo/fix-triviaqa · b018a7d5
  Hailey Schoelkopf authored Jun 14, 2023
```
Fix triviaqa task
```
  b018a7d5
- Update triviaqa.py · 50cf0984
  Hailey Schoelkopf authored Jun 14, 2023
  
  50cf0984
- Merge pull request #579 from SONG-WONHO/master · 137d0c43
  Hailey Schoelkopf authored Jun 14, 2023
```
Add - 4bit-related args
```
  137d0c43