Commits · 4d21ab6b2713cc3a8b4fa7574e89c62ef504e75f · gaoqiong / lm-evaluation-harness

02 May, 2023 1 commit
- address PR comments · 4d21ab6b
  Benjamin Fattori authored May 02, 2023
  
  4d21ab6b
15 Apr, 2023 2 commits
- update type arguments for hf-causal to accept str for batch_size · 9142999e
  Benjamin Fattori authored Apr 15, 2023
  
  9142999e
- compute multiple forward passes in autobatcher to improve robustness to OOMs · e87c0838
  Benjamin Fattori authored Apr 15, 2023
  
  e87c0838
12 Apr, 2023 1 commit
- remove double assign typo of test_batch · d86de51d
  Benjamin Fattori authored Apr 12, 2023
  
  d86de51d
11 Apr, 2023 1 commit
- additional external call to empty_cache + gc collect · 42c6b7df
  Benjamin Fattori authored Apr 11, 2023
  
  42c6b7df
16 Mar, 2023 2 commits
- wrap self._model_call with F.log_softmax + remove empty_cache, not needed · 99304fe5
  Benjamin Fattori authored Mar 16, 2023
  
  99304fe5
- empty cuda cache after determing largest possible batch size · 8a89b30c
  Benjamin Fattori authored Mar 16, 2023
  
  8a89b30c
14 Mar, 2023 1 commit
- add accelerate to deps · 97f936be
  Benjamin Fattori authored Mar 14, 2023
  
  97f936be
10 Mar, 2023 1 commit
- ll_rolling computes adaptive bs separately and passes computed bs to... · f1dedbf0
  Benjamin Fattori authored Mar 10, 2023
```
ll_rolling computes adaptive bs separately and passes computed bs to _ll_tokens to avoid recomputation
```
  f1dedbf0
09 Mar, 2023 2 commits
- accelerate import not needed · b824fc91
  Benjamin Fattori authored Mar 09, 2023
  
  b824fc91
- single GPU automatic batching logic · d5720d5f
  Benjamin Fattori authored Mar 09, 2023
  
  d5720d5f
08 Mar, 2023 2 commits
- Merge pull request #381 from EleutherAI/bigscience_refactor · 2e522e2c
  Stella Biderman authored Mar 08, 2023
```
[WIP] Upstream `hf-causal` and `hf-seq2seq` model implementations
```
  2e522e2c
- Update README.md · d09f32ce
  Stella Biderman authored Mar 08, 2023
  
  d09f32ce
07 Mar, 2023 2 commits
- Merge pull request #390 from sxjscience/master · 9bb0de9c
  Stella Biderman authored Mar 07, 2023
```
Enable "low_cpu_mem_usage" to reduce the memory usage of HF models
```
  9bb0de9c
- enable "low_cpu_mem_usage" · 859f96fd
  Xingjian Shi authored Mar 07, 2023
  
  859f96fd
06 Feb, 2023 1 commit
- Merge pull request #384 from jon-tow/gpt2-vocab-slice · 3d14707a
  Stella Biderman authored Feb 05, 2023
```
hotfix(gpt2): Remove vocab-size logits slice
```
  3d14707a
04 Feb, 2023 1 commit
- hotfix(gpt2): Remove vocab-size logits slice · 1a545447
  jon-tow authored Feb 04, 2023
  
  1a545447
18 Jan, 2023 1 commit
- Add accuracy metric to crows-pairs (#380) · f9eca2c8
  Hailey Schoelkopf authored Jan 18, 2023
```
* add accuracy metric to crows-pairs
```
  f9eca2c8
17 Jan, 2023 2 commits

Aflah authored Jan 18, 2023

* Added CrowSPairs

* Fixed Bug

* Changed to absolute Value

* Docstring Changes

* Update crowspairs.py

Finegrained Tasks

* Update __init__.py

* Finegrained Tasks

* Add `crows_pairs` version metadata

ea3df930

first step: add HF+accelerate implementations · db1f55ff
haileyschoelkopf authored Jan 17, 2023

db1f55ff

15 Jan, 2023 1 commit
- Update CODEOWNERS · 25780307
  Stella Biderman authored Jan 15, 2023
  
  25780307
11 Jan, 2023 1 commit

Added ToxiGen task (#377) · f079e322

Tom Hartvigsen authored Jan 10, 2023

* added toxigen task

* removed previous doc_to_text

* Add train split for few-shot samples

* Add task version metadata

f079e322

29 Dec, 2022 1 commit
- Update Pile Test/Val Download URLs (#373) · 8c048e26
  ben authored Dec 29, 2022
```
* use pile links hosted on the eye

* update links in dataset_infos
```
  8c048e26
17 Dec, 2022 1 commit
- Merge pull request #370 from jon-tow/update-lambada-openai · f29597be
  Stella Biderman authored Dec 16, 2022
```
Update `lambada_openai` multilingual data source
```
  f29597be
16 Dec, 2022 1 commit
- Update `lambada_openai` multilingual data source · 2bae14d2
  jon-tow authored Dec 16, 2022
  
  2bae14d2
13 Dec, 2022 2 commits
- Merge pull request #366 from jon-tow/bump-triviaqa-version · 57f8ff24
  Stella Biderman authored Dec 12, 2022
```
Bump `triviaqa` version
```
  57f8ff24
- Bump `triviaqa` version · f7aeeff2
  jon-tow authored Dec 12, 2022
  
  f7aeeff2
12 Dec, 2022 4 commits
- Update `actions/setup-python`in CI workflows (#365) · 3035550d
  Jonathan Tow authored Dec 12, 2022
```
* Update `actions/setup-python`in  CI workflows
```
  3035550d
- Merge pull request #364 from jon-tow/triviaqa-dataset · 0742f4d7
  Stella Biderman authored Dec 12, 2022
```
Replace stale `triviaqa` dataset link
```
  0742f4d7
- Run pre-commit · 3aff8606
  jon-tow authored Dec 12, 2022
  
  3aff8606
- Replace stale `triviaqa` dataset link · 299e9505
  jon-tow authored Dec 12, 2022
  
  299e9505
08 Dec, 2022 1 commit
- Bump version to 0.3.0 · 62ca1840
  Leo Gao authored Dec 08, 2022
  
  62ca1840
07 Dec, 2022 1 commit
- Merge pull request #362 from EleutherAI/cleanup-for-release · 1d8107bf
  Stella Biderman authored Dec 07, 2022
```
Cleanup `README.md` and package deps
```
  1d8107bf
02 Dec, 2022 2 commits
- Remove redefinition of `jieba` · 1e5d55d9
  jon-tow authored Dec 02, 2022
  
  1e5d55d9
- Cleanup `README.md` and package deps · 0699d774
  jon-tow authored Dec 02, 2022
  
  0699d774
25 Nov, 2022 5 commits
- Merge pull request #357 from jon-tow/add-lambada-orig · fdd3dbc3
  Stella Biderman authored Nov 25, 2022
```
Add the original `LAMBADA` dataset
```
  fdd3dbc3
- Run pre-commit · 1386065d
  jon-tow authored Nov 25, 2022
  
  1386065d
- Merge branch 'master' of https://github.com/EleutherAI/lm-evaluation-harness into add-lambada-orig · 8f442ec1
  jon-tow authored Nov 25, 2022
  
  8f442ec1
- Merge pull request #358 from jon-tow/update-pre-commit · 9ec03596
  Stella Biderman authored Nov 25, 2022
```
Remove unsupported `gitlab` link in pre-commit config
```
  9ec03596
- Revert "Remove old `LAMBADA` dataset" · 0bca4303
  jon-tow authored Nov 25, 2022
```
This reverts commit 63bdb4b0.
```
  0bca4303