Commits · 2a9da9fb3588ca6ed6aeea4aceca6d43623e9f94 · gaoqiong / lm-evaluation-harness

24 Apr, 2023 3 commits
- add metric + agg registries · 2a9da9fb
  haileyschoelkopf authored Apr 24, 2023
  
  2a9da9fb
- add template_vars + make jinja rendering stricter · 460584ca
  haileyschoelkopf authored Apr 23, 2023
  
  460584ca
- make tasks and models registered by decorators · f275301a
  haileyschoelkopf authored Apr 23, 2023
  
  f275301a
23 Apr, 2023 1 commit
- refactor Instance dataclass · e7c18e53
  haileyschoelkopf authored Apr 23, 2023
  
  e7c18e53
19 Apr, 2023 1 commit
- in-place replace main with lm-eval2, keeping old git history · d2a9b759
  haileyschoelkopf authored Apr 19, 2023
  
  d2a9b759
18 Apr, 2023 2 commits
- Merge pull request #416 from trail-of-forks/fix-setup · 814940e8
  Stella Biderman authored Apr 18, 2023
```
Update setup.py dependencies
```
  814940e8
- Update setup.py dependencies · 83d45912
  Riccardo Schirone authored Apr 18, 2023
  
  83d45912
17 Apr, 2023 5 commits
- Update README.md · b315725a
  Stella Biderman authored Apr 17, 2023
  
  b315725a
- Merge pull request #414 from zanussbaum/master · 5c95b907
  Stella Biderman authored Apr 17, 2023
```
feat: evaluation using peft models with CLM
```
  5c95b907
- Merge branch 'master' into master · 83c2ff16
  Stella Biderman authored Apr 17, 2023
  
  83c2ff16
- Cleaned up wording a little · 7548311a
  Stella Biderman authored Apr 17, 2023
  
  7548311a
- docs: update readme · 69826e2c
  Zach Nussbaum authored Apr 17, 2023
  
  69826e2c
13 Apr, 2023 4 commits
- Update README.md · 44275ae9
  Stella Biderman authored Apr 13, 2023
  
  44275ae9
- Update README.md · fcbe193e
  Stella Biderman authored Apr 13, 2023
  
  fcbe193e
- fix: reqs for peft + sentencepiece tokenizers (llama model) · 94ce276c
  Zach Nussbaum authored Apr 13, 2023
  
  94ce276c
- feat: peft model evals · 2b9a6e94
  Zach Nussbaum authored Apr 13, 2023
  
  2b9a6e94
27 Mar, 2023 2 commits
- Merge pull request #407 from EleutherAI/fix-install-instructions · 50eb80ba
  Stella Biderman authored Mar 27, 2023
```
Update README installation instructions
```
  50eb80ba
- update installation instructions · 8b9c0da5
  haileyschoelkopf authored Mar 27, 2023
  
  8b9c0da5
24 Mar, 2023 1 commit
- Merge pull request #403 from Jeffwan/fix_readme · ccc31842
  Stella Biderman authored Mar 24, 2023
```
Change device parameter to cuda:0 to avoid runtime error
```
  ccc31842
23 Mar, 2023 1 commit
- Change device parameter to cuda:0 to avoid runtime error · 92e5632e
  Jiaxin Shan authored Mar 23, 2023
  
  92e5632e
10 Mar, 2023 2 commits
- Hosting wikitext on HuggingFace (#396) · 83dbfbf6
  ben authored Mar 10, 2023
```
* host wikitext on hf

* transfer wikitext to Eleuther HF org
```
  83dbfbf6
- Merge pull request #391 from fattorib/track-upstream-master · 16ac11af
  Hailey Schoelkopf authored Mar 10, 2023
```
Hosting arithmetic dataset on HuggingFace
```
  16ac11af
09 Mar, 2023 1 commit
- transfer data to eleuther HF org · 7502ff3d
  Benjamin Fattori authored Mar 09, 2023
  
  7502ff3d
08 Mar, 2023 7 commits
- Merge branch 'EleutherAI:master' into track-upstream-master · 945a5b01
  ben authored Mar 08, 2023
  
  945a5b01
- inspect import not needed · db501387
  Benjamin Fattori authored Mar 08, 2023
  
  db501387
- Merge pull request #381 from EleutherAI/bigscience_refactor · 2e522e2c
  Stella Biderman authored Mar 08, 2023
```
[WIP] Upstream `hf-causal` and `hf-seq2seq` model implementations
```
  2e522e2c
- Update README.md · d09f32ce
  Stella Biderman authored Mar 08, 2023
  
  d09f32ce
- remove unused import for eval-harness · 1581db0b
  Benjamin Fattori authored Mar 08, 2023
  
  1581db0b
- arithmetic task class now uses HF remote builder · 3c8479ba
  Benjamin Fattori authored Mar 08, 2023
  
  3c8479ba
- remove old dataset builder script under datasets for arithmetic · f022d303
  Benjamin Fattori authored Mar 08, 2023
  
  f022d303
07 Mar, 2023 2 commits
- Merge pull request #390 from sxjscience/master · 9bb0de9c
  Stella Biderman authored Mar 07, 2023
```
Enable "low_cpu_mem_usage" to reduce the memory usage of HF models
```
  9bb0de9c
- enable "low_cpu_mem_usage" · 859f96fd
  Xingjian Shi authored Mar 07, 2023
  
  859f96fd
06 Feb, 2023 1 commit
- Merge pull request #384 from jon-tow/gpt2-vocab-slice · 3d14707a
  Stella Biderman authored Feb 05, 2023
```
hotfix(gpt2): Remove vocab-size logits slice
```
  3d14707a
04 Feb, 2023 1 commit
- hotfix(gpt2): Remove vocab-size logits slice · 1a545447
  jon-tow authored Feb 04, 2023
  
  1a545447
18 Jan, 2023 1 commit
- Add accuracy metric to crows-pairs (#380) · f9eca2c8
  Hailey Schoelkopf authored Jan 18, 2023
```
* add accuracy metric to crows-pairs
```
  f9eca2c8
17 Jan, 2023 2 commits

Aflah authored Jan 18, 2023

* Added CrowSPairs

* Fixed Bug

* Changed to absolute Value

* Docstring Changes

* Update crowspairs.py

Finegrained Tasks

* Update __init__.py

* Finegrained Tasks

* Add `crows_pairs` version metadata

ea3df930

first step: add HF+accelerate implementations · db1f55ff
haileyschoelkopf authored Jan 17, 2023

db1f55ff

15 Jan, 2023 1 commit
- Update CODEOWNERS · 25780307
  Stella Biderman authored Jan 15, 2023
  
  25780307
11 Jan, 2023 1 commit

Added ToxiGen task (#377) · f079e322

Tom Hartvigsen authored Jan 10, 2023

* added toxigen task

* removed previous doc_to_text

* Add train split for few-shot samples

* Add task version metadata

f079e322

29 Dec, 2022 1 commit
- Update Pile Test/Val Download URLs (#373) · 8c048e26
  ben authored Dec 29, 2022
```
* use pile links hosted on the eye

* update links in dataset_infos
```
  8c048e26