Commits · 45737a385c5945aa7182266e1ccf9877d39b6db3 · gaoqiong / lm-evaluation-harness

"llm/ext_server/CMakeLists.txt" did not exist on "22cd5eaab6265dfe77005af1d60f9c71b71d3d17"

06 Jul, 2023 3 commits
- Merge pull request #627 from fattorib/refactor-more-tasks · 45737a38
  Lintang Sutawika authored Jul 06, 2023
```
[Refactor] Add: SWAG,RACE,Arithmetic,Winogrande,PubmedQA
```
  45737a38
- Merge branch 'big-refactor' into refactor-more-tasks · 02362e6a
  Lintang Sutawika authored Jul 06, 2023
  
  02362e6a
- Merge pull request #658 from EleutherAI/edge-case-lowbits · 0ba4ae15
  Lintang Sutawika authored Jul 06, 2023
```
[Refactor] Add Lambada Multilingual
```
  0ba4ae15
05 Jul, 2023 12 commits
- Update README.md · 29940d16
  Hailey Schoelkopf authored Jul 05, 2023
  
  29940d16
- Update README.md · 33add999
  Hailey Schoelkopf authored Jul 05, 2023
  
  33add999
- Merge pull request #650 from EleutherAI/lintangsutawika-patch-1 · b296ff8f
  Hailey Schoelkopf authored Jul 05, 2023
```
Update __init__.py
```
  b296ff8f
- check off Lambada Multilingual from task list · b5697205
  haileyschoelkopf authored Jul 05, 2023
  
  b5697205
- remove batch_size param from config · 38bd2cc8
  haileyschoelkopf authored Jul 05, 2023
  
  38bd2cc8
- Update __init__.py · 797f47c7
  Lintang Sutawika authored Jul 05, 2023
  
  797f47c7
- remove unused line in evaluator.py · 23318d0e
  haileyschoelkopf authored Jul 05, 2023
  
  23318d0e
- fix typo in copa group name · 0e5da3b0
  haileyschoelkopf authored Jul 05, 2023
  
  0e5da3b0
- add lambada_mt tasks · 9735ac34
  haileyschoelkopf authored Jul 05, 2023
  
  9735ac34
- Merge pull request #648 from EleutherAI/edge-case-lowbits · 4f0a7e57
  Lintang Sutawika authored Jul 05, 2023
```
[Refactor] Misc. bugfixes ; edgecase quantized models
```
  4f0a7e57
- Update qa4mre status · 42c10bd6
  Hailey Schoelkopf authored Jul 05, 2023
  
  42c10bd6
- raise ValueError when invalid task passed · 7f557daa
  haileyschoelkopf authored Jul 05, 2023
  
  7f557daa
04 Jul, 2023 7 commits
- address PR comments · b1b5239d
  Benjamin Fattori authored Jul 04, 2023
  
  b1b5239d
- Merge pull request #640 from farzanehnakhaee70/add-prost-config · 66b91e02
  Lintang Sutawika authored Jul 04, 2023
```
[refactor] Add prost config
```
  66b91e02
- Update __init__.py · d289277d
  Lintang Sutawika authored Jul 04, 2023
```
Should fix a potential error if a dataset has no `dataset_name` argument.

cc: @haileyschoelkopf
```
  d289277d
- Merge branch 'big-refactor' into add-prost-config · 66bb89e5
  FarzanehNakhaee authored Jul 04, 2023
  
  66bb89e5
- Update doc to target and gold_alias in the config · e8bb77db
  FarzanehNakhaee authored Jul 04, 2023
  
  e8bb77db
- Merge remote-tracking branch 'upstream/big-refactor' into refactor-more-tasks · 6e3ef5ff
  Benjamin Fattori authored Jul 04, 2023
  
  6e3ef5ff
- edit doc to text in yaml file · ccae768a
  FarzanehNakhaee authored Jul 04, 2023
  
  ccae768a
03 Jul, 2023 18 commits
- update task readme with completed race · 026d2c21
  Benjamin Fattori authored Jul 04, 2023
  
  026d2c21
- acc_norm not needed for pubmed_qa · 81bdd5bd
  Benjamin Fattori authored Jul 04, 2023
  
  81bdd5bd
- cleanup metric loading code · 328f0e85
  haileyschoelkopf authored Jul 03, 2023
  
  328f0e85
- update race preprocess + rename dataset · 2411c9d3
  Benjamin Fattori authored Jul 03, 2023
  
  2411c9d3
- add race task, with custom HF dataset · 7feab5e3
  Benjamin Fattori authored Jul 03, 2023
  
  7feab5e3
- remove output_base_path · 6a2620ad
  haileyschoelkopf authored Jul 03, 2023
  
  6a2620ad
- Update README.md · 070b6b9c
  Hailey Schoelkopf authored Jul 03, 2023
  
  070b6b9c
- Merge branch 'edge-case-lowbits' of... · 93b7a418
  haileyschoelkopf authored Jul 03, 2023
```
Merge branch 'edge-case-lowbits' of https://github.com/EleutherAI/lm-evaluation-harness into edge-case-lowbits
```
  93b7a418
- remove out-of-date description guide · 899a24c7
  haileyschoelkopf authored Jul 03, 2023
  
  899a24c7
- add up-to-date config fields to docs · 97220c9b
  haileyschoelkopf authored Jul 03, 2023
  
  97220c9b
- Update task.py · 0b62e042
  Hailey Schoelkopf authored Jul 03, 2023
  
  0b62e042
- bump torch ver. 1.7->1.8 · 445beade
  haileyschoelkopf authored Jul 03, 2023
  
  445beade
- fix squeeze() in pad_and_concat · 46ca66de
  haileyschoelkopf authored Jul 03, 2023
  
  46ca66de
- don't place quantized models .to(device) · 0b896bd2
  haileyschoelkopf authored Jul 03, 2023
  
  0b896bd2
- Merge pull request #647 from EleutherAI/handle-multigpu-errors · 0d8e206c
  Lintang Sutawika authored Jul 03, 2023
```
[Refactor] Handle `cuda:0` device assignment
```
  0d8e206c
- revert typehint · b3598058
  haileyschoelkopf authored Jul 03, 2023
  
  b3598058
- revert gpus > num processes case · 5cb7af7e
  haileyschoelkopf authored Jul 03, 2023
  
  5cb7af7e
- add typehint · 62f4d663
  haileyschoelkopf authored Jul 03, 2023
  
  62f4d663