Commits · 5da401b070f3cb92e37c96e720319f87aa3eb83d · gaoqiong / lm-evaluation-harness

13 Dec, 2023 14 commits
- add styles 5 -8 · 5da401b0
  lintangsutawika authored Dec 13, 2023
  
  5da401b0
- fixed list · 3c5f347f
  lintangsutawika authored Dec 13, 2023
  
  3c5f347f
- allow doc_to_choice to be a single feature from a dataset · e96c6484
  lintangsutawika authored Dec 13, 2023
  
  e96c6484
- removed comments · cd4ef5e8
  lintangsutawika authored Dec 13, 2023
  
  cd4ef5e8
- moved files · d5c6f0e5
  lintangsutawika authored Dec 13, 2023
  
  d5c6f0e5
- reveresed os.walk · 6fdecc6f
  lintangsutawika authored Dec 13, 2023
  
  6fdecc6f
- format · ec656bb1
  lintangsutawika authored Dec 13, 2023
  
  ec656bb1
- resolved merge conflict · 6f4f9e1c
  lintangsutawika authored Dec 13, 2023
  
  6f4f9e1c
- added mmlu alt world prompts · 0d5748b7
  lintangsutawika authored Dec 13, 2023
  
  0d5748b7
- add alt prompts for truthfulqa · b9e55312
  lintangsutawika authored Dec 13, 2023
  
  b9e55312
- removed · edb21d21
  lintangsutawika authored Dec 13, 2023
  
  edb21d21
- removed · 6e220b17
  lintangsutawika authored Dec 13, 2023
  
  6e220b17
- remove style · 09cac5dc
  lintangsutawika authored Dec 13, 2023
  
  09cac5dc
- fixed file path · ec4cdfed
  lintangsutawika authored Dec 13, 2023
  
  ec4cdfed
12 Dec, 2023 12 commits
- Merge pull request #1035 from baberabb/big-refactor_dp · aed90773
  Hailey Schoelkopf authored Dec 12, 2023
```
[Refactor] vllm data parallel
```
  aed90773
- Update vllm_causallms.py · d588a466
  Hailey Schoelkopf authored Dec 12, 2023
  
  d588a466
- Update README.md · 86311c23
  Hailey Schoelkopf authored Dec 12, 2023
  
  86311c23
- data_parallel -> data_parallel_size · 6409fe6c
  Hailey Schoelkopf authored Dec 12, 2023
  
  6409fe6c
- Describe model_comparator.py in readme · de2a60e3
  Hailey Schoelkopf authored Dec 12, 2023
  
  de2a60e3
- Merge pull request #1104 from EleutherAI/simplify_evaluator · 61b0cd29
  Lintang Sutawika authored Dec 12, 2023
```
Simplified `evaluator.py`
```
  61b0cd29
- Merge branch 'main' into simplify_evaluator · da320e91
  Lintang Sutawika authored Dec 12, 2023
  
  da320e91
- Merge pull request #1106 from EleutherAI/fix_stderr · 6ec258cc
  Stella Biderman authored Dec 12, 2023
```
Fix stderr
```
  6ec258cc
- allow N/A as a possible value for Stderr · d6f8faad
  lintangsutawika authored Dec 12, 2023
  
  d6f8faad
- calculate std if items > 1 · 91279a6e
  lintangsutawika authored Dec 12, 2023
  
  91279a6e
- simplified evaluator by removing unused variables · 62e4ea10
  lintangsutawika authored Dec 12, 2023
  
  62e4ea10
- Merge pull request #1089 from h-albert-lee/K-MMLU · f8c2cfcb
  Hanwool Albert Lee authored Dec 12, 2023
```
Add kmmlu evaluation to tasks
```
  f8c2cfcb
11 Dec, 2023 7 commits
- Merge pull request #1099 from shiweijiezero/main · c86f527f
  Lintang Sutawika authored Dec 11, 2023
```
Change the sub-task name from sst to sst2 in glue
```
  c86f527f
- Delete lm_eval/tasks/glue/sst directory · ce4d9c68
  weijie authored Dec 11, 2023
  
  ce4d9c68
- Change the sub-task name from sst to sst2 in glue · dc556837
  weijie authored Dec 11, 2023
  
  dc556837
- Merge pull request #1097 from shiweijiezero/main · 09703d57
  Hailey Schoelkopf authored Dec 11, 2023
```
Update template of qqp dataset
```
  09703d57
- Update template of qqp dataset · 5dbdf582
  weijie authored Dec 11, 2023
  
  5dbdf582
- Debugging · c256eda8
  h-albert-lee authored Dec 11, 2023
  
  c256eda8
- Change spaces to underline · 489814b7
  h-albert-lee authored Dec 11, 2023
  
  489814b7
10 Dec, 2023 7 commits
- fix `add_special_tokens=False` in generate_until + decoding fixes · 452d49e0
  baberabb authored Dec 10, 2023
  
  452d49e0
- Remove blanks at the end of a file · 4edd9c4c
  h-albert-lee authored Dec 10, 2023
  
  4edd9c4c
- reformatted with Black · 520d8ce2
  h-albert-lee authored Dec 10, 2023
  
  520d8ce2
- Modify code for flake8 · 95908d8b
  h-albert-lee authored Dec 10, 2023
  
  95908d8b
- bugfix #2 remove 'context' · 9221cb57
  h-albert-lee authored Dec 10, 2023
  
  9221cb57
- bug fix (f-string) · 7bc9b82b
  h-albert-lee authored Dec 10, 2023
  
  7bc9b82b
- update yaml config · c64da8ac
  h-albert-lee authored Dec 10, 2023
  
  c64da8ac