Commits · 386d63ea1d6d6c94b3a9f86e206cd5dcb2f88f51 · gaoqiong / lm-evaluation-harness

15 Dec, 2023 2 commits
- fixed brier_score to allow multi-gpu inference · 386d63ea
  lintangsutawika authored Dec 15, 2023
  
  386d63ea
- removed group and task from yaml template · 4efa0b6d
  lintangsutawika authored Dec 15, 2023
  
  4efa0b6d
13 Dec, 2023 15 commits
- fixed style · 50967725
  lintangsutawika authored Dec 13, 2023
  
  50967725
- fixed stderr for metrics like brier_score · 4bff76d5
  lintangsutawika authored Dec 13, 2023
  
  4bff76d5
- add styles 5 -8 · 5da401b0
  lintangsutawika authored Dec 13, 2023
  
  5da401b0
- fixed list · 3c5f347f
  lintangsutawika authored Dec 13, 2023
  
  3c5f347f
- allow doc_to_choice to be a single feature from a dataset · e96c6484
  lintangsutawika authored Dec 13, 2023
  
  e96c6484
- removed comments · cd4ef5e8
  lintangsutawika authored Dec 13, 2023
  
  cd4ef5e8
- moved files · d5c6f0e5
  lintangsutawika authored Dec 13, 2023
  
  d5c6f0e5
- reveresed os.walk · 6fdecc6f
  lintangsutawika authored Dec 13, 2023
  
  6fdecc6f
- format · ec656bb1
  lintangsutawika authored Dec 13, 2023
  
  ec656bb1
- added mmlu alt world prompts · 0d5748b7
  lintangsutawika authored Dec 13, 2023
  
  0d5748b7
- add alt prompts for truthfulqa · b9e55312
  lintangsutawika authored Dec 13, 2023
  
  b9e55312
- removed · edb21d21
  lintangsutawika authored Dec 13, 2023
  
  edb21d21
- removed · 6e220b17
  lintangsutawika authored Dec 13, 2023
  
  6e220b17
- remove style · 09cac5dc
  lintangsutawika authored Dec 13, 2023
  
  09cac5dc
- fixed file path · ec4cdfed
  lintangsutawika authored Dec 13, 2023
  
  ec4cdfed
12 Dec, 2023 5 commits
- Update vllm_causallms.py · d588a466
  Hailey Schoelkopf authored Dec 12, 2023
  
  d588a466
- data_parallel -> data_parallel_size · 6409fe6c
  Hailey Schoelkopf authored Dec 12, 2023
  
  6409fe6c
- allow N/A as a possible value for Stderr · d6f8faad
  lintangsutawika authored Dec 12, 2023
  
  d6f8faad
- calculate std if items > 1 · 91279a6e
  lintangsutawika authored Dec 12, 2023
  
  91279a6e
- simplified evaluator by removing unused variables · 62e4ea10
  lintangsutawika authored Dec 12, 2023
  
  62e4ea10
11 Dec, 2023 5 commits
- Delete lm_eval/tasks/glue/sst directory · ce4d9c68
  weijie authored Dec 11, 2023
  
  ce4d9c68
- Change the sub-task name from sst to sst2 in glue · dc556837
  weijie authored Dec 11, 2023
  
  dc556837
- Update template of qqp dataset · 5dbdf582
  weijie authored Dec 11, 2023
  
  5dbdf582
- Debugging · c256eda8
  h-albert-lee authored Dec 11, 2023
  
  c256eda8
- Change spaces to underline · 489814b7
  h-albert-lee authored Dec 11, 2023
  
  489814b7
10 Dec, 2023 7 commits
- fix `add_special_tokens=False` in generate_until + decoding fixes · 452d49e0
  baberabb authored Dec 10, 2023
  
  452d49e0
- Remove blanks at the end of a file · 4edd9c4c
  h-albert-lee authored Dec 10, 2023
  
  4edd9c4c
- reformatted with Black · 520d8ce2
  h-albert-lee authored Dec 10, 2023
  
  520d8ce2
- Modify code for flake8 · 95908d8b
  h-albert-lee authored Dec 10, 2023
  
  95908d8b
- bugfix #2 remove 'context' · 9221cb57
  h-albert-lee authored Dec 10, 2023
  
  9221cb57
- bug fix (f-string) · 7bc9b82b
  h-albert-lee authored Dec 10, 2023
  
  7bc9b82b
- update yaml config · c64da8ac
  h-albert-lee authored Dec 10, 2023
  
  c64da8ac
08 Dec, 2023 6 commits
- implementing kmmlu · 1b14602e
  h-albert-lee authored Dec 08, 2023
  
  1b14602e
- remove README.md · 1653632c
  lintangsutawika authored Dec 08, 2023
  
  1653632c
- added prompt and output variations · 1ff18c9a
  lintangsutawika authored Dec 08, 2023
  
  1ff18c9a
- added prompt and output variations · af6e46fd
  lintangsutawika authored Dec 08, 2023
  
  af6e46fd
- add prompt variation · 66421b57
  lintangsutawika authored Dec 08, 2023
  
  66421b57
- made prompt and output variations · 55eff889
  lintangsutawika authored Dec 08, 2023
  
  55eff889