Commits · 3bb230d3d8d85e40944a58196b87190d40404a59 · gaoqiong / lm-evaluation-harness

03 Dec, 2023 3 commits
- add tokenizer options to model_args · 3bb230d3
  baberabb authored Dec 03, 2023
  
  3bb230d3
- nits · d0dc5bf9
  baberabb authored Dec 03, 2023
  
  d0dc5bf9
- fix tokenizer · 1a0f57d7
  baberabb authored Dec 03, 2023
  
  1a0f57d7
29 Nov, 2023 10 commits
- fixup · d4edd280
  baberabb authored Nov 29, 2023
  
  d4edd280
- fixed chunking · da2d160b
  baberabb authored Nov 29, 2023
  
  da2d160b
- simpler request chunking · b97c561f
  baberabb authored Nov 29, 2023
  
  b97c561f
- fixed sampler issue with new default num_fewshot value; n_shot value changed · 38ac7b2a
  baberabb authored Nov 29, 2023
  
  38ac7b2a
- change torch req for mps · 7bb147b5
  baberabb authored Nov 29, 2023
  
  7bb147b5
- fix imports · 6359f083
  baberabb authored Nov 29, 2023
  
  6359f083
- rename function · 055d99b5
  baberabb authored Nov 29, 2023
  
  055d99b5
- remove unused imports · 1f504aa4
  baberabb authored Nov 29, 2023
  
  1f504aa4
- fix data + tensor parallel · 694dc642
  baberabb authored Nov 29, 2023
  
  694dc642
- fix chunking of inputs · 5075de60
  baberabb authored Nov 29, 2023
  
  5075de60
28 Nov, 2023 17 commits
- Update evaluator.py · b87ee153
  Lintang Sutawika authored Nov 28, 2023
  
  b87ee153
- test data parallel · 9c0d4e93
  baberabb authored Nov 28, 2023
  
  9c0d4e93
- Delete lm_eval/tasks/siqa/bigbench.yaml · 331d7c51
  Stella Biderman authored Nov 28, 2023
  
  331d7c51
- Update default.yml · 20463a3f
  Stella Biderman authored Nov 28, 2023
  
  20463a3f
- Rename bigbench.yml to default.yml · 5dcbcd63
  Stella Biderman authored Nov 28, 2023
```
Changing the name since this is not actually the big bench format.
```
  5dcbcd63
- reformat · 5ded4d5b
  lintangsutawika authored Nov 28, 2023
  
  5ded4d5b
- reformat · 37a46351
  lintangsutawika authored Nov 28, 2023
  
  37a46351
- update version · 43f615c9
  lintangsutawika authored Nov 28, 2023
  
  43f615c9
- update version · 94cdface
  lintangsutawika authored Nov 28, 2023
  
  94cdface
- add versions · 0d03a9f3
  lintangsutawika authored Nov 28, 2023
  
  0d03a9f3
- fixes how num_fewshot from config is read · 9e65d0ce
  lintangsutawika authored Nov 28, 2023
  
  9e65d0ce
- force configs with num_fewshot=0 to ignore manual num_fewshot setting · 37ccb191
  lintangsutawika authored Nov 28, 2023
  
  37ccb191
- use all_headers for headers · 61d0f137
  lintangsutawika authored Nov 28, 2023
  
  61d0f137
- renamed template file to make it easier to search · 19738ed2
  lintangsutawika authored Nov 28, 2023
  
  19738ed2
- renamed template file to make it easier to search · db8928b0
  lintangsutawika authored Nov 28, 2023
  
  db8928b0
- updated oa_completion and fix import error · 8ea23e2d
  lintangsutawika authored Nov 28, 2023
  
  8ea23e2d
- removed flan from the naming · 3b9640b8
  lintangsutawika authored Nov 28, 2023
  
  3b9640b8
27 Nov, 2023 10 commits
- Update README.md · 5535fbd4
  Stella Biderman authored Nov 27, 2023
  
  5535fbd4
- Update README.md · 4706fbb2
  Stella Biderman authored Nov 27, 2023
  
  4706fbb2
- Create bigbench.yml · 11129906
  Stella Biderman authored Nov 27, 2023
  
  11129906
- Create README.md · babdc8d6
  Stella Biderman authored Nov 27, 2023
  
  babdc8d6
- 'flan' in groupnames optional for all variants · 1e5ca7a4
  haileyschoelkopf authored Nov 27, 2023
  
  1e5ca7a4
- make 'bbh' a group name for flan cot 3-shot, add stopseqs · e408c2ce
  haileyschoelkopf authored Nov 27, 2023
  
  e408c2ce
- fix precommint · 7fa49671
  baberabb authored Nov 27, 2023
  
  7fa49671
- Update vllm_causallms.py · 08074cf9
  Hailey Schoelkopf authored Nov 27, 2023
  
  08074cf9
- Fix gpu_memory_utilization kwarg typing, change to conditional import · b01c4893
  Hailey Schoelkopf authored Nov 27, 2023
  
  b01c4893
- always import vllm model · d86400b5
  Hailey Schoelkopf authored Nov 27, 2023
  
  d86400b5