Commits · c57ca81a36d5b1b832d126765dbe8e74af15b307 · gaoqiong / lm-evaluation-harness

"sgl-router/src/routers/grpc/regular/responses/tool_loop.rs" did not exist on "212f5e482242a88c7e697ad3c9b7e8084e3eb7e0"

29 Nov, 2023 6 commits
- fixed sampler issue with new default num_fewshot value; n_shot value changed · 38ac7b2a
  baberabb authored Nov 29, 2023
  
  38ac7b2a
- fix imports · 6359f083
  baberabb authored Nov 29, 2023
  
  6359f083
- rename function · 055d99b5
  baberabb authored Nov 29, 2023
  
  055d99b5
- remove unused imports · 1f504aa4
  baberabb authored Nov 29, 2023
  
  1f504aa4
- fix data + tensor parallel · 694dc642
  baberabb authored Nov 29, 2023
  
  694dc642
- fix chunking of inputs · 5075de60
  baberabb authored Nov 29, 2023
  
  5075de60
28 Nov, 2023 16 commits
- Update evaluator.py · b87ee153
  Lintang Sutawika authored Nov 28, 2023
  
  b87ee153
- test data parallel · 9c0d4e93
  baberabb authored Nov 28, 2023
  
  9c0d4e93
- Delete lm_eval/tasks/siqa/bigbench.yaml · 331d7c51
  Stella Biderman authored Nov 28, 2023
  
  331d7c51
- Update default.yml · 20463a3f
  Stella Biderman authored Nov 28, 2023
  
  20463a3f
- Rename bigbench.yml to default.yml · 5dcbcd63
  Stella Biderman authored Nov 28, 2023
```
Changing the name since this is not actually the big bench format.
```
  5dcbcd63
- reformat · 5ded4d5b
  lintangsutawika authored Nov 28, 2023
  
  5ded4d5b
- reformat · 37a46351
  lintangsutawika authored Nov 28, 2023
  
  37a46351
- update version · 43f615c9
  lintangsutawika authored Nov 28, 2023
  
  43f615c9
- update version · 94cdface
  lintangsutawika authored Nov 28, 2023
  
  94cdface
- add versions · 0d03a9f3
  lintangsutawika authored Nov 28, 2023
  
  0d03a9f3
- fixes how num_fewshot from config is read · 9e65d0ce
  lintangsutawika authored Nov 28, 2023
  
  9e65d0ce
- force configs with num_fewshot=0 to ignore manual num_fewshot setting · 37ccb191
  lintangsutawika authored Nov 28, 2023
  
  37ccb191
- use all_headers for headers · 61d0f137
  lintangsutawika authored Nov 28, 2023
  
  61d0f137
- renamed template file to make it easier to search · 19738ed2
  lintangsutawika authored Nov 28, 2023
  
  19738ed2
- renamed template file to make it easier to search · db8928b0
  lintangsutawika authored Nov 28, 2023
  
  db8928b0
- removed flan from the naming · 3b9640b8
  lintangsutawika authored Nov 28, 2023
  
  3b9640b8
27 Nov, 2023 12 commits
- Update README.md · 5535fbd4
  Stella Biderman authored Nov 27, 2023
  
  5535fbd4
- Update README.md · 4706fbb2
  Stella Biderman authored Nov 27, 2023
  
  4706fbb2
- Create bigbench.yml · 11129906
  Stella Biderman authored Nov 27, 2023
  
  11129906
- Create README.md · babdc8d6
  Stella Biderman authored Nov 27, 2023
  
  babdc8d6
- 'flan' in groupnames optional for all variants · 1e5ca7a4
  haileyschoelkopf authored Nov 27, 2023
  
  1e5ca7a4
- make 'bbh' a group name for flan cot 3-shot, add stopseqs · e408c2ce
  haileyschoelkopf authored Nov 27, 2023
  
  e408c2ce
- fix precommint · 7fa49671
  baberabb authored Nov 27, 2023
  
  7fa49671
- Update vllm_causallms.py · 08074cf9
  Hailey Schoelkopf authored Nov 27, 2023
  
  08074cf9
- Fix gpu_memory_utilization kwarg typing, change to conditional import · b01c4893
  Hailey Schoelkopf authored Nov 27, 2023
  
  b01c4893
- always import vllm model · d86400b5
  Hailey Schoelkopf authored Nov 27, 2023
  
  d86400b5
- add auto batching · bbd6d051
  baberabb authored Nov 27, 2023
  
  bbd6d051
- add auto batching · 37f10cad
  baberabb authored Nov 27, 2023
  
  37f10cad
26 Nov, 2023 1 commit
- use Seq2Seq backend where either can be loaded from HF · 7ab782ec
  haileyschoelkopf authored Nov 26, 2023
  
  7ab782ec
24 Nov, 2023 4 commits
- udate · a5b65c9b
  lintangsutawika authored Nov 24, 2023
  
  a5b65c9b
- fix missing lines · b64e592d
  lintangsutawika authored Nov 24, 2023
  
  b64e592d
- gen_kwargs is parsed once · 47933b98
  lintangsutawika authored Nov 24, 2023
  
  47933b98
- update on args · fde2a1d5
  lintangsutawika authored Nov 24, 2023
  
  fde2a1d5
22 Nov, 2023 1 commit
- fix args · 4f79fa7b
  baberabb authored Nov 23, 2023
  
  4f79fa7b