Commits · 81a11d6dbe3f86c6215eb58cc1474b22c955f262 · gaoqiong / lm-evaluation-harness

26 Jul, 2023 1 commit
- materialze "out" tensor during batch calculation · 81a11d6d
  Benjamin Fattori authored Jul 26, 2023
  
  81a11d6d
22 Jul, 2023 1 commit
- multi-device: take minimum computed autobatch over all ranks · 0a7720e9
  Benjamin Fattori authored Jul 22, 2023
  
  0a7720e9
18 Jul, 2023 1 commit
- Fix dummy model · 9bd6b0be
  Eddy Yeo authored Jul 18, 2023
  
  9bd6b0be
16 Jul, 2023 1 commit
- set fp32 if device=mps · ce214979
  baberabb authored Jul 16, 2023
  
  ce214979
15 Jul, 2023 2 commits
- add mps · 2bb7ce3b
  baberabb authored Jul 15, 2023
  
  2bb7ce3b
- removed rnd from task.fewshot_context in write_out.py; add mps in huggingface.py · 73b2149f
  baberabb authored Jul 15, 2023
  
  73b2149f
13 Jul, 2023 2 commits
- add fast tokenizer flag · 165f8493
  haileyschoelkopf authored Jul 13, 2023
  
  165f8493
- fix test_evaluator.py · d9b547b7
  baberabb authored Jul 13, 2023
  
  d9b547b7
12 Jul, 2023 1 commit
- remove TODO comment · e4acfcaa
  Benjamin Fattori authored Jul 12, 2023
  
  e4acfcaa
10 Jul, 2023 2 commits
- pass override bs to _loglikelihood_tokens · 0fa9f03f
  Benjamin Fattori authored Jul 10, 2023
  
  0fa9f03f
- initial autobatching commit · d42e1706
  Benjamin Fattori authored Jul 10, 2023
  
  d42e1706
03 Jul, 2023 5 commits
- don't place quantized models .to(device) · 0b896bd2
  haileyschoelkopf authored Jul 03, 2023
  
  0b896bd2
- revert typehint · b3598058
  haileyschoelkopf authored Jul 03, 2023
  
  b3598058
- revert gpus > num processes case · 5cb7af7e
  haileyschoelkopf authored Jul 03, 2023
  
  5cb7af7e
- handle device assignment better · b8154374
  haileyschoelkopf authored Jul 03, 2023
  
  b8154374
- auto-gptq -> gptq and an import warning · 1d8f782f
  gk authored Jul 04, 2023
  
  1d8f782f
02 Jul, 2023 1 commit
- Add PEFT, quantization, remote code, LLaMA fix · 18d842ab
  gk authored Jul 02, 2023
  
  18d842ab
29 Jun, 2023 1 commit
- remove stray print · cb5423d3
  haileyschoelkopf authored Jun 29, 2023
  
  cb5423d3
28 Jun, 2023 7 commits
- cleanup + change to name=parallelize · 1b4fc896
  haileyschoelkopf authored Jun 28, 2023
  
  1b4fc896
- add device_map options for HFLM · 23eaa2db
  haileyschoelkopf authored Jun 28, 2023
  
  23eaa2db
- clean up batched code and add comments · b250b001
  haileyschoelkopf authored Jun 28, 2023
  
  b250b001
- make OAI completions work with new generation kwargs format · 5a5442ff
  haileyschoelkopf authored Jun 28, 2023
  
  5a5442ff
- add cache hooks to API LMs · d1c5abef
  haileyschoelkopf authored Jun 28, 2023
  
  d1c5abef
- change generation_kwargs -> kwargs · 732f7ed2
  haileyschoelkopf authored Jun 28, 2023
  
  732f7ed2
- push WIP batched multi-kwarg code · ffc3a456
  haileyschoelkopf authored Jun 28, 2023
  
  ffc3a456
27 Jun, 2023 1 commit
- update lm_eval.models.__init__ · 2fef6bc5
  haileyschoelkopf authored Jun 27, 2023
  
  2fef6bc5
26 Jun, 2023 1 commit
- remove hf-causal model type · 4f002df5
  haileyschoelkopf authored Jun 26, 2023
  
  4f002df5
24 Jun, 2023 2 commits
- remove stale commented code · 513352ae
  haileyschoelkopf authored Jun 24, 2023
  
  513352ae
- add batched generation · dced6e96
  haileyschoelkopf authored Jun 24, 2023
  
  dced6e96
23 Jun, 2023 1 commit
- add cache hooks to HF · e3960fa0
  haileyschoelkopf authored Jun 23, 2023
  
  e3960fa0
22 Jun, 2023 10 commits
- fix edge cases for seq2seq · 0500fb33
  haileyschoelkopf authored Jun 22, 2023
  
  0500fb33
- remove pure seq2seq LM class · 82a21c36
  haileyschoelkopf authored Jun 22, 2023
  
  82a21c36
- fix issues with encoder_attns, test lambada · 621214e1
  haileyschoelkopf authored Jun 22, 2023
  
  621214e1
- remove stray print in AnthropicLM, add TODO · 1c409035
  haileyschoelkopf authored Jun 21, 2023
  
  1c409035
- rename hf_merged.py · 360f7c3e
  haileyschoelkopf authored Jun 21, 2023
  
  360f7c3e
- discard ctx toks for causal greedy_until · 3a6adddb
  haileyschoelkopf authored Jun 21, 2023
  
  3a6adddb
- more pre-commit · b48f5205
  haileyschoelkopf authored Jun 20, 2023
  
  b48f5205
- add start of docstring to HFLM · 86b71954
  haileyschoelkopf authored Jun 20, 2023
  
  86b71954
- init tensors on-device where possible · 0e4f5361
  haileyschoelkopf authored Jun 20, 2023
  
  0e4f5361
- automatically unwrap model when needed · bc7f52e6
  haileyschoelkopf authored Jun 20, 2023
  
  bc7f52e6