Commits · 4277dc35b375dd7e186162692be587ff4da7dd8f · gaoqiong / lm-evaluation-harness

27 Nov, 2023 12 commits
- fixes for precommit · 4277dc35
  Hailey Schoelkopf authored Nov 27, 2023
  
  4277dc35
- fix precommint · 7fa49671
  baberabb authored Nov 27, 2023
  
  7fa49671
- skip vllm test (requires CUDA) · 05326b04
  baberabb authored Nov 27, 2023
  
  05326b04
- Update unit_tests.yml · b2282739
  Hailey Schoelkopf authored Nov 27, 2023
  
  b2282739
- Update vllm_causallms.py · 08074cf9
  Hailey Schoelkopf authored Nov 27, 2023
  
  08074cf9
- Add vLLM to README · 581dd9ff
  Hailey Schoelkopf authored Nov 27, 2023
  
  581dd9ff
- Fix gpu_memory_utilization kwarg typing, change to conditional import · b01c4893
  Hailey Schoelkopf authored Nov 27, 2023
  
  b01c4893
- always import vllm model · d86400b5
  Hailey Schoelkopf authored Nov 27, 2023
  
  d86400b5
- add auto batching · bbd6d051
  baberabb authored Nov 27, 2023
  
  bbd6d051
- add auto batching · 37f10cad
  baberabb authored Nov 27, 2023
  
  37f10cad
- lazy load import in tests · 2c20df08
  baberabb authored Nov 27, 2023
  
  2c20df08
- Add tests for VLLM · 026005ac
  baberabb authored Nov 27, 2023
  
  026005ac
22 Nov, 2023 10 commits
- fix args · 4f79fa7b
  baberabb authored Nov 23, 2023
  
  4f79fa7b
- Update vllm compatibility in README. · 953bfdd2
  baberabb authored Nov 23, 2023
  
  953bfdd2
- add dependency · b570fce7
  baberabb authored Nov 23, 2023
  
  b570fce7
- fix tensor parallel · af884028
  baberabb authored Nov 23, 2023
  
  af884028
- fix imports · 5ba54539
  baberabb authored Nov 23, 2023
  
  5ba54539
- fix imports · ce164bb1
  baberabb authored Nov 23, 2023
  
  ce164bb1
- disable tqdm · 667fc837
  baberabb authored Nov 23, 2023
  
  667fc837
- fix logliklehood_rolling · ac664bce
  baberabb authored Nov 23, 2023
  
  ac664bce
- bugfix · 2362ab41
  baberabb authored Nov 22, 2023
  
  2362ab41
- fix greedy_until · e5491709
  baberabb authored Nov 22, 2023
  
  e5491709
21 Nov, 2023 8 commits
- (temp) fix import · 24f4e8d7
  baberabb authored Nov 22, 2023
  
  24f4e8d7
- bugfix · f2890cc1
  baberabb authored Nov 22, 2023
  
  f2890cc1
- Merge remote-tracking branch 'origin/big-refactor' into big-refactor_vllm · 91868979
  baberabb authored Nov 22, 2023
```
# Conflicts:
#	lm_eval/models/__init__.py
```
  91868979
- bugfix · 0635af13
  baberabb authored Nov 22, 2023
  
  0635af13
- bugfix · 103a10e3
  baberabb authored Nov 21, 2023
  
  103a10e3
- add vllm model · a2920e3d
  baberabb authored Nov 21, 2023
  
  a2920e3d
- Merge pull request #967 from EleutherAI/upstream-ggml · 815cd29d
  Hailey Schoelkopf authored Nov 21, 2023
```
[Refactor] Upstream ggml from big-refactor branch
```
  815cd29d
- add vllm model · 3d8bc82f
  baberabb authored Nov 21, 2023
  
  3d8bc82f
20 Nov, 2023 4 commits
- Merge pull request #979 from EleutherAI/benchmark-documentation · e3644fcc
  Stella Biderman authored Nov 20, 2023
```
add description on task/group alias
```
  e3644fcc
- add ggml tests · f87b864f
  haileyschoelkopf authored Nov 20, 2023
  
  f87b864f
- add typehints · 1e9c8b59
  baberabb authored Nov 20, 2023
  
  1e9c8b59
- add docs · b22f3440
  baberabb authored Nov 20, 2023
  
  b22f3440
17 Nov, 2023 6 commits
- Merge pull request #958 from EleutherAI/verbosity-rework · afda6551
  Hailey Schoelkopf authored Nov 17, 2023
```
[Refactor] Verbosity rework
```
  afda6551
- add initialize_tasks() to interface guide in docs · 9b596e8f
  haileyschoelkopf authored Nov 17, 2023
  
  9b596e8f
- initialize tasks in test files · 032820dd
  haileyschoelkopf authored Nov 17, 2023
  
  032820dd
- Merge pull request #1002 from EleutherAI/bugfixes-bigrefactor · 2c0c345a
  Hailey Schoelkopf authored Nov 17, 2023
```
[Refactor] Bugfixes
```
  2c0c345a
- update bigbench datasets to mirror, rename groupnames · 487767ab
  haileyschoelkopf authored Nov 17, 2023
  
  487767ab
- fix error when using wildcard task names · 2444b364
  haileyschoelkopf authored Nov 17, 2023
  
  2444b364