Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
14b8ec7e
Unverified
Commit
14b8ec7e
authored
Apr 28, 2022
by
Stella Biderman
Committed by
GitHub
Apr 28, 2022
Browse files
Merge pull request #309 from dirkgr/PytestUpdate
Fixes for using lm_eval as a library
parents
827e2d51
e9f8f654
Changes
25
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
2 additions
and
2 deletions
+2
-2
.github/workflows/python-app.yml
.github/workflows/python-app.yml
+1
-1
docs/task_guide.md
docs/task_guide.md
+1
-1
lm_eval/datasets/arithmetic/__init__.py
lm_eval/datasets/arithmetic/__init__.py
+0
-0
lm_eval/datasets/asdiv/__init__.py
lm_eval/datasets/asdiv/__init__.py
+0
-0
lm_eval/datasets/coqa/__init__.py
lm_eval/datasets/coqa/__init__.py
+0
-0
lm_eval/datasets/drop/__init__.py
lm_eval/datasets/drop/__init__.py
+0
-0
lm_eval/datasets/gsm8k/__init__.py
lm_eval/datasets/gsm8k/__init__.py
+0
-0
lm_eval/datasets/headqa/__init__.py
lm_eval/datasets/headqa/__init__.py
+0
-0
lm_eval/datasets/hendrycks_ethics/__init__.py
lm_eval/datasets/hendrycks_ethics/__init__.py
+0
-0
lm_eval/datasets/hendrycks_math/__init__.py
lm_eval/datasets/hendrycks_math/__init__.py
+0
-0
lm_eval/datasets/lambada/__init__.py
lm_eval/datasets/lambada/__init__.py
+0
-0
lm_eval/datasets/logiqa/__init__.py
lm_eval/datasets/logiqa/__init__.py
+0
-0
lm_eval/datasets/mutual/__init__.py
lm_eval/datasets/mutual/__init__.py
+0
-0
lm_eval/datasets/pile/__init__.py
lm_eval/datasets/pile/__init__.py
+0
-0
lm_eval/datasets/quac/__init__.py
lm_eval/datasets/quac/__init__.py
+0
-0
lm_eval/datasets/sat_analogies/__init__.py
lm_eval/datasets/sat_analogies/__init__.py
+0
-0
lm_eval/datasets/triviaqa/__init__.py
lm_eval/datasets/triviaqa/__init__.py
+0
-0
lm_eval/datasets/truthfulqa/__init__.py
lm_eval/datasets/truthfulqa/__init__.py
+0
-0
lm_eval/datasets/unscramble/__init__.py
lm_eval/datasets/unscramble/__init__.py
+0
-0
lm_eval/datasets/wikitext/__init__.py
lm_eval/datasets/wikitext/__init__.py
+0
-0
No files found.
.github/workflows/python-app.yml
View file @
14b8ec7e
...
@@ -32,7 +32,7 @@ jobs:
...
@@ -32,7 +32,7 @@ jobs:
run
:
|
run
:
|
python -m pip install --upgrade pip
python -m pip install --upgrade pip
pip install flake8 pytest pytest-cov
pip install flake8 pytest pytest-cov
pip install -e .
pip install -e .
[dev]
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
-
name
:
Lint with flake8
-
name
:
Lint with flake8
run
:
|
run
:
|
...
...
docs/task_guide.md
View file @
14b8ec7e
...
@@ -11,7 +11,7 @@ If you haven't already, go ahead and fork the main repo, clone it, create a bran
...
@@ -11,7 +11,7 @@ If you haven't already, go ahead and fork the main repo, clone it, create a bran
git clone https://github.com/<YOUR-USERNAME>/lm-evaluation-harness.git
git clone https://github.com/<YOUR-USERNAME>/lm-evaluation-harness.git
cd
lm-evaluation-harness
cd
lm-evaluation-harness
git checkout
-b
<task-name>
git checkout
-b
<task-name>
pip
install
-
r
requirements.txt
pip
install
-
e
".[dev]"
```
```
## Creating Your Task File
## Creating Your Task File
...
...
lm_eval/datasets/arithmetic/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/asdiv/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/coqa/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/drop/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/gsm8k/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/headqa/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/hendrycks_ethics/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/hendrycks_math/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/lambada/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/logiqa/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/mutual/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/pile/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/quac/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/sat_analogies/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/triviaqa/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/truthfulqa/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/unscramble/__init__.py
0 → 100644
View file @
14b8ec7e
lm_eval/datasets/wikitext/__init__.py
0 → 100644
View file @
14b8ec7e
Prev
1
2
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment