Commit 24321ad8 authored by lintangsutawika's avatar lintangsutawika
Browse files

pre-commit reformat

parent 60e8803f
# Eval Harness Documentation # Eval Harness Documentation
Welcome to the docs for the LM Evaluation Harness! Welcome to the docs for the LM Evaluation Harness!
## Table of Contents ## Table of Contents
...@@ -20,4 +20,4 @@ Tracking progress on revamping documentation pages for the refactor of LM-Evalua ...@@ -20,4 +20,4 @@ Tracking progress on revamping documentation pages for the refactor of LM-Evalua
* [ ] Explaining registries + decorators * [ ] Explaining registries + decorators
* [ ] model_guide.md for adding new model API * [ ] model_guide.md for adding new model API
* [ ] guide to writing an adapter to new advanced codebase (e.g. NeoX) * [ ] guide to writing an adapter to new advanced codebase (e.g. NeoX)
* [ ] Parallelism guide (?) * [ ] Parallelism guide (?)
\ No newline at end of file
...@@ -46,9 +46,9 @@ class MyCustomLM(LM): ...@@ -46,9 +46,9 @@ class MyCustomLM(LM):
#... #...
``` ```
We support We support
The three types of The three types of
...@@ -64,9 +64,9 @@ smth smth tokenizer-agnostic ...@@ -64,9 +64,9 @@ smth smth tokenizer-agnostic
## Registration ## Registration
Congrats on implementing your model! Now it's time to test it out. Congrats on implementing your model! Now it's time to test it out.
To make your model usable via the command line interface to `lm-eval` using `main.py`, you'll need to tell `lm-eval` what your model's name is. To make your model usable via the command line interface to `lm-eval` using `main.py`, you'll need to tell `lm-eval` what your model's name is.
This is done via a *decorator*, `lm_eval.api.registry.register_model`. Using `register_model()`, one can both tell the package what the model's name(s) to be used are when invoking it with `python main.py --model <name>` and alert `lm-eval` to the model's existence. This is done via a *decorator*, `lm_eval.api.registry.register_model`. Using `register_model()`, one can both tell the package what the model's name(s) to be used are when invoking it with `python main.py --model <name>` and alert `lm-eval` to the model's existence.
...@@ -87,4 +87,4 @@ Using this decorator results in the class being added to an accounting of the us ...@@ -87,4 +87,4 @@ Using this decorator results in the class being added to an accounting of the us
## Conclusion ## Conclusion
After reading this guide, you should be able to add new model APIs or implementations to the Eval Harness library! After reading this guide, you should be able to add new model APIs or implementations to the Eval Harness library!
\ No newline at end of file
...@@ -436,7 +436,9 @@ class Task(abc.ABC): ...@@ -436,7 +436,9 @@ class Task(abc.ABC):
# always prepend the (possibly empty) task description # always prepend the (possibly empty) task description
labeled_examples = self._config.description labeled_examples = self._config.description
else: else:
labeled_examples = self._config.description + self.sampler.get_context(doc, num_fewshot) labeled_examples = self._config.description + self.sampler.get_context(
doc, num_fewshot
)
example = self.doc_to_text(doc) example = self.doc_to_text(doc)
return labeled_examples + example return labeled_examples + example
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment