Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
5c4d9c72
Commit
5c4d9c72
authored
Jan 06, 2024
by
daniel-furman
Browse files
first stab at wrap_chat_template, remove system for now
parent
87dff8b0
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
13 deletions
+5
-13
lm_eval/models/huggingface.py
lm_eval/models/huggingface.py
+5
-13
No files found.
lm_eval/models/huggingface.py
View file @
5c4d9c72
...
...
@@ -663,25 +663,17 @@ class HFLM(LM):
elif
self
.
AUTO_MODEL_CLASS
==
transformers
.
AutoModelForSeq2SeqLM
:
return
self
.
tokenizer
.
decode
(
tokens
,
skip_special_tokens
=
True
)
def
tok_wrap_chat_template
(
self
,
requests
:
List
[
Instance
]
,
system
:
bool
=
False
)
->
List
[
Instance
]:
def
tok_wrap_chat_template
(
self
,
requests
:
List
[
Instance
])
->
List
[
Instance
]:
"""
Utility for adding chat templates via the apply_chat_template() method
"""
new_reqs
=
[]
for
req
in
requests
:
context
,
continuation
=
req
.
args
[
0
].
strip
(),
req
.
args
[
1
].
strip
()
if
system
:
chat
=
[
{
"role"
:
"system"
,
"content"
:
system
},
{
"role"
:
"user"
,
"content"
:
context
},
{
"role"
:
"assistant"
,
"content"
:
continuation
},
]
else
:
chat
=
[
{
"role"
:
"user"
,
"content"
:
context
},
{
"role"
:
"assistant"
,
"content"
:
continuation
},
]
chat
=
[
{
"role"
:
"user"
,
"content"
:
context
},
{
"role"
:
"assistant"
,
"content"
:
continuation
},
]
single_tokenized_conversation
=
self
.
tokenizer
.
apply_chat_template
(
chat
,
tokenize
=
False
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment