Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
5d7e8457
"kubernetes/helm/vscode:/vscode.git/clone" did not exist on "e08c144f0b93672eae7eff1f4914a8bc14751f00"
Commit
5d7e8457
authored
Feb 08, 2019
by
thomwolf
Browse files
fix model on cuda
parent
eccb2f01
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
0 deletions
+1
-0
examples/run_openai_gpt.py
examples/run_openai_gpt.py
+1
-0
No files found.
examples/run_openai_gpt.py
View file @
5d7e8457
...
...
@@ -135,6 +135,7 @@ def main():
tokenizer
=
OpenAIGPTTokenizer
.
from_pretrained
(
args
.
model_name
,
special_tokens
=
special_tokens
)
special_tokens_ids
=
list
(
tokenizer
.
convert_tokens_to_ids
(
token
)
for
token
in
special_tokens
)
model
=
OpenAIGPTDoubleHeadsModel
.
from_pretrained
(
args
.
model_name
,
num_special_tokens
=
len
(
special_tokens
))
model
.
to
(
device
)
# Load and encode the datasets
def
tokenize_and_encode
(
obj
):
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment