Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
d3d388b9
"git@developer.sourcefind.cn:dadigang/Ventoy.git" did not exist on "385c806adf77847441e81356bb36abf5ec0ae056"
Unverified
Commit
d3d388b9
authored
Mar 16, 2021
by
Suraj Patil
Committed by
GitHub
Mar 16, 2021
Browse files
fix M2M100 example (#10745)
parent
b5492582
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
1 deletion
+4
-1
docs/source/model_doc/m2m_100.rst
docs/source/model_doc/m2m_100.rst
+4
-1
No files found.
docs/source/model_doc/m2m_100.rst
View file @
d3d388b9
...
...
@@ -43,6 +43,9 @@ multilingual it expects the sequences in a certain format: A special language id
source
and
target
text
.
The
source
text
format
is
:
obj
:`[
lang_code
]
X
[
eos
]`,
where
:
obj
:`
lang_code
`
is
source
language
id
for
source
text
and
target
language
id
for
target
text
,
with
:
obj
:`
X
`
being
the
source
or
target
text
.
The
:
class
:`~
transformers
.
M2M100Tokenizer
`
depends
on
:
obj
:`
sentencepiece
`
so
be
sure
to
install
it
before
running
the
examples
.
To
install
:
obj
:`
sentencepiece
`
run
``
pip
install
sentencepiece
``.
-
Supervised
Training
..
code
-
block
::
...
...
@@ -87,7 +90,7 @@ id for source text and target language id for target text, with :obj:`X` being t
"La vie est comme une boîte de chocolat."
>>>
#
translate
Chinese
to
English
>>>
tokenizer
.
src_lang
=
"
ar_AR
"
>>>
tokenizer
.
src_lang
=
"
zh
"
>>>
encoded_zh
=
tokenizer
(
chinese_text
,
return_tensors
=
"pt"
)
>>>
generated_tokens
=
model
.
generate
(**
encoded_zh
,
forced_bos_token_id
=
tokenizer
.
get_lang_id
(
"en"
))
>>>
tokenizer
.
batch_decode
(
generated_tokens
,
skip_special_tokens
=
True
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment