Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
00934026
Unverified
Commit
00934026
authored
Mar 17, 2023
by
Sylvain Gugger
Committed by
GitHub
Mar 17, 2023
Browse files
LLaMA house-keeping (#22216)
* LLaMA house-keeping * Doc links
parent
42f8f764
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
7 additions
and
5 deletions
+7
-5
docs/source/en/model_doc/llama.mdx
docs/source/en/model_doc/llama.mdx
+4
-2
src/transformers/__init__.py
src/transformers/__init__.py
+1
-1
src/transformers/models/llama/configuration_llama.py
src/transformers/models/llama/configuration_llama.py
+2
-2
No files found.
docs/source/en/model_doc/llama.mdx
View file @
00934026
...
@@ -33,8 +33,10 @@ python src/transformers/models/llama/convert_llama_weights_to_hf.py \
...
@@ -33,8 +33,10 @@ python src/transformers/models/llama/convert_llama_weights_to_hf.py \
-
After
conversion
,
the
model
and
tokenizer
can
be
loaded
via
:
-
After
conversion
,
the
model
and
tokenizer
can
be
loaded
via
:
```
python
```
python
tokenizer
=
transformers
.
LlamaTokenizer
.
from_pretrained
(
"/output/path/tokenizer/"
)
from
transformers
import
LlamaForCausalLM
,
LlamaTokenizer
model
=
transformers
.
LlamaForCausalLM
.
from_pretrained
(
"/output/path/llama-7b/"
)
tokenizer
=
LlamaTokenizer
.
from_pretrained
(
"/output/path/tokenizer/"
)
model
=
LlamaForCausalLM
.
from_pretrained
(
"/output/path/llama-7b/"
)
```
```
-
The
LLaMA
tokenizer
is
based
on
[
sentencepiece
](
https
://
github
.
com
/
google
/
sentencepiece
).
One
quirk
of
sentencepiece
is
that
when
decoding
a
sequence
,
if
the
first
token
is
the
start
of
the
word
(
e
.
g
.
"Banana"
),
the
tokenizer
does
not
prepend
the
prefix
space
to
the
string
.
To
have
the
tokenizer
output
the
prefix
space
,
set
`
decode_with_prefix_space
=
True
`
in
the
`
LlamaTokenizer
`
object
or
in
the
tokenizer
configuration
.
-
The
LLaMA
tokenizer
is
based
on
[
sentencepiece
](
https
://
github
.
com
/
google
/
sentencepiece
).
One
quirk
of
sentencepiece
is
that
when
decoding
a
sequence
,
if
the
first
token
is
the
start
of
the
word
(
e
.
g
.
"Banana"
),
the
tokenizer
does
not
prepend
the
prefix
space
to
the
string
.
To
have
the
tokenizer
output
the
prefix
space
,
set
`
decode_with_prefix_space
=
True
`
in
the
`
LlamaTokenizer
`
object
or
in
the
tokenizer
configuration
.
...
...
src/transformers/__init__.py
View file @
00934026
...
@@ -4486,9 +4486,9 @@ if TYPE_CHECKING:
...
@@ -4486,9 +4486,9 @@ if TYPE_CHECKING:
TypicalLogitsWarper
,
TypicalLogitsWarper
,
top_k_top_p_filtering
,
top_k_top_p_filtering
,
)
)
from
.modeling_utils
import
PreTrainedModel
# PyTorch model imports
# PyTorch model imports
from
.modeling_utils
import
PreTrainedModel
from
.models.albert
import
(
from
.models.albert
import
(
ALBERT_PRETRAINED_MODEL_ARCHIVE_LIST
,
ALBERT_PRETRAINED_MODEL_ARCHIVE_LIST
,
AlbertForMaskedLM
,
AlbertForMaskedLM
,
...
...
src/transformers/models/llama/configuration_llama.py
View file @
00934026
...
@@ -30,7 +30,7 @@ LLAMA_PRETRAINED_CONFIG_ARCHIVE_MAP = {}
...
@@ -30,7 +30,7 @@ LLAMA_PRETRAINED_CONFIG_ARCHIVE_MAP = {}
class
LlamaConfig
(
PretrainedConfig
):
class
LlamaConfig
(
PretrainedConfig
):
r
"""
r
"""
This is the configuration class to store the configuration of a [`
~
LlamaModel`]. It is used to instantiate an LLaMA
This is the configuration class to store the configuration of a [`LlamaModel`]. It is used to instantiate an LLaMA
model according to the specified arguments, defining the model architecture. Instantiating a configuration with the
model according to the specified arguments, defining the model architecture. Instantiating a configuration with the
defaults will yield a similar configuration to that of the LLaMA-7B.
defaults will yield a similar configuration to that of the LLaMA-7B.
...
@@ -41,7 +41,7 @@ class LlamaConfig(PretrainedConfig):
...
@@ -41,7 +41,7 @@ class LlamaConfig(PretrainedConfig):
Args:
Args:
vocab_size (`int`, *optional*, defaults to 32000):
vocab_size (`int`, *optional*, defaults to 32000):
Vocabulary size of the LLaMA model. Defines the number of different tokens that can be represented by the
Vocabulary size of the LLaMA model. Defines the number of different tokens that can be represented by the
`inputs_ids` passed when calling [`
~
LlamaModel`]
`inputs_ids` passed when calling [`LlamaModel`]
hidden_size (`int`, *optional*, defaults to 4096):
hidden_size (`int`, *optional*, defaults to 4096):
Dimension of the hidden representations.
Dimension of the hidden representations.
intermediate_size (`int`, *optional*, defaults to 11008):
intermediate_size (`int`, *optional*, defaults to 11008):
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment