Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
00934026
Unverified
Commit
00934026
authored
Mar 17, 2023
by
Sylvain Gugger
Committed by
GitHub
Mar 17, 2023
Browse files
LLaMA house-keeping (#22216)
* LLaMA house-keeping * Doc links
parent
42f8f764
Changes
3
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
7 additions
and
5 deletions
+7
-5
docs/source/en/model_doc/llama.mdx
docs/source/en/model_doc/llama.mdx
+4
-2
src/transformers/__init__.py
src/transformers/__init__.py
+1
-1
src/transformers/models/llama/configuration_llama.py
src/transformers/models/llama/configuration_llama.py
+2
-2
No files found.
docs/source/en/model_doc/llama.mdx
View file @
00934026
...
...
@@ -33,8 +33,10 @@ python src/transformers/models/llama/convert_llama_weights_to_hf.py \
-
After
conversion
,
the
model
and
tokenizer
can
be
loaded
via
:
```
python
tokenizer
=
transformers
.
LlamaTokenizer
.
from_pretrained
(
"/output/path/tokenizer/"
)
model
=
transformers
.
LlamaForCausalLM
.
from_pretrained
(
"/output/path/llama-7b/"
)
from
transformers
import
LlamaForCausalLM
,
LlamaTokenizer
tokenizer
=
LlamaTokenizer
.
from_pretrained
(
"/output/path/tokenizer/"
)
model
=
LlamaForCausalLM
.
from_pretrained
(
"/output/path/llama-7b/"
)
```
-
The
LLaMA
tokenizer
is
based
on
[
sentencepiece
](
https
://
github
.
com
/
google
/
sentencepiece
).
One
quirk
of
sentencepiece
is
that
when
decoding
a
sequence
,
if
the
first
token
is
the
start
of
the
word
(
e
.
g
.
"Banana"
),
the
tokenizer
does
not
prepend
the
prefix
space
to
the
string
.
To
have
the
tokenizer
output
the
prefix
space
,
set
`
decode_with_prefix_space
=
True
`
in
the
`
LlamaTokenizer
`
object
or
in
the
tokenizer
configuration
.
...
...
src/transformers/__init__.py
View file @
00934026
...
...
@@ -4486,9 +4486,9 @@ if TYPE_CHECKING:
TypicalLogitsWarper
,
top_k_top_p_filtering
,
)
from
.modeling_utils
import
PreTrainedModel
# PyTorch model imports
from
.modeling_utils
import
PreTrainedModel
from
.models.albert
import
(
ALBERT_PRETRAINED_MODEL_ARCHIVE_LIST
,
AlbertForMaskedLM
,
...
...
src/transformers/models/llama/configuration_llama.py
View file @
00934026
...
...
@@ -30,7 +30,7 @@ LLAMA_PRETRAINED_CONFIG_ARCHIVE_MAP = {}
class
LlamaConfig
(
PretrainedConfig
):
r
"""
This is the configuration class to store the configuration of a [`
~
LlamaModel`]. It is used to instantiate an LLaMA
This is the configuration class to store the configuration of a [`LlamaModel`]. It is used to instantiate an LLaMA
model according to the specified arguments, defining the model architecture. Instantiating a configuration with the
defaults will yield a similar configuration to that of the LLaMA-7B.
...
...
@@ -41,7 +41,7 @@ class LlamaConfig(PretrainedConfig):
Args:
vocab_size (`int`, *optional*, defaults to 32000):
Vocabulary size of the LLaMA model. Defines the number of different tokens that can be represented by the
`inputs_ids` passed when calling [`
~
LlamaModel`]
`inputs_ids` passed when calling [`LlamaModel`]
hidden_size (`int`, *optional*, defaults to 4096):
Dimension of the hidden representations.
intermediate_size (`int`, *optional*, defaults to 11008):
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment