Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
text-generation-inference
Commits
d69a0633
"docs/vscode:/vscode.git/clone" did not exist on "c19ae1c1769ef9d5ade592dd3feb389ad09aea7e"
Unverified
Commit
d69a0633
authored
Jun 01, 2023
by
OlivierDehaene
Committed by
GitHub
Jun 01, 2023
Browse files
fix(server): fix has_position_ids (#395)
Fix #389
parent
db2ebe39
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
7 additions
and
5 deletions
+7
-5
server/text_generation_server/models/causal_lm.py
server/text_generation_server/models/causal_lm.py
+0
-5
server/text_generation_server/models/model.py
server/text_generation_server/models/model.py
+7
-0
No files found.
server/text_generation_server/models/causal_lm.py
View file @
d69a0633
...
...
@@ -496,11 +496,6 @@ class CausalLM(Model):
else
:
tokenizer
.
add_special_tokens
({
"pad_token"
:
"[PAD]"
})
self
.
has_position_ids
=
(
inspect
.
signature
(
model
.
forward
).
parameters
.
get
(
"position_ids"
,
None
)
is
not
None
)
super
(
CausalLM
,
self
).
__init__
(
model
=
model
,
tokenizer
=
tokenizer
,
...
...
server/text_generation_server/models/model.py
View file @
d69a0633
import
inspect
import
torch
from
abc
import
ABC
,
abstractmethod
...
...
@@ -29,6 +30,12 @@ class Model(ABC):
self
.
device
=
device
self
.
rank
=
rank
self
.
world_size
=
world_size
self
.
has_position_ids
=
(
inspect
.
signature
(
model
.
forward
).
parameters
.
get
(
"position_ids"
,
None
)
is
not
None
)
self
.
check_initialized
()
@
property
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment