Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
281977bd
Unverified
Commit
281977bd
authored
Jul 26, 2024
by
Michael Goin
Committed by
GitHub
Jul 26, 2024
Browse files
[Doc] Add Nemotron to supported model docs (#6843)
parent
3bbb4936
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
5 additions
and
3 deletions
+5
-3
docs/source/models/supported_models.rst
docs/source/models/supported_models.rst
+4
-0
vllm/model_executor/layers/activation.py
vllm/model_executor/layers/activation.py
+1
-3
No files found.
docs/source/models/supported_models.rst
View file @
281977bd
...
...
@@ -113,6 +113,10 @@ Decoder-only Language Models
- MPT, MPT-Instruct, MPT-Chat, MPT-StoryWriter
- :code:`mosaicml/mpt-7b`, :code:`mosaicml/mpt-7b-storywriter`, :code:`mosaicml/mpt-30b`, etc.
-
* - :code:`NemotronForCausalLM`
- Nemotron-3, Nemotron-4, Minitron
- :code:`nvidia/Minitron-8B-Base`, :code:`mgoin/Nemotron-4-340B-Base-hf-FP8`, etc.
- ✅︎
* - :code:`OLMoForCausalLM`
- OLMo
- :code:`allenai/OLMo-1B-hf`, :code:`allenai/OLMo-7B-hf`, etc.
...
...
vllm/model_executor/layers/activation.py
View file @
281977bd
...
...
@@ -166,9 +166,7 @@ class ReLUSquaredActivation(CustomOp):
def
forward_native
(
self
,
x
:
torch
.
Tensor
)
->
torch
.
Tensor
:
"""PyTorch-native implementation equivalent to forward()."""
relu_applied
=
nn
.
functional
.
relu
(
x
)
squared
=
torch
.
square
(
relu_applied
)
return
squared
return
torch
.
square
(
F
.
relu
(
x
))
def
forward_cuda
(
self
,
x
:
torch
.
Tensor
)
->
torch
.
Tensor
:
return
self
.
forward_native
(
x
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment