Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
b9ce8dd0
Commit
b9ce8dd0
authored
Mar 04, 2025
by
Meenakshi Sharma
Committed by
GitHub
Mar 04, 2025
Browse files
Update README.md (#1)
parent
5215c90d
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
11 additions
and
11 deletions
+11
-11
README.md
README.md
+11
-11
No files found.
README.md
View file @
b9ce8dd0
...
@@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
...
@@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
limitations under the License.
limitations under the License.
-->
-->
#
Triton Distributed
#
Dynemo
<h4>
A Datacenter Scale Distributed Inference Serving Framework
</h4>
<h4>
A Datacenter Scale Distributed Inference Serving Framework
</h4>
...
@@ -23,9 +23,9 @@ limitations under the License.
...
@@ -23,9 +23,9 @@ limitations under the License.
[

](https://github.com/dynemo-ai/dynemo/releases/latest)
[

](https://github.com/dynemo-ai/dynemo/releases/latest)
Triton Distributed
is a flexible, component based, data center scale
Dynemo
is a flexible, component based, data center scale
inference serving framework designed to leverage the strengths of the
inference serving framework designed to leverage the strengths of the
standalone
Triton
Inference Server while expanding its capabilities
standalone
Dynemo
Inference Server while expanding its capabilities
to meet the demands of complex use cases including those of Generative
to meet the demands of complex use cases including those of Generative
AI. It is designed to enable developers to implement and customize
AI. It is designed to enable developers to implement and customize
routing, load balancing, scaling and workflow definitions at the data
routing, load balancing, scaling and workflow definitions at the data
...
@@ -36,17 +36,17 @@ center scale without sacrificing performance or ease of use.
...
@@ -36,17 +36,17 @@ center scale without sacrificing performance or ease of use.
> rapid-prototyping stage and we are actively looking for feedback and
> rapid-prototyping stage and we are actively looking for feedback and
> collaborators.
> collaborators.
## Building
Triton Distributed
## Building
Dynemo
### Requirements
### Requirements
Triton Distributed
development and examples are container based.
Dynemo
development and examples are container based.
*
[
Docker
](
https://docs.docker.com/get-started/get-docker/
)
*
[
Docker
](
https://docs.docker.com/get-started/get-docker/
)
*
[
buildx
](
https://github.com/docker/buildx
)
*
[
buildx
](
https://github.com/docker/buildx
)
### Development
### Development
You can build the
Triton Distributed
container using the build scripts
You can build the
Dynemo
container using the build scripts
in
`container/`
(or directly with
`docker build`
).
in
`container/`
(or directly with
`docker build`
).
We provide 3 types of builds:
We provide 3 types of builds:
...
@@ -61,9 +61,9 @@ For example, if you want to build a container for the `STANDARD` backends you ca
...
@@ -61,9 +61,9 @@ For example, if you want to build a container for the `STANDARD` backends you ca
Please see the instructions in the corresponding example for specific build instructions.
Please see the instructions in the corresponding example for specific build instructions.
## Running
Triton Distributed
for Local Testing and Development
## Running
Dynemo
for Local Testing and Development
You can run the
Triton Distributed
container using the run scripts in
You can run the
Dynemo
container using the run scripts in
`container/`
(or directly with
`docker run`
).
`container/`
(or directly with
`docker run`
).
The run script offers a few common workflows:
The run script offers a few common workflows:
...
@@ -92,9 +92,9 @@ HF_TOKEN```) and mounts common directories such as ```/tmp:/tmp```,
...
@@ -92,9 +92,9 @@ HF_TOKEN```) and mounts common directories such as ```/tmp:/tmp```,
Please see the instructions in the corresponding example for specific
Please see the instructions in the corresponding example for specific
deployment instructions.
deployment instructions.
## Rust Based
Distributed
Runtime
## Rust Based Runtime
Triton distributed
has a new rust based distributed runtime with
Dynemo
has a new rust based distributed runtime with
implementation under development. The rust based runtime enables
implementation under development. The rust based runtime enables
serving arbitrary python code as well as native rust. Please note the
serving arbitrary python code as well as native rust. Please note the
APIs are subject to change.
APIs are subject to change.
...
@@ -113,7 +113,7 @@ bindings.
...
@@ -113,7 +113,7 @@ bindings.
An intermediate example expanding further on the concepts introduced
An intermediate example expanding further on the concepts introduced
in the Hello World example. In this example, we demonstrate
in the Hello World example. In this example, we demonstrate
[Disaggregated Serving](https://arxiv.org/abs/2401.09670) as an
[Disaggregated Serving](https://arxiv.org/abs/2401.09670) as an
application of the components defined in
Triton Distributed
.
application of the components defined in
Dynemo
.
# Disclaimers
# Disclaimers
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment