Commit b9ce8dd0 authored by Meenakshi Sharma's avatar Meenakshi Sharma Committed by GitHub
Browse files

Update README.md (#1)

parent 5215c90d
...@@ -15,7 +15,7 @@ See the License for the specific language governing permissions and ...@@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
limitations under the License. limitations under the License.
--> -->
# Triton Distributed # Dynemo
<h4> A Datacenter Scale Distributed Inference Serving Framework </h4> <h4> A Datacenter Scale Distributed Inference Serving Framework </h4>
...@@ -23,9 +23,9 @@ limitations under the License. ...@@ -23,9 +23,9 @@ limitations under the License.
[![GitHub Release](https://img.shields.io/github/v/release/dynemo-ai/dynemo)](https://github.com/dynemo-ai/dynemo/releases/latest) [![GitHub Release](https://img.shields.io/github/v/release/dynemo-ai/dynemo)](https://github.com/dynemo-ai/dynemo/releases/latest)
Triton Distributed is a flexible, component based, data center scale Dynemo is a flexible, component based, data center scale
inference serving framework designed to leverage the strengths of the inference serving framework designed to leverage the strengths of the
standalone Triton Inference Server while expanding its capabilities standalone Dynemo Inference Server while expanding its capabilities
to meet the demands of complex use cases including those of Generative to meet the demands of complex use cases including those of Generative
AI. It is designed to enable developers to implement and customize AI. It is designed to enable developers to implement and customize
routing, load balancing, scaling and workflow definitions at the data routing, load balancing, scaling and workflow definitions at the data
...@@ -36,17 +36,17 @@ center scale without sacrificing performance or ease of use. ...@@ -36,17 +36,17 @@ center scale without sacrificing performance or ease of use.
> rapid-prototyping stage and we are actively looking for feedback and > rapid-prototyping stage and we are actively looking for feedback and
> collaborators. > collaborators.
## Building Triton Distributed ## Building Dynemo
### Requirements ### Requirements
Triton Distributed development and examples are container based. Dynemo development and examples are container based.
* [Docker](https://docs.docker.com/get-started/get-docker/) * [Docker](https://docs.docker.com/get-started/get-docker/)
* [buildx](https://github.com/docker/buildx) * [buildx](https://github.com/docker/buildx)
### Development ### Development
You can build the Triton Distributed container using the build scripts You can build the Dynemo container using the build scripts
in `container/` (or directly with `docker build`). in `container/` (or directly with `docker build`).
We provide 3 types of builds: We provide 3 types of builds:
...@@ -61,9 +61,9 @@ For example, if you want to build a container for the `STANDARD` backends you ca ...@@ -61,9 +61,9 @@ For example, if you want to build a container for the `STANDARD` backends you ca
Please see the instructions in the corresponding example for specific build instructions. Please see the instructions in the corresponding example for specific build instructions.
## Running Triton Distributed for Local Testing and Development ## Running Dynemo for Local Testing and Development
You can run the Triton Distributed container using the run scripts in You can run the Dynemo container using the run scripts in
`container/` (or directly with `docker run`). `container/` (or directly with `docker run`).
The run script offers a few common workflows: The run script offers a few common workflows:
...@@ -92,9 +92,9 @@ HF_TOKEN```) and mounts common directories such as ```/tmp:/tmp```, ...@@ -92,9 +92,9 @@ HF_TOKEN```) and mounts common directories such as ```/tmp:/tmp```,
Please see the instructions in the corresponding example for specific Please see the instructions in the corresponding example for specific
deployment instructions. deployment instructions.
## Rust Based Distributed Runtime ## Rust Based Runtime
Triton distributed has a new rust based distributed runtime with Dynemo has a new rust based distributed runtime with
implementation under development. The rust based runtime enables implementation under development. The rust based runtime enables
serving arbitrary python code as well as native rust. Please note the serving arbitrary python code as well as native rust. Please note the
APIs are subject to change. APIs are subject to change.
...@@ -113,7 +113,7 @@ bindings. ...@@ -113,7 +113,7 @@ bindings.
An intermediate example expanding further on the concepts introduced An intermediate example expanding further on the concepts introduced
in the Hello World example. In this example, we demonstrate in the Hello World example. In this example, we demonstrate
[Disaggregated Serving](https://arxiv.org/abs/2401.09670) as an [Disaggregated Serving](https://arxiv.org/abs/2401.09670) as an
application of the components defined in Triton Distributed. application of the components defined in Dynemo.
# Disclaimers # Disclaimers
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment