Skip to content

GitLab

"components/backends/trtllm/vscode:/vscode.git/clone" did not exist on "b74b887bad92eb3e31891dffd805b61d9fcdec63"
Unverified Commit cd2389ba authored by Harrison Saturley-Hall's avatar Harrison Saturley-Hall Committed by GitHub
Browse files

chore: pre-0.6.0 activities (#3592) 


Signed-off-by: default avatarHarrison Saturley-Hall <hsaturleyhal@nvidia.com>
parent fa4ce46c
Expand all Show whitespace changes
Inline Side-by-side
Showing with 60230 additions and 36690 deletions
ATTRIBUTIONS-Python.md
View file @ cd2389ba
This diff is collapsed.
ATTRIBUTIONS-Rust.md
View file @ cd2389ba
This diff is collapsed.
ATTRIBUTIONS-container-sglang-runtime.txt
View file @ cd2389ba
......@@ -119282,3 +119282,44 @@ License: Zlib
 
************************************
 
ninja-build
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: ninja
Upstream-Contact: martine@danga.com
Source: https://ninja-build.org/
Files: *
Copyright: 2011-2014 Google
License: Apache-2.0
Files: src/getopt.*
Copyright: 1997 Gregory Pietsch
License: other
This file and the accompanying getopt.h header file are hereby placed in the
public domain without restrictions. Just give the author credit, don't
claim you wrote it or prevent anyone else from using it.
.
Gregory Pietsch's current e-mail address:
gpietsch@comcast.net
Files: debian/*
Copyright: 2012 Gary Kramlich
2016 Felix Geyer
License: Apache-2.0
License: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
.
http://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.
On Debian systems, the complete text of the Apache version 2.0 license
can be found in "/usr/share/common-licenses/Apache-2.0".
\ No newline at end of file
ATTRIBUTIONS-container-tensorrtllm-runtime.txt
View file @ cd2389ba
......@@ -67784,3 +67784,80 @@ ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDI
************************************
wget
This package was debianized by
Christian Schwarz <schwarz@monet.m.isar.de> on Mon, 18 Nov 1996 00:59:57 +0100
J. Ramos Goncalves <ramos@debian.org> on Thu, 13 Feb 1997 23:15:18 +0000
Nicolás Lichtmaier <nick@debian.org> on Sat, 18 Oct 1997 21:23:12 -0300
Noël Köthe <noel@debian.org> on Mon, 18 Feb 2002 09:53:00 +0100
It was downloaded from ftp://ftp.gnu.org/gnu/wget/
Homepage: http://www.gnu.org/directory/wget.html
http://www.gnu.org/software/wget/wget.html
Upstream Author: Giuseppe Scrivano <gscrivano@gnu.org>
Copyright: (C) 2007 Free Software Foundation, Inc.
Released under the terms of the GPL; see
/usr/share/common-licenses/GPL-3.
"In addition, as a special exception, the Free Software Foundation
gives permission to link the code of its release of Wget with the
OpenSSL project's "OpenSSL" library (or with modified versions of it
that use the same license as the "OpenSSL" library), and distribute
the linked executables. You must obey the GNU General Public License
in all respects for all of the code used other than "OpenSSL". If you
modify this file, you may extend this exception to your version of the
file, but you are not obligated to do so. If you do not wish to do
so, delete this exception statement from your version."
The wget(1) manpage and the wget info page are distributed under the
terms of the GNU Free Documentation License ; see
/usr/share/common-licenses/GFDL-1.2
************************************
ninja-build
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: ninja
Upstream-Contact: martine@danga.com
Source: https://ninja-build.org/
Files: *
Copyright: 2011-2014 Google
License: Apache-2.0
Files: src/getopt.*
Copyright: 1997 Gregory Pietsch
License: other
This file and the accompanying getopt.h header file are hereby placed in the
public domain without restrictions. Just give the author credit, don't
claim you wrote it or prevent anyone else from using it.
.
Gregory Pietsch's current e-mail address:
gpietsch@comcast.net
Files: debian/*
Copyright: 2012 Gary Kramlich
2016 Felix Geyer
License: Apache-2.0
License: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
.
http://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.
On Debian systems, the complete text of the Apache version 2.0 license
can be found in "/usr/share/common-licenses/Apache-2.0".
\ No newline at end of file
Cargo.lock
View file @ cd2389ba
......@@ -2065,7 +2065,7 @@ dependencies = [
[[package]]
name = "dynamo-async-openai"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"async-openai-macros",
"backoff",
......@@ -2103,7 +2103,7 @@ dependencies = [
[[package]]
name = "dynamo-engine-mistralrs"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"anyhow",
"async-stream",
......@@ -2121,7 +2121,7 @@ dependencies = [
[[package]]
name = "dynamo-llm"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"ahash",
"aho-corasick",
......@@ -2215,7 +2215,7 @@ dependencies = [
[[package]]
name = "dynamo-parsers"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"anyhow",
"dynamo-async-openai",
......@@ -2234,7 +2234,7 @@ dependencies = [
[[package]]
name = "dynamo-run"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"anyhow",
"async-stream",
......@@ -2262,7 +2262,7 @@ dependencies = [
[[package]]
name = "dynamo-runtime"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"anyhow",
"arc-swap",
......@@ -2327,7 +2327,7 @@ dependencies = [
[[package]]
name = "dynamo-tokens"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"bytemuck",
"derive-getters",
......@@ -4192,7 +4192,7 @@ checksum = "58f929b4d672ea937a23a1ab494143d968337a5f47e56d0815df1e0890ddf174"
[[package]]
name = "libdynamo_llm"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"anyhow",
"async-once-cell",
......@@ -4877,8 +4877,9 @@ dependencies = [
[[package]]
name = "modelexpress-client"
version = "0.1.0"
source = "git+https://github.com/ai-dynamo/modelexpress.git?rev=a232220bf268a475d293914d407f4ae186f443e3#a232220bf268a475d293914d407f4ae186f443e3"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9462ec6cd8b3d6beaa262ad0907a8ba297c7e3a220c11e4159742d8c275588eb"
dependencies = [
"anyhow",
"clap 4.5.48",
......@@ -4898,8 +4899,9 @@ dependencies = [
[[package]]
name = "modelexpress-common"
version = "0.1.0"
source = "git+https://github.com/ai-dynamo/modelexpress.git?rev=a232220bf268a475d293914d407f4ae186f443e3#a232220bf268a475d293914d407f4ae186f443e3"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4cd921793901b2a39105accab89a953d96661cc0eebfecfda307bec266edafbb"
dependencies = [
"anyhow",
"async-trait",
......
Cargo.toml
View file @ cd2389ba
......@@ -28,7 +28,7 @@ default-members = [
resolver = "3"
[workspace.package]
version = "0.5.1"
version = "0.6.0"
edition = "2024"
description = "Dynamo Inference Framework"
authors = ["NVIDIA Inc. <sw-dl-dynamo@nvidia.com>"]
......@@ -39,15 +39,14 @@ keywords = ["llm", "genai", "inference", "nvidia", "distributed"]
[workspace.dependencies]
# Local crates
dynamo-runtime = { path = "lib/runtime", version = "0.5.1" }
dynamo-llm = { path = "lib/llm", version = "0.5.1" }
dynamo-tokens = { path = "lib/tokens", version = "0.5.1" }
dynamo-async-openai = { path = "lib/async-openai", version = "0.5.1", features = [
dynamo-runtime = { path = "lib/runtime", version = "0.6.0" }
dynamo-llm = { path = "lib/llm", version = "0.6.0" }
dynamo-tokens = { path = "lib/tokens", version = "0.6.0" }
dynamo-async-openai = { path = "lib/async-openai", version = "0.6.0", features = [
"byot",
"rustls",
] }
dynamo-parsers = { path = "lib/parsers", version = "0.5.1" }
dynamo-parsers = { path = "lib/parsers", version = "0.6.0" }
# External dependencies
anyhow = { version = "1" }
......@@ -76,8 +75,8 @@ hf-hub = { version = "0.4.2", default-features = false, features = [
] }
# ModelExpress for model downloading
modelexpress-client = { git = "https://github.com/ai-dynamo/modelexpress.git", rev = "a232220bf268a475d293914d407f4ae186f443e3", package = "modelexpress-client" }
modelexpress-common = { git = "https://github.com/ai-dynamo/modelexpress.git", rev = "a232220bf268a475d293914d407f4ae186f443e3", package = "modelexpress-common" }
modelexpress-client = { version = "0.2.0" }
modelexpress-common = { version = "0.2.0" }
humantime = { version = "2.2.0" }
libc = { version = "0.2" }
......
deploy/cloud/helm/crds/Chart.yaml
View file @ cd2389ba
......@@ -16,5 +16,5 @@ apiVersion: v2
name: dynamo-crds
description: A Helm chart for dynamo CRDs
type: application
version: 0.5.0
version: 0.6.0
dependencies: []
\ No newline at end of file
deploy/cloud/helm/platform/Chart.yaml
View file @ cd2389ba
......@@ -19,7 +19,7 @@ maintainers:
url: https://www.nvidia.com
description: A Helm chart for NVIDIA Dynamo Platform.
type: application
version: 0.5.0
version: 0.6.0
home: https://nvidia.com
dependencies:
- name: dynamo-operator
......
deploy/helm/chart/Chart.yaml
View file @ cd2389ba
......@@ -17,5 +17,5 @@ apiVersion: v2
name: dynamo-graph
description: A Helm chart to deploy a Dynamo graph on Kubernetes
type: application
version: 0.5.0
appVersion: 0.5.0
version: 0.6.0
appVersion: 0.6.0
\ No newline at end of file
docs/support_matrix.md
View file @ cd2389ba
......@@ -56,8 +56,8 @@ If you are using a **GPU**, the following GPU models and architectures are suppo
| **Python Package** | **Version** | glibc version | CUDA Version |
| :----------------- | :---------- | :------------------------------------ | :----------- |
| ai-dynamo | 0.5.1 | >=2.28 | |
| ai-dynamo-runtime | 0.5.1 | >=2.28 (Python 3.12 has known issues) | |
| ai-dynamo | 0.6.0 | >=2.28 | |
| ai-dynamo-runtime | 0.6.0 | >=2.28 (Python 3.12 has known issues) | |
| NIXL | 0.6.0 | >=2.27 | >=11.8 |
### Build Dependency
......
lib/bindings/python/Cargo.lock
View file @ cd2389ba
......@@ -1428,7 +1428,7 @@ dependencies = [
[[package]]
name = "dynamo-async-openai"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"async-openai-macros",
"backoff",
......@@ -1454,7 +1454,7 @@ dependencies = [
[[package]]
name = "dynamo-llm"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"ahash",
"aho-corasick",
......@@ -1536,7 +1536,7 @@ dependencies = [
[[package]]
name = "dynamo-parsers"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"anyhow",
"dynamo-async-openai",
......@@ -1554,7 +1554,7 @@ dependencies = [
[[package]]
name = "dynamo-py3"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"anyhow",
"async-stream",
......@@ -1591,7 +1591,7 @@ dependencies = [
[[package]]
name = "dynamo-runtime"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"anyhow",
"arc-swap",
......@@ -3478,8 +3478,9 @@ dependencies = [
[[package]]
name = "modelexpress-client"
version = "0.1.0"
source = "git+https://github.com/ai-dynamo/modelexpress.git?rev=a232220bf268a475d293914d407f4ae186f443e3#a232220bf268a475d293914d407f4ae186f443e3"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9462ec6cd8b3d6beaa262ad0907a8ba297c7e3a220c11e4159742d8c275588eb"
dependencies = [
"anyhow",
"clap",
......@@ -3499,8 +3500,9 @@ dependencies = [
[[package]]
name = "modelexpress-common"
version = "0.1.0"
source = "git+https://github.com/ai-dynamo/modelexpress.git?rev=a232220bf268a475d293914d407f4ae186f443e3#a232220bf268a475d293914d407f4ae186f443e3"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4cd921793901b2a39105accab89a953d96661cc0eebfecfda307bec266edafbb"
dependencies = [
"anyhow",
"async-trait",
......
lib/bindings/python/Cargo.toml
View file @ cd2389ba
......@@ -7,7 +7,7 @@
[package]
name = "dynamo-py3"
version = "0.5.1"
version = "0.6.0"
edition = "2024"
authors = ["NVIDIA"]
license = "Apache-2.0"
......
lib/bindings/python/pyproject.toml
View file @ cd2389ba
......@@ -16,7 +16,7 @@
[project]
name = "ai-dynamo-runtime"
version = "0.5.1"
version = "0.6.0"
description = "Dynamo Inference Framework Runtime"
readme = "README.md"
authors = [
......
lib/runtime/examples/Cargo.lock
View file @ cd2389ba
......@@ -657,7 +657,7 @@ dependencies = [
[[package]]
name = "dynamo-runtime"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"anyhow",
"arc-swap",
......@@ -685,6 +685,9 @@ dependencies = [
"nix",
"nuid",
"once_cell",
"opentelemetry",
"opentelemetry-otlp",
"opentelemetry_sdk",
"prometheus",
"rand 0.9.1",
"rayon",
......@@ -699,6 +702,7 @@ dependencies = [
"tokio-util",
"tower-http",
"tracing",
"tracing-opentelemetry",
"tracing-subscriber",
"url",
"uuid",
......@@ -792,10 +796,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88365f1a5671eb2f7fc240adb216786bc6494b38ce15f1d26ad6eaa303d5e822"
dependencies = [
"http",
"prost",
"prost 0.13.5",
"tokio",
"tokio-stream",
"tonic",
"tonic 0.13.1",
"tonic-build",
"tower",
"tower-service",
......@@ -1032,7 +1036,7 @@ checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "hello_world"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"dynamo-runtime",
]
......@@ -1709,6 +1713,82 @@ version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d05e27ee213611ffe7d6348b942e8f942b37114c00cc03cec254295a4a17852e"
[[package]]
name = "opentelemetry"
version = "0.31.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b84bcd6ae87133e903af7ef497404dda70c60d0ea14895fc8a5e6722754fc2a0"
dependencies = [
"futures-core",
"futures-sink",
"js-sys",
"pin-project-lite",
"thiserror 2.0.12",
"tracing",
]
[[package]]
name = "opentelemetry-http"
version = "0.31.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7a6d09a73194e6b66df7c8f1b680f156d916a1a942abf2de06823dd02b7855d"
dependencies = [
"async-trait",
"bytes",
"http",
"opentelemetry",
"reqwest",
]
[[package]]
name = "opentelemetry-otlp"
version = "0.31.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a2366db2dca4d2ad033cad11e6ee42844fd727007af5ad04a1730f4cb8163bf"
dependencies = [
"http",
"opentelemetry",
"opentelemetry-http",
"opentelemetry-proto",
"opentelemetry_sdk",
"prost 0.14.1",
"reqwest",
"thiserror 2.0.12",
"tokio",
"tonic 0.14.2",
"tracing",
]
[[package]]
name = "opentelemetry-proto"
version = "0.31.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a7175df06de5eaee9909d4805a3d07e28bb752c34cab57fa9cff549da596b30f"
dependencies = [
"opentelemetry",
"opentelemetry_sdk",
"prost 0.14.1",
"tonic 0.14.2",
"tonic-prost",
]
[[package]]
name = "opentelemetry_sdk"
version = "0.31.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e14ae4f5991976fd48df6d843de219ca6d31b01daaab2dad5af2badeded372bd"
dependencies = [
"futures-channel",
"futures-executor",
"futures-util",
"opentelemetry",
"percent-encoding",
"rand 0.9.1",
"thiserror 2.0.12",
"tokio",
"tokio-stream",
]
[[package]]
name = "overload"
version = "0.1.1"
......@@ -1931,7 +2011,17 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2796faa41db3ec313a31f7624d9286acf277b52de526150b7e69f3debf891ee5"
dependencies = [
"bytes",
"prost-derive",
"prost-derive 0.13.5",
]
[[package]]
name = "prost"
version = "0.14.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7231bd9b3d3d33c86b58adbac74b5ec0ad9f496b19d22801d773636feaa95f3d"
dependencies = [
"bytes",
"prost-derive 0.14.1",
]
[[package]]
......@@ -1947,7 +2037,7 @@ dependencies = [
"once_cell",
"petgraph",
"prettyplease",
"prost",
"prost 0.13.5",
"prost-types",
"regex",
"syn 2.0.100",
......@@ -1967,13 +2057,26 @@ dependencies = [
"syn 2.0.100",
]
[[package]]
name = "prost-derive"
version = "0.14.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9120690fafc389a67ba3803df527d0ec9cbbc9cc45e4cc20b332996dfb672425"
dependencies = [
"anyhow",
"itertools",
"proc-macro2",
"quote",
"syn 2.0.100",
]
[[package]]
name = "prost-types"
version = "0.13.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52c2c1bf36ddb1a1c396b3601a3cec27c2462e45f07c386894ec3ccf5332bd16"
dependencies = [
"prost",
"prost 0.13.5",
]
[[package]]
......@@ -2206,6 +2309,7 @@ checksum = "cbc931937e6ca3a06e3b6c0aa7841849b160a90351d6ab467a8b9b9959767531"
dependencies = [
"base64",
"bytes",
"futures-channel",
"futures-core",
"futures-util",
"http",
......@@ -2504,7 +2608,7 @@ dependencies = [
[[package]]
name = "service_metrics"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"dynamo-runtime",
"futures",
......@@ -2680,7 +2784,7 @@ dependencies = [
[[package]]
name = "system_metrics"
version = "0.5.1"
version = "0.6.0"
dependencies = [
"anyhow",
"dynamo-runtime",
......@@ -2990,7 +3094,7 @@ dependencies = [
"hyper-util",
"percent-encoding",
"pin-project",
"prost",
"prost 0.13.5",
"socket2",
"tokio",
"tokio-rustls",
......@@ -3001,6 +3105,32 @@ dependencies = [
"tracing",
]
[[package]]
name = "tonic"
version = "0.14.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eb7613188ce9f7df5bfe185db26c5814347d110db17920415cf2fbcad85e7203"
dependencies = [
"async-trait",
"base64",
"bytes",
"http",
"http-body",
"http-body-util",
"hyper",
"hyper-timeout",
"hyper-util",
"percent-encoding",
"pin-project",
"sync_wrapper",
"tokio",
"tokio-stream",
"tower",
"tower-layer",
"tower-service",
"tracing",
]
[[package]]
name = "tonic-build"
version = "0.13.1"
......@@ -3015,6 +3145,17 @@ dependencies = [
"syn 2.0.100",
]
[[package]]
name = "tonic-prost"
version = "0.14.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "66bd50ad6ce1252d87ef024b3d64fe4c3cf54a86fb9ef4c631fdd0ded7aeaa67"
dependencies = [
"bytes",
"prost 0.14.1",
"tonic 0.14.2",
]
[[package]]
name = "tower"
version = "0.5.2"
......@@ -3109,6 +3250,25 @@ dependencies = [
"tracing-core",
]
[[package]]
name = "tracing-opentelemetry"
version = "0.32.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e6e5658463dd88089aba75c7791e1d3120633b1bfde22478b28f625a9bb1b8e"
dependencies = [
"js-sys",
"opentelemetry",
"opentelemetry_sdk",
"rustversion",
"smallvec",
"thiserror 2.0.12",
"tracing",
"tracing-core",
"tracing-log",
"tracing-subscriber",
"web-time",
]
[[package]]
name = "tracing-serde"
version = "0.2.0"
......
lib/runtime/examples/Cargo.toml
View file @ cd2389ba
......@@ -10,7 +10,7 @@ members = [
resolver = "3"
[workspace.package]
version = "0.5.1"
version = "0.6.0"
edition = "2024"
authors = ["NVIDIA"]
license = "Apache-2.0"
......
pyproject.toml
View file @ cd2389ba
......@@ -3,7 +3,7 @@
[project]
name = "ai-dynamo"
version = "0.5.1"
version = "0.6.0"
description = "Distributed Inference Framework"
readme = "README.md"
authors = [
......@@ -13,7 +13,7 @@ license = { text = "Apache-2.0" }
license-files = ["LICENSE"]
requires-python = ">=3.10"
dependencies = [
"ai-dynamo-runtime==0.5.1",
"ai-dynamo-runtime==0.6.0",
"pytest>=8.3.4",
"types-psutil>=7.0.0.20250218",
"kubernetes>=32.0.1,<33.0.0",
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment