Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
447840c2
Commit
447840c2
authored
Apr 10, 2025
by
Cole
Committed by
Harrison King Saturley-Hall
Apr 10, 2025
Browse files
docs: add docstring for llm.rs (#267)
parent
da38e96a
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
23 additions
and
0 deletions
+23
-0
lib/bindings/python/rust/llm.rs
lib/bindings/python/rust/llm.rs
+23
-0
No files found.
lib/bindings/python/rust/llm.rs
View file @
447840c2
...
@@ -13,6 +13,29 @@
...
@@ -13,6 +13,29 @@
// See the License for the specific language governing permissions and
// See the License for the specific language governing permissions and
// limitations under the License.
// limitations under the License.
/// This module provides a high-performance interface that bridges Python
/// applications with the Rust-powered Dynamo LLM runtime.
///
/// It is organized into several specialized sub-modules, each responsible for a particular aspect of the system:
///
/// - `backend`:
/// Wraps low-level interfaces for LLM inference, manages resource allocation,
/// and integrates with specialized hardware for optimized execution.
/// - `disagg_route`:
/// Implements distributed routing of inference requests with dynamic
/// load balancing and efficient resource allocation across clusters.
/// - `kv`:
/// Implements a high-performance key-value caching system that stores
/// intermediate computations and maintains model state for rapid data access.
/// - `model_card`:
/// Manages model deployment cards containing detailed metadata, configuration
/// settings, and versioning information to ensure consistent deployments.
/// - `preprocessor`:
/// Provides utilities for transforming raw LLM requests—including tokenization,
/// prompt formatting, and validation—into a format required by the Dynamo runtime.
///
/// Each sub-module is designed to encapsulate its functionality for clean
/// integration between Python tools and the Dynamo runtime.
use
super
::
*
;
use
super
::
*
;
pub
mod
backend
;
pub
mod
backend
;
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment