"benchmarks/utils/genai.py" did not exist on "3c4adde596c6ad51906a0eb2d2f1fed588a317b2"
Unverified Commit 39d645e5 authored by Jonathan Tong's avatar Jonathan Tong Committed by GitHub
Browse files

docs: migrate Fern docs from fern/ into docs/ (#6206)


Signed-off-by: default avatarJont828 <jt572@cornell.edu>
parent d381e6ff
<!--
SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
# dynamo.nixl_connect.ReadOperation
An operation which transfers data from a remote worker to the local worker.
To create the operation, NIXL metadata ([RdmaMetadata](rdma_metadata.md)) from a remote worker's [`ReadableOperation`](readable_operation.md)
along with a matching set of local [`Descriptor`](descriptor.md) objects which reference memory intended to receive data from the remote worker must be provided.
The NIXL metadata must be transferred from the remote to the local worker via a secondary channel, most likely HTTP or TCP+NATS.
Once created, data transfer will begin immediately.
Disposal of the object will instruct the NIXL subsystem to cancel the operation,
therefore the operation should be awaited until completed unless cancellation is intended.
## Example Usage
```python
async def read_from_remote(
self,
remote_metadata: dynamo.nixl_connect.RdmaMetadata,
local_tensor: torch.Tensor
) -> None:
descriptor = dynamo.nixl_connect.Descriptor(local_tensor)
with await self.connector.begin_read(remote_metadata, descriptor) as read_op:
# Wait for the operation to complete writing data from the remote worker to local_tensor.
await read_op.wait_for_completion()
```
## Methods
### `cancel`
```python
def cancel(self) -> None:
```
Instructs the NIXL subsystem to cancel the operation.
Completed operations cannot be cancelled.
### `wait_for_completion`
```python
async def wait_for_completion(self) -> None:
```
Blocks the caller until the memory from the remote worker has been transferred to the provided buffers.
## Properties
### `status`
```python
@property
def status(self) -> OperationStatus:
```
Returns [`OperationStatus`](operation_status.md) which provides the current state (aka. status) of the operation.
## Related Classes
- [Connector](connector.md)
- [Descriptor](descriptor.md)
- [Device](device.md)
- [OperationStatus](operation_status.md)
- [RdmaMetadata](rdma_metadata.md)
- [ReadableOperation](readable_operation.md)
- [WritableOperation](writable_operation.md)
- [WriteOperation](write_operation.md)
<!--
SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
# dynamo.nixl_connect.ReadableOperation
An operation which enables a remote worker to read data from the local worker.
To create the operation, a set of local [`Descriptor`](descriptor.md) objects must be provided that reference memory intended to be transferred to a remote worker.
Once created, the memory referenced by the provided descriptors becomes immediately readable by a remote worker with the necessary metadata.
The NIXL metadata ([RdmaMetadata](rdma_metadata.md)) required to access the memory referenced by the provided descriptors is accessible via the operations `.metadata()` method.
Once acquired, the metadata needs to be provided to a remote worker via a secondary channel, most likely HTTP or TCP+NATS.
Disposal of the object will instruct the NIXL subsystem to cancel the operation,
therefore the operation should be awaited until completed unless cancellation is intended.
## Example Usage
```python
async def send_data(
self,
local_tensor: torch.Tensor
) -> None:
descriptor = dynamo.nixl_connect.Descriptor(local_tensor)
with await self.connector.create_readable(descriptor) as read_op:
op_metadata = read_op.metadata()
# Send the metadata to the remote worker via sideband communication.
await self.notify_remote_data(op_metadata)
# Wait for the remote worker to complete its read operation of local_tensor.
# AKA send data to remote worker.
await read_op.wait_for_completion()
```
## Methods
### `metadata`
```python
def metadata(self) -> RdmaMetadata:
```
Generates and returns the NIXL metadata ([RdmaMetadata](rdma_metadata.md)) required for a remote worker to read from the operation.
Once acquired, the metadata needs to be provided to a remote worker via a secondary channel, most likely HTTP or TCP+NATS.
### `wait_for_completion`
```python
async def wait_for_completion(self) -> None:
```
Blocks the caller until the operation has received a completion signal from a remote worker.
## Properties
### `status`
```python
@property
def status(self) -> OperationStatus:
```
Returns [`OperationStatus`](operation_status.md) which provides the current state (aka. status) of the operation.
## Related Classes
- [Connector](connector.md)
- [Descriptor](descriptor.md)
- [Device](device.md)
- [OperationStatus](operation_status.md)
- [RdmaMetadata](rdma_metadata.md)
- [ReadOperation](read_operation.md)
- [WritableOperation](writable_operation.md)
- [WriteOperation](write_operation.md)
<!--
SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
# dynamo.nixl_connect.WritableOperation
An operation which enables a remote worker to write data to the local worker.
To create the operation, a set of local [`Descriptor`](descriptor.md) objects must be provided which reference memory intended to receive data from a remote worker.
Once created, the memory referenced by the provided descriptors becomes immediately writable by a remote worker with the necessary metadata.
The NIXL metadata ([RdmaMetadata](rdma_metadata.md)) required to access the memory referenced by the provided descriptors is accessible via the operations `.metadata()` method.
Once acquired, the metadata needs to be provided to a remote worker via a secondary channel, most likely HTTP or TCP+NATS.
Disposal of the object will instruct the NIXL subsystem to cancel the operation,
therefore the operation should be awaited until completed unless cancellation is intended.
Cancellation is handled asynchronously.
## Example Usage
```python
async def recv_data(
self,
local_tensor: torch.Tensor
) -> None:
descriptor = dynamo.nixl_connect.Descriptor(local_tensor)
with await self.connector.create_writable(descriptor) as write_op:
op_metadata = write_op.metadata()
# Send the metadata to the remote worker via sideband communication.
await self.request_remote_data(op_metadata)
# Wait the remote worker to complete its write operation to local_tensor.
# AKA receive data from remote worker.
await write_op.wait_for_completion()
```
## Methods
### `metadata`
```python
def metadata(self) -> RdmaMetadata:
```
Generates and returns the NIXL metadata ([RdmaMetadata](rdma_metadata.md)) required for a remote worker to write to the operation.
Once acquired, the metadata needs to be provided to a remote worker via a secondary channel, most likely HTTP or TCP+NATS.
### `wait_for_completion`
```python
async def wait_for_completion(self) -> None:
```
Blocks the caller until the operation has received a completion signal from a remote worker.
## Properties
### `status`
```python
@property
def status(self) -> OperationStatus:
```
Returns [`OperationStatus`](operation_status.md) which provides the current state (aka. status) of the operation.
## Related Classes
- [Connector](connector.md)
- [Descriptor](descriptor.md)
- [Device](device.md)
- [OperationStatus](operation_status.md)
- [RdmaMetadata](rdma_metadata.md)
- [ReadOperation](read_operation.md)
- [ReadableOperation](readable_operation.md)
- [WriteOperation](write_operation.md)
<!--
SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
# dynamo.nixl_connect.WriteOperation
An operation which transfers data from the local worker to a remote worker.
To create the operation, NIXL metadata ([RdmaMetadata](rdma_metadata.md)) from a remote worker's [`WritableOperation`](writable_operation.md)
along with a matching set of local [`Descriptor`](descriptor.md) objects which reference memory to be transferred to the remote worker must be provided.
The NIXL metadata must be transferred from the remote to the local worker via a secondary channel, most likely HTTP or TCP+NATS.
Once created, data transfer will begin immediately.
Disposal of the object will instruct the NIXL subsystem to cancel the operation,
therefore the operation should be awaited until completed unless cancellation is intended.
Cancellation is handled asynchronously.
## Example Usage
```python
async def write_to_remote(
self,
remote_metadata: dynamo.nixl_connect.RdmaMetadata,
local_tensor: torch.Tensor
) -> None:
descriptor = dynamo.nixl_connect.Descriptor(local_tensor)
with await self.connector.begin_write(descriptor, remote_metadata) as write_op:
# Wait for the operation to complete writing local_tensor to the remote worker.
await write_op.wait_for_completion()
```
## Methods
### `cancel`
```python
def cancel(self) -> None:
```
Instructs the NIXL subsystem to cancel the operation.
Completed operations cannot be cancelled.
### `wait_for_completion`
```python
async def wait_for_completion(self) -> None:
```
Blocks the caller until all provided buffers have been transferred to the remote worker.
## Properties
### `status`
```python
@property
def status(self) -> OperationStatus:
```
Returns [`OperationStatus`](operation_status.md) which provides the current state (aka. status) of the operation.
## Related Classes
- [Connector](connector.md)
- [Descriptor](descriptor.md)
- [Device](device.md)
- [OperationStatus](operation_status.md)
- [RdmaMetadata](rdma_metadata.md)
- [ReadOperation](read_operation.md)
- [ReadableOperation](readable_operation.md)
- [WritableOperation](writable_operation.md)
File suppressed by a .gitattributes entry or the file's encoding is unsupported.
File suppressed by a .gitattributes entry or the file's encoding is unsupported.
File suppressed by a .gitattributes entry or the file's encoding is unsupported.
File suppressed by a .gitattributes entry or the file's encoding is unsupported.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment