• Tom Birch's avatar
    Single-process control via PipeRPCWrapper (#156) · 5d4f50fb
    Tom Birch authored
    Adds support for:
    * Reused layers (e.g. for weight sharing)
    * Lazily-constructed layers
    * Single-process control via PipeRPCWrapper
    * PipelineStyle.AsyncScheudle, which lays the foundation for asynchronous pipeline work by introducing an event loop for each rank/worker to process either activations or gradients as they arrive
    
    Also added examples for multi-process and PipeRPCWrapper
    5d4f50fb
tutorial_pipe_rpc.py 2.33 KB