• Jesse Gross's avatar
    model: Pass input tensor instead of raw data to models · 0fbfcf3c
    Jesse Gross authored
    Rather than directly giving the input data to models, we can
    pass a tensor instead. In the short term, this saves some duplicated
    code.
    
    Longer term, we will want to overlap setting up the next batch with
    processing of the current one. In this case, we will only have the
    shape of tensor but it will not be loaded with data at the time of
    graph generation. By passing only a tensor to models now, we set up
    this possibility and prevent them from relying on data that they won't
    have in the future.
    
    Although the same could be done for Positions and Outputs, in some
    cases we either need the raw input data or don't use them at all.
    Therefore, for now we leave them as they are and allow models to
    convert them to tensors as needed.
    0fbfcf3c
model.go 4.34 KB