• Jesse Gross's avatar
    kvcache: Remove special case for reservation mask · 1c093e97
    Jesse Gross authored
    We currently short circuit generation of the cache mask and just
    generate an empty tensor of the correct size. However, in some
    cases, this can also skip a cast operation. This can result in the
    worst case graph being not fully worst case.
    
    We don't actually need the fast path for mask generation, so it's
    better to just use the normal code path.
    1c093e97
causal.go 20.5 KB