• Michael Yang's avatar
    refactor layer creation · 70a93057
    Michael Yang authored
    previous layer creation was not ideal because:
    
    1. it required reading the input file multiple times, once to calculate
       the sha256 checksum, another to write it to disk, and potentially one
       more to decode the underlying gguf
    2. used io.ReadSeeker which is prone to user error. if the file isn't
       reset correctly or in the right place, it could end up reading an
       empty file
    
    there are also some brittleness when reading existing layers else
    writing the inherited layers will error reading an already closed file
    
    this commit aims to fix these issues by restructuring layer creation.
    
    1. it will now write the layer to a temporary file as well as the hash
       function and move it to the final location on Commit
    2. layers are read once once when copied to the destination. exception
       is raw model files which still requires a second read to decode the
       model metadata
    70a93057
layers.go 1.92 KB