"sgl-kernel/python/vscode:/vscode.git/clone" did not exist on "129d29927892163670a75b377c329b752ee37a0b"
  • Blake Mizerany's avatar
    server/internal/client: use chunksums for concurrent blob verification (#9746) · eb2b22b0
    Blake Mizerany authored
    Replace large-chunk blob downloads with parallel small-chunk
    verification to solve timeout and performance issues. Registry users
    experienced progressively slowing download speeds as large-chunk
    transfers aged, often timing out completely.
    
    The previous approach downloaded blobs in a few large chunks but
    required a separate, single-threaded pass to read the entire blob back
    from disk for verification after download completion.
    
    This change uses the new chunksums API to fetch many smaller
    chunk+digest pairs, allowing concurrent downloads and immediate
    verification as each chunk arrives. Chunks are written directly to their
    final positions, eliminating the entire separate verification pass.
    
    The result is more reliable downloads that maintain speed throughout the
    transfer process and significantly faster overall completion, especially
    over unstable connections or with large blobs.
    eb2b22b0
chunked.go 1.56 KB