• drbh's avatar
    fix: Handle concurrent grammar requests (#1610) · 343aa7a1
    drbh authored
    This PR fixes parallel grammar requests, currently grammar states are
    not concatenated correctly when a new request is added to the batch and
    this results in incorrect generation. This PR updates the `concatenate`
    function to correctly include the previous states.
    
    fixes: #1601
    343aa7a1
tokens.py 22.4 KB