• AllentDan's avatar
    Decode generated token_ids incrementally (#309) · 9bfe03c6
    AllentDan authored
    * add incremental decoding for turbomind
    
    * update TIS
    
    * fix triton post processing
    
    * update doc
    
    * fix typo
    
    * SentencePieceTokenizer incremental decode, add qwen message prompt
    
    * docstring
    
    * update bot
    9bfe03c6
restful_api.md 5.17 KB