"docs/source/vscode:/vscode.git/clone" did not exist on "00b5887b948dc6e796b089ffa4057d06284b05d5"
  • Matt's avatar
    Allow apply_chat_template to pass kwargs to the template and support a dict of templates (#29658) · 48fbab73
    Matt authored
    * Allow apply_chat_template to pass kwargs to the template
    
    * Fix priority for template_kwargs
    
    * Fix docstring
    
    * style fix
    
    * Add the option for the model to have a dict of templates
    
    * Error message cleanup
    
    * Add test for chat template dicts
    
    * Simplify the chat template dict test and apply it to all tokenizers in self.get_tokenizers()
    
    * Save chat template dicts as lists with fixed key names
    
    * Add test for serialization/reloading
    
    * Add require_jinja just to be safe, even though I don't think we use it
    48fbab73
test_tokenization_common.py 213 KB