GitLab

The modification replaces this method with a token-based approach using a tokenizer. This tokenizes the entire prompt, identifies the token corresponding to the trigger word, removes it, and then decodes the modified tokens back into text.