• Julien Plu's avatar
    Add (M)Luke model training for Token Classification in the examples (#14880) · aa19f478
    Julien Plu authored
    * Add Luke training
    
    * Fix true label tags
    
    * Fix true label tags
    
    * Fix true label tags
    
    * Update the data collator for Luke
    
    * Some training refactor for Luke
    
    * Improve data collator for Luke
    
    * Fix import
    
    * Fix datasets concatenation
    
    * Add the --max_entity_length argument for Luke models
    
    * Remove unused code
    
    * Fix style issues
    
    * Fix style issues
    
    * Move the Luke training into a separate folder
    
    * Fix style
    
    * Fix naming
    
    * Fix filtering
    
    * Fix filtering
    
    * Fix filter
    
    * Update some preprocessing
    
    * Move luke to research_projects
    
    * Checkstyle
    
    * Address comments
    
    * Fix style
    aa19f478
luke_utils.py 4.99 KB