• Lorenzo Ampil's avatar
    Fix Inconsistent NER Grouping (Pipeline) (#4987) · 0cc4eae0
    Lorenzo Ampil authored
    
    
    * Add B I handling to grouping
    
    * Add fix to include separate entity as last token
    
    * move last_idx definition outside loop
    
    * Use first entity in entity group as reference for entity type
    
    * Add test cases
    
    * Take out extra class accidentally added
    
    * Return tf ner grouped test to original
    
    * Take out redundant last entity
    
    * Get last_idx safely
    Co-authored-by: default avatarColleterVi <36503688+ColleterVi@users.noreply.github.com>
    
    * Fix first entity comment
    
    * Create separate functions for group_sub_entities and group_entities (splitting call method to testable functions)
    
    * Take out unnecessary last_idx
    
    * Remove additional forward pass test
    
    * Move token classification basic tests to separate class
    
    * Move token classification basic tests back to monocolumninputtestcase
    
    * Move base ner tests to nerpipelinetests
    
    * Take out unused kwargs
    
    * Add back mandatory_keys argument
    
    * Add unitary tests for group_entities in _test_ner_pipeline
    
    * Fix last entity handling
    
    * Fix grouping fucntion used
    
    * Add typing to group_sub_entities and group_entities
    Co-authored-by: default avatarColleterVi <36503688+ColleterVi@users.noreply.github.com>
    0cc4eae0
test_pipelines.py 23.4 KB