• ghlai9665's avatar
    LayoutLMv2Processor: ensure 1-to-1 mapping between images and samples in case... · e9fd583c
    ghlai9665 authored
    LayoutLMv2Processor: ensure 1-to-1 mapping between images and samples in case of overflowing tokens (#17092)
    
    * add get_overflowing_images function to ensure 1-to-1 mapping between samples and images in LayoutLMv2Processor
    
    * make style
    
    * add test for overflowing_tokens, change assert to ValueError, avoiding unrelated formatting changes
    
    * change line length by passing --preview into black
    e9fd583c
test_processor_layoutlmv2.py 23.3 KB