• anj-s's avatar
    [refactor] Enable benchmarks/pipe.py and merge real and synthetic input pipeline. (#286) · 44b9bcd8
    anj-s authored
    
    
    * [refactor]Remove unused variables and refactor common configurations
    
    * move helper function to call site
    
    * fixed lint errors
    
    * fix lint errors
    
    * fix lint errors
    
    * fix lint errors
    
    * fix import order
    
    * format files
    
    * remove unused imports
    
    * fix lint errors
    
    * fix lint errors
    
    * refactor common utilities
    
    * address PR comments
    
    * sorted imports
    
    * add space
    
    * modify comment
    
    * added doc strings and addressed PR comments.
    
    * addressed PR comments
    
    * added another comment to clarify.
    
    * fixing lint errors
    
    * addressed PR comments
    
    * addressed PR comments
    
    * fixed typos
    
    * initialize var
    
    * rename seq_pred to lm
    
    * fix lint errors
    
    * move datasets and models into separate folders
    
    * add the folders created
    
    * fix lint errors
    
    * create golden config to stats mapping
    
    * add common batching for both synthetic and real data
    
    * fixed lint errors
    
    * enable real pipe benchmakrs with new golden data
    
    * reduce seq len to avoid OOM
    
    * updated golden data
    
    * add logging
    
    * add golden data
    
    * add golden data
    
    * fix lint errors
    
    * add doc string
    
    * remove commented out line
    
    * address comments
    
    * rename imports
    
    * refactor common logic in dataloaders
    
    * add golden configs
    
    * lint changes
    Co-authored-by: default avatarAnjali Sridhar <anj@devfair0443.h2.fair>
    44b9bcd8
transformer_lm.py 3.21 KB