• Matthijs Hollemans's avatar
    [WIP] add EnCodec model (#23655) · 0c3fdccf
    Matthijs Hollemans authored
    
    
    * boilerplate stuff
    
    * messing around with the feature extractor
    
    * fix feature extractor
    
    * unit tests for feature extractor
    
    * rename speech to audio
    
    * quick-and-dirty import of Meta's code
    
    * import weights (sort of)
    
    * cleaning up
    
    * more cleaning up
    
    * move encoder/decoder args into config
    
    * cleanup model
    
    * rename EnCodec -> Encodec
    
    * RVQ parameters in config
    
    * add slow test
    
    * add lstm init and test_init
    
    * Add save & load
    
    * finish EncodecModel
    
    * remove decoder_input_values as they are ont used anywhere (not removed from doc yet)
    
    * fix test feature extraction model name
    
    * Add better slow test
    
    * Fix tests
    
    * some fixup and cleaning
    
    * Improve further
    
    * cleaning up quantizer
    
    * fix up conversion script
    
    * test don't pass, _encode_fram does not work
    
    * update tests with output per encode and decode
    
    * more cleanup
    
    * rename _codebook
    
    * remove old config cruft
    
    * ratios & hop_length
    
    * use ModuleList instead of Sequential
    
    * clean up resnet block
    
    * update types
    
    * update tests
    
    * fixup
    
    * quick cleanup
    
    * fix padding
    
    * more styl,ing
    
    * add patrick feedback
    
    * fix copies
    
    * fixup
    
    * fix lstm
    
    * fix shape issues
    
    * fixup
    
    * rename conv layers
    
    * fixup
    
    * fix decoding
    
    * small conv refactoring
    
    * remove norm_params
    
    * simplify conv layers
    
    * rename conv layers
    
    * stuff
    
    * Clean up
    
    * Add padding logic
    
    use padding mask
    
    small conv refactoring
    
    remove norm_params
    
    simplify conv layers
    
    rename conv layers
    
    stuff
    
    add batched test
    
    update
    
    Clean up
    
    merge and update for padding
    
    fix padding
    
    fixup
    
    * clean up more
    
    * clean up more
    
    * More clean ups
    
    * cleanup convolutions
    
    * typo
    
    * fix typos
    
    * fixup
    
    * build PR doc?
    
    * start refactoring docstring
    
    * fix don't pad when no strid and chunk
    
    * update docstring
    
    * update docstring
    
    * nits
    
    * update going to lunch
    
    * update config and model
    
    * fix broken testse (becaue of the config changes)
    
    * fix scale computation
    
    * fixu[
    
    * only return dict if speciefied or if config returns it
    
    * remove todos
    
    * update defaults in config
    
    * update conversion script
    
    * fix doctest
    
    * more docstring + fixup
    
    * nits on batched_tests
    
    * more nits
    
    * Apply suggestions from code review
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    
    * update basxed on review
    
    * fix update
    
    * updaet tests
    
    * Apply suggestions from code review
    Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
    
    * fixup
    
    * add overlap and chunl_length_s
    
    * cleanup feature extraction
    
    * teste edge cases truncation and padding
    
    * correct processor values
    
    * update config encodec, nits
    
    * fix tests
    
    * fixup
    
    * fix 24Hz test
    
    * elle tests are green
    
    * fix fixup
    
    * Apply suggestions from code review
    
    * revert readme changes
    
    * fixup
    
    * add example
    
    * use facebook checkpoints
    
    * fix typo
    
    * no pipeline tests
    
    * use slef.pad everywhere we can
    
    * Apply suggestions from code review
    Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
    
    * update based on review
    
    * update
    
    * update mdx
    
    * fix bug and tests
    
    * fixup
    
    * fix doctest
    
    * remove comment
    
    * more nits
    
    * add more coverage for `test_truncation_and_padding`
    
    * fixup
    
    * add last test
    
    * fix text
    
    * nits
    
    * Update tests/models/encodec/test_modeling_encodec.py
    Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
    
    * take care of the last comments
    
    * typo
    
    * fix test
    
    * nits
    
    * fixup
    
    * Update src/transformers/models/encodec/feature_extraction_encodec.py
    Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
    
    ---------
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    Co-authored-by: default avatararthur.zucker@gmail.com <arthur.zucker@gmail.com>
    Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
    Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
    Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
    0c3fdccf
README.md 89.4 KB