1. 16 Jun, 2023 1 commit
  2. 14 Jun, 2023 1 commit
    • Matthijs Hollemans's avatar
      [WIP] add EnCodec model (#23655) · 0c3fdccf
      Matthijs Hollemans authored
      
      
      * boilerplate stuff
      
      * messing around with the feature extractor
      
      * fix feature extractor
      
      * unit tests for feature extractor
      
      * rename speech to audio
      
      * quick-and-dirty import of Meta's code
      
      * import weights (sort of)
      
      * cleaning up
      
      * more cleaning up
      
      * move encoder/decoder args into config
      
      * cleanup model
      
      * rename EnCodec -> Encodec
      
      * RVQ parameters in config
      
      * add slow test
      
      * add lstm init and test_init
      
      * Add save & load
      
      * finish EncodecModel
      
      * remove decoder_input_values as they are ont used anywhere (not removed from doc yet)
      
      * fix test feature extraction model name
      
      * Add better slow test
      
      * Fix tests
      
      * some fixup and cleaning
      
      * Improve further
      
      * cleaning up quantizer
      
      * fix up conversion script
      
      * test don't pass, _encode_fram does not work
      
      * update tests with output per encode and decode
      
      * more cleanup
      
      * rename _codebook
      
      * remove old config cruft
      
      * ratios & hop_length
      
      * use ModuleList instead of Sequential
      
      * clean up resnet block
      
      * update types
      
      * update tests
      
      * fixup
      
      * quick cleanup
      
      * fix padding
      
      * more styl,ing
      
      * add patrick feedback
      
      * fix copies
      
      * fixup
      
      * fix lstm
      
      * fix shape issues
      
      * fixup
      
      * rename conv layers
      
      * fixup
      
      * fix decoding
      
      * small conv refactoring
      
      * remove norm_params
      
      * simplify conv layers
      
      * rename conv layers
      
      * stuff
      
      * Clean up
      
      * Add padding logic
      
      use padding mask
      
      small conv refactoring
      
      remove norm_params
      
      simplify conv layers
      
      rename conv layers
      
      stuff
      
      add batched test
      
      update
      
      Clean up
      
      merge and update for padding
      
      fix padding
      
      fixup
      
      * clean up more
      
      * clean up more
      
      * More clean ups
      
      * cleanup convolutions
      
      * typo
      
      * fix typos
      
      * fixup
      
      * build PR doc?
      
      * start refactoring docstring
      
      * fix don't pad when no strid and chunk
      
      * update docstring
      
      * update docstring
      
      * nits
      
      * update going to lunch
      
      * update config and model
      
      * fix broken testse (becaue of the config changes)
      
      * fix scale computation
      
      * fixu[
      
      * only return dict if speciefied or if config returns it
      
      * remove todos
      
      * update defaults in config
      
      * update conversion script
      
      * fix doctest
      
      * more docstring + fixup
      
      * nits on batched_tests
      
      * more nits
      
      * Apply suggestions from code review
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * update basxed on review
      
      * fix update
      
      * updaet tests
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * fixup
      
      * add overlap and chunl_length_s
      
      * cleanup feature extraction
      
      * teste edge cases truncation and padding
      
      * correct processor values
      
      * update config encodec, nits
      
      * fix tests
      
      * fixup
      
      * fix 24Hz test
      
      * elle tests are green
      
      * fix fixup
      
      * Apply suggestions from code review
      
      * revert readme changes
      
      * fixup
      
      * add example
      
      * use facebook checkpoints
      
      * fix typo
      
      * no pipeline tests
      
      * use slef.pad everywhere we can
      
      * Apply suggestions from code review
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * update based on review
      
      * update
      
      * update mdx
      
      * fix bug and tests
      
      * fixup
      
      * fix doctest
      
      * remove comment
      
      * more nits
      
      * add more coverage for `test_truncation_and_padding`
      
      * fixup
      
      * add last test
      
      * fix text
      
      * nits
      
      * Update tests/models/encodec/test_modeling_encodec.py
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * take care of the last comments
      
      * typo
      
      * fix test
      
      * nits
      
      * fixup
      
      * Update src/transformers/models/encodec/feature_extraction_encodec.py
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatararthur.zucker@gmail.com <arthur.zucker@gmail.com>
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      0c3fdccf
  3. 31 May, 2023 1 commit
    • Denisa Roberts's avatar
      Add TensorFlow implementation of EfficientFormer (#22620) · 88f50a1e
      Denisa Roberts authored
      * Add tf code for efficientformer
      
      * Fix return dict bug - return last hidden state after last stage
      
      * Fix corresponding return dict bug
      
      * Override test tol
      
      * Change default values of training to False
      
      * Set training to default False X3
      
      * Rm axis from ln
      
      * Set init in dense projection
      
      * Rm debug stuff
      
      * Make style; all tests pass.
      
      * Modify year to 2023
      
      * Fix attention biases codes
      
      * Update the shape list logic
      
      * Add a batch norm eps config
      
      * Remove extract comments in test files
      
      * Add conditional attn and hidden states return for serving output
      
      * Change channel dim checking logic
      
      * Add exception for withteacher model in training mode
      
      * Revert layer count for now
      
      * Add layer count for conditional layer naming
      
      * Transpose for conv happens only in main layer
      
      * Make tests smaller
      
      * Make style
      
      * Update doc
      
      * Rm from_pt
      
      * Change to actual expect image class label
      
      * Remove stray print in tests
      
      * Update image processor test
      
      * Remove the old serving output logic
      
      * Make style
      
      * Make style
      
      * Complete test
      88f50a1e
  4. 13 Apr, 2023 3 commits
  5. 04 Apr, 2023 1 commit
  6. 28 Mar, 2023 1 commit
  7. 22 Mar, 2023 1 commit
  8. 21 Mar, 2023 2 commits
  9. 02 Mar, 2023 1 commit
    • amyeroberts's avatar
      Use PyAV instead of Decord in examples (#21572) · 3412f597
      amyeroberts authored
      * Use PyAV instead of Decord
      
      * Get frame indices
      
      * Fix number of frames
      
      * Update src/transformers/models/videomae/image_processing_videomae.py
      
      * Fix up
      
      * Fix copies
      
      * Update timesformer doctests
      
      * Update docstrings
      3412f597
  10. 01 Mar, 2023 1 commit
  11. 22 Feb, 2023 1 commit
  12. 20 Feb, 2023 1 commit
    • tanreinama's avatar
      add GPTSAN model (reopen) (#21291) · f56174ac
      tanreinama authored
      * add GPTSAN-Japanese
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN (update for review)
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * fix typo in comment text
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * fix document and comments
      
      * fix class name GPTSAN->GPTSan
      
      * fix import and test for tokenizer
      f56174ac
  13. 16 Feb, 2023 1 commit
    • Arthur's avatar
      [CLAP] Add CLAP to the library (#21370) · c236a621
      Arthur authored
      
      
      * add model like clip
      
      * update
      
      * text model ok
      
      * clap text works
      
      * some refactor
      
      - `CLAPVision` to `CLAPAudio`
      - refactor kwargs of audio modules
      
      * more refactor
      
      * more refactor
      
      * more refactor
      
      * correct fusion
      
      * more refactor
      
      * new modules
      
      * add basic processor
      
      * fixup
      
      * remove whisper copioed from
      
      * audio logits match
      
      * add doc
      
      * correct filters mel and add maxlength
      
      * style
      
      * few fixes
      
      * forward passes
      
      * fixup
      
      * fixup
      
      * some clean up
      
      * remove mels form the dictionnary
      
      * pad after the repeat
      
      * update padding when dsmaller
      
      * fix padding
      
      * style
      
      * use swin patch merging
      
      * use copied from swin
      
      * processor with any tokenizer
      
      * more copied from
      
      * some clean up
      
      * more refactor
      
      * fix mel when rand_trunc
      
      * style
      
      * remove unused imports
      
      * update processing
      
      * remove image processing tests
      
      * add testing fiel
      
      * fixmodeling issues
      
      * replace with `is_longer`
      
      * clap in serialization
      
      * more refactor
      
      * `make fixup`
      
      * make fixup
      
      * fix feature extractor
      
      * update test feature extractor
      
      * `make fixup`
      
      * clean up config
      
      * more clean up
      
      * more cleanup
      
      * update tests
      
      * refactor tests and inits
      
      * removeCLAP vision config
      
      * remove CLAP from image procssing auto and dummy vision objects
      
      * update inits
      
      * style
      
      * re order classes in modeling clap
      
      * Use roberta tokenizer as the other weights are not open sourced
      
      * small cleaup
      
      * remove tokenization CLAP
      
      * processor tokenizr is roberta
      
      * update feature extraction doc
      
      * remove vclap from model zero shot
      
      * update f_min and f_max to frequency_xx
      
      * some changes
      
      - fix modeling keys
      - add `is_longer` in the forward pass
      - make fixup
      
      * make fixup
      
      * consistent behavior ebtween rand_crop and fusion
      
      * add numpy resize and bilinear and documentation
      
      * move resizing to image utils
      
      * clean feature extraction
      
      * import resize from correct file
      
      * resize in image transforms
      
      * update
      
      * style
      
      * style
      
      * nit
      
      * remove unused arguments form the feature extractor
      
      * style
      
      * few fixes + make fixup
      
      * oops
      
      * fix more tests
      
      * add zero shot audio classification pipeline
      
      * update zeroshot classification pipeline
      
      * fixup
      
      * fix copies
      
      * all CI tests pass
      
      * make fixup + fix docs
      
      * fix docs
      
      * fix docs
      
      * update tests pip;eline
      
      * update zero shot pipeline
      
      * update feature extraction clap
      
      * update tokenization auto
      
      * use nested simplify
      
      * update pipeline tests
      
      * Apply suggestions from code review
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * split in two lines
      
      * fixes
      
      * refactor
      
      * clean up
      
      * add integration tests
      
      * update config docstring
      
      * style
      
      * update processor
      
      * fix processor test
      
      * fix feat extractor tests
      
      * update docs
      
      * Apply suggestions from code review
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * fix readmes
      
      * fix tips
      
      * Update src/transformers/models/auto/configuration_auto.py
      
      * update doc and remove todo -> properly explained
      
      * fix idx and typo
      
      * typoe
      
      * cleanup config
      
      * cleanup tests, styles and doc
      
      * ignore docstyle on image transform
      
      * add conversion script
      
      * remove the `clap` indx in favor of `CLAP`
      
      * update __init
      
      * nits
      
      * Update src/transformers/pipelines/__init__.py
      
      * fix bug
      
      * clarifiy config
      
      * fix copy
      
      * fix init
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * fix model output
      
      * fix comment
      
      * make fixup
      
      * make fixup
      
      * rename to `Clap`
      
      * replace to `Clap`
      
      * replace to `Clap`
      
      * repo consistency
      
      * again repo-consistency
      
      * make fixup
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      
      * add config
      
      * changes
      
      * update conversion
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      
      * remove unused function
      
      * update based on code reviews
      
      * style
      
      * more comments
      
      * cleanup
      
      * clean up
      
      * style
      
      * apply suggestions
      
      * Empty commit
      
      * pipeline will be added in a different PR
      
      * update calls to audio utils functions
      
      * update pipeline init
      
      * style
      
      * style
      
      * styling again
      
      * use pad
      
      * fix repo-consistency
      
      * update utils and add doc for audio utils
      
      * clean up resize by using torch. update inits accordingly
      
      * style
      
      * CLap's  tokenizer is RobertA
      
      * add audio utils to internal toctreee
      
      * update totctree
      
      * style
      
      * update documentation and normalize naming accross audio utils and feature extraction clap
      
      * style
      
      * clean up
      
      * update doc and typos
      
      * fix doctest
      
      * update modelin code, got rid of a lot of reshaping
      
      * style on added doc audio utils
      
      * update modeling clap
      
      * style
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * docstringvariables with CLAP
      
      * rename key
      
      * update modeling CLAP
      
      * update audio utils docstring
      
      * update processing clap
      
      * fix readmes
      
      * fix toctree
      
      * udpate configuration clap
      
      * fix init
      
      * make fixup
      
      * fix
      
      * fix
      
      * update naming
      
      * update
      
      * update checkpoint path
      
      * Apply suggestions from code review
      
      * Major refactoring
      
      * Update src/transformers/models/clap/configuration_clap.py
      
      * merge
      
      ---------
      Co-authored-by: default avataryounesbelkada <younesbelkada@gmail.com>
      Co-authored-by: default avatarYounes Belkada <49240599+younesbelkada@users.noreply.github.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      c236a621
  14. 15 Feb, 2023 1 commit
    • Susnato Dhar's avatar
      Add Ernie-M Model to huggingface (#21349) · 0c9c8472
      Susnato Dhar authored
      * config and tokenization(fast too) changed and ErnieEncoder added
      
      * Slow Tokenization Added
      
      * Tokenizer(slow) is now working and Fast Tokenizer removed
      
      * Added Config code
      
      * Added Base Model and utils
      
      * ErnieMModel is now working
      
      * All added except tests
      
      * All tests passed except ErnieUIEM
      
      * All tests passed
      
      * all fixes done
      
      * all fixes done
      
      * fixed MAP
      
      * fixed check_code_quality
      
      * fixed Build PR Documentation issue
      
      * Added changes(comments) and also updated to the latest upstream/main
      
      * Added fixup
      
      * Added # Copied comments
      
      * Added fixup
      
      * Added more comments and some nits
      
      * Added fixup
      
      * Fixed README_hd.md
      
      * Added more fixes
      
      * ErnieMTokenizer (being sentencepiece) protected and other docs edited
      
      * Added code_quality fix
      
      * Fixed for
      
      * Added more fix
      
      * modified AZ
      
      * ernie-m tokenization test added!
      
      * attention mask part fixed(with 0->self.config.pad_token_id)
      
      * applied make fixup
      0c9c8472
  15. 10 Feb, 2023 3 commits
  16. 09 Feb, 2023 1 commit
    • NielsRogge's avatar
      Add BLIP-2 (#21441) · d7f1e7c0
      NielsRogge authored
      
      
      * First draft
      
      * More improvements
      
      * More improvements
      
      * Improve conversion script
      
      * Convert all weights
      
      * Make forward pass work
      
      * Make logits match
      
      * More improvements
      
      * More improvements
      
      * More improvements
      
      * Use get_input_embeddings
      
      * Improve some more
      
      * Improve model tests
      
      * Improve model tests
      
      * More improvements
      
      * Fix processor
      
      * Update files
      
      * Update prepare_inputs_for_generation
      
      * More improvements
      
      * Fix copies
      
      * More fixes
      
      * Make fixup
      
      * More improvements
      
      * Add support for seq2seq language model
      
      * More improvements
      
      * Fix test
      
      * More improvements
      
      * Improve conversion script
      
      * Remove some todo's
      
      * Fix README's
      
      * Improve conversion script
      
      * Fix generation
      
      * Fix style and remove Blip2Model
      
      * Fix model outputs
      
      * More improvements
      
      * Set eos_token_id in config
      
      * Fix quality
      
      * Small improvements
      
      * Add processor tests
      
      * More improvements
      
      * Apply suggestions
      
      * Apply suggestions
      
      * Add integration test
      
      * Update image URL
      
      * Add integration test
      
      * Fix model_type
      
      * Update style
      
      * Improve docs
      
      * Add doc tests
      
      * Fix copies
      
      * Remove tests which are passing
      
      * Improve some more
      
      * Add tests for seq2seq language models
      
      * Minor fix
      
      * Convert more checkpoints
      
      * finalize CI
      
      * Fix blip and blip2 processors
      
      * add `accelerate` support for `blip2`
      
      * clean up
      
      * make style
      
      * Update conversion script
      
      * Update conversion script some more
      
      * Update organization
      
      * revert toc file
      
      * add blip-2 to toc file
      
      * Some more improvements
      
      * Fix docstring
      
      * Improve docs
      
      ---------
      Co-authored-by: default avatarydshieh <ydshieh@users.noreply.github.com>
      Co-authored-by: default avataryounesbelkada <younesbelkada@gmail.com>
      d7f1e7c0
  17. 03 Feb, 2023 1 commit
    • Matthijs Hollemans's avatar
      [WIP] add SpeechT5 model (#18922) · e4bacf66
      Matthijs Hollemans authored
      * make SpeechT5 model by copying Wav2Vec2
      
      * add paper to docs
      
      * whoops added docs in wrong file
      
      * remove SpeechT5Tokenizer + put CTC back in the name
      
      * remove deprecated class
      
      * remove unused docstring
      
      * delete SpeechT5FeatureExtractor, use Wav2Vec2FeatureExtractor instead
      
      * remove classes we don't need right now
      
      * initial stab at speech encoder prenet
      
      * add more speech encoder prenet stuff
      
      * improve SpeechEncoderPrenet
      
      * add encoder (not finished yet)
      
      * add relative position bias to self-attention
      
      * add encoder CTC layers
      
      * fix formatting
      
      * add decoder from BART, doesn't work yet
      
      * make it work with generate loop
      
      * wrap the encoder into a speech encoder class
      
      * wrap the decoder in a text decoder class
      
      * changed my mind
      
      * changed my mind again ;-)
      
      * load decoder weights, make it work
      
      * add weights for text decoder postnet
      
      * add SpeechT5ForCTC model that uses only the encoder
      
      * clean up EncoderLayer and DecoderLayer
      
      * implement _init_weights in SpeechT5PreTrainedModel
      
      * cleanup config + Encoder and Decoder
      
      * add head + cross attention masks
      
      * improve doc comments
      
      * fixup
      
      * more cleanup
      
      * more fixup
      
      * TextDecoderPrenet works now, thanks Kendall
      
      * add CTC loss
      
      * add placeholders for other pre/postnets
      
      * add type annotation
      
      * fix freeze_feature_encoder
      
      * set padding tokens to 0 in decoder attention mask
      
      * encoder attention mask downsampling
      
      * remove features_pen calculation
      
      * disable the padding tokens thing again
      
      * fixup
      
      * more fixup
      
      * code review fixes
      
      * rename encoder/decoder wrapper classes
      
      * allow checkpoints to be loaded into SpeechT5Model
      
      * put encoder into wrapper for CTC model
      
      * clean up conversion script
      
      * add encoder for TTS model
      
      * add speech decoder prenet
      
      * add speech decoder post-net
      
      * attempt to reconstruct the generation loop
      
      * add speech generation loop
      
      * clean up generate_speech
      
      * small tweaks
      
      * fix forward pass
      
      * enable always dropout on speech decoder prenet
      
      * sort declaration
      
      * rename models
      
      * fixup
      
      * fix copies
      
      * more fixup
      
      * make consistency checker happy
      
      * add Seq2SeqSpectrogramOutput class
      
      * doc comments
      
      * quick note about loss and labels
      
      * add HiFi-GAN implementation (from Speech2Speech PR)
      
      * rename file
      
      * add vocoder to TTS model
      
      * improve vocoder
      
      * working on tokenizer
      
      * more better tokenizer
      
      * add CTC tokenizer
      
      * fix decode and batch_code in CTC tokenizer
      
      * fix processor
      
      * two processors and feature extractors
      
      * use SpeechT5WaveformFeatureExtractor instead of Wav2Vec2
      
      * cleanup
      
      * more cleanup
      
      * even more fixup
      
      * notebooks
      
      * fix log-mel spectrograms
      
      * support reduction factor
      
      * fixup
      
      * shift spectrograms to right to create decoder inputs
      
      * return correct labels
      
      * add labels for stop token prediction
      
      * fix doc comments
      
      * fixup
      
      * remove SpeechT5ForPreTraining
      
      * more fixup
      
      * update copyright headers
      
      * add usage examples
      
      * add SpeechT5ProcessorForCTC
      
      * fixup
      
      * push unofficial checkpoints to hub
      
      * initial version of tokenizer unit tests
      
      * add slow test
      
      * fix failing tests
      
      * tests for CTC tokenizer
      
      * finish CTC tokenizer tests
      
      * processor tests
      
      * initial test for feature extractors
      
      * tests for spectrogram feature extractor
      
      * fixup
      
      * more fixup
      
      * add decorators
      
      * require speech for tests
      
      * modeling tests
      
      * more tests for ASR model
      
      * fix imports
      
      * add fake tests for the other models
      
      * fixup
      
      * remove jupyter notebooks
      
      * add missing SpeechT5Model tests
      
      * add missing tests for SpeechT5ForCTC
      
      * add missing tests for SpeechT5ForTextToSpeech
      
      * sort tests by name
      
      * fix Hi-Fi GAN tests
      
      * fixup
      
      * add speech-to-speech model
      
      * refactor duplicate speech generation code
      
      * add processor for SpeechToSpeech model
      
      * add usage example
      
      * add tests for speech-to-speech model
      
      * fixup
      
      * enable gradient checkpointing for SpeechT5FeatureEncoder
      
      * code review
      
      * push_to_hub now takes repo_id
      
      * improve doc comments for HiFi-GAN config
      
      * add missing test
      
      * add integration tests
      
      * make number of layers in speech decoder prenet configurable
      
      * rename variable
      
      * rename variables
      
      * add auto classes for TTS and S2S
      
      * REMOVE CTC!!!
      
      * S2S processor does not support save/load_pretrained
      
      * fixup
      
      * these models are now in an auto mapping
      
      * fix doc links
      
      * rename HiFiGAN to HifiGan, remove separate config file
      
      * REMOVE auto classes
      
      * there can be only one
      
      * fixup
      
      * replace assert
      
      * reformat
      
      * feature extractor can process input and target at same time
      
      * update checkpoint names
      
      * fix commit hash
      e4bacf66
  18. 31 Jan, 2023 1 commit
    • NielsRogge's avatar
      Add DETA (#20983) · 5451f889
      NielsRogge authored
      * First draft
      
      * Add initial draft of conversion script
      
      * Convert all weights
      
      * Fix config
      
      * Add image processor
      
      * Fix DetaImageProcessor
      
      * Run make fix copies
      
      * Remove timm dependency
      
      * Fix dummy objects
      
      * Improve loss function
      
      * Remove conv_encoder attribute
      
      * Update conversion scripts
      
      * Improve postprocessing + docs
      
      * Fix copied from statements
      
      * Add tests
      
      * Improve postprocessing
      
      * Improve postprocessing
      
      * Update READMEs
      
      * More improvements
      
      * Fix rebase
      
      * Add is_torchvision_available
      
      * Add torchvision dependency
      
      * Fix typo and README
      
      * Fix bug
      
      * Add copied from
      
      * Fix style
      
      * Apply suggestions
      
      * Fix thanks to @ydshieh
      
      * Fix another dependency check
      
      * Simplify image processor
      
      * Add scipy
      
      * Improve code
      
      * Add threshold argument
      
      * Fix bug
      
      * Set default threshold
      
      * Improve integration test
      
      * Add another integration test
      
      * Update setup.py
      
      * Address review
      
      * Improve deformable attention function
      
      * Improve copied from
      
      * Use relative imports
      
      * Address review
      
      * Replace assertions
      
      * Address review
      
      * Update dummies
      
      * Remove dummies
      
      * Address comments, update READMEs
      
      * Remove custom kernel code
      
      * Add image processor tests
      
      * Add requires_backends
      
      * Add minor comment
      
      * Update scripts
      
      * Update organization name
      
      * Fix defaults, add doc tests
      
      * Add id2label for object 365
      
      * Fix tests
      
      * Update task guide
      5451f889
  19. 26 Jan, 2023 1 commit
  20. 25 Jan, 2023 1 commit
  21. 19 Jan, 2023 1 commit
    • Jitesh Jain's avatar
      Add OneFormer Model (#20577) · 5b949623
      Jitesh Jain authored
      * Add Oneformer Model
      
      * Add OneFormer Tests
      
      * Add UNIVERSAL_SEGMENTATION_MAPPING
      
      * Fix config
      
      * 馃悰 Fix error encountered while writing tests
      
      * 馃敤 Fix instance segmentation post processing
      
      * Format Files and Add Documentation
      
      * Add Documentation mdx file
      
      * Run make fixup
      
      * Run make fix-copies
      
      * Remove unnecessary code
      
      * Format modeling_oneformer.py
      
      * Add OneFormer to ImageSegmentationPipeline
      
      * Format files
      
      * Add Demo link to Readme
      
      * Fix fomatting errors
      
      * Fix test failures
      
      * Update Table in index.mdx
      
      * Fix version
      
      * Fix style
      
      * Remove OneFormer from TF
      
      * Fix Imports
      
      * Fix dummy objects
      
      * Fix tests
      
      * Add newline
      
      * Remove OneFormerFeatureExtractor
      
      * Remove CUDA Kernels
      
      * Use AutoBackbone for Swin
      
      * Fix description
      
      * Use Image Processor
      
      * Fix copies
      
      * Fix formatting
      
      * Fix import order
      
      * Fix flake8 errors
      
      * Fix doc errors
      
      * Add Hindi Readme entry
      
      * Update supported backbones
      
      * Update supported backbones
      
      * Undo Changes
      
      * Fix type of config
      
      * Fix isort
      
      * Fix auto.mdx
      
      * Fix swin config
      
      * Replace DinatBackbone with AutoBackbone
      
      * Use SwinBackbone
      
      * Use SwinBackbone
      
      * Fix conversion script
      
      * Fix arguments
      
      * Add argument description
      
      * Fix style
      
      * Add OneFormerProcessor
      
      * Fix OneFormerProcessor Tests
      
      * Fix mapping
      
      * Fix imports
      
      * Fix inits
      
      * Fix style
      
      * Fix comment
      
      * Fix docstring
      
      * Move OneFormer to MultiModal
      
      * Fix Copies
      
      * Remove size divisor
      
      * Fix check_repo.py
      
      * Fix copies
      
      * Add Processor for Testing Pipeline
      
      * Fix padding for tokens
      
      * Fix variables
      
      * Fix formatting with correct black version
      
      * Add Image Processor Test
      
      * Apply suggestions
      
      * Revert common modeling
      
      * Add check for task
      
      * Fix conversion script
      
      * Fix initialization order
      
      * Fix tests
      
      * Undo Pipeline Changes
      
      * Fix layers in MLP
      
      * Fix copies
      
      * Update image paths
      
      * Fix copies
      
      * Apply suggestions
      5b949623
  22. 16 Jan, 2023 1 commit
    • NielsRogge's avatar
      Add UperNet (#20648) · 4ed89d48
      NielsRogge authored
      
      
      * First draft
      
      * More improvements
      
      * Add convnext backbone
      
      * Add conversion script
      
      * Add more improvements
      
      * Comment out to_dict
      
      * Add to_dict method
      
      * Add default config
      
      * Fix config
      
      * Fix backbone
      
      * Fix backbone some more
      
      * Add docs, auto mapping, tests
      
      * Fix some tests
      
      * Fix more tests
      
      * Fix more tests
      
      * Add conversion script
      
      * Improve conversion script
      
      * Add support for getting reshaped undownsampled hidden states
      
      * Fix forward pass
      
      * Add print statements
      
      * Comment out set_shift_and_window_size
      
      * More improvements
      
      * Correct downsampling layers conversion
      
      * Fix style
      
      * First draft
      
      * Fix conversion script
      
      * Remove config attribute
      
      * Fix more tests
      
      * Update READMEs
      
      * Update ConvNextBackbone
      
      * Fix ConvNext tests
      
      * Align ConvNext with Swin
      
      * Remove files
      
      * Fix index
      
      * Improve docs
      
      * Add output_attentions to model forward
      
      * Add backbone mixin, improve tests
      
      * More improvements
      
      * Update init_weights
      
      * Fix interpolation of logits
      
      * Add UperNetImageProcessor
      
      * Improve image processor
      
      * Fix image processor
      
      * Remove print statements
      
      * Remove script
      
      * Update import
      
      * Add image processor tests
      
      * Remove print statements
      
      * Fix test
      
      * Add integration test
      
      * Add convnext integration test
      
      * Update docstring
      
      * Fix README
      
      * Simplify config
      
      * Apply suggestions
      
      * Improve docs
      
      * Rename class
      
      * Fix test_initialization
      
      * Fix import
      
      * Address review
      
      * Fix confg
      
      * Convert all checkpoints
      
      * Fix default backbone
      
      * Usage same processor as segformer
      
      * Apply suggestions
      
      * Fix init_weights, update conversion scripts
      
      * Improve config
      
      * Use Auto API instead of creating a new image processor
      
      * Fix docs
      
      * Add doctests
      
      * Remove ResNetConfig dependency
      
      * Add always_partition argument
      
      * Fix rebase茅
      
      * Improve docs
      
      * Convert checkpoints
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MBP.localdomain>
      4ed89d48
  23. 08 Jan, 2023 1 commit
  24. 05 Jan, 2023 1 commit
  25. 03 Jan, 2023 1 commit
    • NielsRogge's avatar
      Add GIT (GenerativeImage2Text) (#20295) · 9c6f7485
      NielsRogge authored
      
      
      * First draft
      
      * Make model instantiation work
      
      * Fix copied from statement
      
      * More fixes
      
      * Add correct output head
      
      * Improve configuration
      
      * Add conversion script
      
      * Improve conversion script
      
      * Remove token_type_ids
      
      * Fix conversion of projection layers
      
      * Convert all weights
      
      * Use cats image
      
      * Make logits match
      
      * Generate caption on cats image
      
      * Add GITProcessor
      
      * Update conversion script
      
      * Add support for more checkpoints
      
      * Fix conversion script
      
      * Add initial tests
      
      * Remove cross-attention
      
      * More improvements
      
      * Remove is_decoder
      
      * Improve model tests
      
      * Improve tests
      
      * Improve model outputs
      
      * Fix model outputs equivalence
      
      * Fix more tests
      
      * Remove unused code
      
      * Use generate to generate text, no use of cache for now
      
      * Use generate more appropriately
      
      * Fix config tests
      
      * Fix style
      
      * Add support for use_cache
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      
      * Fix style
      
      * Fix GIT vision encoder
      
      * Update README
      
      * Fix integration test
      
      * Set bos and eos token ids
      
      * Improve docs
      
      * Improve code
      
      * Add support for provided attention_mask
      
      * Add copied from statement
      
      * Fix gradient checkpointing test
      
      * Set model_input_names
      
      * Investigate model_input_names
      
      * Remove script
      
      * Fix model inputs
      
      * Fix docstring
      
      * Rename GIT to Git
      
      * Support more models
      
      * Add support for textvqa model
      
      * Add video support
      
      * Extend conversion script for video
      
      * Add support for large variant
      
      * Add support for more models
      
      * Fix config archive map
      
      * Update integration test
      
      * Fix README
      
      * Fix CLIP mean and std
      
      * Update processor
      
      * Fix use_cache for video, thanks @gante
      
      * Remove print statements
      
      * Remove assertion
      
      * Add processor tests
      
      * Fix model_input_names
      
      * Use Auto API for processor
      
      * Fix processor tests
      
      * Fix integration test
      
      * Fix pipeline test
      
      * Make tests faster
      
      * Update conversion script
      
      * Update conversion script
      
      * Convert more checkpoints
      
      * Update conversion script
      
      * Fix typo
      
      * Update docstrings
      
      * Improve code snippets
      
      * Fix doc tests
      
      * Add more code examples茅
      
      * Fix doc tests
      
      * Add integration tests
      
      * Fix unused variable
      
      * revert
      
      * Add GIT to Japanese README
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      Co-authored-by: default avatarydshieh <ydshieh@users.noreply.github.com>
      9c6f7485
  26. 21 Dec, 2022 2 commits
  27. 19 Dec, 2022 1 commit
  28. 28 Nov, 2022 1 commit
  29. 23 Nov, 2022 1 commit
  30. 21 Nov, 2022 3 commits
    • NielsRogge's avatar
      Add Audio Spectogram Transformer (#19981) · 4973d2a0
      NielsRogge authored
      
      
      * First draft
      
      * Make conversion script work
      
      * Add id2label mapping, run code quality
      
      * Fix copies
      
      * Add first draft of feature extractor
      
      * Update conversion script to use feature extractor
      
      * Make more tests pass
      
      * Add docs
      
      * update input_features to input_values + pad by default to max length
      
      * Fix doc tests
      
      * Add feature extractor tests
      
      * Add proper padding/truncation to feature extractor
      
      * Add support for conversion of all audioset checkpoints
      
      * Improve docs and extend conversion script
      
      * Fix README
      
      * Rename spectogram to spectrogram
      
      * Fix copies
      
      * Add integration test
      
      * Remove dummy conv
      
      * Update to ast
      
      * Update organization
      
      * Fix init
      
      * Rename model to AST
      
      * Add require_torchaudio annotator
      
      * Move import of ASTFeatureExtractor under a is_speech_available
      
      * Fix rebase
      
      * Add pipeline config
      
      * Update name of classifier head
      
      * Rename time_dimension and frequency_dimension for clarity
      
      * Remove print statement
      
      * Fix pipeline test
      
      * Fix pipeline test
      
      * Fix index table
      
      * Fix init
      
      * Fix conversion script
      
      * Rename to ForAudioClassification
      
      * Fix index table
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      4973d2a0
    • Matthijs Hollemans's avatar
      add MobileNetV1 model (#17799) · d21c97cc
      Matthijs Hollemans authored
      * add model files etc for MobileNetV2
      
      rename files for MobileNetV1
      
      initial implementation of MobileNetV1
      
      fix conversion script
      
      cleanup
      
      write docs
      
      tweaks
      
      fix conversion script
      
      extract hidden states
      
      fix test cases
      
      make fixup
      
      fixup it all
      
      remove main from doc link
      
      fixes
      
      fix tests
      
      fix up
      
      use google org
      
      fix weird assert
      
      * fixup
      
      * use google organization for checkpoints
      d21c97cc
    • Joao Gante's avatar
  31. 18 Nov, 2022 1 commit
    • Ali Hassani's avatar
      Add Neighborhood Attention Transformer (NAT) and Dilated NAT (DiNAT) models (#20219) · fc4a993e
      Ali Hassani authored
      * Add DiNAT
      
      * Adds DiNAT + tests
      
      * Minor fixes
      
      * Added HF model
      
      * Add natten to dependencies.
      
      * Cleanup
      
      * Minor fixup
      
      * Reformat
      
      * Optional NATTEN import.
      
      * Reformat & add doc to _toctree
      
      * Reformat (finally)
      
      * Dummy objects for DiNAT
      
      * Add NAT + minor changes
      
      Adds NAT as its own independent model + docs, tests
      Adds NATTEN to ext deps to ensure ci picks it up.
      
      * Remove natten from `all` and `dev-torch` deps, add manual pip install to ci tests
      
      * Minor fixes.
      
      * Fix READMEs.
      
      * Requested changes to docs + minor fixes.
      
      * Requested changes.
      
      * Add NAT/DiNAT tests to layoutlm_job
      
      * Correction to Dinat doc.
      
      * Requested changes.
      fc4a993e
  32. 17 Nov, 2022 1 commit