1. 01 Mar, 2023 2 commits
  2. 28 Feb, 2023 1 commit
  3. 27 Feb, 2023 1 commit
  4. 24 Feb, 2023 1 commit
  5. 21 Feb, 2023 3 commits
  6. 20 Feb, 2023 3 commits
    • Alara Dirik's avatar
      Add EfficientNet (#21563) · 49ab1623
      Alara Dirik authored
      * Add EfficientNet to transformers
      49ab1623
    • tanreinama's avatar
      add GPTSAN model (reopen) (#21291) · f56174ac
      tanreinama authored
      * add GPTSAN-Japanese
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN (update for review)
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * fix typo in comment text
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * add GPTSAN
      
      * fix document and comments
      
      * fix class name GPTSAN->GPTSan
      
      * fix import and test for tokenizer
      f56174ac
    • Andy Ehrenberg's avatar
      add flax whisper implementation (#20479) · 2840272c
      Andy Ehrenberg authored
      
      
      * add flax whisper implementation
      
      * rever change to setup
      
      * remove unused imports
      
      * revert generation changes
      
      * flax whisper docs
      
      * docs
      
      * import order
      
      * import sorting
      
      * isort
      
      * add dummy objects
      
      * doc formatting
      
      * formatting
      
      * remove trailing whitespaces
      
      * fix flax whisper docs
      
      * add generation logic to unlock flax whisper
      
      * remove scans
      
      * give credits to Flax Bart implementation
      
      * remove unused imports
      
      * add license
      
      * remove assert
      
      * more credits to Bart
      
      * fix style
      
      * formatting
      
      * support left padding
      
      * add flax whisper generation test
      
      * remove copied from comments whenever not a full copy
      
      * fix docstrings for logits processors
      
      * revert change to FlaxForceTokensLogitsProcessor
      
      * revert doc changes
      
      * improve generation docs
      
      * reorganize
      
      * formatting
      
      * cleanup docs
      
      * add tests
      
      * handle empty list case
      
      * fix forced decoder ids in flax tests
      
      * add flax whisper to inits
      
      * upate dummy objects
      
      * docs for FlaxAutoModelForSpeechSeq2Seq
      
      * fix decoder_position_ids computation in pretrained model decode/__call__ fns
      
      * add Copied from statements as necessary
      
      * compute position_ids only in __call__ and decode methods of pretrained model subclasses
      
      * improve readabilityof compute positional embeddings
      
      * check dimensionality of input_features instead of hidden_states
      
      * copied from statement for init_cache
      
      * formatting
      
      * fix copies
      
      * fix copies
      
      * pass attention mask to encoder layers
      
      * fix decoder module outputs
      
      * set dtype
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      
      * smaller flax model for whisper test
      
      * Update src/transformers/generation/flax_utils.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/whisper/modeling_flax_whisper.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update tests/models/whisper/test_modeling_flax_whisper.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * cleanup
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/whisper/modeling_flax_whisper.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * bias cleanup
      
      * doc fix
      
      * align style for force tokens processor
      
      * readability
      
      * fix input shape in tests
      
      * revert FlaxGenerationMixin docstring
      
      * formatting
      
      * fix tests
      
      * fix imports
      
      * consistent encoder hidden states
      
      * consistent hidden states
      
      * input shapes
      
      * typo
      
      * partial class trick
      
      * partial class for input shape
      
      * base_class with correct input shape
      
      * partial base classes
      
      * match by name
      
      * set main_input_name
      
      * compare on names
      
      * formatting
      
      * remove unused import
      
      * safer position ids computation
      
      * safer position id computation
      
      * Update src/transformers/models/whisper/modeling_flax_whisper.py
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      
      * Update src/transformers/models/whisper/modeling_flax_whisper.py
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      
      * remove identical inherited tests
      
      * fix prompt ids in tests
      
      * use generation config
      
      * use jnp array
      
      * better var names
      
      * more explicit bias use
      
      * import transformers
      
      * formatting
      
      * test formatting
      
      * remove unused imports
      
      * remove unused imports
      
      * formatting
      
      * isort
      
      * docs
      
      * fix ln orders for encoder hidden states
      
      * whisper unique generation stuff
      
      * flake
      
      * use finfo for attention bias
      
      * docs
      
      * Update src/transformers/generation/flax_utils.py
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * docs
      
      * add timestamp flax test
      
      * jit for timestamps
      
      * formatting
      
      * clean up timestamps processor
      
      * formatting
      
      * remove if_true
      
      * cleanup
      
      ---------
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      2840272c
  7. 16 Feb, 2023 3 commits
    • Arthur's avatar
      [CLAP] Add CLAP to the library (#21370) · c236a621
      Arthur authored
      
      
      * add model like clip
      
      * update
      
      * text model ok
      
      * clap text works
      
      * some refactor
      
      - `CLAPVision` to `CLAPAudio`
      - refactor kwargs of audio modules
      
      * more refactor
      
      * more refactor
      
      * more refactor
      
      * correct fusion
      
      * more refactor
      
      * new modules
      
      * add basic processor
      
      * fixup
      
      * remove whisper copioed from
      
      * audio logits match
      
      * add doc
      
      * correct filters mel and add maxlength
      
      * style
      
      * few fixes
      
      * forward passes
      
      * fixup
      
      * fixup
      
      * some clean up
      
      * remove mels form the dictionnary
      
      * pad after the repeat
      
      * update padding when dsmaller
      
      * fix padding
      
      * style
      
      * use swin patch merging
      
      * use copied from swin
      
      * processor with any tokenizer
      
      * more copied from
      
      * some clean up
      
      * more refactor
      
      * fix mel when rand_trunc
      
      * style
      
      * remove unused imports
      
      * update processing
      
      * remove image processing tests
      
      * add testing fiel
      
      * fixmodeling issues
      
      * replace with `is_longer`
      
      * clap in serialization
      
      * more refactor
      
      * `make fixup`
      
      * make fixup
      
      * fix feature extractor
      
      * update test feature extractor
      
      * `make fixup`
      
      * clean up config
      
      * more clean up
      
      * more cleanup
      
      * update tests
      
      * refactor tests and inits
      
      * removeCLAP vision config
      
      * remove CLAP from image procssing auto and dummy vision objects
      
      * update inits
      
      * style
      
      * re order classes in modeling clap
      
      * Use roberta tokenizer as the other weights are not open sourced
      
      * small cleaup
      
      * remove tokenization CLAP
      
      * processor tokenizr is roberta
      
      * update feature extraction doc
      
      * remove vclap from model zero shot
      
      * update f_min and f_max to frequency_xx
      
      * some changes
      
      - fix modeling keys
      - add `is_longer` in the forward pass
      - make fixup
      
      * make fixup
      
      * consistent behavior ebtween rand_crop and fusion
      
      * add numpy resize and bilinear and documentation
      
      * move resizing to image utils
      
      * clean feature extraction
      
      * import resize from correct file
      
      * resize in image transforms
      
      * update
      
      * style
      
      * style
      
      * nit
      
      * remove unused arguments form the feature extractor
      
      * style
      
      * few fixes + make fixup
      
      * oops
      
      * fix more tests
      
      * add zero shot audio classification pipeline
      
      * update zeroshot classification pipeline
      
      * fixup
      
      * fix copies
      
      * all CI tests pass
      
      * make fixup + fix docs
      
      * fix docs
      
      * fix docs
      
      * update tests pip;eline
      
      * update zero shot pipeline
      
      * update feature extraction clap
      
      * update tokenization auto
      
      * use nested simplify
      
      * update pipeline tests
      
      * Apply suggestions from code review
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * split in two lines
      
      * fixes
      
      * refactor
      
      * clean up
      
      * add integration tests
      
      * update config docstring
      
      * style
      
      * update processor
      
      * fix processor test
      
      * fix feat extractor tests
      
      * update docs
      
      * Apply suggestions from code review
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * fix readmes
      
      * fix tips
      
      * Update src/transformers/models/auto/configuration_auto.py
      
      * update doc and remove todo -> properly explained
      
      * fix idx and typo
      
      * typoe
      
      * cleanup config
      
      * cleanup tests, styles and doc
      
      * ignore docstyle on image transform
      
      * add conversion script
      
      * remove the `clap` indx in favor of `CLAP`
      
      * update __init
      
      * nits
      
      * Update src/transformers/pipelines/__init__.py
      
      * fix bug
      
      * clarifiy config
      
      * fix copy
      
      * fix init
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * fix model output
      
      * fix comment
      
      * make fixup
      
      * make fixup
      
      * rename to `Clap`
      
      * replace to `Clap`
      
      * replace to `Clap`
      
      * repo consistency
      
      * again repo-consistency
      
      * make fixup
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      
      * add config
      
      * changes
      
      * update conversion
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      
      * remove unused function
      
      * update based on code reviews
      
      * style
      
      * more comments
      
      * cleanup
      
      * clean up
      
      * style
      
      * apply suggestions
      
      * Empty commit
      
      * pipeline will be added in a different PR
      
      * update calls to audio utils functions
      
      * update pipeline init
      
      * style
      
      * style
      
      * styling again
      
      * use pad
      
      * fix repo-consistency
      
      * update utils and add doc for audio utils
      
      * clean up resize by using torch. update inits accordingly
      
      * style
      
      * CLap's  tokenizer is RobertA
      
      * add audio utils to internal toctreee
      
      * update totctree
      
      * style
      
      * update documentation and normalize naming accross audio utils and feature extraction clap
      
      * style
      
      * clean up
      
      * update doc and typos
      
      * fix doctest
      
      * update modelin code, got rid of a lot of reshaping
      
      * style on added doc audio utils
      
      * update modeling clap
      
      * style
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * docstringvariables with CLAP
      
      * rename key
      
      * update modeling CLAP
      
      * update audio utils docstring
      
      * update processing clap
      
      * fix readmes
      
      * fix toctree
      
      * udpate configuration clap
      
      * fix init
      
      * make fixup
      
      * fix
      
      * fix
      
      * update naming
      
      * update
      
      * update checkpoint path
      
      * Apply suggestions from code review
      
      * Major refactoring
      
      * Update src/transformers/models/clap/configuration_clap.py
      
      * merge
      
      ---------
      Co-authored-by: default avataryounesbelkada <younesbelkada@gmail.com>
      Co-authored-by: default avatarYounes Belkada <49240599+younesbelkada@users.noreply.github.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      c236a621
    • Alissa's avatar
      Add OPT resources to the transformers documentation (#21625) · b0f0086f
      Alissa authored
      * Add resources to OPT
      
      * Add additional resources for OPT
      
      * Remove -{" "} after <PipelineTag pipeline="question-answering" />
      
      * Change bitsnbytes to bitsandbytes
      
      * Revert formatting
      
      * Revert automatic format changes
      
      * Remove - sign after <PipelineTag pipeline="question-answering" />
      b0f0086f
    • Jannis Vamvas's avatar
      [WIP] Move X-MOD models to facebook organization (#21640) · 61abe329
      Jannis Vamvas authored
      Move X-MOD models to facebook org
      61abe329
  8. 15 Feb, 2023 3 commits
  9. 13 Feb, 2023 1 commit
  10. 10 Feb, 2023 1 commit
    • Jannis Vamvas's avatar
      Add X-MOD (#20939) · b0d539cc
      Jannis Vamvas authored
      
      
      * Add X-MOD to Readme
      
      * Add documentation for X-MOD
      
      * Implement X-MOD
      
      * Fix formatting of X-MOD docs
      
      * Change signature of X-MOD forward methods to use lang_ids
      
      * Minor changes
      
      * Rebase with main and run make fix-copies
      
      * Make suggested changes to docstrings
      
      * Improve code readability
      Co-authored-by: default avatarYounes Belkada <49240599+younesbelkada@users.noreply.github.com>
      
      * Fix code style
      
      * Conversion script: Remove asserts and type annotations
      
      * Remove _TOKENIZER_FOR_DOC
      
      * XMOD -> Xmod
      
      * Update copyright note
      
      * Fix doctests
      
      * Fix docstring
      
      * Add integration test for FillMaskPipeline
      
      * Revert "Add integration test for FillMaskPipeline"
      
      This reverts commit 4381eb3b1d0f5d85785f89caba83928e6efa6d1f.
      
      * Add end-to-end integration test for mask fill
      
      * make style
      
      * Rebase with main and make fix-copies
      
      ---------
      Co-authored-by: default avatarYounes Belkada <49240599+younesbelkada@users.noreply.github.com>
      b0d539cc
  11. 09 Feb, 2023 1 commit
    • NielsRogge's avatar
      Add BLIP-2 (#21441) · d7f1e7c0
      NielsRogge authored
      
      
      * First draft
      
      * More improvements
      
      * More improvements
      
      * Improve conversion script
      
      * Convert all weights
      
      * Make forward pass work
      
      * Make logits match
      
      * More improvements
      
      * More improvements
      
      * More improvements
      
      * Use get_input_embeddings
      
      * Improve some more
      
      * Improve model tests
      
      * Improve model tests
      
      * More improvements
      
      * Fix processor
      
      * Update files
      
      * Update prepare_inputs_for_generation
      
      * More improvements
      
      * Fix copies
      
      * More fixes
      
      * Make fixup
      
      * More improvements
      
      * Add support for seq2seq language model
      
      * More improvements
      
      * Fix test
      
      * More improvements
      
      * Improve conversion script
      
      * Remove some todo's
      
      * Fix README's
      
      * Improve conversion script
      
      * Fix generation
      
      * Fix style and remove Blip2Model
      
      * Fix model outputs
      
      * More improvements
      
      * Set eos_token_id in config
      
      * Fix quality
      
      * Small improvements
      
      * Add processor tests
      
      * More improvements
      
      * Apply suggestions
      
      * Apply suggestions
      
      * Add integration test
      
      * Update image URL
      
      * Add integration test
      
      * Fix model_type
      
      * Update style
      
      * Improve docs
      
      * Add doc tests
      
      * Fix copies
      
      * Remove tests which are passing
      
      * Improve some more
      
      * Add tests for seq2seq language models
      
      * Minor fix
      
      * Convert more checkpoints
      
      * finalize CI
      
      * Fix blip and blip2 processors
      
      * add `accelerate` support for `blip2`
      
      * clean up
      
      * make style
      
      * Update conversion script
      
      * Update conversion script some more
      
      * Update organization
      
      * revert toc file
      
      * add blip-2 to toc file
      
      * Some more improvements
      
      * Fix docstring
      
      * Improve docs
      
      ---------
      Co-authored-by: default avatarydshieh <ydshieh@users.noreply.github.com>
      Co-authored-by: default avataryounesbelkada <younesbelkada@gmail.com>
      d7f1e7c0
  12. 07 Feb, 2023 2 commits
    • Stefan Schweter's avatar
      Add XLM-V to Model Doc (#21498) · 7e51a441
      Stefan Schweter authored
      * doc: introduce new section for XLM-V model
      
      * doc: mention more details for XLM-V integration
      
      * docs: paper abstract in italics, model identifier for base model added
      
      * doc: mention new XLM-V support
      
      * auto: add XLM-V mapping
      
      * doc: run make fix-copies ;)
      7e51a441
    • Sylvain Gugger's avatar
      Deprecate parallelize API (#21448) · 5b493762
      Sylvain Gugger authored
      * Deprecate parallelize API
      
      * Add documentation
      
      * Fix copies
      5b493762
  13. 03 Feb, 2023 2 commits
    • Matthijs Hollemans's avatar
      [WIP] add SpeechT5 model (#18922) · e4bacf66
      Matthijs Hollemans authored
      * make SpeechT5 model by copying Wav2Vec2
      
      * add paper to docs
      
      * whoops added docs in wrong file
      
      * remove SpeechT5Tokenizer + put CTC back in the name
      
      * remove deprecated class
      
      * remove unused docstring
      
      * delete SpeechT5FeatureExtractor, use Wav2Vec2FeatureExtractor instead
      
      * remove classes we don't need right now
      
      * initial stab at speech encoder prenet
      
      * add more speech encoder prenet stuff
      
      * improve SpeechEncoderPrenet
      
      * add encoder (not finished yet)
      
      * add relative position bias to self-attention
      
      * add encoder CTC layers
      
      * fix formatting
      
      * add decoder from BART, doesn't work yet
      
      * make it work with generate loop
      
      * wrap the encoder into a speech encoder class
      
      * wrap the decoder in a text decoder class
      
      * changed my mind
      
      * changed my mind again ;-)
      
      * load decoder weights, make it work
      
      * add weights for text decoder postnet
      
      * add SpeechT5ForCTC model that uses only the encoder
      
      * clean up EncoderLayer and DecoderLayer
      
      * implement _init_weights in SpeechT5PreTrainedModel
      
      * cleanup config + Encoder and Decoder
      
      * add head + cross attention masks
      
      * improve doc comments
      
      * fixup
      
      * more cleanup
      
      * more fixup
      
      * TextDecoderPrenet works now, thanks Kendall
      
      * add CTC loss
      
      * add placeholders for other pre/postnets
      
      * add type annotation
      
      * fix freeze_feature_encoder
      
      * set padding tokens to 0 in decoder attention mask
      
      * encoder attention mask downsampling
      
      * remove features_pen calculation
      
      * disable the padding tokens thing again
      
      * fixup
      
      * more fixup
      
      * code review fixes
      
      * rename encoder/decoder wrapper classes
      
      * allow checkpoints to be loaded into SpeechT5Model
      
      * put encoder into wrapper for CTC model
      
      * clean up conversion script
      
      * add encoder for TTS model
      
      * add speech decoder prenet
      
      * add speech decoder post-net
      
      * attempt to reconstruct the generation loop
      
      * add speech generation loop
      
      * clean up generate_speech
      
      * small tweaks
      
      * fix forward pass
      
      * enable always dropout on speech decoder prenet
      
      * sort declaration
      
      * rename models
      
      * fixup
      
      * fix copies
      
      * more fixup
      
      * make consistency checker happy
      
      * add Seq2SeqSpectrogramOutput class
      
      * doc comments
      
      * quick note about loss and labels
      
      * add HiFi-GAN implementation (from Speech2Speech PR)
      
      * rename file
      
      * add vocoder to TTS model
      
      * improve vocoder
      
      * working on tokenizer
      
      * more better tokenizer
      
      * add CTC tokenizer
      
      * fix decode and batch_code in CTC tokenizer
      
      * fix processor
      
      * two processors and feature extractors
      
      * use SpeechT5WaveformFeatureExtractor instead of Wav2Vec2
      
      * cleanup
      
      * more cleanup
      
      * even more fixup
      
      * notebooks
      
      * fix log-mel spectrograms
      
      * support reduction factor
      
      * fixup
      
      * shift spectrograms to right to create decoder inputs
      
      * return correct labels
      
      * add labels for stop token prediction
      
      * fix doc comments
      
      * fixup
      
      * remove SpeechT5ForPreTraining
      
      * more fixup
      
      * update copyright headers
      
      * add usage examples
      
      * add SpeechT5ProcessorForCTC
      
      * fixup
      
      * push unofficial checkpoints to hub
      
      * initial version of tokenizer unit tests
      
      * add slow test
      
      * fix failing tests
      
      * tests for CTC tokenizer
      
      * finish CTC tokenizer tests
      
      * processor tests
      
      * initial test for feature extractors
      
      * tests for spectrogram feature extractor
      
      * fixup
      
      * more fixup
      
      * add decorators
      
      * require speech for tests
      
      * modeling tests
      
      * more tests for ASR model
      
      * fix imports
      
      * add fake tests for the other models
      
      * fixup
      
      * remove jupyter notebooks
      
      * add missing SpeechT5Model tests
      
      * add missing tests for SpeechT5ForCTC
      
      * add missing tests for SpeechT5ForTextToSpeech
      
      * sort tests by name
      
      * fix Hi-Fi GAN tests
      
      * fixup
      
      * add speech-to-speech model
      
      * refactor duplicate speech generation code
      
      * add processor for SpeechToSpeech model
      
      * add usage example
      
      * add tests for speech-to-speech model
      
      * fixup
      
      * enable gradient checkpointing for SpeechT5FeatureEncoder
      
      * code review
      
      * push_to_hub now takes repo_id
      
      * improve doc comments for HiFi-GAN config
      
      * add missing test
      
      * add integration tests
      
      * make number of layers in speech decoder prenet configurable
      
      * rename variable
      
      * rename variables
      
      * add auto classes for TTS and S2S
      
      * REMOVE CTC!!!
      
      * S2S processor does not support save/load_pretrained
      
      * fixup
      
      * these models are now in an auto mapping
      
      * fix doc links
      
      * rename HiFiGAN to HifiGan, remove separate config file
      
      * REMOVE auto classes
      
      * there can be only one
      
      * fixup
      
      * replace assert
      
      * reformat
      
      * feature extractor can process input and target at same time
      
      * update checkpoint names
      
      * fix commit hash
      e4bacf66
    • Avi Singhal's avatar
      Added model resources for LayoutLM Issue#19848 (#21377) · 0df80282
      Avi Singhal authored
      
      
      * updated resources for LayoutLM
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * fixed formatting, removed extra section
      
      ---------
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      0df80282
  14. 31 Jan, 2023 2 commits
    • NielsRogge's avatar
      [Docs] Minor fixes (#21383) · c21298a6
      NielsRogge authored
      
      
      * Improve docs
      
      * Add DETA resources
      
      ---------
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      c21298a6
    • NielsRogge's avatar
      Add DETA (#20983) · 5451f889
      NielsRogge authored
      * First draft
      
      * Add initial draft of conversion script
      
      * Convert all weights
      
      * Fix config
      
      * Add image processor
      
      * Fix DetaImageProcessor
      
      * Run make fix copies
      
      * Remove timm dependency
      
      * Fix dummy objects
      
      * Improve loss function
      
      * Remove conv_encoder attribute
      
      * Update conversion scripts
      
      * Improve postprocessing + docs
      
      * Fix copied from statements
      
      * Add tests
      
      * Improve postprocessing
      
      * Improve postprocessing
      
      * Update READMEs
      
      * More improvements
      
      * Fix rebase
      
      * Add is_torchvision_available
      
      * Add torchvision dependency
      
      * Fix typo and README
      
      * Fix bug
      
      * Add copied from
      
      * Fix style
      
      * Apply suggestions
      
      * Fix thanks to @ydshieh
      
      * Fix another dependency check
      
      * Simplify image processor
      
      * Add scipy
      
      * Improve code
      
      * Add threshold argument
      
      * Fix bug
      
      * Set default threshold
      
      * Improve integration test
      
      * Add another integration test
      
      * Update setup.py
      
      * Address review
      
      * Improve deformable attention function
      
      * Improve copied from
      
      * Use relative imports
      
      * Address review
      
      * Replace assertions
      
      * Address review
      
      * Update dummies
      
      * Remove dummies
      
      * Address comments, update READMEs
      
      * Remove custom kernel code
      
      * Add image processor tests
      
      * Add requires_backends
      
      * Add minor comment
      
      * Update scripts
      
      * Update organization name
      
      * Fix defaults, add doc tests
      
      * Add id2label for object 365
      
      * Fix tests
      
      * Update task guide
      5451f889
  15. 30 Jan, 2023 1 commit
  16. 25 Jan, 2023 2 commits
    • Anahita Bhiwandiwalla's avatar
      Add BridgeTower model (#20775) · 3a6e4a22
      Anahita Bhiwandiwalla authored
      
      
      * Commit with BTModel and latest HF code
      
      * Placeholder classes for BTForMLM and BTForITR
      
      * Importing Bert classes from transformers
      
      * Removed objectives.py and dist_utils.py
      
      * Removed swin_transformer.py
      
      * Add image normalization, BridgeTowerForImageAndTextRetrieval
      
      * Add center_crop
      
      * Removing bert tokenizer and LCI references
      
      * Tested config loading from HF transformers hub
      
      * Removed state_dict updates and added path to hub
      
      * Enable center crop
      
      * Getting image_size from config, renaming num_heads and num_layers
      
      * Handling max_length in BridgeTowerProcessor
      
      * Add BridgeTowerForMaskedLM
      
      * Add doc string for BridgeTowerConfig
      
      * Add doc strings for BT config, processor, image processor
      
      * Adding docs, removed swin
      
      * Removed convert_bridgetower_original_to_pytorch.py
      
      * Added doc files for bridgetower, removed is_vision
      
      * Add support attention_mask=None and BridgeTowerModelOutput
      
      * Fix formatting
      
      * Fixes with 'make style', 'make quality', 'make fixup'
      
      * Remove downstream tasks from BridgeTowerModel
      
      * Formatting fixes, add return_dict to BT models
      
      * Clean up after doc_test
      
      * Update BTModelOutput return type, fix todo in doc
      
      * Remove loss_names from init
      
      * implement tests and update tuples returned by models
      
      * Add image reference to bridgetower.mdx
      
      * after make fix-copies, make fixup, make style, make quality, make repo-consistency
      
      * Rename class names with BridgeTower prefix
      
      * Fix for image_size in BTImageProcessor
      
      * implement feature extraction bridgetower tests
      
      * Update image_mean and image_std to be list
      
      * remove unused import
      
      * Removed old comments
      
      * Rework CLIP
      
      * update config in tests followed config update
      
      * Formatting fixes
      
      * Add copied from for BridgeTowerPredictionHeadTransform
      
      * Update bridgetower.mdx
      
      * Update test_feature_extraction_bridgetower.py
      
      * Update bridgetower.mdx
      
      * BridgeTowerForMaskedLM is conditioned on image too
      
      * Add BridgeTowerForMaskedLM
      
      * Fixes
      
      * Call post_init to init weights
      
      * Move freeze layers into method
      
      * Remove BTFeatureExtractor, add BT under multimodal models
      
      * Remove BTFeatureExtractor, add BT under multimodal models
      
      * Code review feedback - cleanup
      
      * Rename variables
      
      * Formatting and style to PR review feedback
      
      * Move center crop after resize
      
      * Use named parameters
      
      * Style fix for modeling_bridgetower.py
      
      * Update docs/source/en/model_doc/bridgetower.mdx
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * Update docs/source/en/model_doc/bridgetower.mdx
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * Update docs/source/en/model_doc/bridgetower.mdx
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * Update src/transformers/models/bridgetower/modeling_bridgetower.py
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * Update src/transformers/models/bridgetower/modeling_bridgetower.py
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * Update docs/source/en/model_doc/bridgetower.mdx
      Co-authored-by: default avatarYounes Belkada <49240599+younesbelkada@users.noreply.github.com>
      
      * Update src/transformers/models/bridgetower/modeling_bridgetower.py
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * Rename config params, copy BERT classes, clean comments
      
      * Cleanup irtr
      
      * Replace Roberta imports, add BTTextConfig and Model
      
      * Update docs, add visionconfig, consistent arg names
      
      * make fixup
      
      * Comments for forward in BTModel and make fixup
      
      * correct tests
      
      * Remove inconsistent roberta copied from
      
      * Add BridgeTowerTextModel to dummy_pt_objects.py
      
      * Add BridgeTowerTextModel to IGNORE_NON_TESTED
      
      * Update docs for BT Text and Vision Configs
      
      * Treat BridgeTowerTextModel as a private model
      
      * BridgeTowerTextModel as private
      
      * Run make fix-copies
      
      * Adding BTTextModel to PRIVATE_MODELS
      
      * Fix for issue with BT Text and Image configs
      
      * make style changes
      
      * Update README_ja.md
      
      Add から to BridgeTower's description
      
      * Clean up config, .mdx and arg names
      
      * Fix init_weights. Remove nn.Sequential
      
      * Formatting and style fixes
      
      * Re-add tie_word_embeddings in config
      
      * update test implementation
      
      * update style
      
      * remove commented out
      
      * fix style
      
      * Update README with abs for BridgeTower
      
      * fix style
      
      * fix mdx file
      
      * Update bridgetower.mdx
      
      * Update img src in bridgetower.mdx
      
      * Update README.md
      
      * Update README.md
      
      * resolve style failed
      
      * Update _toctree.yml
      
      * Update README_ja.md
      
      * Removed mlp_ratio, rename feats, rename BTCLIPModel
      
      * Replace BTCLIP with BTVisionModel,pass in vision_config to BTVisionModel
      
      * Add test_initialization support
      
      * Add support for output_hidden_states
      
      * Update support for output_hidden_states
      
      * Add support for output_attentions
      
      * Add docstring for output_hidden_states
      
      * update tests
      
      * add bridgetowervisionmodel as private model
      
      * rerun the PR test
      
      * Remove model_type, pass configs to classes, renames
      
      * Change self.device to use weight device
      
      * Remove image_size
      
      * Style check fixes
      
      * Add hidden_size and num_hidden_layers to BridgeTowerTransformer
      
      * Update device setting
      
      * cosmetic update
      
      * trigger test again
      
      * trigger tests again
      
      * Update test_modeling_bridgetower.py
      
      trigger tests again
      
      * Update test_modeling_bridgetower.py
      
      * minor update
      
      * re-trigger tests
      
      * Update docs/source/en/model_doc/bridgetower.mdx
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * Remove pad, update max_text_len, doc cleanup, pass eps to LayerNorm
      
      * Added copied to, some more review feedback
      
      * make fixup
      
      * Use BridgeTowerVisionEmbeddings
      
      * Code cleanup
      
      * Fixes for BridgeTowerVisionEmbeddings
      
      * style checks
      
      * re-tests
      
      * fix embedding
      
      * address comment on init file
      
      * retrigger tests
      
      * update import prepare_image_inputs
      
      * update test_image_processing_bridgetower.py to reflect test_image_processing_common.py
      
      * retrigger tests
      Co-authored-by: default avatarShaoyen Tseng <shao-yen.tseng@intel.com>
      Co-authored-by: default avatarTiep Le <tiep.le@intel.com>
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      Co-authored-by: default avatarYounes Belkada <49240599+younesbelkada@users.noreply.github.com>
      Co-authored-by: default avatarTiep Le <97980157+tileintel@users.noreply.github.com>
      3a6e4a22
    • NielsRogge's avatar
      [Mask2Former] Add doc tests (#21232) · f83135eb
      NielsRogge authored
      
      
      * Add doc tests
      
      * Add OneFormer resourcesé
      
      * Fix merge
      
      * Fix style
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      f83135eb
  17. 24 Jan, 2023 1 commit
  18. 20 Jan, 2023 1 commit
  19. 19 Jan, 2023 2 commits
    • Clémentine Fourrier's avatar
      Graphormer model for Graph Classification (#20968) · 87208a05
      Clémentine Fourrier authored
      
      
      * [FT] First commit for graphormer architecture.
      
      The model has no tokenizer, as it uses a collator and preprocessing function for its input management.
      Architecture to be tested against original one.
      The arch might need to be changed to fit the checkpoint, but a revert to the original arch will make the code less nice to read.
      TODO: doc
      
      * [FIX] removed test model
      
      * [FIX] import error
      
      * [FIX] black and flake
      
      * [DOC] added paper refs
      
      * [FIX] [DOC]
      
      * [FIX] black
      
      * [DOC] Updated READMEs
      
      * [FIX] Order of imports + rm Tokenizer calls
      
      * [FIX] Moved assert in class to prevent doc build failure
      
      * [FIX] make fix-copies
      
      * [Doc] update from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * [FIX] Removed Graphormer from Sequence classification model list
      
      * [DOC] Added HF copyright to Cython file
      
      * [DOC] Fixed comments
      
      * [FIX] typos in class doc + removed config classes.
      
      Todo: update doc from paper definitions
      
      * [FIX] Removed dependency to fairseq, and replaced all asserts with Exception management
      
      * [FIX] Homogeneized initialization of weights to pretrained constructor
      
      * [FIX] [CP] Updated multi_hop parameter to get same results as in original implementation
      
      * [DOC] Relevant parameter description in the configuration file
      
      * [DOC] Updated doc and comments in main graphormer file
      
      * [FIX] make style and quality checks
      
      * [DOC] Fix doc format
      
      * [FIX] [WIP] Updated part of the tests, though still a wip
      
      * [FIX] [WIP]
      
      * [FIX] repo consistency
      
      * [FIX] Changed input names for more understandability
      
      * [FIX] [BUG] updated num_classes params for propagation in the model
      
      * simplified collator
      
      * [FIX] Updated tests to follow new naming pattern
      
      * [TESTS] Updated test suite along with model
      
      * |FIX] rm tokenizer import
      
      * [DOC] add link to graphormerdoc
      
      * Changed section in doc from text model to graph model
      
      * Apply suggestions from code review
      
      Spacing, inits
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * [DOC] Explain algos_graphormer functions
      
      * Cython soft import protection
      
      * Rm call to Callable in configuration graphormer
      
      * [FIX] replaced asserts with Exceptions
      
      * Add org to graphormer checkpoints
      
      * Prefixed classes with Graphormer
      
      * Management of init functions
      
      * format
      
      * fixes
      
      * fix length file
      
      * update indent
      
      * relaunching ci
      
      * Errors for missing cython imports
      
      * fix style
      
      * fix style doc
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      87208a05
    • Jitesh Jain's avatar
      Add OneFormer Model (#20577) · 5b949623
      Jitesh Jain authored
      * Add Oneformer Model
      
      * Add OneFormer Tests
      
      * Add UNIVERSAL_SEGMENTATION_MAPPING
      
      * Fix config
      
      * 🐛 Fix error encountered while writing tests
      
      * 🔨 Fix instance segmentation post processing
      
      * Format Files and Add Documentation
      
      * Add Documentation mdx file
      
      * Run make fixup
      
      * Run make fix-copies
      
      * Remove unnecessary code
      
      * Format modeling_oneformer.py
      
      * Add OneFormer to ImageSegmentationPipeline
      
      * Format files
      
      * Add Demo link to Readme
      
      * Fix fomatting errors
      
      * Fix test failures
      
      * Update Table in index.mdx
      
      * Fix version
      
      * Fix style
      
      * Remove OneFormer from TF
      
      * Fix Imports
      
      * Fix dummy objects
      
      * Fix tests
      
      * Add newline
      
      * Remove OneFormerFeatureExtractor
      
      * Remove CUDA Kernels
      
      * Use AutoBackbone for Swin
      
      * Fix description
      
      * Use Image Processor
      
      * Fix copies
      
      * Fix formatting
      
      * Fix import order
      
      * Fix flake8 errors
      
      * Fix doc errors
      
      * Add Hindi Readme entry
      
      * Update supported backbones
      
      * Update supported backbones
      
      * Undo Changes
      
      * Fix type of config
      
      * Fix isort
      
      * Fix auto.mdx
      
      * Fix swin config
      
      * Replace DinatBackbone with AutoBackbone
      
      * Use SwinBackbone
      
      * Use SwinBackbone
      
      * Fix conversion script
      
      * Fix arguments
      
      * Add argument description
      
      * Fix style
      
      * Add OneFormerProcessor
      
      * Fix OneFormerProcessor Tests
      
      * Fix mapping
      
      * Fix imports
      
      * Fix inits
      
      * Fix style
      
      * Fix comment
      
      * Fix docstring
      
      * Move OneFormer to MultiModal
      
      * Fix Copies
      
      * Remove size divisor
      
      * Fix check_repo.py
      
      * Fix copies
      
      * Add Processor for Testing Pipeline
      
      * Fix padding for tokens
      
      * Fix variables
      
      * Fix formatting with correct black version
      
      * Add Image Processor Test
      
      * Apply suggestions
      
      * Revert common modeling
      
      * Add check for task
      
      * Fix conversion script
      
      * Fix initialization order
      
      * Fix tests
      
      * Undo Pipeline Changes
      
      * Fix layers in MLP
      
      * Fix copies
      
      * Update image paths
      
      * Fix copies
      
      * Apply suggestions
      5b949623
  20. 18 Jan, 2023 2 commits
    • Samuel Xu's avatar
      Remove Roberta Dependencies from XLM Roberta Flax and Tensorflow models (#21047) · defdcd28
      Samuel Xu authored
      * Added flax model code
      
      * Added tf changes
      
      * missed some
      
      * Added copy comments
      
      * Added style hints
      
      * Fixed copy statements
      
      * Added suggested fixes
      
      * Made some fixes
      
      * Style fixup
      
      * Added necessary copy statements
      
      * Fixing copy statements
      
      * Added more copies
      
      * Final copy fix
      
      * Some bugfixes
      
      * Adding imports to init
      
      * Fixed up all make fixup errors
      
      * Fixed doc errors
      
      * Auto model changes
      defdcd28
    • Younes Belkada's avatar
      `blip` support for training (#21021) · 023f51fe
      Younes Belkada authored
      * `blip` support for training
      
      * remove labels creation
      
      * remove unneeded `decoder_input_ids` creation
      
      * final changes
      
      - add colab link to documentation
      - reduction = mean for loss
      
      * fix nits
      
      * update link
      
      * clearer error message
      023f51fe
  21. 17 Jan, 2023 2 commits
  22. 16 Jan, 2023 2 commits
    • Alara Dirik's avatar
    • NielsRogge's avatar
      Add UperNet (#20648) · 4ed89d48
      NielsRogge authored
      
      
      * First draft
      
      * More improvements
      
      * Add convnext backbone
      
      * Add conversion script
      
      * Add more improvements
      
      * Comment out to_dict
      
      * Add to_dict method
      
      * Add default config
      
      * Fix config
      
      * Fix backbone
      
      * Fix backbone some more
      
      * Add docs, auto mapping, tests
      
      * Fix some tests
      
      * Fix more tests
      
      * Fix more tests
      
      * Add conversion script
      
      * Improve conversion script
      
      * Add support for getting reshaped undownsampled hidden states
      
      * Fix forward pass
      
      * Add print statements
      
      * Comment out set_shift_and_window_size
      
      * More improvements
      
      * Correct downsampling layers conversion
      
      * Fix style
      
      * First draft
      
      * Fix conversion script
      
      * Remove config attribute
      
      * Fix more tests
      
      * Update READMEs
      
      * Update ConvNextBackbone
      
      * Fix ConvNext tests
      
      * Align ConvNext with Swin
      
      * Remove files
      
      * Fix index
      
      * Improve docs
      
      * Add output_attentions to model forward
      
      * Add backbone mixin, improve tests
      
      * More improvements
      
      * Update init_weights
      
      * Fix interpolation of logits
      
      * Add UperNetImageProcessor
      
      * Improve image processor
      
      * Fix image processor
      
      * Remove print statements
      
      * Remove script
      
      * Update import
      
      * Add image processor tests
      
      * Remove print statements
      
      * Fix test
      
      * Add integration test
      
      * Add convnext integration test
      
      * Update docstring
      
      * Fix README
      
      * Simplify config
      
      * Apply suggestions
      
      * Improve docs
      
      * Rename class
      
      * Fix test_initialization
      
      * Fix import
      
      * Address review
      
      * Fix confg
      
      * Convert all checkpoints
      
      * Fix default backbone
      
      * Usage same processor as segformer
      
      * Apply suggestions
      
      * Fix init_weights, update conversion scripts
      
      * Improve config
      
      * Use Auto API instead of creating a new image processor
      
      * Fix docs
      
      * Add doctests
      
      * Remove ResNetConfig dependency
      
      * Add always_partition argument
      
      * Fix rebaseé
      
      * Improve docs
      
      * Convert checkpoints
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MBP.localdomain>
      4ed89d48
  23. 04 Jan, 2023 1 commit