"docs/source/ko/tasks/token_classification.mdx" did not exist on "2b9513fdabbcfd3ca5d7003a955be633a2f365fc"
  1. 12 Oct, 2022 1 commit
    • NielsRogge's avatar
      Add LiLT (#19450) · 4d367a3c
      NielsRogge authored
      
      
      * First draft
      
      * Fix more things
      
      * Improve more things
      
      * Remove some head models
      
      * Fix more things
      
      * Add missing layers
      
      * Remove tokenizer
      
      * Fix more things
      
      * Fix copied from statements
      
      * Make all tests pass
      
      * Remove print statements
      
      * Remove files
      
      * Fix README and docs
      
      * Add integration test and fix organization
      
      * Add tips
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Make tests faster, improve docs
      
      * Fix doc tests
      
      * Add model to toctree
      
      * Add docs
      
      * Add note about creating new checkpoint
      
      * Remove is_decoder
      
      * Make tests smaller, add docs
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      4d367a3c
  2. 11 Oct, 2022 1 commit
    • Mathieu Jouffroy's avatar
      [CvT] Tensorflow implementation (#18597) · 5ca131f3
      Mathieu Jouffroy authored
      
      
      * implemented TFCvtModel and TFCvtForImageClassification and modified relevant files, added an exception in convert_tf_weight_name_to_pt_weight_name, added quick testing file to compare with pytorch model
      
      * added docstring + testing file in transformers testing suite
      
      * added test in testing file, modified docs to pass repo-consistency, passed formatting test
      
      * refactoring + passing all test
      
      * small refacto, removing unwanted comments
      
      * improved testing config
      
      * corrected import error
      
      * modified acces to pretrained model archive list, to pass tf_test
      
      * corrected import structure in init files
      
      * modified testing for keras_fit with cpu
      
      * correcting PR issues + Refactoring
      
      * Refactoring : improving readability and reducing the number of permutations
      
      * corrected momentum value + cls_token initialization
      
      * removed from_pt as weights were added to the hub
      
      * Update tests/models/cvt/test_modeling_tf_cvt.py
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      5ca131f3
  3. 10 Oct, 2022 2 commits
    • amyeroberts's avatar
      Add TF whisper (#19378) · e3f028f3
      amyeroberts authored
      
      
      * simplify loop
      
      * add featur extractor
      
      * add model
      
      * start conversion
      
      * add dropout
      
      * initial commit of test files
      
      * copnversion for all models
      
      * update processor for correct padding
      
      * update feature extraction
      
      * update integration test logits match
      
      * fmnt: off for the logits
      
      * on the fly mel bank
      
      * small nit
      
      * update test
      
      * update tokenizer
      
      * nit feature extraction
      
      * update
      
      * update tokenizer test
      
      * adds logit processor and update tokenizer to get supress tokens
      
      * style
      
      * clean convert
      
      * revert to original modeling tf utils
      
      * Update
      
      * update
      
      * nit
      
      * clean convert file
      
      * update tests and nits
      
      * quality
      
      * slow generation test
      
      * ffn_dim to allow customization
      
      * update readme
      
      * add to toctreee
      
      * start fixing integration tests
      
      * update tests and code
      
      * fix feature extractor
      
      * fix config tests common
      
      * update code to fix tests
      
      * fix feature exctractor
      
      * nit feature extraction
      
      * update test for new feature extractor
      
      * style
      
      * add absrtact
      
      * large logits wioth custom decoder input ids
      
      * wraap around is otrch available
      
      * fix feature extractor
      
      * correct logits for whisper small.en
      
      * nit
      
      * fix encoder_attentino_mask
      
      * some fixes
      
      * remove unnecessary inputs
      
      * nits
      
      * add normalizer file
      
      * update etst tokenization
      
      * fix attention mask not defined
      
      * fix generate
      
      * remove uncoder attention mask useless
      
      * update test modeling whisper
      
      * update condfig to add second non supress tokens
      
      * nits on feature exrtactor
      
      * nit for test tokenizers
      
      * update etsts
      
      * update tests
      
      * update tokenization test
      
      * fixup
      
      * invalidated hf token. Clean convert openai to whisper
      
      * fix logit tests
      
      * fixup
      
      * Add model to README
      
      * Fix doc tests
      
      * clean merge
      
      * revert toc_tree changes
      
      * remove useless LogitProcessor
      
      * Update whisper .mdx
      
      * update config file doc
      
      * update configuration docstring
      
      * update test tokenization
      
      * update test tokenization
      
      * update tokenization whisper
      Added copied from where needed
      
      * update feature extraction
      
      * nit test name
      
      * style
      
      * quality
      
      * remove get suppress tokens and update non_speech tokens global variables
      
      * Update src/transformers/models/whisper/feature_extraction_whisper.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * clean modeling whisper and test
      Removed the attention mask arguments that are deprecated
      
      * fix large test
      
      * Add multilingual audio test, and translate test
      
      * style
      
      * fix larg multilingual test
      
      * nits
      
      * add copied from for attention layer
      
      * remove attention masks in doc
      
      * add english normalizer
      
      * Update docs/source/en/model_doc/whisper.mdx
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * update tokenization test
      
      * remove copied from in whisper attention : no bias in k_proj only
      
      * wrap around dependencies in english normalizer
      
      * style
      
      * correct import generation logits
      
      * for now, wrap feature extractor with torch
      
      * remove torch depencies for feature extraction and style
      
      * Update src/transformers/models/whisper/convert_openai_whisper_to_tfms.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update docs/source/en/model_doc/whisper.mdx
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * fixup
      
      * nit
      
      * update logitds
      
      * style
      
      * nit
      
      * nits and fix final tests
      
      * add `is_more_itertools_available` to utils
      
      * quality
      
      * add begin supress tokens, supress tokens to generate args and config
      
      * clean supressTokensLogitProcessor in generation logits
      
      * Nit naming
      
      * add supressTokensAtBegin
      
      * udpate tests, supress tokens to None or correct values
      
      * nit and style
      
      * update RAG to fit test and generate_logit
      
      * add copy pasted statment on english normalizer
      
      * add arguments to config_common_kwargs
      
      * Update src/transformers/generation_utils.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update src/transformers/generation_logits_process.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * revert changes based on reviews
      
      * update doc and nits
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * more nits
      
      * last nits
      
      * update test configuration common
      
      * add BART name in decoder attention mask documentation
      
      * Update src/transformers/models/whisper/modeling_whisper.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * style
      
      * nit
      
      * nit
      
      * add english.json file to git
      
      * nits on documentation
      
      * nit
      
      * nits
      
      * last styling
      
      * add main toctree file
      
      * remove sentence piece dependency
      
      * clean init file
      
      * fix tokenizer that has no dependencies on sentencepiece
      
      * update whisper init file, nit
      
      * remove english.json file
      
      * add get decoder prompt id
      
      * All weights loading
      
      * Remove hanging pdb
      
      * Fixup and tidy up
      
      * Use same copied from as PT model
      
      * Remove whitespace changes
      
      * Remove torch references
      
      * Tie embeddings
      
      * Remove logits processor input to generate
      
      * Update logit values
      
      * revert changes and add forced logit processor
      
      * nit
      
      * clean normalizer
      
      * remove protected
      
      * Add logit processors and update generation code & tests
      
      * Some tidy up
      
      * Update docstring
      
      * update
      
      * update based on review
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update to reflect changes on the PT model branch
      
      * Tidy up
      
      * Remove extra whitespace
      
      * Fix test - make input ids small enough we can append
      
      * Include upstream changes on main
      
      * PR comments - add batch tests, remove comments & defaults
      
      * Fix model output imports
      
      * Update src/transformers/models/whisper/modeling_tf_whisper.py
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      
      * Update src/transformers/generation_tf_logits_process.py
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      
      * Update src/transformers/models/whisper/modeling_tf_whisper.py
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      
      * Update src/transformers/models/whisper/modeling_tf_whisper.py
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      
      * Update tests/models/whisper/test_modeling_tf_whisper.py
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      
      * Update src/transformers/models/whisper/modeling_tf_whisper.py
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      
      * Update src/transformers/models/whisper/modeling_tf_whisper.py
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      
      * Update docstring example
      
      * Update src/transformers/models/whisper/modeling_tf_whisper.py
      Co-authored-by: default avatarMatt <Rocketknight1@users.noreply.github.com>
      
      * Remove changes to adjust_logits_during_generation function
      
      * Update src/transformers/models/whisper/modeling_tf_whisper.py
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * Tidy up imports that don't require TF
      
      * Update tests - skip and no more skip
      
      * Update tests/generation/test_generation_tf_logits_process.py
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      
      * Update src/transformers/models/whisper/modeling_tf_whisper.py
      
      * Update src/transformers/models/whisper/modeling_tf_whisper.py
      Co-authored-by: default avatarMatt <Rocketknight1@users.noreply.github.com>
      
      * Add training flags
      
      * Add (skipped) XLA generation tests
      
      * Add embedding correctness test
      
      * Add constant ids for generation tests
      
      * Make logits finding a bit tidier
      
      * Remove unused args
      
      * xla generation enabled
      
      * Don't skip XLA tests anymore
      
      * Fix tests - add position ids to expected signature and update rag generation
      
      * Undo method reorder
      
      * Remove added whitespace
      
      * Remove copy-paste gradient checkopint ref
      
      * Remove
      
      * Trigger CI - (issue with refs when pulling)
      Co-authored-by: default avatarArthur Zucker <arthur.zucker@gmail.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarNielsRogge <niels.rogge1@gmail.com>
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      Co-authored-by: default avatarMatt <Rocketknight1@users.noreply.github.com>
      Co-authored-by: default avatarJoao Gante <joao@huggingface.co>
      e3f028f3
    • APAVOU Cl茅ment's avatar
      Add `OPTForQuestionAnswering` (#19402) · af69360b
      APAVOU Cl茅ment authored
      * Add `OPTForQuestionAnswering`
      
      - added `OPTForQuestionAnswering` class based on `BloomForQuestionAnswering`
      - added `OPTForQuestionAnswering` in common tests
      - all common tests pass
      - make fixup done
      
      * added docstrings for OPTForQuestionAnswering
      
      * Fix docstrings for OPTForQuestionAnswering
      af69360b
  4. 07 Oct, 2022 1 commit
    • Amrit Sahu's avatar
      [WIP] Add ZeroShotObjectDetectionPipeline (#18445) (#18930) · e9a49bab
      Amrit Sahu authored
      * Add ZeroShotObjectDetectionPipeline (#18445)
      
      * Add AutoModelForZeroShotObjectDetection task
      
      This commit also adds the following
      
      - Add explicit _processor method for ZeroShotObjectDetectionPipeline.
        This is necessary as pipelines don't auto infer processors yet and
        `OwlVitProcessor` wraps tokenizer and feature_extractor together, to
        process multiple images at once
      
      - Add auto tests and other tests for ZeroShotObjectDetectionPipeline
      
      * Add AutoModelForZeroShotObjectDetection task
      
      This commit also adds the following
      
      - Add explicit _processor method for ZeroShotObjectDetectionPipeline.
        This is necessary as pipelines don't auto infer processors yet and
        `OwlVitProcessor` wraps tokenizer and feature_extractor together, to
        process multiple images at once
      
      - Add auto tests and other tests for ZeroShotObjectDetectionPipeline
      
      * Add batching for ZeroShotObjectDetectionPipeline
      
      * Fix doc-string ZeroShotObjectDetectionPipeline
      
      * Fix output format: ZeroShotObjectDetectionPipeline
      e9a49bab
  5. 06 Oct, 2022 1 commit
  6. 05 Oct, 2022 2 commits
    • Arthur's avatar
      Add WhisperModel to transformers (#19166) · 45e14038
      Arthur authored
      
      
      * simplify loop
      
      * add featur extractor
      
      * add model
      
      * start conversion
      
      * add dropout
      
      * initial commit of test files
      
      * copnversion for all models
      
      * update processor for correct padding
      
      * update feature extraction
      
      * update integration test logits match
      
      * fmnt: off for the logits
      
      * on the fly mel bank
      
      * small nit
      
      * update test
      
      * update tokenizer
      
      * nit feature extraction
      
      * update
      
      * update tokenizer test
      
      * adds logit processor and update tokenizer to get supress tokens
      
      * style
      
      * clean convert
      
      * revert to original modeling tf utils
      
      * Update
      
      * update
      
      * nit
      
      * clean convert file
      
      * update tests and nits
      
      * quality
      
      * slow generation test
      
      * ffn_dim to allow customization
      
      * update readme
      
      * add to toctreee
      
      * start fixing integration tests
      
      * update tests and code
      
      * fix feature extractor
      
      * fix config tests common
      
      * update code to fix tests
      
      * fix feature exctractor
      
      * nit feature extraction
      
      * update test for new feature extractor
      
      * style
      
      * add absrtact
      
      * large logits wioth custom decoder input ids
      
      * wraap around is otrch available
      
      * fix feature extractor
      
      * correct logits for whisper small.en
      
      * nit
      
      * fix encoder_attentino_mask
      
      * some fixes
      
      * remove unnecessary inputs
      
      * nits
      
      * add normalizer file
      
      * update etst tokenization
      
      * fix attention mask not defined
      
      * Add model to README
      
      * Fix doc tests
      
      * fix generate
      
      * remove uncoder attention mask useless
      
      * update test modeling whisper
      
      * update condfig to add second non supress tokens
      
      * nits on feature exrtactor
      
      * nit for test tokenizers
      
      * update etsts
      
      * update tests
      
      * update tokenization test
      
      * fixup
      
      * invalidated hf token. Clean convert openai to whisper
      
      * fix logit tests
      
      * fixup
      
      * clean merge
      
      * revert toc_tree changes
      
      * remove useless LogitProcessor
      
      * Update whisper .mdx
      
      * update config file doc
      
      * update configuration docstring
      
      * update test tokenization
      
      * update test tokenization
      
      * update tokenization whisper
      Added copied from where needed
      
      * update feature extraction
      
      * nit test name
      
      * style
      
      * quality
      
      * remove get suppress tokens and update non_speech tokens global variables
      
      * Update src/transformers/models/whisper/feature_extraction_whisper.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * clean modeling whisper and test
      Removed the attention mask arguments that are deprecated
      
      * fix large test
      
      * Add multilingual audio test, and translate test
      
      * style
      
      * fix larg multilingual test
      
      * nits
      
      * Update docs/source/en/model_doc/whisper.mdx
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * add copied from for attention layer
      
      * remove attention masks in doc
      
      * add english normalizer
      
      * update tokenization test
      
      * remove copied from in whisper attention : no bias in k_proj only
      
      * wrap around dependencies in english normalizer
      
      * style
      
      * correct import generation logits
      
      * for now, wrap feature extractor with torch
      
      * Update src/transformers/models/whisper/convert_openai_whisper_to_tfms.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update docs/source/en/model_doc/whisper.mdx
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * remove torch depencies for feature extraction and style
      
      * fixup
      
      * nit
      
      * update logitds
      
      * style
      
      * nit
      
      * nits and fix final tests
      
      * add `is_more_itertools_available` to utils
      
      * quality
      
      * add begin supress tokens, supress tokens to generate args and config
      
      * clean supressTokensLogitProcessor in generation logits
      
      * Nit naming
      
      * add supressTokensAtBegin
      
      * udpate tests, supress tokens to None or correct values
      
      * nit and style
      
      * update RAG to fit test and generate_logit
      
      * add copy pasted statment on english normalizer
      
      * add arguments to config_common_kwargs
      
      * Update src/transformers/generation_utils.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update src/transformers/generation_logits_process.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * revert changes based on reviews
      
      * update doc and nits
      
      * more nits
      
      * last nits
      
      * update test configuration common
      
      * add BART name in decoder attention mask documentation
      
      * Update src/transformers/models/whisper/modeling_whisper.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * style
      
      * nit
      
      * nit
      
      * add english.json file to git
      
      * nits on documentation
      
      * nit
      
      * nits
      
      * last styling
      
      * add main toctree file
      
      * remove sentence piece dependency
      
      * clean init file
      
      * fix tokenizer that has no dependencies on sentencepiece
      
      * update whisper init file, nit
      
      * remove english.json file
      
      * add get decoder prompt id
      
      * revert changes and add forced logit processor
      
      * nit
      
      * clean normalizer
      
      * remove protected
      
      * update
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * update based on review
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * add batched tests
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarNielsRogge <niels.rogge1@gmail.com>
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      45e14038
    • Alara Dirik's avatar
      Maskformer post-processing fixes and improvements (#19172) · 07e94bf1
      Alara Dirik authored
      - Improves MaskFormer docs, corrects minor typos
      - Restructures MaskFormerFeatureExtractor.post_process_panoptic_segmentation for better readability, adds target_sizes argument for optional resizing
      - Adds post_process_semantic_segmentation and post_process_instance_segmentation methods.
      - Adds a deprecation warning to post_process_segmentation method in favour of post_process_instance_segmentation
      07e94bf1
  7. 04 Oct, 2022 1 commit
  8. 03 Oct, 2022 1 commit
  9. 30 Sep, 2022 3 commits
    • Kashif Rasul's avatar
      time series forecasting model (#17965) · 5cd16f01
      Kashif Rasul authored
      
      
      * initial files
      
      * initial model via cli
      
      * typos
      
      * make a start on the model config
      
      * ready with configuation
      
      * remove tokenizer ref.
      
      * init the transformer
      
      * added initial model forward to return dec_output
      
      * require gluonts
      
      * update dep. ver table and add as extra
      
      * fixed typo
      
      * add type for prediction_length
      
      * use num_time_features
      
      * use config
      
      * more config
      
      * typos
      
      * opps another typo
      
      * freq can be none
      
      * default via transformation is 1
      
      * initial transformations
      
      * fix imports
      
      * added transform_start_field
      
      * add helper to create pytorch dataloader
      
      * added inital val and test data loader
      
      * added initial distr head and loss
      
      * training working
      
      * remove TimeSeriesTransformerTokenizer
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update src/transformers/__init__.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update src/transformers/models/time_series_transformer/__init__.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * fixed copyright
      
      * removed docs
      
      * remove time series tokenizer
      
      * fixed docs
      
      * fix text
      
      * fix second
      
      * fix default
      
      * fix order
      
      * use config directly
      
      * undo change
      
      * fix comment
      
      * fix year
      
      * fix import
      
      * add additional arguments for training vs. test
      
      * initial greedy inference loop
      
      * fix inference
      
      * comment out token inputs to enc dec
      
      * Use HF encoder/decoder
      
      * fix inference
      
      * Use Seq2SeqTSModelOutput output
      
      * return Seq2SeqTSPredictionOutput
      
      * added default arguments
      
      * fix return_dict true
      
      * scale is a tensor
      
      * output static_features for inference
      
      * clean up some unused bits
      
      * fixed typo
      
      * set return_dict if none
      
      * call model once for both train/predict
      
      * use cache if future_target is none
      
      * initial generate func
      
      * generate arguments
      
      * future_time_feat is required
      
      * return SampleTSPredictionOutput
      
      * removed unneeded classes
      
      * fix when params is none
      
      * fix return dict
      
      * fix num_attention_heads
      
      * fix arguments
      
      * remove unused shift_tokens_right
      
      * add different dropout configs
      
      * implement FeatureEmbedder, Scaler and weighted_average
      
      * remove gluonts dependency
      
      * fix class names
      
      * avoid _variable names
      
      * remove gluonts dependency
      
      * fix imports
      
      * remove gluonts from configuration
      
      * fix docs
      
      * fixed typo
      
      * move utils to examples
      
      * add example requirements
      
      * config has no freq
      
      * initial run_ts_no_trainer
      
      * remove from ignore
      
      * fix output_attentions and removed unsued getters/setters
      
      * removed unsed tests
      
      * add dec seq len
      
      * add test_attention_outputs
      
      * set has_text_modality=False
      
      * add config attribute_map
      
      * make style
      
      * make fix-copies
      
      * add encoder_outputs to TimeSeriesTransformerForPrediction forward
      
      * Improve docs, add model to README
      
      * added test_forward_signature
      
      * More improvements
      
      * Add more copied from
      
      * Fix README
      
      * Fix remaining quality issues
      
      * updated encoder and decoder
      
      * fix generate
      
      * output_hidden_states and use_cache are optional
      
      * past key_values returned too
      
      * initialize weights of distribution_output module
      
      * fixed more tests
      
      * update test_forward_signature
      
      * fix return_dict outputs
      
      * Update src/transformers/models/time_series_transformer/configuration_time_series_transformer.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/time_series_transformer/configuration_time_series_transformer.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/time_series_transformer/configuration_time_series_transformer.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/time_series_transformer/configuration_time_series_transformer.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/time_series_transformer/modeling_time_series_transformer.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/time_series_transformer/modeling_time_series_transformer.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/time_series_transformer/modeling_time_series_transformer.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * removed commented out tests
      
      * added neg. bin and normal output
      
      * Update src/transformers/models/time_series_transformer/configuration_time_series_transformer.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * move to one line
      
      * Add docstrings
      
      * Update src/transformers/models/time_series_transformer/configuration_time_series_transformer.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * add try except for assert and raise
      
      * try and raise exception
      
      * fix the documentation formatting
      
      * fix assert call
      
      * fix docstring formatting
      
      * removed input_ids from DOCSTRING
      
      * Update input docstring
      
      * Improve variable names
      
      * Update order of inputs
      
      * Improve configuration
      
      * Improve variable names
      
      * Improve docs
      
      * Remove key_length from tests
      
      * Add extra docs
      
      * initial unittests
      
      * added test_inference_no_head test
      
      * added test_inference_head
      
      * add test_seq_to_seq_generation
      
      * make style
      
      * one line
      
      * assert mean prediction
      
      * removed comments
      
      * Update src/transformers/models/time_series_transformer/modeling_time_series_transformer.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update src/transformers/models/time_series_transformer/modeling_time_series_transformer.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * fix order of args
      
      * make past_observed_mask optional as well
      
      * added Amazon license header
      
      * updated utils with new fieldnames
      
      * make style
      
      * cleanup
      
      * undo position of past_observed_mask
      
      * fix import
      
      * typo
      
      * more typo
      
      * rename example files
      
      * remove example for now
      
      * Update docs/source/en/_toctree.yml
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/time_series_transformer/configuration_time_series_transformer.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/time_series_transformer/modeling_time_series_transformer.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/models/time_series_transformer/modeling_time_series_transformer.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update modeling_time_series_transformer.py
      
      fix style
      
      * fixed typo
      
      * fix typo and grammer
      
      * fix style
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      Co-authored-by: default avatarNielsRogge <niels.rogge1@gmail.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      5cd16f01
    • Matt's avatar
      Rebase ESM PR and update all file formats (#19055) · 368b649a
      Matt authored
      
      
      * Rebase ESM PR and update all file formats
      
      * Fix test relative imports
      
      * Add __init__.py to the test dir
      
      * Disable gradient checkpointing
      
      * Remove references to TFESM... FOR NOW >:|
      
      * Remove completed TODOs from tests
      
      * Convert docstrings to mdx, fix-copies from BERT
      
      * fix-copies for the README and index
      
      * Update ESM's __init__.py to the modern format
      
      * Add to _toctree.yml
      
      * Ensure we correctly copy the pad_token_id from the original ESM model
      
      * Ensure we correctly copy the pad_token_id from the original ESM model
      
      * Tiny grammar nitpicks
      
      * Make the layer norm after embeddings an optional flag
      
      * Make the layer norm after embeddings an optional flag
      
      * Update the conversion script to handle other model classes
      
      * Remove token_type_ids entirely, fix attention_masking and add checks to convert_esm.py
      
      * Break the copied from link from BertModel.forward to remove token_type_ids
      
      * Remove debug array saves
      
      * Begin ESM-2 porting
      
      * Add a hacky workaround for the precision issue in original repo
      
      * Code cleanup
      
      * Remove unused checkpoint conversion code
      
      * Remove unused checkpoint conversion code
      
      * Fix copyright notices
      
      * Get rid of all references to the TF weights conversion
      
      * Remove token_type_ids from the tests
      
      * Fix test code
      
      * Update src/transformers/__init__.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update src/transformers/__init__.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Update README.md
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Add credit
      
      * Remove _ args and __ kwargs in rotary embedding
      
      * Assertively remove asserts
      
      * Replace einsum with torch.outer()
      
      * Fix docstring formatting
      
      * Remove assertions in tokenization
      
      * Add paper citation to ESMModel docstring
      
      * Move vocab list to single line
      
      * Remove ESMLayer from init
      
      * Add Facebook copyrights
      
      * Clean up RotaryEmbedding docstring
      
      * Fix docstring formatting
      
      * Fix docstring for config object
      
      * Add explanation for new config methods
      
      * make fix-copies
      
      * Rename all the ESM- classes to Esm-
      
      * Update conversion script to allow pushing to hub
      
      * Update tests to point at my repo for now
      
      * Set config properly for tests
      
      * Remove the gross hack that forced loss of precision in inv_freq and instead copy the data from the model being converted
      
      * make fixup
      
      * Update expected values for slow tests
      
      * make fixup
      
      * Remove EsmForCausalLM for now
      
      * Remove EsmForCausalLM for now
      
      * Fix padding idx test
      
      * Updated README and docs with ESM-1b and ESM-2 separately (#19221)
      
      * Updated README and docs with ESM-1b and ESM-2 separately
      
      * Update READMEs, longer entry with 3 citations
      
      * make fix-copies
      Co-authored-by: default avatarYour Name <you@example.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarTom Sercu <tsercu@fb.com>
      Co-authored-by: default avatarYour Name <you@example.com>
      368b649a
    • NielsRogge's avatar
      Add MarkupLM (#19198) · f3d2f7a6
      NielsRogge authored
      
      
      * First draft
      
      * Make basic test work
      
      * Fix most tokenizer tests
      
      * More improvements
      
      * Make more tests pass
      
      * Fix more tests
      
      * Fix some code quality
      
      * Improve truncation
      
      * Implement feature extractor
      
      * Improve feature extractor and add tests
      
      * Improve feature extractor tests
      
      * Fix pair_input test partly
      
      * Add fast tokenizer
      
      * Improve implementation
      
      * Fix rebase
      
      * Fix rebase
      
      * Fix most of the tokenizer tests.
      
      * propose solution for fast
      
      * add: integration test for fasttokenizer, warning for decode, fix template in slow tokenizer
      
      * add: modify markuplmconverter
      
      * add: some modify on converter and tokenizerfast
      
      * Fix style, copies
      
      * Make fixup
      
      * Update tokenization_markuplm.py
      
      * Update test_tokenization_markuplm.py
      
      * Update markuplm related
      
      * Improve processor, add integration test
      
      * Add processor test file
      
      * Improve processor
      
      * Improve processor tests
      
      * Fix more processor tests
      
      * Fix processor tests
      
      * Update docstrings
      
      * Add Copied from statements
      
      * Add more Copied from statements
      
      * Add code examples
      
      * Improve code examples
      
      * Add model to doc tests
      
      * Adding dependency check
      
      * Add dummy file
      
      * Add requires_backends
      
      * Add model to toctree
      
      * Fix more things, disable dependency check for now
      
      * Apply more suggestions
      
      * Add soft dependency
      
      * Add annotators to tests
      
      * Fix style
      
      * Remove from_slow=True
      
      * Remove print statements
      
      * Add sanity check
      
      * Fix processor test
      
      * Fix processor tests, add more docs
      
      * Add doc tests for mdx file
      
      * Add more tips
      
      * Apply suggestions
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      Co-authored-by: default avatarlockon-n <45759388+lockon-n@users.noreply.github.com>
      Co-authored-by: default avatarSaulLu <lucilesaul.com@gmail.com>
      Co-authored-by: default avatarlockon-n <dd098309@126.com>
      f3d2f7a6
  10. 29 Sep, 2022 2 commits
    • mustapha ajeghrir's avatar
      Fix `m2m_100.mdx` doc example missing `labels` (#19149) · ba9e336f
      mustapha ajeghrir authored
      The `labels` variable is not defined, the `model_inputs` already contain this information.
      ba9e336f
    • Aritra Roy Gosthipaty's avatar
      [TensorFlow] Adding GroupViT (#18020) · 0dc7b3a7
      Aritra Roy Gosthipaty authored
      
      
      * chore: initial commit
      
      * chore: adding util methods
      
      yet to work on the nn.functional.interpolate port with align_corener=True
      
      * chore: refactor the utils
      
      * used tf.compat.v1.image.resize to align the F.interpolate function
      * added type hints to the method signatures
      * added references to the gists where one 2 one alignment of torch and tf has been shown
      
      * chore: adding the layers
      
      * chore: porting all the layers from torch to tf
      
      This is the initial draft, nothing is tested yet.
      
      * chore: aligning the layers with reference to tf clip
      
      * chore: aligning the modules
      
      * added demaraction comments
      * added copied and adapted from comments
      
      * chore: aligning with CLIP
      
      * chore: wrangling the layers to keep it tf compatible
      
      * chore: aligning the names of the layers for porting
      
      * chore: style changes
      
      * chore: adding docs and inits
      
      * chore: adding tfp dependencis
      
      the code is taken from TAPAS
      
      * chore: initial commit for testing
      
      * chore: aligning the vision embeddings with the vit implementatino
      
      * chore: changing model prefix
      
      * chore: fixing the name of the model and the layer normalization test case
      
      * chore: every test passes but the slow ones
      
      * chore: fix style and integration test
      
      * chore: moving comments below decorators
      
      * chore: make fixup and fix-copies changes
      
      * chore: adding the Vision and Text Model to check_repo
      
      * chore: modifying the prefix name to align it with the torch implementation
      
      * chore: fix typo in configuration
      
      * choer: changing the name of the model variable
      
      * chore: adding segmentation flag
      
      * chore: gante's review
      
      * chore: style refactor
      
      * chore: amy review
      
      * chore: adding shape_list to parts that have been copied from other snippets
      
      * chore: init batchnorm with torch defaults
      
      * chore: adding shape_list to pass the tests
      
      * test fix: adding seed as 0
      
      * set seed
      
      * chore: changing the straight through trick to fix -ve dimensinos
      
      * chore: adding a dimension to the loss
      
      * chore: adding reviewers and contributors names to the docs
      
      * chore: added changes after review
      
      * chore: code quality fixup
      
      * chore: fixing the segmentation snippet
      
      * chore: adding  to the layer calls
      
      * chore: changing int32 to int64 for inputs of serving
      
      * chore: review changes
      
      * chore: style changes
      
      * chore: remove from_pt=True
      
      * fix: repo consistency
      Co-authored-by: default avatarydshieh <ydshieh@users.noreply.github.com>
      0dc7b3a7
  11. 23 Sep, 2022 1 commit
  12. 22 Sep, 2022 4 commits
  13. 21 Sep, 2022 2 commits
  14. 20 Sep, 2022 1 commit
  15. 19 Sep, 2022 1 commit
  16. 14 Sep, 2022 2 commits
    • Shinya Otani's avatar
      Add support for Japanese GPT-NeoX-based model by ABEJA, Inc. (#18814) · f5f430e5
      Shinya Otani authored
      * add gpt-neox-japanese model and tokenizer as new model
      
      * Correction to PR's comment for GPT NeoX Japanese
      - Fix to be able to use gpu
      - Add comment # Copied... at the top of RotaryEmbedding
      - Implement nn.Linear instead of original linear class
      - Add generation test under @slow
      
      * fix bias treatment for gpt-neox-japanese
      
      * Modidy gpt-neox-japanese following PR
      - add doc for bias_dropout_add
      - style change following a PR comment
      
      * add document for gpt-neox-japanese
      
      * remove unused import from gpt-neox-japanese
      
      * fix README for gpt-neox-japanese
      f5f430e5
    • NielsRogge's avatar
      Add Deformable DETR (#17281) · 59407bbe
      NielsRogge authored
      
      
      * First draft
      
      * More improvements
      
      * Improve model, add custom CUDA code
      
      * Import torch before
      
      * Add script that imports custom layer
      
      * Add everything in new ops directory
      
      * Import custom layer in modeling file
      
      * Fix ARCHIVE_MAP typo
      
      * Creating the custom kernel on the fly.
      
      * Import custom layer in modeling file
      
      * More improvements
      
      * Fix CUDA loading
      
      * More improvements
      
      * Improve conversion script
      
      * Improve conversion script
      
      * Make it work until encoder_outputs
      
      * Make forward pass work
      
      * More improvements
      
      * Make logits match original implementation
      
      * Make implementation also support single_scale model
      
      * Add support for single_scale and dilation checkpoint
      
      * Add support for with_box_refine model
      
      * Support also two stage model
      
      * Improve tests
      
      * Fix more tests
      
      * Make more tests pass
      
      * Upload all models to the hub
      
      * Clean up some code
      
      * Improve decoder outputs
      
      * Rename intermediate hidden states and reference points
      
      * Improve model outputs
      
      * Move tests to dedicated folder
      
      * Improve model outputs
      
      * Fix retain_grad test
      
      * Improve docs
      
      * Clean up and make test_initialization pass
      
      * Improve variable names
      
      * Add copied from statements
      
      * Improve docs
      
      * Fix style
      
      * Improve docs
      
      * Improve docs, move tests to model folder
      
      * Fix rebase
      
      * Remove DetrForSegmentation from auto mapping
      
      * Apply suggestions from code review
      
      * Improve variable names and docstrings
      
      * Apply some more suggestions from code review
      
      * Apply suggestion from code review
      
      * better docs and variables names
      
      * hint to num_queries and two_stage confusion
      
      * remove asserts and code refactor
      
      * add exception if two_stage is True and with_box_refine is False
      
      * use f-strings
      
      * Improve docs and variable names
      
      * Fix code quality
      
      * Fix rebase
      
      * Add require_torch_gpu decorator
      
      * Add pip install ninja to CI jobs
      
      * Apply suggestion of @sgugger
      
      * Remove DeformableDetrForObjectDetection from auto mapping
      
      * Remove DeformableDetrModel from auto mapping
      
      * Add model to toctree
      
      * Add model back to mappings, skip model in pipeline tests
      
      * Apply @sgugger's suggestion
      
      * Fix imports in the init
      
      * Fix copies
      
      * Add CPU implementation
      
      * Comment out GPU function
      
      * Undo previous change
      
      * Apply more suggestions
      
      * Remove require_torch_gpu annotator
      
      * Fix quality
      
      * Add logger.info
      
      * Fix logger
      
      * Fix variable names
      
      * Fix initializaztion
      
      * Add missing initialization
      
      * Update checkpoint name
      
      * Add model to doc tests
      
      * Add CPU/GPU equivalence test
      
      * Add Deformable DETR to pipeline tests
      
      * Skip model for object detection pipeline
      Co-authored-by: default avatarNicolas Patry <patry.nicolas@protonmail.com>
      Co-authored-by: default avatarNouamane Tazi <nouamane98@gmail.com>
      Co-authored-by: default avatarSylvain Gugger <Sylvain.gugger@gmail.com>
      59407bbe
  17. 09 Sep, 2022 1 commit
  18. 08 Sep, 2022 1 commit
    • NielsRogge's avatar
      Add X-CLIP (#18852) · bb6f6d53
      NielsRogge authored
      * First draft
      
      * Improve conversion script
      
      * Make vision encoder work
      
      * More improvements
      
      * Improve conversion script
      
      * Fix quality
      
      * Add MultiframeIntegrationTransformer
      
      * More improvements
      
      * Make MiT output work
      
      * Fix quality
      
      * Add prompts generator
      
      * Add tests
      
      * Fix some tests
      
      * Fix some more tests
      
      * Fix more tests
      
      * Improve conversion script
      
      * Fix model outputs
      
      * Fix more tests
      
      * Add XClipProcessor
      
      * Use processor in conversion script
      
      * Fix integration test
      
      * Update README, fix docs
      
      * Fix all tests
      
      * Add MIT output to XClipOutput
      
      * Create better variable names
      
      * Rename XClip to XCLIP
      
      * Extend conversion script
      
      * Add support for large models
      
      * Add support for 16 frame models
      
      * Add another model'
      
      * Fix module issue
      
      * Apply suggestions from code review
      
      * Add figure to docs
      
      * Fix CLIPProcessor issue
      
      * Apply suggestions from code review
      
      * Delete file
      
      * Convert more checkpoints
      
      * Convert last checkpoint
      
      * Update nielsr to microsoft
      bb6f6d53
  19. 07 Sep, 2022 1 commit
    • Ankur Goyal's avatar
      Add DocumentQuestionAnswering pipeline (#18414) · 2ef77421
      Ankur Goyal authored
      
      
      * [WIP] Skeleton of VisualQuestionAnweringPipeline extended to support LayoutLM-like models
      
      * Fixup
      
      * Use the full encoding
      
      * Basic refactoring to DocumentQuestionAnsweringPipeline
      
      * Cleanup
      
      * Improve args, docs, and implement preprocessing
      
      * Integrate OCR
      
      * Refactor question_answering pipeline
      
      * Use refactored QA code in the document qa pipeline
      
      * Fix tests
      
      * Some small cleanups
      
      * Use a string type annotation for Image.Image
      
      * Update encoding with image features
      
      * Wire through the basic docs
      
      * Handle invalid response
      
      * Handle empty word_boxes properly
      
      * Docstring fix
      
      * Integrate Donut model
      
      * Fixup
      
      * Incorporate comments
      
      * Address comments
      
      * Initial incorporation of tests
      
      * Address Comments
      
      * Change assert to ValueError
      
      * Comments
      
      * Wrap `score` in float to make it JSON serializable
      
      * Incorporate AutoModeLForDocumentQuestionAnswering changes
      
      * Fixup
      
      * Rename postprocess function
      
      * Fix auto import
      
      * Applying comments
      
      * Improve docs
      
      * Remove extra assets and add copyright
      
      * Address comments
      Co-authored-by: default avatarAnkur Goyal <ankur@impira.com>
      2ef77421
  20. 06 Sep, 2022 1 commit
  21. 02 Sep, 2022 2 commits
    • Jason Phang's avatar
      PEGASUS-X (#18551) · 53e33e6f
      Jason Phang authored
      * PegasusX Initial commit
      
      * rename
      
      * pegasus X implementation
      
      * pegx update
      
      * pegx fix
      
      * pegasus-x fixes
      
      * pegx updates
      
      * cleanup
      
      * cleanup
      
      * cleanup
      
      * tests
      
      * stylefixes
      
      * Documentation update
      
      * Model hub fix
      
      * cleanup
      
      * update
      
      * update
      
      * testfix
      
      * Check fix
      
      * tweaks for merging
      
      * style
      
      * style
      
      * updates for pr
      
      * style
      
      * change pegasus-x repo
      53e33e6f
    • NielsRogge's avatar
      [LayoutLM] Add clarification to docs (#18716) · c60dd98e
      NielsRogge authored
      
      
      * Add clarification
      
      * Add another clarification
      
      * Apply suggestion
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      c60dd98e
  22. 01 Sep, 2022 1 commit
  23. 31 Aug, 2022 1 commit
    • Ankur Goyal's avatar
      Add LayoutLMForQuestionAnswering model (#18407) · 5c4c8690
      Ankur Goyal authored
      
      
      * Add LayoutLMForQuestionAnswering model
      
      * Fix output
      
      * Remove TF TODOs
      
      * Add test cases
      
      * Add docs
      
      * TF implementation
      
      * Fix PT/TF equivalence
      
      * Fix loss
      
      * make fixup
      
      * Fix up documentation code examples
      
      * Fix up documentation examples + test them
      
      * Remove LayoutLMForQuestionAnswering from the auto mapping
      
      * Docstrings
      
      * Add better docstrings
      
      * Undo whitespace changes
      
      * Update tokenizers in comments
      
      * Fixup code and remove `from_pt=True`
      
      * Fix tests
      
      * Revert some unexpected docstring changes
      
      * Fix tests by overriding _prepare_for_class
      Co-authored-by: default avatarAnkur Goyal <ankur@impira.com>
      5c4c8690
  24. 30 Aug, 2022 1 commit
  25. 24 Aug, 2022 1 commit
  26. 16 Aug, 2022 1 commit
  27. 12 Aug, 2022 3 commits
    • Niklas Muennighoff's avatar
      Update BLOOM parameter counts (#18531) · 56ef0ba4
      Niklas Muennighoff authored
      * Update BLOOM parameter counts
      
      * Update BLOOM parameter counts
      56ef0ba4
    • NielsRogge's avatar
      Fix URLs (#18604) · 153d1361
      NielsRogge authored
      
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      153d1361
    • NielsRogge's avatar
      Add Donut (#18488) · 2ab790e8
      NielsRogge authored
      
      
      * First draft
      
      * Improve script
      
      * Update script
      
      * Make conversion work
      
      * Add final_layer_norm attribute to Swin's config
      
      * Add DonutProcessor
      
      * Convert more models
      
      * Improve feature extractor and convert base models
      
      * Fix bug
      
      * Improve integration tests
      
      * Improve integration tests and add model to README
      
      * Add doc test
      
      * Add feature extractor to docs
      
      * Fix integration tests
      
      * Remove register_buffer
      
      * Fix toctree and add missing attribute
      
      * Add DonutSwin
      
      * Make conversion script work
      
      * Improve conversion script
      
      * Address comment
      
      * Fix bug
      
      * Fix another bug
      
      * Remove deprecated method from docs
      
      * Make Swin and Swinv2 untouched
      
      * Fix code examples
      
      * Fix processor
      
      * Update model_type to donut-swin
      
      * Add feature extractor tests, add token2json method, improve feature extractor
      
      * Fix failing tests, remove integration test
      
      * Add do_thumbnail for consistency
      
      * Improve code examples
      
      * Add code example for document parsing
      
      * Add DonutSwin to MODEL_NAMES_MAPPING
      
      * Add model to appropriate place in toctree
      
      * Update namespace to appropriate organization
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      2ab790e8