1. 13 Jun, 2022 1 commit
    • Daniel Stancl's avatar
      Add `LongT5` model (#16792) · a72f1c9f
      Daniel Stancl authored
      
      
      * Initial commit
      
      * Make some fixes
      
      * Make PT model full forward pass
      
      * Drop TF & Flax implementation, fix copies etc
      
      * Add Flax model and update some corresponding stuff
      
      * Drop some TF things
      
      * Update config and flax local attn
      
      * Add encoder_attention_type to config
      
      * .
      
      * Update docs
      
      * Do some cleansing
      
      * Fix some issues -> make style; add some docs
      
      * Fix position_bias + mask addition + Update tests
      
      * Fix repo consistency
      
      * Fix model consistency by removing flax operation over attn_mask
      
      * [WIP] Add PT TGlobal LongT5
      
      * .
      
      * [WIP] Add flax tglobal model
      
      * [WIP] Update flax model to use the right attention type in the encoder
      
      * Fix flax tglobal model forward pass
      
      * Make the use of global_relative_attention_bias
      
      * Add test suites for TGlobal model
      
      * Fix minor bugs, clean code
      
      * Fix pt-flax equivalence though not convinced with correctness
      
      * Fix LocalAttn implementation to match the original impl. + update READMEs
      
      * Few updates
      
      * Update: [Flax] improve large model init and loading #16148
      
      * Add ckpt conversion script accoring to #16853 + handle torch device placement
      
      * Minor updates to conversion script.
      
      * Typo: AutoModelForSeq2SeqLM -> FlaxAutoModelForSeq2SeqLM
      
      * gpu support + dtype fix
      
      * Apply some suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * * Remove (de)parallelize stuff
      * Edit shape comments
      * Update README.md
      * make fix-copies
      
      * Remove caching logic for local & tglobal attention
      
      * Apply another batch of suggestions from code review
      
      * Add missing checkpoints
      * Format converting scripts
      * Drop (de)parallelize links from longT5 mdx
      
      * Fix converting script + revert config file change
      
      * Revert "Remove caching logic for local & tglobal attention"
      
      This reverts commit 2a619828f6ddc3e65bd9bb1725a12b77fa883a46.
      
      * Stash caching logic in Flax model
      
      * Make side relative bias used always
      
      * Drop caching logic in PT model
      
      * Return side bias as it was
      
      * Drop all remaining model parallel logic
      
      * Remove clamp statements
      
      * Move test files to the proper place
      
      * Update docs with new version of hf-doc-builder
      
      * Fix test imports
      
      * Make some minor improvements
      
      * Add missing checkpoints to docs
      * Make TGlobal model compatible with torch.onnx.export
      * Replace some np.ndarray with jnp.ndarray
      
      * Fix TGlobal for ONNX conversion + update docs
      
      * fix _make_global_fixed_block_ids and masked neg  value
      
      * update flax model
      
      * style and quality
      
      * fix imports
      
      * remove load_tf_weights_in_longt5 from init and fix copies
      
      * add slow test for TGlobal model
      
      * typo fix
      
      * Drop obsolete is_parallelizable and one warning
      
      * Update __init__ files to fix repo-consistency
      
      * fix pipeline test
      
      * Fix some device placements
      
      * [wip]: Update tests -- need to generate summaries to update expected_summary
      
      * Fix quality
      
      * Update LongT5 model card
      
      * Update (slow) summarization tests
      
      * make style
      
      * rename checkpoitns
      
      * finish
      
      * fix flax tests
      Co-authored-by: default avatarphungvanduy <pvduy23@gmail.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarpatil-suraj <surajp815@gmail.com>
      a72f1c9f
  2. 31 May, 2022 1 commit
    • Arthur's avatar
      Opt in flax and tf (#17388) · 7822a9b7
      Arthur authored
      
      
      * initial commit
      
      * add init file
      
      * update globakl init
      
      * update index and dummy objects
      
      * style
      
      * update modelling auto
      
      * fix initi typo in src/transformers
      
      * fix typo in modeling tf auto, opt was in wrong mapping name
      
      * fixed a slow test : saved_model
      
      * style
      
      * fix positionnal embedding if no position id is provided
      
      * update tf test
      
      * update test flax requirements
      
      * fixed serialization
      
      * update
      
      * update tf name to allow smooth convertion
      
      * update flax tests
      
      * style
      
      * fix test typo
      
      * fix tf typo test
      
      * add xla for generate support in causal LM
      
      * fixed bug
      
      * cleaned tf tests
      
      * style
      
      * removed from PT for slow tests
      
      * fix typp
      
      * opt test as slow
      
      * trying to fix GPT2 undefined
      
      * correct documentation and add to test doc
      
      * update tf doc
      
      * fix doc
      
      * fake commit
      
      * Apply suggestions from code review
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      
      * update test based on review
      
      * merged main layer for functionning test
      
      * fixup + quality
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * update long comment
      
      * make fix copies
      Co-authored-by: default avatarArthur <arthur@huggingface.co>
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      7822a9b7
  3. 24 May, 2022 1 commit
    • NielsRogge's avatar
      Add LayoutLMv3 (#17060) · 31ee80d5
      NielsRogge authored
      
      
      * Make forward pass work
      
      * More improvements
      
      * Remove unused imports
      
      * Remove timm dependency
      
      * Improve loss calculation of token classifier
      
      * Fix most tests
      
      * Add docs
      
      * Add model integration test
      
      * Make all tests pass
      
      * Add LayoutLMv3FeatureExtractor
      
      * Improve integration test + make fixup
      
      * Add example script
      
      * Fix style
      
      * Add LayoutLMv3Processor
      
      * Fix style
      
      * Add option to add visual labels
      
      * Make more tokenizer tests pass
      
      * Fix more tests
      
      * Make more tests pass
      
      * Fix bug and improve docs
      
      * Fix import of processors
      
      * Improve docstrings
      
      * Fix toctree and improve docs
      
      * Fix auto tokenizer
      
      * Move tests to model folder
      
      * Move tests to model folder
      
      * change default behavior add_prefix_space
      
      * add prefix space for fast
      
      * add_prefix_spcae set to True for Fast
      
      * no space before `unique_no_split` token
      
      * add test to hightligh special treatment of added tokens
      
      * fix `test_batch_encode_dynamic_overflowing` by building a long enough example
      
      * fix `test_full_tokenizer` with add_prefix_token
      
      * Fix tokenizer integration test
      
      * Make the code more readable
      
      * Add tests for LayoutLMv3Processor
      
      * Fix style
      
      * Add model to README and update init
      
      * Apply suggestions from code review
      
      * Replace asserts by value errors
      
      * Add suggestion by @ducviet00
      
      * Add model to doc tests
      
      * Simplify script
      
      * Improve README
      
      * a step ahead to fix
      
      * Update pair_input_test
      
      * Make all tokenizer tests pass - phew
      
      * Make style
      
      * Add LayoutLMv3 to CI job
      
      * Fix auto mapping
      
      * Fix CI job name
      
      * Make all processor tests pass
      
      * Make tests of LayoutLMv2 and LayoutXLM consistent
      
      * Add copied from statements to fast tokenizer
      
      * Add copied from statements to slow tokenizer
      
      * Remove add_visual_labels attribute
      
      * Fix tests
      
      * Add link to notebooks
      
      * Improve docs of LayoutLMv3Processor
      
      * Fix reference to section
      Co-authored-by: default avatarSaulLu <lucilesaul.com@gmail.com>
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      31ee80d5
  4. 23 May, 2022 1 commit
    • ghlai9665's avatar
      Correct & Improve Doctests for LayoutLMv2 (#17168) · 7b8cb269
      ghlai9665 authored
      
      
      * add inference example to LayoutLMv2ForQuestionAnswering, passing doctest
      
      * add loss example to LayoutLMv2ForQuestionAnswering, passing doctest
      
      * Add correct doctest for LayoutLMv2ForTokenClassification, passing doctest
      
      * add correct doctest for LayoutLMv2ForSequenceClassification, passing test
      
      * add correct doctest for LayoutLMv2Model, passing test
      
      * make fixup
      
      * fix to address review comments
      
      * make style
      
      * fix doctest line break issue, add to documentaiton_tests.txt, address review comments
      
      * move comment about layoutlmv2 dependencies to the doc page
      
      * format doc page as suggested
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * delete extraneous backtick
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      7b8cb269
  5. 18 May, 2022 1 commit
  6. 17 May, 2022 1 commit
  7. 16 May, 2022 1 commit
    • Patrick von Platen's avatar
      Add Wav2Vec2Conformer (#16812) · 5a995735
      Patrick von Platen authored
      
      
      * save intermediate
      
      * add wav2vec2 conformer
      
      * add more code
      
      * more
      
      * first test passes
      
      * make all checkpoints work
      
      * update
      
      * up
      
      * more clean ups
      
      * save clean-up
      
      * save clean-up
      
      * save more
      
      * remove bogus
      
      * finalize design conformer
      
      * remove vision
      
      * finish all tests
      
      * more changes
      
      * finish code
      
      * add doc tests
      
      * add slow tests
      
      * fix autoconfig test
      
      * up
      
      * correct docstring
      
      * up
      
      * update
      
      * fix
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarAnton Lozhkov <aglozhkov@gmail.com>
      
      * Update docs/source/en/model_doc/wav2vec2-conformer.mdx
      
      * upload
      
      * save copied from
      
      * correct configs
      
      * fix model outputs
      
      * add to docs
      
      * fix imports
      
      * finish
      
      * finish code
      
      * correct copied from
      
      * correct again
      
      * correct make fix
      
      * improve make fix copies
      
      * save
      
      * correct fix copy from
      
      * correct init structure
      
      * correct
      
      * fix import
      
      * apply suggestions
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarAnton Lozhkov <aglozhkov@gmail.com>
      5a995735
  8. 13 May, 2022 1 commit
  9. 04 May, 2022 1 commit
  10. 02 May, 2022 1 commit
    • NielsRogge's avatar
      Add YOLOS (#16848) · 1ac69874
      NielsRogge authored
      
      
      * First draft
      
      * Add YolosForObjectDetection
      
      * Make forward pass work
      
      * Add mid position embeddings
      
      * Add interpolation of position encodings
      
      * Add expected values
      
      * Add YOLOS to tests
      
      * Add integration test
      
      * Support tiny model as well
      
      * Support all models in conversion script
      
      * Remove mid_pe_size attribute
      
      * Make more tests pass
      
      * Add model to README and fix config
      
      * Add copied from statements
      
      * Rename base_model_prefix to vit
      
      * Add missing YOLOS_PRETRAINED_CONFIG_ARCHIVE_MAP
      
      * Apply suggestions from code review
      
      * Apply more suggestions from code review
      
      * Convert remaining checkpoints
      
      * Improve docstrings
      
      * Add YolosFeatureExtractor
      
      * Add feature extractor to docs
      
      * Add corresponding tests
      
      * Fix style
      
      * Fix docs
      
      * Apply suggestion from code review
      
      * Fix bad rebase
      
      * Fix some more bad rebase
      
      * Fix missing character
      
      * Improve docs and variable names
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      1ac69874
  11. 26 Apr, 2022 1 commit
  12. 25 Apr, 2022 1 commit
  13. 22 Apr, 2022 1 commit
  14. 21 Apr, 2022 1 commit
  15. 20 Apr, 2022 1 commit
  16. 18 Apr, 2022 1 commit
  17. 13 Apr, 2022 4 commits
  18. 12 Apr, 2022 2 commits
    • hiromu's avatar
      Add Doc Tests for Reformer PyTorch (#16565) · 1bac40db
      hiromu authored
      * start working
      
      * fix: ReformerForQA doctest
      
      * fix: ReformerModelWithLMHead doctest
      
      * fix: ReformerModelForSC doctest
      
      * fix: ReformerModelForMLM doctest
      
      * add: documentation_tests.txt
      
      * make fixup
      
      * change: ReformerModelForSC doctest
      
      * change: checkpoint
      1bac40db
    • Michael Chung's avatar
      Add Doc Test GPT-2 (#16439) · 924484ee
      Michael Chung authored
      
      
      * First Pass All Tests Pass
      
      * WIP
      
      * Adding file to documentation tests
      
      * Change the base model for the example in the doc test.
      
      * Fix Code Styling by running
      make fixup
      
      * Called Style
      
      * Reverted to gpt2 model rather than distill gpt2
      Then used a token classification model over a sequence model for an example.
      
      * Fix Styling Issue
      
      * Hopefully ignores the formatting issue.
      Co-authored-by: default avatarArEnSc <xx.mike.chung.xx@gmail.com>
      924484ee
  19. 11 Apr, 2022 1 commit
  20. 08 Apr, 2022 2 commits
    • Johannes Kolbe's avatar
      add vit tf doctest with @add_code_sample_docstrings (#16636) · 9db2eebb
      Johannes Kolbe authored
      
      
      * add vit tf doctest with @add_code_sample_docstrings
      
      * add labels string back in
      Co-authored-by: default avatarJohannes Kolbe <johannes.kolbe@tech.better.team>
      9db2eebb
    • NielsRogge's avatar
      Add TAPEX (#16473) · 4ef0abb7
      NielsRogge authored
      
      
      * Add TapexTokenizer
      
      * Improve docstrings and provide option to provide answer
      
      * Remove option for pretokenized inputs
      
      * Add TAPEX to README
      
      * Fix copies
      
      * Remove option for pretokenized inputs
      
      * Initial commit: add tapex fine-tuning examples on both table-based question answering and table-based fact verification.
      
      * - Draft a README file for running the script and introducing some background.
      - Remove unused code lines in tabfact script.
      - Disable the deafult `pad_to_max_length` option which is memory-consuming.
      
      * * Support `as_target_tokenizer` function for TapexTokenizer.
      * Fix the do_lower_case behaviour of TapexTokenizer.
      * Add unit tests for target scenarios and cased/uncased scenarios for both source and target.
      
      * * Replace the label BartTokenizer with TapexTokenizer's as_target_tokenizer function.
      * Fix typos in tapex example README.
      
      * * fix the evaluation script - remove the property `task_name`
      
      * * Make the label space more clear for tabfact tasks
      
      * * Using a new fine-tuning script for tapex-base on tabfact.
      
      * * Remove the lowercase code outside the tokenizer - we use the tokenizer to control whether do_lower_case
      * Guarantee the hyper-parameter can be run without out-of-memory on 16GB card and report the new reproduced number on wikisql
      
      * * Remove the default tokenizer_name option.
      * Provide evaluation command.
      
      * * Support for WikiTableQuestion dataset.
      
      * Fix a typo in README.
      
      * * Fix the datasets's key name in WikiTableQuestions
      
      * Run make fixup and move test to folder
      
      * Fix quality
      
      * Apply suggestions from code review
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSuraj Patil <surajp815@gmail.com>
      
      * Apply suggestions from code review
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Apply some more suggestions from code review
      
      * Improve docstrings
      
      * Overwrite failing test
      
      * Improve comment in example scripts
      
      * Fix rebase
      
      * Add TAPEX to Auto mapping
      
      * Add TAPEX to auto config mappings
      
      * Put TAPEX higher than BART in auto mapping
      
      * Add TAPEX to doc tests
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MBP.localdomain>
      Co-authored-by: default avatarSivilTaram <qianlxc@outlook.com>
      Co-authored-by: default avatarNiels Rogge <nielsrogge@nielss-mbp.home>
      Co-authored-by: default avatarSuraj Patil <surajp815@gmail.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      4ef0abb7
  21. 06 Apr, 2022 1 commit
  22. 05 Apr, 2022 1 commit
  23. 29 Mar, 2022 1 commit
  24. 28 Mar, 2022 1 commit
    • NielsRogge's avatar
      Add DPT (#15991) · 979b039c
      NielsRogge authored
      
      
      * First draft
      
      * More improvements
      
      * Add fusion blocks
      
      * Make conversion script work for dpt_large
      
      * Make conversion script work
      
      * Improve implementation
      
      * Improve conversion script
      
      * Add DPTForSemanticSegmentation
      
      * Make conversion work for semantic segmentation
      
      * Add tests
      
      * Remove print statements
      
      * First draft
      
      * Redesign neck
      
      * Improve tests
      
      * Improve implementation some more
      
      * Make neck output list of tensors
      
      * Improve neck and feature extractor
      
      * Fix integration tests
      
      * Make more tests pass
      
      * Make all tests pass
      
      * Add missing config archive map
      
      * Add in_index attribute to make heads accept list of tensors
      
      * Apply suggestions from code review
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Apply some more suggestions
      
      * Add copied from statements
      
      * Remove assert
      
      * Apply suggestions from code review
      
      * Apply suggestions from code review
      
      * Remove DPTInterpolate in favor of nn.Upsample
      
      * Add comments
      
      * Apply suggestions from code review
      
      * Apply suggestions from code review
      
      * Add proposed design
      
      * Update design
      
      * Add DPTReassembleLayer
      
      * Add DPTFeatureFusionStage
      
      * Apply more suggestions from code review
      
      * Apply suggestions from code review
      
      * Apply suggestions from code review
      
      * Fix rebase
      
      * Update in_index and out_indices
      
      * Fix conversion script
      
      * Fix code quality
      
      * Add model to toctree and use DepthEstimatorOutput
      
      * Fix rebase
      
      * Fix code examples
      
      * Improve code
      
      * Fix copied from statements
      
      * Apply suggestions from code review
      
      * Remove compute_loss method
      
      * Apply suggestions from code review
      
      * Fix documentation tests file
      
      * Remove test.py file
      
      * Improve doc example
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarNiels Rogge <nielsrogge@nielss-mbp.home>
      979b039c
  25. 24 Mar, 2022 1 commit
  26. 23 Mar, 2022 1 commit
  27. 22 Mar, 2022 1 commit
    • NielsRogge's avatar
      Add GLPN (#16199) · 0c55d47c
      NielsRogge authored
      
      
      * First draft
      
      * Fix logits calculation
      
      * Improve tests
      
      * Add copied from statements
      
      * Fix base_model_prefix
      
      * Improve implementation, upload new models
      
      * Update design
      
      * Fix integration test
      
      * Add model to README and toctree
      
      * Add document image
      
      * Apply suggestions from code review
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Add decoder_hidden_size attribute
      
      * Update design of decoder
      
      * Add DepthEstimatorOutput class
      
      * Rename in_index to head_in_index and add feature extractor tests
      
      * Apply suggestions from code review
      
      * Apply suggestions from code review
      
      * Update pretrained model name and add to doc tests
      
      * Remove test.py script
      
      * Update copied from statements and clean up
      Co-authored-by: default avatarNiels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      0c55d47c
  28. 21 Mar, 2022 1 commit
    • Patrick von Platen's avatar
      Add Slack notification support for doc tests (#16253) · c1af180d
      Patrick von Platen authored
      * up
      
      * up
      
      * up
      
      * fix
      
      * yeh
      
      * ups
      
      * Empty test commit
      
      * correct quicktour
      
      * correct
      
      * correct
      
      * up
      
      * up
      
      * uP
      
      * uP
      
      * up
      
      * up
      
      * uP
      
      * up
      
      * up
      
      * up
      
      * up
      
      * up
      
      * up
      
      * up
      
      * up
      
      * up
      
      * up
      
      * Update src/transformers/models/van/modeling_van.py
      
      * finish
      
      * apply suggestions
      
      * remove folder
      
      * revert to daily testing
      c1af180d
  29. 15 Mar, 2022 2 commits
  30. 14 Mar, 2022 1 commit
  31. 10 Mar, 2022 1 commit
  32. 09 Mar, 2022 1 commit
  33. 03 Mar, 2022 2 commits