1. 06 May, 2024 1 commit
    • Arthur's avatar
      [`CI update`] Try to use dockers and no cache (#29202) · 307f632b
      Arthur authored
      
      
      * change cis
      
      * nits
      
      * update
      
      * minor updates
      
      * [push-ci-image]
      
      * nit [push-ci-image]
      
      * nitsssss
      
      * [build-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * both
      
      * [push-ci-image]
      
      * this?
      
      * [push-ci-image]
      
      * pypi-kenlm needs g++
      
      * [push-ci-image]
      
      * nit
      
      * more nits [push-ci-image]
      
      * nits [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * add vision
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * add new dummy file but will need to update them [push-ci-image]
      
      * [push-ci-image]
      
      * show package size as well
      
      * [push-ci-image]
      
      * potentially ignore failures
      
      * workflow updates
      
      * nits [push-ci-image]
      
      * [push-ci-image]
      
      * fix consistency
      
      * clean nciida triton
      
      * also show big packages [push-ci-image]
      
      * nit
      
      * update
      
      * another one
      
      * line escape?
      
      * add accelerate [push-ci-image]
      
      * updates [push-ci-image]
      
      * nits to run tests, no push-ci
      
      * try to parse skip reason to make sure nothing is skipped that should no be skippped
      
      * nit?
      
      * always show skipped reasons
      
      * nits
      
      * better parsing of the test outputs
      
      * action="store_true",
      
      * failure on failed
      
      * show matched
      
      * debug
      
      * update short summary with skipped, failed and errors
      
      * nits
      
      * nits
      
      * coolu pdates
      
      * remove docbuilder
      
      * fix
      
      * always run checks
      
      * oups
      
      * nits
      
      * don't error out on library printing
      
      * non zero exi codes
      
      * no warning
      
      * nit
      
      * WAT?
      
      * format nit
      
      * [push-ci-image]
      
      * fail if fail is needed
      
      * [push-ci-image]
      
      * sound file for torch light?
      
      * [push-ci-image]
      
      * order is important [push-ci-image]
      
      * [push-ci-image] reduce even further
      
      * [push-ci-image]
      
      * use pytest rich !
      
      * yes [push-ci-image]
      
      * oupsy
      
      * bring back the full traceback, but pytest rich should help
      
      * nit
      
      * [push-ci-image]
      
      * re run
      
      * nit
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * empty push to trigger
      
      * [push-ci-image]
      
      * nit? [push-ci-image]
      
      * empty
      
      * try to install timm with no deps
      
      * [push-ci-image]
      
      * oups [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image] ?
      
      * [push-ci-image] open ssh client for git checkout fast
      
      * empty for torch light
      
      * updates [push-ci-image]
      
      * nit
      
      * @v4 for checkout
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * fix fetch tests with parallelism
      
      * [push-ci-image]
      
      * more parallelism
      
      * nit
      
      * more nits
      
      * empty to re-trigger
      
      * empty to re-trigger
      
      * split by timing
      
      * did not work with previous commit
      
      * junit.xml
      
      * no path?
      
      * mmm this?
      
      * junitxml format
      
      * split by timing
      
      * nit
      
      * fix junit family
      
      * now we can test if the xunit1 is compatible!
      
      * this?
      
      * fully list tests
      
      * update
      
      * update
      
      * oups
      
      * finally
      
      * use classname
      
      * remove working directory to make sure the path does not interfere
      
      * okay no juni should have the correct path
      
      * name split?
      
      * sort by classname is what make most sense
      
      * some testing
      
      * naem
      
      * oups
      
      * test something fun
      
      * autodetect
      
      * 18?
      
      * nit
      
      * file size?
      
      * uip
      
      * 4 is best
      
      * update to see versions
      
      * better print
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * please install the correct keras version
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * uv is fucking me up
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * nits
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * install issues an pins
      
      * tapas as well
      
      * nits
      
      * more paralellism
      
      * short tb
      
      * soundfile
      
      * soundfile
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * oups
      
      * [push-ci-image]
      
      * fix some things
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * use torch-light for hub
      
      * small git lfs for hub job
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * fix tf tapas
      
      * [push-ci-image]
      
      * nits
      
      * [push-ci-image]
      
      * don't update the test
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * no use them
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * update tf proba
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * woops
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * test with built dockers
      
      * [push-ci-image]
      
      * skip annoying tests
      
      * revert fix copy
      
      * update test values
      
      * update
      
      * last skip and fixup
      
      * nit
      
      * ALL GOOOD
      
      * quality
      
      * Update tests/models/layoutlmv2/test_image_processing_layoutlmv2.py
      
      * Update docker/quality.dockerfile
      Co-authored-by: default avatarLysandre Debut <hi@lysand.re>
      
      * Update src/transformers/models/tapas/modeling_tf_tapas.py
      Co-authored-by: default avatarLysandre Debut <hi@lysand.re>
      
      * Apply suggestions from code review
      Co-authored-by: default avatarLysandre Debut <hi@lysand.re>
      
      * use torch-speed
      
      * updates
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      * fuck ken-lm [push-ci-image]
      
      * [push-ci-image]
      
      * [push-ci-image]
      
      ---------
      Co-authored-by: default avatarLysandre Debut <hi@lysand.re>
      307f632b
  2. 24 Apr, 2024 1 commit
  3. 22 Mar, 2024 1 commit
  4. 22 Dec, 2023 1 commit
  5. 16 Nov, 2023 1 commit
    • Arthur's avatar
      [`Styling`] stylify using ruff (#27144) · 651408a0
      Arthur authored
      
      
      * try to stylify using ruff
      
      * might need to remove these changes?
      
      * use ruf format andruff check
      
      * use isinstance instead of type comparision
      
      * use # fmt: skip
      
      * use # fmt: skip
      
      * nits
      
      * soem styling changes
      
      * update ci job
      
      * nits isinstance
      
      * more files update
      
      * nits
      
      * more nits
      
      * small nits
      
      * check and format
      
      * revert wrong changes
      
      * actually use formatter instead of checker
      
      * nits
      
      * well docbuilder is overwriting this commit
      
      * revert notebook changes
      
      * try to nuke docbuilder
      
      * style
      
      * fix feature exrtaction test
      
      * remve `indent-width = 4`
      
      * fixup
      
      * more nits
      
      * update the ruff version that we use
      
      * style
      
      * nuke docbuilder styling
      
      * leve the print for detected changes
      
      * nits
      
      * Remove file I/O
      Co-authored-by: default avatarcharliermarsh <charlie.r.marsh@gmail.com>
      
      * style
      
      * nits
      
      * revert notebook changes
      
      * Add # fmt skip when possible
      
      * Add # fmt skip when possible
      
      * Fix
      
      * More `  # fmt: skip` usage
      
      * More `  # fmt: skip` usage
      
      * More `  # fmt: skip` usage
      
      * NIts
      
      * more fixes
      
      * fix tapas
      
      * Another way to skip
      
      * Recommended way
      
      * Fix two more fiels
      
      * Remove asynch
      Remove asynch
      
      ---------
      Co-authored-by: default avatarcharliermarsh <charlie.r.marsh@gmail.com>
      651408a0
  6. 04 Oct, 2023 1 commit
  7. 11 Aug, 2023 2 commits
  8. 18 Jul, 2023 1 commit
  9. 05 Jul, 2023 1 commit
  10. 22 Jun, 2023 2 commits
  11. 06 Jun, 2023 1 commit
  12. 25 May, 2023 1 commit
  13. 16 May, 2023 1 commit
  14. 09 May, 2023 2 commits
    • Sylvain Gugger's avatar
      Revert "[Doctests] Refactor doctests + add CI" (#23245) · 69ee4624
      Sylvain Gugger authored
      Revert "[Doctests] Refactor doctests + add CI (#22987)"
      
      This reverts commit 627f4479.
      69ee4624
    • Arthur's avatar
      [Doctests] Refactor doctests + add CI (#22987) · 627f4479
      Arthur authored
      
      
      * intiial commit
      
      * new styling
      
      * update
      
      * just run doctest in CI
      
      * remove more test for fast dev
      
      * update
      
      * update refs
      
      * update path and fetch upstream
      
      * update documentatyion trests
      
      * typo
      
      * parse pwd
      
      * don't check for files that are in hidden folders
      
      * just give paths relative to transformers
      
      * update
      
      * update
      
      * update
      
      * major refactoring
      
      * make sure options is ok
      
      * lest test that mdx is tested
      
      * doctest glob
      
      * nits
      
      * update doctest nightly
      
      * some cleaning
      
      * run correct test on diff
      
      * debug
      
      * run on a single worker
      
      * skip_cuda_test tampkate
      
      * updates
      
      * add rA and continue on failure
      
      * test options
      
      * parse `py` codeblock?
      
      * we don't need to replace ignore results, don't remember whyu I put it
      
      * cleanup
      
      * more cleaning
      
      * fix arg
      
      * more cleaning
      
      * clean an todo
      
      * more pre-processing
      
      * doctest-module has none so extra `- ` is needed
      
      * remove logs
      
      * nits
      
      * doctest-modules ....
      
      * oups
      
      * let's use sugar
      
      * make dataset go quiet
      
      * add proper timeout
      
      * nites
      
      * spleling timeout
      
      * update
      
      * properly skip tests that have CUDSA
      
      * proper skipping
      
      * cleaning main and get tests to run
      
      * remove make report?
      
      * remove tee
      
      * some updates
      
      * tee was removed but is the full output still available?
      
      * [all-test]
      
      * only our tests
      
      * don't  touch tee in this PR
      
      * no atee-sys
      
      * proper sub
      
      * monkey
      
      * only replace call
      
      * fix sub
      
      * nits
      
      * nits
      
      * fix invalid syntax
      
      * add skip cuda doctest env variable
      
      * make sure all packages are installed
      
      * move file
      
      * update check repo
      
      * revert changes
      
      * nit
      
      * finish cleanup
      
      * fix re
      
      * findall
      
      * update don't test init files
      
      * ignore pycache
      
      * `-ignore-pycache` when running pytests
      
      * try to fix the import missmatch error
      
      * install dec
      
      * pytest is required as doctest_utils imports things from it
      
      * the only log issues were dataset, ignore results should work
      
      * more cleaning
      
      * Update .circleci/create_circleci_config.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * Apply suggestions from code review
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * [ydshieh] empty string if cuda is found
      
      * [ydshieh] fix condition
      
      * style
      
      * [ydshieh] fix
      
      * Add comment
      
      * style
      
      * style
      
      * show failure
      
      * trigger CI
      
      ---------
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarYih-Dar <2521628+ydshieh@users.noreply.github.com>
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      Co-authored-by: default avatarydshieh <ydshieh@users.noreply.github.com>
      627f4479
  15. 28 Apr, 2023 1 commit
  16. 31 Mar, 2023 1 commit
    • Sylvain Gugger's avatar
      Test fetch v2 (#22367) · c6126280
      Sylvain Gugger authored
      
      
      * Test fetcher v2
      
      * Fix regexes
      
      * Remove sanity check
      
      * Fake modification to OPT
      
      * Fixes some .sep issues
      
      * Remove fake OPT change
      
      * Fake modif for BERT
      
      * Fake modif for init
      
      * Exclude SageMaker tests
      
      * Fix test and remove fake modif
      
      * Fake setup modif
      
      * Fake pipeline modif
      
      * Remove all fake modifs
      
      * Adds options to skip/force tests
      
      * [test-all-models] Fake modif for BERT
      
      * Try this way
      
      * Does the command actually work?
      
      * [test-all-models] Try again!
      
      * [skip circleci] Remove fake modif
      
      * Remove debug statements
      
      * Add the list of important models
      
      * Quality
      
      * Update utils/tests_fetcher.py
      Co-authored-by: default avatarLysandre Debut <lysandre.debut@reseau.eseo.fr>
      
      * Address review comments
      
      * Address review comments
      
      * Fix and add test
      
      * Apply suggestions from code review
      Co-authored-by: default avatarYih-Dar <2521628+ydshieh@users.noreply.github.com>
      
      * Address review comments
      
      ---------
      Co-authored-by: default avatarLysandre Debut <lysandre.debut@reseau.eseo.fr>
      Co-authored-by: default avatarYih-Dar <2521628+ydshieh@users.noreply.github.com>
      c6126280
  17. 16 Mar, 2023 1 commit
  18. 07 Feb, 2023 1 commit
  19. 06 Feb, 2023 1 commit
    • Sylvain Gugger's avatar
      Update quality tooling for formatting (#21480) · 6f79d264
      Sylvain Gugger authored
      * Result of black 23.1
      
      * Update target to Python 3.7
      
      * Switch flake8 to ruff
      
      * Configure isort
      
      * Configure isort
      
      * Apply isort with line limit
      
      * Put the right black version
      
      * adapt black in check copies
      
      * Fix copies
      6f79d264
  20. 27 Jan, 2023 1 commit
    • Maria Khalusova's avatar
      Automated compatible models list for task guides (#21338) · 73a2ff69
      Maria Khalusova authored
      * initial commit. added tip placeholders and a script
      
      * removed unused imports, fixed paths
      
      * fixed generated links
      
      * make style
      
      * split language modeling doc into two: causal language modeling and masked language modeling
      
      * added check_task_guides.py to make fix-copies
      
      * review feedback addressed
      73a2ff69
  21. 26 Jan, 2023 1 commit
  22. 03 Jan, 2023 1 commit
  23. 02 Nov, 2022 1 commit
  24. 24 Oct, 2022 2 commits
  25. 21 Oct, 2022 1 commit
  26. 18 Oct, 2022 1 commit
    • Sylvain Gugger's avatar
      Repo utils test (#19696) · a929f81e
      Sylvain Gugger authored
      * Create repo utils test job
      
      * Last occurence
      
      * Add tests for tests_fetcher
      
      * Better filtering
      
      * Let's learn more
      
      * Should fix
      
      * Should fix
      
      * Remove debug
      
      * Style
      
      * WiP
      
      WiP
      
      WiP
      
      WiP
      
      WiP
      
      WiP
      
      WiP
      
      WiP
      
      WiP
      
      * Quality
      
      * address review comments
      
      * Fix link
      a929f81e
  27. 11 Oct, 2022 1 commit
    • Sylvain Gugger's avatar
      Use a dynamic configuration for circleCI tests (#19325) · 69b81c0a
      Sylvain Gugger authored
      * Generate config on the file
      
      * Fake modif for all test launch
      
      * Upload more artifacts
      
      * Typo and quality
      
      * Try converting th yml to txt
      
      * Leave my long lines alone yaml
      
      * Debug prints
      
      * Debug prints v2
      
      * Try without sorting
      
      * Was it really working before?
      
      * Typo
      
      * Use a parameter
      
      * Use a parameter?
      
      * Typo
      
      * Here is some JSON
      
      * Another try
      
      * Learning to read...
      
      * Check default is used
      
      * Does this work?
      
      * With continuation
      
      * WiP
      
      * Use a parameter for test list
      
      * Other fake modif
      
      * With the comma
      
      * Name the test step so it doesn't blow up
      
      * Just one example modification
      
      * Final steps
      
      * Add nightlies
      
      * Move config generator
      
      * Add trigger for nightlies
      
      * Better workflow
      
      * Rebase on recent changes
      
      * Fix config creation
      
      * Fake modif in an example
      
      * Now fake modif in one config file
      
      * Fix install step in custom tokenizers test
      
      * Fix generated config
      
      * Better fix hopefully
      
      * Finally test modif in setup
      
      * final cleanup
      69b81c0a
  28. 07 Oct, 2022 1 commit
    • Sylvain Gugger's avatar
      Rework pipeline tests (#19366) · 9ac586b3
      Sylvain Gugger authored
      * Rework pipeline tests
      
      * Try to fix Flax tests
      
      * Try to put it before
      
      * Use a new decorator instead
      
      * Remove ignore marker since it doesn't work
      
      * Filter pipeline tests
      
      * Woopsie
      
      * Use the fitlered list
      
      * Clean up and fake modif
      
      * Remove init
      
      * Revert fake modif
      9ac586b3
  29. 05 Oct, 2022 1 commit
    • r-terada's avatar
      Add sudachi and jumanpp tokenizers for bert_japanese (#19043) · 2f53ab57
      r-terada authored
      * add sudachipy and jumanpp tokenizers for bert_japanese
      
      * use ImportError instead of ModuleNotFoundError in SudachiTokenizer and JumanppTokenizer
      
      * put test cases of test_tokenization_bert_japanese in one line
      
      * add require_sudachi and require_jumanpp decorator for testing
      
      * add sudachi and pyknp(jumanpp) to dependencies
      
      * remove sudachi_dict_small and sudachi_dict_full from dependencies
      
      * empty commit for ci
      2f53ab57
  30. 29 Sep, 2022 1 commit
    • Sylvain Gugger's avatar
      Fix test fetching for examples (#19237) · 655f72a6
      Sylvain Gugger authored
      * Fix test fetching for examples
      
      * Fake example modif
      
      * Debug statements
      
      * Typo
      
      * You need to persist the file...
      
      * Revert change in example
      
      * Remove debug statements
      655f72a6
  31. 28 Sep, 2022 1 commit
  32. 16 Sep, 2022 1 commit
    • Sylvain Gugger's avatar
      Organize test jobs (#19058) · 820cb97a
      Sylvain Gugger authored
      * Tests conditional run
      
      * Syntax
      
      * Deps
      
      * Try early exit
      
      * Another way
      
      * Test with no tests to run
      
      * Test all
      
      * Typo
      
      * Try this way
      
      * With tests to run
      
      * Mostly finished
      
      * Typo
      
      * With a modification in one file only
      
      * No change, no tests
      
      * Final cleanup
      
      * Address review comments
      820cb97a
  33. 15 Sep, 2022 1 commit
  34. 14 Sep, 2022 1 commit
  35. 25 Aug, 2022 1 commit
    • Craig Chan's avatar
      Determine framework automatically before ONNX export (#18615) · fbf382c8
      Craig Chan authored
      
      
      * Automatic detection for framework to use when exporting to ONNX
      
      * Log message change
      
      * Incorporating PR comments, adding unit test
      
      * Adding tf for pip install for run_tests_onnxruntime CI
      
      * Restoring past changes to circleci yaml and test_onnx_v2.py, tests moved to tests/onnx/test_features.py
      
      * Fixup
      
      * Adding test to fetcher
      
      * Updating circleci config to log more
      
      * Changing test class name
      
      * Comment typo fix in tests/onnx/test_features.py
      Co-authored-by: default avatarlewtun <lewis.c.tunstall@gmail.com>
      
      * Moving torch_str/tf_str to self.framework_pt/tf
      
      * Remove -rA flag in circleci config
      Co-authored-by: default avatarlewtun <lewis.c.tunstall@gmail.com>
      fbf382c8
  36. 23 Aug, 2022 1 commit