Skip to content

GitLab

"tests/extended/test_trainer_ext.py" did not exist on "2df34f4aba7ffbf47974f121767be052bebb23ca"
Switch branch/tag
  1. 07 Jun, 2023 1 commit
  3. 06 Jun, 2023 2 commits
  5. 02 Jun, 2023 2 commits
  7. 31 May, 2023 2 commits
  9. 26 May, 2023 1 commit
  11. 25 May, 2023 3 commits
  13. 24 May, 2023 1 commit
  15. 23 May, 2023 4 commits
  17. 22 May, 2023 2 commits
  19. 18 May, 2023 1 commit
  21. 16 May, 2023 1 commit
  23. 11 May, 2023 2 commits
  25. 10 May, 2023 1 commit
  27. 09 May, 2023 3 commits
  29. 08 May, 2023 1 commit
  31. 05 May, 2023 1 commit
    • Ashwin Mathur's avatar
      Add `no_trainer` scripts to pre-train Vision Transformers (#23156) · fc6c8b0e
      Ashwin Mathur authored 
      

* Add run_mim_no_trainer.py draft from #20412

Add parse_args method and copy over other dependencies

Add Method call for sending telemetry

Initialize Accelerator

Make one log on every process

Set seed and Handle repository creation

Initialize dataset and Set validation split

Create Config

Adapt Config

Update Config

Create Feature Extractor

Create model

Set column names

Create transforms

Create mask generator

Create method to preprocess images

Shuffle datasets if needed and set transforms

Create Dataloaders

Add optimizer

Add learning rate scheduler

Prepare everything with our accelerator

Tie weights for TPU training

Recalculate training steps and training epochs

Set accelerator checkpointing steps

Initialize trackers and store configuration

Set total batch size

Fix typo: mlm -> mim

Log info at the start of training

Load in the weights and states from previous save

update the progress_bar if load from checkpoint

Define train loop

Add evaluation loop to training

Add to parse_args method

Push repo to hub

Save accelerator state

End training and save model and feature extractor

Remove unused imports

Fix trailing whitespace

* Update code based on comments, Rename feature_extractor to image_processor

* Fix linting

* Add argument for learning rate

* Add argument for setting number of training epochs

* Remove incorrect logger argument

* Convert max_train_steps to int for tqdm

---------
Co-authored-by: default avatarSaad Mahmud <shuvro.mahmud79@gmail.com>
      fc6c8b0e
  33. 03 May, 2023 1 commit
  35. 02 May, 2023 3 commits
  37. 25 Apr, 2023 1 commit
  39. 21 Apr, 2023 1 commit
  41. 20 Apr, 2023 1 commit
  43. 17 Apr, 2023 2 commits
  45. 14 Apr, 2023 1 commit
    • Sayak Paul's avatar
      [Examples] TPU-based training of a language model using TensorFlow (#21657) · 390e121f
      Sayak Paul authored 
      

* add: tokenizer training script for TF TPU LM training.

* add: script for preparing the TFRecord shards.

* add: sequence of execution to readme.

* remove limit from the tfrecord shard name.

* Add initial train_model.py

* Add basic training arguments and model init

* Get up to the point of writing the data collator

* Pushing progress so far!

* Complete first draft of model training code

* feat: grouping of texts efficiently.
Co-authored-by: default avatarMatt <rocketknight1@gmail.com>

* Add proper masking collator and get training loop working

* fix: things.

* Read sample counts from filenames

* Read sample counts from filenames

* Draft README

* Improve TPU warning

* Use distribute instead of distribute.experimental

* Apply suggestions from code review
Co-authored-by: default avatarMatt <Rocketknight1@users.noreply.github.com>

* Modularize loading and add MLM probability as arg

* minor refactoring to better use the cli args.

* readme fillup.

* include tpu and inference sections in the readme.

* table of contents.

* parallelize maps.

* polish readme.

* change script name to run_mlm.py

* address PR feedback (round I).

---------
Co-authored-by: default avatarMatt <rocketknight1@gmail.com>
Co-authored-by: default avatarMatt <Rocketknight1@users.noreply.github.com>
      390e121f
  47. 13 Apr, 2023 1 commit
  49. 11 Apr, 2023 1 commit