- 28 Nov, 2022 1 commit
-
-
Matt authored
* Add a test to ensure int dummy inputs are int64 * Move the test into the existing int64 test and update a lot of existing dummies * Fix remaining dummies * Fix remaining dummies * Test for int64 serving sigs as well * Update core tests to use tf.int64 * Add better messages to the assertions * Update all serving sigs to int64 * More sneaky hiding tf.int32s * Add an optional int32 signature in save_pretrained * make fixup * Add Amy's suggestions * Switch all serving sigs back to tf.int32 * Switch all dummies to tf.int32 * Adjust tests to check for tf.int32 instead of tf.int64 * Fix base dummy_inputs dtype * Start casting to tf.int32 in input_processing * Change dtype for unpack_inputs test * Add proper tf.int32 test * Make the alternate serving signature int64
-
- 22 Nov, 2022 1 commit
-
-
Joao Gante authored
-
- 17 Nov, 2022 2 commits
-
-
Joao Gante authored
* move contrastive search test to slow
-
Joao Gante authored
* test hub tf callback * create repo before cloning it
-
- 15 Nov, 2022 1 commit
-
-
Matt authored
* Slightly alter Keras dummy loss * Slightly alter Keras dummy loss * Add sample weight to test_keras_fit * Fix test_keras_fit for datasets * Skip the sample_weight stuff for models where the model tester has no batch_size
-
- 09 Nov, 2022 1 commit
-
-
Joao Gante authored
* move generation_*.py src files into generation/*.py * populate generation.__init__ with lazy loading * move imports and references from generation.xxx.object to generation.object
-
- 07 Nov, 2022 1 commit
-
-
Joao Gante authored
* Add contrastive search
-
- 27 Oct, 2022 1 commit
-
-
Sylvain Gugger authored
* Wip * Add safetensors support for TensorFlow * First tests * Add final test for now * Retrigger CI like this * Update src/transformers/modeling_tf_utils.py Co-authored-by:
Lysandre Debut <lysandre.debut@reseau.eseo.fr> Co-authored-by:
-
- 18 Oct, 2022 1 commit
-
-
David Yang authored
* Clean up deprecation warnings Notes: Changed some strings in tests to raw strings, which will change the literal content of the strings as they are fed into whatever machine handles them. Test cases for past in the past/past_key_values switch changed/removed due to warning of impending removal * Add PILImageResampling abstraction for PIL.Image.Resampling
-
- 14 Oct, 2022 1 commit
-
-
Arthur authored
* simplify loop * fix layer map split * update * update for special variables * add rag test * fixup * revert change : for next PR
-
- 11 Oct, 2022 1 commit
-
-
Dimitre Oliveira authored
* Custom TF signature draft * Apply suggestions from code review Co-authored-by:
Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by:
Matt <Rocketknight1@users.noreply.github.com> Co-authored-by:
Joao Gante <joaofranciscocardosogante@gmail.com> * Adding tf signature tests * Fixing signature check and adding asserts * fixing model load path * Adjusting signature tests * Formatting file Co-authored-by:
Dimitre Oliveira <dimitreoliveira@Dimitres-MacBook-Air.local>
-
- 10 Oct, 2022 1 commit
-
-
amyeroberts authored
* simplify loop * add featur extractor * add model * start conversion * add dropout * initial commit of test files * copnversion for all models * update processor for correct padding * update feature extraction * update integration test logits match * fmnt: off for the logits * on the fly mel bank * small nit * update test * update tokenizer * nit feature extraction * update * update tokenizer test * adds logit processor and update tokenizer to get supress tokens * style * clean convert * revert to original modeling tf utils * Update * update * nit * clean convert file * update tests and nits * quality * slow generation test * ffn_dim to allow customization * update readme * add to toctreee * start fixing integration tests * update tests and code * fix feature extractor * fix config tests common * update code to fix tests * fix feature exctractor * nit feature extraction * update test for new feature extractor * style * add absrtact * large logits wioth custom decoder input ids * wraap around is otrch available * fix feature extractor * correct logits for whisper small.en * nit * fix encoder_attentino_mask * some fixes * remove unnecessary inputs * nits * add normalizer file * update etst tokenization * fix attention mask not defined * fix generate * remove uncoder attention mask useless * update test modeling whisper * update condfig to add second non supress tokens * nits on feature exrtactor * nit for test tokenizers * update etsts * update tests * update tokenization test * fixup * invalidated hf token. Clean convert openai to whisper * fix logit tests * fixup * Add model to README * Fix doc tests * clean merge * revert toc_tree changes * remove useless LogitProcessor * Update whisper .mdx * update config file doc * update configuration docstring * update test tokenization * update test tokenization * update tokenization whisper Added copied from where needed * update feature extraction * nit test name * style * quality * remove get suppress tokens and update non_speech tokens global variables * Update src/transformers/models/whisper/feature_extraction_whisper.py Co-authored-by:
Patrick von Platen <patrick.v.platen@gmail.com> * clean modeling whisper and test Removed the attention mask arguments that are deprecated * fix large test * Add multilingual audio test, and translate test * style * fix larg multilingual test * nits * add copied from for attention layer * remove attention masks in doc * add english normalizer * Update docs/source/en/model_doc/whisper.mdx Co-authored-by:
NielsRogge <48327001+NielsRogge@users.noreply.github.com> * Update src/transformers/models/whisper/configuration_whisper.py Co-authored-by:
Arthur <48595927+ArthurZucker@users.noreply.github.com> * Tidy up imports that don't require TF * Update tests - skip and no more skip * Update tests/generation/test_generation_tf_logits_process.py Co-authored-by:
Arthur Zucker <arthur.zucker@gmail.com> Co-authored-by:
NielsRogge <niels.rogge1@gmail.com> Co-authored-by:
Joao Gante <joao@huggingface.co>
-
- 29 Sep, 2022 1 commit
-
-
Aritra Roy Gosthipaty authored
* chore: initial commit * chore: adding util methods yet to work on the nn.functional.interpolate port with align_corener=True * chore: refactor the utils * used tf.compat.v1.image.resize to align the F.interpolate function * added type hints to the method signatures * added references to the gists where one 2 one alignment of torch and tf has been shown * chore: adding the layers * chore: porting all the layers from torch to tf This is the initial draft, nothing is tested yet. * chore: aligning the layers with reference to tf clip * chore: aligning the modules * added demaraction comments * added copied and adapted from comments * chore: aligning with CLIP * chore: wrangling the layers to keep it tf compatible * chore: aligning the names of the layers for porting * chore: style changes * chore: adding docs and inits * chore: adding tfp dependencis the code is taken from TAPAS * chore: initial commit for testing * chore: aligning the vision embeddings with the vit implementatino * chore: changing model prefix * chore: fixing the name of the model and the layer normalization test case * chore: every test passes but the slow ones * chore: fix style and integration test * chore: moving comments below decorators * chore: make fixup and fix-copies changes * chore: adding the Vision and Text Model to check_repo * chore: modifying the prefix name to align it with the torch implementation * chore: fix typo in configuration * choer: changing the name of the model variable * chore: adding segmentation flag * chore: gante's review * chore: style refactor * chore: amy review * chore: adding shape_list to parts that have been copied from other snippets * chore: init batchnorm with torch defaults * chore: adding shape_list to pass the tests * test fix: adding seed as 0 * set seed * chore: changing the straight through trick to fix -ve dimensinos * chore: adding a dimension to the loss * chore: adding reviewers and contributors names to the docs * chore: added changes after review * chore: code quality fixup * chore: fixing the segmentation snippet * chore: adding to the layer calls * chore: changing int32 to int64 for inputs of serving * chore: review changes * chore: style changes * chore: remove from_pt=True * fix: repo consistency Co-authored-by:ydshieh <ydshieh@users.noreply.github.com>
-
- 16 Sep, 2022 2 commits
-
-
Sylvain Gugger authored
-
Joao Gante authored
* resized models that we can actually load * separate embeddings check * add test for embeddings out of bounds * add fake slows
-
- 15 Sep, 2022 1 commit
-
-
Matt authored
* Override save() to use the serving signature as the default * Replace int32 with int64 in all our serving signatures * Remember one very important line so as not to break every test at once * Dtype fix for TFLED * dtype fix for shift_tokens_right in general * Dtype fixes in mBART and RAG * Fix dtypes for test_unpack_inputs * More dtype fixes * Yet more mBART + RAG dtype fixes * Yet more mBART + RAG dtype fixes * Add a check that the model actually has a serving method
-
- 10 Sep, 2022 1 commit
-
-
Joao Gante authored
-
- 09 Sep, 2022 1 commit
-
-
Matt authored
* Fix train_step and test_step, correctly enable CLIP fit test * Stop using get_args on older Python versions * Don't use get_origin either * UnionType is actually even newer, don't use that either * Apply the same fix to test_loss_computation * Just realized I was accidentally skipping a bunch of tests! * Fix test_loss_computation for models without separable labels * Fix scalar losses in test_step and train_step * Stop committing your breakpoints * Fix Swin loss shape * Fix Tapas loss shape * Shape fixes for TAPAS, DeIT, HuBERT and ViTMAE * Add loss computation to TFMobileBertForPreTraining * make fixup and move copied from statement * make fixup and move copied from statement * Correct copied from * Add labels and next_sentence_label inputs to TFMobileBERT * Make sure total_loss is always defined * Update tests/test_modeling_tf_common.py Co-authored-by:
amyeroberts <22614925+amyeroberts@users.noreply.github.com> * Fix copied from * Ensure CTC models get labels in tests * Ensure CTC models get labels in tests * Fix tests for vit_mae * Fix tests for vit_mae * Fix tests for vit_mae * Reduce batch size for wav2vec2 testing because it was causing OOM * Skip some TAPAS tests that are failing * Skip a failing HuBERT test * make style * Fix mobilebertforpretraining test * Skip Wav2Vec2 tests that use huge amounts of mem * Skip keras_fit for Wav2Vec2 as well Co-authored-by:
-
- 07 Sep, 2022 1 commit
-
-
Ankur Goyal authored
* [WIP] Skeleton of VisualQuestionAnweringPipeline extended to support LayoutLM-like models * Fixup * Use the full encoding * Basic refactoring to DocumentQuestionAnsweringPipeline * Cleanup * Improve args, docs, and implement preprocessing * Integrate OCR * Refactor question_answering pipeline * Use refactored QA code in the document qa pipeline * Fix tests * Some small cleanups * Use a string type annotation for Image.Image * Update encoding with image features * Wire through the basic docs * Handle invalid response * Handle empty word_boxes properly * Docstring fix * Integrate Donut model * Fixup * Incorporate comments * Address comments * Initial incorporation of tests * Address Comments * Change assert to ValueError * Comments * Wrap `score` in float to make it JSON serializable * Incorporate AutoModeLForDocumentQuestionAnswering changes * Fixup * Rename postprocess function * Fix auto import * Applying comments * Improve docs * Remove extra assets and add copyright * Address comments Co-authored-by:Ankur Goyal <ankur@impira.com>
-
- 02 Sep, 2022 1 commit
-
-
Joao Gante authored
-
- 01 Sep, 2022 1 commit
-
-
Joao Gante authored
-
- 29 Aug, 2022 1 commit
-
-
Lucain authored
-
- 05 Aug, 2022 1 commit
-
-
Sylvain Gugger authored
* Draft new cached_file * Initial draft for config and model * Small fixes * Fix first batch of tests * Look in cache when internet is down * Fix last tests * Bad black, not fixing all quality errors * Make diff less * Implement change for TF and Flax models * Add tokenizer and feature extractor * For compatibility with main * Add utils to move the cache and auto-do it at first use. * Quality * Deal with empty commit shas * Deal with empty etag * Address review comments
-
- 04 Aug, 2022 2 commits
-
-
Yih-Dar authored
Co-authored-by: -
nlpcat authored
* change shape to support dynamic batch input in tf.generate * add tests Co-authored-by:nlpcatcode <nlpcodecat@gmail.com>
-
- 03 Aug, 2022 1 commit
-
-
Daniel Suess authored
* Fix failing test_xla_generate_slow tests * Fix failing speech-to-text xla_generate tests
-
- 01 Aug, 2022 1 commit
-
-
Sylvain Gugger authored
* Rewrite push_to_hub to use upload_files * Adapt the doc a bit * Address review comments and clean doc
-
- 22 Jul, 2022 1 commit
-
-
amyeroberts authored
* Add serving_output and serving methods to some vision models * Add serving outputs for DeiT * Don't convert hidden states - differing shapes * Make saveable * Fix up * Make swin saveable * Add in tests * Fix funnel tests (can't convert to tensor) * Fix numpy call * Tidy up a bit * Add in hidden states - resnet * Remove numpy * Fix failing tests - tensor shape and skipping tests * Remove duplicated function * PR comments - formatting and var names * PR comments Add suggestions made by Joao Gante: * Use tf.shape instead of shape_list * Use @tooslow decorator on tests * Simplify some of the logic * PR comments Address Yih-Dar Sheih comments - making tensor names consistent and make types float * Types consistent with docs; disable test on swin (slow) * CI trigger * Change input_features to float32 * Add serving_output for segformer * Fixup Co-authored-by:Amy Roberts <amyeroberts@users.noreply.github.com>
-
- 18 Jul, 2022 1 commit
-
-
Yih-Dar authored
Co-authored-by:
-
- 13 Jul, 2022 1 commit
-
-
Joao Gante authored
-
- 04 Jul, 2022 1 commit
-
-
Matt authored
* Return scalar losses instead of per-sample means * Make loss shape (1,) instead of scalar * Allow scalar losses in test_loss_computation * Allow scalar losses in test_loss_computation * Allow scalar losses in test_loss_computation * Remove XLA loss function for RAG
-
- 01 Jul, 2022 1 commit
-
-
Matt authored
* Copy inputs to train and test step before modifying them, as this breaks things * Add XLA tests, fix our loss functions to be XLA-compatible * make fixup * Update loss computation test to expect vector of per-sample losses * Patch loss for TFLED * Patch loss for TFAlbert * Add a tf_legacy_loss config flag that enables old loss functions * Stop using config.get() because it's not a dict * Skip loss computation test for RAG because its loss is very strange and I'm afraid to rewrite it * make fixup * Add XLA-compatible RAG loss * Fix dtype of loss mask for TFAlbert * Fix test for XLNet too because it overrides the default one * make fixup * Fix config test * No more depending on GPU NaN behaviour * Add test, avoid potential zero division * Fix test item assignment * Fix loss computation masking test * make fixup * Fix dtype bugs
-
- 30 Jun, 2022 1 commit
-
-
Joao Gante authored
* sharded conversion; add flag to control max hidden error * better hidden name matching * Add test: load TF from PT shards * fix test (PT data must be local)
-
- 29 Jun, 2022 1 commit
-
-
Joao Gante authored
TF: XLA beam search + most generation-compatible models are now also XLA-generate-compatible (#17857) * working beam search
馃帀 * XLA generation compatible with ALL classes * add xla generation slow test
-
- 23 Jun, 2022 1 commit
-
-
Matt authored
* Fix tests that broke when models used batchnorm * Initializing the model twice does not actually... ...give you the same weights each time. I am good at machine learning. * Fix speed regression
-
- 21 Jun, 2022 2 commits
-
-
Arthur authored
* initial commit * update modeeling tf utils * quality * clean and update args * update * remove potential bug * code quality * update * update max shard * update tests for sharding from pretrained * fix remaining test * make style * h5py if tf available * update and fix test * fix test * style * modified push to hub to support shard for TF * quick fix * update code * merge branch main and style * Apply suggestions from code review Co-authored-by:
-
Lysandre Debut authored
* Prepare CI for v0.8.0 * pin hfh (revert before merge) * Revert "pin hfh (revert before merge)" This reverts commit a0103140e1c77b810ffcb735192968bc03be3e1f. * Test rc3 * Test latest rc * Unpin to the RC Co-authored-by:Sylvain Gugger <Sylvain.gugger@gmail.com>
-
- 20 Jun, 2022 1 commit
-
-
Joao Gante authored
* Also propagate changes to blenderbot, blenderbot_small, marian, mbart, and pegasus
-
- 09 Jun, 2022 2 commits
-
-
amyeroberts authored
* Use shape_list to safely get shapes * Add relevant test * Tidy and add metrics * Resolve dynamic shaping issues and move test * Tidy up and all samples in batch * Formatting
-
amyeroberts authored
-
