1. 12 Oct, 2022 6 commits
  2. 11 Oct, 2022 9 commits
  3. 10 Oct, 2022 11 commits
  4. 08 Oct, 2022 1 commit
  5. 07 Oct, 2022 5 commits
    • Sylvain Gugger's avatar
      Rework pipeline tests (#19366) · 9ac586b3
      Sylvain Gugger authored
      * Rework pipeline tests
      
      * Try to fix Flax tests
      
      * Try to put it before
      
      * Use a new decorator instead
      
      * Remove ignore marker since it doesn't work
      
      * Filter pipeline tests
      
      * Woopsie
      
      * Use the fitlered list
      
      * Clean up and fake modif
      
      * Remove init
      
      * Revert fake modif
      9ac586b3
    • Alara Dirik's avatar
      Improve and fix ImageSegmentationPipeline (#19367) · 983451a1
      Alara Dirik authored
      - Fixes the image segmentation pipeline test failures caused by changes to the postprocessing methods of supported models
      - Updates the ImageSegmentationPipeline tests
      - Improves docs, adds 'task' argument to optionally perform semantic, instance or panoptic segmentation
      983451a1
    • Dean Wyatte's avatar
      Export TensorFlow models to ONNX with dynamic input shapes (#19255) · a26d71d6
      Dean Wyatte authored
      * validate onnx models with a different input geometry than saved with
      
      * only test working features for now
      
      * simpler test skipping
      
      * rm TODO
      
      * expose batch_size/seq_length on vit
      
      * skip certain name, feature, framework parameterizations known to fail validation
      
      * Trigger CI
      
      * Trigger CI
      a26d71d6
    • Amrit Sahu's avatar
      [WIP] Add ZeroShotObjectDetectionPipeline (#18445) (#18930) · e9a49bab
      Amrit Sahu authored
      * Add ZeroShotObjectDetectionPipeline (#18445)
      
      * Add AutoModelForZeroShotObjectDetection task
      
      This commit also adds the following
      
      - Add explicit _processor method for ZeroShotObjectDetectionPipeline.
        This is necessary as pipelines don't auto infer processors yet and
        `OwlVitProcessor` wraps tokenizer and feature_extractor together, to
        process multiple images at once
      
      - Add auto tests and other tests for ZeroShotObjectDetectionPipeline
      
      * Add AutoModelForZeroShotObjectDetection task
      
      This commit also adds the following
      
      - Add explicit _processor method for ZeroShotObjectDetectionPipeline.
        This is necessary as pipelines don't auto infer processors yet and
        `OwlVitProcessor` wraps tokenizer and feature_extractor together, to
        process multiple images at once
      
      - Add auto tests and other tests for ZeroShotObjectDetectionPipeline
      
      * Add batching for ZeroShotObjectDetectionPipeline
      
      * Fix doc-string ZeroShotObjectDetectionPipeline
      
      * Fix output format: ZeroShotObjectDetectionPipeline
      e9a49bab
    • Bibhabasu Mohapatra's avatar
      add ONNX support for swin transformer (#19390) · e162cebf
      Bibhabasu Mohapatra authored
      
      
      * swin transformer onnx support
      
      * Updated image dimensions as dynamic
      Co-authored-by: default avatarlewtun <lewis.c.tunstall@gmail.com>
      Co-authored-by: default avatarlewtun <lewis.c.tunstall@gmail.com>
      e162cebf
  6. 05 Oct, 2022 6 commits
    • Sylvain Gugger's avatar
      Fix pipeline tests for Roberta-like tokenizers (#19365) · 7e7f62bf
      Sylvain Gugger authored
      * Fix pipeline tests for Roberta-like tokenizers
      
      * Fix fix
      7e7f62bf
    • Arthur's avatar
      Add WhisperModel to transformers (#19166) · 45e14038
      Arthur authored
      
      
      * simplify loop
      
      * add featur extractor
      
      * add model
      
      * start conversion
      
      * add dropout
      
      * initial commit of test files
      
      * copnversion for all models
      
      * update processor for correct padding
      
      * update feature extraction
      
      * update integration test logits match
      
      * fmnt: off for the logits
      
      * on the fly mel bank
      
      * small nit
      
      * update test
      
      * update tokenizer
      
      * nit feature extraction
      
      * update
      
      * update tokenizer test
      
      * adds logit processor and update tokenizer to get supress tokens
      
      * style
      
      * clean convert
      
      * revert to original modeling tf utils
      
      * Update
      
      * update
      
      * nit
      
      * clean convert file
      
      * update tests and nits
      
      * quality
      
      * slow generation test
      
      * ffn_dim to allow customization
      
      * update readme
      
      * add to toctreee
      
      * start fixing integration tests
      
      * update tests and code
      
      * fix feature extractor
      
      * fix config tests common
      
      * update code to fix tests
      
      * fix feature exctractor
      
      * nit feature extraction
      
      * update test for new feature extractor
      
      * style
      
      * add absrtact
      
      * large logits wioth custom decoder input ids
      
      * wraap around is otrch available
      
      * fix feature extractor
      
      * correct logits for whisper small.en
      
      * nit
      
      * fix encoder_attentino_mask
      
      * some fixes
      
      * remove unnecessary inputs
      
      * nits
      
      * add normalizer file
      
      * update etst tokenization
      
      * fix attention mask not defined
      
      * Add model to README
      
      * Fix doc tests
      
      * fix generate
      
      * remove uncoder attention mask useless
      
      * update test modeling whisper
      
      * update condfig to add second non supress tokens
      
      * nits on feature exrtactor
      
      * nit for test tokenizers
      
      * update etsts
      
      * update tests
      
      * update tokenization test
      
      * fixup
      
      * invalidated hf token. Clean convert openai to whisper
      
      * fix logit tests
      
      * fixup
      
      * clean merge
      
      * revert toc_tree changes
      
      * remove useless LogitProcessor
      
      * Update whisper .mdx
      
      * update config file doc
      
      * update configuration docstring
      
      * update test tokenization
      
      * update test tokenization
      
      * update tokenization whisper
      Added copied from where needed
      
      * update feature extraction
      
      * nit test name
      
      * style
      
      * quality
      
      * remove get suppress tokens and update non_speech tokens global variables
      
      * Update src/transformers/models/whisper/feature_extraction_whisper.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * clean modeling whisper and test
      Removed the attention mask arguments that are deprecated
      
      * fix large test
      
      * Add multilingual audio test, and translate test
      
      * style
      
      * fix larg multilingual test
      
      * nits
      
      * Update docs/source/en/model_doc/whisper.mdx
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * add copied from for attention layer
      
      * remove attention masks in doc
      
      * add english normalizer
      
      * update tokenization test
      
      * remove copied from in whisper attention : no bias in k_proj only
      
      * wrap around dependencies in english normalizer
      
      * style
      
      * correct import generation logits
      
      * for now, wrap feature extractor with torch
      
      * Update src/transformers/models/whisper/convert_openai_whisper_to_tfms.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update docs/source/en/model_doc/whisper.mdx
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * remove torch depencies for feature extraction and style
      
      * fixup
      
      * nit
      
      * update logitds
      
      * style
      
      * nit
      
      * nits and fix final tests
      
      * add `is_more_itertools_available` to utils
      
      * quality
      
      * add begin supress tokens, supress tokens to generate args and config
      
      * clean supressTokensLogitProcessor in generation logits
      
      * Nit naming
      
      * add supressTokensAtBegin
      
      * udpate tests, supress tokens to None or correct values
      
      * nit and style
      
      * update RAG to fit test and generate_logit
      
      * add copy pasted statment on english normalizer
      
      * add arguments to config_common_kwargs
      
      * Update src/transformers/generation_utils.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update src/transformers/generation_logits_process.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * revert changes based on reviews
      
      * update doc and nits
      
      * more nits
      
      * last nits
      
      * update test configuration common
      
      * add BART name in decoder attention mask documentation
      
      * Update src/transformers/models/whisper/modeling_whisper.py
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      
      * style
      
      * nit
      
      * nit
      
      * add english.json file to git
      
      * nits on documentation
      
      * nit
      
      * nits
      
      * last styling
      
      * add main toctree file
      
      * remove sentence piece dependency
      
      * clean init file
      
      * fix tokenizer that has no dependencies on sentencepiece
      
      * update whisper init file, nit
      
      * remove english.json file
      
      * add get decoder prompt id
      
      * revert changes and add forced logit processor
      
      * nit
      
      * clean normalizer
      
      * remove protected
      
      * update
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * update based on review
      
      * Update src/transformers/models/whisper/configuration_whisper.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * add batched tests
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarNielsRogge <niels.rogge1@gmail.com>
      Co-authored-by: default avatarNielsRogge <48327001+NielsRogge@users.noreply.github.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      45e14038
    • Alara Dirik's avatar
      Fix MaskFormer failing postprocess tests (#19354) · 7598791c
      Alara Dirik authored
      Ensures post_process_instance_segmentation and post_process_panoptic_segmentation methods return a tensor of shape (target_height, target_width) filled with -1 values if no segment with score > threshold is found.
      7598791c
    • Sylvain Gugger's avatar
      c875a96e
    • r-terada's avatar
      Add sudachi and jumanpp tokenizers for bert_japanese (#19043) · 2f53ab57
      r-terada authored
      * add sudachipy and jumanpp tokenizers for bert_japanese
      
      * use ImportError instead of ModuleNotFoundError in SudachiTokenizer and JumanppTokenizer
      
      * put test cases of test_tokenization_bert_japanese in one line
      
      * add require_sudachi and require_jumanpp decorator for testing
      
      * add sudachi and pyknp(jumanpp) to dependencies
      
      * remove sudachi_dict_small and sudachi_dict_full from dependencies
      
      * empty commit for ci
      2f53ab57
    • Alara Dirik's avatar
      Maskformer post-processing fixes and improvements (#19172) · 07e94bf1
      Alara Dirik authored
      - Improves MaskFormer docs, corrects minor typos
      - Restructures MaskFormerFeatureExtractor.post_process_panoptic_segmentation for better readability, adds target_sizes argument for optional resizing
      - Adds post_process_semantic_segmentation and post_process_instance_segmentation methods.
      - Adds a deprecation warning to post_process_segmentation method in favour of post_process_instance_segmentation
      07e94bf1
  7. 04 Oct, 2022 2 commits