1. 26 Jan, 2024 1 commit
  2. 25 Jan, 2024 2 commits
    • Yusuf's avatar
      Update question_answering.md (#28694) · 24f1a00e
      Yusuf authored
      fix typo:
      
      from:
      
       "model = TFAutoModelForQuestionAnswering("distilbert-base-uncased")"
      
      to:
      model = TFAutoModelForQuestionAnswering.from_pretrained("distilbert-base-uncased")
      24f1a00e
    • NielsRogge's avatar
      Add Depth Anything (#28654) · 963db81a
      NielsRogge authored
      * First draft
      
      * More improvements
      
      * More improvements
      
      * More improvements
      
      * More improvements
      
      * Add docs
      
      * Remove file
      
      * Add copied from
      
      * Address comments
      
      * Address comments
      
      * Address comments
      
      * Fix style
      
      * Update docs
      
      * Convert all checkpoints, add integration test
      
      * Rename checkpoints
      
      * Add pretrained backbone attributes
      
      * Fix default config
      
      * Address comment
      
      * Add figure to docs
      
      * Fix bug thanks to @xenova
      
      * Update conversion script
      
      * Fix integration test
      963db81a
  3. 18 Jan, 2024 1 commit
    • Yoach Lacombe's avatar
      Add new meta w2v2-conformer BERT-like model (#28165) · d2cdefb9
      Yoach Lacombe authored
      
      
      * first commit
      
      * correct default value non causal
      
      * update config and modeling code
      
      * update converting checkpoint
      
      * clean modeling and fix tests
      
      * make style
      
      * add new config parameters to docstring
      
      * fix copied from statements
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      
      * make position_embeddings_type docstrings clearer
      
      * clean converting script
      
      * remove function not used
      
      * clean modeling file
      
      * apply suggestion for test file + add convert script to not_doctested
      
      * modify tests according to review - cleaner logic and more tests
      
      * Apply nit suggestions from code review
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * add checker of valid position embeddings type
      
      * instantiate new layer norm layer with the right eps
      
      * fix freeze_feature_encoder since it can be None in some cases
      
      * add test same output in convert script
      
      * restore wav2vec2conformer and add new model
      
      * create processor and FE + clean
      
      * add new model code
      
      * fix convert script and set default config parameters
      
      * correct model id paths
      
      * make style
      
      * make fix-copies and cleaning files
      
      * fix copied from statements
      
      * complete .md and fixe copies
      
      * clean convert script argument defaults
      
      * fix config parameters docstrings
      
      * fix config docstring
      
      * add copied from and enrich FE tests
      
      * fix copied from and repo-consistency
      
      * add autotokenizer
      
      * make test input length shorter and change docstring code
      
      * fix docstrings and copied from
      
      * add add_adapter to ASR training example
      
      * make testing of adapters more robust
      
      * adapt to multi adapter layers
      
      * refactor input_values->input_features and remove w2v2-bert feature extractor
      
      * remove pretraining model
      
      * remove depreciated features and useless lines
      
      * add copied from and ignore statements to modeling tests
      
      * remove pretraining model #2
      
      * change import in convert script
      
      * change default in convert script
      
      * update readme and remove useless line
      
      * Update tests/models/wav2vec2_bert/test_processor_wav2vec2_bert.py
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * refactor BERT to Bert for consistency
      
      * remove useless ignore copy statement
      
      * add persistent to buffer in rotary
      
      * add eps in LayerNorm init and remove copied from
      
      * add adapter activation parameters and add copied from statements
      
      * Fix copied statements and add unitest.skip reasons
      
      * add copied statement in test_processor
      
      * refactor processor
      
      * make style
      
      * replace numpy random by torch rand
      
      * remove expected output CTC
      
      * improve converting script with processor class
      
      * Apply suggestions from code review
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * remove gumbel class
      
      * remove tests related to previously deleted class
      
      * Update src/transformers/models/wav2vec2_bert/configuration_wav2vec2_bert.py
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * correct typos
      
      * remove uused parameters
      
      * update processor to takes both text and audio
      
      * update checkpoints
      
      * update expected output and add ctc expected output
      
      * add label_attention_mask
      
      * replace pt with np in processor tests
      
      * fix typo
      
      * revert to behaviour with labels_attention_mask
      
      ---------
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      d2cdefb9
  4. 17 Jan, 2024 1 commit
    • Junyang Lin's avatar
      Add qwen2 (#28436) · d6ffe74d
      Junyang Lin authored
      
      
      * add config, modeling, and tokenization
      
      * add auto and init
      
      * update readme
      
      * update readme
      
      * update team name
      
      * fixup
      
      * fixup
      
      * update config
      
      * update code style
      
      * update for fixup
      
      * update for fixup
      
      * update for fixup
      
      * update for testing
      
      * update for testing
      
      * fix bug for config and tokenization
      
      * fix bug for bos token
      
      * not doctest
      
      * debug tokenizer
      
      * not doctest
      
      * debug tokenization
      
      * debug init for tokenizer
      
      * fix style
      
      * update init
      
      * delete if in token auto
      
      * add tokenizer doc
      
      * add tokenizer in init
      
      * Update dummy_tokenizers_objects.py
      
      * update
      
      * update
      
      * debug
      
      * Update tokenization_qwen2.py
      
      * debug
      
      * Update convert_slow_tokenizer.py
      
      * add copies
      
      * add copied from and make style
      
      * update files map
      
      * update test
      
      * fix style
      
      * fix merge reading and update tests
      
      * fix tests
      
      * fix tests
      
      * fix style
      
      * debug a variable in readme
      
      * Update src/transformers/models/qwen2/configuration_qwen2.py
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * update test and copied from
      
      * fix style
      
      * update qwen2 tokenization  and tests
      
      * Update tokenization_qwen2.py
      
      * delete the copied from after property
      
      * fix style
      
      * update tests
      
      * update tests
      
      * add copied from
      
      * fix bugs
      
      * update doc
      
      * add warning for sliding window attention
      
      * update qwen2 tokenization
      
      * fix style
      
      * Update src/transformers/models/qwen2/modeling_qwen2.py
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * fix tokenizer fast
      
      ---------
      Co-authored-by: default avatarRen Xuancheng <jklj077@users.noreply.github.com>
      Co-authored-by: default avatarrenxuancheng.rxc <renxuancheng.rxc@alibaba-inc.com>
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      d6ffe74d
  5. 03 Jan, 2024 2 commits
    • Connor Henderson's avatar
      Add FastSpeech2Conformer (#23439) · d83ff5ee
      Connor Henderson authored
      * start - docs, SpeechT5 copy and rename
      
      * add relevant code from FastSpeech2 draft, have tests pass
      
      * make it an actual conformer, demo ex.
      
      * matching inference with original repo, includes debug code
      
      * refactor nn.Sequentials, start more desc. var names
      
      * more renaming
      
      * more renaming
      
      * vocoder scratchwork
      
      * matching vocoder outputs
      
      * hifigan vocoder conversion script
      
      * convert model script, rename some config vars
      
      * replace postnet with speecht5's implementation
      
      * passing common tests, file cleanup
      
      * expand testing, add output hidden states and attention
      
      * tokenizer + passing tokenizer tests
      
      * variety of updates and tests
      
      * g2p_en pckg setup
      
      * import structure edits
      
      * docstrings and cleanup
      
      * repo consistency
      
      * deps
      
      * small cleanup
      
      * forward signature param order
      
      * address comments except for masks and labels
      
      * address comments on attention_mask and labels
      
      * address second round of comments
      
      * remove old unneeded line
      
      * address comments part 1
      
      * address comments pt 2
      
      * rename auto mapping
      
      * fixes for failing tests
      
      * address comments part 3 (bart-like, train loss)
      
      * make style
      
      * pass config where possible
      
      * add forward method + tests to WithHifiGan model
      
      * make style
      
      * address arg passing and generate_speech comments
      
      * address Arthur comments
      
      * address Arthur comments pt2
      
      * lint  changes
      
      * Sanchit comment
      
      * add g2p-en to doctest deps
      
      * move up self.encoder
      
      * onnx compatible tensor method
      
      * fix is symbolic
      
      * fix paper url
      
      * move models to espnet org
      
      * make style
      
      * make fix-copies
      
      * update docstring
      
      * Arthur comments
      
      * update docstring w/ new updates
      
      * add model architecture images
      
      * header size
      
      * md wording update
      
      * make style
      d83ff5ee
    • lain's avatar
      fix documentation for zero_shot_object_detection (#28267) · 6eba901d
      lain authored
      remove broken space
      6eba901d
  6. 22 Dec, 2023 1 commit
  7. 18 Dec, 2023 1 commit
  8. 11 Dec, 2023 3 commits
  9. 07 Dec, 2023 1 commit
  10. 04 Dec, 2023 1 commit
  11. 30 Nov, 2023 1 commit
    • Yoach Lacombe's avatar
      Add SeamlessM4T v2 (#27779) · 29f1aee3
      Yoach Lacombe authored
      
      
      * add working convertion script
      
      * first non-working version of modeling code
      
      * update modeling code (working)
      
      * make style
      
      * make fix-copies
      
      * add config docstrings
      
      * add config to ignore docstrings formatage due to unconventional markdown
      
      * fix copies
      
      * fix generation num_return_sequences
      
      * enrich docs
      
      * add and fix tests beside integration tests
      
      * update integration tests
      
      * update repo id
      
      * add tie weights and make style
      
      * correct naming in .md
      
      * fix imports and so on
      
      * correct docstrings
      
      * fix fp16 speech forward
      
      * fix speechencoder attention
      
      * make style
      
      * fix copied from
      
      * rename SeamlessM4Tv2-v2 to SeamlessM4Tv2
      
      * Apply suggestions on configuration
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * remove useless public models
      
      * fix private models + better naming for T2U models
      
      * clean speech encoder relative position embeddings
      
      * refactor chunk attention
      
      * add docstrings to chunk attention method
      
      * improve naming and docstrings
      
      * rename some attention variables + add temperature sampling in T2U model
      
      * rename DOCSTRINGS variable names
      
      * make style + remove 2 useless config parameters
      
      * enrich model card
      
      * remove any attention_head reference + fix temperature in T2U
      
      * new fmt and make style
      
      * Apply suggestions from code review
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * rename spkr_id->speaker_id and change docstrings of get_char_input_ids
      
      * simplify v2attention
      
      * make style
      
      * Update seamless_m4t_v2.md
      
      * update code and tests with last update
      
      * update repo ids
      
      * fill article name, abstract andauthors
      
      * update not_doctested and slow_doc tests
      
      ---------
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      29f1aee3
  12. 24 Nov, 2023 1 commit
  13. 23 Nov, 2023 1 commit
  14. 17 Nov, 2023 1 commit
  15. 16 Nov, 2023 1 commit
    • Arthur's avatar
      [`Styling`] stylify using ruff (#27144) · 651408a0
      Arthur authored
      
      
      * try to stylify using ruff
      
      * might need to remove these changes?
      
      * use ruf format andruff check
      
      * use isinstance instead of type comparision
      
      * use # fmt: skip
      
      * use # fmt: skip
      
      * nits
      
      * soem styling changes
      
      * update ci job
      
      * nits isinstance
      
      * more files update
      
      * nits
      
      * more nits
      
      * small nits
      
      * check and format
      
      * revert wrong changes
      
      * actually use formatter instead of checker
      
      * nits
      
      * well docbuilder is overwriting this commit
      
      * revert notebook changes
      
      * try to nuke docbuilder
      
      * style
      
      * fix feature exrtaction test
      
      * remve `indent-width = 4`
      
      * fixup
      
      * more nits
      
      * update the ruff version that we use
      
      * style
      
      * nuke docbuilder styling
      
      * leve the print for detected changes
      
      * nits
      
      * Remove file I/O
      Co-authored-by: default avatarcharliermarsh <charlie.r.marsh@gmail.com>
      
      * style
      
      * nits
      
      * revert notebook changes
      
      * Add # fmt skip when possible
      
      * Add # fmt skip when possible
      
      * Fix
      
      * More `  # fmt: skip` usage
      
      * More `  # fmt: skip` usage
      
      * More `  # fmt: skip` usage
      
      * NIts
      
      * more fixes
      
      * fix tapas
      
      * Another way to skip
      
      * Recommended way
      
      * Fix two more fiels
      
      * Remove asynch
      Remove asynch
      
      ---------
      Co-authored-by: default avatarcharliermarsh <charlie.r.marsh@gmail.com>
      651408a0
  16. 10 Nov, 2023 1 commit
    • Susnato Dhar's avatar
      Add Phi-1 and Phi-1_5 (#26170) · e1c3ac25
      Susnato Dhar authored
      * only dir not even init
      
      * init
      
      * tokenizer removed and reference of codegen added
      
      * modeling file updated a lot remaining app_rotary_emb
      
      * conversion script done
      
      * conversion script fixed, a lot of factoring done and most tests pass
      
      * added token_clf and extractive_QA_head
      
      * integration tests pass
      
      * flash attn tests pass!
      
      * config done
      
      * more docs in modeling file
      
      * some style fix
      
      * style and others
      
      * doc test error fix
      
      * more doc fix
      
      * some attention fixes
      
      * most fixes
      
      * style and other fixes
      
      * docs fix and config
      
      * doc fix
      
      * some comments
      
      * conversion script updated
      
      * conversion script updated
      
      * Revert "conversion script updated"
      
      This reverts commit e92378c54084ec0747041b113083d1746ecb6c7f.
      
      * final comments
      
      * add Phi to language_modeling.md
      
      * edit phi.md file
      
      * rebase and fix
      
      * removed phi-1.5 example
      
      * changed model_type from 'phi'->'mixformer-sequential'
      
      * small change
      
      * small change
      
      * revert \small change
      
      * changed mixformer-sequential->phi
      
      * small change
      
      * added phi-1.5 example instead of phi-1
      
      * doc test might pass now
      
      * rebase and small change
      
      * added the dropout layer
      
      * more fixes
      
      * modified .md file
      
      * very very small doc change
      e1c3ac25
  17. 06 Nov, 2023 1 commit
    • Akshay Chintalapati's avatar
      Update sequence_classification.md (#27281) · e9dbd392
      Akshay Chintalapati authored
      I'm adding accelerate as one of the libraries to install because otherwise when running the Trainer, the model errorr out with the error. 
      
      ImportError: Using the `Trainer` with `PyTorch` requires `accelerate>=0.20.1`: Please run `pip install transformers[torch]` or `pip install accelerate -U`
      
      Further context: 
      1. I've tried this across different environments so I believe that the environment is not the issue. 
      2. I had the latest transformers library version running. 
      3. Typically even after install accelerate and import it, it wouldn't resolve the issue until I restart the notebook and try again.
      e9dbd392
  18. 01 Nov, 2023 1 commit
  19. 23 Oct, 2023 1 commit
    • Yoach Lacombe's avatar
      Add Seamless M4T model (#25693) · cb45f71c
      Yoach Lacombe authored
      
      
      * first raw commit
      
      * still POC
      
      * tentative convert script
      
      * almost working speech encoder conversion scripts
      
      * intermediate code for encoder/decoders
      
      * add modeling code
      
      * first version of speech encoder
      
      * make style
      
      * add new adapter layer architecture
      
      * add adapter block
      
      * add first tentative config
      
      * add working speech encoder conversion
      
      * base model convert works now
      
      * make style
      
      * remove unnecessary classes
      
      * remove unecessary functions
      
      * add modeling code speech encoder
      
      * rework logics
      
      * forward pass of sub components work
      
      * add modeling codes
      
      * some config modifs and modeling code modifs
      
      * save WIP
      
      * new edits
      
      * same output speech encoder
      
      * correct attention mask
      
      * correct attention mask
      
      * fix generation
      
      * new generation logics
      
      * erase comments
      
      * make style
      
      * fix typo
      
      * add some descriptions
      
      * new state
      
      * clean imports
      
      * add tests
      
      * make style
      
      * make beam search and num_return_sequences>1 works
      
      * correct edge case issue
      
      * correct SeamlessM4TConformerSamePadLayer copied from
      
      * replace ACT2FN relu by nn.relu
      
      * remove unecessary return variable
      
      * move back a class
      
      * change name conformer_attention_mask ->conv_attention_mask
      
      * better nit code
      
      * add some Copied from statements
      
      * small nits
      
      * small nit in dict.get
      
      * rename t2u model -> conditionalgeneration
      
      * ongoing refactoring of structure
      
      * update models architecture
      
      * remove SeamlessM4TMultiModal classes
      
      * add tests
      
      * adapt tests
      
      * some non-working code for vocoder
      
      * add seamlessM4T vocoder
      
      * remove buggy line
      
      * fix some hifigan related bugs
      
      * remove hifigan specifc config
      
      * change
      
      * add WIP tokenization
      
      * add seamlessM4T working tokenzier
      
      * update tokenization
      
      * add tentative feature extractor
      
      * Update converting script
      
      * update working FE
      
      * refactor input_values -> input_features
      
      * update FE
      
      * changes in generation, tokenizer and modeling
      
      * make style and add t2u_decoder_input_ids
      
      * add intermediate outputs for ToSpeech models
      
      * add vocoder to speech models
      
      * update valueerror
      
      * update FE with languages
      
      * add vocoder convert
      
      * update config docstrings and names
      
      * update generation code and configuration
      
      * remove todos and update config.pad_token_id to generation_config.pad_token_id
      
      * move block vocoder
      
      * remove unecessary code and uniformize tospeech code
      
      * add feature extractor import
      
      * make style and fix some copies from
      
      * correct consistency + make fix-copies
      
      * add processor code
      
      * remove comments
      
      * add fast tokenizer support
      
      * correct pad_token_id in M4TModel
      
      * correct config
      
      * update tests and codes  + make style
      
      * make some suggested correstion - correct comments and change naming
      
      * rename some attributes
      
      * rename some attributes
      
      * remove unecessary sequential
      
      * remove option to use dur predictor
      
      * nit
      
      * refactor hifigan
      
      * replace normalize_mean and normalize_var with do_normalize + save lang ids to generation config
      
      * add tests
      
      * change tgt_lang logic
      
      * update generation ToSpeech
      
      * add support import SeamlessM4TProcessor
      
      * fix generate
      
      * make tests
      
      * update integration tests, add option to only return text and update tokenizer fast
      
      * fix wrong function call
      
      * update import and convert script
      
      * update integration tests + update repo id
      
      * correct paths and add first test
      
      * update how new attention masks are computed
      
      * update tests
      
      * take first care of batching in vocoder code
      
      * add batching with the vocoder
      
      * add waveform lengths to model outputs
      
      * make style
      
      * add generate kwargs + forward kwargs of M4TModel
      
      * add docstrings forward methods
      
      * reformate docstrings
      
      * add docstrings t2u model
      
      * add another round of modeling docstrings + reformate speaker_id -> spkr_id
      
      * make style
      
      * fix check_repo
      
      * make style
      
      * add seamlessm4t to toctree
      
      * correct check_config_attributes
      
      * write config docstrings + some modifs
      
      * make style
      
      * add docstrings tokenizer
      
      * add docstrings to processor, fe and tokenizers
      
      * make style
      
      * write first version of model docs
      
      * fix FE + correct FE test
      
      * fix tokenizer + add correct integration tests
      
      * fix most tokenization tests
      
      * make style
      
      * correct most processor test
      
      * add generation tests and fix num_return_sequences > 1
      
      * correct integration tests -still one left
      
      * make style
      
      * correct position embedding
      
      * change numbeams to 1
      
      * refactor some modeling code and correct one test
      
      * make style
      
      * correct typo
      
      * refactor intermediate fnn
      
      * refactor feedforward conformer
      
      * make style
      
      * remove comments
      
      * make style
      
      * fix tokenizer tests
      
      * make style
      
      * correct processor tests
      
      * make style
      
      * correct S2TT integration
      
      * Apply suggestions from Sanchit code review
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      
      * correct typo
      
      * replace torch.nn->nn + make style
      
      * change Output naming (waveforms -> waveform) and ordering
      
      * nit renaming and formating
      
      * remove return None when not necessary
      
      * refactor SeamlessM4TConformerFeedForward
      
      * nit typo
      
      * remove almost copied from comments
      
      * add a copied from comment and remove an unecessary dropout
      
      * remove inputs_embeds from speechencoder
      
      * remove backward compatibiliy function
      
      * reformate class docstrings for a few components
      
      * remove unecessary methods
      
      * split over 2 lines smthg hard to read
      
      * make style
      
      * replace two steps offset by one step as suggested
      
      * nice typo
      
      * move warnings
      
      * remove useless lines from processor
      
      * make generation non-standard test more robusts
      
      * remove torch.inference_mode from tests
      
      * split integration tests
      
      * enrich md
      
      * rename control_symbol_vocoder_offset->vocoder_offset
      
      * clean convert file
      
      * remove tgt_lang and src_lang from FE
      
      * change generate docstring of ToText models
      
      * update generate docstring of tospeech models
      
      * unify how to deal withtext_decoder_input_ids
      
      * add default spkr_id
      
      * unify tgt_lang for t2u_model
      
      * simplify tgt_lang verification
      
      * remove a todo
      
      * change config docstring
      
      * make style
      
      * simplify t2u_tgt_lang_id
      
      * make style
      
      * enrich/correct comments
      
      * enrich .md
      
      * correct typo in docstrings
      
      * add torchaudio dependency
      
      * update tokenizer
      
      * make style and fix copies
      
      * modify SeamlessM4TConverter with new tokenizer behaviour
      
      * make style
      
      * correct small typo docs
      
      * fix import
      
      * update docs and add requirement to tests
      
      * add convert_fairseq2_to_hf in utils/not_doctested.txt
      
      * update FE
      
      * fix imports and make style
      
      * remove torchaudio in FE test
      
      * add seamless_m4t.md to utils/not_doctested.txt
      
      * nits and change the way docstring dataset is loaded
      
      * move checkpoints from ylacombe/ to facebook/ orga
      
      * refactor warning/error to be in the 119 line width limit
      
      * round overly precised floats
      
      * add stereo audio behaviour
      
      * refactor .md and make style
      
      * enrich docs with more precised architecture description
      
      * readd undocumented models
      
      * make fix-copies
      
      * apply some suggestions
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * correct bug from previous commit
      
      * refactor a parameter allowing to clean the code + some small nits
      
      * clean tokenizer
      
      * make style and fix
      
      * make style
      
      * clean tokenizers arguments
      
      * add precisions for some tests
      
      * move docs from not_tested to slow
      
      * modify tokenizer according to last comments
      
      * add copied from statements in tests
      
      * correct convert script
      
      * correct parameter docstring style
      
      * correct tokenization
      
      * correct multi gpus
      
      * make style
      
      * clean modeling code
      
      * make style
      
      * add copied from statements
      
      * add copied statements
      
      * add support with ASR pipeline
      
      * remove file added inadvertently
      
      * fix docstrings seamlessM4TModel
      
      * add seamlessM4TConfig to OBJECTS_TO_IGNORE due of unconventional markdown
      
      * add seamlessm4t to assisted generation ignored models
      
      ---------
      Co-authored-by: default avatarSanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      cb45f71c
  20. 18 Oct, 2023 2 commits
  21. 16 Oct, 2023 1 commit
  22. 12 Oct, 2023 1 commit
  23. 09 Oct, 2023 1 commit
    • tom white's avatar
      fix typos in idefics.md (#26648) · c7f01bee
      tom white authored
      * fix typos in idefics.md
      
      Two typos found in reviewing this documentation.
      
      1) max_new_tokens=4, is not sufficient to generate "Vegetables" as indicated - you will get only "Veget". (incidentally - some mention of how to select this value might be useful as it seems to change in each example)
      
      2) inputs = processor(prompts, return_tensors="pt").to(device) as inputs need to be on the same device (as they are in all other examples on the page)
      
      * Update idefics.md
      
      Change device to cuda explicitly to match other examples
      c7f01bee
  24. 27 Sep, 2023 2 commits
  25. 22 Sep, 2023 1 commit
  26. 18 Sep, 2023 1 commit
  27. 15 Sep, 2023 1 commit
  28. 14 Sep, 2023 1 commit
    • Jinho Park's avatar
      Add BROS (#23190) · 17fdd354
      Jinho Park authored
      
      
      * add Bros boilerplate
      
      * copy and pasted modeling_bros.py from official Bros repo
      
      * update copyright of bros files
      
      * copy tokenization_bros.py from official repo and update import path
      
      * copy tokenization_bros_fast.py from official repo and update import path
      
      * copy configuration_bros.py from official repo and update import path
      
      * remove trailing period in copyright line
      
      * copy and paste bros/__init__.py from official repo
      
      * save formatting
      
      * remove unused unnecessary pe_type argument - using only crel type
      
      * resolve import issue
      
      * remove unused model classes
      
      * remove unnecessary tests
      
      * remove unused classes
      
      * fix original code's bug - layer_module's argument order
      
      * clean up modeling auto
      
      * add bbox to prepare_config_and_inputs
      
      * set temporary value to hidden_size (32 is too low because of the of the
      Bros' positional embedding)
      
      * remove decoder test, update create_and_check* input arguemnts
      
      * add missing variable to model tests
      
      * do make fixup
      
      * update bros.mdx
      
      * add boilerate plate for no_head inference test
      
      * update BROS_PRETRAINED_MODEL_ARCHIVE_LIST (add naver-clova-ocr prefix)
      
      * add prepare_bros_batch_inputs function
      
      * update modeling_common to add bbox inputs in Bros Model Test
      
      * remove unnecessary model inference
      
      * add test case
      
      * add model_doc
      
      * add test case for token_classification
      
      * apply fixup
      
      * update modeling code
      
      * update BrosForTokenClassification loss calculation logic
      
      * revert logits preprocessing logic to make sure logits have original shape
      
      * - update class name
      
      * - add BrosSpadeOutput
      - update BrosConfig arguments
      
      * add boilerate plate for no_head inference test
      
      * add prepare_bros_batch_inputs function
      
      * add test case
      
      * add test case for token_classification
      
      * update modeling code
      
      * update BrosForTokenClassification loss calculation logic
      
      * revert logits preprocessing logic to make sure logits have original shape
      
      * apply masking on the fly
      
      * add BrosSpadeForTokenLinking
      
      * update class name
      put docstring to the beginning of the file
      
      * separate the logits calculation logic and loss calculation logic
      
      * update logic for loss calculation so that logits shape doesn't change
      when return
      
      * update typo
      
      * update prepare_config_and_inputs
      
      * update dummy node initialization
      
      * update last_hidden_states getting logic to consider when return_dict is False
      
      * update box first token mask param
      
      * bugfix: remove random attention mask generation
      
      * update keys to ignore on load missing
      
      * run make style and quality
      
      * apply make style and quality of other codes
      
      * update box_first_token_mask to bool type
      
      * update index.md
      
      * apply make style and quality
      
      * apply make fix-copies
      
      * pass check_repo
      
      * update bros model doc
      
      * docstring bugfix fix
      
      * add checkpoint for doc, tokenizer for doc
      
      * Update README.md
      
      * Update docs/source/en/model_doc/bros.md
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * Update bros.md
      
      * Update src/transformers/__init__.py
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * Update docs/source/en/model_doc/bros.md
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * Apply suggestions from code review
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      
      * apply suggestions from code review
      
      * apply suggestions from code review
      
      * revert test_processor_markuplm.py
      
      * Update test_processor_markuplm.py
      
      * apply suggestions from code review
      
      * apply suggestions from code review
      
      * apply suggestions from code review
      
      * update BrosSpadeELForTokenClassification head name to entity linker
      
      * add doc string for config params
      
      * update class, var names to more explicit and apply suggestions from code review
      
      * remove unnecessary keys to ignore
      
      * update relation extractor to be initialized with config
      
      * add bros processor
      
      * apply make style and quality
      
      * update bros.md
      
      * remove bros tokenizer, add bros processor that wraps bert tokenizer
      
      * revert change
      
      * apply make fix-copies
      
      * update processor code, update itc -> initial token, stc -> subsequent token
      
      * add type hint
      
      * remove unnecessary condition branches in embedding forward
      
      * fix auto tokenizer fail
      
      * update docstring for each classes
      
      * update bbox input dimension as standard 2 points and convert them to 4
      points in forward pass
      
      * update bros docs
      
      * apply suggestions from code review : update Bros -> BROS in bros.md
      
      * 1. box prefix var -> bbox
      2. update variable names to be more explicit
      
      * replace einsum with torch matmul
      
      * apply style and quality
      
      * remove unused argument
      
      * remove unused arguments
      
      * update docstrings
      
      * apply suggestions from code review: add BrosBboxEmbeddings, replace
      einsum with classical matrix operations
      
      * revert einsum update
      
      * update bros processor
      
      * apply suggestions from code review
      
      * add conversion script for bros
      
      * Apply suggestions from code review
      
      * fix readme
      
      * apply fix-copies
      
      ---------
      Co-authored-by: default avataramyeroberts <22614925+amyeroberts@users.noreply.github.com>
      17fdd354
  29. 12 Sep, 2023 2 commits
  30. 25 Aug, 2023 1 commit
    • Arthur's avatar
      [`CodeLlama`] Add support for `CodeLlama` (#25740) · 015f8e11
      Arthur authored
      
      
      * add all
      
      * Revert "Delete .github directory"
      
      This reverts commit 9b0ff7b052e2b20b629a26fb13606b78a42944d1.
      
      * make conversion script backward compatible
      
      * fixup
      
      * more styling
      
      * copy to llama changes
      
      * fix repo consistency
      
      * nits
      
      * document correct classes
      
      * updates
      
      * more fixes
      
      * nits
      
      * update auto mappings
      
      * add readmes
      
      * smallupdates
      
      * llama-code replace with llama_code
      
      * make fixup
      
      * updates to the testsing suite
      
      * fix fast nits
      
      * more small fixes
      
      * fix decode
      
      * fix template processing
      
      * properly reset the normalizer
      
      * nits processor
      
      * tokenization tests pass
      
      * styling
      
      * last tests
      
      * additional nits
      
      * one test is left
      
      * nits
      
      Co-authored-by faabian <faabian@users.noreply.github.com>
      
      * update failing test
      
      * fixup
      
      * remove decode infilling users should handle it on their onw after generation, padding can be a problem
      
      * update
      
      * make test slow and more meaningfull
      
      * fixup
      
      * doc update
      
      * fixup
      
      * Apply suggestions from code review
      
      * add kwargs doc
      
      * tokenizer requires `requires_backend`
      
      * type requires_backends
      
      * CodeLlama instead of LlamaCode
      
      * more name cahnges
      
      * nits
      
      * make doctests happy
      
      * small pipeline nits
      
      * last nit
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * update
      
      * add codellama to toctree
      
      ---------
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      015f8e11
  31. 18 Aug, 2023 1 commit
    • Stas Bekman's avatar
      new model: IDEFICS via HuggingFaceM4 (#24796) · 6c811a32
      Stas Bekman authored
      
      
      * rename
      
      * restore
      
      * mappings
      
      * unedited tests+docs
      
      * docs
      
      * fixes
      
      * fix auto-sync breakage
      
      * cleanup
      
      * wip
      
      * wip
      
      * add fetch_images
      
      * remove einops dependency
      
      * update
      
      * fix
      
      * fix
      
      * fix
      
      * fix
      
      * fix
      
      * re-add
      
      * add batching
      
      * rework
      
      * fix
      
      * improve
      
      * add Leo as I am extending his work
      
      * cleanup
      
      * fix
      
      * cleanup
      
      * slow-test
      
      * fix
      
      * fix
      
      * fixes
      
      * deal with warning
      
      * rename modified llama classes
      
      * rework fetch_images
      
      * alternative implementation
      
      * cleanup
      
      * strict version
      
      * cleanup
      
      * [`IDEFICS`]聽Fix idefics ci (#25056)
      
      * Fix IDEFICS CI
      
      * fix test file
      
      * fixup
      
      * some changes to make tests pass
      
      * fix
      
      * fixup
      
      * Update src/transformers/models/idefics/configuration_idefics.py
      Co-authored-by: default avatarStas Bekman <stas00@users.noreply.github.com>
      
      ---------
      Co-authored-by: default avatarStas Bekman <stas00@users.noreply.github.com>
      
      * remove compat checks
      
      * style
      
      * explain that Idefics is not for training from scratch
      
      * require pt>=2.0
      
      * fix idefics vision config (#25092)
      
      * fix idefics vision config
      
      * fixup
      
      * clean
      
      * Update src/transformers/models/idefics/configuration_idefics.py
      
      ---------
      Co-authored-by: default avatarStas Bekman <stas00@users.noreply.github.com>
      
      * cleanup
      
      * style
      
      * cleanup
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * upcase
      
      * sequence of images
      
      * handle the case with no images
      
      * Update src/transformers/image_processing_utils.py
      Co-authored-by: default avatarVictor SANH <victorsanh@gmail.com>
      
      * support pure lm take 2
      
      * support tokenizer options
      
      * parameterize num_channels
      
      * fix upcase
      
      * s|IdeficsForCausalLM|IdeficsForVisionText2Text|g
      
      * manual to one line
      
      * addressing review
      
      * unbreak
      
      * remove clip dependency
      
      * fix test
      
      * consistency
      
      * PIL import
      
      * Idefics prefix
      
      * Idefics prefix
      
      * hack to make tests work
      
      * style
      
      * fix
      
      * fix
      
      * revert
      
      * try/finally
      
      * cleanup
      
      * clean up
      
      * move
      
      * [`IDEFICS`] Fix idefics config refactor (#25149)
      
      * refactor config
      
      * nuke init weights
      
      * more refactor
      
      * oops
      
      * remove visual question answering pipeline support
      
      * Update src/transformers/models/idefics/clip.py
      Co-authored-by: default avatarStas Bekman <stas00@users.noreply.github.com>
      
      * Update src/transformers/models/idefics/modeling_idefics.py
      
      * cleanup
      
      * mv clip.py vision.py
      
      * tidyup
      
      ---------
      Co-authored-by: default avatarStas Bekman <stas00@users.noreply.github.com>
      Co-authored-by: default avatarStas Bekman <stas@stason.org>
      
      * fix
      
      * license
      
      * condition on pt
      
      * fix
      
      * style
      
      * fix
      
      * rm torchvision dependency, allow custom transforms
      
      * address review
      
      * rework device arg
      
      * add_eos_token
      
      * s/transforms/transform/
      
      * fix top level imports
      
      * fix return value
      
      * cleanup
      
      * cleanup
      
      * fix
      
      * style
      
      * license
      
      * license
      
      * Update src/transformers/models/idefics/image_processing_idefics.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * add a wrapper to freeze vision layears
      
      * tidyup
      
      * use the correct std/mean settings
      
      * parameterize values from config
      
      * add tests/models/idefics/test_image_processing_idefics.py
      
      * add test_processor_idefics.py
      
      * cleanup
      
      * cleanups
      
      * fix
      
      * fix
      
      * move to the right group
      
      * style
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * add perceiver config
      
      * reset
      
      * missing arg docs
      
      * Apply suggestions from code review
      Co-authored-by: default avatarLeo Tronchon <leo.tronchon@gmail.com>
      
      * address review comments
      
      * inject automatic end of utterance tokens (#25218)
      
      * inject automatic end of utterance tokens
      
      * fix
      
      * fix
      
      * fix
      
      * rework to not use the config
      
      * not end_of_utterance_token at the end
      
      * Update src/transformers/models/idefics/processing_idefics.py
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      
      * address review
      
      * Apply suggestions from code review
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      
      * Update src/transformers/image_processing_utils.py
      Co-authored-by: default avatarNicolas Patry <patry.nicolas@protonmail.com>
      
      * [`Idefics`] add image_embeddings option in generate-related methods (#25442)
      
      * add image_embeddings option in generate-related methods
      
      * style
      
      * rename image_embeddings and allow perceiver embeddings precomputation
      
      * compute embeddings within generate
      
      * make is_encoder_decoder= True the default in config
      
      * nested if else fix
      
      * better triple check
      
      * switch if elif order for pixel values / img embeds
      
      * update model_kwargs perceiver only at the end
      
      * use _prepare_model_inputs instead of encoder_decoder logic
      
      * fix comment typo
      
      * fix config default for is_encoder_decoder
      
      * style
      
      * add typehints
      
      * precompute in forward
      
      * doc builder
      
      * style
      
      * pop instead of get image hidden states
      
      * Trigger CI
      
      * Update src/transformers/models/idefics/modeling_idefics.py
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * Update src/transformers/models/idefics/modeling_idefics.py
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * fix * + indentation + style
      
      * simplify a bit the use_resampler logic using comments
      
      * update diocstrings
      
      * Trigger CI
      
      ---------
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      
      * fix rebase changes
      
      * unbreak #25237 - to be fixed in follow up PRs
      
      * is_composition = False
      
      * no longer needed
      
      ---------
      Co-authored-by: default avatarleot13 <leo.tronchon@gmail.com>
      Co-authored-by: default avatarYounes Belkada <49240599+younesbelkada@users.noreply.github.com>
      Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
      Co-authored-by: default avatarVictor SANH <victorsanh@gmail.com>
      Co-authored-by: default avatarJoao Gante <joaofranciscocardosogante@gmail.com>
      Co-authored-by: default avatarNicolas Patry <patry.nicolas@protonmail.com>
      Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
      6c811a32
  32. 13 Aug, 2023 1 commit
  33. 09 Aug, 2023 1 commit