"docs/source/vscode:/vscode.git/clone" did not exist on "df40edfb00715880f7432899b7a854aad7ae39d9"
- 07 Apr, 2023 9 commits
-
-
Joao Gante authored
add API warning
-
Arthur authored
* Fix default attention mask size * fixup * add a test to make sure that even if attention mask are not provided, works * style
-
Arthur authored
* do not push special file * Update src/transformers/tokenization_utils_base.py Co-authored-by:
Sylvain Gugger <35901082+sgugger@users.noreply.github.com> --------- Co-authored-by:
-
Arthur authored
* Small nit, Fixes #21986 * Update src/transformers/pipelines/__init__.py
-
Wonhyeong Seo authored
docs: feat: Korean pipeline_tutorial Co-authored-by:
Jungnerd <46880056+jungnerd@users.noreply.github.com> Co-authored-by:
Hyeonseo Yun <0525_hhgus@naver.com> Co-authored-by:
gabrielwithappy <102908949+gabrielwithappy@users.noreply.github.com> Co-authored-by:
Na Yeon Han <nayeon2.han@gmail.com>
-
Yih-Dar authored
* fix --------- Co-authored-by:ydshieh <ydshieh@users.noreply.github.com>
-
Seung-Moo Yang authored
-
Shikhar Chauhan authored
* (feat): Move labels to the same device as logits * Trigger CI * Trigger CI * Trigger CI * (feat): Making changes for Blip2
-
gabrielwithappy authored
translate the autoclass_tutorial and fix the typo of the quicktour
-
- 06 Apr, 2023 13 commits
-
-
Sourab Mangrulkar authored
fix fsdp
-
Yih-Dar authored
* Update tiny model summary file for recent models --------- Co-authored-by: -
Younes Belkada authored
fix slow tests and doctests
-
Nicolas Patry authored
-
Younes Belkada authored
add safety checker
-
Yih-Dar authored
* Auto. add and update pipeline_model_mapping * Fix style and quality * Finalize (comments) * fix --------- Co-authored-by: -
Yih-Dar authored
* Add TFBlipForConditionalGeneration * update pipeline_model_mapping * Add import * Revert changes in GPTSanJapaneseTest --------- Co-authored-by: -
Connor Henderson authored
fix broken link
-
Yih-Dar authored
* Final Tiny things --------- Co-authored-by: -
amyeroberts authored
* Add out_indices to backbones, deprecate out_features * Update - can specify both out_features and out_indices but not both * Add backbone mixin tests * Test tidy up * Add test_backbone for convnext * Remove redefinition of method * Update for Dinat and Nat backbones * Update tests * Smarter indexing * Add checks on config creation for backbone * PR comments
-
Joao Gante authored
-
Nicolas Patry authored
-
Nicolas Patry authored
* Adding Llama FastTokenizer support. - Requires https://github.com/huggingface/tokenizers/pull/1183 version - Only support byte_fallback for llama, raise otherwise (safety net). - Lots of questions are special tokens How to test: ```python from transformers.convert_slow_tokenizer import convert_slow_tokenizer from transformers import AutoTokenizer from tokenizers import Tokenizer tokenizer = AutoTokenizer.from_pretrained("huggingface/llama-7b") if False: new_tokenizer = Tokenizer.from_file("tok.json") else: new_tokenizer = convert_slow_tokenizer(tokenizer) new_tokenizer.save("tok.json") strings = [ "This is a test", "生活的真谛是", "生活的真谛是[MASK]。", # XXX: This one is problematic because of special tokens # "<s> Something something", ] for string in strings: encoded = tokenizer(string)["input_ids"] encoded2 = new_tokenizer.encode(string).ids assert encoded == encoded2, f"{encoded} != {encoded2}" decoded = tokenizer.decode(encoded) decoded2 = new_tokenizer.decode(encoded2) assert decoded.strip() == decoded2, f"{repr(decoded)} != {repr(decoded2)}" ``` The converter + some test script. The test script. Tmp save. Adding Fast tokenizer + tests. Adding the tokenization tests. Correct combination. Small fix. Fixing tests. Fixing with latest update. Rebased. fix copies + normalized added tokens + copies. Adding doc. TMP. Doc + split files. Doc. Versions + try import. Fix Camembert + warnings -> Error. Fix by ArthurZucker. Not a decorator. * Fixing comments. * Adding more to docstring. * Doc rewriting.
-
- 05 Apr, 2023 13 commits
-
-
Kaustubh authored
feat(model parallelism): moving the labels to the same device as the logits for gpt2 and bart (#22591)
-
Matt authored
* Use native TF checkpoints for the TF tests * Remove unneeded exceptions
-
Younes Belkada authored
* add deplot + matcha on `transformers` * more docs * correct path * Update docs/source/en/model_doc/deplot.mdx Co-authored-by:
NielsRogge <48327001+NielsRogge@users.noreply.github.com> * add correct names --------- Co-authored-by:
-
Nicolas Patry authored
* Adding support for BPE merge creation from scores instead of ids. * Revert warn -> raise. * Update src/transformers/convert_slow_tokenizer.py * Quality.
-
Matt authored
Fixes a typo in one of the BLIP pretrained checkpoint names
-
Mikel Penagarikano authored
* Update run_speech_recognition_ctc.py Make sure all processes wait until data is saved before loading the processor from the output_dit * Make sure all processes wait until data is saved before loading the processor from the output_dit * Update run_speech_recognition_ctc.py * Update run_speech_recognition_seq2seq.py
-
Wonhyeong Seo authored
Co-authored-by: -
Quentin Meeus authored
The logger prints a summary at the beginning of training that displays some info such as number of examples, number of parameters, total number of steps, etc. Those numbers can be quite large and difficult to read. I added a thousand separator to improve readability for the following: - num_examples - num_train_epochs - per_device_train_batch_size - total_train_batch_size - max_steps - num_trainable_params
-
Matt authored
* Re-enable skipped test and fix the hidden state shape issue * Actually fix the bug instead of just doing something wrong
-
Joao Gante authored
-
Sylvain Gugger authored
-
Joao Gante authored
-
Sylvain Gugger authored
-
- 04 Apr, 2023 5 commits
-
-
Matt authored
* Fix inverted conditional in TF common test! * Make the same change in the PT tests file * Make sure hidden states for GPT2 have the same output shape in PT/TF * Minor fix to PT implementation of token classification loss * Skip loss equivalence test for TFHubert because it keeps overflowing to inf * Compute LM loss for TF the (weird) way it's computed in PT * Skip loss equivalence test for Wav2Vec2 for the same reason as Hubert * Fix - don't try to access the hidden states property when output is a tuple
-
Sourab Mangrulkar authored
-
Shubhamai authored
* initial commit * review changes * post model PR merge * updating doc
-
Sun Haozhe authored
* corrected/clarified the code comment of find_pruneable_heads_and_indices * have run make style
-
Matt authored
* Initial commit * more stash commit * Yet another stash commit * yet more stash commit * Mostly working except for docs / repo consistency * Stop importing model list from torch file * Add TF BLIP models to docs * Add auto classes * Move get_text_features and get_image_features * Update src/transformers/models/blip/modeling_tf_blip.py Co-authored-by:
amyeroberts <22614925+amyeroberts@users.noreply.github.com> * Update src/transformers/models/blip/modeling_tf_blip.py Co-authored-by:
Joao Gante <joaofranciscocardosogante@gmail.com> * Update tests/models/blip/test_modeling_tf_blip_text.py Co-authored-by:
-
