- 05 Jun, 2024 7 commits
-
-
Huazhong Ji authored
-
Vaibhav Srivastav authored
* doc: add info about wav2vec2 bert in older wav2vec2 models. * apply suggestions from review. * forward contrib credits from review --------- Co-authored-by:Sanchit Gandhi <sanchit-gandhi@users.noreply.github.com>
-
dependabot[bot] authored
Bump transformers in /examples/research_projects/deebert Bumps [transformers](https://github.com/huggingface/transformers) from 3.5.1 to 4.38.0. - [Release notes](https://github.com/huggingface/transformers/releases) - [Commits](https://github.com/huggingface/transformers/compare/v3.5.1...v4.38.0 ) --- updated-dependencies: - dependency-name: transformers dependency-type: direct:production ... Signed-off-by:
dependabot[bot] <support@github.com> Co-authored-by:
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-
amyeroberts authored
* Move label validation checks - fail early * Remove some formatting changes - add back labels change wav2vec2
-
Yih-Dar authored
* benchmark workflow * benchmark workflow * benchmark workflow * benchmark workflow * build * build * build * build * build * build * build * build * build * build * build * build * build * build --------- Co-authored-by:ydshieh <ydshieh@users.noreply.github.com>
-
James Braza authored
Fixed torch definition error
-
Yury Sulsky authored
The StoppingCriteriaList allocates is_done without specifying dtype=torch.bool. On XLA this allocates a float tensor and causes a failure on the following line: is_done = is_done | criteria(input_ids, scores, **kwargs) by attempting to OR float with bool.
-
- 04 Jun, 2024 14 commits
-
-
dependabot[bot] authored
Bump transformers in /examples/research_projects/vqgan-clip Bumps [transformers](https://github.com/huggingface/transformers) from 4.26.0 to 4.38.0. - [Release notes](https://github.com/huggingface/transformers/releases) - [Commits](https://github.com/huggingface/transformers/compare/v4.26.0...v4.38.0 ) --- updated-dependencies: - dependency-name: transformers dependency-type: direct:production ... Signed-off-by:
-
Yih-Dar authored
* build * build * build * build * fix * fix * fix --------- Co-authored-by: -
amyeroberts authored
* Move out common validation * Add missing backbone config arguments
-
Younes Belkada authored
* deprecate blip * mention deprecation on docs
-
Yih-Dar authored
* fix * fix * fix * fix --------- Co-authored-by: -
Manuel Faysse authored
-
Jacklanda authored
✨ Add new line switch before logging "***** Running {description} *****". Signed-off-by:jacklanda <yonyonlau@gmail.com>
-
amyeroberts authored
* Fix pipeline tests - torch imports * Frameowrk dependant float conversion
-
Chujie Zheng authored
* fix logits dtype * Add bf16/fp16 tests for text_classification pipeline * Update test_pipelines_text_classification.py * fix * fix
-
Kristen Pereira authored
* Added interpolate pos encoding feature and test to deit * Added interpolate pos encoding feature and test for deit TF model * readded accidentally delted test for multi_gpu * storing only patch_size instead of entire config and removed commented code * Update modeling_tf_deit.py to remove extra line Co-authored-by:
amyeroberts <22614925+amyeroberts@users.noreply.github.com> --------- Co-authored-by:
-
Raushan Turganbay authored
video-llava can handle more frames
-
Max Strobel authored
* fix(PatchTST): Wrong dropout used for PretainHead * feat(PatchTST): remove unused config.dropout --------- Co-authored-by:Strobel Maximilian (IFAG PSS SIS SCE ACM) <Maximilian.Strobel@infineon.com>
-
DomHudson authored
-
Raushan Turganbay authored
* add device in logits processor * remove device when not needed * codestyle * tests * forgot `melody` version * Update src/transformers/models/whisper/generation_whisper.py Co-authored-by:
Joao Gante <joaofranciscocardosogante@gmail.com> * codestyle * updates --------- Co-authored-by:
-
- 03 Jun, 2024 18 commits
-
-
Aaron Jimenez authored
* add tokenizer_summary to es/_toctree.yml * add tokenizer_summary to es/ * fix link to Transformes XL in en/ * translate until Subword tokenization section * fix GPT link in en/ * fix other GPT link in en/ * fix typo in en/ * translate the doc * run make fixup * Remove .md in Transformer XL link * fix some link issues in es/ * fix typo
-
Yih-Dar authored
* build * build * build * build --------- Co-authored-by: -
miivanov90 authored
* update to not(endswith(loss)) * ruff formatting
-
Younes Belkada authored
Update modeling_cohere.py
-
Jade Choghari authored
Update index.md - Contents = Contenu French typo - Contents = Contenu
-
Qubitium authored
* Rename sanity_evaluation to eval_on_start * move arg back to last
-
Bojun Feng authored
fix typo
-
Sangbum Daniel Choi authored
* fix the get_size_with_aspect_ratio in max_size situation * make fix-up * add more general solution * consider when max_size is not defined * fix typo * fix typo * simple fix * fix error * fix if else error * fix error of size overwrite * fix yolos image processing * fix detr image processing * make * add longest related test script * Update src/transformers/models/yolos/image_processing_yolos.py Co-authored-by:
-
Isotr0py authored
* add qwen2 gguf support * Update docs * fix qwen2 tokenizer * add qwen2 gguf test * fix typo in qwen2 gguf test * format code * Remove mistral, clarify the error message * format code * add typing and update docstring
-
Yih-Dar authored
* fix * fix * fix * fix --------- Co-authored-by: -
NielsRogge authored
Update MLP
-
Joao Gante authored
* tmp commit * sliding window with fewer differences * make fixup + rebase * missing overwrite
-
fxmarty authored
* update non-causal mask for sdpa * add test * update docstrings * add one more test * fix cross attention bug * gentler atol/rtol
-
Pavithra Devi M authored
While running the model.prepare_tf_dataset() method, it raises the error below: ``` TypeError: Cannot convert [array([322., 1.])] to EagerTensor of dtype int64 ``` This happens, in "DataCollatorForSeq2Seq" function when we are try to convert the labels to tensors. While converting the labels to tensors, the labels can be in the format of list of list or list of ndarrays. There is no problem converting the list of list lables. There is a problem when the list of ndarrays are float values(like below). ``` [array([322., 1.])] ``` so the exception raises while trying to convert this label to tensors using below code. ``` batch["labels"] = tf.constant(batch["labels"], dtype=tf.int64) ``` The labels are always integer values, so this got converted to float values in the label padding operation below. ``` batch["labels"] = [ call(label) if padding_side == "right" else np.concatenate([[self.label_pad_token_id] * (max_label_length - len(label)), label]) for label in labels ] ``` Here we have 2 cases: 1 - Concatenating an array having integer padding token value with labels. 2 - Concatenating an empty array with labels. ---------------------------------------------------------------------------------------- case 1: Concatenating an array having integer padding token value with labels. WORKS EXPECTED: ---------------------------------------------------------------------------------------- ``` label = np.array([233, 1]) max_label_length = 4 label_pad_token_id = -100 np.concatenate([[label_pad_token_id] * (max_label_length - len(label)), label]) o/p: array([-100, -100, 233, 1]) ``` ---------------------------------------------------------------------------------------- Case 2: Concatenating an empty array with labels. GIVES THE ISSUE: This scenorio can happen when the label has the maximum label length -- No padding needed. ---------------------------------------------------------------------------------------- ``` label = np.array([233, 1]) max_label_length = 2 label_pad_token_id = -100 np.concatenate([[label_pad_token_id] * (max_label_length - len(label)), label]) o/p: array([233., 1.]) ``` ---------------------------------------------------------------------------------------- Solution: ---------------------------------------------------------------------------------------- We need to concatenate a ndarray of dtype int with labels. AFTER FIX: ---------- case 1: ``` label = np.array([233, 1]) max_label_length = 4 label_pad_token_id = -100 np.concatenate([np.array([label_pad_token_id] * (max_label_length - len(label)), dtype=np.int64),label]) o/p: array([-100, -100, 233, 1]) ``` case 2: ``` label = np.array([233, 1]) max_label_length = 2 label_pad_token_id = -100 np.concatenate([np.array([label_pad_token_id] * (max_label_length - len(label)), dtype=np.int64),label]) o/p: array([233, 1]) ``` -
Arthur authored
* fixes * fix-copies
-
Ahmed Moubtahij authored
* token healing impl + trie with extensions * make fixup * prefix-robust space tokenization * examples readme and requirements * make fixup * allow input prompt and model * redundant defaults * Specialized Trie * make fixup * updated tests with new inherited Tree * input ids to auto device_map * rm unused import * Update src/transformers/generation/utils.py Co-authored-by:
Arthur <48595927+ArthurZucker@users.noreply.github.com> * naming convention * Revert "naming convention" This reverts commit dd39d9c5b7a969e2d8a8d2a8e54f121b82dc44f0. * naming convention * last -hopefully- changes --------- Co-authored-by:
-
amyeroberts authored
* Remove copied froms for deprecated models * Remove automatically in script
-
CharlesCNorton authored
Corrected a typo in security.md. Changed `use_safetenstors` to `use_safetensors` in the section discussing the usage of safe formats for loading models to prevent arbitrary code execution.
-
- 31 May, 2024 1 commit
-
-
Arthur authored
* current working example! * commit regex and result file * update * nit * push the conversion file * oups * roadmap and nits * attempt diffs for 3 files * persimmon * nit * add diff file that is the same as the modeling_llama.py * fix rope nits * updates * updates with converted versions * give some breathing space to the code * delete * update * update * push the actual result * update regex patterns * update regex patterns * fix some issues * fix some issues * fix some issues * updates * updates * updates * updates * updates * revert changes done to llama * updates * update gemma * updates * oups * current state * current state * update * ouiiii * nit * clear diffs * nit * fixup * update * doc
🚀 *🔥 * for now use gemma * deal with comments * style * handle funtions * deal with assigns * todos * process inheritage * keep decorators? *🤗 * deal with duplicates * fixup * correctly remove duplicate code * run ruff post script * ruff deals pretty well with imports, let's leave it to him * ah maybe not lol * for now remove all imports from child. * nit * conversion of llama * okay * convert starcoder2 * synch with main * update llama diff * updates * https://docs.astral.sh/ruff/rules/redefined-while-unused/ fixes the imports, bit needs later version of ruff * updates * okay actual state * non zero exit * update! * revert unrelated * remove other diff files * updates * cleanup * update * less diff! * stash * current updates * updates * No need for call * finished fining deps * update * current changes * current state * current state * new status * nit * finally * fixes * nits * order is now expected * use logger info instead of prints * fixup * up * nit * update * nits * update * correct merge * update * update * update * add warning * update caution message * update * better merging strategy * copy class statements :wink * fixups * nits * update * Apply suggestions from code review Co-authored-by:
-
![dependabot[bot]'s avatar](https://www.gravatar.com/avatar/48ea49be76d0c68403a7f3df87e3487d?s=80&d=identicon "dependabot[bot]")
