GitLab

* fix image-to-text batch incorrect output issue
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* add ci test
Signed-off-by: Wang, Yi <yi.a.wang@intel.com>

* update ci test
Signed-off-by: Wang, Yi <yi.a.wang@intel.com>

---------
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Signed-off-by: Wang, Yi <yi.a.wang@intel.com>

* add sacremoses check

* fix style

* for FlaubertTokenizer

* HerbertTokenizer fix

* add typeHint

* Update src/transformers/testing_utils.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* make less skipped

* make quality

* remove import

---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* left-padding test revisited

* Apply suggestions from code review
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

Potential typo in mlx support

* 🐛 Fix vision encoder decoder positional arg

* ✅

 Add test for VisionEncoderDecoder with LayoutLMv3 encoder

---------
Co-authored-by: Nick DeGroot <1966472+nickthegroot@users.noreply.github.com>

* Set `inputs` as kwarg in `TextClassificationPipeline`

This change has been done to align the `TextClassificationPipeline` with the rest of the pipelines, and to be able to e.g. `pipeline(**{"inputs": "text"})` which wouldn't be possible since the `*args` were being used instead.

* Add `noqa: C409` on `tuple([inputs],)`

Even though is discouraged by the linter, the cast `tuple(list(...),)` is required here, as otherwise the original list in `inputs` will be transformed into a `tuple` and the elements 1...N will be ignored by the `Pipeline`

* Run `ruff format`

* Simplify `tuple` conversion with `(inputs,)`
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

---------
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* Fall back to pytorch model for now

* Fix up

Add support for loading safetensors files saved with metadata format mlx.

* flava multimodal add attn mask

* make style

* check mask is not None

Signed-off-by: Ashok Pon Kumar Sree Prakash <ashokponkumar@gmail.com>

Revert "Automatic safetensors conversion when lacking these files (#29390)"

This reverts commit a69cbf4e.

* Enable BLIP for auto VQA

* Make style

* Add VQA to BLIP pipeline tests

* Fix: Disable torch.autocast in RotaryEmbedding of Gemma and LLaMa for MPS devices

* Update src/transformers/models/gemma/modeling_gemma.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update llama ang gemma rope use cpu in mps device

---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

Substantially reduce memory usage in _update_causal_mask for large batches by using .expand instead of .repeat [needs tests+sanity check] (#29413)

* try to fix gemma mem use

* fix: handle attention mask dim==2 case

* remove logits=logits.float()

* clean up + add llama

* apply formatting

* readability edit: swap order of items being multiplied

* revert change unrelated to PR

* revert black autoformat

* switch to one .to

* Accept style edits
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* added the max_matching_ngram_size parameter into the GenerationConfig, for the PromptLookupCandidateGenerator

* switched back to keyword arguments

* added PromptLookupCandidateGenerator docstring for its parameters

* ruff reformat

* Update src/transformers/generation/configuration_utils.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Fix test failure

* use item

* Fix TrainingArguments regression with torch <2.0.0 for dataloader_prefetch_factor

dataloader_prefetch_factor was added to TrainingArguments in #28498 with the default value None, but  versions of torch<2.0.0 do not accept None and will raise an error if num_workers == 0 and prefetch_factor != 2

* Add is_torch_available() check

* Use is_torch_greater_or_equal_than_2_0

add back check for dataloader_prefetch_factor

* add accelerate docs

* Apply suggestions from code review
Co-authored-by: Loubna Ben Allal <44069155+loubnabnl@users.noreply.github.com>

* Update starcoder2.md

* add correct generation

---------
Co-authored-by: Loubna Ben Allal <44069155+loubnabnl@users.noreply.github.com>

* add docs on exllamav2 + AWQ

* Update docs/source/en/quantization.md

* use require_torch_gpu

* enable on XPU

* fix

* Update ko _toctree.yml

* Create ko: generation_strategies.md

* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Apply suggestions from code review
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

* Apply suggestions from code review
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

* [i18n-zh] Translate add_new_pipeline.md into Chinese

* apply suggestions from Fan-Lin

* Automatic safetensors conversion when lacking these files

* Remove debug

* Thread name

* Typo

* Ensure that raises do not affect the main thread

* Update to pull function from proper lib

* Fix ruff formatting error

* Remove accidently added file

* Fix bug with passing capture_* args to neptune callback

* ruff happy?

* instantiate (frozen)set only once

* code review

* code review 2

* ruff happy?

* code review

* initial-commit

* start cleaning

* small nits

* small nits

* current updates

* add kernels

* small refactoring little step

* add comments

* styling

* nit

* nits

* Style

* Small changes

* Push dummy mambda simple slow

* nit

* Use original names

* Use original names and remove norm

* Updates for inference params

* Style nd updates

* nits

* Match logits

* Add a test

* Add expected generated text

* nits doc, imports and styling

* style

* oups

* dont install kernels, invite users to install the required kernels

* let use use the original packages

* styling

* nits

* fix some copieds

* update doc

* fix-copies

* styling done

* nits

* fix import check

* run but wrong cuda ress

* mamba CUDA works :)

* fix the fast path

* config naming nits

* conversion script is not required at this stage

* finish fixing the fast path: generation make sense now!

* nit

* Let's start working on the CIs

* style

* better style

* more nits

* test nit

* quick fix for now

* nits

* nit

* nit

* nit

* nits

* update test rest

* fixup

* update test

* nit

* some fixes

* nits

* update test values

* fix styling

* nit

* support peft

* integrations tests require torchg

* also add slow markers

* styling

* chose forward wisely

* nits

* update tests

* fix gradient checkpointing

* fixup

* nit

* fix doc

* check copies

* fix the docstring

* fix some more tests

* style

* fix beam search

* add init schene

* update

* nit

* fix

* fixup the doc

* fix the doc

* fixup

* tentative update but slow is no longer good

* nit

* should we always use float32?

* nits

* revert wrong changes

* res in float32

* cleanup

* skip fmt for now

* update generation values

* update test values running original model

* fixup

* update tests + rename inference_params to cache_params + make sure training does not use cache_params

* small nits

* more nits

* fix final CIs

* style

* nit doc

* I hope final doc nits

* nit

* 🫠

* final touch!

* fix torch import

* Apply suggestions from code review
Co-authored-by: Lysandre Debut <hi@lysand.re>

* Apply suggestions from code review

* fix fix and fix

* fix base model prefix!

* nit

* Update src/transformers/models/mamba/__init__.py

* Update docs/source/en/model_doc/mamba.md
Co-authored-by: Lysandre Debut <hi@lysand.re>

* nit

---------
Co-authored-by: Lysandre Debut <hi@lysand.re>

* fix udop imports

* sort imports

* style

* revert with RP

* nit

* exact revert

* update

* ...

* nits

* arf

* 🧼

* beat the last guy

* style everyone

* use torch_device

* Update tests/pipelines/test_pipelines_text_generation.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* fix style

---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

Update starcoder2 paper link