Commits · 4fe9716a793cf077e07f21d0da0b9332d986dd2d · chenpangpang / transformers

"vscode:/vscode.git/clone" did not exist on "c40c7e213bdd0479bdca69df0c500004a7294d39"

12 Jun, 2023 14 commits

Skip RWKV test in past CI (#24204) · 4fe9716a

Yih-Dar authored Jun 12, 2023



* fix

* fix

---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

4fe9716a

Fix steps bugs in no trainer examples (#24197) · f7d80cb3
Ethan authored Jun 12, 2023
```
Fix step bugs in no trainer + load checkpoint + grad acc
```
f7d80cb3
Fix `_load_pretrained_model` (#24200) · 08ae37c8
Marc Sun authored Jun 12, 2023
```
Fix test
```
08ae37c8
🚨🚨🚨 Replace DataLoader logic for Accelerate in Trainer, remove unneeded tests 🚨🚨🚨 (#24028) · ebd94b0f
Zach Mueller authored Jun 12, 2023
```
* Working integration

* Fix failing test

* Revert label host logic

* Bring it back!
```
ebd94b0f

🌐

[i18n-KO] Translated tasks_summary.mdx to Korean (#23977) · dc42a9d7

Kihoon Son authored Jun 13, 2023

* 🌐

 [i18n-KO] Translated tasks_summary.mdx to Korean
Co-Authored-By: Hyeonseo Yun <0525yhs@gmail.com>
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
Co-Authored-By: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-Authored-By: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-Authored-By: Nayeon Han <nayeon2.han@gmail.com>

* Apply suggestions from code review
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>

* Update _toctree.yml

* Delete generation_strategies.mdx

* Delete tasks_explained.mdx

---------
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>

dc42a9d7

Generate: detect special architectures when loaded from PEFT (#24198) · 60b69f7d
Joao Gante authored Jun 12, 2023

60b69f7d

typo: fix typos in CONTRIBUTING.md and deepspeed.mdx (#24184) · 97527898

Jacob authored Jun 12, 2023



* typo: fix typos in CONTRIBUTING.md and deepspeed.mdx

* Update CONTRIBUTING.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

97527898

Update `GPTNeoXLanguageGenerationTest` (#24193) · dadc9fb4

Yih-Dar authored Jun 12, 2023



* fix

* fix

* fix

---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

dadc9fb4

Fix device issue in `OpenLlamaModelTest::test_model_parallelism` (#24195) · a9cdb059
Yih-Dar authored Jun 12, 2023
```
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
```
a9cdb059
Generate: force caching on the main model, in assisted generation (#24177) · 9f81f4f6
Joao Gante authored Jun 12, 2023

9f81f4f6

[i18n]Translated "attention.mdx" to korean (#23878) · 535f92ae

Kihoon Son authored Jun 12, 2023



* [i18n]Translated "attention.mdx" to korean
Co-Authored-By: Hyeonseo Yun <0525yhs@gmail.com>
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
Co-Authored-By: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-Authored-By: Nayeon Han <nayeon2.han@gmail.com>
Co-Authored-By: Jungnerd <46880056+jungnerd@users.noreply.github.com>

* Apply suggestions from code review
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>

* Update _toctree.yml

---------
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

535f92ae

Change ProgressCallback to use dynamic_ncols=True (#24101) · ba64ec07

AinL authored Jun 12, 2023

* Change ProgressCallback to use dynamic_ncols=True

* style: make style

* Revert "style: make style"

This reverts commit dee484904cd30a072d80e3be0a3d74a03cff30c6.

* run make style only trainer_callback

ba64ec07

Fix push to hub (#24187) · 93f73a38
NielsRogge authored Jun 12, 2023
```
Add fix
```
93f73a38
Fix `Wav2Vec2` CI OOM (#24190) · e26c6f03
Yih-Dar authored Jun 12, 2023
```
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
```
e26c6f03

10 Jun, 2023 2 commits
- Avoid OOM in doctest CI (#24139) · 8f093fb7
  Yih-Dar authored Jun 10, 2023
```
* fix

* fix

---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
```
  8f093fb7
- [tests] fix bitsandbytes import issue (#24151) · 0d217f42
  Stas Bekman authored Jun 09, 2023
```
fix bitsandbytes import issue
```
  0d217f42
09 Jun, 2023 16 commits
- Tool types (#24032) · deff5979
  Lysandre Debut authored Jun 09, 2023
```
* Tool types

* Tests + fixes

* Isolate types

* Oops

* Review comments + docs

* Tests + docs

* soundfile -> vision
```
  deff5979
- Fix typo in streamers.py (#24144) · 061580c8
  Freddie Vargus authored Jun 09, 2023
  
  061580c8
- [documentation] grammatical fixes in image_classification.mdx (#24141) · 12bb853c
  LiamSwayne authored Jun 09, 2023
```
Update image_classification.mdx
```
  12bb853c
- Fix Pipeline CI OOM issue (#24124) · d0d16329
  Yih-Dar authored Jun 09, 2023
```
* fix

* fix

* fix

---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
```
  d0d16329
- [BlenderBotSmall] Update doc example (#24092) · a7501f6f
  Arthur authored Jun 09, 2023
```
* small tokenizer uses `__start__` and `__end__`

* fix PR doctest
```
  a7501f6f
- [lamaTokenizerFast] Update documentation (#24132) · 5af3a1aa
  Arthur authored Jun 09, 2023
```
* Update documentation

* nits
```
  5af3a1aa
- [`SAM`] Fix sam slow test (#24140) · 62fe7533
  Younes Belkada authored Jun 09, 2023
```
* fix sam test

* update pipeline typehint
```
  62fe7533
- Fix XGLM OOM on CI (#24123) · 847b47c0
  Yih-Dar authored Jun 09, 2023
```
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
```
  847b47c0
- Fix SAM OOM issue on CI (#24125) · b8fe259f
  Yih-Dar authored Jun 09, 2023
```
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
```
  b8fe259f
- Fix TF Rag OOM issue (#24122) · 707023d1
  Yih-Dar authored Jun 09, 2023
```
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
```
  707023d1
- fix bugs with trainer (#24134) · f2b91835
  Sourab Mangrulkar authored Jun 09, 2023
```
* fix the deepspeed test failures

* apex fix

* FSDP save ckpt fix

* Update src/transformers/trainer.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
```
  f2b91835
- Generate: PT's `top_p` enforces `min_tokens_to_keep` when it is `1` (#24111) · be10092e
  Joao Gante authored Jun 09, 2023
  
  be10092e
- Correctly build models and import call_context for older TF versions (#24138) · 03585f37
  Matt authored Jun 09, 2023
  
  03585f37
- [`bnb`] Fix bnb config json serialization (#24137) · a6d05d55
  Younes Belkada authored Jun 09, 2023
```
* fix bnb config json serialization

* forward contrib credits from discussions

---------
Co-authored-by: Andrechang <Andrechang@users.noreply.github.com>
```
  a6d05d55
- PLAM => PaLM (#24129) · e2972dff
  Elliott Wang authored Jun 09, 2023
  
  e2972dff
- [Lllama] Update tokenization code to ensure parsing of the special tokens [core] (#24042) · 535542d3
  Arthur authored Jun 09, 2023
```
* preventllama fast from returning token type ids

* remove type hints

* normalised False
```
  535542d3
08 Jun, 2023 8 commits
- Avoid `GPT-2` daily CI job OOM (in TF tests) (#24106) · 2e2088f2
  Yih-Dar authored Jun 08, 2023
```
* fix

* fix

---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
```
  2e2088f2
- Fix typo in Llama docstrings (#24020) · 9322c244
  Serge Panev authored Jun 08, 2023
```
* Fix typo in Llama docstrings
Signed-off-by: Serge Panev <spanev@nvidia.com>

* Update
Signed-off-by: Serge Panev <spanev@nvidia.com>

* make style
Signed-off-by: Serge Panev <spanev@nvidia.com>

---------
Signed-off-by: Serge Panev <spanev@nvidia.com>
```
  9322c244
- add trust_remote_code option to CLI download cmd (#24097) · a73883ae
  Radamés Ajna authored Jun 08, 2023
```
* add trust_remote_code option

* require_torch
```
  a73883ae
- [`GPT2`] Add correct keys on `_keys_to_ignore_on_load_unexpected` on all child... · 8b169142
  Younes Belkada authored Jun 08, 2023
```
[`GPT2`] Add correct keys on `_keys_to_ignore_on_load_unexpected` on all child classes of `GPT2PreTrainedModel` (#24113)

* add correct keys on `_keys_to_ignore_on_load_unexpected`

* oops
```
  8b169142
- fix get_keys_to_not_convert function (#24095) · 71a114d3
  Marc Sun authored Jun 08, 2023
```
* fix get_keys_to_not_convert funct

* Fix style
```
  71a114d3
- Update the pin on Accelerate (#24110) · 8c5f3067
  Sylvain Gugger authored Jun 08, 2023
  
  8c5f3067
- [`Trainer`] Correct behavior of `_load_best_model` for PEFT models (#24103) · 2200bf7a
  Younes Belkada authored Jun 08, 2023
```
* v1

* some refactor

- add ST format as well

* fix

* add `ADAPTER_WEIGHTS_NAME` & `ADAPTER_SAFE_WEIGHTS_NAME`
```
  2200bf7a
- reset accelerate env variables after each test (#24107) · 0f236050
  Sourab Mangrulkar authored Jun 08, 2023
  
  0f236050