tests/deepspeed/test_deepspeed.py · 976189a6df796a2ff442dd81b022626c840d8c27 · chenpangpang / transformers

Fix initialization for missing parameters in `from_pretrained` under ZeRO-3 (#28245) · 976189a6

Xuehai Pan authored Jan 09, 2024

* Fix initialization for missing parameters in `from_pretrained` under ZeRO-3

* Test initialization for missing parameters under ZeRO-3

* Add more tests

* Only enable deepspeed context for per-module level parameters

* Enable deepspeed context only once

* Move class definition inside test case body

976189a6

test_deepspeed.py 51.3 KB

Replace test_deepspeed.py