tests/models/mixtral/test_modeling_mixtral.py · e768616afa1a1e20bd9678c8099891d8529e89da · chenpangpang / transformers

Fix load balancing loss func for mixtral (#28256) · e768616a

liangxuZhang authored Jan 11, 2024



* Correct the implementation of auxiliary loss of mixtrtal

* correct the implementation of auxiliary loss of mixtrtal

* Implement a simpler calculation method

---------
Co-authored-by: zhangliangxu3 <zhangliangxu3@jd.com>

e768616a

test_modeling_mixtral.py 21.5 KB

Replace test_modeling_mixtral.py