GitLab

* Fix AttributeError: MixtralModel object has no attribute org_vocab_size.

* Make LoRA logic for Mistral and Mixtral the same

---------
Co-authored-by: Pernekhan Utemuratov <pernekhan@deepinfra.com>

Co-authored-by: Roy <jasonailu87@gmail.com>

* add mixtral lora support

* formatting

* fix incorrectly ported logic

* polish tests

* minor fixes and refactoring

* minor fixes

* formatting

* rename and remove redundant logic

* refactoring

* refactoring

* minor fix

* minor refactoring

* fix code smell

Co-authored-by: Roy <jasonailu87@gmail.com>

This reverts commit 5c976a7e.

Co-authored-by: Chunan Zeng <chunanzeng@Chunans-Air.attlocal.net>

Co-authored-by: shujunhua1 <shujunhua1@jd.com>

Co-authored-by: Jiang Li <jiang1.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>

Co-authored-by: Pernekhan Utemuratov <pernekhan@deepinfra.com>

Signed-off-by: Tao He <sighingnow@gmail.com>

* fix: engine-useray complain

* fix: typo

Co-authored-by: chen shen <scv119@gmail.com>