- [ ] I have searched related issues but cannot get the expected help.
- [ ] The bug has not been fixed in the latest version.
- [ ] Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
- [ ] If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
- [ ] To help the community, I will use English or attach an English translation if using another language. Non-English content without translation may be closed.
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Run '...' command.
2. If you modified optimization rules, uploads the file or clarify what have you changed.
3. How did you install KTransformers? (e.g. from source, from whl)
4. KTransformers version (e.g. 0.2.2 or specific commit number)
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Environments(please complete the following information):**
- [ ] If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
- [ ] To help the community, I will use English or attach an English translation if using another language. Non-English content without translation may be closed.
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
