Merge pull request #1210 from kvcache-ai/support-amx-qwen

Support amx and qwen3

Merge pull request #1210 from kvcache-ai/support-amx-qwen
Support amx and qwen3
ffb1f7bf · wang jiahao · GitHub · ba92cf1a · 8f76c37d · ffb1f7bf
Unverified Commit ffb1f7bf authored Apr 29, 2025 by wang jiahao Committed by GitHub Apr 29, 2025
14 changed files
--- a/ktransformers/optimize/optimize_rules/Qwen3Moe-serve.yaml
+++ b/ktransformers/optimize/optimize_rules/Qwen3Moe-serve.yaml
--- a/ktransformers/server/args.py
+++ b/ktransformers/server/args.py
--- a/ktransformers/server/backend/interfaces/balance_serve.py
+++ b/ktransformers/server/backend/interfaces/balance_serve.py
--- a/ktransformers/server/balance_serve/inference/forward_batch.py
+++ b/ktransformers/server/balance_serve/inference/forward_batch.py
--- a/ktransformers/server/balance_serve/inference/model_runner.py
+++ b/ktransformers/server/balance_serve/inference/model_runner.py
--- a/ktransformers/server/balance_serve/sched_rpc.py
+++ b/ktransformers/server/balance_serve/sched_rpc.py
--- a/ktransformers/server/balance_serve/settings.py
+++ b/ktransformers/server/balance_serve/settings.py
--- a/ktransformers/server/config/config.py
+++ b/ktransformers/server/config/config.py
--- a/ktransformers/server/requirements.txt
+++ b/ktransformers/server/requirements.txt
--- a/ktransformers/tests/test_speed.py
+++ b/ktransformers/tests/test_speed.py
--- a/ktransformers/util/custom_gguf.py
+++ b/ktransformers/util/custom_gguf.py
--- a/pyproject.toml
+++ b/pyproject.toml
--- a/requirements-local_chat.txt
+++ b/requirements-local_chat.txt
--- a/custom_flashinfer @ af4259e8
+++ b/custom_flashinfer @ af4259e8
-Subproject commit fd94393fb5b8ba8bae9c0bd6ab1c2a429d81ac76
+Subproject commit af4259e8a33f095b419d1fd1733a50b22fc84c49