Merge pull request #1013 from kvcache-ai/work-concurrent

In v0.2.4 version, we’ve added highly desired multi-concurrency support to the community through a major refactor of the whole architecture.

Merge pull request #1013 from kvcache-ai/work-concurrent
In v0.2.4 version, we’ve added highly desired multi-concurrency support to the community through a major refactor of the whole architecture.
a41d2163 · wang jiahao · GitHub · f142f4df · 4ed9744e · a41d2163
Unverified Commit a41d2163 authored Apr 02, 2025 by wang jiahao Committed by GitHub Apr 02, 2025
20 changed files
--- a/ktransformers/ktransformers_ext/bench/bench_linear.py
+++ b/ktransformers/ktransformers_ext/bench/bench_linear.py
--- a/ktransformers/ktransformers_ext/bench/bench_linear_torch.py
+++ b/ktransformers/ktransformers_ext/bench/bench_linear_torch.py
--- a/ktransformers/ktransformers_ext/bench/bench_mlp.py
+++ b/ktransformers/ktransformers_ext/bench/bench_mlp.py
--- a/ktransformers/ktransformers_ext/bench/bench_mlp_torch.py
+++ b/ktransformers/ktransformers_ext/bench/bench_mlp_torch.py
--- a/ktransformers/ktransformers_ext/bench/bench_moe.py
+++ b/ktransformers/ktransformers_ext/bench/bench_moe.py
--- a/ktransformers/ktransformers_ext/bench/bench_moe_torch.py
+++ b/ktransformers/ktransformers_ext/bench/bench_moe_torch.py
--- a/ktransformers/ktransformers_ext/cmake/FindSIMD.cmake
+++ b/ktransformers/ktransformers_ext/cmake/FindSIMD.cmake
--- a/ktransformers/ktransformers_ext/cpu_backend/backend.cpp
+++ b/ktransformers/ktransformers_ext/cpu_backend/backend.cpp
--- a/ktransformers/ktransformers_ext/cpu_backend/backend.h
+++ b/ktransformers/ktransformers_ext/cpu_backend/backend.h
--- a/ktransformers/ktransformers_ext/cpu_backend/cpuinfer.h
+++ b/ktransformers/ktransformers_ext/cpu_backend/cpuinfer.h
--- a/ktransformers/ktransformers_ext/cpu_backend/task_queue.cpp
+++ b/ktransformers/ktransformers_ext/cpu_backend/task_queue.cpp
--- a/ktransformers/ktransformers_ext/cpu_backend/task_queue.h
+++ b/ktransformers/ktransformers_ext/cpu_backend/task_queue.h
--- a/ktransformers/ktransformers_ext/cpu_backend/vendors/README.md
+++ b/ktransformers/ktransformers_ext/cpu_backend/vendors/README.md
--- a/ktransformers/ktransformers_ext/cpu_backend/vendors/cuda.h
+++ b/ktransformers/ktransformers_ext/cpu_backend/vendors/cuda.h
--- a/ktransformers/ktransformers_ext/cpu_backend/vendors/hip.h
+++ b/ktransformers/ktransformers_ext/cpu_backend/vendors/hip.h
--- a/ktransformers/ktransformers_ext/cpu_backend/vendors/musa.h
+++ b/ktransformers/ktransformers_ext/cpu_backend/vendors/musa.h
--- a/ktransformers/ktransformers_ext/cpu_backend/vendors/vendor.h
+++ b/ktransformers/ktransformers_ext/cpu_backend/vendors/vendor.h
--- a/ktransformers/ktransformers_ext/cuda/binding.cpp
+++ b/ktransformers/ktransformers_ext/cuda/binding.cpp
--- a/ktransformers/ktransformers_ext/cuda/custom_gguf/dequant.cu
+++ b/ktransformers/ktransformers_ext/cuda/custom_gguf/dequant.cu
--- a/ktransformers/ktransformers_ext/cuda/custom_gguf/ops.h
+++ b/ktransformers/ktransformers_ext/cuda/custom_gguf/ops.h