Merge pull request #1013 from kvcache-ai/work-concurrent

In v0.2.4 version, we’ve added highly desired multi-concurrency support to the community through a major refactor of the whole architecture.

Merge pull request #1013 from kvcache-ai/work-concurrent
In v0.2.4 version, we’ve added highly desired multi-concurrency support to the community through a major refactor of the whole architecture.
a41d2163 · wang jiahao · GitHub · f142f4df · 4ed9744e · a41d2163
Unverified Commit a41d2163 authored Apr 02, 2025 by wang jiahao Committed by GitHub Apr 02, 2025
13 changed files
--- a/ktransformers/tests/mmlu_test_multi.py
+++ b/ktransformers/tests/mmlu_test_multi.py
--- a/ktransformers/tests/test_client.py
+++ b/ktransformers/tests/test_client.py
--- a/ktransformers/tests/test_speed.py
+++ b/ktransformers/tests/test_speed.py
--- a/ktransformers/util/utils.py
+++ b/ktransformers/util/utils.py
--- a/merge_tensors/merge_safetensor_gguf.py
+++ b/merge_tensors/merge_safetensor_gguf.py
--- a/requirements-local_chat.txt
+++ b/requirements-local_chat.txt
--- a/setup.py
+++ b/setup.py
--- a/custom_flashinfer @ fd94393f
+++ b/custom_flashinfer @ fd94393f
--- a/third_party/nlohmann/json.hpp
+++ b/third_party/nlohmann/json.hpp
--- a/third_party/nlohmann/json_fwd.hpp
+++ b/third_party/nlohmann/json_fwd.hpp
--- a/prometheus-cpp @ f13cdd05
+++ b/prometheus-cpp @ f13cdd05
--- a/spdlog @ 48bcf39a
+++ b/spdlog @ 48bcf39a
--- a/xxHash @ 953a09ab
+++ b/xxHash @ 953a09ab