1、同步到最新版本；2、增加batch推理接口；3、解决内存泄漏问题；4、修复llama系列流式输出不流畅的问题

56215723 · zhouxiang · 44be91d3 · 56215723 · 56215723 · 56215723
Commit 56215723 authored Jan 31, 2024 by zhouxiang
Showing with 130 additions and 8 deletions

tools/scripts/qwen2flm.py tools/scripts/qwen2flm.py +4 -3

tools/scripts/setup.py tools/scripts/setup.py +2 -2

tools/src/pytools.cpp tools/src/pytools.cpp +124 -3

No files found.
--- a/tools/scripts/qwen2flm.py
+++ b/tools/scripts/qwen2flm.py
--- a/tools/scripts/setup.py
+++ b/tools/scripts/setup.py
--- a/tools/src/pytools.cpp
+++ b/tools/src/pytools.cpp