GitLab

Update param

Update Function call

优化提示词，解决部分Deepseek r1的兼容性

fix non stream

Fix cmake config error

Move KV cache creation to balance_serve

Signed-off-by: onepick <jiajuku12@163.com>

Enh: Make Ollama perf data more accurate, consistent with OpenAI's implementation

remove hard code max_length

update llama4 tutorial

Fix some build error for ROCM

Signed-off-by: onepick <jiajuku12@163.com>

Modify the performance data calculation module from estimation to retrieving from `raw_usage`.

Feat: Support Non-streaming chat in Ollama backend

Fix the error caused by the client not passing temperature and top_p being empty

1. Fix terrible logic in CMakeLists.txt
2. using the correct typedef for hip
Signed-off-by: onepick <jiajuku12@163.com>

Add bsz_tensors param to torch linear

Update chat.py

Update chat.py

Update chat.py

Update config.py

Feat: Add Function call support

feat(build): display limited tail of subprocesses in real time

this is a followup on #1108

chore: show cmake output in real time during build_ext

Defined new data structures in chat.py to replace OpenAI's original implementation, adding support for tool calling.

Implemented logic for extracting and processing tool calls, enabling dynamic function invocation during conversations.

Added methods in balance_serve.py to retrieve sampling parameters, handling default values and edge cases.

Updated ktransformers.py and transformers.py to support the passing of tool parameters.

Modified the default value of top_p in config.py to 1.0 to increase generation diversity.

Extended the message model in chat.py to support the transmission of tool call information.

These changes enhance the system's flexibility and functionality, enabling more complex interaction patterns.

使用长prompt时，避免rpc进程挂掉

当prompt超过cache_len的时候，rpc进程会crash掉，导致整体不可用。
这里增加一个检查，让过长的prompt在请求早期就被提前过滤掉