GitLab

修复config.architectures[0] == "DeepseekV2ForCausalLM" or "DeepseekV3ForCausalLM" 永远为真

The numa node location would be calculated based on the total number
of worker threads.
So we should always use the actual number of threads instead of using a min() op.

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>

This change explicitly clears CUDA cache during weight loading to mitigate memory fragmentation issues, particularly beneficial for low-VRAM GPUs.

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>

Add fp8 linear kernel;\n Add empty cache to fit in 16G VRAM; By 'wkGCaSS - 知乎 https://zhuanlan.zhihu.com/p/25491611225'