GitLab

[feature] release 0.1.3

Update README.md

Set the reminder to set CUDA_HOME and CUDA_PATH in the README to the "Quick Start" section under "Install CUDA".

Fix: None for load config

[fix] f16 dequantize device ignored

[fix] Fix bugs about static cache and server param;

Update README

[fix] fix broken link in tutorial.

Release v0.1.2

[update] Update readme; Add tutorial

[fix] format classes and files name

[fix] fix ktransformers interface when use new CUDAGraphRunner
[fix] fix YAML and optimize logic, the top rule has the highest priority

[feature] support q2_k & q3_k dequantize on gpu

Update task_queue.h

[Feature] towards 0.1.2

[fix] linux and windows can all find CPUInfer in current Directory

1) Linear and MLP operators support qlen>1; 2) All operators now share a single memory buffer; 3) Refactor CPUInfer submit/sync logic.

Windows Support