GitLab

[fix] f16 dequantize device ignored

[fix] Fix bugs about static cache and server param;

Update README

[fix] fix broken link in tutorial.

Release v0.1.2

[update] Update readme; Add tutorial

[fix] format classes and files name

[fix] fix ktransformers interface when use new CUDAGraphRunner
[fix] fix YAML and optimize logic, the top rule has the highest priority

[feature] support q2_k & q3_k dequantize on gpu

Update task_queue.h

[Feature] towards 0.1.2

[fix] linux and windows can all find CPUInfer in current Directory

1) Linear and MLP operators support qlen>1; 2) All operators now share a single memory buffer; 3) Refactor CPUInfer submit/sync logic.

Windows Support

support windows support q4_0 and q5_0 dequant on cpu Add CopyRight from pygguf(It was added before, but disappear after merge). Add some TODO in the code.

update docker.md to support docker pull image

Feature support multi instruct