GitLab

Set the reminder to set CUDA_HOME and CUDA_PATH in the README to the "Quick Start" section under "Install CUDA".

Fix: None for load config

[fix] f16 dequantize device ignored

[fix] Fix bugs about static cache and server param;

Update README

[fix] fix broken link in tutorial.

Release v0.1.2

[update] Update readme; Add tutorial

[fix] format classes and files name

[fix] fix ktransformers interface when use new CUDAGraphRunner
[fix] fix YAML and optimize logic, the top rule has the highest priority

[feature] support q2_k & q3_k dequantize on gpu

Update task_queue.h

[Feature] towards 0.1.2

[fix] linux and windows can all find CPUInfer in current Directory

1) Linear and MLP operators support qlen>1; 2) All operators now share a single memory buffer; 3) Refactor CPUInfer submit/sync logic.

Windows Support

support windows support q4_0 and q5_0 dequant on cpu Add CopyRight from pygguf(It was added before, but disappear after merge). Add some TODO in the code.

update docker.md to support docker pull image