Skip to content

GitLab

"sgl-kernel/python/vscode:/vscode.git/clone" did not exist on "1da8d23051242b1066943b4c041ee85d33be74d9"
Switch branch/tag
Find file BlameHistoryPermalink
  • tpoisonooo's avatar
    feat(src): add kv cache int8 quantization (#22) · cc93136e
    tpoisonooo authored 
    * feat(src): add int8 and compile passed

* feat(kernels): fix

* feat(llama): update kernel

* feat(src): add debug

* fix(kernel): k_cache use int8_t pointer

* style(llama): clean code

* feat(deploy.py): revert to enable fmha

* style(LlamaV2): clean code

* feat(deploy.py): add default quant policy
    cc93136e
LlamaContextDecoder.cc 13.3 KB