Skip to content

GitLab

Switch branch/tag
  1. 28 Jun, 2023 1 commit
    • tpoisonooo's avatar
      feat(src): add kv cache int8 quantization (#22) · cc93136e
      tpoisonooo authored 
      * feat(src): add int8 and compile passed

* feat(kernels): fix

* feat(llama): update kernel

* feat(src): add debug

* fix(kernel): k_cache use int8_t pointer

* style(llama): clean code

* feat(deploy.py): revert to enable fmha

* style(LlamaV2): clean code

* feat(deploy.py): add default quant policy
      cc93136e
  3. 25 Jun, 2023 1 commit
  5. 21 Jun, 2023 1 commit
  7. 20 Jun, 2023 1 commit
  9. 18 Jun, 2023 1 commit