GitLab

* add performance

* use png

* update

* update

* update

* update

* update

* update internlm model

* update

* update

* update

* update

* update temperature, topk and top_p

* update

* update

* loosen log level

* add demo gif

* add demo gif

* fix(kv_qparams.py): zp use min

* revert(qparams.py): revert format

* fix(kv_qparams.py): update formula

* add cal qparams

* support offload inference

* add collect funtions (mod,weight)

* stats kv scales

* update init

* add user guide

* fix hints

* fix comments & support turbomind format

* update user guide

* fix slice kv cache error & support pileval dataset (used in llm-awq)

* fix wrong num heads slice

* update default dataset

* fix conflict

* fix hints

* fix hints

* add gitignore

* docs(quantization): add more test

* revert(generate.sh): revert ninja

* revert(llama_config.ini): revert empty line

* fix(quantization.md): fix link error

* wip

* wip

* example finish

* fix include and namespace

* wtf

* install lib

* batchize

* update cmake install

* multithread

* fix comment

* fix

* add mmengine

* bind llamamodel

---------
Co-authored-by: grimoire <yaoqian@pjlab.org.cn>

* format-11.1

* md-link-config

* check-in script for tokenizing a file

* use max_input_len

* docs(README): update description

* docs(project): add quantization test results

* docs(README): reorder

* docs(quantization): add more description

* docs(README): remove openmmlab badge

* docs(README): scale up image

* docs(dir): add zh_cn subdir

* [Doc] add persistent batch inference

* update

* update

* Update README.md

* Update README_zh-CN.md

* feat(template): remote diff

* feat(cmake): use c++17

* change target tritonfastertransformerbackend to tritonturbomindbackend

* install targets to backends/turbomind

* changge model_dir

* build turbomind

* change namespace fastertransformer to turbomind

* change logger name

* temp

* fix lint

* csrc->src

* remove clang-format

* skip .rst

* skip doc

* clang-format

version

version

* mat_B

* rename lmdeploy/serve/fastertransformer to lmdeploy/serve/turbomind

* update

* update

* change llmdeploy to lmdeploy

* update logo

* update readme

* add hf tokenizer

* format

* fix for comment

* don't skip speical tokens

* add webui

* update readme

* resolve comments

* readme

* feat(src): add int8 and compile passed

* feat(kernels): fix

* feat(llama): update kernel

* feat(src): add debug

* fix(kernel): k_cache use int8_t pointer

* style(llama): clean code

* feat(deploy.py): revert to enable fmha

* style(LlamaV2): clean code

* feat(deploy.py): add default quant policy