GitLab

* translate turbomind

* keep persistent batching

* revised

* revise

* works on interlm and vicuna

* support GQA

* remove comment

* update readme, add logger, default tp=1

* remove log

* clean-up

* fix lint

* fix lint

* bump version to v0.0.2

* fix command

* update installation and inference section

* add pypi ci

* fix build

* Doc: add Twitter link

* Doc: add a space

* update builder

* remove root permission

* update readme

* update setup.py

* add install cuda 12.1 script

* use generate.sh

* add nccl to install_requires

* update README.md

* fix lint

* update setup.py

---------
Co-authored-by: chenxin <chenxin@pjlab.org.cn>

* translate quantization doc

* revise

* Update README_zh-CN.md

* Update README.md

* Update README_zh-CN.md

* Update README.md

* Update README_zh-CN.md

* defer symlink

* fix lint

Co-authored-by: grimoire <yaoqian@pjlab.org.cn>

* checkin benchmark on real conversation data

* change resolution

* update

* decode only forward pass

* fix lint

* batch embedding

* refactor model.py and support baichuan-7b

* remove model_name

* remove hard session_len

* export tokenizer.py to target dir

* remove model_name from client

* remove model_name

* update

* correct throughput equation

* fix session.response

* update serving.md

* update readme

* update according to review comments

* update

* update

* update

* update

* add profile throughput benchmark

* add output only throughput

* update req/min

* update benckmark readme

* fix lint

---------
Co-authored-by: grimoire <yaoqian@pjlab.org.cn>

* Fix lmdeploy.serve.turbomind bug
* add __init__.py for turbomind
* add resume function
* fix the assignment for session.response

* Fix code style

* add GQA for llama2

* fix model conversion

* fix lint & remove dev log

* update news

* minor

* fix allocation size

* fix split_dim for w_qkv.bias

* [Fix] fix issue 127

* 优化防止接口更改

* 如果没有deepspeed用python启动需要手动加载到GPU上

* rollback the changes about max_out_tokens and delelte torch > 2.0 if statement

* support kernel injection with customized deepspeed

* spelling error

* Update chat.py

---------
Co-authored-by: wangruohui <12756472+wangruohui@users.noreply.github.com>

* add docker action

* update

* fix

* update

* add llama2 template

* update readme and fix lint

* update readme

* add bos

* add bos

* remove bos

* Update model.py

---------
Co-authored-by: grimoire <yaoqian@pjlab.org.cn>

* fix get_dataset error

* fix lint

* add datasets to requirements.txt

* update some msci

* remove copy

* repetition_penalty=1

* add repetition_penalty to chat args

* update readme

* update readme

* update requirements

* update transformers version

* lint

* comments

* lint

* update requirements

* remove setup_requires

---------
Co-authored-by: dongchunyu <dongchunyu@pjlab.org.cn>

* wip

* profile disable tp

* fix profile

* lint

* fix dlpack

* remove comment

* add tp flag

* add session len check

* add eos

* remove tp and session len inputs

* warp tokenizer

* multithread load weight

* update profile

* refactor tokenizer

* remove pre/post process

* remove mpi4py requirement

* remove

* remove bind

* remove mpi requirement

* check backend_tokenizer

* update log info

* format cuda utils

* [Fix] fix attempted_relative_import

* use try...except...else

* docs: fix doc

* fix: fix lint

* [bugfix] Fix some docs' bug in 'serving'

* [bugfix] Fix some docs' bug in 'serving'

* Update README.md: use internlm-chat-7b

* Update README_zh-CN.md: use intern-chat-7b

* Doc: update discord and wechat link

* Doc: update discord and wechat link

* [Doc] add discord and wechat link

* [Doc] add discord and wechat link

* [Doc] add discord and wechat link

* [Doc] add discord and wechat link

* move turbomind.md to docs/en

* update link

* update link