GitLab

Merge branch 'dtk23.10-v0.0.13-qwen' of http://developer.hpccube.com/codes/aicomponent/lmdeploy into dtk23.10-v0.0.13-qwen

* support inference a batch of prompts

* docstring and assert

* add

* import fire in main

* wrap to speed up fire cli

* update

* update docs

* update docs

* fix

* resolve commennts

* resolve confict and add test for cli

* add build from docker section

* update

* install python package

* update

* update

* update

* fix crash

* update profile_generation.py

* format

* use self.bos_id

* remove sys_instruct

This reverts commit 4b5c2bda.

* robust incremental decode for leading space

* speed up lookup as prefix_space_tokens is shorter than no_prefix_space_tokens

* add UT and fix qwen stuff

* move tokenizer

* remove Tokenizer in init

* update deploy.py

* free runner disk

* limit cpu

* docker.yml

* keep swap

* keep swap