GitLab

Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

Possible fix for conflict between Automated Prefix Caching (#2762) and multi-LoRA support (#1804) (#3263)

Co-authored-by: Avnish Narayan <avnish@anyscale.com>

Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

Co-authored-by: Yunmo Chen <16273544+wanmok@users.noreply.github.com>

* [FIX] Minor bug fixes

* Address review comments