GitLab

Only modified the mixed single and double quotes in server/config/config.py

fix local_chat bug and update doc

Update SUMMARY.md

delete sudo install

yaml: fix Marlin AssertionError

fix: refine backend error message to include ROCM_HOME

Update balance-serve.md

In v0.2.4 version, we’ve added highly desired multi-concurrency support to the community through a major refactor of the whole architecture.

Fix db position issue

Signed-off-by: fishingfly <zhoyuzf@163.com>

Update README.md

Marlin quantized linear only supports GPU device, when change generate_op
to "KLinearMarlin", generate_device need to be changed to "cuda" accordingly.

Fixes: e5b001d7 ("Update readme; Format code; Add example yaml.")

fix benchmark params for human eval benchmark

Update gate.py

Fix benchmarking slow issue on self-hosted actions