Skip to content

GitLab

Commit 98baf41a authored by huangwb's avatar huangwb
Browse files

add vllm exllama_kernels install command

parent 173387ea
Hide whitespace changes
Inline Side-by-side
Showing with 9 additions and 0 deletions
server/Makefile
View file @ 98baf41a
...@@ -5,6 +5,15 @@ include Makefile-selective-scan ...@@ -5,6 +5,15 @@ include Makefile-selective-scan
unit-tests: unit-tests:
pytest -s -vv -m "not private" tests pytest -s -vv -m "not private" tests
install-vllm:
cd vllm/ && python setup.py install
install-exllama:
cd exllama_kernels && python setup.py install
install-exllamav2:
cd exllamav2_kernels && python setup.py install
gen-server: gen-server:
# Compile protos # Compile protos
pip install grpcio-tools==1.51.1 mypy-protobuf==3.4.0 'types-protobuf>=3.20.4' --no-cache-dir pip install grpcio-tools==1.51.1 mypy-protobuf==3.4.0 'types-protobuf>=3.20.4' --no-cache-dir
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment