Unverified Commit a0dca443 authored by lewtun's avatar lewtun Committed by GitHub
Browse files

feat(docs): Clarify installation steps (#54)

Adds some bits for first-time users (like me 😄 )
parent 20c3c594
...@@ -93,7 +93,7 @@ curl 127.0.0.1:8080/generate_stream \ ...@@ -93,7 +93,7 @@ curl 127.0.0.1:8080/generate_stream \
-H 'Content-Type: application/json' -H 'Content-Type: application/json'
``` ```
To use GPUs, you will need to install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html). **Note:** To use GPUs, you need to install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html).
### API documentation ### API documentation
...@@ -102,14 +102,31 @@ The Swagger UI is also available at: [https://huggingface.github.io/text-generat ...@@ -102,14 +102,31 @@ The Swagger UI is also available at: [https://huggingface.github.io/text-generat
### Local install ### Local install
You can also opt to install `text-generation-inference` locally. You will need to have cargo and Python installed on your You can also opt to install `text-generation-inference` locally.
machine
First [install Rust](https://rustup.rs/) and create a Python virtual environment with at least
Python 3.9, e.g. using `conda`:
```shell
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
conda create -n text-generation-inference python=3.9
conda activate text-generation-inference
```
Then run:
```shell ```shell
BUILD_EXTENSIONS=True make install # Install repository and HF/transformer fork with CUDA kernels BUILD_EXTENSIONS=True make install # Install repository and HF/transformer fork with CUDA kernels
make run-bloom-560m make run-bloom-560m
``` ```
**Note:** on some machines, you may also need the OpenSSL libraries. On Linux machines, run:
```shell
sudo apt-get install libssl-dev
```
### CUDA Kernels ### CUDA Kernels
The custom CUDA kernels are only tested on NVIDIA A100s. If you have any installation or runtime issues, you can remove The custom CUDA kernels are only tested on NVIDIA A100s. If you have any installation or runtime issues, you can remove
...@@ -153,4 +170,4 @@ make router-dev ...@@ -153,4 +170,4 @@ make router-dev
```shell ```shell
make python-tests make python-tests
make integration-tests make integration-tests
``` ```
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment