Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
text-generation-inference
Commits
169178b9
Unverified
Commit
169178b9
authored
Sep 20, 2024
by
Nicolas Patry
Committed by
GitHub
Sep 20, 2024
Browse files
Preparing for release. (#2540)
* Preparing for release. * Upgrade version in docs.
parent
7e2d1887
Changes
7
Hide whitespace changes
Inline
Side-by-side
Showing
7 changed files
with
8 additions
and
8 deletions
+8
-8
Cargo.toml
Cargo.toml
+1
-1
README.md
README.md
+1
-1
docs/openapi.json
docs/openapi.json
+1
-1
docs/source/installation_amd.md
docs/source/installation_amd.md
+1
-1
docs/source/installation_intel.md
docs/source/installation_intel.md
+2
-2
docs/source/installation_nvidia.md
docs/source/installation_nvidia.md
+1
-1
docs/source/quicktour.md
docs/source/quicktour.md
+1
-1
No files found.
Cargo.toml
View file @
169178b9
...
@@ -20,7 +20,7 @@ default-members = [
...
@@ -20,7 +20,7 @@ default-members = [
resolver
=
"2"
resolver
=
"2"
[workspace.package]
[workspace.package]
version
=
"2.
2
.1-dev0"
version
=
"2.
3
.1-dev0"
edition
=
"2021"
edition
=
"2021"
authors
=
[
"Olivier Dehaene"
]
authors
=
[
"Olivier Dehaene"
]
homepage
=
"https://github.com/huggingface/text-generation-inference"
homepage
=
"https://github.com/huggingface/text-generation-inference"
...
...
README.md
View file @
169178b9
...
@@ -83,7 +83,7 @@ model=HuggingFaceH4/zephyr-7b-beta
...
@@ -83,7 +83,7 @@ model=HuggingFaceH4/zephyr-7b-beta
volume
=
$PWD
/data
volume
=
$PWD
/data
docker run
--gpus
all
--shm-size
1g
-p
8080:80
-v
$volume
:/data
\
docker run
--gpus
all
--shm-size
1g
-p
8080:80
-v
$volume
:/data
\
ghcr.io/huggingface/text-generation-inference:2.
2
.0
--model-id
$model
ghcr.io/huggingface/text-generation-inference:2.
3
.0
--model-id
$model
```
```
And then you can make requests like
And then you can make requests like
...
...
docs/openapi.json
View file @
169178b9
...
@@ -10,7 +10,7 @@
...
@@ -10,7 +10,7 @@
"name"
:
"Apache 2.0"
,
"name"
:
"Apache 2.0"
,
"url"
:
"https://www.apache.org/licenses/LICENSE-2.0"
"url"
:
"https://www.apache.org/licenses/LICENSE-2.0"
},
},
"version"
:
"2.
2
.1-dev0"
"version"
:
"2.
3
.1-dev0"
},
},
"paths"
:
{
"paths"
:
{
"/"
:
{
"/"
:
{
...
...
docs/source/installation_amd.md
View file @
169178b9
...
@@ -11,7 +11,7 @@ volume=$PWD/data # share a volume with the Docker container to avoid downloading
...
@@ -11,7 +11,7 @@ volume=$PWD/data # share a volume with the Docker container to avoid downloading
docker run
--rm
-it
--cap-add
=
SYS_PTRACE
--security-opt
seccomp
=
unconfined
\
docker run
--rm
-it
--cap-add
=
SYS_PTRACE
--security-opt
seccomp
=
unconfined
\
--device
=
/dev/kfd
--device
=
/dev/dri
--group-add
video
\
--device
=
/dev/kfd
--device
=
/dev/dri
--group-add
video
\
--ipc
=
host
--shm-size
256g
--net
host
-v
$volume
:/data
\
--ipc
=
host
--shm-size
256g
--net
host
-v
$volume
:/data
\
ghcr.io/huggingface/text-generation-inference:2.
2
.0-rocm
\
ghcr.io/huggingface/text-generation-inference:2.
3
.0-rocm
\
--model-id
$model
--model-id
$model
```
```
...
...
docs/source/installation_intel.md
View file @
169178b9
...
@@ -12,7 +12,7 @@ volume=$PWD/data # share a volume with the Docker container to avoid downloading
...
@@ -12,7 +12,7 @@ volume=$PWD/data # share a volume with the Docker container to avoid downloading
docker run
--rm
--privileged
--cap-add
=
sys_nice
\
docker run
--rm
--privileged
--cap-add
=
sys_nice
\
--device
=
/dev/dri
\
--device
=
/dev/dri
\
--ipc
=
host
--shm-size
1g
--net
host
-v
$volume
:/data
\
--ipc
=
host
--shm-size
1g
--net
host
-v
$volume
:/data
\
ghcr.io/huggingface/text-generation-inference:2.
2
.0-intel-xpu
\
ghcr.io/huggingface/text-generation-inference:2.
3
.0-intel-xpu
\
--model-id
$model
--cuda-graphs
0
--model-id
$model
--cuda-graphs
0
```
```
...
@@ -29,7 +29,7 @@ volume=$PWD/data # share a volume with the Docker container to avoid downloading
...
@@ -29,7 +29,7 @@ volume=$PWD/data # share a volume with the Docker container to avoid downloading
docker run
--rm
--privileged
--cap-add
=
sys_nice
\
docker run
--rm
--privileged
--cap-add
=
sys_nice
\
--device
=
/dev/dri
\
--device
=
/dev/dri
\
--ipc
=
host
--shm-size
1g
--net
host
-v
$volume
:/data
\
--ipc
=
host
--shm-size
1g
--net
host
-v
$volume
:/data
\
ghcr.io/huggingface/text-generation-inference:2.
2
.0-intel-cpu
\
ghcr.io/huggingface/text-generation-inference:2.
3
.0-intel-cpu
\
--model-id
$model
--cuda-graphs
0
--model-id
$model
--cuda-graphs
0
```
```
...
...
docs/source/installation_nvidia.md
View file @
169178b9
...
@@ -11,7 +11,7 @@ model=teknium/OpenHermes-2.5-Mistral-7B
...
@@ -11,7 +11,7 @@ model=teknium/OpenHermes-2.5-Mistral-7B
volume
=
$PWD
/data
# share a volume with the Docker container to avoid downloading weights every run
volume
=
$PWD
/data
# share a volume with the Docker container to avoid downloading weights every run
docker run
--gpus
all
--shm-size
64g
-p
8080:80
-v
$volume
:/data
\
docker run
--gpus
all
--shm-size
64g
-p
8080:80
-v
$volume
:/data
\
ghcr.io/huggingface/text-generation-inference:2.
2
.0
\
ghcr.io/huggingface/text-generation-inference:2.
3
.0
\
--model-id
$model
--model-id
$model
```
```
...
...
docs/source/quicktour.md
View file @
169178b9
...
@@ -11,7 +11,7 @@ model=teknium/OpenHermes-2.5-Mistral-7B
...
@@ -11,7 +11,7 @@ model=teknium/OpenHermes-2.5-Mistral-7B
volume
=
$PWD
/data
# share a volume with the Docker container to avoid downloading weights every run
volume
=
$PWD
/data
# share a volume with the Docker container to avoid downloading weights every run
docker run
--gpus
all
--shm-size
1g
-p
8080:80
-v
$volume
:/data
\
docker run
--gpus
all
--shm-size
1g
-p
8080:80
-v
$volume
:/data
\
ghcr.io/huggingface/text-generation-inference:2.
2
.0
\
ghcr.io/huggingface/text-generation-inference:2.
3
.0
\
--model-id
$model
--model-id
$model
```
```
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment