Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
a8cb6554
"ssh:/git@developer.sourcefind.cn:2222/OpenDAS/dynamo.git" did not exist on "05913af5e83b3e418218aed8d5ef239976ae8100"
Unverified
Commit
a8cb6554
authored
Jul 29, 2025
by
Anant Sharma
Committed by
GitHub
Jul 29, 2025
Browse files
docs: update readme install instructions (#2170)
parent
ca5b681a
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
19 additions
and
5 deletions
+19
-5
README.md
README.md
+10
-3
components/backends/llama_cpp/README.md
components/backends/llama_cpp/README.md
+9
-2
No files found.
README.md
View file @
a8cb6554
...
@@ -168,12 +168,16 @@ To specify which GPUs to use set environment variable `CUDA_VISIBLE_DEVICES`.
...
@@ -168,12 +168,16 @@ To specify which GPUs to use set environment variable `CUDA_VISIBLE_DEVICES`.
## SGLang
## SGLang
```
```
# Install libnuma
apt install -y libnuma-dev
uv pip install ai-dynamo[sglang]
uv pip install ai-dynamo[sglang]
```
```
Run the backend/worker like this:
Run the backend/worker like this:
```
```
python -m dynamo.sglang.worker --help #Note the '.worker' in the module path for SGLang
# Note the '.worker' in the module path for SGLang
python -m dynamo.sglang.worker --help
```
```
You can pass any sglang flags directly to this worker, see https://docs.sglang.ai/backend/server_arguments.html . See there to use multiple GPUs.
You can pass any sglang flags directly to this worker, see https://docs.sglang.ai/backend/server_arguments.html . See there to use multiple GPUs.
...
@@ -203,7 +207,7 @@ sudo apt-get -y install libopenmpi-dev
...
@@ -203,7 +207,7 @@ sudo apt-get -y install libopenmpi-dev
### After installing the pre-requisites above, install Dynamo
### After installing the pre-requisites above, install Dynamo
```
```
uv pip install --upgrade pip setuptools &&
uv pip install ai-dynamo[trtllm]
uv pip install ai-dynamo[trtllm]
```
```
Run the backend/worker like this:
Run the backend/worker like this:
...
@@ -273,9 +277,12 @@ maturin develop --uv
...
@@ -273,9 +277,12 @@ maturin develop --uv
```
```
cd $PROJECT_ROOT
cd $PROJECT_ROOT
uv pip install .
uv pip install .
# For development, use
export PYTHONPATH="${PYTHONPATH}:$(pwd)/components/frontend/src:$(pwd)/components/planner/src:$(pwd)/components/backends/vllm/src:$(pwd)/components/backends/sglang/src:$(pwd)/components/backends/trtllm/src:$(pwd)/components/backends/llama_cpp/src:$(pwd)/components/backends/mocker/src"
```
```
Note editable (
`-e`
) does not work because the
`dynamo`
package is split over multiple directories, one per backend.
> [!Note]
> Editable (`-e`) does not work because the `dynamo` package is split over multiple directories, one per backend.
You should now be able to run
`python -m dynamo.frontend`
.
You should now be able to run
`python -m dynamo.frontend`
.
...
...
components/backends/llama_cpp/README.md
View file @
a8cb6554
# llama.cpp engine for Dynamo
# llama.cpp engine for Dynamo
Usage:
Usage:
-
`pip install -r requirements.txt`
# Need a recent pip,
`uv pip`
might be too old.
```
-
`python -m dynamo.llama_cpp --model-path /data/models/Qwen3-0.6B-Q8_0.gguf [args]`
# Install ai-dynamo llama.cpp backend (CPU Mode)
pip install "ai-dynamo[llama_cpp]"
# [Optional] To build llama.cpp for CUDA (needs a recent pip)
pip install -r --force-reinstall requirements.gpu.txt
python -m dynamo.llama_cpp --model-path /data/models/Qwen3-0.6B-Q8_0.gguf [args]
```
## Request Migration
## Request Migration
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment