Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
a8cb6554
Unverified
Commit
a8cb6554
authored
Jul 29, 2025
by
Anant Sharma
Committed by
GitHub
Jul 29, 2025
Browse files
docs: update readme install instructions (#2170)
parent
ca5b681a
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
19 additions
and
5 deletions
+19
-5
README.md
README.md
+10
-3
components/backends/llama_cpp/README.md
components/backends/llama_cpp/README.md
+9
-2
No files found.
README.md
View file @
a8cb6554
...
...
@@ -168,12 +168,16 @@ To specify which GPUs to use set environment variable `CUDA_VISIBLE_DEVICES`.
## SGLang
```
# Install libnuma
apt install -y libnuma-dev
uv pip install ai-dynamo[sglang]
```
Run the backend/worker like this:
```
python -m dynamo.sglang.worker --help #Note the '.worker' in the module path for SGLang
# Note the '.worker' in the module path for SGLang
python -m dynamo.sglang.worker --help
```
You can pass any sglang flags directly to this worker, see https://docs.sglang.ai/backend/server_arguments.html . See there to use multiple GPUs.
...
...
@@ -203,7 +207,7 @@ sudo apt-get -y install libopenmpi-dev
### After installing the pre-requisites above, install Dynamo
```
uv pip install --upgrade pip setuptools &&
uv pip install ai-dynamo[trtllm]
uv pip install ai-dynamo[trtllm]
```
Run the backend/worker like this:
...
...
@@ -273,9 +277,12 @@ maturin develop --uv
```
cd $PROJECT_ROOT
uv pip install .
# For development, use
export PYTHONPATH="${PYTHONPATH}:$(pwd)/components/frontend/src:$(pwd)/components/planner/src:$(pwd)/components/backends/vllm/src:$(pwd)/components/backends/sglang/src:$(pwd)/components/backends/trtllm/src:$(pwd)/components/backends/llama_cpp/src:$(pwd)/components/backends/mocker/src"
```
Note editable (
`-e`
) does not work because the
`dynamo`
package is split over multiple directories, one per backend.
> [!Note]
> Editable (`-e`) does not work because the `dynamo` package is split over multiple directories, one per backend.
You should now be able to run
`python -m dynamo.frontend`
.
...
...
components/backends/llama_cpp/README.md
View file @
a8cb6554
# llama.cpp engine for Dynamo
Usage:
-
`pip install -r requirements.txt`
# Need a recent pip,
`uv pip`
might be too old.
-
`python -m dynamo.llama_cpp --model-path /data/models/Qwen3-0.6B-Q8_0.gguf [args]`
```
# Install ai-dynamo llama.cpp backend (CPU Mode)
pip install "ai-dynamo[llama_cpp]"
# [Optional] To build llama.cpp for CUDA (needs a recent pip)
pip install -r --force-reinstall requirements.gpu.txt
python -m dynamo.llama_cpp --model-path /data/models/Qwen3-0.6B-Q8_0.gguf [args]
```
## Request Migration
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment