README.md

# `llama`

This package provides Go bindings to [llama.cpp](https://github.com/ggerganov/llama.cpp).

## Vendoring

Ollama vendors [llama.cpp](https://github.com/ggerganov/llama.cpp/) and [ggml](https://github.com/ggerganov/llama.cpp/tree/master/ggml/src). While we generally strive to contribute changes back upstream to avoid drift, we carry a small set of patches which are applied to the tracking commit.

If you update the vendoring code, start by running the following command to establish the tracking llama.cpp repo in the `./vendor/` directory.

```
make -f Makefile.sync apply-patches
```

### Updating Base Commit

**Pin to new base commit**

To change the base commit, update `FETCH_HEAD` in Makefile.sync.

When updating to a newer base commit, the existing patches may not apply cleanly and require manual merge resolution.

Start by applying the patches. If any of the patches have conflicts, the `git am` will stop at the first failure.

```
make -f Makefile.sync apply-patches
```

If there are conflicts, you will see an error message. Resolve the conflicts in `./vendor/`, and continue the patch series with `git am --continue` and rerun `make -f Makefile.sync apply-patches`. Repeat until all patches are successfully applied.

Once all patches are applied, commit the changes to the tracking repository.

```
make -f Makefile.sync format-patches sync
```

### Generating Patches

When working on new fixes or features that impact vendored code, use the following model. First get a clean tracking repo with all current patches applied:

```
make -f Makefile.sync clean apply-patches
```

Iterate until you're ready to submit PRs. Once your code is ready, commit a change in the `./vendor/` directory, then generate the patches for ollama with

```
make -f Makefile.sync format-patches
```

In your `./vendor/` directory, create a branch, and cherry-pick the new commit to that branch, then submit a PR upstream to llama.cpp.

Commit the changes in the ollama repo and submit a PR to Ollama, which will include the vendored code update with your change, along with the patches.

After your PR upstream is merged, follow the **Updating Base Commit** instructions above, however first remove your patch before running `apply-patches` since the new base commit contains your change already.