Unverified Commit 291700c9 authored by Matt Williams's avatar Matt Williams Committed by GitHub
Browse files

Clean up documentation (#1506)



* Clean up documentation

Will probably need to update with PRs for new release.
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* Correcting to fit in 0.1.15 changes
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* Update README.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* addressing comments
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* more api cleanup
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* its llava not llama
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* Update docs/troubleshooting.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Updated hosting to server and documented all env vars
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* remove last of the cli descriptions
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* Update README.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* update further per conversation with jeff earlier today
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* cleanup the doc readme
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* move upgrade to faq
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* first change
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* updated
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* Update docs/faq.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/README.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update README.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/README.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/README.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/README.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/README.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* examples in parent
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* add exapmle for create model.
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* update faq
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* update create model api
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* Update docs/api.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/faq.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/troubleshooting.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* update the readme in docs
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* update a few more things
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>

* Update docs/troubleshooting.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/faq.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update README.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/modelfile.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

* Update docs/troubleshooting.md
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>

---------
Signed-off-by: default avatarMatt Williams <m@technovangelist.com>
Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>
parent 9db28af8
...@@ -60,7 +60,7 @@ Here are some example open-source models that can be downloaded: ...@@ -60,7 +60,7 @@ Here are some example open-source models that can be downloaded:
| Vicuna | 7B | 3.8GB | `ollama run vicuna` | | Vicuna | 7B | 3.8GB | `ollama run vicuna` |
| LLaVA | 7B | 4.5GB | `ollama run llava` | | LLaVA | 7B | 4.5GB | `ollama run llava` |
> Note: You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models. > Note: You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
## Customize your own model ## Customize your own model
...@@ -129,6 +129,10 @@ For more examples, see the [examples](examples) directory. For more information ...@@ -129,6 +129,10 @@ For more examples, see the [examples](examples) directory. For more information
`ollama create` is used to create a model from a Modelfile. `ollama create` is used to create a model from a Modelfile.
```
ollama create mymodel -f ./Modelfile
```
### Pull a model ### Pull a model
``` ```
......
# Documentation # Documentation
- [Modelfile](./modelfile.md) To get started, see the project's **[quicktart](../README.md#quickstart)**.
- [How to develop Ollama](./development.md)
- [API](./api.md) Ollama is a tool for running AI models on your hardware. Many users will choose to use the Command Line Interface (CLI) to work with Ollama. Learn more about all the commands in the CLI in the **[Main Readme](../README.md)**.
- [Tutorials](./tutorials.md)
Use the RESTful API using any language, including Python, JavaScript, Typescript, Go, Rust, and many more. Learn more about using the API in the **[API Documentation](./api.md)**.
Create new models or modify models already in the library using the Modelfile. Learn more about the Modelfile syntax in the **[Modelfile Documentation](./modelfile.md)**.
Import models using source model weights found on Hugging Face and similar sites by referring to the **[Import Documentation](./import.md)**.
Installing on Linux in most cases is easy using the script on Ollama.ai. To get more detail about the install, including CUDA drivers, see the **[Linux Documentation](./linux.md)**.
Many of our users like the flexibility of using our official Docker Image. Learn more about using Docker with Ollama using the **[Docker Documentation](./docker.md)**.
It is easy to install on Linux and Mac, but many users will choose to build Ollama on their own. To do this, refer to the **[Development Documentation](./development.md)**.
If encountering a problem with Ollama, the best place to start is the logs. Find more information about them here in the **[Troubleshooting Guide](./troubleshooting.md)**.
Finally for all the questions that don't fit anywhere else, there is the **[FAQ](./faq.md)**
[Tutorials](./tutorials.md) apply the documentation to tasks.
For working code examples of using Ollama, see [Examples](../examples).
\ No newline at end of file
This diff is collapsed.
# FAQ # FAQ
## How can I view the logs? ## How can I upgrade Ollama?
On macOS: To upgrade Ollama, run the installation process again. On the Mac, click the Ollama icon in the menubar and choose the restart option if an update is available.
``` ## How can I view the logs?
cat ~/.ollama/logs/server.log
```
On Linux: Review the [Troubleshooting](./troubleshooting.md) docs for more about using logs.
``` ## How do I use Ollama server environment variables on Mac
journalctl -u ollama
```
If you're running `ollama serve` directly, the logs will be printed to the console. On macOS, Ollama runs in the background and is managed by the menubar app. If adding environment variables, Ollama will need to be run manually.
## How can I expose Ollama on my network? 1. Click the menubar icon for Ollama and choose **Quit Ollama**.
2. Open a new terminal window and run the following command (this example uses `OLLAMA_HOST` with an IP address of `123.1.1.1`):
Ollama binds to 127.0.0.1 port 11434 by default. Change the bind address with the `OLLAMA_HOST` environment variable. ```bash
OLLAMA_HOST=123.1.1.1 ollama serve
```
On macOS: ## How do I use Ollama server environment variables on Linux?
```bash If Ollama is installed with the install script, a systemd service was created, running as the Ollama user. To add an environment variable, such as OLLAMA_HOST, follow these steps:
OLLAMA_HOST=0.0.0.0:11434 ollama serve
```
On Linux: 1. Create a `systemd` drop-in directory and add a config file. This is only needed once.
Create a `systemd` drop-in directory and set `Environment=OLLAMA_HOST` ```bash
mkdir -p /etc/systemd/system/ollama.service.d
echo '[Service]' >>/etc/systemd/system/ollama.service.d/environment.conf
```
```bash 2. For each environment variable, add it to the config file:
mkdir -p /etc/systemd/system/ollama.service.d
echo '[Service]' >>/etc/systemd/system/ollama.service.d/environment.conf
```
```bash ```bash
echo 'Environment="OLLAMA_HOST=0.0.0.0:11434"' >>/etc/systemd/system/ollama.service.d/environment.conf echo 'Environment="OLLAMA_HOST=0.0.0.0:11434"' >>/etc/systemd/system/ollama.service.d/environment.conf
``` ```
Reload `systemd` and restart Ollama:
```bash 3. Reload `systemd` and restart Ollama:
systemctl daemon-reload
systemctl restart ollama
```
## How can I allow additional web origins to access Ollama? ```bash
systemctl daemon-reload
systemctl restart ollama
```
Ollama allows cross origin requests from `127.0.0.1` and `0.0.0.0` by default. Add additional origins with the `OLLAMA_ORIGINS` environment variable: ## How can I expose Ollama on my network?
On macOS: Ollama binds to 127.0.0.1 port 11434 by default. Change the bind address with the `OLLAMA_HOST` environment variable. Refer to the section above for how to use environment variables on your platform.
```bash ## How can I allow additional web origins to access Ollama?
OLLAMA_ORIGINS=http://192.168.1.1:*,https://example.com ollama serve
```
On Linux: Ollama allows cross-origin requests from `127.0.0.1` and `0.0.0.0` by default. Add additional origins with the `OLLAMA_ORIGINS` environment variable. For example, to add all ports on 192.168.1.1 and https://example.com, use:
```bash ```shell
echo 'Environment="OLLAMA_ORIGINS=http://192.168.1.1:*,https://example.com"' >>/etc/systemd/system/ollama.service.d/environment.conf OLLAMA_ORIGINS=http://192.168.1.1:*,https://example.com
``` ```
Reload `systemd` and restart Ollama: Refer to the section above for how to use environment variables on your platform.
```bash
systemctl daemon-reload
systemctl restart ollama
```
## Where are models stored? ## Where are models stored?
- macOS: Raw model data is stored under `~/.ollama/models`. - macOS: `~/.ollama/models`.
- Linux: Raw model data is stored under `/usr/share/ollama/.ollama/models` - Linux: `/usr/share/ollama/.ollama/models`
Below the models directory you will find a structure similar to the following:
```shell See [the CLI Documentation](./cli.md) for more on this.
.
├── blobs
└── manifests
└── registry.ollama.ai
├── f0rodo
├── library
├── mattw
└── saikatkumardey
```
There is a `manifests/registry.ollama.ai/namespace` path. In example above, the user has downloaded models from the official `library`, `f0rodo`, `mattw`, and `saikatkumardey` namespaces. Within each of those directories, you will find directories for each of the models downloaded. And in there you will find a file name representing each tag. Each tag file is the manifest for the model. ## How do I set them to a different location?
The manifest lists all the layers used in this model. You will see a `media type` for each layer, along with a digest. That digest corresponds with a file in the `models/blobs directory`. If a different directory needs to be used, set the environment variable `OLLAMA_MODELS` to the chosen directory. Refer to the section above for how to use environment variables on your platform.
### How can I change where Ollama stores models?
To modify where models are stored, you can use the `OLLAMA_MODELS` environment variable. Note that on Linux this means defining `OLLAMA_MODELS` in a drop-in `/etc/systemd/system/ollama.service.d` service file, reloading systemd, and restarting the ollama service.
## Does Ollama send my prompts and answers back to Ollama.ai to use in any way? ## Does Ollama send my prompts and answers back to Ollama.ai to use in any way?
No. Anything you do with Ollama, such as generate a response from the model, stays with you. We don't collect any data about how you use the model. You are always in control of your own data. No, Ollama runs entirely locally, and conversation data will never leave your machine.
## How can I use Ollama in Visual Studio Code? ## How can I use Ollama in Visual Studio Code?
There is already a large collection of plugins available for VSCode as well as other editors that leverage Ollama. You can see the list of [extensions & plugins](https://github.com/jmorganca/ollama#extensions--plugins) at the bottom of the main repository readme. There is already a large collection of plugins available for VSCode as well as other editors that leverage Ollama. See the list of [extensions & plugins](https://github.com/jmorganca/ollama#extensions--plugins) at the bottom of the main repository readme.
## How do I use Ollama behind a proxy? ## How do I use Ollama behind a proxy?
Ollama is compatible with proxy servers if `HTTP_PROXY` or `HTTPS_PROXY` are configured. When using either variables, ensure it is set where `ollama serve` can access the values. Ollama is compatible with proxy servers if `HTTP_PROXY` or `HTTPS_PROXY` are configured. When using either variables, ensure it is set where `ollama serve` can access the values. When using `HTTPS_PROXY`, ensure the proxy certificate is installed as a system certificate. Refer to the section above for how to use environment variables on your platform.
When using `HTTPS_PROXY`, ensure the proxy certificate is installed as a system certificate.
On macOS:
```bash
HTTPS_PROXY=http://proxy.example.com ollama serve
```
On Linux:
```bash
echo 'Environment="HTTPS_PROXY=https://proxy.example.com"' >>/etc/systemd/system/ollama.service.d/environment.conf
```
Reload `systemd` and restart Ollama:
```bash
systemctl daemon-reload
systemctl restart ollama
```
### How do I use Ollama behind a proxy in Docker? ### How do I use Ollama behind a proxy in Docker?
The Ollama Docker container image can be configured to use a proxy by passing `-e HTTPS_PROXY=https://proxy.example.com` when starting the container. The Ollama Docker container image can be configured to use a proxy by passing `-e HTTPS_PROXY=https://proxy.example.com` when starting the container.
Alternatively, Docker daemon can be configured to use a proxy. Instructions are available for Docker Desktop on [macOS](https://docs.docker.com/desktop/settings/mac/#proxies), [Windows](https://docs.docker.com/desktop/settings/windows/#proxies), and [Linux](https://docs.docker.com/desktop/settings/linux/#proxies), and Docker [daemon with systemd](https://docs.docker.com/config/daemon/systemd/#httphttps-proxy). Alternatively, the Docker daemon can be configured to use a proxy. Instructions are available for Docker Desktop on [macOS](https://docs.docker.com/desktop/settings/mac/#proxies), [Windows](https://docs.docker.com/desktop/settings/windows/#proxies), and [Linux](https://docs.docker.com/desktop/settings/linux/#proxies), and Docker [daemon with systemd](https://docs.docker.com/config/daemon/systemd/#httphttps-proxy).
Ensure the certificate is installed as a system certificate when using HTTPS. This may require a new Docker image when using a self-signed certificate. Ensure the certificate is installed as a system certificate when using HTTPS. This may require a new Docker image when using a self-signed certificate.
......
...@@ -72,7 +72,7 @@ docker run --rm -v .:/model ollama/quantize -q q4_0 /model ...@@ -72,7 +72,7 @@ docker run --rm -v .:/model ollama/quantize -q q4_0 /model
This will output two files into the directory: This will output two files into the directory:
- `f16.bin`: the model converted to GGUF - `f16.bin`: the model converted to GGUF
- `q4_0.bin` the model quantized to a 4-bit quantization (we will use this file to create the Ollama model) - `q4_0.bin` the model quantized to a 4-bit quantization (Ollama will use this file to create the Ollama model)
### Step 3: Write a `Modelfile` ### Step 3: Write a `Modelfile`
......
# Ollama Model File # Ollama Model File
> Note: this `Modelfile` syntax is in development > Note: `Modelfile` syntax is in development
A model file is the blueprint to create and share models with Ollama. A model file is the blueprint to create and share models with Ollama.
...@@ -75,7 +75,7 @@ There are two ways to view `Modelfile`s underlying the models in [ollama.ai/libr ...@@ -75,7 +75,7 @@ There are two ways to view `Modelfile`s underlying the models in [ollama.ai/libr
3. Scroll down to "Layers" 3. Scroll down to "Layers"
- Note: if the [`FROM` instruction](#from-required) is not present, - Note: if the [`FROM` instruction](#from-required) is not present,
it means the model was created from a local file it means the model was created from a local file
- Option 2: use `ollama show` to print the `Modelfile` like so: - Option 2: use `ollama show` to print the `Modelfile` for any local models like so:
```bash ```bash
> ollama show --modelfile llama2:13b > ollama show --modelfile llama2:13b
...@@ -206,7 +206,7 @@ LICENSE """ ...@@ -206,7 +206,7 @@ LICENSE """
## Notes ## Notes
- the **`Modelfile` is not case sensitive**. In the examples, we use uppercase for instructions to make it easier to distinguish it from arguments. - the **`Modelfile` is not case sensitive**. In the examples, uppercase instructions are used to make it easier to distinguish it from arguments.
- Instructions can be in any order. In the examples, we start with FROM instruction to keep it easily readable. - Instructions can be in any order. In the examples, the `FROM` instruction is first to keep it easily readable.
[1]: https://ollama.ai/library [1]: https://ollama.ai/library
# How to troubleshoot issues
Sometimes Ollama may not perform as expected. One of the best ways to figure out what happened is to take a look at the logs. Find the logs on Mac by running the command:
```shell
cat ~/.ollama/logs/server.log
```
On Linux systems with systemd, the logs can be found with this command:
```shell
journalctl -u ollama
```
If manually running `ollama serve` in a terminal, the logs will be on that terminal.
Join the [Discord](https://discord.gg/ollama) for help interpreting the logs.
## Known issues
* `signal: illegal instruction (core dumped)`: Ollama requires AVX support from the CPU. This was introduced in 2011 and CPUs started offering it in 2012. CPUs from before that and some lower end CPUs after that may not have AVX support and thus are not supported by Ollama. Some users have had luck with building Ollama on their machines disabling the need for AVX.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment