Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "fa06f822f4eba34a4084a129aec35536fc133616"
Unverified Commit 7774ed4a authored by Michael Chiang's avatar Michael Chiang Committed by GitHub
Browse files

Update README.md for linux + cleanup (#601) 


Co-authored-by: default avatarJeffrey Morgan <jmorganca@gmail.com>
parent 11f920f2
Show whitespace changes
Inline Side-by-side
Showing with 99 additions and 99 deletions
README.md
View file @ 7774ed4a
...@@ -9,19 +9,27 @@ ...@@ -9,19 +9,27 @@
[![Discord](https://dcbadge.vercel.app/api/server/ollama?style=flat&compact=true)](https://discord.gg/ollama) [![Discord](https://dcbadge.vercel.app/api/server/ollama?style=flat&compact=true)](https://discord.gg/ollama)
Run, create, and share large language models (LLMs). Get up and running with large language models locally.
> Note: Ollama is in early preview. Please report any issues you find. ### macOS
## Download [Download](https://ollama.ai/download/Ollama-darwin.zip)
- [Download](https://ollama.ai/download) for macOS ### Linux & WSL2
- Download for Windows and Linux (coming soon)
- Build [from source](#building) ```
curl https://ollama.ai/install.sh | sh
```
[Manual install instructions](https://github.com/jmorganca/ollama/blob/main/docs/linux.md)
### Windows
coming soon
## Quickstart ## Quickstart
To run and chat with [Llama 2](https://ai.meta.com/llama), the new model by Meta: To run and chat with [Llama 2](https://ollama.ai/library/llama2):
``` ```
ollama run llama2 ollama run llama2
...@@ -29,87 +37,49 @@ ollama run llama2 ...@@ -29,87 +37,49 @@ ollama run llama2
## Model library ## Model library
Ollama supports a list of open-source models available on [ollama.ai/library](https://ollama.ai/library 'ollama model library') Ollama supports a list of open-source models available on [ollama.ai/library](https://ollama.ai/library "ollama model library")
Here are some example open-source models that can be downloaded: Here are some example open-source models that can be downloaded:
| Model | Parameters | Size | Download | | Model | Parameters | Size | Download |
| ------------------------ | ---------- | ----- | ------------------------------- | | ------------------ | ---------- | ----- | ------------------------------ |
| Llama2 | 7B | 3.8GB | `ollama pull llama2` | | Llama 2 | 7B | 3.8GB | `ollama run llama2` |
| Llama2 13B | 13B | 7.3GB | `ollama pull llama2:13b` | | Code Llama | 7B | 3.8GB | `ollama run codellama` |
| Llama2 70B | 70B | 39GB | `ollama pull llama2:70b` | | Llama 2 Uncensored | 7B | 3.8GB | `ollama run llama2-uncensored` |
| Llama2 Uncensored | 7B | 3.8GB | `ollama pull llama2-uncensored` | | Llama 2 13B | 13B | 7.3GB | `ollama run llama2:13b` |
| Code Llama | 7B | 3.8GB | `ollama pull codellama` | | Llama 2 70B | 70B | 39GB | `ollama run llama2:70b` |
| Orca Mini | 3B | 1.9GB | `ollama pull orca-mini` | | Orca Mini | 3B | 1.9GB | `ollama run orca-mini` |
| Vicuna | 7B | 3.8GB | `ollama pull vicuna` | | Vicuna | 7B | 3.8GB | `ollama run vicuna` |
| Nous-Hermes | 7B | 3.8GB | `ollama pull nous-hermes` |
| Nous-Hermes 13B | 13B | 7.3GB | `ollama pull nous-hermes:13b` |
| Wizard Vicuna Uncensored | 13B | 7.3GB | `ollama pull wizard-vicuna` |
> Note: You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models. > Note: You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models.
## Examples ## Customize your own model
### Pull a public model ### Import from GGUF or GGML
``` Ollama supports importing GGUF and GGML file formats in the Modelfile. This means if you have a model that is not in the Ollama library, you can create it, iterate on it, and upload it to the Ollama library to share with others when you are ready.
ollama pull llama2
```
> This command can also be used to update a local model. Only updated changes will be pulled. 1. Create a file named Modelfile, and add a `FROM` instruction with the local filepath to the model you want to import.
### Run a model interactively ```
FROM ./vicuna-33b.Q4_0.gguf
```
``` 3. Create the model in Ollama
ollama run llama2
>>> hi
Hello! How can I help you today?
```
For multiline input, you can wrap text with `"""`: ```
ollama create name -f path_to_modelfile
```
``` 5. Run the model
>>> """Hello,
... world!
... """
I'm a basic program that prints the famous "Hello, world!" message to the console.
```
### Run a model non-interactively
```
$ ollama run llama2 'tell me a joke'
Sure! Here's a quick one:
Why did the scarecrow win an award? Because he was outstanding in his field!
```
```
$ cat <<EOF >prompts.txt
tell me a joke about llamas
tell me another one
EOF
$ ollama run llama2 <prompts.txt
>>> tell me a joke about llamas
Why did the llama refuse to play hide-and-seek?
nobody likes to be hided!
>>> tell me another one ```
Sure, here's another one: ollama run name
```
Why did the llama go to the bar? ### Customize a prompt
To have a hay-often good time!
```
### Run a model on contents of a text file Models from the Ollama library can be customized with a prompt. The example
```
$ ollama run llama2 "summarize this file:" "$(cat README.md)"
Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications.
```
### Customize a model
Pull a base model:
``` ```
ollama pull llama2 ollama pull llama2
...@@ -138,30 +108,61 @@ ollama run mario ...@@ -138,30 +108,61 @@ ollama run mario
Hello! It's your friend Mario. Hello! It's your friend Mario.
``` ```
For more examples, see the [examples](./examples) directory. For more information on creating a Modelfile, see the [Modelfile](./docs/modelfile.md) documentation. For more examples, see the [examples](./examples) directory. For more information on working with a Modelfile, see the [Modelfile](./docs/modelfile.md) documentation.
## CLI Reference
### Create a model
`ollama create` is used to create a model from a Modelfile.
### Listing local models ### Pull a model
``` ```
ollama list ollama pull llama2
``` ```
### Removing local models > This command can also be used to update a local model. Only the diff will be pulled.
### Remove a model
``` ```
ollama rm llama2 ollama rm llama2
``` ```
## Model packages ### Copy a model
```
ollama cp llama2 my-llama2
```
### Multiline input
For multiline input, you can wrap text with `"""`:
```
>>> """Hello,
... world!
... """
I'm a basic program that prints the famous "Hello, world!" message to the console.
```
### Pass in prompt as arguments
```
$ ollama run llama2 "summarize this file:" "$(cat README.md)"
Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications.
```
### List models on your computer
### Overview ```
ollama list
```
Ollama bundles model weights, configurations, and data into a single package, defined by a [Modelfile](./docs/modelfile.md). ### Start Ollama
<picture> `ollama serve` is used when you want to start ollama without running the desktop application.
<source media="(prefers-color-scheme: dark)" height="480" srcset="https://github.com/jmorganca/ollama/assets/251292/2fd96b5f-191b-45c1-9668-941cfad4eb70">
<img alt="logo" height="480" src="https://github.com/jmorganca/ollama/assets/251292/2fd96b5f-191b-45c1-9668-941cfad4eb70">
</picture>
## Building ## Building
...@@ -204,19 +205,18 @@ curl -X POST http://localhost:11434/api/generate -d '{ ...@@ -204,19 +205,18 @@ curl -X POST http://localhost:11434/api/generate -d '{
}' }'
``` ```
## Community Projects using Ollama ## Community Integrations
| Project | Description | - [LangChain](https://python.langchain.com/docs/integrations/llms/ollama) and [LangChain.js](https://js.langchain.com/docs/modules/model_io/models/llms/integrations/ollama) with [example](https://js.langchain.com/docs/use_cases/question_answering/local_retrieval_qa)
| -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | - [LlamaIndex](https://gpt-index.readthedocs.io/en/stable/examples/llm/ollama.html)
| [LangChain][1] and [LangChain.js][2] | Also, there is a question-answering [example][3]. | - [Raycast extension](https://github.com/MassimilianoPasquini97/raycast_ollama)
| [Continue](https://github.com/continuedev/continue) | Embeds Ollama inside Visual Studio Code. The extension lets you highlight code to add to the prompt, ask questions in the sidebar, and generate code inline. | - [Discollama](https://github.com/mxyng/discollama) (Discord bot inside the Ollama discord channel)
| [LiteLLM](https://github.com/BerriAI/litellm) | Lightweight Python package to simplify LLM API calls. | - [Continue](https://github.com/continuedev/continue)
| [Discord AI Bot](https://github.com/mekb-turtle/discord-ai-bot) | Interact with Ollama as a chatbot on Discord. | - [Obsidian Ollama plugin](https://github.com/hinterdupfinger/obsidian-ollama)
| [Raycast Ollama](https://github.com/MassimilianoPasquini97/raycast_ollama) | Raycast extension to use Ollama for local llama inference on Raycast. | - [Dagger Chatbot](https://github.com/samalba/dagger-chatbot)
| [Simple HTML UI](https://github.com/rtcfirefly/ollama-ui) | Also, there is a Chrome extension. | - [LiteLLM](https://github.com/BerriAI/litellm)
| [Ollama-GUI](https://github.com/ollama-interface/Ollama-Gui?tab=readme-ov-file) | 🖥️ Mac Chat Interface ⚡️ | - [Discord AI Bot](https://github.com/mekb-turtle/discord-ai-bot)
| [Emacs client](https://github.com/zweifisch/ollama) | | - [HTML UI](https://github.com/rtcfirefly/ollama-ui)
- [Typescript UI](https://github.com/ollama-interface/Ollama-Gui?tab=readme-ov-file)
[1]: https://python.langchain.com/docs/integrations/llms/ollama - [Dumbar](https://github.com/JerrySievert/Dumbar)
[2]: https://js.langchain.com/docs/modules/model_io/models/llms/integrations/ollama - [Emacs client](https://github.com/zweifisch/ollama)
[3]: https://js.langchain.com/docs/use_cases/question_answering/local_retrieval_qa
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment