How to configure Ollama on Linux

In this article, we will cover a few dos and don'ts for configuring Ollama on Linux. This assumes a native installation on bare metal, and is likely not applicable to virtual machines or WSL. It also assumes your distro uses systemd.

Installing Ollama

If you are on a distro like Arch Linux, which keeps repositories up to date and has official Ollama packages, I recommend installing Ollama from the distro's repositories. Make sure to install the appropriate version for your hardware, e.g. ollama for CPU inference, ollama-rocm for AMD cards, or ollama-cuda if you're an Nvidia user.

On other distros, simply run the installation script as documented on the Ollama download page.

Checking configuration

You can print the current configuration with the systemctl cat command. For example, on Arch Linux, you may see output similar to:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

$ systemctl cat ollama.service
# /usr/lib/systemd/system/ollama.service
[Unit]
Description=Ollama Service
Wants=network-online.target
After=network.target network-online.target

[Service]
ExecStart=/usr/bin/ollama serve
WorkingDirectory=/var/lib/ollama
Environment="HOME=/var/lib/ollama"
Environment="OLLAMA_MODELS=/var/lib/ollama"
User=ollama
Group=ollama
Restart=on-failure
RestartSec=3
RestartPreventExitStatus=1
Type=simple
PrivateTmp=yes
ProtectSystem=full
ProtectHome=yes

[Install]
WantedBy=multi-user.target

This shows the path to the default systemd service unit, /usr/lib/systemd/system/ollama.service, along with its content. Never edit this file directly, as your changes may be overriden when you update Ollama.

Note that if you add an override file, its contents will be printed after the default unit.

Modifying settings

Modifications to the settings should be made in files ending in .conf, in the directory /etc/systemd/system/ollama.service.d/. You can launch an editor with an appropriate file via the systemctl edit command:

1

$ sudo systemctl edit ollama.service

Make sure you pay attention to the published instructions, as you typically need to place your modifications in the correct location, if you see text like the following:

1
2
3
4

### Editing /etc/systemd/system/ollama.service.d/override.conf
### Anything between here and the comment below will become the contents of the drop-in file

### Edits below this comment will be discarded

Note: you must make your changes on line 3, as any text added below line 4 will be discarded.

Add your desired changes to this file. Note that all changes must be within a section, defined by the appropriate header, as per systemd specifications. You will most likely only want to set environment variables, which go in the Service section. For example, to keep models loaded in memory, you can add the following:

1
2
3
4
5
6

### Editing /etc/systemd/system/ollama.service.d/override.conf
### Anything between here and the comment below will become the contents of the drop-in file
+[Service]
+Environment="OLLAMA_KEEP_ALIVE=-1"

### Edits below this comment will be discarded

Save the file, and exit the editor, and you can refresh the unit database, and restart Ollama.

1
2

$ sudo systemctl daemon-reload
$ sudo systemctl restart ollama.service

The new settings should now be applied.

Troubleshooting

If you do not have an editor configured, you may get an error such as:

1

Cannot edit files, no editor available. Please set either $SYSTEMD_EDITOR, $EDITOR or $VISUAL.

In that case, pass it the name of your preferred editor with the EDITOR variable, e.g. nvim or nano.

1

$ sudo EDITOR=nvim systemctl edit ollama.service

Do ✅

• Check ollama help serve for a list of available configuration variables.
• Check the ollama documentation for extra information.
• Use systemctl edit to edit system files.
• Use systemctl cat to verify your changes.
• Use systemd to run Ollama.

Don't ❌

• Edit the files in /usr/lib/systemd/system/ directly.
• Run ollama serve manually.
• Forget to add the proper section header, e.g. [Service] into the override file.
• Forget to reload the files after editing.