Detect when Ollama falls back to CPU

Ollama doesn't fail when it can't use your GPU. It quietly runs the model on the CPU instead, and replies that took a second now take a minute. Here's how to spot it, and how to get an alert when it happens.

Check by hand: ollama ps

With a model loaded, run ollama ps and look at the PROCESSOR column:

NAME           ID            SIZE     PROCESSOR          UNTIL
llama3.1:8b    46e0c10c039e  6.2 GB   100% GPU           4 minutes from now
qwen2.5:14b    7cdf5a0187d5  11 GB    48%/52% CPU/GPU    4 minutes from now

100% GPU is what you want. A CPU/GPU split means part of the model didn't fit in GPU memory; 100% CPU means the GPU isn't being used at all.

The same numbers come from the API. GET /api/ps lists each loaded model with size (bytes in memory) and size_vram (bytes on the GPU):

curl -s http://127.0.0.1:11434/api/ps | jq '.models[] | {name, gpu_pct: (100 * .size_vram / .size | floor)}'

Common causes

• The model (plus its context) doesn't fit in VRAM. A bigger model, a larger quantisation or a longer context window (num_ctx) all need more memory; Ollama offloads what doesn't fit to the CPU.
• Something else is using the GPU: another model still loaded, a game, a second inference server.
• The driver broke. After a kernel or driver update, or in a container started without GPU access, Ollama can't see the GPU and runs everything on the CPU. The Ollama server log says which GPU, if any, it found at start-up.

Get an alert when it happens

Paw Pulse is a free, push-based monitor for home LLM servers. Its client script runs from cron on the Ollama box, reads /api/ps and reports the lowest GPU share across loaded models as gpu_offload_pct. Set a floor and you get one "slow" alert when it drops below, and one more when it recovers.

1. Set up a monitor and the client: follow the Ollama monitoring guide (no sign-up, about five minutes). Install jq on the box (apt install jq); the GPU check needs it.
2. Set the floor on the monitor:

   curl -X PATCH https://pulse.dougalpaw.com/v1/monitors/<id> \
     -H 'authorization: Bearer <manage_token>' -H 'content-type: application/json' \
     -d '{"min_gpu_pct":90}'

The alert in Discord, Slack, Teams, Telegram or Google Chat reads like:

🟠 home-ollama is SLOW: only 48% of the model is on the GPU (your floor is 90%). It has probably fallen back to CPU.

Things to know

• It only sees loaded models. If nothing is loaded when the script runs, there's no number and no alert. Turn on the speed probe (MEASURE_TPS=1 MEASURE_MODEL=…) to load a model on every beat; the GPU check runs after it.
• Pick the floor to suit your setup. If a model you use on purpose is always split across CPU and GPU, set the floor below its usual share, or the monitor will stay slow.
• Paw Pulse is open source (MIT), needs no account and never sends your prompts, hostname or IP. Read the client script before you run it.