Ollama Local LLM Monitor

A lightweight terminal monitor for Ubuntu/Linux designed for running local LLMs with Ollama, providing real-time metrics for models, CPU, RAM, swap, NVIDIA GPUs, VRAM, temperature, power usage, and LLM performance benchmarks.

Current version: v2.2.5

The project was created to make it easier to see what is actually happening while local models are running, with particular attention to multi-GPU systems, mobile workstations, and NVIDIA eGPU setups.

Main Features
Requirements
Installation
Quick Start
Monitor Panels
Ollama Metrics
NVIDIA GPU Metrics
System Metrics
LLM Benchmark
Real Prompt Metrics with ollama-monitored-run
Automatic Benchmarking
Active Model and STALE State
Cold and Warm Runs
Recommended GPU Comparison Protocol
Remote Ollama
Shared State File
Running Without Installation
Troubleshooting
Known Limitations
Full Changelog
Recommended Workflow
Possible Future Improvements
License
Version

Main Features

The monitor displays the following information in real time:

currently loaded Ollama models;
model size;
amount of the model resident in VRAM;
CPU/GPU residency ratio;
context length;
remaining time before model unload;
quantization and parameter size when available;
NVIDIA GPU utilization;
memory-controller utilization;
VRAM used and total VRAM;
GPU temperature;
power draw and power limit;
P-State;
GPU and memory clocks;
PCI bus ID;
PCIe generation and link width when available;
NVIDIA driver version;
CUDA processes;
VRAM allocated by Ollama processes;
CPU utilization;
CPU temperature;
load average;
RAM usage;
swap usage;
root filesystem usage;
Ollama processes with CPU/RAM/RSS metrics;
LLM benchmark metrics;
TTFT;
prompt processing tok/s;
generation tok/s;
model load duration;
total request duration;
optional integration with blackwell-egpu status.

No third-party Python packages are required.

Requirements

Required

Linux / Ubuntu
Python 3
Ollama

NVIDIA Metrics

The following command must work:

nvidia-smi

Optional

If the following command is available:

blackwell-egpu status

the monitor automatically adds a dedicated eGPU panel.

Installation

Download and extract the current release package:

unzip ollama-monitor-v2.2.5-ubuntu26.zip
cd ollama-monitor-v2.2.5

Install:

chmod +x install.sh
./install.sh

The installer copies:

~/.local/bin/ollama-monitor
~/.local/bin/ollama-monitored-run

If ~/.local/bin is not in your PATH:

echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

Verify the installation:

command -v ollama-monitor
command -v ollama-monitored-run

Expected output:

/home/USER/.local/bin/ollama-monitor
/home/USER/.local/bin/ollama-monitored-run

The current installer also verifies that both commands can actually start:

ollama-monitor --help
ollama-monitored-run --help

Quick Start

Start the monitor:

ollama-monitor

Refresh every 2 seconds:

ollama-monitor -i 2

Quit with:

or:

Ctrl-C

Monitor Panels

The interface is approximately organized as follows:

OLLAMA LOCAL LLM MONITOR v2.2.5

── OLLAMA ───────────────────────────────────────────────
...

── LLM BENCHMARK ────────────────────────────────────────
...

── NVIDIA GPU ───────────────────────────────────────────
...

── BLACKWELL eGPU ───────────────────────────────────────
...

── SYSTEM ───────────────────────────────────────────────
...

── OLLAMA PROCESSES ─────────────────────────────────────
...

Ollama Metrics

The monitor queries:

GET /api/ps

and displays currently resident models.

Example:

MODEL                          SIZE       CPU/GPU       VRAM      CTX    EXPIRES
qwen3.8:27B                 18.0 GiB        25/75%   13.5 GiB     4096      3m 42s
  params=27B  quant=Q4_K_M

CPU/GPU Residency

Ollama provides:

size
size_vram

The monitor calculates:

GPU % = size_vram / size * 100
CPU % = 100 - GPU %

This value describes how much of the model is resident on the GPU compared with the model's total size.

It does not represent the GPU's instantaneous computational utilization.

NVIDIA GPU Metrics

Each NVIDIA GPU visible through nvidia-smi is displayed separately.

Available metrics include:

GPU utilization
Memory-controller utilization
VRAM used / total
Temperature
Power draw / power limit
P-State
SM clock
Memory clock
PCI Bus ID
PCIe generation
PCIe width
Driver version
CUDA processes
Per-process VRAM

Example:

GPU 1  NVIDIA GeForce RTX 5060 Ti

GPU   ███████████████░░░   78%
VRAM  ████████████████░░   13.5 GiB / 16.0 GiB
Temp  63°C
Power 118W / 180W
PCIe  Gen 3 x4

OLLAMA pid=4318  VRAM=13820 MiB

This is particularly useful on multi-GPU and eGPU systems because it shows not only whether a GPU is visible to Linux, but also whether Ollama has actually allocated VRAM on that device.

BLACKWELL eGPU Panel

When blackwell-egpu status is available, the monitor can display additional eGPU-specific information such as:

mode
wait
eGPU enclosure
NVIDIA GPU name
PCIe link speed
PCIe RX / TX
iGPU state

Example:

── BLACKWELL eGPU ────────────────────────────────────────

NVIDIA GeForce RTX 5060 Ti (AORUS RTX506T AI BOX)

  Same device as NVIDIA GPU 0 — GPU/VRAM/Temp/Power shown above
  Mode 3   Wait 0   Link 16GT/s (Gen4)
  PCIe RX/TX 12 / 1019
  iGPU: Integrated GPU

Automatic Metric Deduplication

If blackwell-egpu identifies the same physical GPU already displayed in the NVIDIA GPU panel, the following duplicate values are omitted from the BLACKWELL panel:

GPU utilization
VRAM usage
Temperature
Power

The BLACKWELL-specific data remains visible.

Deduplication is deliberately conservative. Duplicate metrics are hidden only when the helper GPU can be matched unambiguously to exactly one NVIDIA GPU.

If multiple identical GPUs are present and the match is uncertain, the complete BLACKWELL panel remains visible.

System Metrics

The SYSTEM panel displays:

CPU utilization
CPU temperature
Load average
RAM used / total
Swap used / total
Root filesystem usage

Example:

CPU   36.2%
Temp  62°C
Load  4.21 3.55 2.80

RAM   18.5 GiB / 31.8 GiB
SWAP  1.1 GiB / 8.0 GiB
ROOT  47%

LLM Benchmark

Starting with v2, the monitor includes an integrated Ollama benchmark.

The main metrics are:

TTFT
Prompt eval tok/s
Generation tok/s
Prompt tokens
Generated tokens
Prompt duration
Model load duration
Total Ollama duration
Done reason

Example:

── LLM BENCHMARK ─────────────────────────────────────────

qwen3.8:27B   state=done   [monitor benchmark]

TTFT                2.84 s
Prompt eval       114.36 tok/s
Generation         18.72 tok/s
Generated            128 tok
Prompt tokens          31
Prompt duration      271 ms
Model load            73 ms
Total Ollama         9.77 s
Done reason         length

Token/s Calculation

Ollama returns:

prompt_eval_count
prompt_eval_duration
eval_count
eval_duration

The monitor calculates:

Prompt tok/s =
prompt_eval_count / prompt_eval_duration_seconds

and:

Generation tok/s =
eval_count / eval_duration_seconds

The final token-rate values therefore come from Ollama's actual runtime counters rather than from counting streamed HTTP chunks.

TTFT

TTFT means:

Time To First Token

It is measured client-side from the moment the request is sent until the first non-empty output is received.

Starting with v2.1, the monitor considers both:

thinking

and:

response

This is important for reasoning models, which may start producing thinking content before the final visible response.

Manual Benchmark

When a model is loaded, press:

The monitor benchmarks the model currently considered active.

You can also start a benchmark directly:

ollama-monitor --benchmark qwen3.8:27B

With 256 generated tokens:

ollama-monitor \
  --benchmark qwen3.8:27B \
  --benchmark-tokens 256

With a custom benchmark prompt:

ollama-monitor \
  --benchmark qwen3.8:27B \
  --benchmark-tokens 256 \
  --benchmark-prompt "Explain how transformer inference works."

Real Prompt Metrics with ollama-monitored-run

An important limitation of GET /api/ps is that it shows which models are loaded, but it does not expose the timing counters for a request executed by another client.

Therefore, a request performed with:

ollama run llama3.2

does not allow a separate monitoring process to retrospectively recover:

prompt_eval_count
prompt_eval_duration
eval_count
eval_duration

Starting with v2.2, the project includes:

ollama-monitored-run

Use it when you want the monitor to display metrics for the actual prompt you are executing, rather than for a separate synthetic benchmark.

Interactive Usage

Terminal 1:

ollama-monitor

Terminal 2:

ollama-monitored-run llama3.2

Then:

>>> Explain how a RAG pipeline works
...
>>> Now give me an example using SQL Server
...

The monitor will display:

llama3.2:latest   state=done   [actual prompt]

together with metrics for that exact request.

Monitored One-Shot Prompt

You can run a single monitored request:

ollama-monitored-run llama3.2 \
  "Explain how the KV cache works."

For a more repeatable benchmark:

ollama-monitored-run llama3.2 \
  --tokens 256 \
  --temperature 0 \
  "Explain how transformer inference works."

Automatic Benchmarking

Starting with v2.2, automatic benchmarking is enabled by default.

It starts:

when the first active model is detected
when the model considered active changes

Normal startup:

ollama-monitor

Disable automatic benchmarking with:

ollama-monitor --no-auto-benchmark

The explicit option is also available:

ollama-monitor --auto-benchmark-on-model-change

Active Model and STALE State

When several models are resident at the same time, /api/ps does not expose an explicit last used model field.

Starting with v2.1, the monitor watches changes to:

expires_at

to estimate which model was used most recently.

If the displayed benchmark belongs to a different model from the currently active model, it is marked:

STALE

Example:

qwen3.8:27B   state=STALE

Benchmark belongs to qwen3.8:27B,
while the active model is llama3.2:latest.

Press 'b' to benchmark llama3.2:latest.

Cold and Warm Runs

For meaningful comparisons, it is important to distinguish between cold and warm runs.

Cold Run

The model is not resident in RAM/VRAM before the request.

This is useful for measuring:

model load duration
startup latency
initial TTFT

To explicitly unload a model, send a request with:

keep_alive: 0

Then verify:

ollama ps

The model should no longer appear.

Warm Run

The model is already resident in memory.

This is generally the most useful test for measuring steady-state inference performance.

Recommended GPU Comparison Protocol

When comparing CPU-only, an internal GPU, and an eGPU, keep the following conditions unchanged:

model
quantization
prompt
num_ctx
num_predict
temperature
driver
power profile

Recommended settings:

num_predict = 256
temperature = 0

Recommended sequence:

UNLOAD MODEL
     ↓
COLD RUN #1
     ↓
WARM RUN #1
     ↓
WARM RUN #2
     ↓
WARM RUN #3

For warm runs, use the median of the three results whenever possible.

Recommended metrics to record:

TTFT
Prompt tok/s
Generation tok/s
CPU/GPU residency
VRAM
GPU utilization
GPU temperature
Power draw
RAM
Swap
PCIe generation / width
Model load duration
Total request duration

Example Comparison Table

Configuration	TTFT	Prompt tok/s	Gen tok/s	VRAM	GPU %	Temp	Power
CPU only	-	-	-	-	-	-	-
Internal GPU	-	-	-	-	-	-	-
eGPU	-	-	-	-	-	-	-

Remote Ollama

The monitor can connect to an Ollama server running on another host.

ollama-monitor \
  --ollama-url http://192.168.1.50:11434

Benchmark:

ollama-monitor \
  --ollama-url http://192.168.1.50:11434 \
  --benchmark qwen3.8:27B

Real monitored prompt:

ollama-monitored-run \
  --ollama-url http://192.168.1.50:11434 \
  llama3.2

You can also use:

export OLLAMA_HOST=http://192.168.1.50:11434

Shared State File

ollama-monitor and ollama-monitored-run communicate through a small local JSON state file.

When available:

$XDG_RUNTIME_DIR/ollama-monitor-state.json

otherwise:

~/.cache/ollama-monitor/state.json

You can override the location with:

export OLLAMA_MONITOR_STATE=/path/to/state.json

The file only contains the metrics needed to update the benchmark panel.

Running Without Installation

Monitor:

python3 ollama-monitor.py

Wrapper:

python3 ollama-monitored-run llama3.2

Troubleshooting

`ollama-monitor`: command not found

Check:

ls -l ~/.local/bin/ollama-monitor

Then:

export PATH="$HOME/.local/bin:$PATH"

To make it permanent:

echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

`ollama-monitored-run`: command not found

Check:

ls -l ~/.local/bin/ollama-monitored-run

If it is missing, reinstall the current version:

chmod +x install.sh
./install.sh

Then:

command -v ollama-monitored-run

Ollama API Is Not Reachable

Check:

curl http://127.0.0.1:11434/api/ps

and:

ollama ps

`nvidia-smi` Does Not Work

Check directly:

nvidia-smi

The monitor will continue to display Ollama and system metrics, but NVIDIA-specific information will not be available.

PCIe Link Is Not Displayed

The query field names exposed by nvidia-smi may vary between NVIDIA driver versions.

The monitor tries several commonly used variants.

If the driver does not expose the value, it is shown as unavailable.

TTFT = N/A

Starting with v2.1, TTFT detection considers both:

thinking

and:

response

If the value still remains N/A, verify that the request actually produced streamed output and that you are running the latest monitor version.

The Benchmark Belongs to the Previous Model

Starting with v2.1, the monitor may display:

state=STALE

Press:

to benchmark the active model.

Alternatively, leave automatic benchmarking enabled.

"No benchmark has been run"

With versions older than v2.2, this was expected when the prompt was executed through another client.

Starting with v2.2:

ollama-monitor

automatically starts a standard benchmark when it detects the first active model or a model change.

For metrics from the actual prompt, use:

ollama-monitored-run MODEL

`blackwell-egpu` Panel Is Missing

First verify:

blackwell-egpu status

The command should return valid JSON.

Starting with v2.2.2, if the command exists but returns an error, empty output, or invalid JSON, the monitor displays a diagnostic message instead of silently hiding the panel.

Known Limitations

Prompts from External Clients

The monitor cannot retrospectively reconstruct token/s for a request already executed directly by:

ollama run
AnythingLLM
Open WebUI
another API client

unless the request passes through a wrapper or proxy capable of reading Ollama's final response counters.

ollama-monitored-run solves this for terminal requests executed through the wrapper.

A possible future architecture is an Ollama-compatible proxy:

Client
  ↓
Ollama Monitor Proxy
  ↓
Ollama

This would make it possible to collect metrics from external applications as well.

Active Model Identification

When several models are loaded, detecting the most recently used model by observing expires_at is an heuristic.

For exact request metrics, use:

ollama-monitored-run

Token/s During Streaming

HTTP stream chunks do not necessarily correspond one-to-one with model tokens.

For this reason, the final:

Generation tok/s

value is calculated from Ollama's final runtime counters.

Full Changelog

v1 / Initial Version

First version of the monitor.

Added

terminal TUI monitor;
GET /api/ps integration;
currently loaded Ollama model;
model size;
CPU/GPU residency;
model VRAM residency;
context length;
unload countdown;
quantization;
parameter size;
NVIDIA multi-GPU support;
GPU utilization;
memory-controller utilization;
VRAM used / total;
GPU temperature;
power draw / limit;
P-State;
SM clock;
memory clock;
PCI bus ID;
PCIe generation / width;
NVIDIA driver version;
CUDA processes;
per-process VRAM;
CPU utilization;
CPU temperature;
RAM usage;
swap usage;
load average;
root filesystem usage;
Ollama process monitoring;
optional blackwell-egpu status integration;
remote Ollama support;
no external Python dependencies.

v2

Introduced the integrated LLM benchmark.

Added

TTFT;
Prompt eval tok/s;
Generation tok/s;
prompt token count;
generated token count;
prompt evaluation duration;
model load duration;
total Ollama duration;
done reason;
keyboard benchmark with b;
--benchmark MODEL;
--benchmark-tokens;
--benchmark-prompt;
--benchmark-keep-alive.

Changed

Final token/s values are calculated from Ollama's own counters rather than by counting HTTP streaming chunks.

v2.1

Improved handling of multiple simultaneously loaded models.

Added

detection of the most recently active model through expires_at changes;
STALE state when the displayed benchmark belongs to another model;
b now targets the active model rather than the first entry returned by /api/ps;
--auto-benchmark-on-model-change;
TTFT support for reasoning models by considering thinking in addition to response.

Fixed

A possible:

TTFT = N/A

with models that emitted thinking content before the final response.

v2.2

Introduced monitoring of actual user prompts and improved automatic benchmarking.

Added

automatic benchmark enabled by default;
benchmark when the first active model is detected;
automatic benchmark when the active model changes;
--no-auto-benchmark;
ollama-monitored-run;
monitored interactive chat mode;
monitored one-shot mode;
collection of actual /api/chat request metrics;
[actual prompt] state;
shared JSON state file between the wrapper and monitor;
OLLAMA_MONITOR_STATE environment variable.

Clarified

A normal request such as:

ollama run MODEL

does not expose its request counters to a separate monitor process.

To collect exact metrics for the actual prompt, use:

ollama-monitored-run MODEL

or, in a future version, an Ollama-compatible proxy.

v2.2.1

Hotfix for v2.2.

Fixed

detect_active_model() now always returns:

(active_model, changed)

including when:

/api/ps returns no models;
the first model is detected;
resident model state changes.

This fixes:

TypeError: cannot unpack non-iterable NoneType object

v2.2.2

Restored and hardened blackwell-egpu support.

Fixed

In v2.2.1, the optional blackwell-egpu status panel code was accidentally lost while consolidating v2.2/v2.2.1 changes.

As a result, the panel could not be displayed even when the command was installed and working correctly.

Restored

The panel:

── BLACKWELL eGPU ────────────────────────────────────────

can display, when available:

mode
wait
eGPU enclosure name
NVIDIA GPU name
PCIe link
GPU utilization
VRAM used / total
temperature
power draw
PCIe RX / TX
iGPU

Example supported input:

{"mode":3,"wait":0,"egpu":"AORUS RTX506T AI BOX","egpu2":"NVIDIA GeForce RTX 5060 Ti","link":"16GT/s (Gen4)","gpu_util":0,"pwr_curr":21.02,"vram_used":110,"vram_total":16311,"temp":49,"pcie_rx":339,"pcie_tx":317}

The panel no longer filters by mode value. If blackwell-egpu status returns valid JSON, the panel is displayed for mode=3 as well.

Improved Diagnostics

If blackwell-egpu is present in PATH but:

exits with a non-zero return code;
produces no output;
produces invalid JSON;

the monitor displays a diagnostic message instead of silently hiding the panel.

v2.2.3

Added automatic deduplication for the BLACKWELL eGPU panel.

Improved

When blackwell-egpu status identifies the same GPU already displayed in the NVIDIA GPU panel, duplicate metrics are omitted from the BLACKWELL section:

GPU utilization
VRAM used / total
temperature
power

BLACKWELL-specific information remains visible:

mode
wait
eGPU enclosure
PCIe link
PCIe RX / TX
iGPU

Example:

── NVIDIA GPU ────────────────────────────────────────────
GPU 0 NVIDIA GeForce RTX 5060 Ti
  GPU ...
  VRAM ...
  Temp ...
  Power ...

── BLACKWELL eGPU ────────────────────────────────────────
NVIDIA GeForce RTX 5060 Ti (AORUS RTX506T AI BOX)
  Same device as NVIDIA GPU 0 — GPU/VRAM/Temp/Power shown above
  Mode 3  Wait 0  Link 16GT/s (Gen4)
  PCIe RX/TX 12 / 1019
  iGPU: Integrated GPU

Conservative Matching

Deduplication occurs only when the match with an NVIDIA GPU is unambiguous.

The monitor compares the normalized GPU model name and, when necessary, total VRAM.

If several identical NVIDIA GPUs are present and it is not possible to determine which one is the eGPU with confidence, the complete BLACKWELL panel remains visible to avoid hiding useful information.

v2.2.4

Hotfix for v2.2.3.

Fixed

Added the missing:

import re

required by:

normalize_gpu_name()

v2.2.3 could fail with:

NameError: name 're' is not defined

when rendering the BLACKWELL eGPU panel.

Automatic deduplication between NVIDIA GPU and BLACKWELL eGPU remains unchanged.

v2.2.5

Packaging/execution hotfix for v2.2.4.

Fixed

In v2.2.4, import re was accidentally inserted before the Python shebang:

import re
#!/usr/bin/env python3

When the file was launched directly as a command, Linux interpreted it as a shell script and produced errors such as:

import: not found
from: not found
Syntax error: "(" unexpected

v2.2.5 guarantees that the first line is always:

#!/usr/bin/env python3

and that import re is correctly placed inside the Python source.

Installer Verification

install.sh now also runs:

ollama-monitor --help
ollama-monitored-run --help

after installation so that shebang or execution errors are detected immediately.

Current Status

v2.2.5 is the current consolidated release of the monitor.

Recommended Workflow

Terminal 1:

ollama-monitor

Terminal 2:

ollama-monitored-run llama3.2

For benchmark comparisons:

ollama-monitored-run llama3.2 \
  --tokens 256 \
  --temperature 0 \
  "Explain how transformer inference works."

Repeat with other models or GPU configurations while keeping the prompt and test parameters unchanged.

Possible Future Improvements

Possible future work includes:

CSV / JSON benchmark history
time-series charts
GPU/VRAM/temperature/power peak tracking
automatic Cold + Warm benchmark suite
automatic median calculation for warm runs
Ollama-compatible /api/chat and /api/generate proxy
external client support
model comparison history
Markdown / HTML export
AMD ROCm support
Intel GPU support

License

GPL3

Version

Ollama Local LLM Monitor
Current version: 2.2.5
Platform target: Ubuntu/Linux
Runtime: Python 3
Ollama API: /api/ps, /api/generate, /api/chat
GPU monitoring: nvidia-smi

Name		Name	Last commit message	Last commit date
LICENSE		LICENSE
README.md		README.md
README_IT.md		README_IT.md
install.sh		install.sh
ollama-monitor.py		ollama-monitor.py
ollama-monitored-run		ollama-monitored-run

Latest commit

History

Folders and files

Repository files navigation

Ollama Local LLM Monitor

Table of Contents

Main Features

Requirements

Required

NVIDIA Metrics

Optional

Installation

Quick Start

Monitor Panels

Ollama Metrics

CPU/GPU Residency

NVIDIA GPU Metrics

BLACKWELL eGPU Panel

Automatic Metric Deduplication

System Metrics

LLM Benchmark

Token/s Calculation

TTFT

Manual Benchmark

Real Prompt Metrics with ollama-monitored-run

Interactive Usage

Monitored One-Shot Prompt

Automatic Benchmarking

Active Model and STALE State

Cold and Warm Runs

Cold Run

Warm Run

Recommended GPU Comparison Protocol

Example Comparison Table

Remote Ollama

Shared State File

Running Without Installation

Troubleshooting

ollama-monitor: command not found

ollama-monitored-run: command not found

Ollama API Is Not Reachable

nvidia-smi Does Not Work

PCIe Link Is Not Displayed

TTFT = N/A

The Benchmark Belongs to the Previous Model

"No benchmark has been run"

blackwell-egpu Panel Is Missing

Known Limitations

Prompts from External Clients

Active Model Identification

Token/s During Streaming

Full Changelog

v1 / Initial Version

Added

v2

Added

Changed

v2.1

Added

Fixed

v2.2

Added

Clarified

v2.2.1

Fixed

v2.2.2

Fixed

Restored

Improved Diagnostics

v2.2.3

Improved

Conservative Matching

v2.2.4

Fixed

v2.2.5

Fixed

Installer Verification

Current Status

Recommended Workflow

Possible Future Improvements

`ollama-monitor`: command not found

`ollama-monitored-run`: command not found

`nvidia-smi` Does Not Work

`blackwell-egpu` Panel Is Missing