Open Weight Models · linux · Lesson 7

Manage models, memory, and privacy

Inspect resource use, troubleshoot common problems, and keep local inference under your control.

linux version. Use Linux shell for terminal commands. The exercises and setup instructions below are tailored to this version.

What you will learn

Distinguish downloaded model files from models currently loaded in memory.
Investigate slow responses and common local service errors with a short, repeatable check.
Explain the local/cloud boundary and keep the unauthenticated local API on loopback.

Know what is stored and what is running

Use the model name and tag consistently. A downloaded model can remain on disk while no model is running. Conversely, finishing a chat does not necessarily unload its model immediately. Check the two states separately before trying to solve a storage or memory problem.

The following commands have different purposes. The remove command is a reference for deliberate cleanup after the course; do not run it as part of the observation exercise.

Know what is stored and what is running
Command	What it tells you or changes
ollama list	List models available locally; ollama ls is an alias.
ollama ps	List currently loaded models, including their processor allocation.
ollama show gemma3:1b	Inspect this model's details.
ollama stop gemma3:1b	Unload this model from memory without deleting its local copy.
ollama rm gemma3:1b	Delete this local model copy. You may need to download it again to use that name.

Stop and remove solve different problems

Use stop when you want to release the model's runtime resources. Use rm only when you intend to remove a local model. Removing one name may not reclaim all shared model data used by another named model.

Observe memory and processor use on Linux

Use free -h to inspect system RAM and available memory, then use ollama ps while a model is loaded to observe its CPU/GPU allocation. The free tool is normally supplied by procps/procps-ng; if absent, use your distribution's system monitor or package documentation.

Download size, free RAM, and GPU memory are different measurements. Context and other running work also consume resources. Start with the small course model and short inputs; reduce context or workload when memory is constrained instead of assuming one universal RAM minimum.

For GPU acceleration, check Ollama's current Linux hardware list and the matching vendor-driver instructions. A GPU name alone does not establish driver compatibility. CPU execution is a valid fallback, with performance determined by your machine; verify the actual allocation rather than assuming acceleration worked.

Compare two requests with the same model, prompt, and answer length; note whether the first had to load the model.
Record the actual result and timing alongside the resource observation. Speed does not establish factual accuracy.
Close unnecessary work, shorten a request, or unload an unused model before repeating a constrained test.

Observe resources while the course model is loaded · bash

free -h
ollama ps

Inspect the Linux service or foreground logs

For an installed systemd service, use status and the journal commands below. The journal example limits output to the current boot and the latest 100 entries; sudo is included so you can read the system service log. Preserve the relevant error and time.

For a foreground server on a non-systemd/manual setup, inspect the terminal where you ran ollama serve. systemctl and journalctl are not substitutes for that terminal's output. Diagnose the selected server before changing its model, port, or configuration.

Inspect the Linux service or foreground logs
Symptom	Linux check
Client cannot connect	Check the installed systemd unit or the existing foreground terminal. Start only the selected server if it is stopped.
Address already in use	Try ollama list to identify a responding Ollama server. Investigate the port owner if it fails; do not launch a second copy or kill arbitrary processes.
Unit not found / no systemd	Use the deliberate manual foreground workflow from lesson 2, or follow official service installation instructions. Do not assume every distribution has ollama.service.
Model not found	Compare the exact name against ollama list on this server. Confirm lesson 6 created cybercorps-study and that you did not switch server accounts.
Timeout / insufficient memory	Inspect free -h, ollama ps, and the relevant log. Shorten the request or reduce load before adjusting client timeouts.

Systemd with installed ollama.service: inspect status and logs · bash

systemctl status ollama --no-pager
sudo journalctl -u ollama -b -n 100 --no-pager

Keep inference and the endpoint local

Use the downloaded gemma3:1b base or your cybercorps-study configuration, and retain Ollama's default loopback binding. The local API does not need a key; it should not be exposed to other machines for these exercises.

A local endpoint can route cloud-model requests. To disable Ollama cloud features, apply OLLAMA_NO_CLOUD=1 to the server using the matching method below, then verify its startup log. This is a server setting, not a change to an already-running server from a separate client shell.

Use the instructions for your server mode

Choose the systemd or foreground section below. Keep the same mode and model inventory you used earlier. Disabling cloud features does not disable downloads or make every application on the machine offline.

Systemd service: configure and verify local-only mode

Use this section only for the installed Ollama systemd unit. Finish any inference request first. Open the service override editor, add the shown Environment entry under [Service], preserve unrelated settings, and save and close the editor. The middle block is file content, not a shell command.

Reload unit definitions and restart this service, then inspect its fresh journal entries for the message Ollama cloud disabled: true. Confirm ollama list still answers. If the message is absent, inspect the saved override and startup error before claiming the setting is active.

Systemd only: open the service override · bash

sudo systemctl edit ollama

Add to the service override file · ini

[Service]
Environment="OLLAMA_NO_CLOUD=1"

Systemd only: reload, restart, and verify · bash

sudo systemctl daemon-reload
sudo systemctl restart ollama
systemctl status ollama --no-pager
sudo journalctl -u ollama --since '5 minutes ago' --no-pager
ollama list

Foreground server: set the variable when starting it

Use this alternative for your existing manual foreground setup without a competing service. After requests finish, return to the terminal running your server and stop that specific process with Ctrl+C. Start it again with the command below and leave the terminal open.

Inspect its startup output for Ollama cloud disabled: true, then run ollama list from your client terminal. The variable applies to this server process; record the full startup command for the next session. An address-in-use error means another process still owns the endpoint and needs investigation.

Foreground server only: start with cloud features disabled · bash

OLLAMA_NO_CLOUD=1 ollama serve

Put it into practice

Document Linux operation and local-only mode

Record your chosen server mode and compare ollama list with ollama ps before and after a short gemma3:1b request.
Capture free -h and the observed processor allocation. Stop gemma3:1b with ollama stop, then compare a first and second equivalent request without claiming a universal performance result.
Inspect the systemd journal or your foreground server terminal, according to the mode you actually use. Record a relevant startup entry and where you would look for an error.
Enable OLLAMA_NO_CLOUD=1 using only the matching section. For systemd, save the override, reload, and restart. For foreground mode, stop your own server with Ctrl+C and restart it with the variable.
Verify the cloud-disabled startup message and a successful ollama list. Keep the local model names and loopback endpoint unchanged.
Write a checklist for connection, missing-model, port-conflict, and memory errors, including the correct source of logs for this Linux setup.

You have completed this task when…

Your evidence distinguishes stored models, loaded models, RAM observations, and CPU/GPU allocation.
You can locate the correct Linux log and explain why systemd instructions do not apply to every server setup.
Your actual server reports cloud features disabled, or you document the exact unresolved issue without claiming verification.

Official documentation

Use these references for platform requirements, current options, and further detail.