Open Weight Models · Lesson 7
Manage models, memory, and privacy
Inspect resource use, troubleshoot common problems, and keep local inference under your control.
What you will learn
- Distinguish downloaded model files from models currently loaded in memory.
- Investigate slow responses and common local service errors with a short, repeatable check.
- Explain the local/cloud boundary and keep the unauthenticated local API on loopback.
Know what is stored and what is running
Use the model name and tag consistently. A downloaded model can remain on disk while no model is running. Conversely, finishing a chat does not necessarily unload its model immediately. Check the two states separately before trying to solve a storage or memory problem.
The following commands have different purposes. The remove command is a reference for deliberate cleanup after the course; do not run it as part of the observation exercise.
| Command | What it tells you or changes |
|---|---|
| ollama list | List models available locally; ollama ls is an alias. |
| ollama ps | List currently loaded models, including their processor allocation. |
| ollama show gemma3:1b | Inspect this model's details. |
| ollama stop gemma3:1b | Unload this model from memory without deleting its local copy. |
| ollama rm gemma3:1b | Delete this local model copy. You may need to download it again to use that name. |
Stop and remove solve different problems
Use stop when you want to release the model's runtime resources. Use rm only when you intend to remove a local model. Removing one name may not reclaim all shared model data used by another named model.
Measure your computer instead of guessing
A model's download size is not its total runtime memory requirement. Context, working memory, other running applications, and concurrent requests also matter. There is no single RAM figure or speed promise that applies to every computer and workload.
A larger context window makes room for more text but increases memory demand. Keep our short exercises at the chosen 4096-token setting rather than increasing it without a reason. When a conversation grows, review what must remain in its history before expanding the window.
Supported GPUs can accelerate inference; compatibility depends on the operating system, hardware, and drivers. CPU execution is also possible. Use ollama ps to see CPU, GPU, or mixed allocation, then consult the current hardware documentation if the observed setup differs from what you expected.
- Compare a first request after unloading with a second request using the same short prompt. Record both times; loading can make the first slower.
- Keep the model, prompt, requested answer length, and other running tasks similar when comparing timings.
- Record responsiveness and answer quality together. A faster incorrect response does not improve your study assistant.
- If memory becomes constrained, close unnecessary applications, unload an unused model, and shorten the test before considering larger hardware or model changes.
Troubleshoot one layer at a time
Start with the exact symptom and a small known request. Check the service before changing the prompt, and check the model name before changing hardware settings. Avoid restarting or deleting unrelated processes and files just to clear an error.
If the message does not explain the failure, consult Ollama's troubleshooting guide for your platform's server logs. Keep the time, command, and relevant error when asking for help. Review logs for private information before sharing them.
| Symptom | A useful next check |
|---|---|
| Connection refused or cannot connect | Run ollama list. Start the Ollama application or the Linux service if stopped. For a manual setup without a running service, use ollama serve in a separate terminal. |
| Port 11434 is already in use | Test ollama list first: an existing Ollama service may already own the port. Do not start a second server. If the test fails, identify the port owner before changing anything. |
| Model not found | Compare the request's exact name and tag with ollama list. For cybercorps-study, check that lesson 6's create command completed. |
| Slow response or timeout | Inspect ollama ps and available resources. Retry a shorter single request after checking load; extend the client timeout only when the service is working but needs longer. |
| Out-of-memory failure | Reduce context/workload and unload unused models. A small disk download does not guarantee enough runtime memory. |
Keep the local boundary clear
Local inference with a downloaded model runs on your machine. Cloud models send work to a remote service, even when accessed through your local Ollama server. Check the selected model and configuration; the word localhost alone is not proof of local inference.
Ollama binds to 127.0.0.1:11434 by default, and its local API does not require authentication. Keep that loopback setup for this course. Do not expose it to your LAN or the internet, add a public tunnel, or broaden its bind address for these exercises.
For an explicit local-only setup, the current Ollama FAQ documents OLLAMA_NO_CLOUD=1 on the server. Apply it using the FAQ's method for your operating system, restart Ollama, and verify that the server log reports cloud features disabled. Setting it only in a client's terminal does not reconfigure an already-running server.
This setting disables Ollama cloud models and web search; it is not a firewall or a promise that the whole computer is offline. Downloads and updates still need separate consideration. Your own client, saved histories, backups, and other software also affect where data goes. Use the fictional notes in this course instead of personal records or credentials.
Create a local operation checklist
Record ollama list and ollama show gemma3:1b. Identify which model names you need for the course.
Send a short request, then inspect ollama ps. Record the model and the processor allocation you actually observe.
After the request finishes, run ollama stop gemma3:1b. Compare ollama ps with ollama list to demonstrate unloaded versus still available.
Time two comparable short requests, noting that the first may reload the model. Record observations without presenting them as universal performance claims.
Document your chosen local model, loopback endpoint, and whether you enabled and verified the documented server-side local-only setting.
Write your first diagnostic step for connection failure, port conflict, and a missing model. Keep model removal as an optional later cleanup decision.
You have completed this task when…
- Your evidence distinguishes disk availability from a model loaded in memory.
- You have observations from your own machine and a practical first check for three common errors.
- Your checklist distinguishes local inference from cloud routing and keeps the unauthenticated API on loopback.
Official documentation
Use these references for platform requirements, current options, and further detail.