Open Weight Models · linux · Lesson 5

Use the local Ollama API

Send chat requests from your terminal and Python, then read and validate the response.

linux version. Use Linux shell for terminal commands. The exercises and setup instructions below are tailored to this version.

What you will learn

Send a non-streaming chat request to your local Ollama service.
Extract an answer from JSON and distinguish an API failure from an inaccurate answer.
Use a Python client with a timeout and explicit conversation history.

From a chat window to a request

An API lets your own program ask the model a question. For this lesson, Ollama must already be running and gemma3:1b must be available from lesson 3. The program runs on the same computer as the service; you do not need an API key for this local endpoint.

Send JSON to http://localhost:11434/api/chat. The model field selects gemma3:1b, and messages contains role/content objects. Set stream to false to receive one complete JSON response instead of processing a stream. The answer is in message.content, not in a top-level response field.

A successful HTTP request only establishes that the service answered. Your application must still check the answer against its task. In our example, the supplied note says the workshop starts at 09:30; a fluent answer giving 10:00 is a content failure even if the request succeeded.

Send JSON from the Linux shell

Keep the server from lesson 2 running and check that gemma3:1b is listed. Paste this POSIX-shell curl command into a client terminal. The final backslash on each continued line must have no following spaces; the single quotes keep the JSON together.

The request uses the local chat endpoint, selects the course model, and requests one response with stream set to false. Read message.content in the returned JSON and compare its answer with the 09:30 note. Save the real response rather than substituting an expected answer.

Linux shell: send a local chat request · bash

curl --fail --show-error --max-time 120 \
  http://localhost:11434/api/chat \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gemma3:1b",
    "stream": false,
    "messages": [
      {"role": "user", "content": "Note: the workshop starts at 09:30. Using only this note, what time does it start?"}
    ]
  }'

Build a small Python client

In your plain-text editor, save the complete Python example below as ~/CyberCorps/open-weight-models/local_chat.py. Preserve its indentation and filename. The client uses the standard library and needs no pip installation.

Open a client terminal in that folder and run python3 local_chat.py. The reusable chat function validates the response, preserves explicit conversation history, and reports request failures. The example should print a first answer and a follow-up; check both against the supplied note.

local_chat.py · python

import json
import sys
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen

URL = "http://localhost:11434/api/chat"


def chat(messages, model="gemma3:1b"):
    payload = {"model": model, "messages": messages, "stream": False}
    request = Request(
        URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    try:
        with urlopen(request, timeout=120) as response:
            result = json.loads(response.read().decode("utf-8"))
    except HTTPError as error:
        detail = error.read(2000).decode("utf-8", errors="replace")
        raise RuntimeError(f"HTTP {error.code}: {detail}") from error
    except URLError as error:
        raise RuntimeError(
            f"Cannot reach local Ollama: {error.reason}. Check the service."
        ) from error
    except TimeoutError as error:
        raise RuntimeError("Ollama did not respond before the timeout.") from error
    except (OSError, ValueError, UnicodeError) as error:
        raise RuntimeError(f"Could not read an Ollama response: {error}") from error

    if not isinstance(result, dict):
        raise RuntimeError("Expected a JSON object from Ollama.")
    if result.get("error"):
        raise RuntimeError(f"Ollama reported: {result['error']}")
    message = result.get("message")
    if not isinstance(message, dict):
        raise RuntimeError("The response has no message object.")
    content = message.get("content")
    if message.get("role") != "assistant" or not isinstance(content, str):
        raise RuntimeError("The response has no valid assistant text.")
    if not content.strip():
        raise RuntimeError("Ollama returned an empty answer.")
    return {"role": "assistant", "content": content}


def main():
    history = [
        {"role": "system", "content": "Use the supplied notes. Do not invent missing facts."},
        {"role": "user", "content": "Note: the workshop starts at 09:30. When does it start?"},
    ]
    try:
        first_answer = chat(history)
        print("First answer:", first_answer["content"])
        history.append(first_answer)
        history.append({"role": "user", "content": "Write that time in words."})
        second_answer = chat(history)
        print("Follow-up:", second_answer["content"])
    except RuntimeError as error:
        print(f"Request failed: {error}", file=sys.stderr)
        return 1
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run the saved Python client on Linux · bash

cd "$HOME/CyberCorps/open-weight-models"
python3 --version
python3 local_chat.py

Your program owns the conversation

Treat each chat request as stateless: the caller supplies the conversation it wants the model to use. The second request above includes the original note, the first assistant answer, and the follow-up. Keeping a model loaded in memory does not give a new request your application's previous messages.

For independent tests, start a new messages list. For a conversation, preserve relevant turns in order and keep the history within your context budget. Do not quietly mix one learner's messages with another's. If you remove older turns, decide which source notes the next answer still needs.

A 400 response can indicate malformed input; a 404 can indicate an unavailable model. Read the error, check the exact model name, and correct the cause. A timeout is not evidence of a bad answer, and an invented fact is not fixed by increasing the timeout.

Keep the example local

Use the downloaded gemma3:1b model and the loopback address shown here. A local Ollama endpoint can also route requests to cloud models when configured to do so; the address alone does not prove where inference happens. Lesson 7 explains the distinction.

Put it into practice

Record a Linux API conversation

Check ollama list in a Linux client terminal and confirm gemma3:1b is available on your chosen server.
Run the curl example. Locate message.content, save the returned JSON, and compare its time with the supplied 09:30 note.
Save local_chat.py in ~/CyberCorps/open-weight-models, enter that folder, and run python3 local_chat.py. Record both answers or the real request error.
Explain the four messages sent for the second Python request: system instruction, original user note, assistant answer, and user follow-up.
Temporarily use course-missing-model in one Python call to observe the reported error. Restore gemma3:1b and rerun the client before continuing.

You have completed this task when…

Your curl request and Python client use the local chat endpoint and the intended model.
The saved evidence distinguishes a failed request from an answer with an incorrect fact.
local_chat.py remains in the project folder with its reusable chat function and original model default restored.

Official documentation

Use these references for platform requirements, current options, and further detail.