Understand open weight models

What you will learn

• Distinguish downloadable model weights from a fully open source AI system.
• Explain inference, training, tokens, context, and quantisation in plain language.
• Identify the licence, hardware, and data-handling checks needed before choosing a model.
• Explain where a local request is processed and how that differs from a cloud request.

What you are downloading

A language model combines a design, called its architecture, with learned numerical parameters called weights. Training adjusts those parameters using examples. Inference uses the resulting model to produce an answer to a new input. In this course you will run an existing model, rather than train one from scratch.

Ollama is the software that loads and runs a compatible model. It is not the model itself. Think of the distinction between a music player and an album: installing the player does not automatically install every album you might want. Likewise, downloading model weights and loading them into memory are separate events.

Typing a correction into a chat can affect the next answer through the conversation context. Ordinary chat does not retrain the model's weights. Fine-tuning is a separate training process; a saved prompt or a Modelfile instruction is not fine-tuning.

Open weights do not settle the licence question

Open weights means the model's learned parameters are made available for people to obtain and use under specified terms. It does not automatically mean that all training data, training code, or unrestricted rights are available. The Open Source Initiative's definition considers freedoms to use, study, modify, and share, together with access to the materials needed for modification.

The runner and the model have separate terms. For our example, Gemma 3, read Google's Gemma Terms of Use and the model card. Check the rules for your intended use and any sharing or hosting. Record the exact model and the terms you reviewed; a familiar family name is not a substitute for checking the specific release.

• Model card: what inputs, tasks, limitations, and evaluations are described?
• Licence and use terms: what conditions apply to using, adapting, or distributing this model?
• Provenance: who published the model, and where did this particular package come from?

Tokens are the model's working units

A tokenizer converts text into tokens: pieces that may represent words, parts of words, punctuation, or other text units. A word count is therefore not a reliable token count. The same sentence can be split differently by different tokenizers.

Context is the working material available for a request, including instructions, relevant conversation, and supplied notes. Space is also needed for generated text. A model's advertised maximum context and the context actually configured in the runner can differ. A long chat is not an unlimited memory, and a larger context does not guarantee that every detail is used correctly.

Budget for storage and working memory

Disk space holds downloaded files. RAM and, where supported, GPU memory hold the model and working data during inference. The download size is not the total memory requirement: the runtime and the context also need space. Longer contexts can increase memory demand.

Quantisation stores numerical values with reduced precision to lower resource requirements. It involves trade-offs that depend on the method, model, and task. A smaller file is useful only if the resulting answers are adequate for your purpose. Parameter count, quantisation, and context length are different properties; none alone describes overall quality.

We use gemma3:1b, a small text-only example, to make the workflow approachable. This choice is not a promise that it will fit every computer or answer difficult questions reliably. Start with short inputs and inspect actual behaviour before considering a larger model.

Know where each request goes

For a local model, inference runs on your computer. A cloud model runs on remote infrastructure and receives the input needed for its request. The Ollama application can support both workflows, so the application's name alone does not establish where a particular request is processed.

This course uses the local gemma3:1b model and synthetic learning material. Downloading the software and model needs network access; local inference itself does not require an Ollama API key. Local processing still leaves you responsible for files, copied answers, integrations, and access to the computer.

Create a model decision card

1. Open the linked Gemma 3 model card, gemma3:1b library entry, and Gemma Terms of Use. Do not install or download a model yet.

2. Write a short card with the exact tag, publisher, supported input type, licence link, and one limitation relevant to summarising notes.

3. Record your computer's operating system, available disk space, installed RAM, and GPU or Apple chip if known. Mark unknown details as unknown instead of guessing.

4. Draw a simple request path: your prompt → local Ollama service → locally loaded model → answer. Draw a separate path for a cloud request.

5. Explain why an advertised download size is insufficient to decide whether inference will fit, and why chatting with a model does not normally update its weights.

Official documentation

Use these references for platform requirements, current options, and further detail.

• Open Source Initiative: Open Source AI Definition 1.0
• Google: Gemma Terms of Use
• Hugging Face: training, architectures, and checkpoints
• Hugging Face: tokenization algorithms
• Hugging Face: quantization overview
• Ollama: context length
• Ollama: local and cloud behaviour
• Google: Gemma 3 model card
• Ollama: gemma3:1b