Control AI risks and governance

Hallucination occurs when an AI system presents unsupported or incorrect information as though it were grounded. Data exposure concerns sensitive inputs or outputs reaching unauthorised people or systems through submission, storage, logging, sharing, or subsequent use. Local execution may reduce some external transfers, but access controls, local records, connected tools, and output handling still matter.

Model poisoning targets the data or learning process that shapes model behaviour. Malicious prompts instead try to redirect the system through instructions in input, such as an email, retrieved document, or log field. These are different mechanisms and need different controls. A prompt embedded in an artifact is evidence to analyse, not authority to change the analyst’s task.

Treat retrieved pages, attachments, messages, and tool results as untrusted data. Delimit them clearly and instruct the assistant to analyse their content, but do not rely on wording alone as a security boundary. Restrict tool permissions, isolate sensitive contexts, validate requested actions, and require appropriate approval for changes. A model should not be able to grant itself broader access because a document requests it.

For poisoning risk, govern dataset sources, update processes, feedback channels, and model changes. Track provenance, restrict who may change them, and evaluate behaviour before and after an update. When outputs become systematically unreliable, investigate the pipeline and inputs rather than assuming every error is deliberate poisoning. Distinguish a plausible attack explanation from a confirmed finding.

An AI usage policy should identify approved tools and purposes, permitted data classes, retention expectations, human review requirements, and accountable owners. Establish how staff report mistakes or suspected misuse. Involve the relevant security, privacy, legal, or regulatory advisers when evaluating data handling and obligations; analysts should not infer permission from a vendor’s general marketing statement.

Before adoption, define evaluation criteria and the situations where use must stop or fall back to a manual process. Keep an inventory of integrations and credentials, review access, and document material model or workflow changes. Train analysts to question unsupported claims without rejecting useful assistance. Governance works when it makes the safe next action clear during ordinary casework.