Securing a Local LLM: Prompt Injection and the OWASP Top 10

Running a model on your own hardware removes one category of risk — your data stops leaving the building — and people often stop thinking about security there. It does not remove the others. A local model that reads your documents, browses a page or calls a tool has an attack surface, and the most important vulnerability class in it has no known reliable fix. That is not a dramatisation; it is what the standard says.

Quick answer

• Prompt injection is the number one risk in the OWASP Top 10 for LLM Applications, listed as LLM01.
• It cannot be reliably prevented — OWASP says it is “unclear if there are fool-proof methods of prevention”.
• Local does not mean safe. Self-hosting fixes data egress, not injection, output handling or excessive agency.
• Design for containment, not for a filter that catches everything.

What the standard actually lists

OWASP publishes a Top 10 specifically for LLM applications. The current edition is the 2025 list:

Self-hosting addresses part of exactly one of these well — LLM02, and only the “sent to a third party” half of it. The other nine apply to a local deployment essentially unchanged, and two of them (LLM08, vector and embedding weaknesses; LLM06, excessive agency) get more relevant when you build your own retrieval pipeline instead of buying one.

LLM01: the one with no fix

OWASP’s definition is deliberately broad: “A Prompt Injection Vulnerability occurs when user prompts alter the LLM’s behavior or output in unintended ways.” Note what it does not say — it does not require an attacker, or malice. Unintentional triggers count.

The distinction that matters operationally is direct versus indirect. Direct injection is someone typing something into your chat box. Indirect injection is the model ingesting external content — a web page, a PDF, an email — that contains text which changes its behaviour when read. The second is the one that catches local setups, because the whole point of a private RAG system is to feed the model documents you did not write.

And then the sentence that should shape your design:

“Given the stochastic influence at the heart of the way models work, it is unclear if there are fool-proof methods of prevention for prompt injection.”

This is a security standard saying the primary risk in its own list has no guaranteed mitigation. That is unusual, and it has a clear design consequence: stop trying to build a filter that catches every injection, and start limiting what a successful one can do.

Why a document pipeline is an injection surface

If you have built anything that retrieves text and puts it in a prompt, you have built an indirect injection path. The retrieved chunk arrives in the same context window as your instructions, and the model has no reliable way to tell which came from you.

This applies to more of a local stack than people expect:

• Retrieval. Any document a user can add to the index is untrusted input. See building a private RAG chatbot for how that pipeline is assembled.
• Scanned documents and images. Text extracted by a vision model is still text the model will read — and it can carry instructions a human skimming the page would not notice. Our guide to local OCR with Ollama covers that extraction step.
• Web content. Anything fetched at runtime is authored by someone else.
• Tool output. If a tool returns attacker-influenced data, that data re-enters the context.

Containment, since prevention is not on offer

The mitigations OWASP describes are about constraining behaviour and limiting blast radius — privilege control, input and output filtering, and adversarial testing. Translated into a self-hosted setup:

• Give the model the least authority that works. LLM06 is “Excessive Agency” for a reason. A model that can only read cannot be talked into writing. If it must act, put a human confirmation on anything destructive or outward-facing.
• Treat model output as untrusted input. That is LLM05, Improper Output Handling. Never pass generated text into a shell, a SQL query or an HTML page without the escaping you would apply to a form field.
• Bind the inference server to localhost. Ollama’s documentation describes no authentication of its own, and its OpenAI-compatible API key is required but ignored. Whatever sits in front of it is what decides access — see the Ollama server and Docker guide.
• Separate who can add documents from who can query. This is the single highest-value control in a RAG system, and it is an access-control decision rather than an AI one.
• Cap consumption. LLM10 is Unbounded Consumption. On self-hosted hardware the denial-of-service target is your own GPU.

What self-hosting genuinely fixes

It is worth being precise, because the honest case is strong enough without overstating it. Running locally means prompts and documents are not transmitted to a third party, not retained under someone else’s policy, and not subject to a vendor changing terms. For confidential material that is often decisive on its own — see self-hosted AI for the full argument.

What it does not do is make the model harder to manipulate, or make a careless integration safe. A locally hosted model wired to a shell with no confirmation step is more dangerous than a cloud model behind a read-only API, not less.

One more that is easy to overlook: LLM03, Supply Chain. Every model you pull is a binary from the internet. Prefer .safetensors over pickle-based formats — the reasons are covered in checkpoints, LoRAs and VAEs explained — and prefer publishers you can identify. The software around the model counts too: in March 2026, two LiteLLM releases on PyPI carried a credential stealer for about 40 minutes, as our guide to LiteLLM and its alternatives explains.

FAQ

Is a self-hosted LLM more secure?

For data confidentiality, yes — nothing is transmitted to a third party. For prompt injection, output handling and excessive agency, it is exactly as exposed as any other deployment.

Can prompt injection be prevented?

Not reliably. OWASP states it is “unclear if there are fool-proof methods of prevention”. Design so a successful injection has limited consequences.

What is indirect prompt injection?

When the model reads external content — a document, page or file — containing text that changes its behaviour. It is the form that affects retrieval systems, because retrieved text shares a context window with your instructions.

Does Ollama have authentication?

Its documentation describes none, and the OpenAI-compatible API key is required but ignored. Bind it to localhost and put access control in front of it.

What is the OWASP Top 10 for LLMs?

A ten-item risk list for LLM applications, currently the 2025 edition, running from LLM01 Prompt Injection to LLM10 Unbounded Consumption.

Sources

• OWASP: Top 10 for LLM Applications (2025 edition, the ten categories) and LLM01:2025 Prompt Injection (the definition, the direct/indirect distinction and the prevention statement), read 24 September 2026

Updated 24 September 2026. Risk categories and quotations are from the OWASP 2025 list as published on that date. This is an overview for self-hosted deployments, not a substitute for a security review.