How to Use Ollama with LangChain in Python (2026)

LangChain is the usual way to put a local model behind real application code — chains, retrieval, tool calls — and Ollama is the easiest way to serve that model. This guide covers the current integration package, a working example of each of the two things you will actually use (chat and embeddings), and the settings that cause most of the trouble.

Quick answer

• Install one package: pip install -qU langchain-ollama. It is the dedicated integration package, at version 1.1.0.
• Two classes matter: ChatOllama for generation, OllamaEmbeddings for retrieval.
• Python 3.10 or newer is required (the package declares >=3.10, <4.0).
• Raise the context length before you do anything involving documents. This is the single most common cause of “it ignores my data”.

Before you start

You need Ollama running locally with at least one model pulled. If it isn’t set up yet, our comparison of local LLM runners covers the install, and the Ollama server guide covers running it as a service or in Docker.

ollama pull qwen3:8b
ollama pull nomic-embed-text
pip install -qU langchain-ollama

Not sure which model your machine can hold? The model recommender starts from your RAM or VRAM, and the VRAM calculator sizes a specific model at the context length you intend to use — which, as below, you should raise.

Chat: ChatOllama

The integration follows LangChain’s standard chat-model interface, so anything written against another provider mostly transfers. If you would rather wire the same calls together without writing code, self-hosting n8n with a local model covers the no-code route. Instantiate with a model name, then call invoke():

from langchain_ollama import ChatOllama

llm = ChatOllama(
    model="qwen3:8b",
    temperature=0,
)

messages = [
    ("system", "You are a helpful assistant that translates English to French."),
    ("human", "I love programming."),
]

ai_msg = llm.invoke(messages)
print(ai_msg.content)

The returned AIMessage carries more than text. Its response_metadata includes Ollama’s own timings — total_duration, load_duration, prompt_eval_count and eval_count — and usage_metadata gives input, output and total token counts. Those eval counts are the honest way to measure throughput in your own application rather than trusting a headline figure; our guide to tokens per second explains what they mean and what actually sets the number.

What the integration supports

Per LangChain’s own integration table for ChatOllama: tool calling, structured output, image input, token-level streaming, native async and logprobs are all supported. Audio and video input are not. The table also marks token usage as unsupported even though the example response carries usage_metadata, so treat the counts above as what the integration actually returns rather than a guaranteed feature. Image input means a vision-capable model in Ollama can be driven through this same interface, so reading an image or a scanned document stays inside the same code path as ordinary chat.

Embeddings: OllamaEmbeddings

Retrieval needs a second, much smaller model. The same package provides it:

from langchain_ollama import OllamaEmbeddings

embeddings = OllamaEmbeddings(model="nomic-embed-text")

vector = embeddings.embed_query("Local RAG keeps documents on your own machine.")
vectors = embeddings.embed_documents(["first chunk", "second chunk"])

LangChain’s docs note that vector stores and retrievers call embed_documents() and embed_query() under the hood, so you rarely call them directly — but it is worth knowing which is which, because indexing and querying must use the same embedding model. Vectors from two different models are not comparable, and mixing them degrades retrieval silently rather than erroring. Which embedding model to pick, with sizes and licences, is covered in our guide to local embedding models for RAG.

The setting that breaks most first attempts

Ollama’s default context window is small, and it is not one fixed number. Open WebUI’s documentation explains that Ollama “picks a model’s default context length from your GPU’s VRAM, and on GPUs with less than 24 GiB that is 4096 tokens”. It warns that this “severely limits Retrieval-Augmented Generation (RAG) performance”, because retrieved data “may not be used at all or only partially processed”. The same page suggests raising it to 8192+, and for retrieval work “more than 16000” tokens.

In LangChain you raise it on the model object with num_ctx:

llm = ChatOllama(
    model="qwen3:8b",
    temperature=0,
    num_ctx=8192,
)

A larger context costs memory, and the cost is not trivial — it scales with the number of tokens you allow — LLM context windows explained shows exactly how much. Size it with the VRAM calculator before raising it on a machine that is already close to its limit.

When to use LangChain, and when not to

LangChain earns its complexity when you need composition: retrieval feeding generation, tool calls, structured outputs, swapping providers behind one interface. For a single prompt and a single reply, Ollama’s own API is fewer moving parts, and its OpenAI-compatible endpoint works with existing OpenAI client code.

If what you actually want is to chat with your own documents rather than build an application, a front end will get you there faster: see building a private RAG chatbot, or the Open WebUI setup guide.

FAQ

Which package should I install?

langchain-ollama. LangChain’s documentation states the integration “lives in the langchain-ollama package”, and both ChatOllama and OllamaEmbeddings import from langchain_ollama.

What Python version do I need?

3.10 or newer. The package declares support for >=3.10, <4.0.

Why does my chain ignore the documents I gave it?

Almost always the context window. Ollama derives a default from your GPU’s VRAM — 4096 tokens on cards below 24 GiB — so retrieved passages are truncated before the model ever sees them. Set num_ctx=8192 or higher and try again.

Can I use a vision model through LangChain?

Yes — ChatOllama supports image input. Audio and video input are not supported. For what those vision models can and cannot do on documents, see local OCR with Ollama.

Do I need a GPU?

Not for the embedding model, which is small. The chat model is what benefits. A small model runs on CPU, slowly.

Sources

• LangChain documentation: ChatOllama and OllamaEmbeddings
• PyPI: langchain-ollama — version and Python requirement checked 17 September 2026
• Open WebUI documentation: Retrieval Augmented Generation (Ollama’s default context length)

Updated 17 September 2026. Package versions and API details were read from the projects’ own documentation on that date.