Local OCR with Ollama: Reading Documents Without the Cloud

Sending a scanned contract, an invoice or a passport photo to a cloud OCR service means uploading the document. The same reasoning carries to the next step: once the text is out, local text-to-speech can read it aloud without the document leaving the machine either. A local vision model avoids that entirely: the image never leaves your machine. This guide covers how to read text and structure out of documents with Ollama, which models are actually built for it, what they cost in disk and memory, and — the part most guides skip — where this approach is genuinely worse than a traditional OCR engine.

Quick answer

• The command is one line: ollama run gemma4 ./image.png what is in this image?
• Best documented for documents: IBM’s granite3.2-vision at 2.4GB — its model card says it is “specifically designed for visual document understanding”.
• Smallest: moondream at 1.7GB, if you are tight on memory.
• Important caveat: a vision language model is not a dedicated OCR engine. It can misread and it can invent. For high-stakes text extraction, verify the output.

How to pass an image to Ollama

On the command line, give the file path as part of the prompt. Ollama’s vision documentation gives this exact form in its Quick start:

ollama run gemma4 ./image.png what is in this image?

Worth knowing because it trips people up: the CLI reference documents a second shape, with the path inside the quoted prompt rather than before it — ollama run gemma4 "What's in this image? /Users/jmorgan/Desktop/smile.png". Both are official and both work; the difference is only quoting and word order.

Through the API you send an images array instead. Ollama’s API reference describes it as “a list of base64-encoded images (for multimodal models such as llava)”. Its vision documentation adds a useful distinction: the SDKs accept file paths, URLs or raw bytes, while the REST API expects base64-encoded image data. So in Python you can hand over a path; with curl you must encode it first:

IMG=$(base64 < invoice.jpg | tr -d '\n')

curl -X POST http://localhost:11434/api/chat \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"granite3.2-vision\",
    \"messages\": [{
      \"role\": \"user\",
      \"content\": \"Extract every line item and total as JSON.\",
      \"images\": [\"$IMG\"]
    }],
    \"stream\": false
  }"

If you are calling this from application code rather than a shell, the same capability is available through LangChain’s ChatOllama, which supports image input directly. Extracted text is usually not the end of the job either — it is the input to a search or retrieval system, which is covered in building a private RAG chatbot. For a broader view of what runs locally, start at our local LLM hub.

The models, by size

Download sizes are the default Ollama tags, read from the registry on 17 September 2026. As always, the download is not the memory requirement — a vision model also holds the encoded image while it works.

If any of these will not fit alongside what you already run, size it first with the VRAM calculator, or see what each card class holds on our VRAM tier pages.

Why Granite Vision is the document pick

Most vision models are trained to describe photographs. Documents are a different problem: dense small text, tables, multi-column layouts, charts. IBM’s model card for Granite Vision 3.2 2B states it is “specifically designed for visual document understanding, enabling automated content extraction from tables, charts, infographics, plots” and diagrams.

IBM also publishes benchmark comparisons against other vision models in the 1B–4B range, including OCRBench 0.77 and ChartQA 0.87, both the highest in its comparison table. Those are the vendor’s own reported figures, so treat them as a signal about intent and tier rather than an independent result — BFWR does not run benchmarks.

The practical reading: at 2.4GB it is smaller than most general vision models here and aimed squarely at the job. It is the one to try first for invoices, forms and reports.

Where this is worse than real OCR

This is the honest limit, and it matters more than the model choice.

• A VLM can hallucinate text. A traditional OCR engine that cannot read a character produces garbage or nothing. A language model produces something plausible — a total that looks like a total, an invoice number in the right format. That failure is much harder to spot.
• No confidence scores. Dedicated OCR engines return per-character or per-word confidence you can threshold on. A chat response gives you prose.
• No coordinates. If you need to know where on the page text sits — for redaction, form-filling or highlighting — a VLM does not give you bounding boxes.
• Resolution limits. Dense multi-column pages at low resolution degrade badly. Scan at a higher DPI, or split the page and process it in parts.

Where a VLM wins is on the things traditional OCR is bad at: reading a chart and answering a question about it, pulling structured fields out of an unfamiliar layout, or summarising a page. For “turn this scan into exact text”, conventional OCR remains more reliable — and combining the two, OCR for text and a VLM for interpretation, beats either alone.

Getting structured output

Asking for JSON in the prompt works better when you also constrain the response format. Ollama’s API accepts a format parameter for this, and if you are driving it through LangChain, ChatOllama supports structured output directly. Either way, validate what comes back rather than trusting it — see the hallucination point above.

Running this as a service for several machines or users? The Ollama server and Docker guide covers binding it safely, and self-hosted AI covers what staying local does and doesn’t protect.

FAQ

Can Ollama do OCR?

It can run vision language models that read text from images, which covers many OCR use cases. It is not a dedicated OCR engine and gives no confidence scores or text coordinates.

Which Ollama model is best for documents?

granite3.2-vision (2.4GB) is the one explicitly built for visual document understanding — tables, charts and infographics. Try it before the larger general-purpose models.

How do I send an image on the command line?

Include the file path in the prompt: ollama run gemma4 ./image.png what is in this image?

Why does the model invent numbers that aren’t on the page?

Because it is a language model. When the image is unclear it produces plausible text rather than failing. Raise the scan resolution, crop to the region you care about, and verify anything that matters.

Do I need a GPU?

These models run on CPU, slowly. A 2GB model is usable on modest hardware; the larger ones benefit substantially from a GPU.

Sources

• Ollama documentation: Vision, CLI reference, and the API reference (the images parameter)
• IBM: Granite Vision 3.2 2B model card (document understanding, OCRBench and ChartQA figures)
• Model download sizes read from the Ollama registry on 17 September 2026

Updated 17 September 2026. Benchmark figures are as published by the model’s authors; BFWR does not run its own benchmarks.