Best Vector Database for Local RAG in 2026: What It Costs in RAM

Quick answer

• For a personal RAG corpus the database is not the expensive part. Fifty thousand chunks embedded at 768 dimensions need roughly 188 MiB of RAM once index and overhead are counted — less than the embedding model that produced them.
• Four different kinds of software get called “a vector database”. Choosing the wrong type costs you far more than choosing the wrong product.
• If you run Open WebUI, the choice is already narrow: its documentation lists exactly three external providers — Qdrant, Milvus and pgvector.
• Weaviate is the only one here that is not purely permissive: code in its wl/ directory is available only under a separate enterprise licence.
• Chroma, Open WebUI’s default, has shipped no release since 5 May 2026 despite committing daily.

Sizes on this page use GiB and MiB (1024³ and 1024²), because that is how both the vendor documentation and your operating system report memory. Dimensions for specific models come from our guide to local embedding models for RAG, which sizes the raw vectors; this page is about what the database adds on top of them.

Four different things called “a vector database”

Roundups usually rank these against each other as if they were interchangeable. They are not. An embedded library runs inside your Python process; a server is a separate daemon you operate; an extension lives inside a database you already run; and a raw index library is not a database at all. Decide the shape first, then the product.

Star counts and release dates were read from the GitHub API on 25 September 2026 and will drift. The useful signal is the shape of each row, not the exact number.

What the index actually costs in RAM

Qdrant is the only project here that publishes its sizing arithmetic in full, so its formulas are used below. They are not Qdrant-specific in spirit — every HNSW-based store pays the same three costs — but the constants are Qdrant’s own:

dense_size      = points × dimensions × bytes_per_dim
hnsw_size       = points × m × 2 × 4 bytes × 1.2      # m defaults to 16
id_tracker_size = points × 52 bytes

bytes_per_dim is 4 for float32 (the default), 2 for float16, 1 for uint8 and 0.5 for 4-bit. The docs then advise adding about 20% headroom. Running those formulas over a single-node setup with no replication:

The numbers are reproducible: the same formulas regenerate every worked example in Qdrant’s own documentation exactly, including its 5.72 GiB figure for two million 768-dimension vectors.

The graph everyone worries about is 5% of the bill

At 768 dimensions the HNSW graph costs about 154 bytes per point against 3,072 bytes for the vector itself — roughly 4.7% of the total. Tuning m down to save memory is therefore close to pointless, and it costs recall. The dimension count of your embedding model is the lever that matters, and it is chosen before the database ever sees the data.

Put the other way round: at 768 dimensions you would need about 2.6 million chunks to fill 8 GiB. At 4096 dimensions — a large embedding model — that falls to roughly 518,000. Dimensionality moves the ceiling by 5×; the graph does not.

Quantization is the real lever

Storing vectors at lower precision is where the savings are. The same 250,000-chunk corpus at 768 dimensions:

Qdrant’s documentation attaches an explicit warning to this: “Only use quantization if the resulting search quality is acceptable. Some embedding models don’t quantize efficiently, so verify recall against unquantized results.” Treat the table as a budget, not a free win.

The one licence worth reading

GitHub reports NOASSERTION for two of these projects, and the two cases are opposites.

pgvector is a false alarm. Its licence is the PostgreSQL License — a permissive BSD-style grant that GitHub simply does not auto-detect. There is nothing to worry about.

Weaviate genuinely is split. Its top-level LICENSE states that code “Outside of the ‘wl’ directory… is licensed under the BSD-3-Clause License”, while code inside wl/ “is only available under a separate enterprise license”. It adds: “Enterprise features are enabled via license key. The BSD license does not grant the right to use these features or circumvent any license key.” So without a key the enterprise features stay switched off. What the licence no longer does is promise, in so many words, that only BSD-3 code runs in that case.

Corrected 10 October 2026. When this guide was published on 25 September, wl/LICENSE-WEAVIATE was a five-line placeholder that stated no terms, and the top-level LICENSE said that without a key “only BSD-3-Clause-licensed code is executed”. A commit on 30 September 2026, titled “Update wl license text per legal”, replaced both: the placeholder now carries the enterprise-licence wording quoted above, and the sentence promising BSD-only execution is gone. That is the second change to this mechanism in five weeks, so treat it as a moving target. The same “read the licence, not the README” caution applies as with Open WebUI’s branding clause.

If you run Open WebUI, the decision is mostly made

Open WebUI ships with Chroma. Its own troubleshooting documentation is blunt about the limit: the “default ChromaDB vector database (which uses a local SQLite-backed PersistentClient)” crashes when UVICORN_WORKERS > 1, because of “SQLite being not fork-safe”. Single-worker installs are unaffected; the moment you scale workers, you must move.

And when you do move, the list is short. Open WebUI’s external-knowledge feature supports exactly three providers — “Qdrant, Milvus and pgvector” — with the caveat that the provider’s Python client must already be in your image. That rules out LanceDB and Weaviate for this particular integration regardless of their merits. Our private RAG chatbot guide covers the surrounding setup.

A note on Chroma’s release gap

Chroma is the default in the most popular local AI front end, and it has not cut a release since 5 May 2026 — on GitHub or on PyPI, where chromadb also sits at 1.5.9. That is nearly five months. It is not an abandoned project: the repository was pushed to on 24 September 2026 and carries 29,369 stars. But it had been releasing every week or two before May, so the cadence has plainly changed, and its GitHub description no longer calls it a vector database at all — it now reads “Search infrastructure for AI”. If you depend on Chroma, pin your version and watch it.

How to choose

• One machine, one process, a personal corpus: stay embedded. Chroma or LanceDB. The index is a few hundred megabytes; none of this needs a server.
• You already run PostgreSQL: use pgvector. One fewer daemon, permissive licence, and your vectors sit in the database you are already backing up.
• You want a server and a permissive licence: Qdrant. It also happens to be the only project here that documents its memory arithmetic properly, which is why this page could be written at all.
• You are building the retrieval layer yourself: FAISS — but understand you are adopting an index, not a database, and persistence is your problem.

The honest summary is that for local, single-user RAG, most of this choice does not matter much. Getting the embedding model and chunking right matters considerably more, and the context window you feed the results into will cost you more VRAM than the whole vector index costs RAM.

FAQ

What is the best vector database for local RAG?

For a single machine, an embedded store — Chroma or LanceDB — because a personal corpus needs only a few hundred megabytes and a separate server buys nothing. Choose Qdrant if you want a server with a permissive licence, or pgvector if you already run PostgreSQL.

How much RAM does a vector database need?

Mostly points × dimensions × bytes_per_dim. At 768 dimensions in float32 that is about 3.2 KB per chunk including graph and ID overhead, so 50,000 chunks need roughly 188 MiB and one million need about 3.66 GiB.

Is FAISS a vector database?

No. Its own description calls it “a library for efficient similarity search and clustering of dense vectors”. It provides the index; persistence, filtering, an API and concurrency are left to you.

Is Weaviate open source?

Partly. Code outside the wl/ directory is BSD-3-Clause; code inside it is available only under a separate enterprise licence, and those enterprise features are enabled by a licence key.

Sources

• Qdrant capacity planning documentation — memory formulas, quantization guidance (qdrant.tech)
• Weaviate LICENSE and wl/LICENSE-WEAVIATE (github.com/weaviate/weaviate)
• Open WebUI RAG troubleshooting and retrieval documentation (docs.openwebui.com)
• pgvector LICENSE (github.com/pgvector/pgvector)
• Release versions and star counts from the GitHub API, and chromadb from the PyPI JSON API, both read 25 September 2026

Corrected 10 October 2026: Weaviate changed its licence wording on 30 September 2026, and the section on it now quotes the current text. Updated 25 September 2026: licences, release versions and star counts checked on this date; all memory figures computed from Qdrant’s published formulas.