Best open-source embedding models for RAG in 2026

Why the choice is hard to reverse

In a retrieval-augmented generation (RAG) setup, the text embedding model turns each document into a vector, and those vectors are what you search at query time. The critical property is that the vector length is not just a number — it is baked into your storage schema, your distance index, and your vector database. That text embedding becomes the basis for semantic search and similarity search across your entire corpus, which is how information retrieval actually works in RAG. Change the embedding model later and the new vectors will not line up with the old ones, which means re-embedding the whole corpus. For a growing knowledge base, that is a real migration, not a drop-in swap.

That is why “best” is the wrong question to lead with. The right question is: which model's dimensions, context and cost fit your corpus, your language mix and your budget — because you are going to live with the answer for a long time.

The three numbers that decide it

• Dimensions. The vector length. More dimensions capture more nuance but cost more to store, compare and retrieve. If your corpus is large, dimension count is a storage and latency budget, not a bragging right.
• Context window. How many tokens a single embedding call accepts. A bigger window lets you embed one large document the way you want, instead of chunking into fixed pieces. Chunking strategy and context window interact more than people expect.
• Cost per token. Embeddings are billed on input tokens only (there is no output rate), so the number is simple: your corpus size × the input price. For a large corpus that is the number that dominates.

Notice what is absent: MTEB score. It is a useful signal at the margin, but it does not tell you how the model behaves on your language mix or your chunking, and its leaderboard position shifts constantly. The three numbers above are what you can act on.

The live open-source options in 2026

Here are the open-source embedding models we serve, with the facts we measured by a real call — dimensions and context verified, not read off a spec sheet. A curated selection, not the whole market.

Model	Dimensions	Context	Input €/100k (client)
BGE-M3 (multilingual, BAAI)	1,024	8,192	€0.0013
BGE Multilingual Gemma2 (BAAI)	3,584	8,192	€0.0013
Qwen3 Embedding 8B	4,096	40,960	€0.014

All three are multilingual and run on EU-sovereign operators (OVHcloud and Scaleway).

The multilingual question

If your corpus is not single-language English, multilingual strength matters far more than raw score. BGE-M3 is built for multilingual retrieval and is a famous default. BGE Multilingual Gemma2 trades a larger 3,584-dimension vector for a different multilingual profile. Qwen3 Embedding 8B brings a much larger 40,960-token context window, which changes your chunking freedom more than your dimensional budget.

Here is the practical rule: dimensions are a commitment, context is a convenience, cost is a constant. If your corpus is huge, lean toward lower dimensions to control storage and retrieve faster. If your documents are long or you want chunking flexibility, the larger context window buys you that. Both are legitimate — they are just different optimizations of the same three-number trade.

How embeddings fit the retrieval pipeline

To use these numbers well, it helps to see where the embedding model sits in the retrieval pipeline. RAG works in two phases. At index time you cut each document into chunks, run each chunk through the embedding model to get a vector, and store those vectors in a vector database alongside their text. At query time you embed the user's question the same way and look up the nearest vectors by distance — those are your candidate documents, handed to a chat model as context.

The embedding model is used in both phases, and both use the same three numbers. The dimensions determine the size and shape of your vector index: a 4,096-dimension vector takes four times the storage and slows nearest-neighbour search versus a 1,024-dimension one at the same row count. The context window decides how the chunking works: a small window forces short fixed chunks, while a 40,960-token window lets you embed a long section as a single vector, which can improve recall for documents that are coherent as a whole. And the input cost multiplies by the total number of tokens you index.

One detail frequently missed: the query side also costs input tokens. Embedding a short question is nearly free, but if you re-rank or embed long queries, that cost is real too. None of this changes the decision framework — it just explains why the three numbers are the ones that matter and why changing the model later is so expensive.

Common mistakes that cost recall

Most retrieval problems are not the model's fault — they come from indexing mistakes that survive a model swap. The cheapest wins come from fixing these, not from chasing a slightly better MTEB score.

• Chunking against the context window instead of with it. Arbitrary fixed-size chunks can split a sentence or a coherent idea across two vectors. Recipe: size chunks so that most semantic units fit inside one vector, with a small overlap to catch boundaries. A bigger context window gives you more freedom here.
• Ignoring preprocessing. Headers, boilerplate and navigation text get embedded into vectors and pollute nearest-neighbour results. Stripping them before embedding is cheap and reliably improves recall.
• No query-side care. If you embed the user's raw question verbatim, inconsistencies (“API” vs “api key”) can hurt. Applying the same normalization to queries as to documents closes that gap.
• Judging recall on a toy set. Testing on a hundred clean documents never predicts behaviour on ten million messy ones. Test on a representative sample of your real data, including the ugly corner cases.

None of these requires switching models — they are indexing hygiene. Get them right first, and the model choice becomes a much smaller lever.

How to pick before you index

1. Measure your corpus size and language mix. The numbers that matter are token count and how multilingual it is.
2. Do a small retrieval test with your real queries, not a leaderboard, on a few hundred documents. Recall quality is a property of your data, not of MTEB.
3. Cost it out. Corpus tokens × input price = your indexing bill. At these rates a large corpus is cheap to index; the dimensional budget is what scales with your database.
4. Lock it in. Once the vectors are in your schema, the model is effectively committed. Certainty now beats a marginal MTEB gain later.

Running embeddings on EU-sovereign infrastructure

Vectorizing documents is often the step that most exposes sensitive data — you send whole documents through the model. If those documents must stay under EU jurisdiction, the operator matters more than the model. All three models above are served via EU-owned operators (OVHcloud, Scaleway), so the embedding half of your RAG pipeline never leaves EU control. The generation half — the chat model that answers from retrieved context — is the same tool-calling catalog, served sovereign or fast access with the label shown before you call.

Both halves together — embedding on EU infrastructure, generation on EU infrastructure, behind one OpenAI-compatible API — is what lets a team build a fully sovereign RAG pipeline. Billed per token from prepaid credit: if you need millions of embeddings a month, the hosting-vs-API analysis tells you where the break-even sits, and the broader model guide fills in the rest of the catalog.

FAQ