Best open-source embedding models for RAG in 2026

Why the choice is hard to reverse

In a retrieval-augmented generation (RAG) setup, the text embedding model turns each document into a vector, and those vectors are what you search at query time. The critical property is that the vector length is not just a number — it is baked into your storage schema, your distance index, and your vector database. That text embedding becomes the basis for semantic search and similarity search across your entire corpus, which is how information retrieval actually works in RAG. Change the embedding model later and the new vectors will not line up with the old ones, which means re-embedding the whole corpus. For a growing knowledge base, that is a real migration, not a drop-in swap.

That is why “best” is the wrong question to lead with. The right question is: which model's dimensions, context and cost fit your corpus, your language mix and your budget — because you are going to live with the answer for a long time.

The three numbers that decide it

• Dimensions. The vector length. More dimensions capture more nuance but cost more to store, compare and retrieve. If your corpus is large, dimension count is a storage and latency budget, not a bragging right.
• Context window. How many tokens a single embedding call accepts. A bigger window lets you embed one large document the way you want, instead of chunking into fixed pieces. Chunking strategy and context window interact more than people expect.
• Cost per token. Embeddings are billed on input tokens only (there is no output rate), so the number is simple: your corpus size × the input price. For a large corpus that is the number that dominates.

The MTEB score is not one of the three. It is a useful signal at the margin (verified figures are below), but it does not tell you how the model behaves on your language mix or your chunking, and its leaderboard position shifts constantly. The three numbers above are what you can act on.

The live open-source options in 2026

Here are the open-source embedding models we serve, with verified facts — dimensions measured by a real call, context windows as stated by the provider (measured at the limit for Qwen3 Embedding 8B). A curated selection, not the whole market; live prices are on the embeddings page.

All three are multilingual and are served by EU-sovereign operators: OVHcloud for BGE-M3, Scaleway for the other two. BGE-M3 has been paused since 20 October 2026, while OVHcloud, its only European provider, is unavailable to us; BGE Multilingual Gemma2 and Qwen3 Embedding 8B stay live. Their licences differ: BGE-M3 is MIT and Qwen3 Embedding 8B is Apache-2.0, while BGE Multilingual Gemma2 carries Google's Gemma Terms of Use — open weights, but not a permissive licence.

What the MTEB scores say, and what they don't

MTEB, the Massive Text Embedding Benchmark, averages a model over dozens of public tasks. Two of the three models above appear in the same table of the Qwen3 Embedding technical report, on MTEB Multilingual:

How to read them:

• Whose numbers these are. They come from the Qwen team's own report, published in June 2025; its model card claimed first place on the MTEB multilingual leaderboard on 5 June 2025. We have not re-run them, and leaderboard positions have moved since.
• The Retrieval column is the one that matters for RAG. On it the gap is about 16 points (70.88 against 54.60), a bigger gap than the one on the overall mean.
• What the gap costs. At the prices in the table above (VAT included), indexing a 100-million-token corpus costs about €1.70 with BGE-M3 (when it is live) and about €16.80 with Qwen3 Embedding 8B. Both are small bills. The real price of the larger model is storage and search speed: 4,096 dimensions take four times the space of 1,024.
• BGE Multilingual Gemma2. Its model card publishes per-task results but no aggregate comparable to the table above, so we do not quote a score for it rather than guess one.
• A benchmark is an average of public datasets. It does not know your documents. The retrieval test on your own queries, described below, still decides.

The multilingual question

If your corpus is not single-language English, multilingual strength matters far more than raw score. BGE-M3 is built for multilingual retrieval and is a famous default (paused on Frontière AI since 20 October 2026). BGE Multilingual Gemma2 trades a larger 3,584-dimension vector for a different multilingual profile. Qwen3 Embedding 8B brings a much larger 40,960-token context window, which changes your chunking freedom more than your dimensional budget.

Here is the practical rule: dimensions are a commitment, context is a convenience, cost is a constant. If your corpus is huge, lean toward lower dimensions to control storage and retrieve faster. If your documents are long or you want chunking flexibility, the larger context window buys you that. Both are legitimate — they are just different optimizations of the same three-number trade.

How embeddings fit the retrieval pipeline

To use these numbers well, it helps to see where the embedding model sits in the retrieval pipeline. RAG works in two phases. At index time you cut each document into chunks, run each chunk through the embedding model to get a vector, and store those vectors in a vector database alongside their text. At query time you embed the user's question the same way and look up the nearest vectors by distance — those are your candidate documents, handed to a chat model as context.

The embedding model is used in both phases, and both use the same three numbers. The dimensions determine the size and shape of your vector index: a 4,096-dimension vector takes four times the storage and slows nearest-neighbour search versus a 1,024-dimension one at the same row count. The context window decides how the chunking works: a small window forces short fixed chunks, while a 40,960-token window lets you embed a long section as a single vector, which can improve recall for documents that are coherent as a whole. And the input cost multiplies by the total number of tokens you index.

One detail frequently missed: the query side also costs input tokens. Embedding a short question is nearly free, but if you re-rank or embed long queries, that cost is real too. None of this changes the decision framework — it just explains why the three numbers are the ones that matter and why changing the model later is so expensive.

Common mistakes that cost recall

Most retrieval problems are not the model's fault — they come from indexing mistakes that survive a model swap. The cheapest wins come from fixing these, not from chasing a slightly better MTEB score.

• Chunking against the context window instead of with it. Arbitrary fixed-size chunks can split a sentence or a coherent idea across two vectors. Recipe: size chunks so that most semantic units fit inside one vector, with a small overlap to catch boundaries. A bigger context window gives you more freedom here.
• Ignoring preprocessing. Headers, boilerplate and navigation text get embedded into vectors and pollute nearest-neighbour results. Stripping them before embedding is cheap and reliably improves recall.
• No query-side care. If you embed the user's raw question verbatim, inconsistencies (“API” vs “api key”) can hurt. Applying the same normalization to queries as to documents closes that gap.
• Judging recall on a toy set. Testing on a hundred clean documents never predicts behaviour on ten million messy ones. Test on a representative sample of your real data, including the ugly corner cases.

None of these requires switching models — they are indexing hygiene. Get them right first, and the model choice becomes a much smaller lever.

How to pick before you index

1. Measure your corpus size and language mix. The numbers that matter are token count and how multilingual it is.
2. Do a small retrieval test with your real queries, not a leaderboard, on a few hundred documents. Recall quality is a property of your data, not of MTEB.
3. Cost it out. Corpus tokens × input price = your indexing bill. At these rates a large corpus is cheap to index; the dimensional budget is what scales with your database.
4. Lock it in. Once the vectors are in your schema, the model is effectively committed. Certainty now beats a marginal MTEB gain later.

Running embeddings on EU-sovereign infrastructure

Vectorizing documents is often the step that most exposes sensitive data — you send whole documents through the model. If those documents must stay under EU jurisdiction, the operator matters more than the model. All three models above are served via EU-owned operators (OVHcloud, Scaleway), so the embedding half of your RAG pipeline is computed under EU control; the gateway that relays your requests runs on Vercel's Paris region, a US sub-processor that stores no prompt or response (see our privacy page). The generation half — the chat model that answers from retrieved context — is the same tool-calling catalog, served sovereign or fast access with the label shown before you call.

Both halves together — embedding on EU infrastructure, generation on EU infrastructure, behind one OpenAI-compatible API — is what lets a team build a RAG pipeline whose inference runs entirely at EU-controlled providers. Billed per token from prepaid credit: if you need millions of embeddings a month, the hosting-vs-API analysis tells you where the break-even sits, and the broader model guide fills in the rest of the catalog.

FAQ