E5 Embeddings Expect 'query:' and 'passage:' Prefixes. Drop Them and Measure Recall in Node

Book: AI That Reads The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub The e5-small-v2 model card has an FAQ entry that asks whether you need to add "query: " and "passage: " to your input texts. The answer is one word and one sentence: "Yes, this is how the model is trained, otherwise you will see a performance degradation." Now open the Transformers.js port of the same model. Its usage example is extractor('This is a simple test.'). No prefix. If you copy that into a Node RAG service, you index every chunk raw, embed every user question raw, and get vectors that look fine. Nothing errors. Retrieval just quietly runs below what the model can do. E5 and nomic-embed-text were both trained to expect a marker on each side. Below is the reason, and a TypeScript script that embeds your own corpus four ways and prints recall@5 for each, so you get a number from your data instead of a claim from mine. What the prefix actually does inside E5 E5 comes from the paper Text Embeddings by Weakly-Supervised Contrastive Pre-training (Wang et al., 2022). The training recipe is contrastive: take a pair of texts that belong together, a query q and a passage d, and push their vectors close while pushing the other passages in the batch away. The pairs come from a dataset the authors built called CCPairs, mined from Reddit post/comment pairs, Stack Exchange question/answer pairs, Wikipedia, paper titles and abstracts, and web pages, filtered down to about 270M pairs, per section 3 of the paper. The detail that matters for your code is in section 4.1. E5 uses one shared encoder for both sides. The authors write that they "break the symmetry by adding two prefix identifiers 'query:' and 'passage:' to q and d respectively," and that "such an asymmetric design turns out to be important for some retrieval tasks where there exist paraphrases of the query in the target corpus." Read that last part slowly. Without a marker, the encoder has no way to know whether a string is a question looking for an answer or a passage that might be one. The text most similar to "how do I rotate an API key" is another sentence that says almost the same thing. In a corpus full of FAQ headings, ticket titles and duplicated questions, a symmetric model happily returns the paraphrase instead of the paragraph that answers it. The prefix tells the model which job each string has, and it learned that job over hundreds of millions of pairs. One shared encoder, two kinds of input. The prefix is the only tag that tells them apart. Section 4.1 also sets the contrastive temperature to 0.01, and the model card's FAQ says that low temperature is why cosine scores cluster between roughly 0.7 and 1.0. So with E5, rank by score and stop trusting fixed thresholds like "anything above 0.8 is relevant." A prefix change moves scores around too. Nomic made the same call, for the same reason The Nomic Embed technical report cites E5 for this exact idea in section 4.2.2. Its explanation is short: "Without prefixes, the model receives conflicting reward signal." Their example is a query like "What is the capital of France?", where a similarity task and a question-answering task disagree about which text should be closest. Nomic went further and trained four prefixes: search_query:, search_document:, classification: and clustering:. The model card states that the input "must include a task instruction prefix." For RAG you want search_document: on chunks and search_query: on questions. Not every model works this way, which is why you read the card each time. BGE small v1.5 puts an instruction on the query side only ("Represent this sentence for searching relevant passages: "), says passages need none, and notes that for v1.5, dropping the instruction "only has a slight degradation." Three models, three conventions. None of them is applied for you by the embedding call. The examples you copy from skip it The defaults in the JavaScript ecosystem embed raw strings. The Xenova/e5-small-v2 card shows an unprefixed call. The Ollama page for nomic-embed-text shows ollama.embed on a plain sentence. LangChain.js's HuggingFaceTransformersEmbeddings lists options for model, batch size, newline stripping and pipeline settings, and nothing for a query or document prefix. Python has a slot for this: Sentence Transformers supports named prompts stored in a model's config, and its docs use multilingual E5 as the example of a model where queries get "query: " and passages get "passage: ". In Node you add the strings yourself. That is a two-line fix, but only if you know it exists. Four ways to embed the same corpus The script needs two files. docs.json is your chunk set, and queries.json is a list of real questions with the ids of the chunks that answer them: [ { "id": "d1", "text": "Rotate keys from Settings > API." }, { "id": "d2", "text": "Refunds post within 5 business days." } ] [ { "text": "how do I rotate an api key", "relevant": ["d1"] } ] Pull the questions from your support inbox or search logs, not from your own head, and use more than a handful. Small sets swing hard: with 20 queries, each one is worth 0.05 of recall. Give every query at least one id in relevant, or the score turns into NaN. Install Transformers.js with npm i @huggingface/transformers (4.3.2 is the latest release at the time of writing). The file is .mts, so Node 22.18+ or 23.6+ runs it directly, since type stripping is on by default from those versions. The script is one file, recall.mts, shown in four parts. Paste them in order into the same file. First comes the model table, where each entry pairs a model id with the prefix strings its card asks for: // recall.mts (run: node recall.mts e5) import { readFile } from 'node:fs/promises'; import { pipeline, layer_norm } from '@huggingface/transformers'; type Doc = { id: string; text: string }; type Query = { text: string; relevant: string[] }; const MODELS = { e5: { id: 'Xenova/e5-small-v2', q: 'query: ', d: 'passage: ', }, nomic: { id: 'nomic-ai/nomic-embed-text-v1.5', q: 'search_query: ', d: 'search_document: ', }, }; const K = 5; const name = (process.argv[2] ?? 'e5') as keyof typeof MODELS; const model = MODELS[name]; if (!model) throw new Error(`unknown model: ${name}`); Then load both JSON files and build the embedder. It runs 32 texts per batch and returns plain number arrays: const docs: Doc[] = JSON.parse( await readFile('docs.json', 'utf8'), ); const queries: Query[] = JSON.parse( await readFile('queries.json', 'utf8'), ); const extractor = await pipeline( 'feature-extraction', model.id, ); async function embed(texts: string[]): Promise { const out: number[][] = []; for (let i = 0; i < texts.length; i += 32) { const batch = texts.slice(i, i + 32); let t = await extractor(batch, { pooling: 'mean' }); // nomic's card applies layer_norm before normalizing if (name === 'nomic') t = layer_norm(t, [t.dims[1]]); out.push(...(t.normalize(2, -1).tolist() as number[][])); } return out; } // vectors are unit length, so dot product = cosine const dot = (a: number[], b: number[]) => a.reduce((s, x, i) => s + x * b[i], 0); For one question, recall@5 is the share of its relevant chunk ids that show up in the top five results. The function averages that over every question: function recallAtK(qv: number[][], dv: number[][]): number { let sum = 0; qv.forEach((q, i) => { const top = dv .map((d, j) => ({ id: docs[j].id, s: dot(q, d) })) .sort((a, b) => b.s - a.s) .slice(0, K) .map((r) => r.id); const rel = queries[i].relevant; const hit = rel.filter((id) => top.includes(id)).length; sum += hit / rel.length; }); return sum / qv.length; } Last, the four variants. Each one embeds both sides with its own prefix pair and prints one line: const VARIANTS = [ { label: 'no prefix', q: '', d: '' }, { label: 'both', q: model.q, d: model.d }, { label: 'swapped', q: model.d, d: model.q }, { label: 'query only', q: model.q, d: '' }, ]; for (const v of VARIANTS) { const dv = await embed(docs.map((d) => v.d + d.text)); const qv = await embed(queries.map((q) => v.q + q.text)); const r = recallAtK(qv, dv).toFixed(3); console.log(`${v.label.padEnd(11)} recall@${K} = ${r}`); } Mean pooling and L2 normalization match the E5 card's reference code. The layer_norm step for nomic comes from the Transformers.js snippet on nomic's card. The first run downloads the model weights from the Hugging Face Hub, so expect a pause. You get four lines, one per variant. I'm not putting numbers here. They depend on your chunks, your questions and your chunk size, and a figure from someone else's corpus tells you very little about yours. Reading the four numbers Same corpus, same questions, four ways of tagging them. The script tells you which sieve your data needs. both is what the model was trained on. Treat it as the baseline everything else is compared to. no prefix is what most copied examples give you. If it lands close to both on your data, your corpus may not contain many paraphrases of its own questions. The card's advice still stands, and a corpus changes as you add to it, so rerun before you decide. swapped is a sanity check. It asks the model to treat questions as passages and passages as questions. If it scores as well as both, look at your eval set before trusting any of the numbers. query only is the half-migrated index. It is what you get when someone fixes the query path in the API handler but never re-embeds the stored chunks. Changing the passage-side prefix means re-indexing, because the stored vectors were computed from different input text. The query side changes at request time, which is why this half-fixed state is so easy to ship. If your task is matching questions to other questions (FAQ dedup, "similar tickets"), add a fifth line: { label: 'query both', q: model.q, d: model.q }. The E5 card recommends "query: " on both sides for symmetric tasks like paraphrase retrieval. Wiring it into your RAG code Once the numbers convince you, the change in a LangChain.js pipeline is small. Wrap the embedder so callers can't forget: type Embedder = { embedQuery(t: string): Promise; embedDocuments(t: string[]): Promise; }; export function withE5Prefixes(e: Embedder): Embedder { return { embedQuery: (t) => e.embedQuery('query: ' + t), embedDocuments: (ts) => e.embedDocuments(ts.map((t) => 'passage: ' + t)), }; } Two things to keep straight. The prefix goes into the text you embed, not the text you store and hand to the LLM, so your chunk content stays clean. And the prefix is a property of the model, so keep it next to the model id in config. Swap E5 for nomic and the strings change; swap to BGE and only the query side gets one. Run the script again whenever you change the model, the chunker or the prefix. Commit queries.json next to the indexer so the check survives the next model swap. If this was useful Prefixes are one of those details that sit between "the RAG demo works" and "retrieval holds up on real questions." AI That Reads is the book in my AI in TypeScript series that covers that ground in Node: embeddings, vector search, and grounding answers in your own documents, so the model answers from the chunks that actually hold the answer. The whole series, from your first LLM call to agents in production, is at xgabriel.com/ai-in-typescript.