Reciprocal rank fusion

• RRF needs only positions. A result's score is the sum, over the lists it appears in, of 1 / (k + its rank there); six of the eight engines below set k = 60, Azure's page says only "such as 60", and Qdrant defaults to 2.
• The 60 was fixed in a pilot, and the pilot called it "not critical". Every k its table tried, from 0 to 500, beat the best single system.
• Agreement beats one list's enthusiasm, and score gaps are lost. At k = 60 a result ranked 61st in both lists outscores one ranked first in only one, and a first place won by a mile scores the same as one won by a hair.
• On our benches, fusion never beat the stronger list where one list was much stronger: 58 of 68 claims for keyword search alone against 56 fused. On LongMemEval-S, where the lists were close, fusion beat keyword search alone, 409 to 400 of 470 with one embedding model, which the registered test read as no material difference, and 417 to 400 with another (no registered test). A learned-sparse list added later hit 416 alone and 425 fused with BM25 and nomic-embed-text together, and all five of its registered comparisons read no material difference.

If you're new here: strata→signal is a small workshop that builds on its own machines and writes up what it measures. Every outside claim below is quoted from the paper or documentation named beside it, read on 2026-10-15 (UTC) and read again on 2026-10-29 (UTC), except one paper of May 2026, read on 2026-11-07 and 2026-11-08 (UTC), and one model's card, read on 2026-11-07 (UTC). Our figures are from the published pages named beside them, except the LongMemEval figures for keyword search alone, for nomic-embed-text alone and for the two fusions, and the learned-sparse figures, from a second registered run on the same questions (2026-11-07): both runs' registered records are in this page's data kit, with their sha256s and a README naming the file behind each figure, except the records behind four figures (deep-recall's own code returning the identical top 10 on 499 of 500 questions, the two fused scores at rank 10, the cross-encoder's 432 of 470 and the bench's written tie rule) and the two registrations themselves, which are not published; the nomic-embed-text fusion's figures are also printed on Nemotron 3.5 Rerank 8B, measured on our own workloads.

Three words this page leans on. A ranked list is what a search returns: results in order, best first, usually with a score each. Keyword search here means BM25, the classic formula that scores documents by the query's words they contain. An embedding model turns text into a list of numbers so that texts about the same thing get similar lists; searching by those lists is search by meaning (How an embedding model works).

What it does

Keyword search finds the exact name or number in a question and misses a paraphrase; an embedding model finds the paraphrase and can miss the exact name. Running both and merging their lists covers more of both, but the merge has a problem: a BM25 score has no upper limit, while an embedding model's similarity sits between −1 and 1, so adding or averaging them mixes units. RRF ignores the scores. Each list votes for its results by position, the votes are added, and the totals give the merged order. RuleSage, our board-game rules helper, merges its keyword list and its embedding list this way before a slower model re-reads the shortlist.

The formula, by hand

For each result d, add one term for every list d appears in; rank 1 is the top:

RRF score(d) = sum over lists of 1 / (k + rank of d in that list)

The paper writes the same sum, "where k = 60" (Cormack, Clarke and Büttcher, 2009, §1).

A worked example. Keyword search returns A, B, C in that order. Search by meaning returns B, then C, and not A at all. With k = 60:

The merged list is B, C, A. C was never first anywhere, yet it beats A, which topped the keyword list, because both searches found C and only one found A.

What k does. With k = 0 the same lists give A 1/1 = 1, B 1/2 + 1/1 = 1.5 and C 1/3 + 1/2 ≈ 0.833: now A beats C, because a first place is worth twice a second. A larger k flattens the points, so first place barely beats third and appearing in both lists becomes the main way up. At k = 60, a result 10th in both lists scores 2/70 ≈ 0.028571, well above a lone first place's 1/61 ≈ 0.016393, and a result 62nd in both scores 2/122, exactly 1/61: a dead tie.

Why 60

The paper says: "k = 60 was fixed during a pilot investigation and not altered during subsequent validation." The first of its four pilots fused 30 configurations of one search engine (Wumpus) on topics 351 to 400 of TREC, the US National Institute of Standards and Technology's long-running search evaluation, and "indicated that k = 60 was near-optimal, but that the choice was not critical." In its Table 1, measured in mean average precision (MAP, a 0-to-1 score of how high the relevant documents sit), fusion scored 0.2072 at k = 0, 0.2145 at k = 60, 0.2147 at k = 80 (the table's highest) and 0.2098 at k = 500, against 0.2016 for the best single system.

Its one sentence of reasoning: "while highly-ranked documents are more important, the importance of lower-ranked documents does not vanish as it would were, say, an exponential function used. The constant k mitigates the impact of high rankings by outlier systems."

What the paper does not say. It derives nothing about 60, and for other collections it says only that "similar results were seen" on three more TREC collections. None of its inputs was an embedding search: it fused configurations of one search engine, runs submitted to TREC and the outputs of learning-to-rank methods. The engines' own words differ from the paper's. Azure AI Search: "Experiments show the algorithm performs best when you set k to a small value, such as 60" (Microsoft Learn). MongoDB: "60 is a sensitivity parameter that MongoDB determined" (MongoDB docs).

Where it came from

Two eighteenth-century voting rules. Merging rankings is an election in which the searches vote and the documents stand. Jean-Charles de Borda's count (1781) gives "the top ranked candidate … c points, the second ranked candidate … c−1 points, and so on"; the Marquis de Condorcet's rule (1785) elects "the candidate(s) that beats or ties with every other candidate in a pair-wise comparison". Both lines are summaries, and both originals are cited, in two Dartmouth papers below (Aslam and Montague, 2001, §3.1; Montague and Aslam, 2002, §3.1, references 8 and 9).

Adding up scores, 1993. At TREC-2 (1993-08-31 to 09-02; proceedings March 1994), Edward Fox and Joseph Shaw of Virginia Tech combined five retrieval runs by their scores and named the formulas: CombSUM, the sum of a document's scores, and CombMNZ, that sum times the "Number of Nonzero Similarities", so documents found by more runs rise (Fox and Shaw, Table 6). They also left a warning: "Most of the combinations of two runs performed worse than the better of the two runs while performing better than the poorer of the two runs."

Votes instead of scores, 2001 and 2002. Javed Aslam and Mark Montague turned Borda's count into Borda-fuse: "No relevance scores are required, no training data is required, and the combination algorithm is simple and efficient" (SIGIR 2001, §3.1). Their Condorcet-fuse, which sorts documents by head-to-head majorities, "significantly outperforms Borda-fuse" (CIKM 2002, abstract).

Reciprocal rank fusion, 2009. Gordon Cormack and Charles Clarke of the University of Waterloo and Stefan Büttcher of Google, looking for an unsupervised method, "came up with Reciprocal Rank Fusion (RRF) to serve as a baseline" (SIGIR 2009, pp. 758–759). The baseline won: discounting the first pilot, "RRF outperformed Condorcet all 7 times (p ≈ 0.008), outperformed CombMNZ 6 of 7 times (p ≈ .04)", beating Condorcet, CombMNZ and the best single system "by 4% to 5%" on average. Its tables hold the exceptions: CombMNZ edged it on TREC 3 (MAP 0.4381 against 0.4350) and on the LETOR 3 set (0.6107 against 0.6051, p ≈ .2), and on TREC 9 a best run "derived using a human-in-the-loop" scored 0.3519 against 0.2830.

Where it runs today

Each line is from the engine's own documentation, read on 2026-10-15 (UTC) and again on 2026-10-29 (UTC).

Two things are easy to miss. Not every default is 60: in Qdrant a lone first place (1/2) ties a result both lists placed third and beats one both placed fourth. And Weaviate kept RRF but stopped making it the default, because relative score fusion "retains more information from the original searches than rankedFusion, which only retains the rankings."

What it cannot do

It ignores score gaps. A result that won its list by a mile and one that won by a hair both get 1/61. Because RRF "is a function of ranks, it disregards the distribution of scores and, as such, discards useful information" (Bruch, Gai and Ingber, 2024, §1).

It ties. Two results each first in one list and absent from the other score exactly 1/61, and only the engine's tie rule separates them. Our LongMemEval bench wrote its rule down before any run (equal scores keep file-name order, written for its keyword and vector lists), because the code it reproduced left ties unspecified. Near-ties are fragile too: when that code and our rebuild of it ranked the same 500 questions, one list differed at rank 10, where the fused scores were 0.020282187 and 0.020277778, traced to two embedding similarities about 0.0000001 apart (consistent with rounding in 32-bit arithmetic; not proven).

A list's depth decides who can score. A result beyond a list's cut-off gets nothing from it; Elasticsearch's RRF page says rank_window_size "Defaults to the size parameter", the number of results the request asks for (its RRF retriever reference says 10).

A tuned combination can beat it. Bruch, Gai and Ingber found "that CC outperforms RRF in in-domain and out-of-domain settings" (CC: a weighted sum of the two searches' scores, each rescaled first, whose one weight is tuned, "requiring only a small set of training examples"), and found RRF "to be sensitive to its parameters", against the 2009 pilot's "not critical" (abstract). Weights inside RRF are the cheaper fix; Qdrant says why it added them: "Assigning equal weight to both can let the weaker retriever drag down results."

What it did on our benches

LongMemEval-S, a chat-memory benchmark. LongMemEval (Wu et al., 2024, arXiv:2410.10813) asks 500 questions about long chat histories; in its S version each question comes with 38 to 62 past chat sessions, and the dataset labels the sessions that hold the answer. We rebuilt the search stage of deep-recall, an open-source (MIT) memory tool whose author published LongMemEval figures: a BM25 list and an embedding list from BAAI/bge-small-en-v1.5, each cut at 50, fused by RRF at k = 60 with weight 1.0 on the embedding list and 0.5 on BM25, as deep-recall ships it. Every setting and measure was written down before any score. every@5 counts the 470 answerable questions whose every answer-holding session is in the top 5. Every run here passed its integrity checks, and a second, independently written scorer reproduced every count.

The fusion with a registered test. With nomic-embed-text in place of BGE-small, the embedding list alone scored 403 of 470, and its fusion with the same BM25 list, at the same k and weights, 409, against 400 for BM25 alone (registered on 2026-10-17; that fusion's column in the table below is from the page that reranks that fusion). This pair had its test written in advance: a win needed a gain of at least 10 questions and McNemar's exact two-sided p at or below 0.05 after a Holm adjustment (which raises each p-value to allow for several comparisons tested at once) for the four comparisons the bench registered. It read +9 (23 questions only the fusion hit, 14 only BM25), p 0.1877 before that adjustment: no material difference.

The fusion deep-recall ships, untested. With BGE-small the fusion reached 417 of 470, with no registered test against BM25 alone; its net gain over BM25 alone was 17 questions (the table above has the types; no per-question count of gains and losses was made). The limits: the BGE-small list alone was not scored, and nothing here grades an answer. The +17 here and the +9 above come from different arms run through different code (BGE-small through deep-recall's own embedding code, nomic-embed-text through our own harness); only the +9 was gated by a test written in advance, and no test was registered between the two. Against deep-recall's own figures: its published search figure is 410 of 470, from a run that named files by the dataset's own labels, where ours were blind; and its own code, run unchanged and offline on our files, returned the identical top 10 on 499 of 500 questions, with the same 417. In that same run, deep-recall's local cross-encoder (a small model that reads the question beside each candidate) re-read the fused shortlist plus a rare-word search's hits and reached 432 of 470 (exploratory: not gated, no test). Their own committed results, dated 2026-09-29, give 97.4 % (458 of 470) for a hosted model reranking over files named by the dataset's labels; we have not checked their figure, and neither it nor our 432 (91.9 %) tests the other.

A learned-sparse list on the same questions, run 2026-11-07. Learned sparse retrieval is keyword search with learned weights: a model reads each document once and weights every vocabulary word the document should be found by, contained or not, and a query scores a document by adding, over the query's words, each word's weight in the document times its weight in the query, so the index is still words and weights, not a dense list of numbers. We added one such model, the OpenSearch project's doc-v3-distill (Apache-2.0; documents encoded by the model, questions by a fixed word-weight table with no model call), as a third list on the same 500 questions, run once on a laptop's CPU with the earlier lists read from their records, and fused it by RRF at k = 60 with the weights above (1.0 on each model's list, 0.5 on BM25): with BM25; with nomic-embed-text at equal weights, the sparse-plus-dense hybrid, to ask whether it retrieves worse than sparse alone; and with both. Five comparisons were registered before the run under the rule above, Holm-adjusted across five, each with the words it would print; integrity checks and a second scorer passed on every list. All five read NO MATERIAL DIFFERENCE, which the registration says "is not the same as equal":

(The Holm adjustment, across these five, never lets an adjusted p fall below the one before it in order of size, which is why three rows share 0.4014.)

The first row is the one to read slowly: it met the margin (+16) at an unadjusted p of 0.0139, but the Holm-adjusted p was 0.0693, and the rule written first reads that as no material difference; the reading stands for this set, since a second test on the same 500 questions would not be a replication. The three-way fusion's 425 is the highest count in the run's record, and the comparison carrying it read no material difference; on whether the hybrid is worse than sparse alone, the +6 toward the hybrid is a number, not a verdict. In the equal-weight fusion, 10 of the 500 questions had an exact tie across ranks 5 and 6, the boundary every@5 counts at; the fusion code gives such a tie to the list named first, here the sparse one, and an exploratory recomputation naming the dense list first changed no question's every@5 outcome. The run cannot say anything beyond one English chat-memory dataset; anything about such models as a family (SPLADE, a family of learned-sparse models that encodes the query with a model as well, was to be represented, as a reference only, by naver/splade-v3, which was skipped before any download: its host required accepting a licence first, and its weights ship only as a Python pickle, a format that can run code when loaded); whether other fusion weights would do better (none was tuned); or anything about answers. Texts longer than 510 tokens (word pieces; 54,802 of 125,667 texts) were encoded in consecutive 510-token windows with the maximum kept across them, where the model card's own example cuts at 512; no cutting run was made, so the page cannot say how much this moved the counts.

68 factual claims (published 2026-10-09). Each claim about one historical town searched 2,569 passages; a hit was a right passage in the top 8, meaning a passage that holds the claim's quoted source text word for word, which that page names as keyword search's home ground. BM25 alone found 58, EmbeddingGemma 2 alone 45, nomic-embed-text alone 44; fused with BM25 at k = 60 with equal weights, fixed in advance, EmbeddingGemma 2 reached 56 and nomic 52 (both exploratory). The prediction written before the run, "Fusing BM25 with a dense model beats BM25 (59 or more)", was REFUTED.

316 code searches (published 2026-10-09). Each docstring searched 746 functions for its own; the share with the right function first was 0.547468 for EmbeddingGemma 2 alone (173 of 316), 0.389241 for nomic (123), 0.275316 for BM25 (87), 0.408228 for RRF of BM25 and EmbeddingGemma 2 and 0.389241 for RRF of BM25 and nomic (both exploratory).

In production. RuleSage fuses two or three lists at k = 60, "from the original RRF paper; not retuned", so no fused score can pass the number of lists over 61 (Three librarians and a careful reader).

On the two benches where one list was much stronger, no fused row beat its stronger input: the shape Fox and Shaw reported for two-run combinations in 1993. On LongMemEval-S, where the two lists were close, the fusion with a registered test beat both of its inputs, by less than the margin that test demanded. A later run on the same 500 questions added a learned-sparse list that alone counted 416, above both; its fusions counted 417, 422 and 425, each above it by less than that margin, and all five registered comparisons read no material difference. The 2009 paper fused many runs at once (30 in each pilot); this page cannot say whether that is why its results and ours differ.

An outside reading on the same benchmark

Read on 2026-11-07 and 2026-11-08 (UTC). In May 2026 Sahil Sen, Akhil Kasturi, Elias Lumer, Anmol Gulati and Vamse Kumar Subbiah posted "Is Grep All You Need? How Agent Harnesses Reshape Agentic Search" (arXiv:2605.15184, v1 of 2026-05-14). It measures a different task on LongMemEval: a language model working as an agent (choosing and running tools in a loop) answers a 116-question subset of LongMemEval-S, called representative without saying how it was drawn, and GPT-4o grades the answers. The search tool is grep (a pattern match over the raw text) or vector retrieval (an unnamed embedding model with a reranker), inside four harnesses, the programs that give a model its tools: the authors' own, Claude Code, Codex CLI and Gemini CLI. There is no BM25 arm and no fused arm. With results returned inline, grep scored above vector retrieval in all ten model-harness pairs, by 1.7 to 23.3 points (Claude Opus 4.6 in the authors' harness: 93.1 against 83.6, accuracy in percent; in Claude Code, 76.7 against 75.0, about two questions of 116); with results written to files, vector retrieval led in five of ten. The paper reports no statistical tests or intervals; its second experiment, which varied how much unrelated history each question carried, prints different figures at the full history, the first experiment's own setting, for the same model and harness (Claude Opus 4.6 in Claude Code with grep: 76.7 in the first, 94.0 in the second), without saying why. It says "We do not claim that grep 'beats' vector in general, only that it can win end-to-end under the task distribution and corpora we study", notes that LongMemEval's answers often sit in verbatim spans, which may favour word matching, and leaves some rows pending. For this page: it fused nothing, so it tests no fusion; it grades an agent's answers on 116 questions where our rows count retrieval on all 500 (every@5 over the 470 answerable), so neither ranks the other; and its own summary, that "overall scores still depend strongly on which harness and tool-calling style is used, even when the underlying conversation data are the same", is the part that carries.

What this page does not say

• That RRF is the best fusion for your data: our fused rows are exploratory or untested against their inputs, except the registered comparisons on LongMemEval-S, each of which read no material difference; settling it takes a tuned fusion scored on held-out questions.
• Anything about answers: every figure of ours measures retrieval only.
• That 60 is right or wrong anywhere: the paper fixed it on one pilot, and our benches never varied it.
• Anything about the eight engines beyond their documentation: we ran none of them for this page.
• Anything about learned sparse retrieval beyond one model on one dataset, or about agents and grep as a search tool: the May 2026 paper is quoted, not reproduced.
• What the 1781 and 1785 originals say: we read Aslam and Montague's summaries, not the originals.

What to take with you

• RRF is one line of arithmetic on positions: in our worked example, a result both searches found (0.032002) beat one that topped a single list (0.016393).
• The 60 is a pilot's setting, not a law: every k the 2009 pilot's table tried beat the best single system, and Qdrant defaults to 2.
• It rewards agreement and loses score gaps, which is why Weaviate's default moved to score fusion in v1.24.
• Fusing in a much weaker list did not beat the stronger list on our benches (58 against 56 of 68 claims), and on LongMemEval-S fusion lifted BM25 from 400 to 409 of 470 with one embedding model, under the registered margin of 10, and to 417 with another (no registered test for that pair); a learned-sparse list added later hit 416 alone and 425 fused with BM25 and nomic-embed-text together, and all five of its registered comparisons read no material difference.

How to check our work — and see it live

The worked example needs only a calculator, and the 2009 paper is two pages: read it. The 68-claim and code figures are on the two pages linked above, with their gates and audits. And the live door: ask RuleSage a rules question, free and without an account, and open the wrench on the ruling to see the fused score, how many lists voted and the ceiling it is measured against.

The rest of the seminar

• Three librarians and a careful reader: keyword search, embeddings, fusion and a reranker working together in RuleSage.
• How an embedding model turns anything into a list of numbers: what the meaning list is made of, and where keyword search still wins.
• Four homes for one reranker: the slower model that re-reads the fused shortlist.
• OpenJev as a reranker for RuleSage and Nemotron 3.5 Rerank 8B, measured on our own workloads: two more rerankers re-reading RuleSage's fused list, and, on the second page, the LongMemEval fusion this page cites.
• How the search works: this hub's own search, one BM25 list ranked in your browser, with nothing to fuse.

The whole shelf holds the rest. If there is a fusion comparison you want measured next, say so; the suggestion box is read.

Who ran this, and thanks

This page stands on the authors of every paper cited here, on deep-recall's and LongMemEval's authors, and on the teams who document their engines plainly enough to quote. A small human team asked for it and chose what it would cover; a fleet of AI agents read the sources, computed the examples and drafted it under that team's rulings.

Sources

All read on 2026-10-15 (UTC) and read again on 2026-10-29 (UTC), except the May 2026 paper, read on 2026-11-07 and 2026-11-08 (UTC), and the learned-sparse model's card, read on 2026-11-07 (UTC).

• G. V. Cormack, C. L. A. Clarke, S. Büttcher, "Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods", SIGIR 2009, pp. 758–759, doi:10.1145/1571941.1572114; the authors' copy: https://cormack.uwaterloo.ca/cormacksigir09-rrf.pdf (the earlier address under plg.uwaterloo.ca now redirects there)
• E. A. Fox, J. A. Shaw, "Combination of Multiple Searches", The Second Text REtrieval Conference (TREC-2), NIST Special Publication 500-215 (March 1994), pp. 243–252: https://trec.nist.gov/pubs/trec2/papers/txt/23.txt
• J. A. Aslam, M. Montague, "Models for Metasearch", SIGIR 2001, pp. 276–284, doi:10.1145/383952.384007 (read in the authors' copy on Javed Aslam's university publications page)
• M. Montague, J. A. Aslam, "Condorcet Fusion for Improved Retrieval", CIKM 2002, pp. 538–548, doi:10.1145/584792.584881 (read in the authors' copy on the same page)
• S. Bruch, S. Gai, A. Ingber, "An Analysis of Fusion Functions for Hybrid Retrieval", ACM Transactions on Information Systems 42(1), January 2024 (online 2023), doi:10.1145/3596512; arXiv:2210.11934v2
• D. Wu et al., "LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory", arXiv:2410.10813
• S. Sen, A. Kasturi, E. Lumer, A. Gulati, V. K. Subbiah, "Is Grep All You Need? How Agent Harnesses Reshape Agentic Search", arXiv:2605.15184 (cs.CL), v1 submitted 2026-05-14: https://arxiv.org/abs/2605.15184
• deep-recall (MIT): https://github.com/turlockmike/deep-recall
• Our LongMemEval bench: its registered readings of keyword search alone, of deep-recall's BGE-small fusion, and (2026-10-17) of nomic-embed-text alone and its fusion; the last is printed on https://research.strata2signal.com/nemotron-rerank-8b-on-our-own-workloads/
• Our learned-sparse run (2026-11-07): one registered record of the sparse list (S1) and its three fusions (F1, F2, F3) beside A0, A1, A2 and A5, with a second, independently written scorer's recount of each list and the reading of the five comparisons; in this page's data kit: /reciprocal-rank-fusion/data/. The model: opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill (Apache-2.0), revision babf71f3: https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill
• Engine documentation, linked in the table above: Elasticsearch (the RRF page and the RRF retriever reference), OpenSearch (score ranker processor), Azure AI Search (hybrid search ranking, last updated 2026-06-08), MongoDB ($rankFusion), Milvus (RRF Ranker), Qdrant (hybrid queries), Weaviate (hybrid search concepts), Vespa (phased ranking).