You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(joiners): treat non-finite scores as zero when merging documents - #13805
A non-finite input score therefore propagates instead of being neutralised:
NaN poisons the merged score (and, because comparisons with NaN are false, the document keeps its position - sorting becomes insertion-order dependent);
+inf dominates every real document even with a weight of 0.1;
-inf vetoes the document entirely (its merged score is -inf regardless of the other backend).
Scores like these arrive from remote embedding/reranking backends and from user-supplied Document(score=...); they are not scores, and the joiner's contract is to merge comparable relevance values.
before: merged score of the NaN document is NaN; 'non-finite' sorted before 'finite' (0.5)
+inf -> inf, -inf -> -inf
after : non-finite contributions count as 0; finite documents keep their exact merged scores and order
Fix
Weight a non-finite contribution as zero (0.0 if not math.isfinite(score) else score * weight); finite inputs are unchanged.
Tests
test/components/joiners/test_document_joiner.py (NaN, +inf, -inf with weights):
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You have signed the CLA already but the status is still pending? Let us recheck it.
We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.
To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
DocumentJoinermerges the weighted contributions with a plain sum:A non-finite input score therefore propagates instead of being neutralised:
NaNpoisons the merged score (and, because comparisons with NaN are false, the document keeps its position - sorting becomes insertion-order dependent);+infdominates every real document even with a weight of0.1;-infvetoes the document entirely (its merged score is-infregardless of the other backend).Scores like these arrive from remote embedding/reranking backends and from user-supplied
Document(score=...); they are not scores, and the joiner's contract is to merge comparable relevance values.Fix
Weight a non-finite contribution as zero (
0.0 if not math.isfinite(score) else score * weight); finite inputs are unchanged.Tests
test/components/joiners/test_document_joiner.py(NaN, +inf, -inf with weights):