Tests for the Mongo semantics the heuristics calculator does not reproduce - #1762
Merged
jgaleotti merged 1 commit intoSep 16, 2026
Merged
Conversation
LautaroPetaccio
force-pushed
the
tests/mongo-heuristics-semantics-gaps
branch
8 times, most recently
from
September 14, 2026 18:14
d6337fc to
0d9344d
Compare
…oduce Twenty queries where the calculator disagrees with the database, and three neighbouring cases that it already answers correctly, kept as guards so that a fix can be told apart from a regression in the path next to it. They extend the section at the end of MongoHeuristicsCalculatorTest, which was written for exactly this, and follow its convention: every expected value was taken from a MongoDB 7.0.41 server answering the same query against the same document, one test per defect, and @disabled with the reason on the ones that fail today. Where the database refuses the query outright there is no answer to copy, so those tests assert that the calculator answers false without throwing, that being the only answer available to it. Four of them concern values compareNonNullValues has no branch for, being two sub-documents or two binary values, which reach the case for types that cannot be compared. That answers false for $eq, missing the plainest nested-object query there is, and true for $ne and $nin, which is a match the database does not have. On master these threw instead, so this is a step forward that stops short. Six of the queries make the calculator throw, and MongoHandler catches nothing around computeDistanceDocuments, so the exception costs the action its whole ExtraHeuristicsDto, the SQL heuristics included. Two of those six are queries the database answers normally: $regex against an array holding no strings, and $not holding a bare regex.
LautaroPetaccio
force-pushed
the
tests/mongo-heuristics-semantics-gaps
branch
from
September 14, 2026 18:40
0d9344d to
02c4534
Compare
jgaleotti
approved these changes
Sep 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion to the review on #1750, one test per defect noted there. Based on
fix_all_semanticsrather thanmaster, since most of these are about code that only exists on that branch.Twenty-three tests, added to the section at the end of
MongoHeuristicsCalculatorTestthat was written for exactly this. Twenty fail and carry@Disabledwith the reason, three pass and are there as guards. Every expected value came from a MongoDB 7.0.41 server answering the same query against the same document, so the assertions say what the database does rather than what the documentation implies.Queries that make the calculator throw
MongoHandlercatches nothing aroundcomputeDistanceDocuments, so the exception costs the action its wholeExtraHeuristicsDto, the SQL heuristics computed before it included.{a:{$regex:"x"}}vs{a:[1,2,3]}IllegalArgumentException{a:{$bitsAllSet: BinData(0,"Ag==")}}vs{a:2}NullPointerException{a:{$not:/x/}}vs{a:"aa"}NullPointerException{a:{$gt:null}}vs{a:1}, and$gte$lt$lteNullPointerException{a:{$exists:1}}vs{a:1}NullPointerException{a:{$mod:[0,0]}}ArithmeticException{a:{$size:-1}}NullPointerExceptionThe first row is a regression: the code this PR replaces returned
C_FALSEfor it. The last four are queries the database answers or refuses on its own terms, and their selectors are untouched here, so they arrive through the routetestOperatorsThatAreNotModelledDoNotThrowalready documents.$eqand$negiven null answer normally, which is what makes the ordering operators stand out in their own family.Queries that are answered, but not the way the database answers them
{a:{$ne:{x:1}}}vs{a:{x:1}}{a:{$ne:BinData(0,"AQI=")}}vs the same value{a:{$bitsAllSet:[-1]}}vs{a: Long.MIN_VALUE}{a:{$bitsAllSet: 3.9}}vs{a:3}{a:{$bitsAllSet: -1}}vs{a:-1}{a:{x:1}}vs{a:{x:1}}{a:{$eq:BinData(0,"AQI=")}}vs the same value{a:{$all:[1,1]}}vs{a:1}{a:{$all:[null,null]}}vs{b:1}{a:{$all:[{$elemMatch:{$gt:2}}]}}vs{a:[1,5]}{a:{$eq:[1,2]}}vs{a:[[1,2],3]}{a:{$eq:{$comment:"note",x:1}}}vs the same document{a:{$in:[/x/]}}vs{a:"xy"}The rows in bold are the harmful direction, a match the database does not have, which points the search at a condition it can never satisfy.
Two causes account for most of the table.
compareNonNullValueshas no branch for two sub-documents or two binary values, so they reach the case for types that cannot be compared, which answers false for$eqand true for$ne. On master these threw instead, so this is a step forward that stops short. Separately, nothing validates the bitmask itself, while the field side of the same comparison is validated bygetIntegralLongValue.One that is not about MongoDB semantics
computeDistanceDocumentswalks itsIterablethree times, andMongoHandlerpeeks at it once before that. What is passed in today is aFindIterable, which replays, so this is fourfindround trips per action rather than a wrong answer. The test uses anIterablethat cannot be replayed, which the signature currently accepts, and that one is scored as if the collection were empty while still reporting the documents counted on the first pass.The three that pass
One beside each group of defects, covering the neighbouring case that is already handled: a
byte[]bitmask,$commentat the top level, and$allholding a single null. They are there so a fix can be told apart from a regression in the path next to it.Running them
Green as they stand. The class runs 174 tests with 25 skipped, 5 of those pre-existing and 20 new. Dropping the new annotations turns those 20 into failures. The whole
mongopackage is 358 tests, 0 failures, 25 skipped with this branch applied.