test: the inequality scan covered two operators of ten (#1030)

CHANGELOG.md

@@ -203,6 +203,69 @@ true until the next version shipped.
 
   The gate's printed recipe said `<log>`, singular, so following it exactly produced
   the broken row. It now names one log per gated major and says why.
+- The hand-rolled-inequality scan covered two operators of ten, so its scanned
+  class was empty while 27 live sites sat outside it (#1030).
+
+  `test_layer.py`'s `_hand_rolled_inequalities` refuses `int(<Compare>)` passed to
+  an `expect` call -- the idiom that throws both values away. Its docstring said
+  exactly that. The code required an `Eq` or a `NotEq`:
+
+      and any(isinstance(op, (ast.NotEq, ast.Eq)) for op in arg.args[0].ops)
+
+  So `int(a != b)` was refused and `int("x" in got)` was not. A name that outran
+  its content, and the corpus reported clean because the shapes it actually used
+  were the ones the scan could not see.
+
+  Widened to any comparison, and to ANY boolean combination, which is the worse
+  case: `int(a and b)` cannot say which half was false. The operator list is NAMED
+  and pinned against `ast` itself, so a future operator reddens an arm rather than
+  silently narrowing the rule.
+
+  THE FIRST ATTEMPT AT THE BOOLEAN HALF INHERITED THE BUG IT WAS FIXING. It
+  required a comparison inside the BoolOp, which left
+  `int(p.exists() and q.exists())` live -- a PREMISE arm about a pair, whose whole
+  job is to say which half is missing, reporting `got 0 want 1`. The operator
+  widening fixed the comparison half completely and the boolean half kept the
+  original narrowing. Reported by @jdatcmd, who probed the boundary rather than
+  reading the branch. A single truthiness is still honest: `int(p.exists())` has
+  one value and nothing to disambiguate; it is the COMBINING that loses the answer.
+
+  THE POPULATION WAS 28, NOT THE 7 THE ISSUE MEASURED -- main moved between the
+  measurement and the fix. Twenty-one `in`, five `> 0`, two boolean pairs, across
+  eight files.
+
+  `Expect.contains(got, want, name, absent=False)` is new, because 21 sites of one
+  shape is a missing word in the vocabulary rather than 21 local mistakes. It
+  reports what was actually there:
+
+      collapsed : how-to names clustering: got 0 want 1
+      contains  : how-to names clustering: 'cluster' is absent from 'this document
+                  talks about join keys and nothing else at all'
+
+  Its parameters are `got` and `want` deliberately: `test_failed_query_sentinel.py`
+  partitions the layer's assertions by their first two parameter names, so any
+  other spelling would have put it in neither bucket and opened the silent hole
+  that file exists to refuse. It is registered in that file's shape table, so the
+  failed-query sentinel sweep covers it like every other comparison.
+
+  The five `int(len(x) > 0)` sites became `at_least`, which reports the number. The
+  boolean pair became two `at_least` arms, each naming its own half.
+
+  Removal proof, both directions:
+
+      control                                        44 passed
+      one collapsed `in` site put back               the sweep FAILS
+      the same site, with the OLD Eq/NotEq scanner   the sweep PASSES
+
+  The third line is the finding: the old scan reports a clean corpus with the
+  collapsed site still in it.
+
+  Eleven false-positive arms, five of them real `int()` calls from this corpus --
+  a parsed regex group, a driver flag, a path premise, a value `num()` would refuse
+  as a string, and a single call, which has one value and nothing to disambiguate.
+
+  Guard half 347 passed, 913 checks; the two cluster-side files touched, 22 passed,
+  79 checks. `guard_tests` re-derived by collection, 346 -> 347.
 
 - Four secret-leak claims over the PG server log could pass having read nothing
   (#1032).

test/pytest/TESTS.md

@@ -361,7 +361,8 @@ the general case was hand-rolled.
 | `test_differ_refuses_a_failed_query_on_either_side` | a failed arm is refused, left and right |
 | `test_differ_refuses_two_failed_queries` | **the inverse of #930's trap**; see below |
 | `test_the_inequality_scan_finds_a_planted_offence` | the AST scan fires on both spellings |
-| `test_the_inequality_scan_does_not_flag_honest_code` | five shapes it must not flag |
+| `test_the_inequality_scan_does_not_flag_honest_code` | eleven shapes it must not flag, five of them real int() calls from this corpus |
+| `test_the_operator_list_is_what_ast_offers` | #1030, `_COMPARE_OPS` pinned against `ast`, so a new operator cannot narrow the rule |
 | `test_no_test_in_this_corpus_hand_rolls_an_inequality` | the population is zero, across 17 files |
 | `test_a_conftest_cannot_switch_off_the_order_collapse_scan` | #924, the route still open after #958 |
 | `test_a_conftest_cannot_switch_off_the_broad_except_scan` | the same hatch, a second scan |

test/pytest/expected_tests.txt

@@ -173,7 +173,7 @@
 # 361 here, which is a coincidence of this merge rather than a method: the deltas were
 # measured against different trees. Re-derived by collection on the merged tree, which
 # is the only resolution this number has.
-guard_tests 361
+guard_tests 362
 
 # The complement: tests that need the driver and a throwaway cluster. Until #1016 these ran
 # in no CI job at all -- a quarter of the corpus, green when somebody ran them by hand and

test/pytest/pgc_vacuity.py

@@ -716,6 +716,49 @@ def text(self, got, want, name):
         if got != want:
             raise AssertionError(f"{name}: got {got!r} want {want!r}")
 
+    # -- containment -------------------------------------------------------
+    #
+    # A MISSING WORD IN THE VOCABULARY, not 21 local mistakes (#1030). Every site
+    # that wanted this wrote `expect.num(int(needle in hay), 1, name)`, which throws
+    # BOTH values away: the failure reads `got 0 want 1`, and a reader cannot tell a
+    # haystack that was EMPTY from one that was WRONG. That is the reason `differ`
+    # exists, applied to containment.
+    @_resolving
+    def contains(self, got, want, name, *, absent=False):
+        """Assert `want in got`, SHOWING `got` when it is not there.
+
+        `got` is the haystack and `want` the text sought in it. The names are the
+        layer's own, deliberately: `test_failed_query_sentinel.py` partitions the
+        assertions by their FIRST TWO PARAMETER NAMES, so calling these anything
+        else would put this method in neither bucket and open the silent hole that
+        file exists to refuse. It is a caller-supplied value on the left, so a
+        failed-query sentinel can arrive in it and must be refused like any other.
+
+        `absent=True` asserts the opposite and reports WHERE it was found, because
+        "it is present" is not useful without "here".
+        """
+        self._refuse_failed_query(name, got, want)
+        if _empty(want):
+            raise VacuityError(
+                f"{name}: the text sought is empty, so every value contains it."
+            )
+        # An empty haystack satisfies an absence claim without testing anything,
+        # which is the same vacuity `row_set` refuses without an explicit flag.
+        if absent and _empty(got):
+            raise VacuityError(
+                f"{name}: the value searched is empty, so nothing could have been "
+                f"found in it and this could not have failed."
+            )
+        self._record(name)
+        shown = got if len(got) <= 300 else got[:300] + "...[clipped]"
+        if absent and want in got:
+            raise AssertionError(
+                f"{name}: {want!r} is present at offset {got.index(want)} and "
+                f"should not be, in {shown!r}"
+            )
+        if not absent and want not in got:
+            raise AssertionError(f"{name}: {want!r} is absent from {shown!r}")
+
     # -- SQLSTATE ----------------------------------------------------------
     @_resolving
     def sqlstate(self, exc, want, name):

test/pytest/test_compare_to_bash.py

@@ -274,22 +274,25 @@ def test_the_loop_reader_invents_nothing_in_this_corpus(expect):
     expect.text(", ".join(_loop_names(ast.parse(split))), "one two three",
                 "premise: the reader joins a wrapped name, which is why the relaxation "
                 "is needed at all")
-    expect.num(int("one two three" in split), 0,
-               "and the raw source does NOT contain it, so the old predicate called a "
-               "wrapped name fabricated")
-    expect.num(int("one two three" in _joined(split)), 1,
-               "collapsing the file's own concatenation finds it")
+    expect.contains(
+        split, "one two three",
+        "and the raw source does NOT contain it, so the old predicate called a "
+        "wrapped name fabricated", absent=True)
+    expect.contains(
+        _joined(split), "one two three",
+        "collapsing the file's own concatenation finds it")
 
     built = ('P = "two"\n'
              'for label, sql in ((f"one {P} three", "q"),):\n'
              '    expect.num(g, 1, label)\n')
     expect.text(", ".join(_loop_names(ast.parse(built))), "one {} three",
                 "premise: an f-string name is read as a TEMPLATE, not refused")
-    expect.num(int("one {} three" in _joined(built)), 0,
-               "and collapsing the concatenation does NOT rescue it -- a template is "
-               "constructed, not found, so the relaxation keeps the guarantee it was "
-               "relaxed from. A port that writes an f-string loop name reddens this arm "
-               "by name, which is the designed outcome and not a new one")
+    expect.contains(
+        _joined(built), "one {} three",
+        "and collapsing the concatenation does NOT rescue it -- a template is "
+        "constructed, not found, so the relaxation keeps the guarantee it was "
+        "relaxed from. A port that writes an f-string loop name reddens this arm "
+        "by name, which is the designed outcome and not a new one", absent=True)
 
 
 # A port that parametrises a family its bash twin unrolls, which is the whole of
@@ -503,13 +506,16 @@ def test_a_parametrized_name_is_resolved_from_the_decorator(expect):
         '    expect.num(got, 1, "an unrelated property")\n'
     )
     got = _names(src)
-    expect.num(int("a role with only schema USAGE is refused" in got), 1,
-               "a parametrized name is resolved through the module-level constant")
-    expect.num(int("and is refused reconstruct" in got), 1, "for every row of it")
-    expect.num(int("read_projection" in got), 0,
-               "while the OTHER column of the same decorator is not a name")
-    expect.num(int("reconstruct" in got), 0,
-               "and a parametrize with no name column contributes nothing")
+    expect.contains(
+        got, "a role with only schema USAGE is refused",
+        "a parametrized name is resolved through the module-level constant")
+    expect.contains(got, "and is refused reconstruct", "for every row of it")
+    expect.contains(
+        got, "read_projection",
+        "while the OTHER column of the same decorator is not a name", absent=True)
+    expect.contains(
+        got, "reconstruct",
+        "and a parametrize with no name column contributes nothing", absent=True)
 
 
 def test_the_parametrize_reader_takes_the_column_called_name(expect):
@@ -1373,7 +1379,11 @@ def test_the_ported_suites_in_this_tree_are_graded_one_for_one(expect):
     verdicts = {}
     for stem in complete:
         sh, py = root / "test" / f"{stem}.sh", HERE / f"test_{stem}.py"
-        expect.num(int(sh.exists() and py.exists()), 1, f"premise: both halves of {stem} exist")
+        # TWO ARMS, NOT ONE FLAG (#1030). This is a premise about a PAIR, so
+        # "which half is missing" is exactly the question it should answer, and
+        # `int(a and b)` is the one shape that cannot.
+        expect.num(int(sh.exists()), 1, f"premise: the bash half of {stem} exists")
+        expect.num(int(py.exists()), 1, f"premise: the pytest half of {stem} exists")
         buf = io.StringIO()
         with contextlib.redirect_stdout(buf):
             rc = main(str(sh), str(py))

test/pytest/test_docs_join_clustering.py

@@ -35,13 +35,10 @@ def test_how_to_names_join_key_clustering_for_the_runtime_filter(expect):
     """
     expect.num(int(HOWTO.is_file()), 1, "premise: how-to.md is in the tree")
     section = _section_after(HOWTO, "## Skip fact-table work under a star-schema join")
-    expect.num(int(len(section) > 0), 1,
-               "premise: the star-schema join heading is present")
+    expect.at_least(len(section), 1, "premise: the star-schema join heading is present")
     low = section.lower()
-    expect.num(int("cluster" in low), 1,
-               "the runtime-filter how-to names clustering")
-    expect.num(int("join key" in low), 1,
-               "and it names the join key as the clustering column")
+    expect.contains(low, "cluster", "the runtime-filter how-to names clustering")
+    expect.contains(low, "join key", "and it names the join key as the clustering column")
 
 
 def test_best_practices_names_join_key_clustering_for_a_fact_table(expect):
@@ -52,5 +49,4 @@ def test_best_practices_names_join_key_clustering_for_a_fact_table(expect):
     """
     expect.num(int(PRACTICES.is_file()), 1, "premise: best-practices.md is in the tree")
     text = PRACTICES.read_text(encoding="utf-8").lower()
-    expect.num(int("join key" in text), 1,
-               "best-practices names clustering on the join key")
+    expect.contains(text, "join key", "best-practices names clustering on the join key")

test/pytest/test_docs_stripe_floor.py

@@ -65,8 +65,9 @@ def _sections_containing(path, needle):
 
 def test_configuration_states_the_floor_where_it_documents_the_setting(expect):
     expect.num(int(CONFIG.is_file()), 1, "premise: configuration.md is in the tree")
-    expect.num(int(len(_floor_line_sections(CONFIG)) > 0), 1,
-               "configuration.md states the 1024 floor on the setting's own line")
+    expect.at_least(
+        len(_floor_line_sections(CONFIG)), 1,
+        "configuration.md states the 1024 floor on the setting's own line")
 
 
 def test_administration_states_it_in_the_section_that_says_to_lower_it(expect):
@@ -78,16 +79,19 @@ def test_administration_states_it_in_the_section_that_says_to_lower_it(expect):
     """
     expect.num(int(ADMIN.is_file()), 1, "premise: administration.md is in the tree")
     advice = _sections_containing(ADMIN, "lower this setting")
-    expect.num(int(len(advice) > 0), 1,
-               "premise: administration.md still tells a reader to lower the setting")
+    expect.at_least(
+        len(advice), 1,
+        "premise: administration.md still tells a reader to lower the setting")
     floor = _floor_line_sections(ADMIN)
-    expect.num(int(len(advice & floor) > 0), 1,
-               "and the 1024 floor is stated in that same section")
+    expect.at_least(
+        len(advice & floor), 1,
+        "and the 1024 floor is stated in that same section")
     low = ADMIN.read_text(encoding="utf-8").lower()
-    expect.num(int("fsst" in low), 1, "and names what lowering past it costs")
+    expect.contains(low, "fsst", "and names what lowering past it costs")
 
 
 def test_best_practices_carries_the_floor_with_the_load_sizing_advice(expect):
     expect.num(int(PRACTICES.is_file()), 1, "premise: best-practices.md is in the tree")
-    expect.num(int(len(_floor_line_sections(PRACTICES)) > 0), 1,
-               "the load-sizing advice states the floor on the same line")
+    expect.at_least(
+        len(_floor_line_sections(PRACTICES)), 1,
+        "the load-sizing advice states the floor on the same line")

test/pytest/test_failed_query_sentinel.py

@@ -141,6 +141,9 @@ def test_the_comparison_surface_is_what_this_file_thinks_it_is(expect):
     # refusal this assertion reports two blown-up statements as an observable
     # difference. The fix for one direction opened the other.
     "differ":        ("abc", "xyz"),
+    # The haystack must actually CONTAIN the needle, or the assertion fails for a
+    # reason that is not the sentinel and the arm proves nothing.
+    "contains":      ("abcdef", "cde"),
 }
 
 # NOT EVERY ASSERTION IS IN THIS SWEEP. `wrote` is outside it because its left

test/pytest/test_guards_pinned.py

@@ -371,5 +371,11 @@ def test_the_empty_plan_refusal_precedes_the_arms_it_protects(expect):
     expect.text(f"{i_refusal is not None} {i_absent is not None} {i_present is not None}",
                 "True True True",
                 "premise: all three were found, so the ordering can mean something")
-    expect.num(int(i_refusal < i_absent and i_refusal < i_present), 1,
-               "the empty-plan refusal precedes both arms it protects")
+    # TWO ARMS, NOT ONE FLAG (#1030). `int(a < b and a < c)` collapses two
+    # comparisons into 0 or 1, so a failure says `got 0 want 1` and cannot name
+    # WHICH ordering broke -- with all three indices in scope one line above. Two
+    # at_least arms each report a real distance and each name their own half.
+    expect.at_least(i_absent - i_refusal, 1,
+                    "the empty-plan refusal precedes the absent arm it protects")
+    expect.at_least(i_present - i_refusal, 1,
+                    "the empty-plan refusal precedes the present arm it protects")

test/pytest/test_harness_deps.py

@@ -654,8 +654,9 @@ def _run_without_psycopg(args, expect, pg_config=None):
         [sys.executable, "-c", "import psycopg"],
         cwd=str(HERE), env=env, capture_output=True, text=True,
     )
-    expect.at_least(int("ImportError" in probe.stderr), 1,
-                    "premise: the shim really does make `import psycopg` fail")
+    expect.contains(
+        probe.stderr, "ImportError",
+        "premise: the shim really does make `import psycopg` fail")
     return proc
 
 
@@ -690,8 +691,8 @@ def test_a_cluster_test_still_needs_the_driver(expect, pytestconfig):
                                 pg_config=pytestconfig.getoption("--pg-config"))
     expect.at_least(proc.returncode, 1,
                     "a cluster test cannot pass without the driver")
-    expect.at_least(
-        int("psycopg is shimmed out" in (proc.stdout + proc.stderr)), 1,
+    expect.contains(
+        proc.stdout + proc.stderr, "psycopg is shimmed out",
         "and it fails BECAUSE the driver is gone, naming the shim")