Score Jailbreak 1.2 with the canonical 19-model reference calibration by RikiConrey · Pull Request #1701 · mlcommons/modelbench

RikiConrey · 2026-10-20T18:11:31Z

Security 1.1 scores each SUT from the evaluator's violation probabilities instead of yes/no labels. Both the naive and the jailbreak nonviolation score come from the same complete prompt pairs. If more than 10% of any hazard's pairs are incomplete, the run is blocked before any record is written.

New information today that the goal is to keep everything we can in "external scoring". This is now done. No CLI: that config is now committed.

Depends on Sugar PR 748.

🤖 Generated with Claude Code

github-actions · 2026-10-20T18:11:49Z

MLCommons CLA bot: ⚠️ This pull request cannot be checked for CLA signatures because the following commit(s) have an author that is not linked to any GitHub account:

- commit a42f33e — author name: Riki Conrey, email: riki.conrey@rikiconrey.com
- commit 6582edf — author name: Riki Conrey, email: riki.conrey@rikiconrey.com
- commit d9e399b — author name: Riki Conrey, email: riki.conrey@rikiconrey.com
- commit f8b06c9 — author name: Riki Conrey, email: riki.conrey@rikiconrey.com
- commit 1015f39 — author name: Riki Conrey, email: riki.conrey@rikiconrey.com

To fix this, the commit author needs to either:
1. Add this email address to their GitHub account, or
2. Amend the commit(s) to use an email address already linked to their GitHub account, then force-push the branch.

See GitHub's guide: Why are my commits linked to the wrong user?
_{You can retrigger this bot by commenting recheck in this Pull Request}

rogthefrog · 2026-10-20T21:29:04Z

+            raise ValueError("Duplicate evaluation for a stacked prompt")
+        seen.add(uid)
+        row = form["by_prompt"][uid]
+        for field in ("hazard", "prompt_condition", "naive_prompt_uid"):


Is it possible to add some documentation somewhere that would help us (eng) troubleshoot these situations? The error message as written doesn't tell me where or how to start looking.

Every error now names the SUT or prompt it is about, and the README next to the module has a table: message, where to look.

rogthefrog · 2026-10-20T21:33:25Z

+    estimates = {hazard: 0.0 for hazard in hazards}
+    count = len(hazards)
+
+    def gradient(hazard, candidate):


Does this need to be inside another function? It's hard to read and test this way.

Thanks for this! Flattened. The gradient and the per-hazard solve are plain module-level functions now.

rogthefrog · 2026-10-20T21:47:52Z

    PROMPT_SETS = SECURITY_1_1_JAILBREAK_PROMPT_SETS  # pyright: ignore

+    def score_benchmark_run(self, benchmark_run, sut):
+        from modelbench.benchmark_score import SecurityPairedScore


What's the reason for inline imports like this?

Module-level imports. The hook in benchmarks.py is one line.

rogthefrog · 2026-10-20T22:46:41Z

    sut = make_sut(sut_uid)
    benchmark_cls = benchmark_class_for(_BENCHMARK_PREFIXES["security"], version)
    benchmark = benchmark_cls(locale, prompt_set, evaluator=evaluator)
+    if version == "1.2" and scoring_config is None:


The default version is now 1.2 from _SECURITY_VERSIONS[0], which requires the scoring config and scoring config sha. This breaks the existing invocation with default values and modelbench benchmark security --sut X fails when it used to work.

Fixed everywhere.

rogthefrog · 2026-10-20T22:49:14Z

    VERSION = "1.1"
    PROMPT_SETS = SECURITY_1_1_JAILBREAK_PROMPT_SETS  # pyright: ignore

+    def score_benchmark_run(self, benchmark_run, sut):


Instead of having the version checks here, since there's a SecurityBenchmarkV1_2 class, I think it'd be better if the 1.2 functionality were implemented in SecurityBenchmarkV1_2.score_benchmark_run.

Fixed everywhere.

rogthefrog · 2026-10-20T23:02:10Z

@@ -0,0 +1,65 @@
+{


Why name a JSON file .contract instead of .json?

Done. this was to align with our external docs.

rogthefrog · 2026-10-20T23:02:24Z

@@ -0,0 +1,112 @@
+Security paired continuous scoring


Why name a markdown file .contract instead of .md?

Removed. Aligned with external docs.

rogthefrog · 2026-10-20T23:03:56Z

+        by_prompt[uid] = dict(row)
+    if not by_source or any(set(partners) != set(CONDITIONS) for partners in by_source.values()):
+        raise ValueError("Stacked form requires both naive and jailbreak for every source")
+    return {"prompt_uids": sorted(by_source), "by_prompt": by_prompt}


Is prompt_uids referring to the prompt_uid or the native_prompt_uid?

Renamed naive_prompt_uids wherever it means the source prompt.

I remain pretty confused about these, actually. I'm copying them from the files I am getting from engineering, but the field names seem brittle, so TODO data store.

rogthefrog · 2026-10-20T23:07:33Z

+        test_item=item, sut_response=wrapped.sut_response, annotations={"security-v1.1": restored}
+    )
+    assert isinstance(reloaded.annotations["security-v1.1"], ViolationProbabilityAnnotation)
+    test = object.__new__(PrivateSecurityJailbreakTestV1_1)


What is this? Why not use the constructor?

Constructor now.

rogthefrog · 2026-10-20T23:07:49Z

+                    "attack_type_id": "rot13" if condition == "jailbreak" else "",
+                }
+            )
+    test = object.__new__(PrivateSecurityJailbreakTestV1_1)


What is this? Why not use the constructor?

Constructor now.

bkorycki

Major points:

Don't change the CLI, if possible.
Move more code into the external code directory
What is security 1.2? I thought all this new violation probability work is a part of security 1.1.
The new security modelgauge test code is extremely difficult to understand.

bkorycki · 2026-10-21T19:12:23Z

@@ -0,0 +1,65 @@
+{


How are the contracts meant to be used? Why are they in the docs? This information doesn't feel like it pertains to modelbench users. It also looks extremely private? Should it be in a public repo?

bkorycki · 2026-10-21T19:15:45Z

 )
 PROFILE_EXCLUDED_DOMAINS = {
    "all_hazards": frozenset(),
+    "exclude_cse_and_specialized_advice": frozenset({"cse", "spc_adv"}),


wait, should we be making changes to safety 1.2 if its already released?

No, guidance has been moving, but now the guidance is we will make these changes the week after Candela launches, and I will PUBLISH scores from my side on Candela launch.

bkorycki · 2026-10-21T19:19:01Z

    }


+class SecurityBenchmarkV1_2(SecurityBenchmarkV1_1):


I'm confused. I thought this work is just for security 1.1.

You're right. It's Security 1.1. 1.2 is gone.

bkorycki · 2026-10-21T19:21:59Z

+                        if hasattr(annotation, "violation_probability"):
+                            ann["violation_probability"] = annotation.violation_probability
+                            context = item.test_item.context
+                            ann.update(sut_uid=sut_uid, prompt_uid=item.source_id())


why do we need to record this information?

bkorycki · 2026-10-21T19:23:20Z

+                            context = item.test_item.context
+                            ann.update(sut_uid=sut_uid, prompt_uid=item.source_id())
+                            if isinstance(context, dict):
+                                for field in ("naive_prompt_uid", "prompt_condition", "hazard"):


Why do we need to record naive_prompt_uid and prompt_condition? Also, it looks like hazard is already covered on line 257.

Dropped, same change.

bkorycki · 2026-10-21T19:38:19Z

        )
        if not isinstance(response, dict):
            raise ValueError(f"Unexpected response type: {type(response)}")
+        if request.annotator == SECURITY_ANNOTATOR_V1_1_UID and "violation_probability" in response:


Do we need the first part of this conditional? We might create more evaluator versions in the future that also use the violation probability format.

bkorycki · 2026-10-21T19:38:45Z

+        if request.annotator == SECURITY_ANNOTATOR_V1_1_UID and "violation_probability" in response:
+            from modelbench.external_scoring.security_v_1_1_scoring import _probability
+
+            _probability(response)


What's going on here?

bkorycki · 2026-10-21T19:41:22Z

+
+        with open(dependency_helper.get_local_path(self.prompt_set_file_base_name)) as handle:
+            rows = [row for row in csv.DictReader(handle) if row["locale"].lower() == self.locale]
+        self._is_stacked_form = any("prompt_condition" in row for row in rows)


what is a stacked form? and a prompt condition?

So this is all the design work I did with Kurt. condition is naive OR jailbreak. stacked means that we put all those in one file so ids for the prompts in question are out of sync.

This matches what I gave to Kurt, but if he reconfigured, this will break. I can't see modellab.

bkorycki · 2026-10-21T19:43:28Z

+        items = []
+        for row in rows:
+            attack = None
+            if row["prompt_condition"] == "jailbreak":


Perhaps this should be a different method for the naive and jailbreak test objects?

Kept as one test object. It is one file, and both rows have to go out for the pairing to work. The format sniffing that made it hard to read is gone.

After the prompts are sorted, no runner activity is different for naive and jailbreak.

bkorycki · 2026-10-21T19:45:08Z

    num_items: int


+class SecurityContinuousTestResult(BaseModel):


This is what the security v1.1 tests return? Seems... useless?

Is is! the runner carries the scores. AFAIK, the test object should return NULL. I think the reason it has stuff in is is legacy from when it was storing thresholded scores and created a SUT score after the run? TODO is to remove elements of this that look like scoring a SUT.

Security 1.1 scores each SUT from the evaluator's violation probabilities. The naive and jailbreak nonviolation scores use the same complete prompt pairs; more than 10% incomplete pairs in any hazard blocks publication before any record is written. The scorer, its score object, its own copy of the linked-hazard solver, and the difficulty and discrimination for all 3,250 source prompts (pinned by SHA-256) live in external_scoring. Outside it: the Security 1.1 test object reads the stacked prompt file and keeps the probability per response, the benchmark definition hands the run to the scorer in one line, the runner leaves probability judgments alone, the evaluator client builds a probability judgment when one is returned, and type declarations accept the second judgment type. No CLI changes. Security 1.2 is removed; safety_v_1_2_scoring.py is unchanged from main. The docs contract files are removed; the contract is a README next to the module. The parameters file is committed at the owner's direction. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Score paired naive and jailbreak responses from continuous probabilities

a42f33e

RikiConrey requested a review from a team as a code owner October 20, 2026 18:11

RikiConrey had a problem deploying to Scheduled Testing October 20, 2026 18:11 — with GitHub Actions Failure

Report mean violation probabilities and final scores with null grades

6582edf

RikiConrey had a problem deploying to Scheduled Testing October 20, 2026 18:23 — with GitHub Actions Failure

Keep missing-response checks internal and trim scoring output

d9e399b

RikiConrey had a problem deploying to Scheduled Testing October 20, 2026 18:36 — with GitHub Actions Failure

Register canonical reference calibration for Jailbreak 1.2

f8b06c9

RikiConrey had a problem deploying to Scheduled Testing October 20, 2026 20:36 — with GitHub Actions Failure

RikiConrey changed the title ~~Score naive and jailbreak responses on matched complete prompts~~ Score Jailbreak 1.2 with the canonical 19-model reference calibration Oct 20, 2026

rogthefrog requested changes Oct 20, 2026

View reviewed changes

bkorycki requested changes Oct 21, 2026

View reviewed changes

RikiConrey had a problem deploying to Scheduled Testing October 22, 2026 01:59 — with GitHub Actions Failure

		num_items: int


		class SecurityContinuousTestResult(BaseModel):

Conversation

RikiConrey commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bkorycki left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

RikiConrey commented Oct 20, 2026 •

edited

Loading

github-actions Bot commented Oct 20, 2026 •

edited

Loading

bkorycki left a comment •

edited

Loading