biolink: migrate the AI-curated corpus to Biolink 4.4.4 (qualifier model)

.claude/commands/curate.md

@@ -73,7 +73,8 @@ See AGENTS.md §4.4 for the `...`/`[...]` operators and the read-the-context gua
 - Walk through 2–6 intermediate biological entities (proteins, pathways, processes, phenotypes) toward the disease node (`label: Disease`, prefix `MESH`).
 - **One connected chain — no shortcut/bypass edge** (AGENTS.md §0). The path is a single chain drug→…→disease. **Do not** add an edge straight from the drug to the disease, and **do not** add an edge from the drug (or any node) to a node already reached through intermediate steps. End the chain at the disease with a mechanistic/causal predicate (`causes`, `contributes to`, …), **never** with a clinical-outcome predicate (`treats`, `prevents`, `ameliorates`, …). Those are reserved for the no-mechanism stub. If you worked out a mechanism, curate *only* that chain — never also append a `Drug —treats→ Disease` edge. (This is the `clinical_shortcut` / `short_circuit` HARD error; the semantic critic checks for it too.)
 - Each edge:
-  - `key`: pick from the canonical 67 in `src/drugmechdb/schema/biolink_predicates.yaml`.
+  - `key`: pick one marked `status: current` in `src/drugmechdb/schema/biolink_predicate_status.yaml` (47 of the 70 enum values; the other 23 are legacy-only and Layer 3 will reject them).
+  - `object_aspect_qualifier` / `object_direction_qualifier`: **required** where the predicate carries no polarity of its own — `affects` needs both, `regulates` needs a direction. This is where `decreases activity of` went: `key: affects` + `object_aspect_qualifier: activity` + `object_direction_qualifier: decreased`. See AGENTS.md §3.0.
   - `evidence`: ≥1 `EvidenceItem` with:
     - `reference: PMID:xxxxxxxx`
     - `snippet:` **verbatim** substring of the cached source (abstract, or full text if you escalated) — copy/paste from `references_cache/`, never typed from memory.

AGENTS.md

@@ -111,21 +111,47 @@ The CURIE prefix of every node `id` **must** match the canonical ontology for th
 
 ## 3. Predicate vocabulary
 
-Every edge `key` must be an **exact** member of the `BiolinkPredicate` enum in `src/drugmechdb/schema/biolink_predicates.yaml` (67 predicates as of May 2026). The list includes the high-frequency ones the agent will reach for first:
+Every edge `key` must be an **exact** member of the `BiolinkPredicate` enum in `src/drugmechdb/schema/biolink_predicates.yaml`. That enum spans two eras — it still accepts the v1.3.0-era vocabulary the 4,846 legacy records are written in — so membership alone is not enough. **New curation must use only the predicates marked `status: current` in `src/drugmechdb/schema/biolink_predicate_status.yaml`** (47 of 70, pinned to Biolink 4.4.4). QC Layer 3 enforces this for any record carrying evidence.
+
+The high-frequency ones you will reach for:
 
 ```
-positively regulates    negatively regulates    decreases activity of    increases activity of
-positively correlated with    negatively correlated with    causes    contributes to
-participates in    occurs in    located in    part of    manifestation of    in taxon
-treats    prevents    has metabolite    binds    affects    disrupts    affects risk for
-…
+regulates    affects    causes    contributes to    treats    prevents
+positively correlated with    negatively correlated with    manifestation of
+participates in    occurs in    located in    part of    in taxon
+has metabolite    directly physically interacts with    disrupts
 ```
 
-Read the file; the descriptions distinguish near-synonyms (e.g. `regulates` vs `positively regulates`, `correlated with` vs `causes`).
+### 3.0 Polarity lives in qualifiers, not in the predicate name
+
+Biolink's v2→v4 refactor removed `positively regulates`, `decreases activity of`, and the whole
+`increases/decreases <aspect> of` family. **Do not use them** — they are `status: legacy_only`.
+Say the same thing with a canonical predicate plus qualifiers on the edge:
+
+```yaml
+  - key: affects                              # was: decreases activity of
+    object_aspect_qualifier: activity         # activity | abundance | expression | synthesis | ...
+    object_direction_qualifier: decreased     # increased | decreased | upregulated | downregulated
+    source: MESH:D000068877
+    target: UniProt:A9UF07
+
+  - key: regulates                            # was: positively regulates
+    object_direction_qualifier: increased
+    source: UniProt:A9UF07
+    target: GO:0008283
+```
+
+`affects` requires **both** qualifiers; `regulates` requires `object_direction_qualifier`. Bare, they
+assert almost nothing, so Layer 3 rejects them without direction. The permissible values are the
+`ObjectAspectEnum` / `ObjectDirectionEnum` enums in `src/drugmechdb/schema/drugmechdb.yaml`, and
+`biolink_predicate_status.yaml` gives the exact replacement for every removed predicate.
+
+Read the enum file; the descriptions distinguish near-synonyms (e.g. `regulates` vs `affects`,
+`correlated with` vs `causes`).
 
 **You may emit predicates in casual surface forms** — `positively_regulates`, `biolink:positively_regulates`, `Positively Regulates` — and `scripts/canonicalize_predicates.py --write` will normalize them. But the **final** YAML must use the canonical form (lowercase, spaces, no prefix). Always canonicalize before validation.
 
-**You may NOT invent new predicates.** If no canonical predicate fits, escalate per PRD §7 Open Q #3 (adding to the enum requires curator approval + Biolink Model citation).
+**You may NOT invent new predicates.** If no canonical predicate fits, escalate per PRD §7 Open Q #3 (adding to the enum requires curator approval + Biolink Model citation). "Valid in Biolink" is not sufficient on its own — the predicate has to be in *our* enum and marked `current`.
 
 ### 3.1 Predicate domain/range
 
@@ -134,15 +160,15 @@ Some predicates only make sense between certain node types. Apply each predicate
 | Predicate(s) | Subject (`source`) label | Object (`target`) label |
 |---|---|---|
 | `treats`, `prevents`, `ameliorates`, `exacerbates`, `contraindicated for` | `Drug`, `ChemicalSubstance` | `Disease`, `PhenotypicFeature` |
-| `decreases activity of`, `increases activity of` | any | `Protein`, `GeneFamily`, `MacromolecularComplex`, `MolecularActivity`, `Pathway`, `BiologicalProcess` |
-| `decreases abundance of`, `increases abundance of`, `increases degradation of`, `decreases synthesis of` | any | `Protein`, `GeneFamily`, `MacromolecularComplex`, `ChemicalSubstance`, `MolecularActivity` |
-| `increases expression of`, `decreases expression of` | any | `Protein`, `GeneFamily`, `MacromolecularComplex`, `ChemicalSubstance` |
+| `affects` + aspect `activity` | any | `Protein`, `GeneFamily`, `MacromolecularComplex`, `MolecularActivity`, `Pathway`, `BiologicalProcess` |
+| `affects` + aspect `abundance`, `degradation`, `synthesis`, `stability`, `transport`, `uptake` | any | `Protein`, `GeneFamily`, `MacromolecularComplex`, `ChemicalSubstance`, `MolecularActivity` |
+| `affects` + aspect `expression` | any | `Protein`, `GeneFamily`, `MacromolecularComplex`, `ChemicalSubstance` |
 | `has metabolite` | `Drug`, `ChemicalSubstance` | `ChemicalSubstance`, `Drug` |
 | `has phenotype` | any | `PhenotypicFeature` |
 | `in taxon` | any | `OrganismTaxon` |
 | `occurs in`, `located in`, `expressed in` | any | `CellularComponent`, `GrossAnatomicalStructure`, `Cell`, `OrganismTaxon`, `Pathway` |
 
-The activity / abundance / expression / degradation / synthesis predicates (rows 2–4) never take a `Disease` or `PhenotypicFeature` object — reach the disease with a mechanistic/causal predicate (e.g. `causes`, `contributes to`) instead, per §0.
+Since Biolink 4.4.4 the object constraint follows the **aspect**, not the predicate name, so the lexicon keys those rows `affects|<aspect>` (the legacy `decreases activity of`-style keys are kept for the legacy corpus). An `affects` edge never takes a `Disease` or `PhenotypicFeature` object — reach the disease with a mechanistic/causal predicate (e.g. `causes`, `contributes to`) instead, per §0.
 
 The gate enforces these: an edge whose known endpoint types fall outside the allowed set is flagged as a SOFT `type_violation` and routed to the semantic critic (§5 step 8), so a mis-typed edge is surfaced and returned for correction rather than passing silently. Getting the types right here keeps the edge from bouncing.
 

CLAUDE.md

@@ -21,8 +21,11 @@ new ones) — keys: `directed`, `multigraph`, `graph`
 Edges may carry per-edge `evidence` (an `EvidenceItem` with a verbatim PubMed `snippet`).
 `_index.yaml` is a **generated** aggregate — don't hand-edit it.
 
-Full structure, the 14 node types, the 67-predicate vocabulary, and the path-quality
+Full structure, the 14 node types, the predicate vocabulary, and the path-quality
 conventions are in **`AGENTS.md`** and **`CurationGuide.md`**. Read those before curating.
+The predicate enum spans two Biolink eras — new curation uses only the `status: current`
+values in `src/drugmechdb/schema/biolink_predicate_status.yaml` (pinned to Biolink 4.4.4),
+with polarity carried in edge qualifiers. See **AGENTS.md §3.0**.
 
 ## The QC gate (`scripts/qc.py`)
 
@@ -35,7 +38,8 @@ and runs the matching layers:
 | `ai_curated` | any edge has `evidence:`      | 1, 2, 3, 4  |
 
 1. **schema** (LinkML `MechanisticPath`) · 2. **node ontology** (prefix↔label) ·
-3. **predicate enum** (67 Biolink predicates) · 4. **reference** (snippet verbatim in source).
+3. **predicate enum** (era-aware: `ai_curated` records must use current-Biolink predicates)
+· 4. **reference** (snippet verbatim in source).
 
 ```bash
 just qc                       # whole corpus, auto profile

CurationGuide.md

@@ -218,6 +218,15 @@ to one of the predicates previously used, if a relationship found in the process
 any of these types listed below, a predicate may be selected
 from the [full set of biolink predicates](https://biolink.github.io/biolink-model/docs/predicates.html).
 
+> **Biolink 4.4.4 note (2026-09).** The list below is the vocabulary *found in the legacy
+> corpus* — it is v1.3.0-era, and 23 of these predicates no longer exist in Biolink. It stays
+> here because `kb/paths/` is still written in it. **New curation must not use them.** Pick a
+> predicate marked `status: current` in `src/drugmechdb/schema/biolink_predicate_status.yaml`,
+> and carry polarity in the edge's `object_aspect_qualifier` / `object_direction_qualifier`
+> rather than in the predicate name — `decreases activity of` is now
+> `affects` + `activity` + `decreased`. See AGENTS.md §3.0 for the contract and
+> `docs/biolink_version_decision.md` for why.
+
 The predicates currently found within DrugMechDB are as follows:
 
 [affects risk for](https://biolink.github.io/biolink-model/docs/affects_risk_for.html)  

README.md

@@ -77,8 +77,9 @@ profile per file and runs the matching layers:
 | `ai_curated` | any edge has `evidence:` | 1, 2, 3, 4 |
 
 1. **schema** (LinkML `MechanisticPath`) · 2. **node ontology** (CURIE prefix ↔ label,
-plus an invisible/whitespace-character guard) · 3. **predicate enum** (67 Biolink
-predicates) · 4. **reference** (every snippet is verbatim in its cited source).
+plus an invisible/whitespace-character guard) · 3. **predicate enum** (Biolink predicates,
+era-aware — newly curated records must use the pinned current release) · 4. **reference**
+(every snippet is verbatim in its cited source).
 
 ```bash
 just qc                       # whole corpus

docs/biolink_drift_monitor.md

@@ -12,7 +12,8 @@
 ## 1. Why this exists
 
 The corpus stores edges keyed by a fixed vocabulary of Biolink predicates
-(`src/drugmechdb/schema/biolink_predicates.yaml`, 67 predicates), each also assigned a
+(`src/drugmechdb/schema/biolink_predicates.yaml`; `biolink_predicate_status.yaml` records
+which of them the pinned Biolink release still defines), each also assigned a
 direction-of-influence sign in `scripts/quality/predicate_polarity.yaml`. Biolink evolves
 independently — the v2→v4 **qualifier refactor** already folded many of our high-frequency
 predicates (`positively regulates`, `increases activity of`, …) into `regulates`/`affects`

docs/biolink_version_decision.md

@@ -1,44 +1,68 @@
 # Biolink-version strategy
 
-> **Decision:** the corpus is **re-curated from scratch in the latest Biolink version**
-> (forwardfill). There is **no legacy vocabulary to preserve or to translate** — the
-> re-curated records are written directly in current Biolink, so storage *is* the current
-> vocabulary. Recorded here as the single source of truth; the publish layer references this.
+> **Decision:** DrugMechDB targets a **pinned current Biolink release** — **4.4.4** — and
+> AI-curated records are written in it, qualifiers and all. The legacy corpus stays in its
+> own vocabulary until forwardfill replaces it. Recorded here as the single source of truth;
+> the publish layer references this.
 
 ## The choice
 
 The drift audit (`docs/biolink_ontology_drift_report.md`) framed three options: **(A)** pin the old
 version, **(B)** migrate stored records, **(C)** keep legacy in storage and translate at publish.
-Its leaning was **C** (translate-at-publish), which minimizes churn *when the legacy corpus is kept*.
 
-That premise no longer holds. Because the corpus is **forwardfilled** — every record re-curated from
-scratch by the AI curator — the output is authored directly in the **latest Biolink version**. This
-is effectively a clean rewrite, not a migration of existing records. So:
+An earlier revision of this document recorded the strategy as *"re-curated from scratch in the latest
+Biolink version, so storage **is** the current vocabulary"* and closed #19 and #54 as moot on that
+basis. **That premise did not hold.** The curator was handed the legacy 67-predicate enum
+(`AGENTS.md` §3, `.claude/commands/curate.md`) and QC Layer 3 enforced it, so forwardfilled records
+came out in the *legacy* vocabulary: measured against Biolink 4.4.4, **323 of 558 edges (58%)** in
+the 135 AI-curated records used predicates Biolink had already removed. Writing "latest Biolink" in
+a policy document does not make the gate produce it.
 
-- **No version pin to an old Biolink** (rejects A).
-- **No record migration** of legacy vocabulary (B is moot — there is no legacy vocabulary in the
-  re-curated corpus to migrate).
-- **No publish-time translation layer** (C is moot — storage already holds current Biolink; there is
-  nothing to translate at export).
+The corrected strategy is **B, scoped to the AI-curated corpus**:
 
-## Consequences (issues closed as moot)
+- **Pin the Biolink release** (4.4.4) in `src/drugmechdb/schema/biolink_predicate_status.yaml`,
+  rather than tracking a moving `master`.
+- **The enum spans both eras.** `biolink_predicates.yaml` still accepts the v1.3.0-era vocabulary,
+  because the 4,846 legacy records in `kb/paths/` are written in it and are not being rewritten.
+- **The gate decides per profile.** Layer 3 permits any enum member for `legacy` records and only
+  `status: current` members for `ai_curated` ones. This is what actually holds new curation to
+  current Biolink — the contract text alone did not.
+- **Records already curated were migrated**, not re-curated, by `scripts/migrate_predicates.py`
+  (a deterministic transform driven by the same status file the gate reads).
 
-- **#19 (publish-time translation to current Biolink)** — moot. Storage is already current Biolink;
-  the export layer emits it directly, with no qualifier remap.
-- **#54 (predicate→qualifier remap rules)** — moot. The ~20 deterministic remap rules were only
-  needed to translate *legacy* predicates at publish; the forwardfilled corpus never uses them.
+## Consequences
 
-*Transition caveat:* until forwardfill completes, the stored corpus is still legacy vocabulary. If a
-publish of the **legacy** corpus is ever needed **before** forwardfill finishes, #19/#54 can be
-reopened for that interim need. The target state (forwardfilled) needs neither.
+- **#19 (publish-time translation to current Biolink)** — still moot *for AI-curated records*
+  (they are stored in current Biolink, so the exporter emits it directly), but **live again for the
+  legacy corpus**: publishing `kb/paths/` as current Biolink needs the translation, because those
+  records keep the legacy vocabulary until forwardfill reaches them.
+- **#54 (predicate→qualifier remap rules)** — **not moot.** The rules are implemented as
+  `scripts/migrate_predicates.py` + the `replacement:` blocks in
+  `src/drugmechdb/schema/biolink_predicate_status.yaml`. They were used to migrate the AI-curated
+  corpus and are what a legacy publish would reuse.
+
+## What has no mechanical answer
+
+Three removed predicates have no faithful qualifier form and are never rewritten automatically —
+they carry `needs_human_decision: true` in the status file:
+
+| Predicate | Why |
+|---|---|
+| `affects risk for` | Direction is unspecified; 4.4.4 splits it into `associated with increased likelihood of` / `associated with decreased likelihood of`. Needs a per-edge reading of the evidence. |
+| `increases response to` / `decreases response to` | Biolink 4.4.4 has no `response` aspect in `GeneOrGeneProductOrChemicalEntityAspectEnum`. |
+
+These appear only in the legacy corpus (1–82 edges each), never in the AI-curated records.
 
 ## Going forward
 
-Keeping the schema aligned with future Biolink releases is handled by the **Biolink-drift monitor**
-(`docs/biolink_drift_monitor.md`, `scripts/check_biolink_drift.py`): it detects predicate drift and
-opens a human-reviewed draft PR — it never auto-applies a schema change and never auto-assigns a
-polarity sign.
+Keeping the pin current is handled by the **Biolink-drift monitor**
+(`docs/biolink_drift_monitor.md`, `scripts/check_biolink_drift.py`): it compares the committed enum
+against a Biolink ref and reports the delta for a human-reviewed draft PR — it never auto-applies a
+schema change and never auto-assigns a polarity sign. When a release moves the pin, update
+`biolink_predicate_status.yaml` and re-run `scripts/migrate_predicates.py --check`.
 
 ### See also
 - `docs/biolink_ontology_drift_report.md` — the drift audit that framed the options.
 - `docs/biolink_drift_monitor.md` — how future Biolink releases are tracked.
+- `src/drugmechdb/schema/biolink_predicate_status.yaml` — the pin, the per-predicate status, and the
+  replacement each removed predicate maps to.

experiments/opus_vs_sonnet/opus/outputs/P01.yaml

@@ -24,7 +24,9 @@ nodes:
     name: Myocardial infarction
     label: Disease
 links:
-  - key: decreases activity of
+  - key: affects
+    object_aspect_qualifier: activity
+    object_direction_qualifier: decreased
     source: MESH:D001241
     target: UniProt:P23219
     evidence:
@@ -37,7 +39,8 @@ links:
         snippet: "the acetylation of platelet cyclooxygenase (COX)-1 at serine-529 is the direct mechanism of action of low-dose aspirin"
         supports: SUPPORT
         evidence_source: HUMAN_CLINICAL
-  - key: positively regulates
+  - key: regulates
+    object_direction_qualifier: increased
     source: UniProt:P23219
     target: CHEBI:50124
     evidence:
@@ -46,7 +49,8 @@ links:
         supports: SUPPORT
         evidence_source: HUMAN_CLINICAL
         explanation: "COX-1 activity drives thromboxane A2 formation; inhibiting COX-1 reduces thromboxane A2."
-  - key: positively regulates
+  - key: regulates
+    object_direction_qualifier: increased
     source: CHEBI:50124
     target: GO:0070527
     evidence: