NVIDIA-BioNeMo · scal444 · Sep 19, 2026 · Sep 18, 2026 · Sep 18, 2026 · Sep 18, 2026
diff --git a/skills/nvmolkit-usage/BENCHMARK.md b/skills/nvmolkit-usage/BENCHMARK.md
@@ -0,0 +1,141 @@
+# Skill Benchmark: nvmolkit-usage
+
+> ✅ **Overall verdict: PASS — Recommended for publication**
+
+## Publication Recommendation
+
+Recommended for publication based on the completed evaluation evidence in this report.
+
+## Evaluation Metadata
+
+- Skill: `nvmolkit-usage`
+- Evaluation date: 2026-09-18
+- Evaluator version: `1.5.6`
+- Agents: Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`), Codex (`openai/openai/gpt-5.5`)
+- Tasks: 12 evaluation tasks (12 positive)
+- Dataset digest: `sha256:ce8098e0dd2fc0698933b7d4d303fe13bdd1d9be7e87d142bdfc44baae7a9f90` (skill-evaluator-dataset-snapshot/1)
+- Attempts per task: 3
+- Environment: `k8s-sandbox`
+- Tier 2 evidence: required for publication
+- Tier 3 evidence: required for publication
+
+Each task attempt ran in its own isolated sandbox pod.
+
+## What This Report Answers
+
+The three-tier evaluation checks whether the skill:
+
+- is safe to use;
+- produces correct answers;
+- is discovered and activated when needed;
+- helps the agent complete the user's goal and expected workflow; and
+- avoids wasted skill and tool usage.
+
+## Results at a Glance
+
+| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) |
+|---|---:|---:|
+| Overall | 92.6% — baseline ran, but no comparable score was available; uplift unavailable | 93.4% — baseline ran, but no comparable score was available; uplift unavailable |
+| Security | 70.8% → 95.8% (+25.0 points) | 100.0% → 100.0% (±0.0 points) |
+| Correctness | 96.7% → 100.0% (+3.3 points) | 93.3% → 100.0% (+6.7 points) |
+| Discoverability | 94.2% — baseline ran, but no comparable score was available; uplift unavailable | 95.0% — baseline ran, but no comparable score was available; uplift unavailable |
+| Effectiveness | 90.3% → 90.8% (+0.5 points) | 84.3% → 86.5% (+2.2 points) |
+| Efficiency | 82.2% — baseline ran, but no comparable score was available; uplift unavailable | 85.7% — baseline ran, but no comparable score was available; uplift unavailable |
+
+**How to read this table:** baseline is the same task attempted without the target skill. Scores are rounded to one decimal; threshold-adjacent values use additional precision so their displayed band matches the verdict. Uplift is derived from those displayed scores and shown in percentage points.
+
+Example: `47.0% → 92.0% (+45.0 points)` means the skill-assisted run scored 92.0%, 45.0 percentage points above its 47.0% no-skill baseline.
+
+## Token Usage
+
+Actual Tier 3 execution usage is reported for every observed agent/case pair and both conditions.
+
+| Agent | Dataset case | With skill | Without skill | Delta | Change | Coverage |
+|---|---|---:|---:|---:|---:|---|
+| claude-code | All cases | 5,388,789 | 6,485,536 | -1,096,747 | -16.91% | skill 12/12; base 12/12 |
+| claude-code | nvmolkit-usage-001 | 206,219 | 690,603 | -484,384 | -70.14% | skill 1/1; base 1/1 |
+| claude-code | nvmolkit-usage-002 | 1,257,795 | 177,892 | +1,079,903 | +607.06% | skill 1/1; base 1/1 |
+| claude-code | nvmolkit-usage-003 | 285,979 | 297,646 | -11,667 | -3.92% | skill 1/1; base 1/1 |
+| claude-code | nvmolkit-usage-004 | 245,297 | 588,508 | -343,211 | -58.32% | skill 1/1; base 1/1 |
+| claude-code | nvmolkit-usage-005 | 648,715 | 740,940 | -92,225 | -12.45% | skill 1/1; base 1/1 |
+| claude-code | nvmolkit-usage-006 | 74,549 | 561,517 | -486,968 | -86.72% | skill 1/1; base 1/1 |
+| claude-code | nvmolkit-usage-007 | 302,671 | 550,115 | -247,444 | -44.98% | skill 1/1; base 1/1 |
+| claude-code | nvmolkit-usage-008 | 1,160,762 | 708,853 | +451,909 | +63.75% | skill 1/1; base 1/1 |
+| claude-code | nvmolkit-usage-009 | 69,766 | 416,700 | -346,934 | -83.26% | skill 1/1; base 1/1 |
+| claude-code | nvmolkit-usage-010 | 389,838 | 1,152,170 | -762,332 | -66.16% | skill 1/1; base 1/1 |
+| claude-code | nvmolkit-usage-011 | 69,928 | 61,294 | +8,634 | +14.09% | skill 1/1; base 1/1 |
+| claude-code | nvmolkit-usage-012 | 677,270 | 539,298 | +137,972 | +25.58% | skill 1/1; base 1/1 |
+| codex | All cases | 1,063,506 | 653,365 | +410,141 | +62.77% | skill 12/12; base 12/12 |
+| codex | nvmolkit-usage-001 | 42,626 | 30,919 | +11,707 | +37.86% | skill 1/1; base 1/1 |
+| codex | nvmolkit-usage-002 | 42,737 | 47,031 | -4,294 | -9.13% | skill 1/1; base 1/1 |
+| codex | nvmolkit-usage-003 | 119,838 | 142,275 | -22,437 | -15.77% | skill 1/1; base 1/1 |
+| codex | nvmolkit-usage-004 | 73,963 | 104,844 | -30,881 | -29.45% | skill 1/1; base 1/1 |
+| codex | nvmolkit-usage-005 | 136,297 | 38,676 | +97,621 | +252.41% | skill 1/1; base 1/1 |
+| codex | nvmolkit-usage-006 | 50,646 | 33,189 | +17,457 | +52.60% | skill 1/1; base 1/1 |
+| codex | nvmolkit-usage-007 | 75,109 | 35,311 | +39,798 | +112.71% | skill 1/1; base 1/1 |
+| codex | nvmolkit-usage-008 | 130,297 | 59,105 | +71,192 | +120.45% | skill 1/1; base 1/1 |
+| codex | nvmolkit-usage-009 | 85,560 | 18,457 | +67,103 | +363.56% | skill 1/1; base 1/1 |
+| codex | nvmolkit-usage-010 | 147,972 | 27,105 | +120,867 | +445.92% | skill 1/1; base 1/1 |
+| codex | nvmolkit-usage-011 | 51,430 | 20,374 | +31,056 | +152.43% | skill 1/1; base 1/1 |
+| codex | nvmolkit-usage-012 | 107,031 | 96,079 | +10,952 | +11.40% | skill 1/1; base 1/1 |
+| ALL AGENTS | Dataset aggregate | 6,452,295 | 7,138,901 | -686,606 | -9.62% | skill 24/24; base 24/24 |
+
+Prompt tokens include cached reads, so total tokens are `prompt + completion` (cached is not added twice). The Efficiency score uses `(prompt - cached) + completion`. N/A means the relevant trajectory counters were not available; coverage is never estimated.
+
+## Tier Status
+
+| Tier | Purpose | Status | Evidence |
+|---|---|---|---|
+| Tier 1 | Static validation | **PASSED WITH OBSERVATIONS** | 11 validator(s); 10 finding(s) |
+| Tier 2 | Semantic deduplication | **PASSED** | 2 validator(s); 0 finding(s) |
+| Tier 3 | Live agent evaluation | **PASS** | 2 agent(s); 12 task(s) |
+
+## Findings and Observations
+
+<details>
+<summary>Show detailed findings and successful checks</summary>
+
+- **MEDIUM** QUALITY/quality_correctness: SKILL_SPEC recommended field missing: 'metadata.tags' (`skills/nvmolkit-usage/SKILL.md`)
+- **MEDIUM** QUALITY/quality_efficiency: Large skill (5312 tokens, recommended max <5000). Per agentskills.io, SKILL.md should be concise (~500 lines) — large skill bodies increase token cost after invocation; long or unfocused top-level descriptions can degrade agent routing accuracy (`skills/nvmolkit-usage/SKILL.md`)
+- **LOW** QUALITY/quality_discoverability: Description very long (567 chars, recommend 50-150) (`skills/nvmolkit-usage/SKILL.md`)
+- **LOW** QUALITY/quality_discoverability: No '## Purpose' section (`skills/nvmolkit-usage/SKILL.md`)
+- **LOW** QUALITY/quality_reliability: No prerequisites/requirements documented (`skills/nvmolkit-usage/SKILL.md`)
+- 5 additional finding(s) are available in the full evaluation artifacts.
+
+</details>
+
+## Scoring Methodology
+
+<details>
+<summary>Show dimension definitions, source signals, and thresholds</summary>
+
+| Dimension | Question | Scored signals |
+|---|---|---|
+| Security | Is it safe to use? | `security` (100%) |
+| Correctness | Is the answer correct? | `accuracy` (100%) |
+| Discoverability | Was the right skill loaded when needed? | `skill_execution` (100%) |
+| Effectiveness | Did the skill help complete the task? | `goal_accuracy` (50%) + `behavior_check` (50%) |
+| Efficiency | Did it avoid wasted tool calls and token usage? | `skill_efficiency` (50%) + `token_efficiency` (50%) |
+
+- Dimension bands: PASS at 50% or above; NEUTRAL from 40% to below 50%; FAIL below 40%.
+- Overall Tier 3 lift: PASS at +5 points or more; FAIL at -10 points or less; values between those bands are NEUTRAL.
+- Overall verdict: PASS only when every configured dimension passes for at least one supported agent. Lift is reported as diagnostic evidence and does not override this gate.
+- The 50% attempt pass threshold is a separate per-task gate; it is not the dimension pass threshold.
+- Effectiveness is the equal-weight mean of goal completion (`goal_accuracy`) and expected workflow adherence (`behavior_check`).
+- Efficiency is 50% tool-call productivity (the backward-compatible `skill_efficiency` wire id) and 50% `token_efficiency`. Positive-case skill routing is scored under Discoverability, not Efficiency; a negative case without a routing target is N/A. N/A sources are omitted, remaining weights are renormalized, and the dimension is marked partial.
+
+Signals present in this run:
+
+- `security` (Security): unsafe operations, secret leakage, and unauthorized access.
+- `skill_execution` (Skill Execution): whether the expected skill was selected, decoys were avoided, and the workflow executed.
+- `skill_efficiency` (Tool Productivity): tool-call productivity (legacy wire id; routing is scored under Discoverability).
+- `accuracy` (Accuracy): final-answer correctness against the reference answer.
+- `goal_accuracy` (Goal Accuracy): whether the user's goal was achieved.
+- `behavior_check` (Behavior Check): whether the expected workflow behavior was followed.
+- `token_efficiency` (Token Efficiency): actual uncached prompt plus completion usage (50% of Efficiency).
+
+</details>
+
+## Freshness
+
+Regenerate this benchmark when the skill, evaluation dataset, target agent/model, evaluator version, environment, or scoring policy changes.
diff --git a/skills/nvmolkit-usage/SKILL.md b/skills/nvmolkit-usage/SKILL.md
@@ -12,8 +12,9 @@ description: >-
   source.
 license: Apache-2.0
 metadata:
+  author: Kevin Boyd (@scal444)
   owner: Kevin Boyd (@scal444)
-  risk_tier: skill
+  risk-tier: skill
 ---
 
 # nvMolKit usage
@@ -43,10 +44,12 @@ If CUDA is unavailable, nvMolKit calls raise. There is no CPU fallback - if the
 When helping with installation, make the user choose a PyTorch CUDA backend that the host driver supports before installing nvMolKit. nvMolKit's PyPI wheels are built with CUDA Toolkit 12.9 and depend on CUDA 12 runtime packages, but pip/uv can still select a CUDA 13 PyTorch wheel unless the install command says otherwise.
 
 - Conda: prefer conda-forge `pytorch-gpu`; pin `cuda-version=12.6` or another CUDA version supported by the driver.
-- pip: send the user to the PyTorch install selector (`https://pytorch.org/get-started/locally/`) or previous-versions page (`https://pytorch.org/get-started/previous-versions/`) to install `torch` for a CUDA 12.x backend before installing nvMolKit.
+- pip: send the user to the [PyTorch install selector](https://pytorch.org/get-started/locally/) or [previous-versions page](https://pytorch.org/get-started/previous-versions/) to install `torch` for a CUDA 12.x backend before installing nvMolKit.
 - uv: install nvMolKit with an explicit backend, e.g. `uv pip install --torch-backend=cu128 nvmolkit`.
 
-## Verify the install before writing real code
+## Instructions
+
+### Verify the install before writing real code
 
 Run this once to confirm nvMolKit is importable and a GPU op works end to end:
 
@@ -69,7 +72,7 @@ print("fps shape:", tuple(fps.shape), "dtype:", fps.dtype)
 # Expected: shape (3, 32), dtype torch.int32  (1024 bits packed into 32 int32s per row)
 ```
 
-If this fails, point the user at the install guide on the docs site rather than guessing - see "Going deeper" below.
+If this fails, point the user at the [installation guide](https://nvidia-bionemo.github.io/nvMolKit/#installation) rather than guessing.
 
 ## Entry points
 
@@ -202,7 +205,7 @@ autoselect execution settings; an empty `gpuIds` list uses the current device.
 `MCSConfig` supports `to_dict()` / `from_dict()` and can also be persisted with
 `nvmolkit.autotune.save()` / `load()`.
 
-## Recipes
+## Examples
 
 ### Morgan fingerprints + bulk Tanimoto similarity
 
@@ -378,4 +381,4 @@ All conformers of each input molecule are minimized in one batch. Constraints at
 
 - Full feature list, API reference, and guides: <https://nvidia-bionemo.github.io/nvMolKit/>
 - What changed in each release: <https://nvidia-bionemo.github.io/nvMolKit/changelog.html>
-- Worked examples (Jupyter notebooks): the `examples/` directory in the GitHub repo
+- Worked examples (Jupyter notebooks): the [examples/ directory](https://github.com/NVIDIA-BioNeMo/nvMolKit/tree/main/examples) in the GitHub repo
diff --git a/skills/nvmolkit-usage/skill-card.md b/skills/nvmolkit-usage/skill-card.md
@@ -1,5 +1,5 @@
 ## Description: <br>
-Write code that calls the installed nvMolKit Python API for GPU-accelerated, batched RDKit-style operations — Morgan fingerprints, Tanimoto/cosine similarity, ETKDG conformer embedding, MMFF/UFF optimization, TFD, conformer RMSD, Butina clustering, substructure search, and maximum common substructure (MCS) search. <br>
+Write code that calls the installed nvMolKit Python API for GPU-accelerated, batched RDKit-style operations including Morgan fingerprints, Tanimoto/cosine similarity, ETKDG conformer embedding, MMFF/UFF optimization, TFD, conformer RMSD, Butina clustering, substructure search, and maximum common substructure (MCS) search. <br>
 
 This skill is ready for commercial/non-commercial use. <br>
 
@@ -9,14 +9,14 @@ NVIDIA <br>
 ### License/Terms of Use: <br>
 Apache-2.0 <br>
 ## Use Case: <br>
-Developers and computational chemists writing Python code that calls the nvMolKit API for GPU-accelerated, batched cheminformatics operations such as fingerprinting, similarity search, conformer generation, force-field optimization, clustering, substructure search, and MCS search. <br>
+Developers and engineers writing GPU-accelerated cheminformatics code using the nvMolKit Python API for batched molecular operations such as fingerprinting, similarity search, conformer embedding, force field optimization, clustering, and substructure/MCS search. <br>
 
 ### Deployment Geography for Use: <br>
 Global <br>
 
 ## Requirements / Dependencies: <br>
-**Requires API Key or External Credential:** [No] <br>
-**Credential Type(s):** [None] <br>
+**Requires API Key or External Credential:** [Not Specified] <br>
+**Credential Type(s):** [None identified] <br>
 
 Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. <br>
 
@@ -27,6 +27,7 @@ Mitigation: Review and scan skill before deployment. <br>
 ## Reference(s): <br>
 - [nvMolKit Documentation](https://nvidia-bionemo.github.io/nvMolKit/) <br>
 - [nvMolKit Changelog](https://nvidia-bionemo.github.io/nvMolKit/changelog.html) <br>
+- [nvMolKit Examples (Jupyter Notebooks)](https://github.com/NVIDIA-BioNeMo/nvMolKit/tree/main/examples) <br>
 
 
 ## Skill Output: <br>
@@ -42,36 +43,36 @@ Mitigation: Review and scan skill before deployment. <br>
 
 
 ## Evaluation Tasks: <br>
-12 evaluation tasks (12 positive), each run in an isolated sandbox pod. Dataset digest: sha256:16ead845d201c386f3f059697aff38c0e6978ce5e90370c6548d7bade427c6fb. <br>
+12 evaluation tasks with 3 attempts per task, each in an isolated sandbox pod. Dataset digest: sha256:ce8098e0dd2fc0698933b7d4d303fe13bdd1d9be7e87d142bdfc44baae7a9f90. <br>
 
 ## Evaluation Metrics Used: <br>
 Reported benchmark dimensions: <br>
-- Security: Checks for unsafe operations, secret leakage, and unauthorized access. <br>
-- Correctness: Checks final-answer correctness against the reference answer. <br>
-- Discoverability: Checks whether the expected skill was selected, decoys were avoided, and the workflow executed. <br>
-- Effectiveness: Equal-weight mean of goal completion (goal_accuracy) and expected workflow adherence (behavior_check). <br>
-- Efficiency: 50% tool-call productivity (skill_efficiency) and 50% token efficiency. <br>
+- Security: Whether the skill is safe to use — checks for unsafe operations, secret leakage, and unauthorized access. <br>
+- Correctness: Whether the final answer is correct against the reference answer. <br>
+- Discoverability: Whether the right skill was loaded when needed — skill selection, decoy avoidance, and workflow execution. <br>
+- Effectiveness: Whether the skill helped complete the user's goal, scored as equal-weight mean of goal completion and expected workflow adherence. <br>
+- Efficiency: Whether the skill avoided wasted tool calls and token usage, scored as 50% tool-call productivity and 50% token efficiency. <br>
 
 Underlying evaluation signals used in this run: <br>
-- `security`: Checks for unsafe operations, secret leakage, and unauthorized access. <br>
+- `security`: Unsafe operations, secret leakage, and unauthorized access. <br>
 - `skill_execution`: Whether the expected skill was selected, decoys were avoided, and the workflow executed. <br>
-- `skill_efficiency`: Tool-call productivity (routing scored under Discoverability, not Efficiency). <br>
 - `accuracy`: Final-answer correctness against the reference answer. <br>
 - `goal_accuracy`: Whether the user's goal was achieved. <br>
 - `behavior_check`: Whether the expected workflow behavior was followed. <br>
+- `skill_efficiency`: Tool-call productivity (routing scored under Discoverability, not Efficiency). <br>
 - `token_efficiency`: Actual uncached prompt plus completion token usage. <br>
 
 
 
 ## Evaluation Results: <br>
 | Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) |
 |---|---:|---:|
-| Overall | 92.9% | 93.2% |
-| Security | 79.2% → 100.0% (+20.8 points) | 100.0% → 100.0% (±0.0 points) |
-| Correctness | 100.0% → 96.7% (-3.3 points) | 90.0% → 100.0% (+10.0 points) |
-| Discoverability | 98.3% | 94.6% |
-| Effectiveness | 90.9% → 86.5% (-4.4 points) | 74.8% → 91.6% (+16.8 points) |
-| Efficiency | 82.9% | 79.7% |
+| Overall | 92.6% | 93.4% |
+| Security | 70.8% → 95.8% (+25.0 pp) | 100.0% → 100.0% (±0.0 pp) |
+| Correctness | 96.7% → 100.0% (+3.3 pp) | 93.3% → 100.0% (+6.7 pp) |
+| Discoverability | 94.2% | 95.0% |
+| Effectiveness | 90.3% → 90.8% (+0.5 pp) | 84.3% → 86.5% (+2.2 pp) |
+| Efficiency | 82.2% | 85.7% |
 
 ## Skill Version(s): <br>
 0.6.0 (source: pyproject.toml, CHANGELOG, released 2026-08-13) <br>