From d03cf0b1aea72f164e35d385b0e0d6a9a622db7e Mon Sep 17 00:00:00 2001 From: root Date: Fri, 7 Aug 2026 22:28:35 +0800 Subject: [PATCH 01/24] docs(skills): add DPA4 workflows --- doc/agent-skills.md | 13 +- skills/deepmd-finetune-dpa3/SKILL.md | 15 +- skills/deepmd-finetune-dpa4/SKILL.md | 131 ++++++++++++++++++ skills/deepmd-python-inference/SKILL.md | 21 +-- .../references/model-artifacts.md | 85 ++++++++++++ skills/deepmd-train/SKILL.md | 14 +- skills/deepmd-train/models/dpa4.md | 100 +++++++++++++ skills/lammps-deepmd/SKILL.md | 8 +- .../references/commands-and-workflow.md | 2 +- .../references/model-deployment.md | 89 ++++++++++++ 10 files changed, 456 insertions(+), 22 deletions(-) create mode 100644 skills/deepmd-finetune-dpa4/SKILL.md create mode 100644 skills/deepmd-python-inference/references/model-artifacts.md create mode 100644 skills/deepmd-train/models/dpa4.md create mode 100644 skills/lammps-deepmd/references/model-deployment.md diff --git a/doc/agent-skills.md b/doc/agent-skills.md index 54abd87d2e..dda701ace0 100644 --- a/doc/agent-skills.md +++ b/doc/agent-skills.md @@ -18,14 +18,17 @@ in the DeePMD-kit repository under `skills/`. The skill uses progressive disclosure: the top-level workflow handles common training steps and model selection, while model-specific configuration lives under `skills/deepmd-train/models/` and is read only after a model is chosen. - Current references include DPA3 and se_e2_a. + Current references include DPA3, DPA4/SeZM, and se_e2_a. - `deepmd-finetune-dpa3`: Fine-tune DPA3 models from self-trained checkpoints, multi-task pretrained models, or built-in models downloaded by `dp pretrained download`. +- `deepmd-finetune-dpa4`: Fine-tune DPA4/SeZM checkpoints with the PyTorch + backend using standard or LoRA fine-tuning, then validate and export to `.pt2`. - `deepmd-python-inference`: Run Python and CLI inference with trained or - frozen DeePMD-kit models, including energy, force, virial, descriptor, and - model-deviation workflows. + frozen DeePMD-kit models, including DPA4/SeZM `.pt2` archives and energy, + force, virial, descriptor, embedding, and model-deviation workflows. - `lammps-deepmd`: Prepare, explain, and run LAMMPS simulations with DeePMD-kit - potentials, including common NVE, NVT, and NPT setups. + potentials, including DPA4/SeZM `.pt2` deployment and common NVE, NVT, and + NPT setups. ## Related reference @@ -78,5 +81,7 @@ without launching an expensive calculation. For example: for loading a frozen DeePMD-kit model and evaluating one frame.” - “Use the `deepmd-train` skill to choose between DPA3 and se_e2_a for a small water dataset and draft a training input, but do not start training.” +- “Use the `deepmd-finetune-dpa4` skill to inspect a DPA4 checkpoint and draft + a LoRA fine-tuning input, but do not start training.” - “Use the `lammps-deepmd` skill to prepare an NVT LAMMPS input file for a DeePMD-kit model, and explain each command.” diff --git a/skills/deepmd-finetune-dpa3/SKILL.md b/skills/deepmd-finetune-dpa3/SKILL.md index d166676280..f39910e02a 100644 --- a/skills/deepmd-finetune-dpa3/SKILL.md +++ b/skills/deepmd-finetune-dpa3/SKILL.md @@ -1,11 +1,11 @@ --- name: deepmd-finetune-dpa3 -description: Fine-tune a DPA3 model in DeePMD-kit using the PyTorch backend. Use when the user wants to adapt a pre-trained DPA3 model to a new downstream dataset. Supports fine-tuning from a self-trained DPA3 model (.pt checkpoint), from a multi-task pre-trained model, or from a built-in pretrained model downloaded via `dp pretrained download` (e.g., DPA-3.1-3M, DPA-3.2-5M, DPA-3.3-1M). Covers single-task and multi-task fine-tuning workflows. +description: Fine-tune a DPA3 model in DeePMD-kit using the PyTorch backend. Use when the user wants to adapt a pre-trained DPA3 model to a new downstream dataset. Supports fine-tuning from a self-trained DPA3 model (.pt checkpoint), from a multi-task pre-trained model, or from a built-in pretrained model downloaded via `dp pretrained download`. Do not select this skill from the .pt suffix alone; inspect the stored descriptor first. compatibility: Requires deepmd-kit with PyTorch backend installed. GPU strongly recommended. license: LGPL-3.0-or-later metadata: author: iProzd - version: '1.0' + version: '1.1' repository: https://github.com/deepmodeling/deepmd-kit --- @@ -17,6 +17,17 @@ Fine-tune a pre-trained DPA3 model on a downstream dataset. This skill covers th 1. Fine-tuning from a multi-task pre-trained DPA3 model 1. Fine-tuning from a built-in pretrained model (e.g., DPA-3.1-3M, DPA-3.2-5M, DPA-3.3-1M) downloaded via `dp pretrained download` +Both DPA3 and DPA4 checkpoints use the `.pt` suffix. If the user has not +already established the model family, inspect it before proceeding: + +```bash +dp --pt show pretrained.pt descriptor fitting-net type-map +``` + +Use this skill only when the stored descriptor identifies DPA3. Use +`deepmd-finetune-dpa4` for DPA4/SeZM, and stop when the family cannot be +established from the checkpoint. + ## Quick Start ```bash diff --git a/skills/deepmd-finetune-dpa4/SKILL.md b/skills/deepmd-finetune-dpa4/SKILL.md new file mode 100644 index 0000000000..0011706877 --- /dev/null +++ b/skills/deepmd-finetune-dpa4/SKILL.md @@ -0,0 +1,131 @@ +--- +name: deepmd-finetune-dpa4 +description: Fine-tune a DPA4/SeZM model in DeePMD-kit with the PyTorch backend. Use for standard or LoRA fine-tuning from a DPA4/SeZM .pt checkpoint, then validation and .pt2 export. Do not select this skill from the .pt suffix alone; inspect the stored descriptor first. +compatibility: Requires deepmd-kit with the PyTorch backend. DPA4/SeZM training is GPU-oriented. +license: LGPL-3.0-or-later +metadata: + author: SchrodingersCattt + version: '1.0' + repository: https://github.com/deepmodeling/deepmd-kit +--- + +# DeePMD-kit Fine-tuning: DPA4 + +Fine-tune a DPA4/SeZM checkpoint on downstream DeePMD data. This skill covers +single-task standard and LoRA fine-tuning. Do not infer the model family from a +`.pt` suffix or filename: DPA3 and DPA4 checkpoints use the same suffix. + +## Route the checkpoint + +If the user has not already established the model family, inspect the stored +configuration: + +```bash +dp --pt show pretrained.pt descriptor fitting-net type-map +``` + +Use this skill only when the descriptor/model configuration identifies DPA4 or +SeZM. If the checkpoint is multi-task, inspect its branches before selecting a +head: + +```bash +dp --pt show pretrained.pt model-branch descriptor type-map +``` + +Do not guess a branch. Use `deepmd-finetune-dpa3` instead when the descriptor is +DPA3, and stop when the family cannot be established. + +## Before fine-tuning + +1. Confirm the checkpoint exists and can be inspected. +1. Confirm training and validation systems, labels, and element type maps. +1. Keep a held-out test set that is not used for training or model selection. +1. Start from the exact checkpoint architecture. Introducing new element types, + changing architecture, or combining specialized spin/property/multi-task + configurations requires separate compatibility validation. +1. Choose standard fine-tuning or LoRA. Do not assume a built-in DPA4 model name; + check `dp pretrained download -h` for the installed version. + +## Standard fine-tuning + +The model section in `input.json` must match the checkpoint unless the standard +pretrained-script mechanism is deliberately used: + +```bash +dp --pt train input.json --finetune pretrained.pt +``` + +If the architecture is unknown, `--use-pretrain-script` can inherit the stored +model configuration except for `type_map`: + +```bash +dp --pt train input.json --finetune pretrained.pt --use-pretrain-script +``` + +Inspect the resulting configuration and run a bounded initial segment before a +long training job. Do not combine model-specific additions with +`--use-pretrain-script` unless that combination has been validated. + +## LoRA fine-tuning + +DPA4/SeZM supports LoRA adapters for single-task fine-tuning. Copy the exact base +architecture into `lora_ft.json`, then add: + +```json +{ + "model": { + "type": "dpa4", + "lora": { + "rank": 16, + "alpha": 16.0 + } + } +} +``` + +Run: + +```bash +dp --pt train lora_ft.json --finetune pretrained.pt +``` + +The JSON fragment above is not a complete training input. Adapt the full public +example at `examples/water/dpa4/lora_ft.json`. Do not add +`--use-pretrain-script` to this LoRA command unless a targeted test confirms that +the intended LoRA configuration is retained. + +## Monitor and validate + +Monitor `lcurve.out` for non-finite values and train/validation divergence. +Select a checkpoint using validation data, then evaluate the selected checkpoint +on the complete held-out test systems. Report energy and force errors, plus +virial errors when those labels are part of the task. + +## Export and test + +DPA4/SeZM uses the `.pt2` AOTInductor export path rather than the conventional +PyTorch `.pth` freeze path: + +```bash +dp --pt freeze -c ckpt/model.ckpt.pt -o finetuned_model +dp test -m finetuned_model.pt2 -s /path/to/test_system -n 30 +``` + +The freeze command detects DPA4/SeZM and writes `finetuned_model.pt2`. Validate +the exported archive in the target environment before deployment. + +## Checklist + +- [ ] The stored descriptor identifies DPA4/SeZM; the `.pt` suffix was not used as proof. +- [ ] The intended branch is explicit for a multi-task checkpoint. +- [ ] Training, validation, and held-out test systems are separate. +- [ ] The input architecture is compatible with the checkpoint. +- [ ] LoRA uses a complete base configuration and is not silently overwritten. +- [ ] Training and held-out metrics are finite and reported with units. +- [ ] The selected `.pt` checkpoint was exported to and tested as `.pt2`. + +## References + +- [DPA4 model and LoRA documentation](https://docs.deepmodeling.com/projects/deepmd/en/latest/model/dpa4.html) +- [Fine-tuning documentation](https://docs.deepmodeling.com/projects/deepmd/en/latest/train/finetuning.html) +- [Show model information](https://docs.deepmodeling.com/projects/deepmd/en/latest/model/show-model-info.html) diff --git a/skills/deepmd-python-inference/SKILL.md b/skills/deepmd-python-inference/SKILL.md index 84348f835d..42796b8c22 100644 --- a/skills/deepmd-python-inference/SKILL.md +++ b/skills/deepmd-python-inference/SKILL.md @@ -1,12 +1,12 @@ --- name: deepmd-python-inference -description: Run Python inference with DeePMD-kit models using the DeepPot API. Use when the user wants to load a trained/frozen DeePMD model (.pth or .pb) or a built-in pretrained model (e.g., DPA-3.2-5M) in Python, predict energy/force/virial for atomic configurations, evaluate descriptors, or calculate model deviation between multiple models. Also covers using `dp test` CLI for batch evaluation against labeled data. -compatibility: Requires deepmd-kit Python package installed. PyTorch backend for .pth models, TensorFlow for .pb models. +description: Run Python inference with DeePMD-kit models using the DeepPot API. Use when the user wants to load a checkpoint, frozen model (.pb, .pth, or DPA4 .pt2), or built-in pretrained model in Python; predict energy/force/virial; evaluate descriptors; calculate model deviation; or use `dp test` against labeled data. +compatibility: Requires deepmd-kit installed with the backend required by the selected model artifact. license: LGPL-3.0-or-later metadata: - author: iProzd - version: '1.0' - repository: https://github.com/deepmodeling/deepmd-kit + author: iProzd + version: '1.1' + repository: https://github.com/deepmodeling/deepmd-kit --- # DeePMD-kit Python Inference @@ -29,9 +29,10 @@ e, f, v = dp.eval(coord, cell, atype) ## Agent Responsibilities 1. Determine the model source: - - Frozen model file (`.pth` for PyTorch, `.pb` for TensorFlow) + - Frozen model file (`.pth` for conventional PyTorch, `.pb` for TensorFlow, or `.pt2` for DPA4/SeZM) - Built-in pretrained model name (e.g., `DPA-3.2-5M`) - - Checkpoint file (requires freezing first) + - PyTorch checkpoint (`.pt`), whose stored model configuration must be inspected before choosing an inference or export path +1. Read `references/model-artifacts.md` for `.pt`/`.pt2` models or whenever the artifact route is unclear. 1. Determine the inference task: - Single-frame prediction (energy, force, virial) - Batch prediction over multiple frames @@ -54,6 +55,9 @@ dp = DeepPot("model.pth") # From a frozen TensorFlow model dp = DeepPot("graph.pb") +# From a frozen DPA4/SeZM model +dp = DeepPot("model.pt2") + # From a built-in pretrained model (auto-downloads if not cached) dp = DeepPot("DPA-3.2-5M") ``` @@ -285,7 +289,8 @@ dp pretrained download DPA-3.2-5M --cache-dir ./models ## Agent Checklist -- [ ] Model file exists and is accessible (`.pth`, `.pb`, or valid pretrained name) +- [ ] Model file exists and is accessible (`.pb`, `.pth`, `.pt`, `.pt2`, or valid pretrained name) +- [ ] An ambiguous `.pt` checkpoint was classified from its stored configuration, not its filename - [ ] `coord` array is shaped (nframes, natoms\*3) and in Angstrom - [ ] `cell` array is shaped (nframes, 9) or `None` for non-periodic systems - [ ] `atype` indices match the model's `type_map` ordering diff --git a/skills/deepmd-python-inference/references/model-artifacts.md b/skills/deepmd-python-inference/references/model-artifacts.md new file mode 100644 index 0000000000..e066ae5a44 --- /dev/null +++ b/skills/deepmd-python-inference/references/model-artifacts.md @@ -0,0 +1,85 @@ +# DeePMD model artifacts for inference + +Read this reference when the model is a training checkpoint, its extension is +`.pt2`, or the correct backend/export path is unclear. + +## Identify the artifact + +A suffix identifies a serialization/backend route, not necessarily a model +family. In particular, both DPA3 and DPA4 training checkpoints use `.pt`. Never +classify a `.pt` checkpoint from its filename alone. Inspect its stored model +configuration when needed: + +```bash +dp --pt show model.pt descriptor fitting-net type-map +``` + +| Artifact | Typical role | Inference guidance | +| --- | --- | --- | +| `.pb` | TensorFlow frozen model | Load with `DeepPot` or use `dp test`. | +| `.pth` | Conventional PyTorch frozen model | Load with `DeepPot` or use `dp test`. | +| `.pt` | PyTorch training checkpoint | Inspect before use. DPA4 supports eager Python evaluation and embedding extraction from a checkpoint; deployment normally uses a frozen artifact. | +| `.pt2` | DPA4/SeZM AOTInductor deployment archive | Load with `DeepPot`, use `dp test`, or deploy to a compatible LAMMPS build. | + +Backend selection for inference is normally determined from the model artifact. +Do not add a backend flag merely from the assumed model family. + +## DPA4/SeZM + +DPA4/SeZM supports Python evaluation from its `.pt` checkpoint, but a `.pt2` +archive is the normal frozen deployment artifact. Freeze with: + +```bash +dp --pt freeze -c model.ckpt.pt -o frozen_model +``` + +The command writes `frozen_model.pt2` for a detected DPA4/SeZM checkpoint. +Evaluate the archive with: + +```python +from deepmd.infer import DeepPot + +model = DeepPot("frozen_model.pt2") +energy, force, virial = model.eval(coord, cell, atype) +``` + +For labeled data: + +```bash +dp test -m frozen_model.pt2 -s /path/to/system -n 30 +``` + +Check that `atype` follows the model `type_map` and that coordinates/cells use +the units and shapes documented by `DeepPot`. + +## Descriptors and DPA4 embeddings + +Generic descriptor evaluation remains available through `eval_descriptor` or: + +```bash +dp eval-desc -m frozen_model.pt2 -s /path/to/system -o desc_output +``` + +DPA4 additionally exposes model embeddings from a training checkpoint: + +```bash +dp embed -m model.ckpt.pt -s /path/to/system -o embedding.hdf5 +``` + +Do not substitute a `.pt2` archive in the documented `dp embed` checkpoint +workflow without checking the installed command's support. + +## Validation + +- Confirm that the artifact exists and can be loaded in the target environment. +- Inspect the stored descriptor when `.pt` could mean DPA3 or DPA4. +- Confirm the type map before constructing `atype`. +- Run a small finite energy/force/virial evaluation before a large batch. +- Validate an exported `.pt2` where it will be deployed; the suffix alone does + not guarantee environment compatibility. + +## References + +- [DPA4 export, inference, and embeddings](https://docs.deepmodeling.com/projects/deepmd/en/latest/model/dpa4.html) +- [Python inference](https://docs.deepmodeling.com/projects/deepmd/en/latest/inference/python.html) +- [Show model information](https://docs.deepmodeling.com/projects/deepmd/en/latest/model/show-model-info.html) diff --git a/skills/deepmd-train/SKILL.md b/skills/deepmd-train/SKILL.md index d38e7858d7..6d5c685678 100644 --- a/skills/deepmd-train/SKILL.md +++ b/skills/deepmd-train/SKILL.md @@ -1,11 +1,11 @@ --- name: deepmd-train -description: Train DeePMD-kit models with progressive disclosure. Use when the user wants to train a DeePMD-kit potential, prepare an input.json, choose between model families such as se_e2_a/DeepPot-SE and DPA3, run `dp train`, monitor learning curves, freeze checkpoints, or test trained models. Start with model selection and read only the selected model reference under `models/` when model-specific configuration is needed. +description: Train DeePMD-kit models with progressive disclosure. Use when the user wants to train a DeePMD-kit potential, prepare an input.json, choose between model families such as se_e2_a/DeepPot-SE, DPA3, and DPA4/SeZM, run `dp train`, monitor learning curves, freeze checkpoints, or test trained models. Start with model selection and read only the selected model reference under `models/` when model-specific configuration is needed. compatibility: Requires deepmd-kit installed. The selected backend and model may require PyTorch, TensorFlow, JAX, Paddle, GPU support, or custom OP libraries. license: LGPL-3.0-or-later metadata: author: iProzd - version: '1.1' + version: '1.2' repository: https://github.com/deepmodeling/deepmd-kit --- @@ -33,6 +33,7 @@ Available model references: | ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | [`models/se-e2-a.md`](models/se-e2-a.md) | The user wants a classical DeepPot-SE baseline, broad compatibility, or a smaller/established production model. | | [`models/dpa3.md`](models/dpa3.md) | The user wants a high-accuracy DPA3/LAM workflow, large/diverse datasets, dynamic neighbor selection, or pretrained DPA3-style training. | +| [`models/dpa4.md`](models/dpa4.md) | The user wants the PyTorch-only DPA4/SeZM SO(3)-equivariant architecture and its `.pt2` deployment path. | ## Model selection @@ -51,6 +52,7 @@ Recommended defaults: - Choose **se_e2_a** for a robust baseline, small to medium systems, compatibility-focused workflows, or when compute is limited. - Choose **DPA3** for high accuracy on diverse datasets, LAM-style training, or when the user explicitly asks for DPA3, DPA-3, LiGS, dynamic neighbor selection, or pretrained DPA3 variants. +- Choose **DPA4/SeZM** when the user explicitly requests it or wants its SO(3)-equivariant message-passing architecture and accepts a GPU-oriented, PyTorch-only workflow. ## Common workflow @@ -104,12 +106,16 @@ Training progress is usually written to `lcurve.out`. Check for: ### 6. Freeze and test +Read the selected model reference before choosing the output format. For +conventional PyTorch models, a typical flow is: + ```bash dp --pt freeze -o model.pth -dp --pt test -m model.pth -s /path/to/test_system -n 30 +dp test -m model.pth -s /path/to/test_system -n 30 ``` -Adjust the backend flags and output extension for non-PyTorch models. +DPA4/SeZM checkpoints instead freeze to `.pt2`; follow `models/dpa4.md`. +Adjust the backend and output format for other model families. ## Agent checklist diff --git a/skills/deepmd-train/models/dpa4.md b/skills/deepmd-train/models/dpa4.md new file mode 100644 index 0000000000..de74a1d9a8 --- /dev/null +++ b/skills/deepmd-train/models/dpa4.md @@ -0,0 +1,100 @@ +# DPA4 training reference + +Read this file only after the user chooses DPA4/SeZM, or when it is the best fit +for the task. Keep shared data checks and the train/monitor workflow in +`../SKILL.md`; this file records DPA4-specific choices. + +## When to choose DPA4 + +Choose DPA4 when the user explicitly requests DPA4/SeZM or wants its +SO(3)-equivariant message-passing architecture and accepts a GPU-oriented, +PyTorch-only workflow. The aliases `DPA4`, `SeZM`, and `sezm` select the same +implementation. + +DPA4 is not selected merely because a checkpoint ends in `.pt`. Inspect an +existing checkpoint with: + +```bash +dp --pt show model.pt descriptor fitting-net type-map +``` + +## Minimal model configuration + +Start from the maintained example at `examples/water/dpa4/input.json`. A minimal +model section is: + +```json +{ + "model": { + "type": "dpa4", + "type_map": [ + "O", + "H" + ], + "descriptor": { + "rcut": 6.0 + } + } +} +``` + +`model.type: "dpa4"` selects the DPA4/SeZM descriptor and its default energy +fitting network. DPA4 defaults to `float32`; double precision is unnecessary and +not recommended for the normal workflow. + +## Parameters to choose deliberately + +- `rcut` sets the local environment cutoff. +- On the conservative energy path, `sel` is an initial neighbor-search capacity + that grows on demand; it does not truncate the neighbor list. It may also be + set to `auto` or `auto:factor` from training data. +- `lmax`/`l_schedule` and `mmax`/`m_schedule` control angular resolution and are + primary accuracy-cost levers. +- `n_blocks` controls depth; `channels` and `n_radial` control width. +- `n_focus` and `n_atten_head` control aggregation. + +Use documented defaults or a maintained example unless the user has evidence for +changing these parameters. Do not copy DPA3 descriptor parameters into DPA4. + +## Train and monitor + +Use the PyTorch backend: + +```bash +dp --pt train input.json +``` + +Monitor `lcurve.out`, validation metrics, checkpoint creation, and non-finite +values. DPA4 also supports advanced property, spin, denoising, ZBL, multitask, +and LoRA configurations; follow the DPA4 documentation and examples rather than +combining those features from memory. For checkpoint adaptation and LoRA, use +the `deepmd-finetune-dpa4` skill. + +## Freeze and test + +DPA4 checkpoints are `.pt`, but deployment uses an AOTInductor `.pt2` archive: + +```bash +dp --pt freeze -c model.ckpt.pt -o frozen_model +dp test -m frozen_model.pt2 -s /path/to/test_system -n 30 +``` + +The command detects DPA4/SeZM and appends `.pt2`. DPA4 does not use the ordinary +TorchScript `.pth` freeze path and does not support model compression. Validate +the exported archive in the target inference or LAMMPS environment. + +## DPA4 checklist + +- [ ] The PyTorch backend is available. +- [ ] `model.type` is `dpa4`/`sezm`, or the stored checkpoint configuration proves it. +- [ ] `type_map`, data labels, and train/validation systems are consistent. +- [ ] Parameter changes are based on DPA4 documentation, not DPA3 defaults. +- [ ] Training and validation metrics are finite. +- [ ] The selected checkpoint is exported to `.pt2` and tested. +- [ ] `dp compress` is not used for DPA4. + +## References + +- [DPA4 model documentation](https://docs.deepmodeling.com/projects/deepmd/en/latest/model/dpa4.html) +- [DPA4 training example](../../../examples/water/dpa4/input.json) +- [Energy model training](https://docs.deepmodeling.com/projects/deepmd/en/latest/model/train-energy.html) diff --git a/skills/lammps-deepmd/SKILL.md b/skills/lammps-deepmd/SKILL.md index 4ec87c9adc..02dbaf7bcf 100644 --- a/skills/lammps-deepmd/SKILL.md +++ b/skills/lammps-deepmd/SKILL.md @@ -2,12 +2,12 @@ name: lammps-deepmd description: > A tool and knowledge base for running molecular dynamics (MD) simulations in LAMMPS with the DeePMD-kit plugin. It handles input script preparation, ensemble selection (NVE/NVT/NPT), and job execution via `uv` or offline binaries. - USE WHEN you need to set up, write, explain, or execute a LAMMPS molecular dynamics simulation using a DeePMD machine learning potential (e.g., `graph.pb`). + USE WHEN you need to set up, write, explain, or execute a LAMMPS molecular dynamics simulation using a DeePMD machine learning potential (for example `.pb`, `.pth`, or DPA4/SeZM `.pt2`). compatibility: Requires LAMMPS with DeePMD-kit support. Online mode prefers `uvx --from lammps --with deepmd-kit[gpu,torch,lmp] lmp`; offline mode requires a user-provided LAMMPS executable or module. license: LGPL-3.0-or-later metadata: author: OpenClaw - version: '1.0' + version: '1.1' repository: https://github.com/deepmodeling/deepmd-kit lammps_docs: https://docs.lammps.org/ --- @@ -24,7 +24,7 @@ Use this skill when the user wants to run molecular dynamics in LAMMPS with a De - **Offline mode**: do **not** guess the executable. Ask the user which LAMMPS command, module, or container should be used. 1. Confirm the minimum simulation inputs: - structure/data file (for example `data.system`) - - DeePMD model file (for example `graph.pb` or compressed model) + - DeePMD model artifact; read `references/model-deployment.md` for a training checkpoint, DPA4/SeZM, or an unclear export path - atom type to element mapping, including required per-type masses if the data file does not define them - target ensemble (NVE, NVT, NPT, or another explicitly requested setup) - temperature, pressure if applicable, timestep, and total number of steps @@ -81,6 +81,7 @@ Ask only for what is missing: ## Recommended workflow 1. Inspect available files in the working directory. +1. Read `references/model-deployment.md` when the model needs export, its artifact type is unclear, or explicit element mapping is required. 1. Draft `input.lammps`. 1. Explain the script to the user if they asked for an explanation or if the script is nontrivial. 1. Run a short smoke test first when reasonable. @@ -321,3 +322,4 @@ After a run, report at least: - DeePMD-kit: https://github.com/deepmodeling/deepmd-kit - User-provided tutorial reference: https://github.com/tongzhugroup/Chapter13-tutorial/blob/master/input.lammps - Detailed notes: `references/commands-and-workflow.md` +- Model artifact, export, and type-mapping notes: `references/model-deployment.md` diff --git a/skills/lammps-deepmd/references/commands-and-workflow.md b/skills/lammps-deepmd/references/commands-and-workflow.md index 7365057d17..da21fa129a 100644 --- a/skills/lammps-deepmd/references/commands-and-workflow.md +++ b/skills/lammps-deepmd/references/commands-and-workflow.md @@ -32,7 +32,7 @@ Then replace the short run with the intended production length. - `input.lammps`: input script - `data.system`: atomic structure and box -- `graph.pb` or `graph_compressed.pb`: DeePMD model +- a supported DeePMD deployment artifact such as `.pb`, `.pth`, or DPA4/SeZM `.pt2`; see `model-deployment.md` - `log.lammps`: main textual log - `traj.lammpstrj`: trajectory output diff --git a/skills/lammps-deepmd/references/model-deployment.md b/skills/lammps-deepmd/references/model-deployment.md new file mode 100644 index 0000000000..bbbc07aa50 --- /dev/null +++ b/skills/lammps-deepmd/references/model-deployment.md @@ -0,0 +1,89 @@ +# DeePMD model deployment in LAMMPS + +Read this reference when choosing a model artifact, exporting a checkpoint, or +mapping LAMMPS atom types to model elements. General simulation setup and +execution remain in `commands-and-workflow.md`. + +## Choose the model artifact + +A training checkpoint is not automatically a LAMMPS deployment artifact. The +`.pt` suffix also does not distinguish DPA3 from DPA4. Inspect an unfamiliar +PyTorch checkpoint before choosing an export path: + +```bash +dp --pt show model.pt descriptor fitting-net type-map +``` + +| Model artifact | Deployment route | +| --- | --- | +| TensorFlow frozen `.pb` | Use directly with a compatible `pair_style deepmd`. | +| Conventional PyTorch frozen `.pth` | Use directly with a compatible DeePMD-enabled LAMMPS build. | +| PyTorch checkpoint `.pt` | Inspect the stored model configuration and freeze using its model-specific route. | +| DPA4/SeZM archive `.pt2` | Use with a compatible DeePMD-enabled LAMMPS build. | + +Do not call a DPA4 `.pt2` archive a compressed model: DPA4 does not support +`dp compress`. + +## DPA4/SeZM deployment + +Freeze a DPA4/SeZM checkpoint with the standard PyTorch command: + +```bash +dp --pt freeze -c model.ckpt.pt -o frozen_model +``` + +The backend detects DPA4/SeZM and writes `frozen_model.pt2`. Validate the archive +in the actual target environment. + +A basic energy-model input uses: + +```lammps +atom_style atomic +atom_modify map yes +read_data data.system + +pair_style deepmd frozen_model.pt2 +pair_coeff * * O H +``` + +`atom_modify map yes` must appear before `read_data` for the documented DPA4 +route. Ordinary DPA4 energy models use `pair_style deepmd`; spin models may +require a different documented route and must not be treated as ordinary energy +models without inspection. + +## Atom-type mapping + +LAMMPS atom types are local integer IDs. Model types are ordered element names. +Build an explicit mapping instead of assuming the numbers are interchangeable. +For example: + +```lammps +mass 1 15.999 +mass 2 1.008 +pair_coeff * * O H +``` + +Here LAMMPS type 1 maps to `O` and type 2 maps to `H`. Require that: + +- every LAMMPS atom type has a mass; +- every mapped element is supported by the model type map; +- the `pair_coeff` element order matches LAMMPS type order; +- the structure's atom and species counts are unchanged during conversion. + +An implicit `pair_coeff * *` is acceptable only when the model and LAMMPS type +orders have been verified to match. Prefer explicit element names for auditable +workflows. + +## Pre-production validation + +1. Confirm the model loads without format or backend errors. +1. Run `run 0` or a bounded short run before production. +1. Require finite energy, force, pressure, and temperature where applicable. +1. Check atom count, masses, element mapping, box, and units when values are + anomalous or atoms are lost. +1. Preserve the generated LAMMPS input, model path, command, log, and exit code. + +## References + +- [DPA4 export and LAMMPS](https://docs.deepmodeling.com/projects/deepmd/en/latest/model/dpa4.html) +- [DeePMD-kit LAMMPS commands](https://docs.deepmodeling.com/projects/deepmd/en/latest/third-party/lammps-command.html) From 4eb0665ab579a88f2119c611986e1f0498c0fb17 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 7 Aug 2026 14:30:55 +0000 Subject: [PATCH 02/24] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- skills/deepmd-python-inference/SKILL.md | 6 +++--- .../references/model-artifacts.md | 12 ++++++------ skills/lammps-deepmd/SKILL.md | 10 ++++++---- skills/lammps-deepmd/references/model-deployment.md | 12 ++++++------ 4 files changed, 21 insertions(+), 19 deletions(-) diff --git a/skills/deepmd-python-inference/SKILL.md b/skills/deepmd-python-inference/SKILL.md index 42796b8c22..ec14fc6202 100644 --- a/skills/deepmd-python-inference/SKILL.md +++ b/skills/deepmd-python-inference/SKILL.md @@ -4,9 +4,9 @@ description: Run Python inference with DeePMD-kit models using the DeepPot API. compatibility: Requires deepmd-kit installed with the backend required by the selected model artifact. license: LGPL-3.0-or-later metadata: - author: iProzd - version: '1.1' - repository: https://github.com/deepmodeling/deepmd-kit + author: iProzd + version: '1.1' + repository: https://github.com/deepmodeling/deepmd-kit --- # DeePMD-kit Python Inference diff --git a/skills/deepmd-python-inference/references/model-artifacts.md b/skills/deepmd-python-inference/references/model-artifacts.md index e066ae5a44..79dffaea30 100644 --- a/skills/deepmd-python-inference/references/model-artifacts.md +++ b/skills/deepmd-python-inference/references/model-artifacts.md @@ -14,12 +14,12 @@ configuration when needed: dp --pt show model.pt descriptor fitting-net type-map ``` -| Artifact | Typical role | Inference guidance | -| --- | --- | --- | -| `.pb` | TensorFlow frozen model | Load with `DeepPot` or use `dp test`. | -| `.pth` | Conventional PyTorch frozen model | Load with `DeepPot` or use `dp test`. | -| `.pt` | PyTorch training checkpoint | Inspect before use. DPA4 supports eager Python evaluation and embedding extraction from a checkpoint; deployment normally uses a frozen artifact. | -| `.pt2` | DPA4/SeZM AOTInductor deployment archive | Load with `DeepPot`, use `dp test`, or deploy to a compatible LAMMPS build. | +| Artifact | Typical role | Inference guidance | +| -------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | +| `.pb` | TensorFlow frozen model | Load with `DeepPot` or use `dp test`. | +| `.pth` | Conventional PyTorch frozen model | Load with `DeepPot` or use `dp test`. | +| `.pt` | PyTorch training checkpoint | Inspect before use. DPA4 supports eager Python evaluation and embedding extraction from a checkpoint; deployment normally uses a frozen artifact. | +| `.pt2` | DPA4/SeZM AOTInductor deployment archive | Load with `DeepPot`, use `dp test`, or deploy to a compatible LAMMPS build. | Backend selection for inference is normally determined from the model artifact. Do not add a backend flag merely from the assumed model family. diff --git a/skills/lammps-deepmd/SKILL.md b/skills/lammps-deepmd/SKILL.md index 02dbaf7bcf..3b6b50beca 100644 --- a/skills/lammps-deepmd/SKILL.md +++ b/skills/lammps-deepmd/SKILL.md @@ -24,10 +24,12 @@ Use this skill when the user wants to run molecular dynamics in LAMMPS with a De - **Offline mode**: do **not** guess the executable. Ask the user which LAMMPS command, module, or container should be used. 1. Confirm the minimum simulation inputs: - structure/data file (for example `data.system`) - - DeePMD model artifact; read `references/model-deployment.md` for a training checkpoint, DPA4/SeZM, or an unclear export path - - atom type to element mapping, including required per-type masses if the data file does not define them - - target ensemble (NVE, NVT, NPT, or another explicitly requested setup) - - temperature, pressure if applicable, timestep, and total number of steps + +- DeePMD model artifact; read `references/model-deployment.md` for a training checkpoint, DPA4/SeZM, or an unclear export path +- atom type to element mapping, including required per-type masses if the data file does not define them +- target ensemble (NVE, NVT, NPT, or another explicitly requested setup) +- temperature, pressure if applicable, timestep, and total number of steps + 1. Write the LAMMPS input script yourself instead of asking the user to hand-write it. 1. Keep the example readable and fully explained. If you include an example input script, explain what **every command** does. 1. When possible, validate command availability against the LAMMPS docs or local `lmp -h` output before execution. diff --git a/skills/lammps-deepmd/references/model-deployment.md b/skills/lammps-deepmd/references/model-deployment.md index bbbc07aa50..077446ae11 100644 --- a/skills/lammps-deepmd/references/model-deployment.md +++ b/skills/lammps-deepmd/references/model-deployment.md @@ -14,12 +14,12 @@ PyTorch checkpoint before choosing an export path: dp --pt show model.pt descriptor fitting-net type-map ``` -| Model artifact | Deployment route | -| --- | --- | -| TensorFlow frozen `.pb` | Use directly with a compatible `pair_style deepmd`. | -| Conventional PyTorch frozen `.pth` | Use directly with a compatible DeePMD-enabled LAMMPS build. | -| PyTorch checkpoint `.pt` | Inspect the stored model configuration and freeze using its model-specific route. | -| DPA4/SeZM archive `.pt2` | Use with a compatible DeePMD-enabled LAMMPS build. | +| Model artifact | Deployment route | +| ---------------------------------- | --------------------------------------------------------------------------------- | +| TensorFlow frozen `.pb` | Use directly with a compatible `pair_style deepmd`. | +| Conventional PyTorch frozen `.pth` | Use directly with a compatible DeePMD-enabled LAMMPS build. | +| PyTorch checkpoint `.pt` | Inspect the stored model configuration and freeze using its model-specific route. | +| DPA4/SeZM archive `.pt2` | Use with a compatible DeePMD-enabled LAMMPS build. | Do not call a DPA4 `.pt2` archive a compressed model: DPA4 does not support `dp compress`. From 979ba46e3692d4c215aa2ef529450828c8e1afe3 Mon Sep 17 00:00:00 2001 From: Ming-Yu Guo Date: Fri, 7 Aug 2026 22:36:28 +0800 Subject: [PATCH 03/24] Refine description for deepmd-finetune-dpa4 skill Updated the description for the deepmd-finetune-dpa4 skill to simplify language and remove references to SeZM. Signed-off-by: Ming-Yu Guo --- skills/deepmd-finetune-dpa4/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/deepmd-finetune-dpa4/SKILL.md b/skills/deepmd-finetune-dpa4/SKILL.md index 0011706877..4ee41d4aeb 100644 --- a/skills/deepmd-finetune-dpa4/SKILL.md +++ b/skills/deepmd-finetune-dpa4/SKILL.md @@ -1,6 +1,6 @@ --- name: deepmd-finetune-dpa4 -description: Fine-tune a DPA4/SeZM model in DeePMD-kit with the PyTorch backend. Use for standard or LoRA fine-tuning from a DPA4/SeZM .pt checkpoint, then validation and .pt2 export. Do not select this skill from the .pt suffix alone; inspect the stored descriptor first. +description: Fine-tune a DPA4 model in DeePMD-kit. Use for standard or LoRA fine-tuning from a DPA4/SeZM .pt checkpoint, validation and .pt2 export. compatibility: Requires deepmd-kit with the PyTorch backend. DPA4/SeZM training is GPU-oriented. license: LGPL-3.0-or-later metadata: From a801ee6b37ec47f1581d1791ade2c5b13da4e806 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 7 Aug 2026 22:35:12 +0800 Subject: [PATCH 04/24] docs(skills): leave DPA3 skill unchanged --- skills/deepmd-finetune-dpa3/SKILL.md | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) diff --git a/skills/deepmd-finetune-dpa3/SKILL.md b/skills/deepmd-finetune-dpa3/SKILL.md index f39910e02a..d166676280 100644 --- a/skills/deepmd-finetune-dpa3/SKILL.md +++ b/skills/deepmd-finetune-dpa3/SKILL.md @@ -1,11 +1,11 @@ --- name: deepmd-finetune-dpa3 -description: Fine-tune a DPA3 model in DeePMD-kit using the PyTorch backend. Use when the user wants to adapt a pre-trained DPA3 model to a new downstream dataset. Supports fine-tuning from a self-trained DPA3 model (.pt checkpoint), from a multi-task pre-trained model, or from a built-in pretrained model downloaded via `dp pretrained download`. Do not select this skill from the .pt suffix alone; inspect the stored descriptor first. +description: Fine-tune a DPA3 model in DeePMD-kit using the PyTorch backend. Use when the user wants to adapt a pre-trained DPA3 model to a new downstream dataset. Supports fine-tuning from a self-trained DPA3 model (.pt checkpoint), from a multi-task pre-trained model, or from a built-in pretrained model downloaded via `dp pretrained download` (e.g., DPA-3.1-3M, DPA-3.2-5M, DPA-3.3-1M). Covers single-task and multi-task fine-tuning workflows. compatibility: Requires deepmd-kit with PyTorch backend installed. GPU strongly recommended. license: LGPL-3.0-or-later metadata: author: iProzd - version: '1.1' + version: '1.0' repository: https://github.com/deepmodeling/deepmd-kit --- @@ -17,17 +17,6 @@ Fine-tune a pre-trained DPA3 model on a downstream dataset. This skill covers th 1. Fine-tuning from a multi-task pre-trained DPA3 model 1. Fine-tuning from a built-in pretrained model (e.g., DPA-3.1-3M, DPA-3.2-5M, DPA-3.3-1M) downloaded via `dp pretrained download` -Both DPA3 and DPA4 checkpoints use the `.pt` suffix. If the user has not -already established the model family, inspect it before proceeding: - -```bash -dp --pt show pretrained.pt descriptor fitting-net type-map -``` - -Use this skill only when the stored descriptor identifies DPA3. Use -`deepmd-finetune-dpa4` for DPA4/SeZM, and stop when the family cannot be -established from the checkpoint. - ## Quick Start ```bash From 6e27dc225fae7fa4e80a1af0e4c6676802fe574f Mon Sep 17 00:00:00 2001 From: root Date: Fri, 7 Aug 2026 23:20:17 +0800 Subject: [PATCH 05/24] docs(skills): clarify pt2 inference limits --- skills/deepmd-python-inference/SKILL.md | 6 ++++- .../references/model-artifacts.md | 26 ++++++++++++------- 2 files changed, 21 insertions(+), 11 deletions(-) diff --git a/skills/deepmd-python-inference/SKILL.md b/skills/deepmd-python-inference/SKILL.md index ec14fc6202..060f60ba8d 100644 --- a/skills/deepmd-python-inference/SKILL.md +++ b/skills/deepmd-python-inference/SKILL.md @@ -1,6 +1,6 @@ --- name: deepmd-python-inference -description: Run Python inference with DeePMD-kit models using the DeepPot API. Use when the user wants to load a checkpoint, frozen model (.pb, .pth, or DPA4 .pt2), or built-in pretrained model in Python; predict energy/force/virial; evaluate descriptors; calculate model deviation; or use `dp test` against labeled data. +description: Run Python inference with DeePMD-kit models using the DeepPot API. Use when the user wants to load a checkpoint, frozen model (.pb or .pth), DPA4/SeZM AOTInductor deployment archive (.pt2), or built-in pretrained model in Python; predict energy/force/virial; evaluate supported descriptors; calculate model deviation; or use `dp test` against labeled data. compatibility: Requires deepmd-kit installed with the backend required by the selected model artifact. license: LGPL-3.0-or-later metadata: @@ -39,6 +39,9 @@ e, f, v = dp.eval(coord, cell, atype) - Descriptor evaluation - Model deviation calculation - CLI-based testing against labeled data +1. Before using descriptor or embedding hooks, confirm that the selected + artifact supports them; a loadable `.pt2` does not necessarily contain the + serialized model required by those hooks. 1. Help the user prepare input arrays in the correct format. 1. Run inference and report results. @@ -291,6 +294,7 @@ dp pretrained download DPA-3.2-5M --cache-dir ./models - [ ] Model file exists and is accessible (`.pb`, `.pth`, `.pt`, `.pt2`, or valid pretrained name) - [ ] An ambiguous `.pt` checkpoint was classified from its stored configuration, not its filename +- [ ] The requested descriptor or embedding operation is supported by the specific artifact, not inferred from its suffix - [ ] `coord` array is shaped (nframes, natoms\*3) and in Angstrom - [ ] `cell` array is shaped (nframes, 9) or `None` for non-periodic systems - [ ] `atype` indices match the model's `type_map` ordering diff --git a/skills/deepmd-python-inference/references/model-artifacts.md b/skills/deepmd-python-inference/references/model-artifacts.md index 79dffaea30..711d665c73 100644 --- a/skills/deepmd-python-inference/references/model-artifacts.md +++ b/skills/deepmd-python-inference/references/model-artifacts.md @@ -19,7 +19,7 @@ dp --pt show model.pt descriptor fitting-net type-map | `.pb` | TensorFlow frozen model | Load with `DeepPot` or use `dp test`. | | `.pth` | Conventional PyTorch frozen model | Load with `DeepPot` or use `dp test`. | | `.pt` | PyTorch training checkpoint | Inspect before use. DPA4 supports eager Python evaluation and embedding extraction from a checkpoint; deployment normally uses a frozen artifact. | -| `.pt2` | DPA4/SeZM AOTInductor deployment archive | Load with `DeepPot`, use `dp test`, or deploy to a compatible LAMMPS build. | +| `.pt2` | AOTInductor deployment archive | Use supported inference paths in a compatible runtime; the suffix alone does not imply descriptor hooks, portability, or multi-rank support. | Backend selection for inference is normally determined from the model artifact. Do not add a backend flag merely from the assumed model family. @@ -49,16 +49,22 @@ For labeled data: dp test -m frozen_model.pt2 -s /path/to/system -n 30 ``` +`DeepPot.eval` on DPA4/SeZM `.pt2` archives is covered for energy, force, +virial, atomic energy, and atomic virial. `dp test` uses the same model dispatch. +Both require an installed DeePMD-kit/PyTorch runtime compatible with the +compiled archive. + Check that `atype` follows the model `type_map` and that coordinates/cells use the units and shapes documented by `DeepPot`. ## Descriptors and DPA4 embeddings -Generic descriptor evaluation remains available through `eval_descriptor` or: - -```bash -dp eval-desc -m frozen_model.pt2 -s /path/to/system -o desc_output -``` +Descriptor evaluation is conditional for `.pt2`. It requires an archive that +contains the serialized `model.json`; metadata-only archives can run the main +`DeepPot.eval` path but raise `NotImplementedError` for `eval_descriptor`. +In particular, do not run `dp eval-desc` on a DPA4 `.pt2` produced by the +`dp --pt freeze` command above, because that export is metadata-only. Use a +supported checkpoint or verify the archive contents and backend first. DPA4 additionally exposes model embeddings from a training checkpoint: @@ -66,8 +72,7 @@ DPA4 additionally exposes model embeddings from a training checkpoint: dp embed -m model.ckpt.pt -s /path/to/system -o embedding.hdf5 ``` -Do not substitute a `.pt2` archive in the documented `dp embed` checkpoint -workflow without checking the installed command's support. +`dp embed` supports the DPA4/SeZM `.pt` checkpoint and does not support `.pt2`. ## Validation @@ -75,8 +80,9 @@ workflow without checking the installed command's support. - Inspect the stored descriptor when `.pt` could mean DPA3 or DPA4. - Confirm the type map before constructing `atype`. - Run a small finite energy/force/virial evaluation before a large batch. -- Validate an exported `.pt2` where it will be deployed; the suffix alone does - not guarantee environment compatibility. +- Treat `.pt2` as a compiled deployment artifact, not a portable checkpoint. + Export and validate it with a device and toolchain compatible with the final + Python, C++, or LAMMPS runtime. ## References From dc8ec953ee0064968b7153b1dc2b8d8056cdaca5 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 7 Aug 2026 23:21:26 +0800 Subject: [PATCH 06/24] docs(skills): preserve selected DPA4 heads --- skills/deepmd-finetune-dpa4/SKILL.md | 23 ++++++++++++++++++- .../references/model-artifacts.md | 2 ++ skills/deepmd-train/models/dpa4.md | 10 ++++++++ 3 files changed, 34 insertions(+), 1 deletion(-) diff --git a/skills/deepmd-finetune-dpa4/SKILL.md b/skills/deepmd-finetune-dpa4/SKILL.md index 4ee41d4aeb..a44878f0cb 100644 --- a/skills/deepmd-finetune-dpa4/SKILL.md +++ b/skills/deepmd-finetune-dpa4/SKILL.md @@ -55,6 +55,18 @@ pretrained-script mechanism is deliberately used: dp --pt train input.json --finetune pretrained.pt ``` +When fine-tuning a single-task target from a multi-task checkpoint and the +intent is to preserve a particular pretrained fitting head, pass the branch +selected above: + +```bash +dp --pt train input.json --finetune pretrained.pt --model-branch SELECTED_BRANCH +``` + +If `--model-branch` is omitted, the fitting net may be initialized from the +`RANDOM` branch instead. A multi-task target uses `finetune_head` in each target +branch rather than the command-line option. + If the architecture is unknown, `--use-pretrain-script` can inherit the stored model configuration except for `type_map`: @@ -90,7 +102,7 @@ dp --pt train lora_ft.json --finetune pretrained.pt ``` The JSON fragment above is not a complete training input. Adapt the full public -example at `examples/water/dpa4/lora_ft.json`. Do not add +example at `../../examples/water/dpa4/lora_ft.json`. Do not add `--use-pretrain-script` to this LoRA command unless a targeted test confirms that the intended LoRA configuration is retained. @@ -114,6 +126,15 @@ dp test -m finetuned_model.pt2 -s /path/to/test_system -n 30 The freeze command detects DPA4/SeZM and writes `finetuned_model.pt2`. Validate the exported archive in the target environment before deployment. +For a multi-task checkpoint, freeze the selected head explicitly: + +```bash +dp --pt freeze -c ckpt/model.ckpt.pt -o finetuned_model --head SELECTED_BRANCH +``` + +The resulting `.pt2` contains the selected single head; do not pass a branch +again when loading that archive. + ## Checklist - [ ] The stored descriptor identifies DPA4/SeZM; the `.pt` suffix was not used as proof. diff --git a/skills/deepmd-python-inference/references/model-artifacts.md b/skills/deepmd-python-inference/references/model-artifacts.md index 711d665c73..640fc87273 100644 --- a/skills/deepmd-python-inference/references/model-artifacts.md +++ b/skills/deepmd-python-inference/references/model-artifacts.md @@ -34,6 +34,8 @@ dp --pt freeze -c model.ckpt.pt -o frozen_model ``` The command writes `frozen_model.pt2` for a detected DPA4/SeZM checkpoint. +For a multi-task checkpoint, select the head during export with +`--head SELECTED_BRANCH`; the resulting `.pt2` is already single-head. Evaluate the archive with: ```python diff --git a/skills/deepmd-train/models/dpa4.md b/skills/deepmd-train/models/dpa4.md index de74a1d9a8..ce1c9197aa 100644 --- a/skills/deepmd-train/models/dpa4.md +++ b/skills/deepmd-train/models/dpa4.md @@ -83,6 +83,16 @@ The command detects DPA4/SeZM and appends `.pt2`. DPA4 does not use the ordinary TorchScript `.pth` freeze path and does not support model compression. Validate the exported archive in the target inference or LAMMPS environment. +If the checkpoint is multi-task, inspect its branches and pass the selected +head during export: + +```bash +dp --pt show model.ckpt.pt model-branch descriptor type-map +dp --pt freeze -c model.ckpt.pt -o frozen_model --head SELECTED_BRANCH +``` + +The frozen `.pt2` is a selected single-head artifact. + ## DPA4 checklist - [ ] The PyTorch backend is available. From 92476d3db82a73a0c52c6f241b2b2c4b03ddbce0 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 7 Aug 2026 23:22:53 +0800 Subject: [PATCH 07/24] docs(skills): qualify DPA4 pt2 deployment --- .../references/model-deployment.md | 26 +++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/skills/lammps-deepmd/references/model-deployment.md b/skills/lammps-deepmd/references/model-deployment.md index 077446ae11..2bd4963138 100644 --- a/skills/lammps-deepmd/references/model-deployment.md +++ b/skills/lammps-deepmd/references/model-deployment.md @@ -19,7 +19,7 @@ dp --pt show model.pt descriptor fitting-net type-map | TensorFlow frozen `.pb` | Use directly with a compatible `pair_style deepmd`. | | Conventional PyTorch frozen `.pth` | Use directly with a compatible DeePMD-enabled LAMMPS build. | | PyTorch checkpoint `.pt` | Inspect the stored model configuration and freeze using its model-specific route. | -| DPA4/SeZM archive `.pt2` | Use with a compatible DeePMD-enabled LAMMPS build. | +| AOTInductor archive `.pt2` | Inspect its metadata and use only with a compatible DeePMD-enabled LAMMPS build. | Do not call a DPA4 `.pt2` archive a compressed model: DPA4 does not support `dp compress`. @@ -33,7 +33,19 @@ dp --pt freeze -c model.ckpt.pt -o frozen_model ``` The backend detects DPA4/SeZM and writes `frozen_model.pt2`. Validate the archive -in the actual target environment. +in the actual target environment. For a multi-task checkpoint, select the head +during export: + +```bash +dp --pt freeze -c model.ckpt.pt -o frozen_model --head SELECTED_BRANCH +``` + +Two DPA4 `.pt2` export contracts exist. `dp --pt freeze` uses the DPA4-specific +`edge_vec` ABI. `dp --pt_expt freeze --lower-kind graph` uses the NeighborGraph +ABI. They share a suffix but are not interchangeable contracts, and a `.pt2` +suffix alone does not prove multi-rank support. A multi-rank archive must report +`has_comm_artifact=true` and contain +`model/extra/forward_lower_with_comm.pt2`. A basic energy-model input uses: @@ -51,6 +63,11 @@ route. Ordinary DPA4 energy models use `pair_style deepmd`; spin models may require a different documented route and must not be treated as ordinary energy models without inspection. +Single-rank DPA4 execution is covered for supported `edge_vec`, graph, and +dense/nlist archives. Multi-rank execution is supported only when the archive +contains the with-communication artifact required by its ABI; fail closed when +that metadata or nested artifact is absent. + ## Atom-type mapping LAMMPS atom types are local integer IDs. Model types are ordered element names. @@ -77,6 +94,11 @@ workflows. ## Pre-production validation 1. Confirm the model loads without format or backend errors. +1. Treat `.pt2` as a device/toolchain-compiled artifact. Verify that the export + and final LAMMPS runtime use compatible PyTorch, DeePMD-kit, device, and C++ + interfaces. +1. For multi-rank execution, verify the archive's communication metadata and + nested with-comm artifact before launching MPI. 1. Run `run 0` or a bounded short run before production. 1. Require finite energy, force, pressure, and temperature where applicable. 1. Check atom count, masses, element mapping, box, and units when values are From 0e24272f99ab8e54bda22c2629093b8d1008bd43 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 7 Aug 2026 23:26:56 +0800 Subject: [PATCH 08/24] docs(skills): pin online LAMMPS runtime --- skills/lammps-deepmd/SKILL.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/skills/lammps-deepmd/SKILL.md b/skills/lammps-deepmd/SKILL.md index 3b6b50beca..2ebf087d80 100644 --- a/skills/lammps-deepmd/SKILL.md +++ b/skills/lammps-deepmd/SKILL.md @@ -3,7 +3,7 @@ name: lammps-deepmd description: > A tool and knowledge base for running molecular dynamics (MD) simulations in LAMMPS with the DeePMD-kit plugin. It handles input script preparation, ensemble selection (NVE/NVT/NPT), and job execution via `uv` or offline binaries. USE WHEN you need to set up, write, explain, or execute a LAMMPS molecular dynamics simulation using a DeePMD machine learning potential (for example `.pb`, `.pth`, or DPA4/SeZM `.pt2`). -compatibility: Requires LAMMPS with DeePMD-kit support. Online mode prefers `uvx --from lammps --with deepmd-kit[gpu,torch,lmp] lmp`; offline mode requires a user-provided LAMMPS executable or module. +compatibility: Requires LAMMPS with DeePMD-kit support. Online mode uses `uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp`; offline mode requires a user-provided LAMMPS executable or module. license: LGPL-3.0-or-later metadata: author: OpenClaw @@ -20,7 +20,7 @@ Use this skill when the user wants to run molecular dynamics in LAMMPS with a De 1. Confirm the available execution mode: - **Online mode**: if internet access is available and `uv` is installed, prefer - `uvx --from lammps --with deepmd-kit[gpu,torch,lmp] lmp ...` + `uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp ...` - **Offline mode**: do **not** guess the executable. Ask the user which LAMMPS command, module, or container should be used. 1. Confirm the minimum simulation inputs: - structure/data file (for example `data.system`) @@ -42,19 +42,21 @@ Use this skill when the user wants to run molecular dynamics in LAMMPS with a De Use: ```bash -uvx --from lammps --with deepmd-kit[gpu,torch,lmp] lmp -in input.lammps +uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -in input.lammps ``` If you need to inspect the local command-line help: ```bash -uvx --from lammps --with deepmd-kit[gpu,torch,lmp] lmp -h | tee /dev/tty +uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -h | tee /dev/tty ``` Notes: - This is the preferred path because it can provision LAMMPS and DeePMD-kit on demand. -- The `gpu,torch,lmp` extras match the requested runtime pattern from the user. +- The pins match the LAMMPS dependency declared by the DPA4-capable + `deepmd-kit==3.2.0b0` release. Update both pins together after validating a + newer pair. - If the environment is slow or the packages are large, warn the user that the first run may take time. ### Offline mode @@ -283,13 +285,13 @@ When using NPT, it is often useful to keep `vol`, `lx`, `ly`, and `lz` in the th ### Online run ```bash -uvx --from lammps --with deepmd-kit[gpu,torch,lmp] lmp -in input.lammps +uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -in input.lammps ``` ### Online help ```bash -uvx --from lammps --with deepmd-kit[gpu,torch,lmp] lmp -h | tee /dev/tty +uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -h | tee /dev/tty ``` ### Offline run From c41e68af5968f3e38145ad5cda5b9e77f12d9ef5 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 7 Aug 2026 15:29:29 +0000 Subject: [PATCH 09/24] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- skills/deepmd-python-inference/SKILL.md | 4 ++-- .../references/model-artifacts.md | 12 ++++++------ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/skills/deepmd-python-inference/SKILL.md b/skills/deepmd-python-inference/SKILL.md index 060f60ba8d..f3cb9e148e 100644 --- a/skills/deepmd-python-inference/SKILL.md +++ b/skills/deepmd-python-inference/SKILL.md @@ -40,8 +40,8 @@ e, f, v = dp.eval(coord, cell, atype) - Model deviation calculation - CLI-based testing against labeled data 1. Before using descriptor or embedding hooks, confirm that the selected - artifact supports them; a loadable `.pt2` does not necessarily contain the - serialized model required by those hooks. + artifact supports them; a loadable `.pt2` does not necessarily contain the + serialized model required by those hooks. 1. Help the user prepare input arrays in the correct format. 1. Run inference and report results. diff --git a/skills/deepmd-python-inference/references/model-artifacts.md b/skills/deepmd-python-inference/references/model-artifacts.md index 640fc87273..9e86824cbd 100644 --- a/skills/deepmd-python-inference/references/model-artifacts.md +++ b/skills/deepmd-python-inference/references/model-artifacts.md @@ -14,12 +14,12 @@ configuration when needed: dp --pt show model.pt descriptor fitting-net type-map ``` -| Artifact | Typical role | Inference guidance | -| -------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `.pb` | TensorFlow frozen model | Load with `DeepPot` or use `dp test`. | -| `.pth` | Conventional PyTorch frozen model | Load with `DeepPot` or use `dp test`. | -| `.pt` | PyTorch training checkpoint | Inspect before use. DPA4 supports eager Python evaluation and embedding extraction from a checkpoint; deployment normally uses a frozen artifact. | -| `.pt2` | AOTInductor deployment archive | Use supported inference paths in a compatible runtime; the suffix alone does not imply descriptor hooks, portability, or multi-rank support. | +| Artifact | Typical role | Inference guidance | +| -------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | +| `.pb` | TensorFlow frozen model | Load with `DeepPot` or use `dp test`. | +| `.pth` | Conventional PyTorch frozen model | Load with `DeepPot` or use `dp test`. | +| `.pt` | PyTorch training checkpoint | Inspect before use. DPA4 supports eager Python evaluation and embedding extraction from a checkpoint; deployment normally uses a frozen artifact. | +| `.pt2` | AOTInductor deployment archive | Use supported inference paths in a compatible runtime; the suffix alone does not imply descriptor hooks, portability, or multi-rank support. | Backend selection for inference is normally determined from the model artifact. Do not add a backend flag merely from the assumed model family. From adc861655503bd9c1a43e093563ef373bf3d5cb2 Mon Sep 17 00:00:00 2001 From: root Date: Sat, 8 Aug 2026 00:16:32 +0800 Subject: [PATCH 10/24] docs(skills): qualify pt2 atomic outputs --- .../references/model-artifacts.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/skills/deepmd-python-inference/references/model-artifacts.md b/skills/deepmd-python-inference/references/model-artifacts.md index 9e86824cbd..038053ea6a 100644 --- a/skills/deepmd-python-inference/references/model-artifacts.md +++ b/skills/deepmd-python-inference/references/model-artifacts.md @@ -52,9 +52,11 @@ dp test -m frozen_model.pt2 -s /path/to/system -n 30 ``` `DeepPot.eval` on DPA4/SeZM `.pt2` archives is covered for energy, force, -virial, atomic energy, and atomic virial. `dp test` uses the same model dispatch. -Both require an installed DeePMD-kit/PyTorch runtime compatible with the -compiled archive. +virial, and atomic energy. Atomic virial is available only when the archive +metadata reports `do_atomic_virial=true`; the ordinary non-spin +`dp --pt freeze` route enables it, but specialized spin or conversion routes +may not. `dp test` uses the same model dispatch. Both require an installed +DeePMD-kit/PyTorch runtime compatible with the compiled archive. Check that `atype` follows the model `type_map` and that coordinates/cells use the units and shapes documented by `DeepPot`. @@ -75,6 +77,11 @@ dp embed -m model.ckpt.pt -s /path/to/system -o embedding.hdf5 ``` `dp embed` supports the DPA4/SeZM `.pt` checkpoint and does not support `.pt2`. +For a multi-task checkpoint, preserve the selected head explicitly: + +```bash +dp embed -m model.ckpt.pt -s /path/to/system -o embedding.hdf5 --head SELECTED_BRANCH +``` ## Validation From 263c3408fee4ee974ca4ee6f903485238acf8d8f Mon Sep 17 00:00:00 2001 From: root Date: Sat, 8 Aug 2026 00:17:26 +0800 Subject: [PATCH 11/24] docs(skills): make LAMMPS help noninteractive --- skills/lammps-deepmd/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/skills/lammps-deepmd/SKILL.md b/skills/lammps-deepmd/SKILL.md index 2ebf087d80..7957a2f5f2 100644 --- a/skills/lammps-deepmd/SKILL.md +++ b/skills/lammps-deepmd/SKILL.md @@ -48,7 +48,7 @@ uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' l If you need to inspect the local command-line help: ```bash -uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -h | tee /dev/tty +uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -h ``` Notes: @@ -291,7 +291,7 @@ uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' l ### Online help ```bash -uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -h | tee /dev/tty +uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -h ``` ### Offline run From 588023caa9df011fae0f5feda2a503e65e84d414 Mon Sep 17 00:00:00 2001 From: SchrodingersCattt Date: Sat, 8 Aug 2026 08:08:25 +0800 Subject: [PATCH 12/24] docs(skills): clarify DPA4 checkpoint and LoRA selection Document how to obtain and verify compatible DPA4 checkpoints, decide between standard fine-tuning and LoRA, inspect active adapter state, distinguish merged best checkpoints, and avoid the incompatible bundled checkpoint/example pairing. --- skills/deepmd-finetune-dpa4/SKILL.md | 127 ++++++++++++++++++++++++++- 1 file changed, 124 insertions(+), 3 deletions(-) diff --git a/skills/deepmd-finetune-dpa4/SKILL.md b/skills/deepmd-finetune-dpa4/SKILL.md index a44878f0cb..f1ac9194d3 100644 --- a/skills/deepmd-finetune-dpa4/SKILL.md +++ b/skills/deepmd-finetune-dpa4/SKILL.md @@ -35,6 +35,41 @@ dp --pt show pretrained.pt model-branch descriptor type-map Do not guess a branch. Use `deepmd-finetune-dpa3` instead when the descriptor is DPA3, and stop when the family cannot be established. +## Obtain a pretrained checkpoint + +Fine-tuning requires a DPA4/SeZM **training checkpoint** (`.pt`), not a compiled +`.pt2` deployment archive. First check the registry exposed by the installed +version: + +```bash +dp pretrained download -h +``` + +Use a built-in model only when an exact DPA4/SeZM name is listed there. Do not +guess `dp pretrained download DPA4`: some releases have no registered DPA4 +checkpoint. + +For a workflow smoke test, a DeePMD-kit source checkout contains: + +```text +examples/water/dpa4/lmp/pretrained.pt +``` + +Use that file directly: + +```bash +cp examples/water/dpa4/lmp/pretrained.pt ./pretrained.pt +dp --pt show pretrained.pt descriptor fitting-net type-map +``` + +It is a compact O/H smoke-test model, not a general-purpose pretrained +potential. For scientific fine-tuning, obtain a checkpoint from its documented +publisher or train one with the same DeePMD-kit revision that will perform the +fine-tuning. Pin and record the model source, checksum, DeePMD-kit revision, +architecture, task branch, and `type_map`. Reject a checkpoint whose descriptor, +element set/order, or architecture does not match the intended target. Prefer a +bounded one-step compatibility run before a long job. + ## Before fine-tuning 1. Confirm the checkpoint exists and can be inspected. @@ -46,6 +81,86 @@ DPA3, and stop when the family cannot be established. 1. Choose standard fine-tuning or LoRA. Do not assume a built-in DPA4 model name; check `dp pretrained download -h` for the installed version. +## Decide whether to use LoRA + +Use **standard fine-tuning** by default when the target is multi-task, the domain +shift is large, all parameters should adapt, or the workflow combines untested +spin/property/denoising/ZBL changes. Use **LoRA** when the target is single-task, +parameter-efficient adaptation is desired, the downstream domain is reasonably +close to pretraining, and the exact base architecture is known. DPA4 LoRA is +supported by `dp --pt`; do not use it with the exportable training backend. + +The pretrained checkpoint does not need to contain LoRA. LoRA is enabled by the +new fine-tuning input through a non-null `model.lora` block. Check an input with: + +```bash +python - input.json <<'PY' +import json +import sys + +with open(sys.argv[1], encoding="utf-8") as stream: + config = json.load(stream) + +model = config.get("model", {}) +branches = model.get("model_dict") +lora = model.get("lora") +print("multi_task =", isinstance(branches, dict)) +print("lora =", lora) +print("use_lora =", lora is not None) +if isinstance(branches, dict) and lora is not None: + raise SystemExit("DPA4 LoRA targets must be single-task") +PY +``` + +Interpretation: + +- absent or `null` `model.lora` means standard fine-tuning; +- a `model.lora` mapping means LoRA fine-tuning; +- a multi-task target plus LoRA is unsupported. + +To diagnose whether a `.pt` file still contains **active, unmerged** LoRA state, +inspect its saved model parameters and adapter tensors without executing pickled +code: + +```bash +python - pretrained.pt <<'PY' +import sys +import torch + +raw = torch.load(sys.argv[1], map_location="cpu", weights_only=True) +state = raw["model"] if isinstance(raw, dict) and "model" in raw else raw +if not isinstance(state, dict): + raise TypeError(f"Unsupported checkpoint payload: {type(state).__name__}") + +extra = state.get("_extra_state", {}) +params = extra.get("model_params", {}) if isinstance(extra, dict) else {} +configured = params.get("lora") if isinstance(params, dict) else None +markers = (".A_by_l", ".B_by_l", ".A_m0", ".B_m0", ".A_m.", ".B_m.", ".lora_scaling") +adapter_keys = sorted( + key for key in state if any(marker in key for marker in markers) +) + +print("configured_lora =", configured) +print("adapter_tensor_count =", len(adapter_keys)) +for key in adapter_keys[:20]: + print("adapter_tensor =", key) + +if configured is not None and adapter_keys: + print("classification = active/unmerged LoRA checkpoint") +elif configured is not None or adapter_keys: + print("classification = inconsistent or transitional; inspect before use") +else: + print("classification = plain checkpoint or merged LoRA checkpoint") +PY +``` + +Do not infer training history from the last classification. Validation-selected +best LoRA checkpoints fold adapter deltas into ordinary DPA4 weights and remove +LoRA metadata/tensors, so they intentionally look plain. Such merged checkpoints +are suitable for evaluation, new fine-tuning, and `.pt2` export, but do not carry +the optimizer/EMA state needed to resume the same LoRA run. Periodic/final LoRA +checkpoints retain active adapters and are the resumable form. + ## Standard fine-tuning The model section in `input.json` must match the checkpoint unless the standard @@ -102,9 +217,12 @@ dp --pt train lora_ft.json --finetune pretrained.pt ``` The JSON fragment above is not a complete training input. Adapt the full public -example at `../../examples/water/dpa4/lora_ft.json`. Do not add -`--use-pretrain-script` to this LoRA command unless a targeted test confirms that -the intended LoRA configuration is retained. +example at `../../examples/water/dpa4/lora_ft.json`, but copy the architecture +from the actual source checkpoint before adding `model.lora`. The compact +`examples/water/dpa4/lmp/pretrained.pt` does not match the larger architecture +in the maintained `lora_ft.json` and must not be paired with it unchanged. Do +not add `--use-pretrain-script` to this LoRA command unless a targeted test +confirms that the intended LoRA configuration is retained. ## Monitor and validate @@ -138,9 +256,12 @@ again when loading that archive. ## Checklist - [ ] The stored descriptor identifies DPA4/SeZM; the `.pt` suffix was not used as proof. +- [ ] The checkpoint source, checksum, DeePMD-kit revision, architecture, and type map are recorded. - [ ] The intended branch is explicit for a multi-task checkpoint. - [ ] Training, validation, and held-out test systems are separate. - [ ] The input architecture is compatible with the checkpoint. +- [ ] Standard fine-tuning versus LoRA was selected from the task layout and domain shift. +- [ ] Active LoRA state versus a merged best checkpoint is interpreted correctly. - [ ] LoRA uses a complete base configuration and is not silently overwritten. - [ ] Training and held-out metrics are finite and reported with units. - [ ] The selected `.pt` checkpoint was exported to and tested as `.pt2`. From d5e2d6c66d6541ba312ca0a856bf0f24e968aec3 Mon Sep 17 00:00:00 2001 From: SchrodingersCattt Date: Sat, 8 Aug 2026 08:40:16 +0800 Subject: [PATCH 13/24] docs(skills): streamline DPA4 finetuning guidance Remove smoke-model-specific discussion and detailed checkpoint inspection scripts, retaining concise general requirements for checkpoint selection and LoRA usage. --- skills/deepmd-finetune-dpa4/SKILL.md | 129 ++++----------------------- 1 file changed, 18 insertions(+), 111 deletions(-) diff --git a/skills/deepmd-finetune-dpa4/SKILL.md b/skills/deepmd-finetune-dpa4/SKILL.md index f1ac9194d3..30a6d9ef55 100644 --- a/skills/deepmd-finetune-dpa4/SKILL.md +++ b/skills/deepmd-finetune-dpa4/SKILL.md @@ -37,38 +37,16 @@ DPA3, and stop when the family cannot be established. ## Obtain a pretrained checkpoint -Fine-tuning requires a DPA4/SeZM **training checkpoint** (`.pt`), not a compiled -`.pt2` deployment archive. First check the registry exposed by the installed -version: +Fine-tuning requires a DPA4/SeZM training checkpoint (`.pt`), not a `.pt2` +deployment archive. Check whether the installed version provides one: ```bash dp pretrained download -h ``` -Use a built-in model only when an exact DPA4/SeZM name is listed there. Do not -guess `dp pretrained download DPA4`: some releases have no registered DPA4 -checkpoint. - -For a workflow smoke test, a DeePMD-kit source checkout contains: - -```text -examples/water/dpa4/lmp/pretrained.pt -``` - -Use that file directly: - -```bash -cp examples/water/dpa4/lmp/pretrained.pt ./pretrained.pt -dp --pt show pretrained.pt descriptor fitting-net type-map -``` - -It is a compact O/H smoke-test model, not a general-purpose pretrained -potential. For scientific fine-tuning, obtain a checkpoint from its documented -publisher or train one with the same DeePMD-kit revision that will perform the -fine-tuning. Pin and record the model source, checksum, DeePMD-kit revision, -architecture, task branch, and `type_map`. Reject a checkpoint whose descriptor, -element set/order, or architecture does not match the intended target. Prefer a -bounded one-step compatibility run before a long job. +Use only a listed model or a checkpoint supplied by the user or its publisher. +Record its source and DeePMD-kit version, then verify its descriptor, branch, +architecture, and `type_map` before use. ## Before fine-tuning @@ -83,83 +61,15 @@ bounded one-step compatibility run before a long job. ## Decide whether to use LoRA -Use **standard fine-tuning** by default when the target is multi-task, the domain -shift is large, all parameters should adapt, or the workflow combines untested -spin/property/denoising/ZBL changes. Use **LoRA** when the target is single-task, -parameter-efficient adaptation is desired, the downstream domain is reasonably -close to pretraining, and the exact base architecture is known. DPA4 LoRA is -supported by `dp --pt`; do not use it with the exportable training backend. - -The pretrained checkpoint does not need to contain LoRA. LoRA is enabled by the -new fine-tuning input through a non-null `model.lora` block. Check an input with: - -```bash -python - input.json <<'PY' -import json -import sys - -with open(sys.argv[1], encoding="utf-8") as stream: - config = json.load(stream) - -model = config.get("model", {}) -branches = model.get("model_dict") -lora = model.get("lora") -print("multi_task =", isinstance(branches, dict)) -print("lora =", lora) -print("use_lora =", lora is not None) -if isinstance(branches, dict) and lora is not None: - raise SystemExit("DPA4 LoRA targets must be single-task") -PY -``` - -Interpretation: - -- absent or `null` `model.lora` means standard fine-tuning; -- a `model.lora` mapping means LoRA fine-tuning; -- a multi-task target plus LoRA is unsupported. - -To diagnose whether a `.pt` file still contains **active, unmerged** LoRA state, -inspect its saved model parameters and adapter tensors without executing pickled -code: - -```bash -python - pretrained.pt <<'PY' -import sys -import torch - -raw = torch.load(sys.argv[1], map_location="cpu", weights_only=True) -state = raw["model"] if isinstance(raw, dict) and "model" in raw else raw -if not isinstance(state, dict): - raise TypeError(f"Unsupported checkpoint payload: {type(state).__name__}") - -extra = state.get("_extra_state", {}) -params = extra.get("model_params", {}) if isinstance(extra, dict) else {} -configured = params.get("lora") if isinstance(params, dict) else None -markers = (".A_by_l", ".B_by_l", ".A_m0", ".B_m0", ".A_m.", ".B_m.", ".lora_scaling") -adapter_keys = sorted( - key for key in state if any(marker in key for marker in markers) -) - -print("configured_lora =", configured) -print("adapter_tensor_count =", len(adapter_keys)) -for key in adapter_keys[:20]: - print("adapter_tensor =", key) - -if configured is not None and adapter_keys: - print("classification = active/unmerged LoRA checkpoint") -elif configured is not None or adapter_keys: - print("classification = inconsistent or transitional; inspect before use") -else: - print("classification = plain checkpoint or merged LoRA checkpoint") -PY -``` +Use standard fine-tuning by default. Use LoRA only for a single-task target when +parameter-efficient adaptation is wanted and the exact base architecture is +known. LoRA is enabled by a non-null `model.lora` block in the new input; the +pretrained checkpoint does not need to contain LoRA. Multi-task LoRA targets are +unsupported. -Do not infer training history from the last classification. Validation-selected -best LoRA checkpoints fold adapter deltas into ordinary DPA4 weights and remove -LoRA metadata/tensors, so they intentionally look plain. Such merged checkpoints -are suitable for evaluation, new fine-tuning, and `.pt2` export, but do not carry -the optimizer/EMA state needed to resume the same LoRA run. Periodic/final LoRA -checkpoints retain active adapters and are the resumable form. +Periodic LoRA checkpoints retain adapters and can resume training. Best +checkpoints may merge the adapters into ordinary DPA4 weights, so absence of +LoRA metadata does not prove LoRA was never used. ## Standard fine-tuning @@ -217,12 +127,9 @@ dp --pt train lora_ft.json --finetune pretrained.pt ``` The JSON fragment above is not a complete training input. Adapt the full public -example at `../../examples/water/dpa4/lora_ft.json`, but copy the architecture -from the actual source checkpoint before adding `model.lora`. The compact -`examples/water/dpa4/lmp/pretrained.pt` does not match the larger architecture -in the maintained `lora_ft.json` and must not be paired with it unchanged. Do -not add `--use-pretrain-script` to this LoRA command unless a targeted test -confirms that the intended LoRA configuration is retained. +example at `../../examples/water/dpa4/lora_ft.json`, but copy the exact +architecture from the source checkpoint before adding `model.lora`. Do not add +`--use-pretrain-script` unless a targeted test confirms that LoRA is retained. ## Monitor and validate @@ -256,12 +163,12 @@ again when loading that archive. ## Checklist - [ ] The stored descriptor identifies DPA4/SeZM; the `.pt` suffix was not used as proof. -- [ ] The checkpoint source, checksum, DeePMD-kit revision, architecture, and type map are recorded. +- [ ] The checkpoint source, DeePMD-kit revision, architecture, and type map are recorded. - [ ] The intended branch is explicit for a multi-task checkpoint. - [ ] Training, validation, and held-out test systems are separate. - [ ] The input architecture is compatible with the checkpoint. - [ ] Standard fine-tuning versus LoRA was selected from the task layout and domain shift. -- [ ] Active LoRA state versus a merged best checkpoint is interpreted correctly. +- [ ] A resumable LoRA checkpoint is distinguished from a merged best checkpoint. - [ ] LoRA uses a complete base configuration and is not silently overwritten. - [ ] Training and held-out metrics are finite and reported with units. - [ ] The selected `.pt` checkpoint was exported to and tested as `.pt2`. From f42fddeba69ecb90a3d821ced2bc9913a99e0b8c Mon Sep 17 00:00:00 2001 From: weiqichen77 Date: Sun, 9 Aug 2026 08:53:58 +0800 Subject: [PATCH 14/24] docs(skills): add MatMaster DPA4 workflows Signed-off-by: weiqichen77 --- doc/agent-skills.md | 5 + skills/matmaster-dpa4-workflows/SKILL.md | 111 ++++++++ .../assets/bohrium-job/job.json | 19 ++ .../assets/bohrium-job/manifest.json | 30 ++ .../references/matmaster-operations.md | 86 ++++++ .../references/workflow-handoffs.md | 52 ++++ .../scripts/audit_cases.py | 261 ++++++++++++++++++ .../scripts/check_environment.py | 140 ++++++++++ .../scripts/make_job_spec.py | 92 ++++++ 9 files changed, 796 insertions(+) create mode 100644 skills/matmaster-dpa4-workflows/SKILL.md create mode 100644 skills/matmaster-dpa4-workflows/assets/bohrium-job/job.json create mode 100644 skills/matmaster-dpa4-workflows/assets/bohrium-job/manifest.json create mode 100644 skills/matmaster-dpa4-workflows/references/matmaster-operations.md create mode 100644 skills/matmaster-dpa4-workflows/references/workflow-handoffs.md create mode 100755 skills/matmaster-dpa4-workflows/scripts/audit_cases.py create mode 100755 skills/matmaster-dpa4-workflows/scripts/check_environment.py create mode 100755 skills/matmaster-dpa4-workflows/scripts/make_job_spec.py diff --git a/doc/agent-skills.md b/doc/agent-skills.md index dda701ace0..ae2112a71e 100644 --- a/doc/agent-skills.md +++ b/doc/agent-skills.md @@ -29,6 +29,9 @@ in the DeePMD-kit repository under `skills/`. - `lammps-deepmd`: Prepare, explain, and run LAMMPS simulations with DeePMD-kit potentials, including DPA4/SeZM `.pt2` deployment and common NVE, NVT, and NPT setups. +- `matmaster-dpa4-workflows`: Run prepared DPA4 training, fine-tuning, + inference, and LAMMPS cases on MatMaster/Bohrium while delegating scientific + setup and acceptance to the sibling DeePMD-kit skills. ## Related reference @@ -85,3 +88,5 @@ without launching an expensive calculation. For example: a LoRA fine-tuning input, but do not start training.” - “Use the `lammps-deepmd` skill to prepare an NVT LAMMPS input file for a DeePMD-kit model, and explain each command.” +- “Use the `matmaster-dpa4-workflows` skill to prepare and validate a Bohrium + job specification for an existing DPA4 case, but do not submit it.” diff --git a/skills/matmaster-dpa4-workflows/SKILL.md b/skills/matmaster-dpa4-workflows/SKILL.md new file mode 100644 index 0000000000..f2a46f8705 --- /dev/null +++ b/skills/matmaster-dpa4-workflows/SKILL.md @@ -0,0 +1,111 @@ +--- +name: matmaster-dpa4-workflows +description: Run prepared DPA4 training, fine-tuning, inference, and LAMMPS cases on MatMaster/Bohrium. Use when an agent must stage persistent inputs, resolve Bohrium projects/images/GPU resources, generate and validate a job specification, submit or monitor jobs, collect outputs, recover checkpoints or restarts, and record platform provenance. Delegate scientific setup and acceptance to the installed DeePMD-kit skills. +compatibility: Install with deepmd-train, deepmd-finetune-dpa4, deepmd-python-inference, and lammps-deepmd. Requires Python 3; production submission requires a MatMaster platform tool or a Bohrium CLI compatible with 2.5.17. +license: LGPL-3.0-or-later +metadata: + author: MatMaster + version: '1.0' + repository: https://github.com/deepmodeling/deepmd-kit +--- + +# Run DPA4 workflows on MatMaster + +Act only as the MatMaster/Bohrium platform layer. Own storage, resource +resolution, job packaging, submission, monitoring, collection, and retry +lineage. Do not own model architecture, training parameters, checkpoint/head +selection, inference semantics, LAMMPS input design, or scientific acceptance. + +## Route scientific work + +- Use `deepmd-train` for a new DPA4 training case. +- Use `deepmd-finetune-dpa4` for standard or LoRA fine-tuning. +- Use `deepmd-python-inference` for Python inference or `dp test`. +- Use `lammps-deepmd` for deployment, minimization, and MD. + +Require the owning skill to return a prepared case contract. If that skill is +unavailable, install or reload the complete DeePMD-kit skill set instead of +reconstructing its instructions here. + +Read references only when needed: + +- [workflow-handoffs.md](references/workflow-handoffs.md): required inputs, + outputs, and acceptance boundary for each workflow; +- [matmaster-operations.md](references/matmaster-operations.md): storage, + resources, Bohrium CLI fallback, lifecycle, and recovery. + +## Execute + +### 1. Accept a prepared case + +Require a relative entry command, normally `bash run.sh`, plus all referenced +files or durable paths, checksums, runtime/resource requirements, declared +outputs, smoke-test criteria, and checkpoint/restart policy. Do not change the +scientific command to fit an available resource. + +### 2. Inventory and stage + +Run: + +```bash +python scripts/check_environment.py --case-dir CASE_DIR +``` + +Use `--require-file`, `--require-deepmd`, and `--require-lammps` as required by +the handoff. Keep durable inputs and results under `/personal` or `/share`. +Use relative paths inside job packages; the Bohrium working directory is not a +stable absolute path. + +### 3. Resolve execution resources + +Discover a real project ID, full image address, current machine type, wall +time, and output destination. Never reuse a historical project, image tag, GPU +SKU, or case count as a default. Prove image/data/output compatibility with the +bounded smoke test supplied by the owning skill. + +### 4. Build and validate the job + +Generate a concrete job specification: + +```bash +python scripts/make_job_spec.py CASE_DIR \ + --output CASE_DIR/job.json \ + --project-id PROJECT_ID \ + --image FULL_IMAGE_ADDRESS \ + --machine MACHINE_TYPE \ + --name JOB_NAME +``` + +Preview submission, then use the installed platform tool or the native Bohrium +CLI dry run. Submit only after validation and explicit authorization. Stop if +neither a submission tool nor a compatible `bohr` executable exists. + +### 5. Monitor and collect + +Reconcile every job/group ID with `assets/bohrium-job/manifest.json`. Download +finished and failed cases, retain partial checkpoints/restarts, and run a +filesystem/log first pass: + +```bash +python scripts/audit_cases.py ROOT --json REPORT.json +``` + +Add `--mode` and repeat `--require` for declared outputs. A scheduler success +or file-audit pass is not scientific acceptance; return artifacts to the owning +DeePMD skill. + +### 6. Recover the smallest subset + +Classify packaging, path, image, quota, queue, resource, interruption, and +wall-time failures here. Route training divergence/configuration, checkpoint +validity, inference correctness, model export, type mapping, LAMMPS input, and +numerical failures back to the owning skill. Retry only invalid cases and set +`retry_of`; never overwrite evidence from an earlier attempt. + +## Return the platform ledger + +Return persistent paths, project/image/machine selections, case/job/group and +attempt mappings, expected/submitted/downloaded/platform-valid counts, retry +lineage, logs, declared outputs, audit report, and unresolved platform risks. +Keep the platform verdict separate from training, inference, MD, and scientific +verdicts. diff --git a/skills/matmaster-dpa4-workflows/assets/bohrium-job/job.json b/skills/matmaster-dpa4-workflows/assets/bohrium-job/job.json new file mode 100644 index 0000000000..3aa1c134f3 --- /dev/null +++ b/skills/matmaster-dpa4-workflows/assets/bohrium-job/job.json @@ -0,0 +1,19 @@ +{ + "job_name": "__JOB_NAME__", + "command": "bash run.sh", + "log_file": "run.log", + "backward_files": [ + "run.log", + "runtime.txt", + "logs/", + "results/", + "provenance.json" + ], + "project_id": "__PROJECT_ID__", + "machine_type": "__MACHINE_TYPE__", + "image_address": "__FULL_IMAGE_ADDRESS__", + "job_type": "container", + "max_reschedule_times": 0, + "max_run_time": "__MINUTES__", + "nnode": 1 +} diff --git a/skills/matmaster-dpa4-workflows/assets/bohrium-job/manifest.json b/skills/matmaster-dpa4-workflows/assets/bohrium-job/manifest.json new file mode 100644 index 0000000000..a1f49d6878 --- /dev/null +++ b/skills/matmaster-dpa4-workflows/assets/bohrium-job/manifest.json @@ -0,0 +1,30 @@ +{ + "run_id": "__RUN_ID__", + "workflow": "__TRAIN_FINETUNE_INFERENCE_OR_LAMMPS__", + "prepared_by": "__DEEPMD_SKILL__", + "project_id": "__PROJECT_ID__", + "image_address": "__FULL_IMAGE_ADDRESS__", + "machine_type": "__MACHINE_TYPE__", + "cases": [ + { + "case_id": "__CASE_ID__", + "case_dir": "__RELATIVE_CASE_DIR__", + "entry_command": "bash run.sh", + "input_checksums": {}, + "declared_outputs": [], + "attempts": [ + { + "attempt": 0, + "job_id": null, + "group_id": null, + "retry_of": null, + "platform_status": "prepared", + "platform_verdict": null, + "reason": null, + "changed_platform_fields": [], + "result_dir": null + } + ] + } + ] +} diff --git a/skills/matmaster-dpa4-workflows/references/matmaster-operations.md b/skills/matmaster-dpa4-workflows/references/matmaster-operations.md new file mode 100644 index 0000000000..cd46fb5b2f --- /dev/null +++ b/skills/matmaster-dpa4-workflows/references/matmaster-operations.md @@ -0,0 +1,86 @@ +# MatMaster and Bohrium operations + +## Capability order + +Use the first available capability: + +1. installed MatMaster/Bohrium skill or platform tool; +1. an installed Bohrium CLI whose help matches the required operation; +1. explicit prerequisite installation or user/platform intervention. + +This skill bundles no executable, credential, project, image, or GPU default. +Do not invent an OpenAPI route when a platform tool or compatible CLI is +missing. Use `python scripts/check_environment.py --probe --require-bohr` to +inventory the fallback without exposing secret values. + +## Storage and job boundaries + +| Layer | Use | Persistence | +| ---------------------- | ---------------------------------- | --------------------------------- | +| MatMaster session node | prepare, inspect, and post-process | system disk is not durable | +| `/personal` | user-scoped inputs and results | durable | +| `/share` | project-shared inputs and results | durable, permission scoped | +| Bohrium job workspace | production execution | temporary except declared outputs | + +Do not assume that a session-node package exists in the job image. Do not save +mounted `/personal` or `/share` data into an image. Confirm project permission +before writing shared storage, and stop unused billable nodes according to the +accepted lifecycle policy. + +## Images and resources + +- Query current projects, full image addresses, and machine availability. +- Pin the image address and version/digest; a display name is insufficient. +- Select resources that satisfy the prepared handoff, then run its smoke test. +- Treat image-pull/cache startup as distinct from a hung calculation. +- Confirm project, quota, budget, wall time, and reschedule policy before submit. + +## Package and submit + +Keep the command in a short `run.sh`, use relative paths, write an incremental +log, and declare every result needed after the temporary workspace disappears. +Start from `assets/bohrium-job/job.json` or generate the resolved specification +with `scripts/make_job_spec.py`. Reject every remaining `__PLACEHOLDER__`. + +For Bohrium CLI 2.5.17, the validation shape is: + +```bash +bohr version +bohr job submit -i job.json \ + --input_directory ./case_dir/ \ + --dry-run --output json +``` + +Inspect `bohr job submit --help` when the installed interface differs. Remove +`--dry-run` only for the authorized production submission. Prefer a MatMaster +built-in submit tool when present. + +Record the case ID, attempt, job/group IDs, project, image, machine, input +checksums, declared outputs, and `retry_of` in the manifest. A reschedule may +restart `run.sh` from the beginning; enable rescheduling only when the wrapper +is safely restart-aware. + +## Monitor, collect, and recover + +For each job group: + +1. reconcile its members with the manifest; +1. inspect representative running and failed logs; +1. distinguish queue/image startup from execution progress; +1. download finished and failed terminal jobs; +1. run the file/log audit and return results to the owning skill; +1. retry only invalid cases in a new attempt directory. + +`Finished` is a platform state, not proof of a valid model or simulation. +Preserve partial logs, checkpoints, and restarts before any retry. + +Use terminate when recoverable outputs should normally remain. Use kill only +for an unrecoverable job after resolving its exact ID. Delete jobs or persistent +files only when explicitly requested and after verifying the target. + +## Authentication + +Let platform tools handle credentials when possible. For a CLI fallback, read +the access key from the injected environment (`BOHR_ACCESS_KEY`, or +`ACCESS_KEY` when required by that CLI version). Never print, hard-code, copy +into a job package, or return credentials, storage tokens, or node passwords. diff --git a/skills/matmaster-dpa4-workflows/references/workflow-handoffs.md b/skills/matmaster-dpa4-workflows/references/workflow-handoffs.md new file mode 100644 index 0000000000..b967d08d12 --- /dev/null +++ b/skills/matmaster-dpa4-workflows/references/workflow-handoffs.md @@ -0,0 +1,52 @@ +# DeePMD workflow handoffs + +## Common contract + +Before staging, require the owning DeePMD skill to provide: + +- workflow kind, owning skill, case ID, and relative entry command; +- every input file or durable path plus checksums/manifests; +- image/runtime, accelerator, memory, node/rank, and wall-time requirements; +- incremental logs, checkpoints/restarts, final outputs, and output cadence; +- smoke-test contract, retry/resume constraints, and acceptance owner. + +Wrap the exact accepted command in `run.sh`. Do not reinterpret scientific +configuration during packaging. + +## Training and fine-tuning + +`deepmd-train` or `deepmd-finetune-dpa4` owns the training input, data split, +starting checkpoint/head, checkpoint cadence, command, metrics, and scientific +acceptance. + +- Reference large durable datasets/checkpoints rather than copying partial data. +- Prove data access, representative batch size, accelerator compatibility, and + checkpoint writing with the supplied bounded smoke test. +- Preserve resolved inputs, logs, metrics, periodic checkpoints, runtime record, + and all attempts. +- On interruption, return every checkpoint. The owning skill selects the valid + resume checkpoint and command. + +The platform layer may report progress and file growth, but must not choose the +best checkpoint or interpret convergence. + +## Inference + +`deepmd-python-inference` owns the model artifact, input ordering, type mapping, +command, batch size, declared predictions, and accuracy acceptance. Preserve +ordering and case identity; return predictions, logs, runtime metadata, and +failures without interpreting accuracy. + +## LAMMPS + +`lammps-deepmd` owns deployment, type mapping, LAMMPS inputs, ensemble, +timestep, ranks, output cadence, restart policy, and simulation acceptance. +Preserve trajectories, logs, final structures, and partial restarts without +rewriting those scientific choices. + +## Return boundary + +Return case/attempt/job/group identity, platform selections and versions, +input/output paths, scheduler/exit evidence, logs, retry lineage, and a platform +verdict. The owning skill determines training, inference, MD, and scientific +validity. diff --git a/skills/matmaster-dpa4-workflows/scripts/audit_cases.py b/skills/matmaster-dpa4-workflows/scripts/audit_cases.py new file mode 100755 index 0000000000..77ba54935b --- /dev/null +++ b/skills/matmaster-dpa4-workflows/scripts/audit_cases.py @@ -0,0 +1,261 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: LGPL-3.0-or-later +"""Run a non-destructive first-pass audit of DPA4 workflow case folders.""" + +# ruff: noqa: T201 -- stdout is this command-line tool's result interface. + +from __future__ import ( + annotations, +) + +import argparse +import json +import math +import re +from pathlib import ( + Path, +) + +FATAL_PATTERNS = { + "unknown_dpa4": re.compile(r"Unknown model type:\s*dpa4", re.I), + "missing_mass": re.compile( + r"Not all per-type masses are set|Type\s+\d+\s+is missing", re.I + ), + "cuda_error": re.compile(r"CUDA Runtime|CUDA error|operation not supported", re.I), + "oom": re.compile(r"out of memory|\bOOM\b|std::bad_alloc", re.I), + "lost_atoms": re.compile(r"Lost atoms|Bond atoms missing|Out of range atoms", re.I), + "non_numeric": re.compile( + r"Non-numeric (?:atom coords|box dimensions|pressure|energy)|\bnan\b|\binf\b", + re.I, + ), + "segfault": re.compile(r"Segmentation fault|SIGSEGV", re.I), + "lammps_error": re.compile(r"^ERROR(?: on proc \d+)?:", re.I | re.M), + "python_exception": re.compile( + r"Traceback \(most recent call last\):|FileNotFoundError|RuntimeError:", re.I + ), + "distributed_error": re.compile( + r"NCCL (?:error|failure)|ProcessGroupNCCL|ChildFailedError", re.I + ), +} + +COMPLETE_PATTERNS = [ + re.compile(r"Total wall time:", re.I), + re.compile(r"Loop time of", re.I), + re.compile(r"Minimization stats:", re.I), +] + +ENERGY_RE = re.compile( + r"(?:PotEng|\bpe\b|potential[_ ]?energy|final[_ ]?energy)\s*[=:, ]+" + r"([-+]?\d+(?:\.\d*)?(?:[eE][-+]?\d+)?)", + re.I, +) + + +def read_text(path: Path, limit: int = 50_000_000) -> str: + try: + with path.open("r", errors="replace") as handle: + return handle.read(limit) + except OSError: + return "" + + +def find_case_dirs(root: Path) -> list[Path]: + markers = ( + "in.lammps", + "log.lammps", + "run.log", + "train.log", + "lcurve.out", + "input.json", + "structure.data", + "input.data", + ) + cases: set[Path] = set() + for marker in markers: + for path in root.rglob(marker): + if path.is_file(): + cases.add(path.parent) + return sorted(cases) + + +def infer_mode(case: Path, requested: str) -> str: + if requested != "auto": + return requested + inputs = [path for path in (case / "in.lammps", case / "in.lmp") if path.is_file()] + text = "\n".join(read_text(path) for path in inputs) + has_run = bool(re.search(r"^\s*run\s+\S+", text, re.M)) + has_minimize = bool(re.search(r"^\s*minimize\s+", text, re.M)) + if has_run: + return "md" + if has_minimize: + return "minimize" + training_input = case / "input.json" + if training_input.is_file() and re.search( + r'"training"\s*:', read_text(training_input) + ): + return "train" + return "unknown" + + +def extract_energy(case: Path, log_text: str) -> float | None: + for name in ("energy.dat", "final_energy.dat"): + path = case / name + if path.is_file(): + values = re.findall(r"[-+]?\d+(?:\.\d*)?(?:[eE][-+]?\d+)?", read_text(path)) + if values: + value = float(values[-1]) + return value if math.isfinite(value) else None + matches = ENERGY_RE.findall(log_text) + if matches: + value = float(matches[-1]) + return value if math.isfinite(value) else None + return None + + +def match_required(case: Path, patterns: list[str]) -> dict[str, list[str]]: + matched: dict[str, list[str]] = {} + for pattern in patterns: + paths = sorted( + str(path.relative_to(case)) for path in case.glob(pattern) if path.exists() + ) + matched[pattern] = paths + return matched + + +def audit_case(case: Path, root: Path, mode: str, required: list[str]) -> dict: + logs = [ + path + for path in ( + case / "log.lammps", + case / "run.log", + case / "train.log", + case / "stderr.log", + case / "log", + ) + if path.is_file() + ] + log_text = "\n".join(read_text(path) for path in logs) + fatal = [ + name for name, pattern in FATAL_PATTERNS.items() if pattern.search(log_text) + ] + inferred_mode = infer_mode(case, mode) + complete = any(pattern.search(log_text) for pattern in COMPLETE_PATTERNS) + required_matches = match_required(case, required) + missing_required = [ + pattern for pattern, matches in required_matches.items() if not matches + ] + reasons: list[str] = [] + if not logs: + reasons.append("missing_log") + if fatal: + reasons.extend(fatal) + if logs and inferred_mode in ("md", "minimize") and not complete: + reasons.append("missing_completion_marker") + reasons.extend(f"missing_required:{pattern}" for pattern in missing_required) + verdict = "passes_file_audit" if not reasons else "needs_review" + artifacts = { + "freeze_log": str(case / "freeze.log") + if (case / "freeze.log").is_file() + else None, + "final_structure": next( + ( + str(case / name) + for name in ("final.data", "relaxed.data") + if (case / name).is_file() + ), + None, + ), + "restart_files": sorted(str(path) for path in case.glob("restart*")), + "trajectory_candidates": sorted( + str(path) + for pattern in ("*.lammpstrj", "dump*", "trajectory/*") + for path in case.glob(pattern) + if path.is_file() + ), + "checkpoint_candidates": sorted( + str(path) + for pattern in ("*.pt", "checkpoint*", "ckpt/*") + for path in case.glob(pattern) + if path.is_file() + ), + "metric_candidates": sorted( + str(path) + for pattern in ("lcurve.out", "metrics*", "results/*") + for path in case.glob(pattern) + if path.is_file() + ), + } + return { + "case_id": str(case.relative_to(root)) or ".", + "case_dir": str(case), + "mode": inferred_mode, + "verdict": verdict, + "reasons": reasons, + "completion_marker": complete, + "fatal_signatures": fatal, + "last_detected_energy": extract_energy(case, log_text), + "required_matches": required_matches, + "artifacts": artifacts, + } + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "root", type=Path, help="Root directory containing case directories" + ) + parser.add_argument( + "--mode", + choices=("auto", "train", "inference", "minimize", "md"), + default="auto", + ) + parser.add_argument( + "--require", + action="append", + default=[], + metavar="GLOB", + help="Require a case-relative file glob; repeat for multiple artifacts", + ) + parser.add_argument( + "--json", type=Path, help="Write the full JSON report to this path" + ) + args = parser.parse_args() + + root = args.root.resolve() + if not root.is_dir(): + parser.error(f"not a directory: {root}") + cases = [ + audit_case(case, root, args.mode, args.require) for case in find_case_dirs(root) + ] + counts: dict[str, int] = {} + for case in cases: + counts[case["verdict"]] = counts.get(case["verdict"], 0) + 1 + report = { + "root": str(root), + "case_count": len(cases), + "mode_request": args.mode, + "required_patterns": args.require, + "verdict_counts": counts, + "warning": "A file/log audit pass is not training, inference, MD-quality, or scientific acceptance.", + "cases": cases, + } + if args.json: + args.json.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n") + print( + json.dumps( + { + key: report[key] + for key in ("root", "case_count", "verdict_counts", "warning") + }, + ensure_ascii=False, + indent=2, + ) + ) + for case in cases: + if case["verdict"] != "passes_file_audit": + print(f"REVIEW\t{case['case_id']}\t{','.join(case['reasons'])}") + return 0 if cases and counts.get("needs_review", 0) == 0 else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/skills/matmaster-dpa4-workflows/scripts/check_environment.py b/skills/matmaster-dpa4-workflows/scripts/check_environment.py new file mode 100755 index 0000000000..7553404c8e --- /dev/null +++ b/skills/matmaster-dpa4-workflows/scripts/check_environment.py @@ -0,0 +1,140 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: LGPL-3.0-or-later +"""Inventory a MatMaster/Bohrium job environment without exposing secrets.""" + +# ruff: noqa: T201 -- stdout is this command-line tool's result interface. + +from __future__ import ( + annotations, +) + +import argparse +import hashlib +import json +import os +import shutil +import subprocess +from pathlib import ( + Path, +) + +COMMANDS = { + "bohr": ["bohr", "version"], + "dp": ["dp", "--version"], + "lmp": ["lmp", "-h"], + "lmp_mpi": ["lmp_mpi", "-h"], + "nvidia_smi": [ + "nvidia-smi", + "--query-gpu=name,memory.total,driver_version", + "--format=csv,noheader", + ], +} + + +def sha256(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as handle: + for chunk in iter(lambda: handle.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +def probe(command: list[str]) -> dict: + executable = shutil.which(command[0]) + result = {"path": executable, "available": executable is not None} + if executable is None: + return result + completed = subprocess.run( + command, capture_output=True, text=True, timeout=20, check=False + ) + output = (completed.stdout or completed.stderr).strip().splitlines() + result.update( + { + "returncode": completed.returncode, + "first_line": output[0][:500] if output else "", + } + ) + return result + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--model", type=Path) + parser.add_argument("--case-dir", type=Path) + parser.add_argument( + "--require-file", + action="append", + default=[], + help="Case-relative required file", + ) + parser.add_argument("--probe", action="store_true", help="Run safe version probes") + parser.add_argument("--require-bohr", action="store_true") + parser.add_argument("--require-deepmd", action="store_true") + parser.add_argument("--require-lammps", action="store_true") + parser.add_argument( + "--require-runtime", action="store_true", help="Require both DeePMD and LAMMPS" + ) + args = parser.parse_args() + + commands = {} + for name, command in COMMANDS.items(): + path = shutil.which(command[0]) + commands[name] = ( + probe(command) + if args.probe + else {"path": path, "available": path is not None} + ) + report = { + "commands": commands, + "environment_present": { + "BOHR_ACCESS_KEY": bool(os.environ.get("BOHR_ACCESS_KEY")), + "ACCESS_KEY": bool(os.environ.get("ACCESS_KEY")), + "PROJECT_ID": bool(os.environ.get("PROJECT_ID")), + "OPENAPI_HOST": bool(os.environ.get("OPENAPI_HOST")), + "TIEFBLUE_HOST": bool(os.environ.get("TIEFBLUE_HOST")), + }, + "persistent_paths": { + "/personal": { + "exists": Path("/personal").exists(), + "writable": os.access("/personal", os.W_OK), + }, + "/share": { + "exists": Path("/share").exists(), + "writable": os.access("/share", os.W_OK), + }, + }, + } + if args.model: + model = args.model.resolve() + report["model"] = { + "path": str(model), + "exists": model.is_file(), + "size": model.stat().st_size if model.is_file() else None, + "sha256": sha256(model) if model.is_file() else None, + } + if args.case_dir: + case = args.case_dir.resolve() + required = ["run.sh", *args.require_file] + report["case"] = { + "path": str(case), + "exists": case.is_dir(), + "required": {name: (case / name).is_file() for name in required}, + } + failures = [] + if args.require_bohr and not commands["bohr"]["available"]: + failures.append("bohr_cli_missing") + if (args.require_deepmd or args.require_runtime) and not commands["dp"][ + "available" + ]: + failures.append("dp_missing") + if (args.require_lammps or args.require_runtime) and not ( + commands["lmp"]["available"] or commands["lmp_mpi"]["available"] + ): + failures.append("lammps_missing") + report["failures"] = failures + print(json.dumps(report, ensure_ascii=False, indent=2)) + return 1 if failures else 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/skills/matmaster-dpa4-workflows/scripts/make_job_spec.py b/skills/matmaster-dpa4-workflows/scripts/make_job_spec.py new file mode 100755 index 0000000000..33f4404542 --- /dev/null +++ b/skills/matmaster-dpa4-workflows/scripts/make_job_spec.py @@ -0,0 +1,92 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: LGPL-3.0-or-later +"""Generate and validate a Bohrium container job specification.""" + +# ruff: noqa: T201 -- stdout is this command-line tool's result interface. + +from __future__ import ( + annotations, +) + +import argparse +import json +import shlex +from pathlib import ( + Path, +) + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("case_dir", type=Path) + parser.add_argument("--output", type=Path, required=True) + parser.add_argument("--project-id", type=int, required=True) + parser.add_argument("--image", required=True) + parser.add_argument("--machine", required=True) + parser.add_argument("--name", required=True) + parser.add_argument("--command", default="bash run.sh") + parser.add_argument("--log-file", default="run.log") + parser.add_argument("--backward", action="append", default=[]) + parser.add_argument("--result-path") + parser.add_argument("--max-run-time", type=int) + parser.add_argument("--max-reschedule-times", type=int, default=0) + parser.add_argument("--nnode", type=int, default=1) + parser.add_argument("--disk-size", type=int) + args = parser.parse_args() + + case_dir = args.case_dir.resolve() + errors = [] + if not case_dir.is_dir(): + errors.append(f"case directory does not exist: {case_dir}") + if "/" not in args.image or ":" not in args.image: + errors.append("image must be a full registry path with a tag") + if "/root/input" in args.command: + errors.append("command must not assume /root/input; use relative paths") + try: + tokens = shlex.split(args.command) + except ValueError as exc: + errors.append(f"invalid command quoting: {exc}") + tokens = [] + if ( + tokens[:2] == ["bash", "run.sh"] + and case_dir.is_dir() + and not (case_dir / "run.sh").is_file() + ): + errors.append("default command requires case_dir/run.sh") + if args.project_id <= 0: + errors.append("project ID must be positive") + if args.max_run_time is not None and args.max_run_time <= 0: + errors.append("max run time must be positive minutes") + if args.max_reschedule_times < 0 or args.nnode <= 0: + errors.append("reschedule count must be nonnegative and nnode positive") + if errors: + for error in errors: + print(f"ERROR: {error}") + return 2 + + spec = { + "job_name": args.name, + "command": args.command, + "log_file": args.log_file, + "backward_files": args.backward, + "project_id": args.project_id, + "machine_type": args.machine, + "image_address": args.image, + "job_type": "container", + "max_reschedule_times": args.max_reschedule_times, + "nnode": args.nnode, + } + if args.result_path: + spec["result_path"] = args.result_path + if args.max_run_time is not None: + spec["max_run_time"] = args.max_run_time + if args.disk_size is not None: + spec["disk_size"] = args.disk_size + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps(spec, ensure_ascii=False, indent=2) + "\n") + print(args.output.resolve()) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) From bbe64e039cbc71ee43f2d957ec4874080cdc2f43 Mon Sep 17 00:00:00 2001 From: SchrodingersCattt Date: Wed, 12 Aug 2026 04:43:54 +0000 Subject: [PATCH 15/24] docs(skills): address DPA4 workflow validation gaps --- skills/deepmd-finetune-dpa4/SKILL.md | 9 +++++++++ skills/matmaster-dpa4-workflows/SKILL.md | 11 ++++++++++- .../matmaster-dpa4-workflows/scripts/audit_cases.py | 2 ++ .../scripts/check_environment.py | 10 ++++++++++ 4 files changed, 31 insertions(+), 1 deletion(-) diff --git a/skills/deepmd-finetune-dpa4/SKILL.md b/skills/deepmd-finetune-dpa4/SKILL.md index 30a6d9ef55..4f8361dbd7 100644 --- a/skills/deepmd-finetune-dpa4/SKILL.md +++ b/skills/deepmd-finetune-dpa4/SKILL.md @@ -126,6 +126,15 @@ Run: dp --pt train lora_ft.json --finetune pretrained.pt ``` +When `pretrained.pt` is multi-task, preserve the selected fitting head: + +```bash +dp --pt train lora_ft.json --finetune pretrained.pt \ + --model-branch SELECTED_BRANCH +``` + +Use the shorter command only for a single-task source checkpoint. + The JSON fragment above is not a complete training input. Adapt the full public example at `../../examples/water/dpa4/lora_ft.json`, but copy the exact architecture from the source checkpoint before adding `model.lora`. Do not add diff --git a/skills/matmaster-dpa4-workflows/SKILL.md b/skills/matmaster-dpa4-workflows/SKILL.md index f2a46f8705..fc443bf0f4 100644 --- a/skills/matmaster-dpa4-workflows/SKILL.md +++ b/skills/matmaster-dpa4-workflows/SKILL.md @@ -73,9 +73,18 @@ python scripts/make_job_spec.py CASE_DIR \ --project-id PROJECT_ID \ --image FULL_IMAGE_ADDRESS \ --machine MACHINE_TYPE \ - --name JOB_NAME + --name JOB_NAME \ + --max-run-time MAX_RUN_TIME_MINUTES \ + --backward run.log \ + --backward CHECKPOINT_OR_RESTART_PATH \ + --backward RESULT_PATH \ + --result-path DURABLE_OUTPUT_DESTINATION ``` +Replace each output placeholder with a path declared by the owning skill and +repeat `--backward` for every log, checkpoint/restart, result, and provenance +artifact that must survive the temporary workspace. + Preview submission, then use the installed platform tool or the native Bohrium CLI dry run. Submit only after validation and explicit authorization. Stop if neither a submission tool nor a compatible `bohr` executable exists. diff --git a/skills/matmaster-dpa4-workflows/scripts/audit_cases.py b/skills/matmaster-dpa4-workflows/scripts/audit_cases.py index 77ba54935b..eb33a0cae6 100755 --- a/skills/matmaster-dpa4-workflows/scripts/audit_cases.py +++ b/skills/matmaster-dpa4-workflows/scripts/audit_cases.py @@ -147,6 +147,8 @@ def audit_case(case: Path, root: Path, mode: str, required: list[str]) -> dict: reasons: list[str] = [] if not logs: reasons.append("missing_log") + elif not log_text.strip(): + reasons.append("empty_log") if fatal: reasons.extend(fatal) if logs and inferred_mode in ("md", "minimize") and not complete: diff --git a/skills/matmaster-dpa4-workflows/scripts/check_environment.py b/skills/matmaster-dpa4-workflows/scripts/check_environment.py index 7553404c8e..877790c222 100755 --- a/skills/matmaster-dpa4-workflows/scripts/check_environment.py +++ b/skills/matmaster-dpa4-workflows/scripts/check_environment.py @@ -121,6 +121,16 @@ def main() -> int: "required": {name: (case / name).is_file() for name in required}, } failures = [] + if args.model and not report["model"]["exists"]: + failures.append("model_missing") + if args.case_dir: + if not report["case"]["exists"]: + failures.append("case_dir_missing") + failures.extend( + f"required_file_missing:{name}" + for name, present in report["case"]["required"].items() + if not present + ) if args.require_bohr and not commands["bohr"]["available"]: failures.append("bohr_cli_missing") if (args.require_deepmd or args.require_runtime) and not commands["dp"][ From a9f12797925a9ba4c2023c6abb6ebbdc4a0f0dda Mon Sep 17 00:00:00 2001 From: root Date: Fri, 14 Aug 2026 22:11:24 +0800 Subject: [PATCH 16/24] Revert "docs(skills): add MatMaster DPA4 workflows" This reverts commit f42fddeba69ecb90a3d821ced2bc9913a99e0b8c. --- doc/agent-skills.md | 5 - skills/matmaster-dpa4-workflows/SKILL.md | 120 -------- .../assets/bohrium-job/job.json | 19 -- .../assets/bohrium-job/manifest.json | 30 -- .../references/matmaster-operations.md | 86 ------ .../references/workflow-handoffs.md | 52 ---- .../scripts/audit_cases.py | 263 ------------------ .../scripts/check_environment.py | 150 ---------- .../scripts/make_job_spec.py | 92 ------ 9 files changed, 817 deletions(-) delete mode 100644 skills/matmaster-dpa4-workflows/SKILL.md delete mode 100644 skills/matmaster-dpa4-workflows/assets/bohrium-job/job.json delete mode 100644 skills/matmaster-dpa4-workflows/assets/bohrium-job/manifest.json delete mode 100644 skills/matmaster-dpa4-workflows/references/matmaster-operations.md delete mode 100644 skills/matmaster-dpa4-workflows/references/workflow-handoffs.md delete mode 100755 skills/matmaster-dpa4-workflows/scripts/audit_cases.py delete mode 100755 skills/matmaster-dpa4-workflows/scripts/check_environment.py delete mode 100755 skills/matmaster-dpa4-workflows/scripts/make_job_spec.py diff --git a/doc/agent-skills.md b/doc/agent-skills.md index ae2112a71e..dda701ace0 100644 --- a/doc/agent-skills.md +++ b/doc/agent-skills.md @@ -29,9 +29,6 @@ in the DeePMD-kit repository under `skills/`. - `lammps-deepmd`: Prepare, explain, and run LAMMPS simulations with DeePMD-kit potentials, including DPA4/SeZM `.pt2` deployment and common NVE, NVT, and NPT setups. -- `matmaster-dpa4-workflows`: Run prepared DPA4 training, fine-tuning, - inference, and LAMMPS cases on MatMaster/Bohrium while delegating scientific - setup and acceptance to the sibling DeePMD-kit skills. ## Related reference @@ -88,5 +85,3 @@ without launching an expensive calculation. For example: a LoRA fine-tuning input, but do not start training.” - “Use the `lammps-deepmd` skill to prepare an NVT LAMMPS input file for a DeePMD-kit model, and explain each command.” -- “Use the `matmaster-dpa4-workflows` skill to prepare and validate a Bohrium - job specification for an existing DPA4 case, but do not submit it.” diff --git a/skills/matmaster-dpa4-workflows/SKILL.md b/skills/matmaster-dpa4-workflows/SKILL.md deleted file mode 100644 index fc443bf0f4..0000000000 --- a/skills/matmaster-dpa4-workflows/SKILL.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -name: matmaster-dpa4-workflows -description: Run prepared DPA4 training, fine-tuning, inference, and LAMMPS cases on MatMaster/Bohrium. Use when an agent must stage persistent inputs, resolve Bohrium projects/images/GPU resources, generate and validate a job specification, submit or monitor jobs, collect outputs, recover checkpoints or restarts, and record platform provenance. Delegate scientific setup and acceptance to the installed DeePMD-kit skills. -compatibility: Install with deepmd-train, deepmd-finetune-dpa4, deepmd-python-inference, and lammps-deepmd. Requires Python 3; production submission requires a MatMaster platform tool or a Bohrium CLI compatible with 2.5.17. -license: LGPL-3.0-or-later -metadata: - author: MatMaster - version: '1.0' - repository: https://github.com/deepmodeling/deepmd-kit ---- - -# Run DPA4 workflows on MatMaster - -Act only as the MatMaster/Bohrium platform layer. Own storage, resource -resolution, job packaging, submission, monitoring, collection, and retry -lineage. Do not own model architecture, training parameters, checkpoint/head -selection, inference semantics, LAMMPS input design, or scientific acceptance. - -## Route scientific work - -- Use `deepmd-train` for a new DPA4 training case. -- Use `deepmd-finetune-dpa4` for standard or LoRA fine-tuning. -- Use `deepmd-python-inference` for Python inference or `dp test`. -- Use `lammps-deepmd` for deployment, minimization, and MD. - -Require the owning skill to return a prepared case contract. If that skill is -unavailable, install or reload the complete DeePMD-kit skill set instead of -reconstructing its instructions here. - -Read references only when needed: - -- [workflow-handoffs.md](references/workflow-handoffs.md): required inputs, - outputs, and acceptance boundary for each workflow; -- [matmaster-operations.md](references/matmaster-operations.md): storage, - resources, Bohrium CLI fallback, lifecycle, and recovery. - -## Execute - -### 1. Accept a prepared case - -Require a relative entry command, normally `bash run.sh`, plus all referenced -files or durable paths, checksums, runtime/resource requirements, declared -outputs, smoke-test criteria, and checkpoint/restart policy. Do not change the -scientific command to fit an available resource. - -### 2. Inventory and stage - -Run: - -```bash -python scripts/check_environment.py --case-dir CASE_DIR -``` - -Use `--require-file`, `--require-deepmd`, and `--require-lammps` as required by -the handoff. Keep durable inputs and results under `/personal` or `/share`. -Use relative paths inside job packages; the Bohrium working directory is not a -stable absolute path. - -### 3. Resolve execution resources - -Discover a real project ID, full image address, current machine type, wall -time, and output destination. Never reuse a historical project, image tag, GPU -SKU, or case count as a default. Prove image/data/output compatibility with the -bounded smoke test supplied by the owning skill. - -### 4. Build and validate the job - -Generate a concrete job specification: - -```bash -python scripts/make_job_spec.py CASE_DIR \ - --output CASE_DIR/job.json \ - --project-id PROJECT_ID \ - --image FULL_IMAGE_ADDRESS \ - --machine MACHINE_TYPE \ - --name JOB_NAME \ - --max-run-time MAX_RUN_TIME_MINUTES \ - --backward run.log \ - --backward CHECKPOINT_OR_RESTART_PATH \ - --backward RESULT_PATH \ - --result-path DURABLE_OUTPUT_DESTINATION -``` - -Replace each output placeholder with a path declared by the owning skill and -repeat `--backward` for every log, checkpoint/restart, result, and provenance -artifact that must survive the temporary workspace. - -Preview submission, then use the installed platform tool or the native Bohrium -CLI dry run. Submit only after validation and explicit authorization. Stop if -neither a submission tool nor a compatible `bohr` executable exists. - -### 5. Monitor and collect - -Reconcile every job/group ID with `assets/bohrium-job/manifest.json`. Download -finished and failed cases, retain partial checkpoints/restarts, and run a -filesystem/log first pass: - -```bash -python scripts/audit_cases.py ROOT --json REPORT.json -``` - -Add `--mode` and repeat `--require` for declared outputs. A scheduler success -or file-audit pass is not scientific acceptance; return artifacts to the owning -DeePMD skill. - -### 6. Recover the smallest subset - -Classify packaging, path, image, quota, queue, resource, interruption, and -wall-time failures here. Route training divergence/configuration, checkpoint -validity, inference correctness, model export, type mapping, LAMMPS input, and -numerical failures back to the owning skill. Retry only invalid cases and set -`retry_of`; never overwrite evidence from an earlier attempt. - -## Return the platform ledger - -Return persistent paths, project/image/machine selections, case/job/group and -attempt mappings, expected/submitted/downloaded/platform-valid counts, retry -lineage, logs, declared outputs, audit report, and unresolved platform risks. -Keep the platform verdict separate from training, inference, MD, and scientific -verdicts. diff --git a/skills/matmaster-dpa4-workflows/assets/bohrium-job/job.json b/skills/matmaster-dpa4-workflows/assets/bohrium-job/job.json deleted file mode 100644 index 3aa1c134f3..0000000000 --- a/skills/matmaster-dpa4-workflows/assets/bohrium-job/job.json +++ /dev/null @@ -1,19 +0,0 @@ -{ - "job_name": "__JOB_NAME__", - "command": "bash run.sh", - "log_file": "run.log", - "backward_files": [ - "run.log", - "runtime.txt", - "logs/", - "results/", - "provenance.json" - ], - "project_id": "__PROJECT_ID__", - "machine_type": "__MACHINE_TYPE__", - "image_address": "__FULL_IMAGE_ADDRESS__", - "job_type": "container", - "max_reschedule_times": 0, - "max_run_time": "__MINUTES__", - "nnode": 1 -} diff --git a/skills/matmaster-dpa4-workflows/assets/bohrium-job/manifest.json b/skills/matmaster-dpa4-workflows/assets/bohrium-job/manifest.json deleted file mode 100644 index a1f49d6878..0000000000 --- a/skills/matmaster-dpa4-workflows/assets/bohrium-job/manifest.json +++ /dev/null @@ -1,30 +0,0 @@ -{ - "run_id": "__RUN_ID__", - "workflow": "__TRAIN_FINETUNE_INFERENCE_OR_LAMMPS__", - "prepared_by": "__DEEPMD_SKILL__", - "project_id": "__PROJECT_ID__", - "image_address": "__FULL_IMAGE_ADDRESS__", - "machine_type": "__MACHINE_TYPE__", - "cases": [ - { - "case_id": "__CASE_ID__", - "case_dir": "__RELATIVE_CASE_DIR__", - "entry_command": "bash run.sh", - "input_checksums": {}, - "declared_outputs": [], - "attempts": [ - { - "attempt": 0, - "job_id": null, - "group_id": null, - "retry_of": null, - "platform_status": "prepared", - "platform_verdict": null, - "reason": null, - "changed_platform_fields": [], - "result_dir": null - } - ] - } - ] -} diff --git a/skills/matmaster-dpa4-workflows/references/matmaster-operations.md b/skills/matmaster-dpa4-workflows/references/matmaster-operations.md deleted file mode 100644 index cd46fb5b2f..0000000000 --- a/skills/matmaster-dpa4-workflows/references/matmaster-operations.md +++ /dev/null @@ -1,86 +0,0 @@ -# MatMaster and Bohrium operations - -## Capability order - -Use the first available capability: - -1. installed MatMaster/Bohrium skill or platform tool; -1. an installed Bohrium CLI whose help matches the required operation; -1. explicit prerequisite installation or user/platform intervention. - -This skill bundles no executable, credential, project, image, or GPU default. -Do not invent an OpenAPI route when a platform tool or compatible CLI is -missing. Use `python scripts/check_environment.py --probe --require-bohr` to -inventory the fallback without exposing secret values. - -## Storage and job boundaries - -| Layer | Use | Persistence | -| ---------------------- | ---------------------------------- | --------------------------------- | -| MatMaster session node | prepare, inspect, and post-process | system disk is not durable | -| `/personal` | user-scoped inputs and results | durable | -| `/share` | project-shared inputs and results | durable, permission scoped | -| Bohrium job workspace | production execution | temporary except declared outputs | - -Do not assume that a session-node package exists in the job image. Do not save -mounted `/personal` or `/share` data into an image. Confirm project permission -before writing shared storage, and stop unused billable nodes according to the -accepted lifecycle policy. - -## Images and resources - -- Query current projects, full image addresses, and machine availability. -- Pin the image address and version/digest; a display name is insufficient. -- Select resources that satisfy the prepared handoff, then run its smoke test. -- Treat image-pull/cache startup as distinct from a hung calculation. -- Confirm project, quota, budget, wall time, and reschedule policy before submit. - -## Package and submit - -Keep the command in a short `run.sh`, use relative paths, write an incremental -log, and declare every result needed after the temporary workspace disappears. -Start from `assets/bohrium-job/job.json` or generate the resolved specification -with `scripts/make_job_spec.py`. Reject every remaining `__PLACEHOLDER__`. - -For Bohrium CLI 2.5.17, the validation shape is: - -```bash -bohr version -bohr job submit -i job.json \ - --input_directory ./case_dir/ \ - --dry-run --output json -``` - -Inspect `bohr job submit --help` when the installed interface differs. Remove -`--dry-run` only for the authorized production submission. Prefer a MatMaster -built-in submit tool when present. - -Record the case ID, attempt, job/group IDs, project, image, machine, input -checksums, declared outputs, and `retry_of` in the manifest. A reschedule may -restart `run.sh` from the beginning; enable rescheduling only when the wrapper -is safely restart-aware. - -## Monitor, collect, and recover - -For each job group: - -1. reconcile its members with the manifest; -1. inspect representative running and failed logs; -1. distinguish queue/image startup from execution progress; -1. download finished and failed terminal jobs; -1. run the file/log audit and return results to the owning skill; -1. retry only invalid cases in a new attempt directory. - -`Finished` is a platform state, not proof of a valid model or simulation. -Preserve partial logs, checkpoints, and restarts before any retry. - -Use terminate when recoverable outputs should normally remain. Use kill only -for an unrecoverable job after resolving its exact ID. Delete jobs or persistent -files only when explicitly requested and after verifying the target. - -## Authentication - -Let platform tools handle credentials when possible. For a CLI fallback, read -the access key from the injected environment (`BOHR_ACCESS_KEY`, or -`ACCESS_KEY` when required by that CLI version). Never print, hard-code, copy -into a job package, or return credentials, storage tokens, or node passwords. diff --git a/skills/matmaster-dpa4-workflows/references/workflow-handoffs.md b/skills/matmaster-dpa4-workflows/references/workflow-handoffs.md deleted file mode 100644 index b967d08d12..0000000000 --- a/skills/matmaster-dpa4-workflows/references/workflow-handoffs.md +++ /dev/null @@ -1,52 +0,0 @@ -# DeePMD workflow handoffs - -## Common contract - -Before staging, require the owning DeePMD skill to provide: - -- workflow kind, owning skill, case ID, and relative entry command; -- every input file or durable path plus checksums/manifests; -- image/runtime, accelerator, memory, node/rank, and wall-time requirements; -- incremental logs, checkpoints/restarts, final outputs, and output cadence; -- smoke-test contract, retry/resume constraints, and acceptance owner. - -Wrap the exact accepted command in `run.sh`. Do not reinterpret scientific -configuration during packaging. - -## Training and fine-tuning - -`deepmd-train` or `deepmd-finetune-dpa4` owns the training input, data split, -starting checkpoint/head, checkpoint cadence, command, metrics, and scientific -acceptance. - -- Reference large durable datasets/checkpoints rather than copying partial data. -- Prove data access, representative batch size, accelerator compatibility, and - checkpoint writing with the supplied bounded smoke test. -- Preserve resolved inputs, logs, metrics, periodic checkpoints, runtime record, - and all attempts. -- On interruption, return every checkpoint. The owning skill selects the valid - resume checkpoint and command. - -The platform layer may report progress and file growth, but must not choose the -best checkpoint or interpret convergence. - -## Inference - -`deepmd-python-inference` owns the model artifact, input ordering, type mapping, -command, batch size, declared predictions, and accuracy acceptance. Preserve -ordering and case identity; return predictions, logs, runtime metadata, and -failures without interpreting accuracy. - -## LAMMPS - -`lammps-deepmd` owns deployment, type mapping, LAMMPS inputs, ensemble, -timestep, ranks, output cadence, restart policy, and simulation acceptance. -Preserve trajectories, logs, final structures, and partial restarts without -rewriting those scientific choices. - -## Return boundary - -Return case/attempt/job/group identity, platform selections and versions, -input/output paths, scheduler/exit evidence, logs, retry lineage, and a platform -verdict. The owning skill determines training, inference, MD, and scientific -validity. diff --git a/skills/matmaster-dpa4-workflows/scripts/audit_cases.py b/skills/matmaster-dpa4-workflows/scripts/audit_cases.py deleted file mode 100755 index eb33a0cae6..0000000000 --- a/skills/matmaster-dpa4-workflows/scripts/audit_cases.py +++ /dev/null @@ -1,263 +0,0 @@ -#!/usr/bin/env python3 -# SPDX-License-Identifier: LGPL-3.0-or-later -"""Run a non-destructive first-pass audit of DPA4 workflow case folders.""" - -# ruff: noqa: T201 -- stdout is this command-line tool's result interface. - -from __future__ import ( - annotations, -) - -import argparse -import json -import math -import re -from pathlib import ( - Path, -) - -FATAL_PATTERNS = { - "unknown_dpa4": re.compile(r"Unknown model type:\s*dpa4", re.I), - "missing_mass": re.compile( - r"Not all per-type masses are set|Type\s+\d+\s+is missing", re.I - ), - "cuda_error": re.compile(r"CUDA Runtime|CUDA error|operation not supported", re.I), - "oom": re.compile(r"out of memory|\bOOM\b|std::bad_alloc", re.I), - "lost_atoms": re.compile(r"Lost atoms|Bond atoms missing|Out of range atoms", re.I), - "non_numeric": re.compile( - r"Non-numeric (?:atom coords|box dimensions|pressure|energy)|\bnan\b|\binf\b", - re.I, - ), - "segfault": re.compile(r"Segmentation fault|SIGSEGV", re.I), - "lammps_error": re.compile(r"^ERROR(?: on proc \d+)?:", re.I | re.M), - "python_exception": re.compile( - r"Traceback \(most recent call last\):|FileNotFoundError|RuntimeError:", re.I - ), - "distributed_error": re.compile( - r"NCCL (?:error|failure)|ProcessGroupNCCL|ChildFailedError", re.I - ), -} - -COMPLETE_PATTERNS = [ - re.compile(r"Total wall time:", re.I), - re.compile(r"Loop time of", re.I), - re.compile(r"Minimization stats:", re.I), -] - -ENERGY_RE = re.compile( - r"(?:PotEng|\bpe\b|potential[_ ]?energy|final[_ ]?energy)\s*[=:, ]+" - r"([-+]?\d+(?:\.\d*)?(?:[eE][-+]?\d+)?)", - re.I, -) - - -def read_text(path: Path, limit: int = 50_000_000) -> str: - try: - with path.open("r", errors="replace") as handle: - return handle.read(limit) - except OSError: - return "" - - -def find_case_dirs(root: Path) -> list[Path]: - markers = ( - "in.lammps", - "log.lammps", - "run.log", - "train.log", - "lcurve.out", - "input.json", - "structure.data", - "input.data", - ) - cases: set[Path] = set() - for marker in markers: - for path in root.rglob(marker): - if path.is_file(): - cases.add(path.parent) - return sorted(cases) - - -def infer_mode(case: Path, requested: str) -> str: - if requested != "auto": - return requested - inputs = [path for path in (case / "in.lammps", case / "in.lmp") if path.is_file()] - text = "\n".join(read_text(path) for path in inputs) - has_run = bool(re.search(r"^\s*run\s+\S+", text, re.M)) - has_minimize = bool(re.search(r"^\s*minimize\s+", text, re.M)) - if has_run: - return "md" - if has_minimize: - return "minimize" - training_input = case / "input.json" - if training_input.is_file() and re.search( - r'"training"\s*:', read_text(training_input) - ): - return "train" - return "unknown" - - -def extract_energy(case: Path, log_text: str) -> float | None: - for name in ("energy.dat", "final_energy.dat"): - path = case / name - if path.is_file(): - values = re.findall(r"[-+]?\d+(?:\.\d*)?(?:[eE][-+]?\d+)?", read_text(path)) - if values: - value = float(values[-1]) - return value if math.isfinite(value) else None - matches = ENERGY_RE.findall(log_text) - if matches: - value = float(matches[-1]) - return value if math.isfinite(value) else None - return None - - -def match_required(case: Path, patterns: list[str]) -> dict[str, list[str]]: - matched: dict[str, list[str]] = {} - for pattern in patterns: - paths = sorted( - str(path.relative_to(case)) for path in case.glob(pattern) if path.exists() - ) - matched[pattern] = paths - return matched - - -def audit_case(case: Path, root: Path, mode: str, required: list[str]) -> dict: - logs = [ - path - for path in ( - case / "log.lammps", - case / "run.log", - case / "train.log", - case / "stderr.log", - case / "log", - ) - if path.is_file() - ] - log_text = "\n".join(read_text(path) for path in logs) - fatal = [ - name for name, pattern in FATAL_PATTERNS.items() if pattern.search(log_text) - ] - inferred_mode = infer_mode(case, mode) - complete = any(pattern.search(log_text) for pattern in COMPLETE_PATTERNS) - required_matches = match_required(case, required) - missing_required = [ - pattern for pattern, matches in required_matches.items() if not matches - ] - reasons: list[str] = [] - if not logs: - reasons.append("missing_log") - elif not log_text.strip(): - reasons.append("empty_log") - if fatal: - reasons.extend(fatal) - if logs and inferred_mode in ("md", "minimize") and not complete: - reasons.append("missing_completion_marker") - reasons.extend(f"missing_required:{pattern}" for pattern in missing_required) - verdict = "passes_file_audit" if not reasons else "needs_review" - artifacts = { - "freeze_log": str(case / "freeze.log") - if (case / "freeze.log").is_file() - else None, - "final_structure": next( - ( - str(case / name) - for name in ("final.data", "relaxed.data") - if (case / name).is_file() - ), - None, - ), - "restart_files": sorted(str(path) for path in case.glob("restart*")), - "trajectory_candidates": sorted( - str(path) - for pattern in ("*.lammpstrj", "dump*", "trajectory/*") - for path in case.glob(pattern) - if path.is_file() - ), - "checkpoint_candidates": sorted( - str(path) - for pattern in ("*.pt", "checkpoint*", "ckpt/*") - for path in case.glob(pattern) - if path.is_file() - ), - "metric_candidates": sorted( - str(path) - for pattern in ("lcurve.out", "metrics*", "results/*") - for path in case.glob(pattern) - if path.is_file() - ), - } - return { - "case_id": str(case.relative_to(root)) or ".", - "case_dir": str(case), - "mode": inferred_mode, - "verdict": verdict, - "reasons": reasons, - "completion_marker": complete, - "fatal_signatures": fatal, - "last_detected_energy": extract_energy(case, log_text), - "required_matches": required_matches, - "artifacts": artifacts, - } - - -def main() -> int: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument( - "root", type=Path, help="Root directory containing case directories" - ) - parser.add_argument( - "--mode", - choices=("auto", "train", "inference", "minimize", "md"), - default="auto", - ) - parser.add_argument( - "--require", - action="append", - default=[], - metavar="GLOB", - help="Require a case-relative file glob; repeat for multiple artifacts", - ) - parser.add_argument( - "--json", type=Path, help="Write the full JSON report to this path" - ) - args = parser.parse_args() - - root = args.root.resolve() - if not root.is_dir(): - parser.error(f"not a directory: {root}") - cases = [ - audit_case(case, root, args.mode, args.require) for case in find_case_dirs(root) - ] - counts: dict[str, int] = {} - for case in cases: - counts[case["verdict"]] = counts.get(case["verdict"], 0) + 1 - report = { - "root": str(root), - "case_count": len(cases), - "mode_request": args.mode, - "required_patterns": args.require, - "verdict_counts": counts, - "warning": "A file/log audit pass is not training, inference, MD-quality, or scientific acceptance.", - "cases": cases, - } - if args.json: - args.json.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n") - print( - json.dumps( - { - key: report[key] - for key in ("root", "case_count", "verdict_counts", "warning") - }, - ensure_ascii=False, - indent=2, - ) - ) - for case in cases: - if case["verdict"] != "passes_file_audit": - print(f"REVIEW\t{case['case_id']}\t{','.join(case['reasons'])}") - return 0 if cases and counts.get("needs_review", 0) == 0 else 1 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/skills/matmaster-dpa4-workflows/scripts/check_environment.py b/skills/matmaster-dpa4-workflows/scripts/check_environment.py deleted file mode 100755 index 877790c222..0000000000 --- a/skills/matmaster-dpa4-workflows/scripts/check_environment.py +++ /dev/null @@ -1,150 +0,0 @@ -#!/usr/bin/env python3 -# SPDX-License-Identifier: LGPL-3.0-or-later -"""Inventory a MatMaster/Bohrium job environment without exposing secrets.""" - -# ruff: noqa: T201 -- stdout is this command-line tool's result interface. - -from __future__ import ( - annotations, -) - -import argparse -import hashlib -import json -import os -import shutil -import subprocess -from pathlib import ( - Path, -) - -COMMANDS = { - "bohr": ["bohr", "version"], - "dp": ["dp", "--version"], - "lmp": ["lmp", "-h"], - "lmp_mpi": ["lmp_mpi", "-h"], - "nvidia_smi": [ - "nvidia-smi", - "--query-gpu=name,memory.total,driver_version", - "--format=csv,noheader", - ], -} - - -def sha256(path: Path) -> str: - digest = hashlib.sha256() - with path.open("rb") as handle: - for chunk in iter(lambda: handle.read(1024 * 1024), b""): - digest.update(chunk) - return digest.hexdigest() - - -def probe(command: list[str]) -> dict: - executable = shutil.which(command[0]) - result = {"path": executable, "available": executable is not None} - if executable is None: - return result - completed = subprocess.run( - command, capture_output=True, text=True, timeout=20, check=False - ) - output = (completed.stdout or completed.stderr).strip().splitlines() - result.update( - { - "returncode": completed.returncode, - "first_line": output[0][:500] if output else "", - } - ) - return result - - -def main() -> int: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--model", type=Path) - parser.add_argument("--case-dir", type=Path) - parser.add_argument( - "--require-file", - action="append", - default=[], - help="Case-relative required file", - ) - parser.add_argument("--probe", action="store_true", help="Run safe version probes") - parser.add_argument("--require-bohr", action="store_true") - parser.add_argument("--require-deepmd", action="store_true") - parser.add_argument("--require-lammps", action="store_true") - parser.add_argument( - "--require-runtime", action="store_true", help="Require both DeePMD and LAMMPS" - ) - args = parser.parse_args() - - commands = {} - for name, command in COMMANDS.items(): - path = shutil.which(command[0]) - commands[name] = ( - probe(command) - if args.probe - else {"path": path, "available": path is not None} - ) - report = { - "commands": commands, - "environment_present": { - "BOHR_ACCESS_KEY": bool(os.environ.get("BOHR_ACCESS_KEY")), - "ACCESS_KEY": bool(os.environ.get("ACCESS_KEY")), - "PROJECT_ID": bool(os.environ.get("PROJECT_ID")), - "OPENAPI_HOST": bool(os.environ.get("OPENAPI_HOST")), - "TIEFBLUE_HOST": bool(os.environ.get("TIEFBLUE_HOST")), - }, - "persistent_paths": { - "/personal": { - "exists": Path("/personal").exists(), - "writable": os.access("/personal", os.W_OK), - }, - "/share": { - "exists": Path("/share").exists(), - "writable": os.access("/share", os.W_OK), - }, - }, - } - if args.model: - model = args.model.resolve() - report["model"] = { - "path": str(model), - "exists": model.is_file(), - "size": model.stat().st_size if model.is_file() else None, - "sha256": sha256(model) if model.is_file() else None, - } - if args.case_dir: - case = args.case_dir.resolve() - required = ["run.sh", *args.require_file] - report["case"] = { - "path": str(case), - "exists": case.is_dir(), - "required": {name: (case / name).is_file() for name in required}, - } - failures = [] - if args.model and not report["model"]["exists"]: - failures.append("model_missing") - if args.case_dir: - if not report["case"]["exists"]: - failures.append("case_dir_missing") - failures.extend( - f"required_file_missing:{name}" - for name, present in report["case"]["required"].items() - if not present - ) - if args.require_bohr and not commands["bohr"]["available"]: - failures.append("bohr_cli_missing") - if (args.require_deepmd or args.require_runtime) and not commands["dp"][ - "available" - ]: - failures.append("dp_missing") - if (args.require_lammps or args.require_runtime) and not ( - commands["lmp"]["available"] or commands["lmp_mpi"]["available"] - ): - failures.append("lammps_missing") - report["failures"] = failures - print(json.dumps(report, ensure_ascii=False, indent=2)) - return 1 if failures else 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/skills/matmaster-dpa4-workflows/scripts/make_job_spec.py b/skills/matmaster-dpa4-workflows/scripts/make_job_spec.py deleted file mode 100755 index 33f4404542..0000000000 --- a/skills/matmaster-dpa4-workflows/scripts/make_job_spec.py +++ /dev/null @@ -1,92 +0,0 @@ -#!/usr/bin/env python3 -# SPDX-License-Identifier: LGPL-3.0-or-later -"""Generate and validate a Bohrium container job specification.""" - -# ruff: noqa: T201 -- stdout is this command-line tool's result interface. - -from __future__ import ( - annotations, -) - -import argparse -import json -import shlex -from pathlib import ( - Path, -) - - -def main() -> int: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("case_dir", type=Path) - parser.add_argument("--output", type=Path, required=True) - parser.add_argument("--project-id", type=int, required=True) - parser.add_argument("--image", required=True) - parser.add_argument("--machine", required=True) - parser.add_argument("--name", required=True) - parser.add_argument("--command", default="bash run.sh") - parser.add_argument("--log-file", default="run.log") - parser.add_argument("--backward", action="append", default=[]) - parser.add_argument("--result-path") - parser.add_argument("--max-run-time", type=int) - parser.add_argument("--max-reschedule-times", type=int, default=0) - parser.add_argument("--nnode", type=int, default=1) - parser.add_argument("--disk-size", type=int) - args = parser.parse_args() - - case_dir = args.case_dir.resolve() - errors = [] - if not case_dir.is_dir(): - errors.append(f"case directory does not exist: {case_dir}") - if "/" not in args.image or ":" not in args.image: - errors.append("image must be a full registry path with a tag") - if "/root/input" in args.command: - errors.append("command must not assume /root/input; use relative paths") - try: - tokens = shlex.split(args.command) - except ValueError as exc: - errors.append(f"invalid command quoting: {exc}") - tokens = [] - if ( - tokens[:2] == ["bash", "run.sh"] - and case_dir.is_dir() - and not (case_dir / "run.sh").is_file() - ): - errors.append("default command requires case_dir/run.sh") - if args.project_id <= 0: - errors.append("project ID must be positive") - if args.max_run_time is not None and args.max_run_time <= 0: - errors.append("max run time must be positive minutes") - if args.max_reschedule_times < 0 or args.nnode <= 0: - errors.append("reschedule count must be nonnegative and nnode positive") - if errors: - for error in errors: - print(f"ERROR: {error}") - return 2 - - spec = { - "job_name": args.name, - "command": args.command, - "log_file": args.log_file, - "backward_files": args.backward, - "project_id": args.project_id, - "machine_type": args.machine, - "image_address": args.image, - "job_type": "container", - "max_reschedule_times": args.max_reschedule_times, - "nnode": args.nnode, - } - if args.result_path: - spec["result_path"] = args.result_path - if args.max_run_time is not None: - spec["max_run_time"] = args.max_run_time - if args.disk_size is not None: - spec["disk_size"] = args.disk_size - args.output.parent.mkdir(parents=True, exist_ok=True) - args.output.write_text(json.dumps(spec, ensure_ascii=False, indent=2) + "\n") - print(args.output.resolve()) - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) From 06bc4d908bc5260303f79aee1302306f05961977 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 14 Aug 2026 22:29:27 +0800 Subject: [PATCH 17/24] docs(skills): gate LAMMPS runtime by capability --- skills/lammps-deepmd/SKILL.md | 84 +++++++------------ .../references/commands-and-workflow.md | 2 +- source/tests/common/test_agent_skills.py | 36 ++++++++ 3 files changed, 68 insertions(+), 54 deletions(-) create mode 100644 source/tests/common/test_agent_skills.py diff --git a/skills/lammps-deepmd/SKILL.md b/skills/lammps-deepmd/SKILL.md index 7957a2f5f2..8d8ee53569 100644 --- a/skills/lammps-deepmd/SKILL.md +++ b/skills/lammps-deepmd/SKILL.md @@ -1,9 +1,9 @@ --- name: lammps-deepmd description: > - A tool and knowledge base for running molecular dynamics (MD) simulations in LAMMPS with the DeePMD-kit plugin. It handles input script preparation, ensemble selection (NVE/NVT/NPT), and job execution via `uv` or offline binaries. + A tool and knowledge base for running molecular dynamics (MD) simulations in LAMMPS with the DeePMD-kit plugin. It handles input script preparation, ensemble selection (NVE/NVT/NPT), and execution with a verified DeePMD-enabled runtime. USE WHEN you need to set up, write, explain, or execute a LAMMPS molecular dynamics simulation using a DeePMD machine learning potential (for example `.pb`, `.pth`, or DPA4/SeZM `.pt2`). -compatibility: Requires LAMMPS with DeePMD-kit support. Online mode uses `uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp`; offline mode requires a user-provided LAMMPS executable or module. +compatibility: Requires a user-provided, containerized, or source-built LAMMPS runtime with DeePMD-kit support. Verify capabilities in the target environment. license: LGPL-3.0-or-later metadata: author: OpenClaw @@ -18,55 +18,45 @@ Use this skill when the user wants to run molecular dynamics in LAMMPS with a De ## Agent responsibilities -1. Confirm the available execution mode: - - **Online mode**: if internet access is available and `uv` is installed, prefer - `uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp ...` - - **Offline mode**: do **not** guess the executable. Ask the user which LAMMPS command, module, or container should be used. +1. Confirm the available execution runtime: + - For a build from the current checkout, record the resolved Git commit SHA. + - For an installed binary, module, or container, record the exact command and + runtime versions. + - Do not infer capabilities from a future release or an artifact suffix. 1. Confirm the minimum simulation inputs: - structure/data file (for example `data.system`) - -- DeePMD model artifact; read `references/model-deployment.md` for a training checkpoint, DPA4/SeZM, or an unclear export path -- atom type to element mapping, including required per-type masses if the data file does not define them -- target ensemble (NVE, NVT, NPT, or another explicitly requested setup) -- temperature, pressure if applicable, timestep, and total number of steps + - DeePMD model artifact; read `references/model-deployment.md` for a training + checkpoint, DPA4/SeZM, or an unclear export path + - atom type to element mapping, including required per-type masses if the data + file does not define them + - target ensemble (NVE, NVT, NPT, or another explicitly requested setup) + - temperature, pressure if applicable, timestep, and total number of steps 1. Write the LAMMPS input script yourself instead of asking the user to hand-write it. 1. Keep the example readable and fully explained. If you include an example input script, explain what **every command** does. 1. When possible, validate command availability against the LAMMPS docs or local `lmp -h` output before execution. 1. Report clearly which command was run, which files were used, and where outputs were written. -## Decide the execution mode - -### Online mode (preferred when internet access is available) - -Use: - -```bash -uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -in input.lammps -``` - -If you need to inspect the local command-line help: - -```bash -uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -h -``` +## Verify the execution runtime -Notes: +Use an existing site-installed binary, container, or build from the current +checkout. Before execution: -- This is the preferred path because it can provision LAMMPS and DeePMD-kit on demand. -- The pins match the LAMMPS dependency declared by the DPA4-capable - `deepmd-kit==3.2.0b0` release. Update both pins together after validating a - newer pair. -- If the environment is slow or the packages are large, warn the user that the first run may take time. +- record `git rev-parse HEAD` for a source checkout, or record `dp --version` and + the exact installed package/container identity; +- inspect the selected LAMMPS command with `-h` and confirm that it provides the + required DeePMD pair style and model-artifact route; +- follow `references/model-deployment.md` for DPA4 `.pt2` and multi-rank + capability gates; +- do not install or upgrade packages silently, and do not claim support from an + unreleased version number. -### Offline mode - -If internet access is unavailable or the user explicitly wants a site-installed binary, ask a concrete question such as: +If no verified runtime is available, ask a concrete question such as: - "Which LAMMPS executable should I use, for example `lmp`, `lmp_mpi`, `mpirun -np 8 lmp`, or an HPC module command?" -- "Do you already have a DeePMD-enabled LAMMPS build on this machine or cluster?" +- "Do you already have a DeePMD-enabled LAMMPS build or container on this machine or cluster?" -Do not invent a binary name or module name. +Do not invent a binary, module, container, or package version. ## Minimal information to collect @@ -80,7 +70,7 @@ Ask only for what is missing: - timestep - run length in steps - whether velocities should be generated from scratch -- preferred execution command if offline +- execution command, module, container, or source checkout ## Recommended workflow @@ -282,21 +272,9 @@ When using NPT, it is often useful to keep `vol`, `lx`, `ly`, and `lz` in the th ## Execution templates -### Online run - -```bash -uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -in input.lammps -``` - -### Online help - -```bash -uvx --from 'lammps==2025.7.22.2.0' --with 'deepmd-kit[gpu,torch,lmp]==3.2.0b0' lmp -h -``` - -### Offline run +### Verified run -Only after the user specifies the executable, use a command such as one of these exact patterns: +Only after the runtime is identified, use the matching command, for example: ```bash lmp -in input.lammps @@ -304,7 +282,7 @@ mpirun -np 8 lmp_mpi -in input.lammps srun lmp -in input.lammps ``` -The agent must not choose one of these on its own without user guidance in offline mode. +Do not choose one of these without evidence that it is the verified runtime. ## Output checklist diff --git a/skills/lammps-deepmd/references/commands-and-workflow.md b/skills/lammps-deepmd/references/commands-and-workflow.md index da21fa129a..96815a4b3f 100644 --- a/skills/lammps-deepmd/references/commands-and-workflow.md +++ b/skills/lammps-deepmd/references/commands-and-workflow.md @@ -7,7 +7,7 @@ This reference expands the main skill with practical operating guidance. 1. Prefer small, explicit input scripts over clever but opaque templates. 1. Explain every command in the example script, because many users treat the example as a starting point for their own production run. 1. If the user asks to run a simulation, always confirm the structure file and DeePMD model file before execution. -1. If the user asks for offline execution, ask which exact LAMMPS command should be used instead of guessing. +1. Ask which exact LAMMPS command, module, container, or source-built runtime should be used instead of guessing or installing one silently. 1. If the user only asks for a template, do not overcomplicate it with advanced computes or fixes unless they are needed. ## Suggested smoke test strategy diff --git a/source/tests/common/test_agent_skills.py b/source/tests/common/test_agent_skills.py new file mode 100644 index 0000000000..b0da5ba92c --- /dev/null +++ b/source/tests/common/test_agent_skills.py @@ -0,0 +1,36 @@ +# SPDX-License-Identifier: LGPL-3.0-or-later +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[3] +LAMMPS_SKILL = ROOT / "skills" / "lammps-deepmd" / "SKILL.md" + + +def test_lammps_skill_uses_capability_gated_runtime() -> None: + text = LAMMPS_SKILL.read_text(encoding="utf-8") + + assert "3.2.0b0" not in text + assert "uvx --from" not in text + assert "record the resolved Git commit SHA" in text + assert "do not claim support from an" in text + assert "unreleased version number" in text + assert "do not install or upgrade packages silently" in text + + +def test_lammps_required_inputs_remain_nested() -> None: + text = LAMMPS_SKILL.read_text(encoding="utf-8") + section = text.split("1. Confirm the minimum simulation inputs:", 1)[1].split( + "1. Write the LAMMPS input script", 1 + )[0] + + for required in ( + "structure/data file", + "DeePMD model artifact", + "atom type to element mapping", + "target ensemble", + "temperature, pressure", + ): + assert any( + line.startswith(" - ") and required in line + for line in section.splitlines() + ) From 2541ae00cf316dd82107e179f5749089becede41 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 14 Aug 2026 14:30:44 +0000 Subject: [PATCH 18/24] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- skills/lammps-deepmd/SKILL.md | 6 ++++++ source/tests/common/test_agent_skills.py | 5 +++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/skills/lammps-deepmd/SKILL.md b/skills/lammps-deepmd/SKILL.md index 8d8ee53569..bd2b961d01 100644 --- a/skills/lammps-deepmd/SKILL.md +++ b/skills/lammps-deepmd/SKILL.md @@ -19,11 +19,14 @@ Use this skill when the user wants to run molecular dynamics in LAMMPS with a De ## Agent responsibilities 1. Confirm the available execution runtime: + - For a build from the current checkout, record the resolved Git commit SHA. - For an installed binary, module, or container, record the exact command and runtime versions. - Do not infer capabilities from a future release or an artifact suffix. + 1. Confirm the minimum simulation inputs: + - structure/data file (for example `data.system`) - DeePMD model artifact; read `references/model-deployment.md` for a training checkpoint, DPA4/SeZM, or an unclear export path @@ -33,8 +36,11 @@ Use this skill when the user wants to run molecular dynamics in LAMMPS with a De - temperature, pressure if applicable, timestep, and total number of steps 1. Write the LAMMPS input script yourself instead of asking the user to hand-write it. + 1. Keep the example readable and fully explained. If you include an example input script, explain what **every command** does. + 1. When possible, validate command availability against the LAMMPS docs or local `lmp -h` output before execution. + 1. Report clearly which command was run, which files were used, and where outputs were written. ## Verify the execution runtime diff --git a/source/tests/common/test_agent_skills.py b/source/tests/common/test_agent_skills.py index b0da5ba92c..70c9d40171 100644 --- a/source/tests/common/test_agent_skills.py +++ b/source/tests/common/test_agent_skills.py @@ -1,6 +1,7 @@ # SPDX-License-Identifier: LGPL-3.0-or-later -from pathlib import Path - +from pathlib import ( + Path, +) ROOT = Path(__file__).resolve().parents[3] LAMMPS_SKILL = ROOT / "skills" / "lammps-deepmd" / "SKILL.md" From 4ccbd9153774d456f05298433277c4966ee83b11 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 14 Aug 2026 22:36:26 +0800 Subject: [PATCH 19/24] docs(skills): define complete held-out evaluation --- skills/deepmd-finetune-dpa4/SKILL.md | 20 +++--- skills/deepmd-python-inference/SKILL.md | 4 ++ .../references/held-out-evaluation.md | 61 +++++++++++++++++++ source/tests/common/test_agent_skills.py | 46 ++++++++++++++ 4 files changed, 124 insertions(+), 7 deletions(-) create mode 100644 skills/deepmd-python-inference/references/held-out-evaluation.md diff --git a/skills/deepmd-finetune-dpa4/SKILL.md b/skills/deepmd-finetune-dpa4/SKILL.md index 4f8361dbd7..fa33f5ab45 100644 --- a/skills/deepmd-finetune-dpa4/SKILL.md +++ b/skills/deepmd-finetune-dpa4/SKILL.md @@ -51,8 +51,12 @@ architecture, and `type_map` before use. ## Before fine-tuning 1. Confirm the checkpoint exists and can be inspected. -1. Confirm training and validation systems, labels, and element type maps. -1. Keep a held-out test set that is not used for training or model selection. +1. Confirm training, validation, and held-out systems, labels, and element type maps. +1. Split correlated frames by independent system, trajectory, or source family; + do not create a nominal held-out set by randomly splitting adjacent frames. +1. Validate each DeePMD system before training: `natoms` is the number of tokens + in `type.raw`, coordinate and force widths are `3 * natoms`, and every used + label is finite and frame-aligned. 1. Start from the exact checkpoint architecture. Introducing new element types, changing architecture, or combining specialized spin/property/multi-task configurations requires separate compatibility validation. @@ -143,9 +147,10 @@ architecture from the source checkpoint before adding `model.lora`. Do not add ## Monitor and validate Monitor `lcurve.out` for non-finite values and train/validation divergence. -Select a checkpoint using validation data, then evaluate the selected checkpoint -on the complete held-out test systems. Report energy and force errors, plus -virial errors when those labels are part of the task. +Select a checkpoint using validation data, then follow the +[complete held-out evaluation](../deepmd-python-inference/references/held-out-evaluation.md) +with that exact native checkpoint and every held-out system. Export for deployment +only after the complete evaluation meets the task's declared thresholds. ## Export and test @@ -174,12 +179,13 @@ again when loading that archive. - [ ] The stored descriptor identifies DPA4/SeZM; the `.pt` suffix was not used as proof. - [ ] The checkpoint source, DeePMD-kit revision, architecture, and type map are recorded. - [ ] The intended branch is explicit for a multi-task checkpoint. -- [ ] Training, validation, and held-out test systems are separate. +- [ ] Training, validation, and held-out systems are independent by source family. +- [ ] Every admitted system has consistent atom counts, shapes, labels, and type maps. - [ ] The input architecture is compatible with the checkpoint. - [ ] Standard fine-tuning versus LoRA was selected from the task layout and domain shift. - [ ] A resumable LoRA checkpoint is distinguished from a merged best checkpoint. - [ ] LoRA uses a complete base configuration and is not silently overwritten. -- [ ] Training and held-out metrics are finite and reported with units. +- [ ] Complete held-out metrics, sample counts, and reference-label scales are reported. - [ ] The selected `.pt` checkpoint was exported to and tested as `.pt2`. ## References diff --git a/skills/deepmd-python-inference/SKILL.md b/skills/deepmd-python-inference/SKILL.md index f3cb9e148e..59811bcb15 100644 --- a/skills/deepmd-python-inference/SKILL.md +++ b/skills/deepmd-python-inference/SKILL.md @@ -33,6 +33,7 @@ e, f, v = dp.eval(coord, cell, atype) - Built-in pretrained model name (e.g., `DPA-3.2-5M`) - PyTorch checkpoint (`.pt`), whose stored model configuration must be inspected before choosing an inference or export path 1. Read `references/model-artifacts.md` for `.pt`/`.pt2` models or whenever the artifact route is unclear. +1. Read `references/held-out-evaluation.md` for complete labeled evaluation used for checkpoint selection or production admission. 1. Determine the inference task: - Single-frame prediction (energy, force, virial) - Batch prediction over multiple frames @@ -203,6 +204,9 @@ Virial RMSE/Natoms : 2.957e-04 eV With `-d test_detail`, per-frame predictions are saved to files for further analysis. +The 30-frame commands above are bounded examples, not complete held-out evaluation. +Use `references/held-out-evaluation.md` when the result admits a model for production. + ## Complete Example: Train, Freeze, and Inference ```python diff --git a/skills/deepmd-python-inference/references/held-out-evaluation.md b/skills/deepmd-python-inference/references/held-out-evaluation.md new file mode 100644 index 0000000000..2405587d2f --- /dev/null +++ b/skills/deepmd-python-inference/references/held-out-evaluation.md @@ -0,0 +1,61 @@ +# Complete held-out evaluation + +Read this reference when labeled DeePMD systems are used to select a checkpoint +or admit a model for production. A bounded smoke test is not a complete +held-out evaluation. + +## Admit the data + +- Keep held-out systems independent of training and validation by system, + trajectory, or source family; do not randomly split correlated adjacent frames. +- Enumerate every held-out system and all of its `set.*` directories. +- Set `natoms` to the number of whitespace-separated entries in `type.raw`. + Require coordinate and force widths of `3 * natoms`, one energy row per frame, + nine box values per periodic frame, and finite values for every evaluated label. +- Interpret `type.raw` as zero-based indices into that system's ordered + `type_map.raw`. Compare model and data types by element identity, not by copying + dataset indices into another type map. + +## Run every system + +Use the backend required by the exact candidate artifact. For a DPA4/SeZM native +checkpoint, run one command per held-out system: + +```bash +dp --pt test -m selected.pt -s held_out/system.000 -n 0 -d details/system.000 +``` + +`-n 0` evaluates all frames. Require explicit `-m`, `-s`, and a unique `-d` +prefix for each system. Preserve the command, log, true exit code, checkpoint +SHA256, and dataset identity. Do not overwrite existing detail files silently. + +## Validate detail outputs + +For an energy model, retain the emitted total-energy (`.e.out`), +energy-per-atom (`.e_peratom.out`), and force (`.f.out`) details. Retain +virial/stress details only when both reference labels and model outputs exist. +Require: + +- energy rows equal the evaluated frame count; +- force rows equal `frames * natoms`, with xyz stored as columns; +- all reference and prediction values are finite; +- energy-per-atom errors come directly from `.e_peratom.out`, or total-energy + errors are divided by `natoms` exactly once. + +Reject missing systems, partial frame coverage, reused detail prefixes, and +results tied to another checkpoint hash. + +## Report and decide + +For every available label, report per-system MAE, RMSE, units, sample count, and +the population standard deviation (`ddof=0`) of the corresponding held-out +reference values. Report an absent label as `N/A`, never zero. Aggregate from all +retained rows; do not average per-system RMSE values. + +Build parity plots from unrounded detail rows, with reference on x, prediction on +y, an equal-aspect `y=x` line, system identity, checkpoint hash, units, sample +count, RMSE, and reference-label standard deviation. + +Admit the candidate only when every declared held-out system is complete and the +declared thresholds pass. Training logs, a successful freeze, or a LAMMPS +canary cannot replace this evaluation. diff --git a/source/tests/common/test_agent_skills.py b/source/tests/common/test_agent_skills.py index 70c9d40171..b6fe98a709 100644 --- a/source/tests/common/test_agent_skills.py +++ b/source/tests/common/test_agent_skills.py @@ -5,6 +5,17 @@ ROOT = Path(__file__).resolve().parents[3] LAMMPS_SKILL = ROOT / "skills" / "lammps-deepmd" / "SKILL.md" +DPA4_FINETUNE_SKILL = ROOT / "skills" / "deepmd-finetune-dpa4" / "SKILL.md" +INFERENCE_SKILL = ROOT / "skills" / "deepmd-python-inference" / "SKILL.md" +HELD_OUT_REFERENCE = ( + ROOT + / "skills" + / "deepmd-python-inference" + / "references" + / "held-out-evaluation.md" +) +DP_TEST_ENTRYPOINT = ROOT / "deepmd" / "entrypoints" / "test.py" +ENERGY_TESTER = ROOT / "deepmd" / "infer" / "model_test" / "ener.py" def test_lammps_skill_uses_capability_gated_runtime() -> None: @@ -35,3 +46,38 @@ def test_lammps_required_inputs_remain_nested() -> None: line.startswith(" - ") and required in line for line in section.splitlines() ) + + +def test_complete_held_out_evaluation_is_routed_and_evidence_complete() -> None: + finetune = DPA4_FINETUNE_SKILL.read_text(encoding="utf-8") + inference = INFERENCE_SKILL.read_text(encoding="utf-8") + held_out = HELD_OUT_REFERENCE.read_text(encoding="utf-8") + + assert "references/held-out-evaluation.md" in inference + assert "held-out-evaluation.md" in finetune + assert "dp --pt test -m selected.pt" in held_out + assert "-n 0 -d details/system.000" in held_out + assert "one command per held-out system" in held_out + assert "population standard deviation (`ddof=0`)" in held_out + assert "do not average per-system RMSE values" in held_out + assert "Training logs, a successful freeze" in held_out + + +def test_complete_held_out_evaluation_checks_dataset_shapes() -> None: + held_out = HELD_OUT_REFERENCE.read_text(encoding="utf-8") + + assert "number of whitespace-separated entries in `type.raw`" in held_out + assert "coordinate and force widths of `3 * natoms`" in held_out + assert "force rows equal `frames * natoms`" in held_out + assert "divided by `natoms` exactly once" in held_out + assert "zero-based indices" in held_out + + +def test_held_out_contract_matches_dp_test_source() -> None: + entrypoint = DP_TEST_ENTRYPOINT.read_text(encoding="utf-8") + energy_tester = ENERGY_TESTER.read_text(encoding="utf-8") + + assert "if numb_test == 0:" in entrypoint + assert 'detail_path.with_suffix(".e.out")' in energy_tester + assert 'detail_path.with_suffix(".e_peratom.out")' in energy_tester + assert 'detail_path.with_suffix(".f.out")' in energy_tester From 0807ca6199559d6c3e439d36ad7c0726082b2732 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 14 Aug 2026 22:50:52 +0800 Subject: [PATCH 20/24] docs(skills): harden DPA4 LAMMPS deployment --- skills/lammps-deepmd/SKILL.md | 36 +++++++----- .../references/commands-and-workflow.md | 29 ++++++---- .../references/model-deployment.md | 55 ++++++++++++++----- source/tests/common/test_agent_skills.py | 48 ++++++++++++++++ 4 files changed, 130 insertions(+), 38 deletions(-) diff --git a/skills/lammps-deepmd/SKILL.md b/skills/lammps-deepmd/SKILL.md index bd2b961d01..0305a05f92 100644 --- a/skills/lammps-deepmd/SKILL.md +++ b/skills/lammps-deepmd/SKILL.md @@ -84,7 +84,7 @@ Ask only for what is missing: 1. Read `references/model-deployment.md` when the model needs export, its artifact type is unclear, or explicit element mapping is required. 1. Draft `input.lammps`. 1. Explain the script to the user if they asked for an explanation or if the script is nontrivial. -1. Run a short smoke test first when reasonable. +1. Follow the staged canary in `references/commands-and-workflow.md` before production. 1. Run the full simulation. 1. Summarize outputs such as `log.lammps`, dump trajectories, restart files, and thermodynamic data. @@ -105,15 +105,17 @@ atom_style atomic neighbor 1.0 bin +atom_modify map yes read_data data.system mass 1 28.0855 mass 2 15.999 pair_style deepmd graph_compressed.pb -pair_coeff * * +pair_coeff * * Si O thermo_style custom step temp pe ke etotal press vol lx ly lz xy xz yz thermo ${THERMO_FREQ} -dump 1 all custom ${DUMP_FREQ} traj.lammpstrj id type x y z +dump 1 all custom ${DUMP_FREQ} traj.lammpstrj id type element x y z +dump_modify 1 element Si O sort id velocity all create ${TEMP} 743574 fix 1 all nvt temp ${TEMP} ${TEMP} ${TAU_T} @@ -170,9 +172,15 @@ run ${NSTEPS} - Uses the `bin` neighbor-building method. - Neighbor lists help LAMMPS efficiently find nearby atoms for force evaluation. +- `atom_modify map yes` + + - Creates an atom-ID map required by the documented DPA4 route. + - It must appear before `read_data`. + - `read_data data.system` - Reads the initial atomic structure, atom types, simulation box, and related information from the LAMMPS data file `data.system`. + - The data file's first line is a skipped title; actual header counts begin after it. - Replace this filename with the actual user file. - `mass 1 28.0855`, `mass 2 15.999` @@ -186,10 +194,11 @@ run ${NSTEPS} - Loads the DeePMD model from `graph_compressed.pb`. - Replace the model filename with the actual model path, for example `graph.pb`, `graph-compress.pb`, or another supported exported model. -- `pair_coeff * *` +- `pair_coeff * * Si O` - - Activates the previously selected pair style for all atom types. - - For DeePMD this often takes the simple form `* *` because the mapping is embedded in the model workflow rather than through conventional pairwise parameters. + - Activates the pair style for all local atom types. + - Maps LAMMPS type 1 to `Si` and type 2 to `O`, matching the masses and dump labels. + - Inspect the actual model type map and replace this example order; do not infer it from integer type IDs. - `thermo_style custom step temp pe ke etotal press vol lx ly lz xy xz yz` @@ -208,14 +217,15 @@ run ${NSTEPS} - Prints the thermo block every `THERMO_FREQ` timesteps. -- `dump 1 all custom ${DUMP_FREQ} traj.lammpstrj id type x y z` +- `dump 1 all custom ${DUMP_FREQ} traj.lammpstrj id type element x y z` + + - Creates dump ID `1` and writes every `DUMP_FREQ` steps. + - Saves `id`, local type, mapped element, and coordinates to `traj.lammpstrj`. + +- `dump_modify 1 element Si O sort id` - - Creates dump ID `1`. - - Dumps atoms from group `all`. - - Uses the `custom` dump format. - - Writes every `DUMP_FREQ` steps. - - Saves to `traj.lammpstrj`. - - Outputs per-atom columns `id type x y z`. + - Uses the same local type-to-element order as `pair_coeff`. + - Sorts each frame by stable atom ID, not by element or model type-map position. - `velocity all create ${TEMP} 743574` diff --git a/skills/lammps-deepmd/references/commands-and-workflow.md b/skills/lammps-deepmd/references/commands-and-workflow.md index 96815a4b3f..48edb9ebb8 100644 --- a/skills/lammps-deepmd/references/commands-and-workflow.md +++ b/skills/lammps-deepmd/references/commands-and-workflow.md @@ -8,25 +8,31 @@ This reference expands the main skill with practical operating guidance. 1. Explain every command in the example script, because many users treat the example as a starting point for their own production run. 1. If the user asks to run a simulation, always confirm the structure file and DeePMD model file before execution. 1. Ask which exact LAMMPS command, module, container, or source-built runtime should be used instead of guessing or installing one silently. +1. Keep shell environment variables and LAMMPS variables distinct; pass values + with an explicit LAMMPS mechanism instead of copying shell syntax into input. +1. Keep a vacuum or nonperiodic slab axis fixed; do not barostat that direction + unless the scientific task explicitly requires changing it. 1. If the user only asks for a template, do not overcomplicate it with advanced computes or fixes unless they are needed. -## Suggested smoke test strategy +## Suggested canary strategy -Before a long production run, consider a short test such as: +Before a long production run, stage validation as: -```lammps -run 100 +```text +run 0 -> short NVE when physically appropriate -> short requested ensemble -> production ``` -This helps catch obvious issues such as: +Check the complete LAMMPS exit code and log at every stage. A canary passes only +when the model loads, thermodynamic values are finite, the atom count is stable, +and early temperature, pressure, and controlled variables are physically +compatible with the initial state. Exit code zero alone is insufficient. -- missing model file -- unsupported pair style in the local LAMMPS build -- malformed data file -- missing per-type masses in the data file or input script -- immediate numerical instability +This catches obvious issues such as: -Then replace the short run with the intended production length. +- unsupported model artifact or pair style in the selected runtime; +- malformed data headers, boxes, coordinates, or triclinic tilt factors; +- missing masses or inconsistent element/type mapping; +- immediate numerical instability or lost atoms. ## Typical files in a DeePMD-LAMMPS job @@ -39,6 +45,7 @@ Then replace the short run with the intended production length. ## Caution points - The correct timestep depends on the physical system and the DeePMD model quality. +- The first line of a LAMMPS data file is a skipped title; put header counts after it. - Ensure every atom type has a mass, either in the LAMMPS data file `Masses` section or via explicit `mass` commands after `read_data`. - `velocity ... create ...` should usually not be repeated when continuing from a restart. - NPT settings need physically sensible damping constants; avoid copying values blindly. diff --git a/skills/lammps-deepmd/references/model-deployment.md b/skills/lammps-deepmd/references/model-deployment.md index 2bd4963138..4f3bb59718 100644 --- a/skills/lammps-deepmd/references/model-deployment.md +++ b/skills/lammps-deepmd/references/model-deployment.md @@ -40,6 +40,11 @@ during export: dp --pt freeze -c model.ckpt.pt -o frozen_model --head SELECTED_BRANCH ``` +Create and consume the archive on the same target physical compute node and +allocation: inspect the native checkpoint -> freeze `.pt2` -> `run 0` -> bounded +MD -> production. Do not freeze in job or node A and move the archive to B unless +portability has been independently validated for that exact device and toolchain. + Two DPA4 `.pt2` export contracts exist. `dp --pt freeze` uses the DPA4-specific `edge_vec` ABI. `dp --pt_expt freeze --lower-kind graph` uses the NeighborGraph ABI. They share a suffix but are not interchangeable contracts, and a `.pt2` @@ -70,40 +75,62 @@ that metadata or nested artifact is absent. ## Atom-type mapping -LAMMPS atom types are local integer IDs. Model types are ordered element names. -Build an explicit mapping instead of assuming the numbers are interchangeable. -For example: +LAMMPS atom types, dataset type indices, and model types are separate namespaces. +Inspect the artifact's ordered type map, for example with +`dp --pt show model.pt type-map`, and treat element identity as the bridge. +For DeePMD data, decode each zero-based `type.raw` index through that system's +ordered `type_map.raw`; do not reuse the dataset integer as a LAMMPS type. + +Use compact one-based LAMMPS types for the elements present in the structure and +write the same element order in masses, `pair_coeff`, and dump metadata: ```lammps mass 1 15.999 mass 2 1.008 pair_coeff * * O H +dump 1 all custom 100 traj.lammpstrj id type element x y z +dump_modify 1 element O H sort id ``` -Here LAMMPS type 1 maps to `O` and type 2 maps to `H`. Require that: +Here LAMMPS type 1 maps to `O` and type 2 maps to `H`. `dump_modify ... element` +labels each local type with that same mapping, while `sort id` gives a stable +per-frame atom order. Do not sort atoms by element or model type-map position. +Require that: - every LAMMPS atom type has a mass; -- every mapped element is supported by the model type map; -- the `pair_coeff` element order matches LAMMPS type order; +- every mapped element is supported by the inspected model type map; +- `pair_coeff`, masses, and dump element labels share one LAMMPS type order; - the structure's atom and species counts are unchanged during conversion. An implicit `pair_coeff * *` is acceptable only when the model and LAMMPS type orders have been verified to match. Prefer explicit element names for auditable workflows. +## LAMMPS data and box checks + +- The first line of a LAMMPS data file is a title and is skipped by `read_data`; + place the actual header counts after it. +- Put `atom_modify map yes` before `read_data` for the documented DPA4 route. +- For a restricted triclinic box, preserve the tilt mapping `xy = b_x`, + `xz = c_x`, and `yz = c_y`; never write `c_z` or `lz` into `yz`. +- After conversion, compare atom count, species counts, and box volume with the + source structure before running dynamics. + ## Pre-production validation 1. Confirm the model loads without format or backend errors. -1. Treat `.pt2` as a device/toolchain-compiled artifact. Verify that the export - and final LAMMPS runtime use compatible PyTorch, DeePMD-kit, device, and C++ - interfaces. +1. Keep DPA4 freeze, `run 0`, canary, and production on the same target physical + compute node and allocation unless exact artifact portability is proven. 1. For multi-rank execution, verify the archive's communication metadata and nested with-comm artifact before launching MPI. -1. Run `run 0` or a bounded short run before production. -1. Require finite energy, force, pressure, and temperature where applicable. -1. Check atom count, masses, element mapping, box, and units when values are - anomalous or atoms are lost. -1. Preserve the generated LAMMPS input, model path, command, log, and exit code. +1. Stage `run 0` -> short NVE when physically appropriate -> short requested + ensemble -> production; do not jump from a successful load to a long run. +1. Require finite thermodynamics, stable atom count, and no mapping, box, or + lost-atom errors. Exit code zero alone is not a passed canary. +1. Require early temperature, pressure, and controlled variables to remain + physically compatible with the initial state and requested ensemble. +1. Preserve the generated data and input files, model path and SHA256, runtime + identity, command, complete log, and true exit code. ## References diff --git a/source/tests/common/test_agent_skills.py b/source/tests/common/test_agent_skills.py index b6fe98a709..7bdc1e18ed 100644 --- a/source/tests/common/test_agent_skills.py +++ b/source/tests/common/test_agent_skills.py @@ -5,6 +5,12 @@ ROOT = Path(__file__).resolve().parents[3] LAMMPS_SKILL = ROOT / "skills" / "lammps-deepmd" / "SKILL.md" +LAMMPS_DEPLOYMENT = ( + ROOT / "skills" / "lammps-deepmd" / "references" / "model-deployment.md" +) +LAMMPS_WORKFLOW = ( + ROOT / "skills" / "lammps-deepmd" / "references" / "commands-and-workflow.md" +) DPA4_FINETUNE_SKILL = ROOT / "skills" / "deepmd-finetune-dpa4" / "SKILL.md" INFERENCE_SKILL = ROOT / "skills" / "deepmd-python-inference" / "SKILL.md" HELD_OUT_REFERENCE = ( @@ -81,3 +87,45 @@ def test_held_out_contract_matches_dp_test_source() -> None: assert 'detail_path.with_suffix(".e.out")' in energy_tester assert 'detail_path.with_suffix(".e_peratom.out")' in energy_tester assert 'detail_path.with_suffix(".f.out")' in energy_tester + + +def test_dpa4_freeze_and_lammps_stay_on_target_node() -> None: + deployment = LAMMPS_DEPLOYMENT.read_text(encoding="utf-8") + + assert "same target physical compute node" in deployment + assert "inspect the native checkpoint -> freeze `.pt2` -> `run 0`" in deployment + assert "move the archive to B" in deployment + assert "artifact portability is proven" in deployment + + +def test_lammps_mapping_data_and_dump_contract() -> None: + skill = LAMMPS_SKILL.read_text(encoding="utf-8") + deployment = LAMMPS_DEPLOYMENT.read_text(encoding="utf-8") + workflow = LAMMPS_WORKFLOW.read_text(encoding="utf-8") + example = skill.split("## Example: annotated NVT input", 1)[1].split( + "### What every command means", 1 + )[0] + + assert example.index("atom_modify map yes") < example.index( + "read_data data.system" + ) + assert "pair_coeff * * Si O" in example + assert "dump_modify 1 element Si O sort id" in example + assert "zero-based `type.raw` index" in deployment + assert "Do not sort atoms by element" in deployment + assert "`xy = b_x`" in deployment + assert "`xz = c_x`" in deployment + assert "`yz = c_y`" in deployment + assert "first line of a LAMMPS data file is a title" in deployment + assert "first line of a LAMMPS data file is a skipped title" in workflow + + +def test_lammps_canary_requires_physical_stability() -> None: + deployment = LAMMPS_DEPLOYMENT.read_text(encoding="utf-8") + workflow = LAMMPS_WORKFLOW.read_text(encoding="utf-8") + + assert "short NVE when physically appropriate" in deployment + assert "Exit code zero alone is not a passed canary" in deployment + assert "early temperature, pressure, and controlled variables" in deployment + assert "do not barostat that direction" in workflow + assert "shell environment variables and LAMMPS variables distinct" in workflow From fe8168c34fe13d3b77f2b311ce6572022f7afe1f Mon Sep 17 00:00:00 2001 From: root Date: Sun, 16 Aug 2026 10:04:53 +0800 Subject: [PATCH 21/24] docs(skills): fix DPA4 evaluation and export contracts --- skills/deepmd-finetune-dpa4/SKILL.md | 11 ++++++-- .../references/dpa4-freeze-policy.md | 24 +++++++++++++++++ .../references/held-out-evaluation.md | 26 ++++++++++++++++--- .../references/model-artifacts.md | 4 ++- skills/deepmd-train/models/dpa4.md | 13 +++++++--- skills/lammps-deepmd/assets/input.nvt.lammps | 6 +++-- .../references/model-deployment.md | 11 +++++--- 7 files changed, 79 insertions(+), 16 deletions(-) create mode 100644 skills/deepmd-python-inference/references/dpa4-freeze-policy.md diff --git a/skills/deepmd-finetune-dpa4/SKILL.md b/skills/deepmd-finetune-dpa4/SKILL.md index fa33f5ab45..3e096280d8 100644 --- a/skills/deepmd-finetune-dpa4/SKILL.md +++ b/skills/deepmd-finetune-dpa4/SKILL.md @@ -96,8 +96,11 @@ If `--model-branch` is omitted, the fitting net may be initialized from the `RANDOM` branch instead. A multi-task target uses `finetune_head` in each target branch rather than the command-line option. -If the architecture is unknown, `--use-pretrain-script` can inherit the stored -model configuration except for `type_map`: +`--use-pretrain-script` replaces only the target `model.descriptor` and +`model.fitting_net` from the checkpoint. It does not restore the complete model +configuration. Inspect and reproduce all other required model-level fields in +the target input, including `type`, spin settings, bridging method/radii, and +task-specific options: ```bash dp --pt train input.json --finetune pretrained.pt --use-pretrain-script @@ -154,6 +157,10 @@ only after the complete evaluation meets the task's declared thresholds. ## Export and test +Before export, read +`../deepmd-python-inference/references/dpa4-freeze-policy.md` and record the +selected freeze-time inference environment. + DPA4/SeZM uses the `.pt2` AOTInductor export path rather than the conventional PyTorch `.pth` freeze path: diff --git a/skills/deepmd-python-inference/references/dpa4-freeze-policy.md b/skills/deepmd-python-inference/references/dpa4-freeze-policy.md new file mode 100644 index 0000000000..370f0345a3 --- /dev/null +++ b/skills/deepmd-python-inference/references/dpa4-freeze-policy.md @@ -0,0 +1,24 @@ +# DPA4 freeze-time inference policy + +Read this reference before exporting a DPA4 `.pt2`. The generated archive may +embed inference choices, so do not inherit unknown values of +`DP_TRITON_INFER`, `DP_TF32_INFER`, or `DP_AMP_INFER` from the shell. + +Choose one explicit policy for the target node. A conservative production +baseline that avoids the slow default while retaining full-precision +accumulation is: + +```bash +export DP_TRITON_INFER=1 +export DP_TF32_INFER=0 +export DP_AMP_INFER=0 +dp --pt freeze -c model.ckpt.pt -o frozen_model +``` + +`DP_TRITON_INFER=2` autotunes for the current hardware and therefore reinforces +the requirement to freeze and run on the same physical node. Levels 1 and 2 keep +FP32 accumulation. Level 3, TF32, and AMP change the numerical policy and require +task-specific accuracy and stability validation before production. + +Record all three values, device/runtime identity, input and output hashes, freeze +command, log, and true exit code with the artifact. diff --git a/skills/deepmd-python-inference/references/held-out-evaluation.md b/skills/deepmd-python-inference/references/held-out-evaluation.md index 2405587d2f..559a1ce37b 100644 --- a/skills/deepmd-python-inference/references/held-out-evaluation.md +++ b/skills/deepmd-python-inference/references/held-out-evaluation.md @@ -12,9 +12,11 @@ held-out evaluation. - Set `natoms` to the number of whitespace-separated entries in `type.raw`. Require coordinate and force widths of `3 * natoms`, one energy row per frame, nine box values per periodic frame, and finite values for every evaluated label. -- Interpret `type.raw` as zero-based indices into that system's ordered - `type_map.raw`. Compare model and data types by element identity, not by copying - dataset indices into another type map. +- When `type_map.raw` is present, interpret `type.raw` as zero-based indices into + that ordered map and compare model and data types by element identity. When it + is absent, require provenance that the dataset indices already follow the + candidate model's ordered type map. Fail closed when neither contract is + established; never copy dataset indices into an assumed model map. ## Run every system @@ -22,13 +24,29 @@ Use the backend required by the exact candidate artifact. For a DPA4/SeZM native checkpoint, run one command per held-out system: ```bash -dp --pt test -m selected.pt -s held_out/system.000 -n 0 -d details/system.000 +detail_prefix="details/system.000" +mkdir -p "$(dirname "$detail_prefix")" +for suffix in e.out e_peratom.out f.out v.out; do + test ! -e "${detail_prefix}.${suffix}" || exit 1 +done +dp --pt test -m selected.pt -s held_out/system.000 -n 0 -d "$detail_prefix" ``` `-n 0` evaluates all frames. Require explicit `-m`, `-s`, and a unique `-d` prefix for each system. Preserve the command, log, true exit code, checkpoint SHA256, and dataset identity. Do not overwrite existing detail files silently. +For a native multi-task checkpoint, inspect its branches and pass the admitted +branch during evaluation: + +```bash +dp --pt show selected.pt model-branch +dp --pt test -m selected.pt -s held_out/system.000 -n 0 \ + -d "$detail_prefix" --head SELECTED_BRANCH +``` + +A frozen selected `.pt2` is already single-head; do not pass `--head` to it. + ## Validate detail outputs For an energy model, retain the emitted total-energy (`.e.out`), diff --git a/skills/deepmd-python-inference/references/model-artifacts.md b/skills/deepmd-python-inference/references/model-artifacts.md index 038053ea6a..a5c9608931 100644 --- a/skills/deepmd-python-inference/references/model-artifacts.md +++ b/skills/deepmd-python-inference/references/model-artifacts.md @@ -27,7 +27,9 @@ Do not add a backend flag merely from the assumed model family. ## DPA4/SeZM DPA4/SeZM supports Python evaluation from its `.pt` checkpoint, but a `.pt2` -archive is the normal frozen deployment artifact. Freeze with: +archive is the normal frozen deployment artifact. First read +`dpa4-freeze-policy.md`; choose and record the freeze-time inference environment +instead of inheriting unknown shell values. Then freeze with: ```bash dp --pt freeze -c model.ckpt.pt -o frozen_model diff --git a/skills/deepmd-train/models/dpa4.md b/skills/deepmd-train/models/dpa4.md index ce1c9197aa..2abb71993c 100644 --- a/skills/deepmd-train/models/dpa4.md +++ b/skills/deepmd-train/models/dpa4.md @@ -33,14 +33,17 @@ model section is: ], "descriptor": { "rcut": 6.0 + }, + "fitting_net": { + "type": "dpa4_ener" } } } ``` -`model.type: "dpa4"` selects the DPA4/SeZM descriptor and its default energy -fitting network. DPA4 defaults to `float32`; double precision is unnecessary and -not recommended for the normal workflow. +Both `model.descriptor` and `model.fitting_net` are required. DPA4 defaults to +`float32`; double precision is unnecessary and not recommended for the normal +workflow. ## Parameters to choose deliberately @@ -72,7 +75,9 @@ the `deepmd-finetune-dpa4` skill. ## Freeze and test -DPA4 checkpoints are `.pt`, but deployment uses an AOTInductor `.pt2` archive: +DPA4 checkpoints are `.pt`, but deployment uses an AOTInductor `.pt2` archive. +Read `../../deepmd-python-inference/references/dpa4-freeze-policy.md` and choose +the freeze-time inference policy before exporting: ```bash dp --pt freeze -c model.ckpt.pt -o frozen_model diff --git a/skills/lammps-deepmd/assets/input.nvt.lammps b/skills/lammps-deepmd/assets/input.nvt.lammps index 90317704dd..c751ee0def 100644 --- a/skills/lammps-deepmd/assets/input.nvt.lammps +++ b/skills/lammps-deepmd/assets/input.nvt.lammps @@ -7,6 +7,7 @@ variable TAU_T equal 0.1 units metal boundary p p p atom_style atomic +atom_modify map yes neighbor 1.0 bin @@ -14,11 +15,12 @@ read_data data.system mass 1 28.0855 mass 2 15.999 pair_style deepmd graph_compressed.pb -pair_coeff * * +pair_coeff * * Si O thermo_style custom step temp pe ke etotal press vol lx ly lz xy xz yz thermo ${THERMO_FREQ} -dump 1 all custom ${DUMP_FREQ} traj.lammpstrj id type x y z +dump 1 all custom ${DUMP_FREQ} traj.lammpstrj id type element x y z +dump_modify 1 element Si O sort id velocity all create ${TEMP} 743574 fix 1 all nvt temp ${TEMP} ${TEMP} ${TAU_T} diff --git a/skills/lammps-deepmd/references/model-deployment.md b/skills/lammps-deepmd/references/model-deployment.md index 4f3bb59718..70602e7cbd 100644 --- a/skills/lammps-deepmd/references/model-deployment.md +++ b/skills/lammps-deepmd/references/model-deployment.md @@ -26,7 +26,10 @@ Do not call a DPA4 `.pt2` archive a compressed model: DPA4 does not support ## DPA4/SeZM deployment -Freeze a DPA4/SeZM checkpoint with the standard PyTorch command: +Before export, read +`../../deepmd-python-inference/references/dpa4-freeze-policy.md` and explicitly +choose the freeze-time inference environment. Then freeze a DPA4/SeZM checkpoint +with the standard PyTorch command: ```bash dp --pt freeze -c model.ckpt.pt -o frozen_model @@ -78,8 +81,10 @@ that metadata or nested artifact is absent. LAMMPS atom types, dataset type indices, and model types are separate namespaces. Inspect the artifact's ordered type map, for example with `dp --pt show model.pt type-map`, and treat element identity as the bridge. -For DeePMD data, decode each zero-based `type.raw` index through that system's -ordered `type_map.raw`; do not reuse the dataset integer as a LAMMPS type. +For DeePMD data with `type_map.raw`, decode each zero-based `type.raw` index +through that ordered map. Without `type_map.raw`, require provenance that dataset +indices already follow the candidate model's ordered type map. Fail closed when +neither contract is established; do not reuse a dataset integer as a LAMMPS type. Use compact one-based LAMMPS types for the elements present in the structure and write the same element order in masses, `pair_coeff`, and dump metadata: From dd3fb6aa2c16050ab19ba93ca83d61576648ce7a Mon Sep 17 00:00:00 2001 From: root Date: Sun, 16 Aug 2026 10:08:01 +0800 Subject: [PATCH 22/24] test(skills): cover DPA4 review contracts --- source/tests/common/test_agent_skills.py | 132 ++++++++++++++++++++++- 1 file changed, 131 insertions(+), 1 deletion(-) diff --git a/source/tests/common/test_agent_skills.py b/source/tests/common/test_agent_skills.py index 7bdc1e18ed..727ddcf234 100644 --- a/source/tests/common/test_agent_skills.py +++ b/source/tests/common/test_agent_skills.py @@ -1,4 +1,12 @@ # SPDX-License-Identifier: LGPL-3.0-or-later +import json +import os +import re +import shutil +import subprocess + +import pytest + from pathlib import ( Path, ) @@ -11,6 +19,11 @@ LAMMPS_WORKFLOW = ( ROOT / "skills" / "lammps-deepmd" / "references" / "commands-and-workflow.md" ) +LAMMPS_ASSET = ROOT / "skills" / "lammps-deepmd" / "assets" / "input.nvt.lammps" +DPA4_TRAIN_REFERENCE = ROOT / "skills" / "deepmd-train" / "models" / "dpa4.md" +DPA4_FREEZE_POLICY = ( + ROOT / "skills" / "deepmd-python-inference" / "references" / "dpa4-freeze-policy.md" +) DPA4_FINETUNE_SKILL = ROOT / "skills" / "deepmd-finetune-dpa4" / "SKILL.md" INFERENCE_SKILL = ROOT / "skills" / "deepmd-python-inference" / "SKILL.md" HELD_OUT_REFERENCE = ( @@ -22,6 +35,7 @@ ) DP_TEST_ENTRYPOINT = ROOT / "deepmd" / "entrypoints" / "test.py" ENERGY_TESTER = ROOT / "deepmd" / "infer" / "model_test" / "ener.py" +FINETUNE_SOURCE = ROOT / "deepmd" / "utils" / "finetune.py" def test_lammps_skill_uses_capability_gated_runtime() -> None: @@ -62,7 +76,7 @@ def test_complete_held_out_evaluation_is_routed_and_evidence_complete() -> None: assert "references/held-out-evaluation.md" in inference assert "held-out-evaluation.md" in finetune assert "dp --pt test -m selected.pt" in held_out - assert "-n 0 -d details/system.000" in held_out + assert '-n 0 -d "$detail_prefix"' in held_out assert "one command per held-out system" in held_out assert "population standard deviation (`ddof=0`)" in held_out assert "do not average per-system RMSE values" in held_out @@ -129,3 +143,119 @@ def test_lammps_canary_requires_physical_stability() -> None: assert "early temperature, pressure, and controlled variables" in deployment assert "do not barostat that direction" in workflow assert "shell environment variables and LAMMPS variables distinct" in workflow + + +def test_held_out_command_runs_from_clean_directory_without_overwrite( + tmp_path: Path, +) -> None: + held_out = HELD_OUT_REFERENCE.read_text(encoding="utf-8") + command = held_out.split("```bash", 1)[1].split("```", 1)[0].strip() + bash = shutil.which("bash") + assert bash is not None + + fake_bin = tmp_path / "bin" + fake_bin.mkdir() + fake_dp = fake_bin / "dp" + fake_dp.write_text( + """#!/bin/sh +while [ "$#" -gt 0 ]; do + if [ "$1" = "-d" ]; then + shift + detail_prefix=$1 + fi + shift +done +: "${detail_prefix:?missing detail prefix}" +touch "${detail_prefix}.e.out" "${detail_prefix}.e_peratom.out" \ + "${detail_prefix}.f.out" +""", + encoding="utf-8", + ) + fake_dp.chmod(0o755) + env = os.environ.copy() + env["PATH"] = f"{fake_bin}{os.pathsep}{env['PATH']}" + + first = subprocess.run( + [bash, "-eu", "-c", command], cwd=tmp_path, env=env, check=False + ) + second = subprocess.run( + [bash, "-eu", "-c", command], cwd=tmp_path, env=env, check=False + ) + + assert first.returncode == 0 + assert second.returncode != 0 + assert (tmp_path / "details" / "system.000.e.out").is_file() + + +def test_dpa4_minimal_model_configuration_normalizes() -> None: + pytest.importorskip("deepmd.lib", reason="requires a built DeePMD checkout") + from deepmd.utils.argcheck import normalize + from deepmd.utils.compat import update_deepmd_input + + text = DPA4_TRAIN_REFERENCE.read_text(encoding="utf-8") + section = text.split("## Minimal model configuration", 1)[1] + fenced_json = re.search(r"```json\n(.*?)\n```", section, flags=re.DOTALL) + assert fenced_json is not None + model_fragment = json.loads(fenced_json.group(1)) + config = { + **model_fragment, + "training": { + "training_data": {"systems": ["dummy"]}, + "numb_steps": 1, + }, + "loss": {"type": "ener"}, + "learning_rate": {"type": "exp", "start_lr": 1e-3}, + } + + normalized = normalize(update_deepmd_input(config, warning=False)) + + assert normalized["model"]["fitting_net"]["type"] == "dpa4_ener" + + +def test_lammps_asset_matches_mapping_contract() -> None: + asset = LAMMPS_ASSET.read_text(encoding="utf-8") + + assert asset.index("atom_modify map yes") < asset.index( + "read_data data.system" + ) + assert "pair_coeff * * Si O" in asset + assert "id type element x y z" in asset + assert "dump_modify 1 element Si O sort id" in asset + + +def test_dpa4_freeze_policy_is_explicit_and_routed() -> None: + policy = DPA4_FREEZE_POLICY.read_text(encoding="utf-8") + train = DPA4_TRAIN_REFERENCE.read_text(encoding="utf-8") + finetune = DPA4_FINETUNE_SKILL.read_text(encoding="utf-8") + deployment = LAMMPS_DEPLOYMENT.read_text(encoding="utf-8") + + for variable in ("DP_TRITON_INFER", "DP_TF32_INFER", "DP_AMP_INFER"): + assert f"export {variable}=" in policy + assert "Levels 1 and 2 keep" in policy + assert "Level 3, TF32, and AMP" in policy + assert "dpa4-freeze-policy.md" in train + assert "dpa4-freeze-policy.md" in finetune + assert "dpa4-freeze-policy.md" in deployment + + +def test_held_out_multitask_and_optional_type_map_contracts() -> None: + held_out = HELD_OUT_REFERENCE.read_text(encoding="utf-8") + + assert "When `type_map.raw` is present" in held_out + assert "is absent, require provenance" in held_out + assert "--head SELECTED_BRANCH" in held_out + assert "already single-head; do not pass `--head`" in held_out + + +def test_use_pretrain_script_guidance_matches_source_scope() -> None: + guidance = DPA4_FINETUNE_SKILL.read_text(encoding="utf-8") + source = FINETUNE_SOURCE.read_text(encoding="utf-8") + function = source.split("def _apply_pretrained_model_params", 1)[1].split( + "\ndef ", 1 + )[0] + + assert "does not restore the complete model" in guidance + assert "model.descriptor" in guidance + assert "model.fitting_net" in guidance + assert 'pretrained_config["descriptor"]' in function + assert 'pretrained_config["fitting_net"]' in function From 3a871cd803e10c0e891cc19a8037de933c3527cf Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sun, 16 Aug 2026 02:11:23 +0000 Subject: [PATCH 23/24] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- .../references/held-out-evaluation.md | 4 ++-- source/tests/common/test_agent_skills.py | 13 ++++++++----- 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/skills/deepmd-python-inference/references/held-out-evaluation.md b/skills/deepmd-python-inference/references/held-out-evaluation.md index 559a1ce37b..d8423f37aa 100644 --- a/skills/deepmd-python-inference/references/held-out-evaluation.md +++ b/skills/deepmd-python-inference/references/held-out-evaluation.md @@ -27,7 +27,7 @@ checkpoint, run one command per held-out system: detail_prefix="details/system.000" mkdir -p "$(dirname "$detail_prefix")" for suffix in e.out e_peratom.out f.out v.out; do - test ! -e "${detail_prefix}.${suffix}" || exit 1 + test ! -e "${detail_prefix}.${suffix}" || exit 1 done dp --pt test -m selected.pt -s held_out/system.000 -n 0 -d "$detail_prefix" ``` @@ -42,7 +42,7 @@ branch during evaluation: ```bash dp --pt show selected.pt model-branch dp --pt test -m selected.pt -s held_out/system.000 -n 0 \ - -d "$detail_prefix" --head SELECTED_BRANCH + -d "$detail_prefix" --head SELECTED_BRANCH ``` A frozen selected `.pt2` is already single-head; do not pass `--head` to it. diff --git a/source/tests/common/test_agent_skills.py b/source/tests/common/test_agent_skills.py index 727ddcf234..f46bf82308 100644 --- a/source/tests/common/test_agent_skills.py +++ b/source/tests/common/test_agent_skills.py @@ -4,13 +4,12 @@ import re import shutil import subprocess - -import pytest - from pathlib import ( Path, ) +import pytest + ROOT = Path(__file__).resolve().parents[3] LAMMPS_SKILL = ROOT / "skills" / "lammps-deepmd" / "SKILL.md" LAMMPS_DEPLOYMENT = ( @@ -189,8 +188,12 @@ def test_held_out_command_runs_from_clean_directory_without_overwrite( def test_dpa4_minimal_model_configuration_normalizes() -> None: pytest.importorskip("deepmd.lib", reason="requires a built DeePMD checkout") - from deepmd.utils.argcheck import normalize - from deepmd.utils.compat import update_deepmd_input + from deepmd.utils.argcheck import ( + normalize, + ) + from deepmd.utils.compat import ( + update_deepmd_input, + ) text = DPA4_TRAIN_REFERENCE.read_text(encoding="utf-8") section = text.split("## Minimal model configuration", 1)[1] From 84923b8a17d4baa6bace51abea458ba138a1fd5b Mon Sep 17 00:00:00 2001 From: root Date: Sun, 16 Aug 2026 10:40:11 +0800 Subject: [PATCH 24/24] docs(skills): reserve held-out detail roots --- .../references/held-out-evaluation.md | 9 ++++----- source/tests/common/test_agent_skills.py | 2 +- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/skills/deepmd-python-inference/references/held-out-evaluation.md b/skills/deepmd-python-inference/references/held-out-evaluation.md index d8423f37aa..b2dd29cb95 100644 --- a/skills/deepmd-python-inference/references/held-out-evaluation.md +++ b/skills/deepmd-python-inference/references/held-out-evaluation.md @@ -24,11 +24,10 @@ Use the backend required by the exact candidate artifact. For a DPA4/SeZM native checkpoint, run one command per held-out system: ```bash -detail_prefix="details/system.000" -mkdir -p "$(dirname "$detail_prefix")" -for suffix in e.out e_peratom.out f.out v.out; do - test ! -e "${detail_prefix}.${suffix}" || exit 1 -done +detail_root="details/selected-SHA256" +test ! -e "$detail_root" || exit 1 +mkdir -p "$detail_root" +detail_prefix="$detail_root/system.000" dp --pt test -m selected.pt -s held_out/system.000 -n 0 -d "$detail_prefix" ``` diff --git a/source/tests/common/test_agent_skills.py b/source/tests/common/test_agent_skills.py index f46bf82308..d55e8de3c1 100644 --- a/source/tests/common/test_agent_skills.py +++ b/source/tests/common/test_agent_skills.py @@ -183,7 +183,7 @@ def test_held_out_command_runs_from_clean_directory_without_overwrite( assert first.returncode == 0 assert second.returncode != 0 - assert (tmp_path / "details" / "system.000.e.out").is_file() + assert (tmp_path / "details" / "selected-SHA256" / "system.000.e.out").is_file() def test_dpa4_minimal_model_configuration_normalizes() -> None: