Skip to content

V1 testing#14

Merged
btmonier merged 18 commits into
mainfrom
v1-testing
May 25, 2026
Merged

V1 testing#14
btmonier merged 18 commits into
mainfrom
v1-testing

Conversation

@btmonier
Copy link
Copy Markdown
Member

Summary

This PR re-platforms the assembly alignment steps onto PHGv2's align-assemblies command, adds orchestrator support for the remaining PS4G pipeline steps (ropebwt-mem, build-spline-knots, convert-ropebwt2ps4g), and provides a full "end-to-end" (E2E) suite of integration tests.

Features

  • Add commands/align/ shared module (PhgAlignParams, PhgAlignRunner, PhgAlignSharedOptions) and rewrite align-assemblies and align-mutated-assemblies as thin wrappers around PHGv2's align-assemblies (drives AnchorWave + minimap2 internally) while preserving the existing --ref-gff / --ref-fasta / --query-fasta / maf_file_paths.txt contract.
  • Add new PHGv2 alignment knobs to both align steps and the YAML config: --in-parallel, --ref-max-align-cov, --query-max-align-cov, --conda-env-prefix, --just-ref-prep.
  • Add ropebwt_mem, build_spline_knots, and convert_ropebwt2ps4g sections to orchestrate with output auto-chaining and config dataclasses.
  • Add mutated-maf-to-gvcf as a documented orchestrate-only step 11.
  • Add a separate Gradle e2eTest task and applyHeavyTestConfig() helper that forks one JVM per heavy test class and restricts class-file scanning so a single fork failure doesn't blow up the whole task.
  • Add various Docker improvements for better integration with pixi environments and larger local test memory usage.
  • Add IntegrationGuard test helper and ~17 new unit test files covering every pipeline command (AlignAssemblies, AlignMutatedAssemblies, ConvertCoordinates, ConvertRopebwt2Ps4g, ConvertToFasta, CreateChainFiles, DownsampleGvcf, FormatRecombinedFastas, GenerateRecombinedSequences, MafToGvcf, PickCrossovers, RopeBwtChrIndex, RopeBwtMem, PhgAlignRunner) plus a substantially expanded OrchestrateE2ETest.
  • Add PHGv2 binary existence check to orchestrate's environment validation so missing PHG triggers an automatic setup-environment retry.

Bug Fixes

  • Fix setup-environment failing to find the extracted MLImpute directory after the upstream repo was renamed to grits (archive extracts as grits-main/).
  • Fix ropebwt-mem auto-calculating the wrong -l value: the keyfile written by rope-bwt-chr-index/orchestrate has no header row, so the previous lines.size - 1 count was off by one.
  • Fix pixi and PHGv2 install permission failures with Docker.

Breaking Changes

  • Minor renumbering of pipeline steps (added to docs).
  • align-* steps now use PHGv2 instead of AnchorWave directly.

Checklist

  • I have updated the version in build.gradle.kts (currently bumped to 0.3.0 in the working tree; commit before opening the PR)
  • I have tested these changes locally
  • I have added/updated tests for new functionality
  • I have updated documentation (if applicable)
  • Breaking changes are clearly documented above

@btmonier btmonier requested review from aberthel, smm477 and zrm22 May 22, 2026 13:43
@btmonier btmonier self-assigned this May 22, 2026
@btmonier btmonier added the enhancement New feature or request label May 22, 2026

// Parse align_assemblies
@Suppress("UNCHECKED_CAST")
val alignAssembliesMap = configMap["align_assemblies"] as? Map<String, Any>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Future idea. Maybe we should change the type to be a data class instead of a Map<String,Any>. It would help not needing to do type casting.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally valid refactoring idea - will consider

@btmonier btmonier merged commit 7a8a5cb into main May 25, 2026
3 checks passed
@btmonier btmonier deleted the v1-testing branch May 25, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants