diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..444a458 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,33 @@ +--- +name: Bug report +about: Report a reproducible Hyperion defect +title: "fix: " +labels: bug +assignees: "" +--- + +## Problem + +Describe the incorrect behavior and the expected behavior. + +## Reproduction + +1. +2. +3. + +## Affected layer + +- [ ] DSL +- [ ] Trace +- [ ] IR +- [ ] Backend +- [ ] Inference +- [ ] Diagnostics +- [ ] API/experiments + +## Acceptance criteria + +- [ ] A failing test or minimal reproduction is included. +- [ ] The fix preserves existing public API behavior unless documented. +- [ ] Relevant local checks pass. diff --git a/.github/ISSUE_TEMPLATE/engineering_task.md b/.github/ISSUE_TEMPLATE/engineering_task.md new file mode 100644 index 0000000..6963d10 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/engineering_task.md @@ -0,0 +1,25 @@ +--- +name: Engineering task +about: Track a scoped implementation, refactor, test, docs, or CI task +title: "" +labels: enhancement +assignees: "" +--- + +## Problem + +What needs to improve? + +## Proposed solution + +What will change? + +## Acceptance criteria + +- [ ] +- [ ] +- [ ] + +## Notes + +Relevant files, commands, or design constraints. diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..6d330a8 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,22 @@ +## Summary + +- +- + +## Changes + +- +- +- + +## Validation + +- [ ] `cd python && pip install -e ".[dev]"` +- [ ] `cd python && ruff check .` +- [ ] `cd python && mypy hyperion_dsl hyperion_trace hyperion_ir hyperion_backends hyperion_inference hyperion_diagnostics --ignore-missing-imports` +- [ ] `cd python && python -m pytest tests/ -v --tb=short` +- [ ] Demo or targeted smoke test completed when relevant + +## Related issue + +Closes # diff --git a/README.md b/README.md index f04cbba..0c259ac 100644 --- a/README.md +++ b/README.md @@ -381,3 +381,11 @@ python -m pytest tests/test_integration/test_mcmc_multichain.py -v ## Лицензия MIT + +## Developer docs + +- `docs/development/architecture_overview.md` — module boundaries and ownership. +- `docs/development/data_flow.md` — model-to-diagnostics execution path. +- `docs/development/debugging.md` — focused debugging paths by layer. +- `docs/development/release_checklist.md` — release validation checklist. +- `docs/development/contribution_workflow.md` — branch, commit, and PR workflow. diff --git a/docs/development/architecture_overview.md b/docs/development/architecture_overview.md new file mode 100644 index 0000000..3fee70c --- /dev/null +++ b/docs/development/architecture_overview.md @@ -0,0 +1,18 @@ +# Architecture overview + +Hyperion is organized as a compiler-backed probabilistic programming stack. Each +layer owns one contract and should stay independently testable. + +| Layer | Packages | Responsibility | +| --- | --- | --- | +| DSL | `hyperion_dsl` | User-facing `@model`, `sample`, `plate`, distributions, constraints, and transforms. | +| Trace | `hyperion_trace` | Effect handlers, substitution, replay, blocking, and trace collection. | +| IR | `hyperion_ir` | Model graph representation, compilation, and graph optimization. | +| Backend | `hyperion_backends` | JAX potential function, flatten/unflatten, gradients, and prior sampling. | +| Inference | `hyperion_inference` | HMC, NUTS, SMC, VI, flows, Laplace, and high-level MCMC orchestration. | +| Diagnostics | `hyperion_diagnostics` | ESS, R-hat, BFMI, posterior predictive checks, and report generation. | +| API/experiments | `hyperion_api`, `hyperion_exp` | Service boundary, experiment runner, and serialization. | + +Keep new features close to the lowest layer that can own the behavior. For +example, a transform validation belongs in DSL/bijectors, while a warning about +divergences belongs in diagnostics. diff --git a/docs/development/contribution_workflow.md b/docs/development/contribution_workflow.md new file mode 100644 index 0000000..d624634 --- /dev/null +++ b/docs/development/contribution_workflow.md @@ -0,0 +1,29 @@ +# Contribution workflow + +## Branching + +Create focused branches from `main`: + +- `docs/` for documentation; +- `test/` for coverage; +- `fix/` for behavior fixes; +- `refactor/` for internal structure; +- `ci/` for automation changes. + +## Commit messages + +Use Conventional Commits: + +- `test: cover diagnostics report warnings` +- `refactor: extract trace replay helper` +- `fix: handle scalar observed values` +- `docs: add backend debugging guide` + +## Pull request notes + +Every pull request should explain: + +- what changed and why; +- which module boundary it touches; +- which tests/checks were run; +- whether examples, docs, or benchmarks need updates. diff --git a/docs/development/data_flow.md b/docs/development/data_flow.md new file mode 100644 index 0000000..68b335e --- /dev/null +++ b/docs/development/data_flow.md @@ -0,0 +1,21 @@ +# Model data flow + +The core execution path is: + +1. A user defines a Python function with `@model`. +2. DSL primitives call the active trace handler. +3. `hyperion_trace` records sample, param, factor, and deterministic sites. +4. `hyperion_ir` compiles the trace into an `IRGraph`. +5. `hyperion_backends.JAXBackend` builds a differentiable potential function. +6. Inference engines consume the backend contract and return `InferenceResult`. +7. Diagnostics summarize samples, convergence, and report warnings. + +## Boundary rules + +- DSL code should not know about HMC/NUTS internals. +- Backends should not format user-facing diagnostic reports. +- Inference engines should emit structured diagnostics rather than printing. +- Diagnostics should accept plain arrays and `InferenceResult` objects. + +These boundaries keep simple models usable in notebooks while preserving a path +to service and experiment execution. diff --git a/docs/development/debugging.md b/docs/development/debugging.md new file mode 100644 index 0000000..653e69e --- /dev/null +++ b/docs/development/debugging.md @@ -0,0 +1,39 @@ +# Debugging guide + +## Trace and DSL issues + +Start with a minimal model and inspect the trace: + +```bash +cd python +python -m pytest tests/test_trace -v --tb=short +``` + +If a site is missing, check handler stack ordering and whether the model function +is wrapped by `@model`. + +## Backend issues + +Reduce the model to one latent variable, then verify: + +- latent names in the compiled graph; +- transform forward/inverse round trips; +- potential function values are finite; +- gradients are finite for a small unconstrained vector. + +## Inference issues + +Run the smallest engine-specific test first: + +```bash +cd python +python -m pytest tests/test_inference/test_components.py -v --tb=short +``` + +For HMC/NUTS, inspect acceptance rate, divergences, tree depth, and BFMI before +tuning sample counts. More samples do not fix a broken geometry. + +## Diagnostics issues + +Diagnostics should be deterministic. Prefer small NumPy arrays in tests and +assert concrete warning messages, table keys, and summary fields. diff --git a/docs/development/release_checklist.md b/docs/development/release_checklist.md new file mode 100644 index 0000000..48e6603 --- /dev/null +++ b/docs/development/release_checklist.md @@ -0,0 +1,25 @@ +# Release checklist + +Use this checklist before tagging or presenting Hyperion as a stable research +artifact. + +## Local validation + +- [ ] `cd python && pip install -e ".[dev]"` +- [ ] `cd python && ruff check .` +- [ ] `cd python && mypy hyperion_dsl hyperion_trace hyperion_ir hyperion_backends hyperion_inference hyperion_diagnostics --ignore-missing-imports` +- [ ] `cd python && python -m pytest tests/ -v --tb=short` +- [ ] `python scripts/run_demo.py` + +## Evidence + +- [ ] README examples still match the public API. +- [ ] Diagnostics reports include warnings for unsafe sampler behavior. +- [ ] Benchmarks were re-run when inference hot paths changed. +- [ ] Java/protobuf integration notes were updated when proto contracts changed. + +## Packaging + +- [ ] Version was bumped intentionally. +- [ ] Dependency ranges still match supported Python versions. +- [ ] Generated or local cache artifacts are not included in the diff.