Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ matches what you want to try first.
| Run tensor operations | [Quickstart](#quickstart) (below) | 2–5 min |
| Build and train a neural net | [Hello Neural Net](#hello-neural-net) (below) | 5 min |
| Run a local GGUF model | [SKaiNET Transformers starter](https://github.com/SKaiNET-developers/SKaiNET-transformers#start-in-5-minutes) | 5 min after model setup |
| Export a secure MCU bundle | [Minerva getting started](docs/modules/ROOT/pages/tutorials/minerva-getting-started.adoc) | 10 min without firmware flashing |

Working in Java? SKaiNET ships first-class Java support — see the
[Java getting-started guide](docs/modules/ROOT/pages/tutorials/java-getting-started.adoc).
Expand Down Expand Up @@ -153,6 +154,26 @@ deployment, the StableHLO path for native and edge targets.

---

## Important Addition: Minerva Secure MCU Export

SKaiNET now includes a Minerva export backend for secure MCU deployment. It is a sibling to StableHLO and Arduino/C99 export: it starts from a supported `ComputeGraph`, lowers static MLPs to a Minerva compiler input, invokes libminerva when configured, and packages generated weights, host fixtures, firmware skeletons, and a fingerprinted `manifest.json`.

Start here:

- [Minerva getting started](docs/modules/ROOT/pages/tutorials/minerva-getting-started.adoc) — run the maintained tiny MLP dry sample, then the real libminerva runtime profile.
- [Minerva export how-to](docs/modules/ROOT/pages/how-to/minerva-export.adoc) — configure compiler paths, keys, calibration, CMake/CTest host verification, and troubleshooting.
- [How Minerva secure MCU export fits](docs/modules/ROOT/pages/explanation/minerva-secure-mcu-export.adoc) — understand why Minerva is not an Arduino replacement and when to choose StableHLO instead.

Runnable examples:

```bash
./gradlew :skainet-compile:skainet-compile-minerva:runMinervaSecureMcuExamples
./gradlew :skainet-compile:skainet-compile-minerva:runMinervaSecureMcuExamples \
-Pminerva.example=sensor-classifier
```

---

## Features

### Kotlin Multiplatform
Expand Down Expand Up @@ -190,7 +211,7 @@ deployment, the StableHLO path for native and edge targets.

- Export supported static MLP graphs to Minerva project bundles for secure MCU inference
- Emits compiler NPZ input, libminerva weights, a fingerprinted manifest, host harness, firmware example, and host verification results
- Start with the [Minerva export guide](docs/modules/ROOT/pages/how-to/minerva-export.adoc)
- Start with the [Minerva getting started guide](docs/modules/ROOT/pages/tutorials/minerva-getting-started.adoc)

### Compiler: MLIR / StableHLO

Expand Down
49 changes: 34 additions & 15 deletions docs/export/minerva.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
# Minerva Secure MCU Export

Minerva export packages a supported SKaiNET compute graph for secure MCU inference through libminerva. The maintained docs-site version is [`docs/modules/ROOT/pages/how-to/minerva-export.adoc`](../modules/ROOT/pages/how-to/minerva-export.adoc); this Markdown entrypoint keeps the repository path requested by the planning issue and is friendly to GitHub browsing.
Minerva export packages a supported SKaiNET compute graph for secure MCU inference through libminerva. The maintained docs-site pages are:

- [Minerva getting started](../modules/ROOT/pages/tutorials/minerva-getting-started.adoc)
- [Minerva export how-to](../modules/ROOT/pages/how-to/minerva-export.adoc)
- [How Minerva secure MCU export fits](../modules/ROOT/pages/explanation/minerva-secure-mcu-export.adoc)

This Markdown entrypoint keeps the repository path requested by the planning issue and is friendly to GitHub browsing.

## Setup

Expand All @@ -9,19 +15,18 @@ Inside this repository, use `project(":skainet-compile:skainet-compile-minerva")
Configure libminerva through `MinervaExportOptions` or the JVM sample environment:

```bash
export MINERVA_COMPILER_SCRIPT=/opt/libminerva/tools/compile_model.py
export MINERVA_COMPILER_SCRIPT=/opt/libminerva/compiler/minerva_compile.py
export MINERVA_RUNTIME_ROOT=/opt/libminerva
export MINERVA_CALIBRATION_NPZ=/secure/project/calibration.npz
export MINERVA_KEY_FILE=/secure/project/device.key
export MINERVA_RUN_CMAKE=true
export MINERVA_RUN_CTEST=true
export MINERVA_HOST_TOLERANCE=1.0
export MINERVA_HOST_ADAPTER_SOURCE=/secure/project/minerva_runtime_adapter.c
export MINERVA_HOST_INCLUDE_DIRS=/opt/libminerva/include
export MINERVA_HOST_LIBRARY_DIRS=/opt/libminerva/lib
export MINERVA_HOST_LIBRARIES=minerva
export MINERVA_HOST_INCLUDE_DIRS=/secure/project/minerva-host-secrets
```

Do not commit real device keys. `include/secrets.example.h` contains placeholders only.
Do not commit real device keys. `include/secrets.example.h` contains placeholders only. For host verification against libminerva, `MINERVA_HOST_INCLUDE_DIRS` can point at an untracked directory that contains a host-only `secrets.h` defining `MNV_DEVICE_KEY`.

## Compatibility

Expand All @@ -36,13 +41,15 @@ Do not commit real device keys. `include/secrets.example.h` contains placeholder
| Activations | `Relu`, `Sigmoid`, `Tanh` after a dense layer |
| Out of scope | CNNs, attention, recurrent models, dynamic shapes, branching graphs, transformers, and arbitrary imported operator sets |

`model.npz` stores activation names as scalar NumPy Unicode arrays (`relu`, `sigmoid`, `tanh`, or `linear`) because the current libminerva compiler reads `layer_i_act` as a string.

## Export API

```kotlin
val options = MinervaExportOptions(
outputDir = "build/minerva",
projectName = "TinySecureMlp",
compilerScript = "/opt/libminerva/tools/compile_model.py",
compilerScript = "/opt/libminerva/compiler/minerva_compile.py",
runtimeRoot = "/opt/libminerva",
calibrationNpz = "/secure/project/calibration.npz",
keyFile = "/secure/project/device.key"
Expand Down Expand Up @@ -93,7 +100,7 @@ The generated host harness has a stable adapter ABI:
int minerva_run_inference(const float *input, int input_count, float *output, int output_count);
```

`host/runtime_adapter.example.c` implements that ABI against the libminerva public C API (`mnv_init`, `mnv_seed_prng`, `mnv_run`, and `mnv_verify_output`). The adapter converts SKaiNET's normalized float fixtures to libminerva Q8 activation buffers and converts Q8 outputs back to floats for parity comparison.
`host/runtime_adapter.example.c` implements that ABI against the current libminerva runtime symbols (`mnv_init`, `mnv_seed_prng`, `mnv_run_with_model`, and `mnv_verify_output_with_key`). The adapter keeps compile-time switches for runtimes that expose the older `mnv_run` / `mnv_verify_output` names. It converts SKaiNET's normalized float fixtures to libminerva Q8 activation buffers and converts Q8 outputs back to floats for parity comparison.

Copy the adapter outside the generated bundle when product-specific scaling or entropy seeding needs local edits, then point CMake at the copied source. This keeps the generated host harness stable while leaving runtime policy in one reviewable adapter file.

Expand All @@ -105,14 +112,12 @@ metadata = mapOf(
MinervaHostVerificationMetadata.RUN_CTEST to "true",
MinervaHostVerificationMetadata.HOST_OUTPUT_PATH to "host-output.txt",
MinervaHostVerificationMetadata.HOST_ADAPTER_SOURCE to "/secure/project/minerva_runtime_adapter.c",
MinervaHostVerificationMetadata.HOST_INCLUDE_DIRS to "/opt/libminerva/include",
MinervaHostVerificationMetadata.HOST_LIBRARY_DIRS to "/opt/libminerva/lib",
MinervaHostVerificationMetadata.HOST_LIBRARIES to "minerva"
MinervaHostVerificationMetadata.HOST_INCLUDE_DIRS to "/secure/project/minerva-host-secrets"
)
```

`HOST_OUTPUT_PATH` is optional when the host run writes `host/observed-output.txt`.
The include, library directory, and library values are passed to CMake as semicolon-separated lists, matching CMake list syntax.
The generated CMake can build a checkout-style libminerva runtime from `runtimeRoot` when that directory contains `CMakeLists.txt`. Include, library directory, and library values are passed to CMake as semicolon-separated lists, matching CMake list syntax; use `HOST_LIBRARY_DIRS` and `HOST_LIBRARIES` only when linking an already-built runtime.

CI recipe:

Expand All @@ -124,13 +129,19 @@ CI recipe:
-Pminerva.compilerScript="$MINERVA_COMPILER_SCRIPT" \
-Pminerva.calibrationNpz="$MINERVA_CALIBRATION_NPZ" \
-Pminerva.keyFile="$MINERVA_KEY_FILE" \
-Pminerva.hostVerification.tolerance="${MINERVA_HOST_TOLERANCE:-1.0}" \
-Pminerva.hostVerification.hostAdapterSource="$MINERVA_HOST_ADAPTER_SOURCE" \
-Pminerva.hostVerification.hostIncludeDirs="$MINERVA_HOST_INCLUDE_DIRS" \
-Pminerva.hostVerification.hostLibraryDirs="$MINERVA_HOST_LIBRARY_DIRS" \
-Pminerva.hostVerification.hostLibraries="$MINERVA_HOST_LIBRARIES"
-Pminerva.hostVerification.hostIncludeDirs="$MINERVA_HOST_INCLUDE_DIRS"
```

`minervaHostVerification` is skipped by default. When enabled, it runs `jvmTest` and `runMinervaTinyMlpSample` with CMake and CTest host verification enabled unless `-Pminerva.hostVerification.runCmakeBuild=false` or `-Pminerva.hostVerification.runCTest=false` is set.
The default parity tolerance remains `1e-3`; the real checkout profile sets `MINERVA_HOST_TOLERANCE=1.0` by default because current libminerva Q8 host outputs are useful as a runtime smoke proof but are not yet numerically close to the SKaiNET float reference.

For a local checkout proof, the helper below creates an untracked key, calibration archive, host-only `secrets.h`, and host-only AVR `pgmspace.h` compatibility shim under `build/minerva-real-runtime-profile`, then runs the gated verification task with CMake and CTest enabled:

```bash
MINERVA_RUNTIME_ROOT=/opt/libminerva ./scripts/run-minerva-real-runtime-profile.sh
```

## Model Sources

Expand Down Expand Up @@ -161,6 +172,14 @@ The generated firmware example intentionally contains integration placeholders.
./gradlew :skainet-compile:skainet-compile-minerva:runMinervaTinyMlpSample
```

Additional secure MCU examples live in `MinervaSecureMcuExportSamples`:

```bash
./gradlew :skainet-compile:skainet-compile-minerva:runMinervaSecureMcuExamples
./gradlew :skainet-compile:skainet-compile-minerva:runMinervaSecureMcuExamples \
-Pminerva.example=sensor-classifier
```

Without `MINERVA_COMPILER_SCRIPT`, the task runs a dry validation through compatibility, lowering, and in-memory NPZ generation. Add `-Pminerva.compilerScript`, `-Pminerva.runtimeRoot`, `-Pminerva.calibrationNpz`, and `-Pminerva.keyFile` to run the real compiler path. `MinervaTinyMlpExportSampleTest` validates the sample graph and NPZ generation without real device keys.

## Export Path Choice
Expand Down
2 changes: 2 additions & 0 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
* Tutorials
** xref:tutorials/java-getting-started.adoc[Java getting started]
** xref:tutorials/hlo-getting-started.adoc[StableHLO getting started]
** xref:tutorials/minerva-getting-started.adoc[Minerva getting started]
** xref:tutorials/graph-dsl.adoc[Graph DSL]
** xref:tutorials/turboquant-getting-started.adoc[TurboQuant: KV-cache compression]
* How-to guides
Expand All @@ -22,6 +23,7 @@
** xref:reference/api.adoc[API reference (Dokka)]
* Explanation
** xref:explanation/skainet-for-ai.adoc[SKaiNET for AI/ML]
** xref:explanation/minerva-secure-mcu-export.adoc[How Minerva secure MCU export fits]
** xref:explanation/operator-design.adoc[Operator documentation system]
** xref:explanation/theory/index.adoc[Mathematical theory]
*** xref:explanation/theory/matmul.adoc[Matrix multiplication]
Expand Down
90 changes: 90 additions & 0 deletions docs/modules/ROOT/pages/explanation/minerva-secure-mcu-export.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
= How Minerva Secure MCU Export Fits
:description: Conceptual overview of SKaiNET's Minerva export backend and how it relates to StableHLO and Arduino/C99 export.

Minerva export is SKaiNET's secure MCU packaging backend. It takes a supported `ComputeGraph`, lowers it to a compact Minerva layer model, writes a NumPy `.npz` compiler input for libminerva, invokes the Minerva compiler when configured, and packages the generated C artifacts into a reviewable project bundle.

The important distinction is scope. StableHLO is a portable compiler IR path for broad native and accelerator deployment. Arduino/C99 export is standalone generated C for small devices without an external secure runtime. Minerva export is for the cases where the deployment artifact must be a libminerva project with secure runtime integration, generated weights, host verification, and manifest provenance.

== Same Export Level, Different Target

StableHLO and Minerva sit at the same architectural level in SKaiNET: both are graph export backends. They share the source boundary, diagnostics model, and export result style, but they produce different artifacts.

[cols="1,2,2",options="header"]
|===
| Backend | Output | Primary target

| StableHLO
| MLIR module text in the StableHLO dialect.
| Portable compiler flows such as IREE, native code, accelerators, and ecosystem tooling.

| Arduino / C99
| Standalone C99 and Arduino library structure.
| Simple MCU deployment with static memory allocation and no secure runtime dependency.

| Minerva
| Minerva compiler input, generated libminerva weights, host and firmware harnesses, manifest, and verification evidence.
| Secure MCU deployment through libminerva.
|===

Minerva is therefore not a mode of the Arduino exporter. It is a smaller, stricter sibling backend with a security-oriented package shape.

== Source Models

Minerva export starts from a SKaiNET `ComputeGraph`. That graph can be produced in several ways:

* a model written in the Kotlin neural-network DSL,
* a traced representative forward pass,
* a hand-built graph,
* a model-import workflow that validates and converts into the supported graph shape.

ONNX is not special in the Minerva pipeline. It can be useful as an input format for inspection or conversion, but the first Minerva implementation does not include a general ONNX-to-Minerva importer. The contract is the SKaiNET graph handed to `MinervaExportFacade`.

== Example Patterns

The maintained examples intentionally stay close to libminerva's own MCU story:

* `sensor-classifier` uses an 8->16->8->4 static MLP. Four slots can be direct ADC inputs, and the remaining slots can hold rolling features or zero padding. Firmware can map the four outputs to class LEDs or action states.
* `safety-guard` uses a 6->10->4->3 static MLP for protect / warn / allow decisions. It demonstrates the same Minerva export path with a different input contract and class vocabulary.

Both examples are source-side SKaiNET graphs. libminerva still owns the encrypted weight packaging, runtime initialization, integrity checks, and secure MCU execution.

== Why the Scope Is Narrow

The phase-one backend intentionally supports static sequential MLPs only:

* known rank-2 tensor shapes,
* dense `MatMul` layers with optional bias `Add`,
* `Relu`, `Sigmoid`, or `Tanh` activations,
* Q8 quantization,
* ATmega328P target metadata.

This keeps failures early and understandable. If a graph contains attention, convolution, branching, dynamic shapes, or an unsupported imported operator, Minerva rejects it before invoking libminerva. For general compiler coverage, use StableHLO.

== What the Bundle Proves

The generated bundle is meant to be auditable:

* `generated/model.npz` is the compiler input SKaiNET produced.
* `generated/weights.c` and `include/weights.h` are generated compiler artifacts.
* `host/` contains reference fixtures, a host harness, and CMake integration.
* `firmware/` contains a starting point for target integration.
* `manifest.json` fingerprints generated files and records export metadata.

Host verification checks the generated structure, validates NPZ contents, guards against accidental secret leakage, and can build and run the generated C harness when libminerva and CMake are available.

== Secret Handling

The repository-generated files must stay free of real device secrets. `secrets.example.h` is a placeholder. Real keys belong in private provisioning systems or local, untracked include directories used only for host verification and firmware integration.

The compiler command summary recorded in `manifest.json` redacts key-file arguments. The manifest should prove which artifacts were generated together, but it should not contain material that can provision or emulate a real device.

== When to Choose Minerva

Choose Minerva when all of these are true:

* the graph fits the supported static MLP contract,
* the target is a secure MCU flow managed by libminerva,
* release evidence needs a manifest and generated-file fingerprints,
* host verification matters before firmware handoff.

Choose StableHLO when you need broad operator coverage or a general compiler ecosystem. Choose Arduino/C99 when the deployment should be plain generated C without libminerva.
Loading
Loading