Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .grok/skills/unsga3-oracle/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,18 @@ Checks sphere residual, niche occupancy, coverage vs true PF. Known historical r
3. If algorithm code changed: `dotnet test Unsga3.slnx -c Release` must stay green.
4. Do **not** commit `tools/oracle/out/*.csv` (gitignored).

### 4. Constrained problems and mating opt-ins

```powershell
python tools/oracle/run_new_surfaces.py --seeds 15
```

Writes `docs/NEW-SURFACES-RESULTS.md`. Does **not** write `docs/WILCOXON-RESULTS.md`. C# defaults stay `RankNicheDistance` + `IndependentWithReplacement`. C1-DTLZ1 passes pymoo `n_var=7` (k=5). Every cell is measured or `skip:`.

## Do not

- Claim bit-identical fronts (different RNG streams).
- Tighten CI IGD bars without multi-seed evidence.
- Run full n=15 when the user only asked for a smoke check.
- Invent IGD, HV, or Wilcoxon numbers. A missing front or a seed with no feasible point is `skip:`.
- Rewrite `docs/WILCOXON-RESULTS.md` from `run_new_surfaces.py` or from a mating-opt-in sweep.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/).

### Added

- Measured 15-seed tables for OSY, TNK, and C1-DTLZ1 (feasible non-dominated IGD vs pymoo `UNSGA3`) and for mating opt-ins (`TwoShuffledPasses`, `Algorithm2`, and both) in `docs/NEW-SURFACES-RESULTS.md`. Protocol is in `docs/EQUIVALENCE.md`. Package defaults are unchanged. `docs/WILCOXON-RESULTS.md` is not rewritten.
- Optional **TypeSafe / Jev System One** pass (`tools/typesafe-pareto`) for Score + Choice judgments over a small Pareto candidate sample. Additive semantic layer — does not replace NSGA-III / U-NSGA-III objectives. Live calls only when `TYPESAFE_API_KEY` is set; CI uses mocks. Metrics append to `metrics/typesafe-runs.jsonl`.

### Changed
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

> **v0.2.0** — opt-in two-shuffle mating, Algorithm 2 tournament, odd-N pairing, feasible-only niching, equal-CV Deb fix, two-layer Das–Dennis, and OSY/TNK/C1-DTLZ1. Defaults stay unconstrained `RankNicheDistance`, independent mating, and single-layer Das–Dennis. The published 15-seed table is not rewritten.
> **15-seed IGD vs pymoo `UNSGA3`:** ZDT1 **median 0.053 vs 0.070** (MWU *p*≈0.05) compares the full C# non-dominated front with pymoo `res.F`. DTLZ2 **median 0.0045 vs 0.0028** (~1.6×) compares C# n_var=12 with pymoo's default n_var=10. Neither pair is a same-set, same-problem ranking. Notes: [`docs/ORACLE-RESULTS.md`](docs/ORACLE-RESULTS.md).
> Details: [`docs/WILCOXON-RESULTS.md`](docs/WILCOXON-RESULTS.md) · single-seed notes: [`docs/ORACLE-RESULTS.md`](docs/ORACLE-RESULTS.md)
> Details: [`docs/WILCOXON-RESULTS.md`](docs/WILCOXON-RESULTS.md) · single-seed notes: [`docs/ORACLE-RESULTS.md`](docs/ORACLE-RESULTS.md) · constrained + mating opt-ins: [`docs/NEW-SURFACES-RESULTS.md`](docs/NEW-SURFACES-RESULTS.md)

```text
https://github.com/AppSprout-dev/Unsga3
Expand Down Expand Up @@ -123,6 +123,7 @@ API docs used: [HTTP](https://docs.typesafe.ai/api.md) · [Python SDK](https://d
|-----|----------|
| [docs/ORACLE-RESULTS.md](docs/ORACLE-RESULTS.md) | Single-seed C# vs pymoo |
| [docs/WILCOXON-RESULTS.md](docs/WILCOXON-RESULTS.md) | Multi-seed Mann–Whitney / Wilcoxon |
| [docs/NEW-SURFACES-RESULTS.md](docs/NEW-SURFACES-RESULTS.md) | 15-seed constrained IGD and mating opt-ins |
| [docs/EQUIVALENCE.md](docs/EQUIVALENCE.md) | Protocol & intentional deltas (ZDT2 quality A/B = gens=250, `PymooCompatible`; matches unsga3-bend) |
| [docs/RESEARCH-STANDARDS.md](docs/RESEARCH-STANDARDS.md) | Literature + indicator standards |
| [docs/NOTICE.md](docs/NOTICE.md) | Attribution (papers + validation tools) |
Expand Down
37 changes: 35 additions & 2 deletions docs/EQUIVALENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,13 @@ ZDT2 **gens=100** is an early-stress snapshot (collapse on Bend, C#, and pymoo),
| Bi | ZDT1–4, ZDT6 | 2 | yes |
| Many | DTLZ1–4, DTLZ7 | 3+ | yes |
| Hard | WFG1, WFG2, WFG9 | 3–5 | planned |
| Constrained | OSY, TNK, C1-DTLZ1 | 2–3 | yes (self-tests; no IGD table) |
| Constrained | OSY, TNK, C1-DTLZ1 | 2–3 | yes (self-tests; 15-seed IGD in [NEW-SURFACES-RESULTS.md](NEW-SURFACES-RESULTS.md)) |

## Unit checks

- Reference association & niche counts
- Non-dominated ranks / constraint domination (equal CV is mutual non-domination)
- Feasible niching, CV fill, and OSY / TNK / C1-DTLZ1 (`ConstrainedSurvivalTests`, `ConstrainedProblemTests`). No IGD table.
- Feasible niching, CV fill, and OSY / TNK / C1-DTLZ1 (`ConstrainedSurvivalTests`, `ConstrainedProblemTests`). Multi-seed IGD is in [NEW-SURFACES-RESULTS.md](NEW-SURFACES-RESULTS.md).
- **Normalization intercepts / ASF axis extremes** (`NormalizationTests`)
- Tournament pressure
- `populationSize: null` ⇒ `|refs|`
Expand Down Expand Up @@ -95,6 +95,39 @@ pymoo 0.6.2 `TournamentSelection` (`pressure=2`) **is** the two-shuffled-pass po

Odd N (Das–Dennis M=2, p=12 → 13; M=3, p=12 → 91): `CreateOffspring` pairs the leftover parent with parent 0, so `parents[N-1]` enters SBX, then the cursor restarts at 0. Even N stays consecutive pairs `(0,1), (2,3), …` and wraps to `(0,1)` with no extra random draw. Published ZDT1 (52) and DTLZ2 (92) are even, so their pair sequence is unchanged.

## New surfaces (constrained + mating opt-ins)

Measured tables live in [NEW-SURFACES-RESULTS.md](NEW-SURFACES-RESULTS.md). They are written by `tools/oracle/run_new_surfaces.py`. [WILCOXON-RESULTS.md](WILCOXON-RESULTS.md) is not a target of that script. Package defaults stay `RankNicheDistance` and `IndependentWithReplacement`.

Seeds are 1..15, the same indexes as the published ZDT1 and DTLZ2 tables. C# and pymoo do not share an RNG. Pairing is by seed index. Operators: SBX η=30, p_c=1, PM η=20, p_m=1/n, duplicate elimination on, single-layer Das–Dennis.

### Constrained

| Problem | M | n_var | Partitions | Pop | Gens | C# settings |
|---------|---|------:|------------|----:|-----:|-------------|
| OSY | 2 | 6 | 12 | 52 | 250 | constructor defaults |
| TNK | 2 | 2 | 12 | 52 | 250 | constructor defaults |
| C1-DTLZ1 | 3 | **7** (k=5) | 12 | 92 | 150 | constructor defaults |

Pop and partitions follow the ZDT (M=2, pop=52) and DTLZ2 (M=3, pop=92) oracles. OSY and TNK use the ZDT2 quality generation count (250). C1-DTLZ1 uses the DTLZ2 generation count (150). `C1Dtlz1Problem` defaults to k=5. pymoo `get_problem("c1dtlz1")` defaults to n_var=12; the harness passes **n_var=7**.

IGD is pymoo's mean nearest distance on the **feasible non-dominated subset of the final population**, both solvers, one shared reference front. OSY and TNK use pymoo's shipped front file. C1-DTLZ1 uses the DTLZ1 simplex at the same Das–Dennis partitions (the C1 inequality does not remove that front). `res.F` is a different set and is not the comparison. A seed with no feasible point is `skip:`, not a filled-in IGD.

pymoo `UNSGA3` still uses its own tournament and two-shuffle pool. This is default Unsga3 against default pymoo, on a shared front. It is not a claim that the mating operators match. A second pymoo run of the same seed is not always bit-identical with SciPy OpenBLAS. The results file keeps the first run in the table and lists the second run when it differs. The two values are not averaged.

### Mating opt-ins

Same populations and generation counts as the quality protocols above. ZDT1 gens=100, ZDT2 gens=250, DTLZ2 gens=150. IGD is the C# feasible non-dominated front against `ParetoFronts` (the published ZDT1 C# definition).

| Column | Tournament | Pool |
|--------|------------|------|
| default | `RankNicheDistance` | `IndependentWithReplacement` |
| two_shuffled | `RankNicheDistance` | `TwoShuffledPasses` |
| algorithm2 | `Algorithm2` | `IndependentWithReplacement` |
| both | `Algorithm2` | `TwoShuffledPasses` |

The ZDT2 and DTLZ2 default columns in that table are `RankNicheDistance`. The published DTLZ2 Wilcoxon column is `PymooCompatible`. ZDT2 has no published Wilcoxon column. Neither published table is rewritten.

## DTLZ2 gap history

| Stage | C# pymoo-mode IGD | vs pymoo 0.0035 |
Expand Down
Loading
Loading