Skip to content

feat(python): add ontology models to the shared SDK - #651

Open
AndrewNepogoda wants to merge 5 commits into
apache:mainfrom
AndrewNepogoda:python-ontology-sdk
Open

AndrewNepogoda wants to merge 5 commits into
apache:mainfrom
AndrewNepogoda:python-ontology-sdk

Conversation

@AndrewNepogoda

@AndrewNepogoda AndrewNepogoda commented Oct 27, 2026 •

Copy link
Copy Markdown

Summary

The ontology converter in apache/ossie-converters defines its own Pydantic models for ontology documents, in ontology/src/ossie_ontology/spec.py (imported as ossie_ontology.spec). The other converters already use the shared ossie.models package from this repo for semantic models.

This PR adds equivalent ontology models to ossie.models, so the ontology converter can switch to the shared package instead of keeping its own copy. It also gives both document types one base class, and OssieDocument becomes that base (see below).

Proposed document hierarchy

OssieDocument            version, to_ossie_yaml / to_ossie_json, from_ossie_yaml / from_ossie_json
├── OssieSemanticModel   a semantic model file (core-spec/ossie-schema.json)
└── OssieOntologyModel   an ontology file (ontology/ontology.json)

Before this PR, the same classes looked like this:

OssieSemanticModel       datasets, relationships, metrics (no version)
└── OssieDocument        + version and the file I/O: the semantic model file

Why the new hierarchy is clearer:

  • One class per kind of file, named after the spec. The core spec calls its document a semantic model and ontology.json calls its document an ontology, so the classes are OssieSemanticModel and OssieOntologyModel. Before, the general name OssieDocument was used for one specific kind of file. That no longer works now that ontology files need a class too, and later mapping files will as well.
  • No near-duplicate pair. OssieSemanticModel and OssieDocument described the same thing, with and without version. Since 0.2.0.dev0 an embedded semantic_model also needs its own version, so a whole semantic model always has one. A standalone file and its embedded copy are now the same class.
  • File handling is written once. version, writing version first, and the strict loaders (from_ossie_yaml / from_ossie_json) live in OssieDocument. The loaders reject duplicate keys, nulls, non-finite numbers, cyclic aliases and a missing version. Ontology files get the same checks without copying the code.
  • Room for the mapping file. When converters support standalone mapping documents (ontology/mapping.json), they become a third OssieDocument subclass, and nothing else changes.
  • One type for "any Ossie file". Code that loads or writes files without caring about the kind can be typed OssieDocument.

name, description and ai_context stay on each subclass rather than the base, because the schemas differ: the core schema requires a non-empty name, while ontology.json doesn't.

The cost is a breaking change for converters, described below.

What's added

New models that mirror ontology/ontology.json node for node:

  • Ontology: OssieOntologyModel, OssieConceptComponent (schema OntologyComponent), OssieOntologyRelationship, OssieRole, OssieConceptType, OssieMultiplicity
  • Mappings: OssieConceptMapping, OssieObjectMapping, OssieReferentMapping, OssieLinkMapping, OssieOntologyMapping (the deprecated embedded form)
  • OssieDocument as the shared base of OssieSemanticModel and OssieOntologyModel (see above)

OssieOntologyModel gets the same I/O methods and document checks (duplicate keys, nulls, missing version) that semantic model files already had. A new test checks that each model has the same properties, required set and additionalProperties: false as its schema node.

The mapping classes are included because #458 has already merged, so they won't need to move a second time. The standalone mapping document (ontology/mapping.json) is left out, because no converter reads it yet.

Breaking change: OssieDocument is now the base class

OssieSemanticModel takes over the role OssieDocument used to have. It carries version (defaulting to 0.2.0.dev0) and the I/O methods.

Old code still imports, but fails at runtime: OssieDocument(name=..., datasets=[...]) and OssieDocument.from_ossie_yaml(...) on a semantic model file raise ValidationError, because name and datasets are not fields of the base.

Before After
OssieDocument(name=..., datasets=[...]) OssieSemanticModel(name=..., datasets=[...])
OssieDocument.from_ossie_yaml(text) / from_ossie_json(text) OssieSemanticModel.from_ossie_yaml(text) / from_ossie_json(text)
OssieSemanticModel without version OssieSemanticModel with version, which is serialized first
OssieOntologyMapping.semantic_model: OssieDocument OssieOntologyMapping.semantic_model: OssieSemanticModel

Converters that use OssieDocument need the rename when they upgrade apache-ossie: dbt, hex, sigma, wisdom, and the ontology converter's ossie_to_spec / spec_to_ossie. Code that builds an OssieSemanticModel and serializes it now also gets a version key in the output.

The ontology converter doesn't depend on the new ontology models yet and still uses its own ossie_ontology.spec. The name OssieOntologyModel doesn't clash with the converter's runtime class OssieOntology (ossie_ontology/model.py), so both can be imported side by side when it migrates.

Behaviour change when a converter moves from ossie_ontology.spec to ossie.models

The new models follow the existing ossie.models approach, not spec.py's. These are the deliberate differences:

ossie_ontology.spec ossie.models
Mutability mutable frozen; assignment raises ValidationError
Unknown keys extra="forbid" extra="forbid" (unchanged; only ai_context allows extras, per the schema)
Absent lists [] None; iterate with or []
Required fields loose per schema: concept type, relationship verbalizes, OntologyMap.concept_mappings. OntologyMap.name becomes optional
Either-or fields not checked per schema anyOf: object and referent mappings need expression or referent_mappings; concept mappings need object_mappings or link_mappings
Enums str literals OssieConceptType, OssieMultiplicity, OssieDataType enums
ai_context object dict OssieAIContextObject
Expression dialects only sql and/or dialects; {} is rejected
Join path JoinPath.from_ OssieRelationship.from_dataset (alias from)
File I/O OssieSpec.load_yaml / dump_yaml / dump_dict OssieOntologyModel.from_ossie_yaml / to_ossie_yaml / model_dump(mode="json", by_alias=True, exclude_none=True). Loading is stricter: duplicate keys, nulls and a missing version are rejected

19 of the 31 ontology fixtures in ossie-converters/ontology/tests fail to validate under the new models. All 19 omit verbalizes, and 4 also have expression: {}. These fixtures already fail ossie validate, so the fix belongs in the fixtures, not the models.

Two converter checks to keep in mind during the migration:

  • The converter's runtime model (ossie_ontology/model.py) requires exactly one of expression / referent_mappings. The schema's anyOf allows both, so ossie.models does too, and the stricter check stays in the converter.
  • The converter's runtime OssieOntology.add_ontology_mapping (in ossie_ontology/model.py) indexes mappings by name. Now that name is optional, spec_to_ossie should either reject a missing name with a clear error or default it (for example to semantic_model.name).

Transition plan for ossie_ontology.spec

When ossie_ontology moves to ossie.models, ossie_ontology/spec.py stays as a thin re-export under the old names. That keeps open work (for example the LinkML converter, formerly #389) importing during the transition:

ossie_ontology.spec ossie.models
OssieSpec OssieOntologyModel (load_yaml → from_ossie_yaml, dump_yaml → to_ossie_yaml)
ConceptComponent OssieConceptComponent
Relationship OssieOntologyRelationship
Role OssieRole
OntologyMapping / ConceptMapping / ObjectMapping / ReferentMapping / LinkMapping OssieOntologyMapping / OssieConceptMapping / OssieObjectMapping / OssieReferentMapping / OssieLinkMapping
SemanticModel OssieSemanticModel
Dataset / DatasetField / Metric / Dimension OssieDataset / OssieField / OssieMetric / OssieDimension
JoinPath OssieRelationship
Expression / DialectExpression OssieExpression / OssieDialectExpression
CustomExtension OssieCustomExtension
AiContext / DataType / CoreVersion OssieAIContext / OssieDataType / OssieVersion

The re-export keeps the names working; the behaviour still changes as described in the table above. The re-export is removed once nothing in progress imports the old path.

Related Issues

No linked issue. The converter-side migration will be tracked in apache/ossie-converters.

Checklist

Specification

  • Spec changes are included in core-spec/ and follow the existing structure (N/A: no spec change)
  • Spec changes have been discussed on the mailing list or in a linked issue (N/A)
  • Breaking changes to the spec are clearly called out in the summary (N/A)

Ontology

  • Ontology changes in ontology/ are consistent with spec changes (N/A: ontology/ is unchanged; the SDK now mirrors it)
  • New or modified terms are defined and documented (N/A)

Converters

  • Spec or ontology changes that affect converters have a matching issue or PR in apache/ossie-converters (no spec change; switching OssieDocument to OssieSemanticModel needs a converters PR before they upgrade apache-ossie)

Validation

  • Validation rules in validation/ are updated if the spec changed (N/A)
  • New validation cases are covered by tests (N/A)

Documentation

  • docs/ is updated to reflect any user-facing changes (N/A: python/README.md documents OssieOntologyModel)
  • New features or behaviors are documented with examples where appropriate
  • CONTRIBUTING.md is updated if the contribution process changed (N/A)

Examples

  • examples/ are added or updated for any new spec constructs (N/A: the tests validate examples/flights.ontology.yaml)

Tests

  • All existing tests pass (pytest / CI green): uv run pytest in python/ gives 288 passed
  • New functionality is covered by tests (python/tests/test_ontology_models.py)

Compliance

  • ASF license headers are present on all new source files (dev/check-license passes)
  • No third-party dependencies are added without PMC/IPMC approval
  • If third-party source code is vendored/copied (not just declared as a dependency), NOTICE and/or LICENSE have been updated per ASF policy (N/A: nothing vendored)

Add Pydantic models mirroring ontology/ontology.json so converters can
share them instead of keeping their own DTOs (ossie_ontology.spec in
apache/ossie-converters):

- OssieOntologyDocument, OssieConcept, OssieOntologyRelationship,
  OssieRole, OssieConceptType, OssieMultiplicity
- OssieConceptMapping, OssieObjectMapping, OssieReferentMapping,
  OssieLinkMapping and the deprecated embedded OssieOntologyMapping
- OssieVersion, shared with OssieDocument

The models are frozen and forbid unknown keys like the existing ones,
and follow the schema's required fields and anyOf rules. OssieDocument
is unchanged apart from typing its version as OssieVersion. Tests check
each model against its schema node.
Copilot AI balanced review requested due to automatic review settings October 27, 2026 12:41
@github-actions

Copy link
Copy Markdown

Thank you for your first contribution to Apache Ossie (incubating)!

A committer will review your PR. In the meantime, please ensure:

  • All CI checks pass
  • ASF license headers are present on new source files
  • Tests cover new functionality

No ICLA is required to contribute. If you are elected as a committer in the future, you will need an Individual Contributor License Agreement (ICLA) on file with the ASF.

Questions? Ask in the PR comments or join us on Slack.

We appreciate your contribution!

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Static review identified no blocking issues in the additive models, exports, documentation, or tests; tests were not executed during this review.

No open issues 🎉

What changed in this PR

Adds shared ontology data models to the Python SDK for future converter migration, without changing current converter dependencies or file I/O behavior.

Changes:

  • Adds ontology and mapping models, including the deprecated embedded mapping form.
  • Exports the new models and shares OssieVersion between semantic-model and ontology documents.
  • Adds schema-parity, validation, nesting, and round-trip tests, plus a brief README description.
File Description
python/​tests/​test_ontology_models.py Tests schema alignment, validation constraints, nesting, and round trips.
python/​src/​ossie/​models.py Defines ontology and mapping models and the shared version type.
python/​src/​ossie/​__init__.py Exports the new SDK types.
python/​README.md Documents ontology model coverage and distinguishes relationship types.

Review effort: Balanced

@AndrewNepogoda

Copy link
Copy Markdown
Author

@jbonofre, @kayemkim would you mind taking a look?

…ontologies

Make OssieDocument the base of every Ossie file root. It holds the
version, version-first serialization and the to_ossie_* / from_ossie_*
methods with their document checks (duplicate keys, nulls, non-finite
numbers, cyclic aliases, missing version).

- OssieSemanticModel extends OssieDocument and takes over the role of
  the old OssieDocument: it now carries version and the I/O methods.
- OssieOntologyDocument is renamed OssieOntology and extends
  OssieDocument, so ontology files load with the same checks.
- OssieOntologyMapping.semantic_model is typed OssieSemanticModel.

This is a breaking change for code that builds or loads OssieDocument
as a semantic model; it switches to OssieSemanticModel.
Pair the ontology file class with OssieSemanticModel, and avoid a clash
with the ontology converter's runtime OssieOntology class in
ossie_ontology/model.py, so both can be imported without an alias.
Match the ConceptComponent name the ontology converter already uses in
ossie_ontology.spec, so the migration keeps the familiar name.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants