Skip to content

ATLAS-5397:Add 'location' attribute to trino_schema type definition - #746

Open
bhor-sanket wants to merge 1 commit into
apache:masterfrom
bhor-sanket:ATLAS-5397
Open

ATLAS-5397:Add 'location' attribute to trino_schema type definition#746
bhor-sanket wants to merge 1 commit into
apache:masterfrom
bhor-sanket:ATLAS-5397

Conversation

@bhor-sanket

@bhor-sanket bhor-sanket commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Background

Trino sends ENTITY_CREATE_V2 notification events to Atlas when a CREATE SCHEMA statement is executed with an explicit HDFS location. The event payload includes a trino_schema entity with a location attribute containing the HDFS path.
A parallel event from the Hive Metastore bridge creates a hive_db entity with the same location value, which is persisted successfully.

Problem Statement

  • Existing behavior: The location attribute sent in the trino_schema entity payload is silently dropped by Atlas. The HDFS location is only visible on the associated hive_db entity created by the Hive
    Metastore bridge — not on the trino_schema entity itself.
  • Expected behavior: The location attribute should be persisted on the trino_schema entity and visible in Atlas UI / REST API.
  • Root cause: The trino_schema type definition in addons/models/6000-Trino/6010-trino-model.json does not include location in its attributeDefs. During ENTITY_CREATE_V2 processing,
    EntityGraphMapper.mapAttributes() iterates only over type-defined attributes (structType.getAllAttributes()), so any attribute present in the incoming entity but absent from the type definition is silently
    discarded with no error or warning logged.
  • Impact: Users relying on trino_schema entities in Atlas cannot see the schema's HDFS location. They must navigate to the linked hive_db entity to find it, which breaks the expected metadata completeness
    for Trino-originated schemas.

How the patch resolves it

  • Added location (type: string, optional, single cardinality) to the trino_schema attributeDefs in addons/models/6000-Trino/6010-trino-model.json — ensures fresh installs include the attribute at type
    creation time.
  • Added a new patch file addons/models/6000-Trino/patches/002-trino_schema_add_location.json with ADD_ATTRIBUTE action (applyToVersion: "1.0" → updateToVersion: "1.1") — ensures existing deployments pick up
    the attribute on Atlas restart without requiring a fresh bootstrap.

Files changed:

  • addons/models/6000-Trino/6010-trino-model.json — added location attribute to trino_schema entity definition
  • addons/models/6000-Trino/patches/003-trino_schema_add_location.json — new patch file for existing deployments

Design decisions:

  • The base model typeVersion is kept at 1.0 (consistent with the existing convention where trino_table and trino_column also remain at 1.0 in the base model despite patch 001 updating them). The patch
    handles version bumping for existing deployments.
  • Patch ordering is guaranteed by Atlas's filename-based alphabetical sort (001-* before 002-*). Since no prior patch touches trino_schema, the applyToVersion is 1.0.

How was this patch tested?

Use-cases validation

  • CREATE SCHEMA with explicit HDFS location
    • Operation: Execute CREATE SCHEMA hive_catalog.trino_schema_hdfs_test4 WITH (location = 'hdfs://...')
    • Expected: trino_schema entity (hive_catalog.trino_schema_hdfs_test4@cm) stores the location attribute; hive_db entity also retains its location via the Hive Metastore bridge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant