feat(gooddata-sdk): [AUTO] Add AI Lake ObjectStorage listing and ColumnExpression for pipe tables

[yenkins-admin]

Summary

Added CatalogObjectStorageInfo, CatalogColumnExpression models and list_object_storages()/create_pipe_table() methods to CatalogAILakeService, exposing the new listAiLakeObjectStorages endpoint and the new columnExpressions field on CreatePipeTableRequest.

Impact: new_feature | Services: gooddata-afm-client, gooddata-metadata-client

Files changed

• packages/gooddata-sdk/src/gooddata_sdk/catalog/ai_lake/entity_model/__init__.py
• packages/gooddata-sdk/src/gooddata_sdk/catalog/ai_lake/entity_model/object_storage.py
• packages/gooddata-sdk/src/gooddata_sdk/catalog/ai_lake/entity_model/column_expression.py
• packages/gooddata-sdk/src/gooddata_sdk/catalog/ai_lake/service.py
• packages/gooddata-sdk/src/gooddata_sdk/__init__.py
• packages/gooddata-sdk/tests/catalog/test_catalog_ai_lake.py
• packages/gooddata-sdk/tests/catalog/unit_tests/test_ai_lake_service.py

Agent decisions

Decisions (5)

entity_model directory placement — Created catalog/ai_lake/entity_model/ subdirectory with separate files per model

• Alternatives: Inline new classes in service.py alongside CatalogAILakeOperation
• Why: Keeps service.py focused on behavior; entity models are independently testable and may grow independently. Follows the pattern used in other domains like catalog/organization/entity_model/.

ColumnExpression API model import — Import ColumnExpression from gooddata_api_client at module top-level

• Alternatives: Import inside as_api_model() body (lazy import)
• Why: gooddata_api_client is a hard dependency, not an optional heavy lib, so a top-level import is correct. ruff PLC0415 would flag a function-body import here.

create_pipe_table() return type — Return None (void)

• Alternatives: Return CatalogPipeTable wrapper for the response
• Why: The PipeTable response model is complex and not yet wrapped. The cluster diff only adds columnExpressions as an input; returning None is consistent with fire-and-monitor patterns like analyze_statistics.

CatalogObjectStorageInfo.from_dict() key format — Accept snake_case keys (e.g. storage_id, storage_type)

• Alternatives: Accept camelCase keys (storageId, storageType)
• Why: The API client's to_dict(camel_case=False) default returns snake_case Python attribute names. Confirmed by tracing model_to_dict(serialize=False) in model_utils.py.

list_object_storages() empty dict fallback — data.get('storages', []) returning empty list on missing key

• Alternatives: Raise error if 'storages' key is missing
• Why: The API contract guarantees the storages field is always present, but defensively returning [] matches the pattern used in get_operation() for optional result/error fields.

Assumptions to verify (4)

• ObjectStorageInfo.to_dict() returns snake_case keys (storage_id, storage_type, storage_config) when called with default camel_case=False; verified by tracing model_to_dict(serialize=False) in model_utils.py.
• The listAiLakeObjectStorages endpoint always returns the storages array (never null); if it can be null the from_dict loop would need a None guard.
• CatalogColumnExpression.function field accepts only the four enum values defined in ColumnExpression.allowed_values; the _check_type=False bypass in as_api_model() means invalid values would be sent to the server without client-side validation.
• The create_pipe_table() method may need to be updated if the API starts returning a meaningful PipeTable response that callers need to inspect.

Risks (2)

• test_list_ai_lake_object_storages will fail if the staging instance has no registered ObjectStorages — the test would pass with an empty list, but the assertions (storage.name, storage.storage_id non-empty) only run if the list is non-empty.
• If the server returns storageConfig as null rather than an empty dict, from_dict() handles it via 'or {}' but older cassettes or unexpected API changes could cause failures.

Layers touched (3)

• entity_model — New entity model classes for ObjectStorageInfo and ColumnExpression
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/ai_lake/entity_model/__init__.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/ai_lake/entity_model/object_storage.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/ai_lake/entity_model/column_expression.py
• public_api — Added list_object_storages() and create_pipe_table() to CatalogAILakeService; exported new classes from package init.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/ai_lake/service.py
  • packages/gooddata-sdk/src/gooddata_sdk/__init__.py
• tests — Integration test for list_object_storages + VCR cassette; unit tests for all new methods and models
  • packages/gooddata-sdk/tests/catalog/test_catalog_ai_lake.py
  • packages/gooddata-sdk/tests/catalog/unit_tests/test_ai_lake_service.py

Source commits (gdc-nas)

• 313a512 Merge pull request #22519 from gooddata/jacek/meta-openapi

OpenAPI diff

@@ gooddata-afm-client.json @@
+      "ColumnExpression": {
+        "description": "Single column projection override: applies `function(column)` to a source column.",
+        "properties": {
+          "column": { "type": "string" },
+          "function": { "enum": ["HLL_HASH","BITMAP_HASH","BITMAP_HASH64","TO_BITMAP"] }
+        }
+      }
+          "columnExpressions": { "additionalProperties": { "$ref": "ColumnExpression" }, "description": "Per-target-column projection overrides..." }
+      "ListObjectStoragesResponse": { "description": "Response for listing ObjectStorages registered for the organization.", ... }
+      "ObjectStorageInfo": { "description": "Descriptor of a registered ObjectStorage. Provider credentials are stripped.", ... }
+    "/api/v1/ailake/object-storages": { "get": { "operationId": "listAiLakeObjectStorages", "summary": "(BETA) List registered AI Lake ObjectStorages" } }

Workflow run

---

Generated by SDK OpenAPI Sync workflow