Skip to content

fix(ingestion): make the role grants runnable and drop the CREATE ROLE - #844

Merged
jirhiker merged 2 commits into
stagingfrom
fix/ingestion-role-sql-runnable
Aug 19, 2026
Merged

fix(ingestion): make the role grants runnable and drop the CREATE ROLE#844
jirhiker merged 2 commits into
stagingfrom
fix/ingestion-role-sql-runnable

Conversation

@jirhiker

Copy link
Copy Markdown
Member

ingestion_role.sql could not be run as written, and it instructed the operator
to do something that would have failed. Found while trying to execute it against
dataservices.

The CREATE ROLE was wrong

The script told you to uncomment a CREATE ROLE for the IAM principal. That
role already exists — registering a service account as a Cloud SQL IAM user
creates the Postgres role automatically, which Terraform does via
google_sql_user.ingestion:

$ gcloud sql users list --instance=dataservices
ocotillo-ingestion@waterdatainitiative-271000.iam   CLOUD_IAM_SERVICE_ACCOUNT

So the script now only grants, and says so. Reaching for a CREATE ROLE is a
signal the Terraform half has not been applied, not something to work around.

It could not run

role_name was hardcoded with \set while db_name was referenced and never
set, so every GRANT would have failed on an undefined variable.

Both are now required parameters, with a readable abort if either is missing:

psql "host=... dbname=ocotillo user=postgres" \
  -v db_name=ocotillo \
  -v role_name=ocotillo-ingestion@waterdatainitiative-271000.iam \
  -f automated_ingestion/sql/ingestion_role.sql

Requiring them is the point, not a formality. The instance hosts both
ocotillo and ocotillo-staging. Granting against the wrong one succeeds
silently and leaves you debugging the database you did not touch.

The role name contains @ and dots, so every reference uses :"role_name",
which quotes it as an identifier — unquoted is a syntax error.

Also

Stops tracking automated_ingestion/iac/.terraform.tfstate.lock.info, which was
committed with the us-west4 change. It is written while a run holds the state
lock and left behind when one is interrupted. A stale copy in a fresh checkout
is worse than useless: Terraform reports it as another user holding the lock.

Not verified against a live database

I could not run this — no psql here, the superuser is a built-in account whose
password I will not handle, and the account I am authenticated as has no
database login on the instance. The grants themselves are unchanged from the
reviewed version; what changed is that the script now runs and targets what you
tell it to.

🤖 Generated with Claude Code

jirhiker and others added 2 commits August 18, 2026 20:57
Two things were wrong with this script.

It told the operator to create the role. Registering the service account as a
Cloud SQL IAM user already creates the Postgres role, which Terraform does via
google_sql_user -- `gcloud sql users list` shows
ocotillo-ingestion@waterdatainitiative-271000.iam as CLOUD_IAM_SERVICE_ACCOUNT
on the instance. A CREATE ROLE would fail, and wanting one is a sign the
Terraform half has not been applied.

It also hardcoded the role name via \set while referencing an unset db_name, so
it could not run as written. Both are now required parameters and the script
stops with a readable message if either is missing. That matters more than
convenience here: the instance hosts both `ocotillo` and `ocotillo-staging`,
and granting against the wrong one would succeed silently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
.terraform.tfstate.lock.info was committed with the us-west4 change. It is
written while a plan or apply holds the lock and left behind when a run is
interrupted -- which is how it got picked up. It is machine-specific, and a
stale one in a fresh checkout is actively misleading, since Terraform reports it
as another user holding the lock.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Coverage

79.27% total — gate is 75%.

@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Your pull request is automatically being deployed to Dagster Cloud.

Location Status Link Updated
ocotillo-automated-ingestion View in Cloud Aug 19, 2026 at 04:15 AM (UTC)

@jirhiker
jirhiker merged commit 0bd2213 into staging Aug 19, 2026
10 checks passed
@jirhiker
jirhiker deleted the fix/ingestion-role-sql-runnable branch August 19, 2026 04:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant