[firestore-bigquery-export] v0.2.6 YAML parsing error with multi-wildcard collection paths

[Suyyala]

Bug Report: firestore-bigquery-export v0.2.6 - Wildcard Paths Cause YAML Parsing Error

Bug Information

Extension: firebase/firestore-bigquery-export
Broken Version: v0.2.6
Working Version: v0.1.56
Severity: Critical - Breaks multi-tenant architectures using wildcard paths
Status: Regression introduced in v2 function migration

---

Summary

The firestore-bigquery-export extension v0.2.6 fails to deploy when COLLECTION_PATH contains multiple wildcards (e.g., {org_id}/{client_id}/data/call_records/data). This is a regression from v0.1.56 which handled wildcard paths correctly.

The error occurs during Google Cloud Deployment Manager YAML parsing, caused by unquoted curly braces in the v2 function eventFilters configuration.

---

Environment

• Firebase CLI Version: 13.x
• Project Type: Multi-region (nam5)
• Extension Version Tested: v0.2.6
• Extension Version Working: v0.1.56
• GCP Project: Multiple projects tested (dev, demo, prod)

---

Reproduction Steps

1. Configure Extension with Multi-Wildcard Path

Create an extension configuration with a collection path containing multiple wildcards:

firebase.json:

{
  "extensions": {
    "firestore-bigquery-export-callinsights": "firebase/firestore-bigquery-export@0.2.6"
  }
}

extensions/firestore-bigquery-export-callinsights.env:

BIGQUERY_PROJECT_ID=${param:PROJECT_ID}
DATABASE_REGION=nam5
COLLECTION_PATH={org_id}/{client_id}/data/call_records/data
DATASET_ID=my_dataset
DATASET_LOCATION=us
TABLE_ID=call_insights
WILDCARD_IDS=true

2. Deploy Extension

firebase deploy --only extensions --force

3. Observe Error

Extensions deploy had errors:

- update firestore-bigquery-export-callinsights
failed to startDeploymentManagerWork for instance id: firestore-bigquery-export-callinsights
error updating deployment: DM HTTP 412 error

Error parsing configuration: while parsing a block mapping
in 'string', line 71, column 19:
    - attribute: document
      ^
expected <block end>, but found '<scalar>'
in 'string', line 73, column 34:
    value: {org_id}/{client_id}/data/call_records/data/{documentId}
           ^

---

Expected Behavior

The extension should deploy successfully with multi-wildcard collection paths, as it did in v0.1.56.

Working in v0.1.56:

• ✅ Single wildcard paths: posts/{postId}/comments
• ✅ Multi-wildcard paths: {org_id}/{client_id}/data/call_records/data
• ✅ All 16 instances deployed successfully

---

Actual Behavior

Broken in v0.2.6:

• ✅ Single wildcard paths: posts/{postId}/comments (works)
• ❌ Multi-wildcard paths: {org_id}/{client_id}/data/call_records/data (YAML parsing error)
• ❌ 9 out of 16 instances failed to deploy in our test
• ❌ Extensions left in ERRORED state

---

Root Cause Analysis

v0.1.56 Implementation (Working)

resources:
  - name: fsexportbigquery
    type: firebaseextensions.v1beta.function
    properties:
      eventTrigger:
        eventType: providers/cloud.firestore/eventTypes/document.write
        resource: projects/${param:PROJECT_ID}/databases/(default)/documents/${param:COLLECTION_PATH}/{documentId}

• Uses v1 Cloud Functions
• COLLECTION_PATH is substituted into a simple resource string
• No YAML parsing of the path value
• Works with any number of curly braces

v0.2.6 Implementation (Broken)

resources:
  - name: fsexportbigquery
    type: firebaseextensions.v1beta.v2function
    properties:
      eventTrigger:
        eventFilters:
          - attribute: document
            value: ${COLLECTION_PATH}/{documentId}  # ← UNQUOTED!
            operator: match-path-pattern

• Uses v2 Cloud Functions
• value field is not quoted
• After substitution: value: {org_id}/{client_id}/data/call_records/data/{documentId}
• YAML parser interprets {org_id} as start of a flow mapping (object)
• Fails parsing when it encounters / after the opening brace

YAML Parsing Issue

When the Deployment Manager tries to parse the generated YAML:

eventFilters:
  - attribute: document
    value: {org_id}/{client_id}/data/call_records/data/{documentId}

YAML parser sees:

• {org_id} - Start of a mapping object
• / - Invalid inside a mapping
• Error: Expected closing } or key-value pair, found /

---

Proposed Fix

Option 1: Quote the Value (Recommended)

eventFilters:
  - attribute: document
    value: "${COLLECTION_PATH}/{documentId}"  # ← Add quotes
    operator: match-path-pattern

Option 2: Use YAML Literal String

eventFilters:
  - attribute: document
    value: |-
      ${COLLECTION_PATH}/{documentId}
    operator: match-path-pattern

Option 3: Escape in Template Generation

If the template is generated programmatically, escape the curly braces or quote the entire value before writing to YAML.

---

Impact

Who Is Affected

1. Multi-tenant architectures using org/client ID wildcards
2. Anyone with multiple wildcards in collection paths
3. Existing v0.1.56 users attempting to upgrade

Scope

• Our testing: 9 out of 16 extensions failed (56% failure rate)
• Pattern: All failures had paths like {x}/{y}/...
• Pattern: All successes had simple paths or single wildcards

Business Impact

• ❌ Cannot upgrade to v0.2.6
• ❌ Missing out on v2 functions benefits (Node.js 20, better performance)
• ❌ Stuck on v0.1.56 indefinitely
• ❌ Multi-tenant SaaS platforms blocked from upgrading

---

Workarounds

Current Workaround: Stay on v0.1.56

{
  "extensions": {
    "firestore-bigquery-export-callinsights": "firebase/firestore-bigquery-export@0.1.56"
  }
}

Pros:

• ✅ Stable and production-proven
• ✅ Full wildcard support

Cons:

• ❌ Uses deprecated v1 Cloud Functions
• ❌ Node.js 18 (older runtime)
• ❌ Missing new features from v0.2.x

No Alternative Workaround

• Cannot modify extension source (it's published)
• Cannot manually deploy v2 functions (managed by extension)
• Must wait for Firebase to fix and publish v0.2.7

---

Test Cases

Should Pass (Regression Tests)

✅ COLLECTION_PATH=posts
✅ COLLECTION_PATH=users/{userId}/posts
✅ COLLECTION_PATH=orgs/{orgId}/data
❌ COLLECTION_PATH={orgId}/{clientId}/data (FAILS in v0.2.6)
❌ COLLECTION_PATH={a}/{b}/c/{d}/data (FAILS in v0.2.6)
❌ COLLECTION_PATH={org_id}/{client_id}/voice_assistant_app/data/chats (FAILS in v0.2.6)

Edge Cases to Test

COLLECTION_PATH={a}/{b}/{c}/data
COLLECTION_PATH={tenant}/{workspace}/{project}/collections/{collectionId}/documents
COLLECTION_PATH=customers/{customerId}/sites/{siteId}/devices/{deviceId}/readings

---

Additional Context

Related Issues

• Wildcard support is documented: https://firebase.google.com/docs/extensions/official/firestore-bigquery-export
• Validation regex still allows wildcards: ^[^/]+(/[^/]+/[^/]+)*$
• No deprecation notice in CHANGELOG.md for v0.2.0
• Issue Firestore BigQuery Export: The backfill script doesn't support wildcards #484: "Firestore BigQuery Export: The backfill script doesn't support wildcards" (different but related)

Changelog Entry (v0.2.0)

## Version 0.2.0
- feat - migrate to v2 functions and support non-default firestore instances
- fix - fix enqueue logic and types

No mention of breaking changes to wildcard support.

Version Comparison

Feature	v0.1.56	v0.2.6
Wildcard paths	✅ Works	❌ Broken
Multi-wildcard paths	✅ Works	❌ Broken
v2 Functions	❌ No	✅ Yes
Node.js 20	❌ No	✅ Yes
DATABASE_REGION param	❌ No	✅ Yes (undocumented)

---

Reproduction Project

We can provide:

• Full extension configuration files
• Deployment logs
• Multiple test cases
• Before/after comparisons

Available for testing/debugging upon request.

---

Suggested Verification

After fix is implemented, please verify:

1. ✅ Single wildcard paths still work
2. ✅ Multi-wildcard paths work
3. ✅ Paths with 3+ wildcards work
4. ✅ Special characters in wildcard names (underscore, dash)
5. ✅ Migration path from v0.1.56 → v0.2.x works smoothly
6. ✅ Existing deployed extensions can be updated in-place

---

Timeline

• v0.1.56 Released: January 2025 (stable)
• v0.2.0 Released: Unknown (migration to v2 functions)
• v0.2.6 Current: Latest version (broken)
• Bug Discovered: October 2, 2025
• Environments Affected: Dev (recovered), Demo (stable on v0.1.56), Prod (stable on v0.1.56)

---

Request

Please:

1. ✅ Confirm this is a bug (not intentional removal of feature)
2. ✅ Add regression tests for multi-wildcard paths
3. ✅ Fix by quoting the value field in eventFilters
4. ✅ Document DATABASE_REGION parameter (currently undocumented)
5. ✅ Publish v0.2.7 with fix
6. ✅ Update documentation if any limitations exist

---

Contact

Available for:

• Further testing
• Providing detailed logs
• Testing pre-release versions
• Verification of fix

---

Thank you for maintaining this excellent extension! We rely on it heavily for our multi-tenant SaaS platform and would love to upgrade to v2 functions once this is resolved.

[HassanBahati]

Hello @Suyyala, thank you for opening this issue. It has been received and we shall provide updates when available. Cheers!

[HassanBahati]

Hi @Suyyala, we were able to reproduce this issue via this minimum reproducible example: MRE. We will share updates on any developments as soon as they are available. Cheers!

[IzaakGough]

Hi, I’ve done some further investigation and this issue appears to have been introduced by the v2 trigger migration in this repo, but the actual failure seems to occur later in the Extensions deployment/rendering path.

Repro

• {org_id}/{client_id}/data/call_records/data -> fails with a Deployment Manager YAML parse error
• root/{org_id}/{client_id}/data/call_records -> deploys successfully

I also tried quoting the trigger filter in extension.yaml:

value: "${COLLECTION_PATH}/{documentId}"

I then installed from local source but the same error still occurs when the path starts with a wildcard.

Takeaway
This does not appear to be an issue with multiple wildcards in general. It appears to fail specifically when COLLECTION_PATH starts with {.

It also appears that quoting in extension.yaml is not preserved through the Extensions -> Deployment Manager rendering path.

So while this seems to have been introduced by the v2 trigger migration in this repo, the actual fix may need to happen in the Firebase Extensions platform/backend layer that renders eventFilters, rather than in firebase/extensions alone.