feat(metadata): sidecar CSVs for nested data + Firestore-only metadata transaction

[htsukamoto5]

⚠️ Stacked on #151

This branch includes the three commits of #151 (fix/metadata-version-drift) — review only the last two commits here (refactor(metadata) + feat(metadata)), or view the diff against the PR A branch. Base is test so CI runs; merge #151 first, then this.

Part 1: Firestore-only metadata transaction (refactor)

blockMetadata's entire body — including every OSF network call — ran inside db.runTransaction. Firestore retries the transaction callback on contention (e.g. two participants submitting concurrently), which would re-run OSF uploads: a latent duplicate-write risk. The transaction is now strictly read → merge → write on the Firestore metadata doc; OSF I/O happens before it (existence check) and after it (mirror upload). When the merge needs the OSF copy as its base (Firestore empty, OSF populated — first submission after a wipe), the transaction aborts via a sentinel, the copy is downloaded outside it, and the transaction re-runs.

Fixes three pre-existing bugs found in the process:

• The Firestore-only branch checked putFileOSF's result with errorCode !== 210, which throws even on success (success returns errorCode: null), so that branch could never complete.
• The OSF-only branch uploaded the unmerged incoming metadata to OSF while Firestore got the merged version; both now receive the merged one.
• The create branch ignored putFileOSF's result entirely (silent metadata loss); it is now checked like the other branches.

blockMetadata also now receives the token api-data already resolved via resolveToken, instead of re-deriving it with ~25 lines of duplicated logic that could trigger a second token refresh per submission.

Part 2: sidecar CSVs for nested array/object columns

The re-vendored library (from #151) expands nested trial fields — survey response objects, mouse_tracking_data arrays — into dotted sub-variables (response.Q0, mouse_tracking_data.x) and exposes the per-row data behind them. Previously the metadata described these variables while the data itself remained locked inside JSON blobs.

DataPipe now writes what the standalone metadata CLI writes per data file: one sidecar CSV per extracted array column (rows keyed by join keys + element_index) and per plain-object column (one row per trial). Since the CLI's sidecars are per source file, per-submission sidecars are the exact incremental equivalent — no cross-session accumulation or dedup state is needed.

• Naming reuses the library's own Psych-DS helpers (deriveFallbackBase + deriveArrayFilename + disambiguateArrayFilename), so DataPipe's sidecar names match the CLI's for the same data (e.g. abc123.json with mouse tracking → subject-abc123_measure-mouseTrackingData_data.csv), placed in the same one-level subfolder as the data file. Zero custom naming code to maintain across re-vendors.
• Ordering: sidecars upload only after the participant's data file lands in OSF — a 409 on the main file can't leave orphan sidecars.
• Best-effort: the data file is already safe and sidecars are derivable from it, so a sidecar failure is queued in the existing uploadQueue (whose retry worker already handles arbitrary filenames and treats 409 as done) and logged — it never fails the submission. If the main file gets queued because OSF is down, the sidecars are queued alongside it.
• Flat data (or CSV submissions) produce no sidecars; nothing changes for those experiments.

Testing

• New metadata-sidecars.test.js unit suite (7 tests: naming convention, subfolder placement, CSV shape for arrays/objects, disambiguation, no-extension filenames, empty case).
• metadata-production.test.js extended for the new return shape + extraction contents (dotted per-row data verified against the real library).
• All 4 metadata unit suites pass locally (23 tests); tsc build clean. Emulator suites (which pin each metadata-state branch's response message — preserved by the refactor) run in this PR's CI.

🤖 Generated with Claude Code