Skip to content

feat(oracle): persist oracle comparisons and expose history and windo… - #549

Merged
Miracle656 merged 1 commit into
Divinity-Labs:mainfrom
Ingenious-02:feat/issue-304-oracle-comparison-history
Oct 26, 2026
Merged

Miracle656 merged 1 commit into
Divinity-Labs:mainfrom
Ingenious-02:feat/issue-304-oracle-comparison-history

Conversation

@Ingenious-02

@Ingenious-02 Ingenious-02 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Persist each comparison and expose historical series and window summary statistics:

  • Add OracleComparison Prisma model mapped to oracle_comparisons with network-leading composite indexes.
  • Persist comparison observations in compareOracles carrying Lens provenance (venues, sampleCount, freshness timestamp, age).
  • Add queryOracleComparisonHistory and computeOracleSummaryStatistics to calculate mean deviation, worst deviation, time spent outside a band (seconds and percentage), and status transitions over a window.
  • Add GET /compare/:asset/history and GET /compare/:asset/summary routes with ?network= filtering/echoing, window/date ranges, band threshold, and honest empty/gap responses (nulls and insufficient_data).
  • Cache policy set to public, max-age=60 with Vary: x-network.
  • Add oracle retention job pruneOldOracleComparisons with bounded batching (SKIP LOCKED).
  • Document endpoints and schemas in openapi.yaml and regenerate openapi.json.
  • Add comprehensive vitest suite covering oracle drift and return, retention, network handling, provenance, and honest answers.

Related issue

Fixes #304

Type of change

  • New feature
  • Tests
  • Docs

Checklist

  • I have read CONTRIBUTING.md
  • npx tsc --noEmit passes
  • npm run build passes
  • I added / updated tests where relevant
  • I updated docs where relevant

…w summary statistics (Divinity-Labs#304)

- Add OracleComparison model to Prisma schema with network-leading composite indexes
- Persist comparison observations in compareOracles with Lens provenance (venues, sampleCount, freshness)
- Add queryOracleComparisonHistory and computeOracleSummaryStatistics in oracleComparison service
- Support window, date ranges (from/to), band threshold, and oracle filtering
- Add GET /compare/:asset/history and GET /compare/:asset/summary routes
- Set cache policy to public, max-age=60 with Vary: x-network
- Add oracle retention job pruneOldOracleComparisons in bounded batches with SKIP LOCKED
- Document new endpoints and schemas in openapi.yaml and regenerate openapi.json
- Add comprehensive vitest suite for oracle drift and return, retention, network handling, provenance, and honest answers
@Miracle656

Copy link
Copy Markdown
Collaborator

Correction to my earlier review.

I described this branch as hundreds of commits behind with tens of thousands of lines of deletions. The deletion figure was wrong — it came from git diff origin/main <branch>, which compares the two tips, so everything main gained since you branched appeared as a deletion on your side. A merge works from the merge base and keeps those files.

Measured properly against current main:

conflicts: 5
files a merge would delete: 0

So the real cost is resolving 5 conflicts, not reconstructing the repository. I apologise for the framing.

https://claude.ai/code/session_01USgemLt4Rnz4SGB1Srf3GB

@Miracle656
Miracle656 merged commit 15f06a8 into Divinity-Labs:main Oct 26, 2026
1 check passed
@Miracle656

Copy link
Copy Markdown
Collaborator

Merged into main (15f06a8). tsc clean, 1,874 tests pass. Four things needed fixing, and the first would have failed silently in production.

1. The model had no migration.

OracleComparison was added to prisma/schema.prisma with nothing in prisma/migrations/. That used to be survivable — Lens ran prisma db push at startup, which reconciled drift automatically. Since #577 it runs prisma migrate deploy, so the table is simply never created. Every test passes, because the generated Prisma client is built from schema.prisma and knows the model; every write in production then fails against a table that does not exist.

I generated it offline with:

npx prisma migrate diff --from-schema-datamodel <main's schema> \
  --to-schema-datamodel prisma/schema.prisma --script

which produces 20261026120000_add_oracle_comparisons: CREATE TABLE plus your two declared indexes, no ALTER, no DROP. Worth knowing that command exists — it needs no database connection, so you can generate a migration without a local Postgres.

2. Both new routes declared no access control.

GET /compare/{asset}/history and GET /compare/{asset}/summary had no scope. The guard added in 4cd27d5 caught them by name. They are price reads, so both now carry read:prices — matching /compare/{asset} immediately above them. These have never shipped, so adding the requirement breaks no existing key.

3. The spec paths were unversioned. Everything documented has carried /v1 since 97c50bf. Moved, with the spec, client, snippets and README regenerated.

4. schema.prisma and .env.example were additive on both sides — main's venue-dispersion block against your oracle retention block — so both are kept.

What's good here. Persisting the comparison rather than computing it per request is the right call: a deviation you can only see live is one nobody can investigate afterwards. deviationPct at Decimal(10,4) with prices at Decimal(36,18) is also correctly chosen — the deviation is a ratio and does not need the price's precision, while the prices do.

ORACLE_COMPARISON_PRUNE_BATCH_SIZE with batched deletes that skip locked rows and yield between batches is the right shape for a retention job on a shared database. A single unbounded DELETE on a growing table is how a retention job takes the service down with it.

Note Lens's Render service is suspended, so the migration will apply on the next deploy after it is brought back up — and it should be applied deliberately, since migrate deploy is now the only thing that creates this table.

https://claude.ai/code/session_01USgemLt4Rnz4SGB1Srf3GB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The oracle comparison has no history

2 participants