Skip to content

fix(monitoring,analytics,crowdfunding): bound growth and replace placeholder metrics - #1241

Closed
JamesEjembi wants to merge 1 commit into
MettaChain:mainfrom
JamesEjembi:fix/1195-1198-integrity-and-bounded-growth
Closed

JamesEjembi wants to merge 1 commit into
MettaChain:mainfrom
JamesEjembi:fix/1195-1198-integrity-and-bounded-growth

Conversation

@JamesEjembi

@JamesEjembi JamesEjembi commented Sep 26, 2026 •

Copy link
Copy Markdown

Fixes #1195
Fixes #1196
Fixes #1197
Fixes #1198

Summary

Two storage structures that grew for the lifetime of the contract are now capped, and four metrics that were fabricated constants are now derived from recorded state. Along the way two silent-divergence bugs turned up in analytics that are the exact failure mode #1195 is about.

144 tests pass across the three touched crates (analytics 43, crowdfunding 40, monitoring 61), up from 98. cargo fmt and cargo clippy are clean on the changed code.

Issue Subject Outcome
#1198 Crowdfunding success metrics blended placeholder analytics Placeholders replaced with values from recorded state; absent data returns None; duplicate line.rs deleted
#1196 quorum_guard history grew without bound History capped to a rolling window; reads bounded by a constant
#1197 Alerts were records with no delivery or retry Bounded alert log, batch read, self-contained payload, acknowledgement, documented contract
#1195 No test tied reported metrics to stored ones Integrity checksum, provenance, override tracing, consistency tests

Two unbounded Vecs

QuorumGuard.history (#1196) appended one entry per proposal forever. It is now a rolling window of MONITORING_MAX_QUORUM_HISTORY, mirroring the snapshot buffer the crate already uses. A separate lifetime total_recorded survives eviction, so the number of proposals observed is never lost — only per-proposal detail of old ones. participation_bps returns None for an evicted proposal rather than a stale value.

The alert log (#1197) is capped the same way at MONITORING_MAX_ALERT_LOG.

Placeholder metrics, and the state they needed

Location Was Now
investor_retention_rate 8_000 — a literal 80% placeholder share of recorded investors who have not refunded
accredited_investors total_investors * 7 / 10 — "assume 70%" counted from stored profiles
jurisdictions hardcoded US 60 / CA 20 / EU 10 / Other 10 grouped from InvestorProfile.jurisdiction
investment_distribution hardcoded 30/40/20/10 split histogram over real investments amounts
top_investor_amount max_investment = 0, so always zero real maximum from the same scan
get_funding_timeline 30 points on a straight target_amount / 30 ramp None

Two of these became Option because the honest answer is "undefined", not zero: retention over an empty cohort, and a jurisdiction split when no profile is stored. Reporting 8_000 there is what the issue was about.

get_investor_demographics also compounded this — it scanned every campaign id to find one, then reported figures unrelated to that campaign's investors.

get_funding_timeline is a deliberate None, not an oversight. investments is keyed by (campaign_id, investor) and holds a running total with no per-investment timestamp, and campaign_investors is an unordered Vec rather than a chronological log. No cumulative curve is reconstructible, and producing one from that data would be the same fabrication the issue reports. The docs describe the state change that would make it real.

line.rs deleted

1 835 lines, never compiled because it is not declared as a module in lib.rs, referenced nowhere in the repo, and all 45 of its public function names duplicate methods already in lib.rs. It was a live copy of every placeholder above.

Its unused ink_e2e dev-dependency is also removed. There is no tests/ directory and no reference to it, but it pulled in the substrate stack, and trie-db 0.28.0 does not compile on the nightly this repo pins — so cargo test -p propchain-crowdfunding failed before running a single test, on main too.

Alert delivery (#1197)

AlertTriggered stays authoritative; the ring buffer is the operational retry surface.

  • get_recent_alerts(since_alert_id, limit) — gap-free batch read. A cursor older than the window is clamped forward, so a worker that was offline resumes instead of silently receiving nothing and assuming it is up to date. limit = 0 means "everything retained" rather than "nothing", since an unset limit returning empty is indistinguishable from a healthy system.
  • alert_payload(alert_id) — self-contained blob with the alert type's stable name, a severity ranking, and canonical JSON, so a consumer needs no follow-up calls and no SCALE enum decoding.
  • acknowledge_alert(alert_id) — idempotent, so a crash between POST and ack is safe to redeliver.
  • pending_alert_count() — the retry-set size; a value that is not falling means delivery is stuck even while the contract is healthy.

Two bugs in my own ring-buffer readers, both caught by the eviction tests: the batch cursor treated since_alert_id = 0 as "skip alert 0" instead of the "handled nothing" sentinel, and pending_alert_count read without the buffer modulo, silently missing every id at or above the cap.

docs/alert_delivery.md specifies the contract, including a recommended worker loop and an explicit list of what the contract does not do.

Two further silent-divergence bugs

Both are in analytics and both are what #1195 is about, so they are fixed here.

1. batch_update_metrics discarded all but the last entry. It looped assigning self.current_metrics, so only the final MetricUpdate survived while BatchMetricsUpdated reported the full count as though all had been applied. Entries are now combined — volumes and counts sum, average_price is the volume-weighted mean — and the event carries the resulting value so it cannot disagree with storage.

⚠️ This is a behaviour change and needs a maintainer's acknowledgement: a caller that passed several entries and expected last-write-wins now gets a combined view. The previous behaviour was data loss, so I did not think it should be accepted silently.

2. batch_add_trends emitted BatchMetricsUpdated. Adding a trend emitted the market-metrics event, so a consumer tailing that event got a spurious one per trend with no way to tell them apart. It now emits BatchTrendsAdded.

A correction to #1195's premise

The issue asks for tests pinning get_market_metrics to a recomputed view. Two things make that not possible as written:

  1. get_market_metrics returns self.current_metrics.clone() — a single stored value. There is no second derivation path to pin against.
  2. There is nothing to recompute from. The contract stores no property valuations, no listing set and no trade tape. historical_trends, property_sentiments and portfolio_positions cannot yield an average price, a total volume or a listing count. average_price and total_volume are prices and amounts, and no aggregation of this contract's own state produces them.

The proposed oracle + staking snapshot would mean building an oracle integration that does not exist here — a large feature well beyond a consistency-test issue — and inventing a derivation that returns plausible-looking numbers would reproduce #1198 in a different contract. That is left as an explicit follow-up and documented on update_market_metrics rather than faked.

What is achievable without a second source, and what the acceptance criteria ask for, is now done: make provenance explicit and make a silent divergence impossible to miss.

  • Integrity checksum. An FNV-1a digest over the three metric fields is written on every update; verify_market_metrics_integrity() recomputes and compares. This is the on-chain expression of the recompute-equals-stored invariant. It is an integrity check, not a commitment — it does not constrain a malicious writer, who recomputes it anyway.
  • Provenance. get_metrics_provenance() returns the live metrics plus writer, timestamp, lifetime update_count, is_override and is_intact, so a consumer can tell an admin-supplied figure from a contract-derived one.
  • Override tracing. Every write emits MarketMetricsOverridden with previous and new values, writer and timestamp.
  • Single write path. set_market_metrics is the only writer, so the checksum, provenance and event cannot be left out of an update path.

Testing

98 → 144 tests, all new ones listed in MONITORING_ANALYTICS_RESOLUTIONS.md. Two #1198 tests need a state the public API cannot reach — invest requires an onboarded, KYC-approved, accredited profile, so a campaign built through the public API always has exactly one profile per investor. Two #[cfg(test)]-gated helpers (erase_investor_profile, set_investor_accredited) reach a missing or revoked profile; they are compiled out of non-test builds and are not part of the contract interface.

cargo test -p propchain-crowdfunding -p propchain-monitoring -p propchain-analytics --lib

Known remaining gap

Eight crates declare ink_e2e, but only contracts/lib uses it and no tests/ directory references it at all. Seven unused declarations remain after this change, so those crates' cargo test is likely still broken for the same trie-db reason. Left out of scope here, but worth its own PR.

Docs

…eholder metrics

Resolves MettaChain#1195, MettaChain#1196, MettaChain#1197 and MettaChain#1198. 144 tests pass across the three
touched crates, up from 98; fmt and clippy are clean on the changed code.

Two unbounded Vecs that grew for the lifetime of the contract are now
capped, and four placeholder metrics are now derived from recorded state.

MettaChain#1198 - crowdfunding reported fabricated analytics as measured outcomes.
get_campaign_analytics returned a hardcoded 8_000 bps "80% placeholder"
retention rate; get_investor_demographics assumed 70% accredited investors
and invented both a jurisdiction split and an investment-size histogram,
while reporting them for a campaign found by scanning every campaign id.
Retention is now the share of recorded investors who have not refunded,
accreditation is counted from stored profiles, and the split and histogram
are grouped and bucketed from real per-investor data. Because retention over
an empty cohort and a jurisdiction split with no stored profile are
undefined rather than zero, both became Option and return None.

get_funding_timeline built 30 points on a straight target_amount/30 ramp.
investments is keyed by (campaign_id, investor) and holds a running total
with no per-investment timestamp, and campaign_investors is an unordered
Vec, so no cumulative curve is reconstructible. It now returns None and
documents the state change that would make it real.

line.rs is deleted: 1 835 lines, never compiled because it is not declared
as a module, referenced nowhere, and all 45 of its public function names
duplicate methods that already exist in lib.rs.

Its unused ink_e2e dev-dependency is also removed. It pulled in trie-db
0.28.0, which does not compile on the nightly this repo pins, so
cargo test -p propchain-crowdfunding failed before running a single test.

MettaChain#1196 - QuorumGuard appended one entry per proposal to an uncapped Vec, so
storage and the participation read grew without bound. History is now a
rolling window of MONITORING_MAX_QUORUM_HISTORY, mirroring the snapshot
buffer the crate already uses, with a lifetime total_recorded that survives
eviction and None rather than a stale value for an evicted proposal.

MettaChain#1197 - alerts were on-chain records with no delivery path, so an operator
whose indexer was down during an incident missed the signal. Alerts are now
also appended to a bounded ring buffer, with a gap-free batch read, a
self-contained JSON payload, idempotent acknowledgement, and
pending_alert_count as the retry-set size. A stale cursor is clamped forward
so a worker that was away resumes rather than silently receiving nothing.
docs/alert_delivery.md specifies the contract, including what it does not do.

Two further silent-divergence bugs found on the way:

batch_update_metrics assigned self.current_metrics in a loop, so only the
last entry survived while the event reported the full count. Entries are now
combined - volumes and counts sum, average_price is the volume-weighted mean
- and the event carries the resulting value so it cannot disagree with
storage. This is a behaviour change: last-write-wins became a combined view.

batch_add_trends emitted BatchMetricsUpdated, so a consumer tailing that
event for metric changes got a spurious one per trend. It now emits
BatchTrendsAdded.

MettaChain#1195 - the premise needed correcting: get_market_metrics returns stored
metrics rather than building a second view, and this contract has no
property valuations, listing set or trade tape, so there is nothing on chain
to recompute an average price or volume from. An oracle + staking snapshot
would mean building an integration that does not exist, and inventing a
derivation would repeat MettaChain#1198 here. What is achievable without a second
source is now done: an FNV-1a checksum over the metric fields is written on
every update and verified by verify_market_metrics_integrity, provenance
exposes the writer, timestamp, count and override flag, every write emits
MarketMetricsOverridden with the previous and new values, and
set_market_metrics is the only writer so none of that can be bypassed.

MONITORING_ANALYTICS_RESOLUTIONS.md covers all four issues in detail,
including the parts that are deliberately left as follow-ups.
@drips-wave

drips-wave Bot commented Sep 26, 2026

Copy link
Copy Markdown

@JamesEjembi Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@JamesEjembi

Copy link
Copy Markdown
Author

Superseded. The analysis of these issues is recorded in #1242 (docs only); the code fix is being reworked.

@JamesEjembi
JamesEjembi deleted the fix/1195-1198-integrity-and-bounded-growth branch September 26, 2026 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment