Skip to content

feat(customer-analytics): add explainable account health scores#31

Open
xPolar wants to merge 5 commits into
masterfrom
capy/usage-metric-account-health
Open

feat(customer-analytics): add explainable account health scores#31
xPolar wants to merge 5 commits into
masterfrom
capy/usage-metric-account-health

Conversation

@xPolar

@xPolar xPolar commented Jun 19, 2026

Copy link
Copy Markdown

Problem

Customer analytics users need a quick way to prioritize account outreach without relying on an opaque score. Existing usage metrics already describe the activity each team cares about, but the accounts list does not turn those signals into a comparable, inspectable health indicator.

Changes

  • Adds a typed, query-time 0–100 health score derived from each team's existing usage metrics, comparing the current metric window with the immediately preceding equal period.
  • Batches scoring for only the accounts on the current page, groups compatible metric queries by source and interval, and leaves callers that do not request health unchanged.
  • Adds a mandatory Health column and first/default Health detail tab with status, formula, contributing factors, period values, trends, and no-data guidance.
  • Keeps health computation fail-open so usage-metric errors do not make the core accounts list unavailable.
  • Extends the account seeder with an opt-in healthy/at-risk demo dataset while preserving its event-free default behavior.
  • Adds deterministic Storybook coverage for the list-to-detail interaction and updates the Accounts agent guide.

How did you test this code?

I am an agent. I ran the focused Python scoring and accounts-query integration suites, the seed-command suite, the focused Jest suites, Ruff, oxfmt, schema generation, and git diff --check. I also exercised the Storybook account list in a browser and recorded clicking a 91/100 Healthy account into its Health detail; the same view showed a 32/100 At risk account and a No data account.

The repository-wide TypeScript check has one pre-existing error in frontend/src/scenes/experiments/MetricsView/new/VariantTimeseriesChart.tsx; none of the changed files report TypeScript errors.

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Docs update

No public documentation change is required for this prototype; the product-local Accounts agent guide is updated.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Captain Capy implemented the prototype from a human-directed request. The design deliberately reuses GroupUsageMetric definitions rather than inventing a second configuration surface or a black-box model. Scores are computed at query time, bounded to the current account page, cached against metric/config fingerprints, and degrade to no_data if optional scoring fails.

Open POST-078 POST-078


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@xPolar xPolar added the capy Generated by capy.ai label Jun 19, 2026 — with Capy AI

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 5 comments

Comment thread products/customer_analytics/backend/services/account_health.py Outdated
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@Scrapybara Scrapybara deleted a comment from capy-ai Bot Jun 19, 2026
@capy-ai

capy-ai Bot commented Jun 19, 2026

Copy link
Copy Markdown

🖼️ Visual QA — Accounts health score UI

Tested against the PR branch frontend + backend with one seeded demo account (--with-health-demo).

Before (master) — desktop 1280px

before-desktop.png

Columns: ACCOUNT | TAGS | NOTES | CSM | ACCOUNT EXECUTIVE | ACCOUNT OWNER — no Health column present.

After (PR branch) — desktop 1280px

after-desktop.png

After (PR branch) — narrow 390px

after-narrow.png

After (PR branch) — health detail expansion

after-health-detail.png

Observations

  • HEALTH column pinned as 2nd column (between ACCOUNT and TAGS), shows 90/100 with a green Healthy badge for the seeded account
  • Clicking the Health cell jumps directly to the Health tab in the expansion panel
  • Health detail panel shows score, status badge, methodology note, and factor cards with current/previous values and % trend
  • Fail-open confirmed — accounts list loads normally even when health scoring runs
  • ⚠️ Narrow viewport (390px): HEALTH column scrolls off-screen at 390px — expected table overflow behavior, not a regression

xPolar and others added 3 commits June 19, 2026 19:02
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@capy-ai

capy-ai Bot commented Jun 19, 2026

Copy link
Copy Markdown

🖼️ Visual QA — PR #31 (explainable account health scores)

Tested via Storybook with deterministic mock data (MSW). Verified three health states across desktop (1440 px) and narrow (375 px) viewports.


Accounts list — desktop (1440 px)

Three accounts, three health states, all rendered in the mandatory Health column:

Account Score Status
Acme Inc 91/100 🟢 Healthy
Globex 32/100 🔴 At risk
Hooli No data

Accounts list — desktop


Health detail panel — desktop (Acme Inc, 91/100 Healthy)

Clicking the health cell opens the Health expansion tab, showing:

  • Large coloured score (91) + "out of 100" label
  • Healthy status tag
  • Formula description text ("Each factor is how much of the previous period's usage was retained this period…")
  • Factor cards: "Events ingested" (7-day window, score 92, -8%), "Active users" (score 90, -10%), each with a green progress bar and current/previous values

Health detail — desktop


No-data expansion — desktop (Hooli)

When no usage metrics are configured for an account, the Health tab shows the graceful empty state:

"We don't have enough usage data to score this account yet. Health scores are derived from the activity measured by your team's usage metrics."

With a Set up usage metrics link. All three health states are visible simultaneously in the list.

No-data state — desktop


Health detail — narrow (375 px / mobile)

At 375 px the table collapses to ACCOUNT | HEALTH columns; the expanded health detail fits in a single-column layout. Score, status tag, and partial formula text are all legible.

Health detail — narrow


Findings

Area Status Notes
Health column in accounts list ✅ Pass Correct color-coding: green (≥80), red (<50)
Status tags ✅ Pass Healthy / At risk / No data render with correct variant colors
Health cell click → opens Health tab ✅ Pass Whole cell is the click target
Score display (header section) ✅ Pass Large bold score, "out of 100" sub-label, formula text
Factor cards ✅ Pass Metric name, interval label, score, progress bar, current/previous, trend %
No-data empty state ✅ Pass Correct guidance text + "Set up usage metrics" link
Narrow layout (375 px) ✅ Pass Table truncates cleanly; health detail single-column
Dashboards upsell toast ℹ️ Noise Storybook mock artifact; unrelated to this PR

No visual regressions found. All new surfaces render as expected across both viewport sizes.


Screenshots captured via Storybook (MSW mocks, AccountsTab.stories.tsx). Branch capy/qa-screenshots-pr31 hosts the raw images.

@capy-ai

capy-ai Bot commented Jun 19, 2026

Copy link
Copy Markdown

🎥 Visual QA — automated screenshot audit

Branch: capy/usage-metric-account-healthmaster
Surface: Customer analytics > Accounts (products/customer_analytics/frontend/components/Accounts/)
Commit: d0a3f5bd
Widths tested: 1456 px (desktop) · 768 px (narrow/tablet)


Before (master) — no Health column

Columns: ACCOUNT · TAGS · NOTES · CSM · ACCOUNT EXECUTIVE · ACCOUNT OWNER

Before — desktop 1456 px — no Health column


After (PR) — Health column added

HEALTH is inserted as the 2nd column (between ACCOUNT and TAGS) at both widths.

Desktop 1456 px — Health column visible, table empty state:

After — desktop 1456 px — Health column visible

Narrow 768 px — Health column still visible, sidebar collapsed to hamburger:

After — narrow 768 px — Health column visible with collapsed sidebar


Health detail expansion — default "Health" tab (no-data state)

Clicking the HEALTH cell opens the row expansion and lands on the Health tab (new default). When no usage metrics are configured the panel shows a friendly empty state with a "Set up usage metrics" CTA. Tabs: Health · Notes · Users · Usage · Spend.

Desktop — health detail panel expanded:

After — desktop — health detail expansion open (no-data state)

Narrow 768 px — health detail expansion open:

After — narrow 768 px — health detail expansion open


Column diff

Before After
Columns ACCOUNT, TAGS, NOTES, CSM, AE, AO ACCOUNT, HEALTH, TAGS, NOTES, CSM, AE, AO
Default expansion tab Notes Health
Health cell state Clickable chip (score or "No data")

Checklist

  • ✅ HEALTH column renders at position 2 (col 2) at both 1456 px and 768 px
  • ✅ Health tab is the new first/default tab in the row expansion
  • ✅ Clicking "No data" in HEALTH column opens the health detail expansion
  • ✅ Health empty-state panel renders with correct messaging and "Set up usage metrics" CTA link
  • ✅ Navigation link in empty state points to the correct config URL
  • ✅ Sidebar collapses to hamburger at 768 px — HEALTH column remains visible
  • ✅ All pre-existing columns (TAGS, NOTES, CSM, AE, AO) preserved in correct order
  • ✅ Expansion tabs (Health/Notes/Users/Usage/Spend) are all accessible
  • ⚠️ Health score shows No data in the Hedgebox demo — expected: the demo project has only 1 group-analytics group; the 7-day window comparison needs ≥ 2 accounts with two full periods of data. The no-data empty state renders correctly.
  • ℹ️ Seeded CSM/AE/Owner emails show as ca-seed-N@team2.customer-analytics.invalid — cosmetic, expected placeholder values from the seeder

Automated visual QA by Captain Capy on 2026-06-19 against commit d0a3f5bd (HEAD of capy/usage-metric-account-health).

@capy-ai

capy-ai Bot commented Jun 19, 2026

Copy link
Copy Markdown

🔍 Visual QA — PR #31 feat(customer-analytics): add explainable account health scores

Tested on branch capy/usage-metric-account-health (debug banner confirms 3c505165) against the Customer Analytics Accounts scene (/project/{team_id}/customer_analytics/accounts).


Before (master — no health column)

Desktop (1920px)

Before — desktop accounts list

Columns: ACCOUNT · TAGS · NOTES · CSM · ACCOUNT EXECUTIVE · ACCOUNT OWNER — no HEALTH column present.


After (PR branch — health column + expandable detail)

Desktop (1920px) — accounts list with health scores

After — desktop: accounts list with health column and scores

  • HEALTH is inserted as the mandatory second column (non-hideable in column configurator).
  • Three demo accounts seeded by --with-health-demo:
    • Acme Inc91/100 Healthy (green badge)
    • Globex Corp32/100 At risk (orange-red badge)
    • HooliNo data (neutral)
  • Clicking 91/100 on Acme Inc expands the Health detail inline:
    • Large "91 out of 100" score with Healthy status tag
    • Formula prose: "Each factor is how much of the previous period's usage was retained this period (current ÷ previous, capped at 100). The overall score is the average of the factor scores."
    • Events ingested (7-day window): score 92, "920 now · 1,000 before · −8%"
    • Active users (7-day window): score 90, "90 now · 100 before · −10%"
    • Health is the default tab (selected automatically when clicking a health cell); other tabs (Notes, Users, Usage, Spend) are still accessible.

Narrow (375px) — accounts list + health expansion

List view Health detail expanded
After — narrow: accounts list After — narrow: health detail open

At 375px, ACCOUNT and HEALTH columns are both visible; remaining columns (TAGS, NOTES, etc.) scroll horizontally. Health badges remain legible. The expansion panel stacks cleanly below the row.


Observations

Result
HEALTH column mandatory and in second position
Score badge colour-coding (green / orange-red / neutral)
Health tab is default on row expand
Factor cards: metric name, window, score bar, values, trend %
No data state: helpful prose + "Set up usage metrics" link
HEALTH column sortable (↕ indicator present)
Narrow layout: columns visible, expansion in-row
ACCOUNTS tile shows while loading (spinner, clears on load) ✅ expected

No layout breaks, overflow clipping, or z-index stacking issues spotted at either breakpoint. Health scores computed and rendered without crashing the accounts list when ClickHouse event data is absent (no_data path degrades gracefully as claimed).

Overall: ✅ Pass — frontend changes render correctly at desktop and narrow widths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capy Generated by capy.ai

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants