[OPIK-8411] MPM presence in growth-report via --mpm (OPIK-8411)

README-ADMIN.md

@@ -144,9 +144,13 @@ HTML page, built entirely from the **admin chargeback report**. Distinct from
 `usage-report` (an experiment-count PDF): `growth-report` gives an org-wide view
 of workspaces, users, and platform adoption, broken down by workspace/department.
 
-**Requires an admin API key.** The report is derived entirely from the admin
-chargeback endpoint; with a non-admin key the command prints an error and exits
-non-zero — there is no fallback.
+**Requires an admin user's API key.** The report is derived entirely from the
+admin chargeback endpoint, which accepts only server admins (users in the
+server's admin user list) and, on self-hosted installs, organization admins of
+the install's organization. Workspace roles such as Manage don't count. With any
+other key the command prints an error and exits non-zero — there is no fallback.
+If the server restricts admin calls to localhost, even an admin key is refused
+("invalid connection") from anywhere else.
 
 ### Basic Usage
 
@@ -172,6 +176,7 @@ report is scoped to just those workspaces.
 - **`--no-open`**: Don't automatically open the generated HTML file after generation.
 - **`--csv-dir DIR`**: Also write Glue-ready CSV fact tables (`growth_users.csv`, `growth_workspaces.csv`, `growth_org_kpis.csv`) into `DIR`. `DIR` is created if it doesn't exist.
 - **`--no-html`**: Skip the HTML report. Requires `--csv-dir` (otherwise there is nothing to write, and the command errors out).
+- **`--mpm`**: Include MPM presence — whether each workspace uses MPM and how many models it monitors (registry models flagged `is_monitored`). Chargeback has no MPM data, so this is collected from `/api/mpm/v3/workspaces` (one call, covering the workspaces the API key's user belongs to) and, for the rest, the REST v2 model registry (one request per registry model). The key's user must be an **organization admin** for this to be complete. That's a different check from the one chargeback applies: the chargeback endpoint accepts server admins, while the registry lets organization admins read every workspace. For a workspace the user isn't a member of, a key that isn't an organization admin gets only the workspace's *public* models, so private monitored models are silently missed and the workspace still counts as checked. Off by default because of the extra requests.
 - **`--chargeback-report FILE`**: Read the chargeback report from a local JSON file (as saved by `cometx admin chargeback-report`) instead of calling the chargeback endpoint. Useful for CSV-only pipelines that already have a saved snapshot, or for re-running without re-fetching it. Note this skips the *chargeback* request only — the report still queries `/api/admin/service-accounts` to classify accounts, and falls back to a name-pattern heuristic if that request fails (`service_account_source` records which was used).
 
 ### The two time concepts
@@ -209,16 +214,17 @@ cometx admin growth-report --active-window 30d --leaderboard-top-n 10 \
 
 The report generates a single self-contained HTML file containing:
 
-- An **Organization overview (chargeback)** section with org-wide KPIs (Total workspaces, Total EM projects, New in {window} (% of base), Active workspaces %), a workspace platform-mix chart (EM / Opik / both / neither), workspace total-vs-active and added-vs-deleted charts, and a by-workspace table.
+- An **Organization overview (chargeback)** section with org-wide KPIs (Total workspaces, Total EM projects, New in {window} (% of base), Active workspaces %, plus MPM workspaces and Monitored models with `--mpm`), a workspace platform-mix chart (EM / Opik / both / neither), workspace total-vs-active and added-vs-deleted charts, and a by-workspace table (with an MPM models column under `--mpm`).
 - A **Users** section with Total / Active (`--active-window`) / Active % / New in {window} KPIs, plus active-vs-total, adoption-rate, per-capability, and user-churn charts.
-- A **Leaderboards** section ranking workspaces (by experiments and EM projects, exact from chargeback) and users (by Opik spans and EM activity), as top-N and active-aware bottom-N. Metrics with no data are omitted.
+- A **Leaderboards** section ranking workspaces (by experiments and EM projects, exact from chargeback; by MPM monitored models under `--mpm`) and users (by Opik spans and EM activity), as top-N and active-aware bottom-N. Metrics with no data are omitted.
 - A **Personal vs Service accounts** section splitting experiments / data / spans between personal and service accounts. Service accounts are identified from the admin service-accounts API when available, falling back to a labeled regex heuristic; the source is shown in the panel hint.
 
 ### Caveats
 
 - **Chargeback is required.** The whole report is derived from the admin chargeback report; without admin access the command errors out (non-zero exit).
 - **Workspace "created" is a proxy** — the earliest member `createdAt` in that workspace, since chargeback has no workspace-creation timestamp. The added-vs-deleted "deleted" series is also a best-effort proxy (all members removed) and typically reads ~0.
 - **"Total projects" counts EM projects only** — chargeback's per-workspace `projects[]` covers Experiment Management. Opik projects and MPM aren't represented there (Opik appears only as a per-user span count; MPM is absent), so the platform mix uses an Opik per-user proxy and excludes MPM.
+- **MPM comes from `--mpm`, not chargeback.** It reflects which models are monitored *now*, so past months can't be reconstructed; a trend builds up from monthly `--csv-dir` exports. A workspace whose MPM lookup fails is reported as unknown (empty in the CSV), never as zero, and the report says how many workspaces were checked. Counts are only complete when the key's user is an organization admin or a member of every workspace (see `--mpm`). `nb_models_registered` from the monthly usage report is not used: it counts every registry model and overstates MPM adoption.
 - **The people layer degrades independently.** If the chargeback payload parses but a section's inputs are missing, a warning is printed and only that section is dropped — the rest of the report still generates.
 
 ### CSV export
@@ -249,7 +255,9 @@ numeric/date types instead of typing everything as `string`.
 this table. The column exists so it is present and typed for a Glue crawler.
 Use the `deleted_users` KPI to see how many were excluded.
 
-**`growth_workspaces.csv`**: `report_date, workspace, member_count, num_projects, num_experiments, data_mb`
+**`growth_workspaces.csv`**: `report_date, workspace, member_count, num_projects, num_experiments, data_mb, mpm_enabled, num_monitored_models`
+
+`mpm_enabled` is `1`/`0` (so `SUM(mpm_enabled)` counts MPM workspaces) and `num_monitored_models` is a count; both are empty unless `--mpm` was given and that workspace could be checked. With `--mpm`, `growth_org_kpis.csv` also gets `mpm_workspaces`, `total_monitored_models`, `mpm_workspaces_unchecked`, and `mpm_member_lookup`. That last one is a `label` metric: its `metric_text` says how the `/api/mpm/v3/workspaces` call went (`ok`, `refused` for a 401/403, `not_found` for a 404, or `error`). Anything but `ok` means every workspace was checked through the registry instead. `refused` is the one to watch: the counts may be low, because the registry hides private models in other workspaces from users who aren't organization admins. The two totals cover only the workspaces that could be checked, so they are exact only when `mpm_workspaces_unchecked` is 0; otherwise they are lower bounds, and the HTML report labels them that way.
 
 **`growth_org_kpis.csv`**: `report_date, metric_name, metric_value, metric_unit, metric_text` — long format so new metrics arrive as new rows without ever changing the Glue schema. `metric_unit` is one of `count`, `percent`, `megabytes`, `label`.
 

README.md

@@ -585,8 +585,10 @@ page, built entirely from the **admin chargeback report**. Distinct from
 `usage-report` (experiment-count PDF): `growth-report` gives an org-wide view of
 workspaces, users, and platform adoption, per workspace/department.
 
-**Requires an admin API key** — the report is derived entirely from the admin
-chargeback endpoint, and with a non-admin key the command errors out (non-zero
+**Requires an admin user's API key** — the report is derived entirely from the
+admin chargeback endpoint, which accepts only server admins (the server's admin
+user list) and, on self-hosted installs, organization admins. Workspace roles
+such as Manage don't count. With any other key the command errors out (non-zero
 exit).
 
 ```
@@ -605,6 +607,7 @@ cometx admin growth-report [WORKSPACE ...]
 * `--exclude-personal` - Drop workspaces whose name matches `--personal-pattern` from the chargeback data (default: off; no effect without `--personal-pattern`).
 * `--personal-pattern REGEX` - Regex used with `--exclude-personal` to identify personal-workspace names to drop, e.g. `'^user-'` (default: none).
 * `--no-open` - Don't automatically open the generated HTML file after generation.
+* `--mpm` - Include MPM presence (monitored models per workspace), collected from the MPM and model-registry APIs. The key's user must be an organization admin (a separate check from chargeback's server-admin one) or private models in workspaces they aren't a member of are silently missed; off by default because it makes one request per registry model in workspaces the key's user isn't a member of.
 
 **Two time concepts:**
 * `--units` controls chart *granularity* — every chart shows the full all-time history bucketed at this resolution.
@@ -628,6 +631,7 @@ that section is dropped.
 * **Chargeback is required** — the whole report is derived from the admin chargeback report; without admin access the command errors out.
 * **Workspace "created" is a proxy** — the earliest member `createdAt`, since chargeback has no workspace-creation timestamp. The added-vs-deleted "deleted" series is also a best-effort proxy (all members removed) and typically reads ~0.
 * **"Total projects" counts EM projects only** — chargeback's per-workspace `projects[]` covers Experiment Management; Opik projects and MPM aren't represented (Opik appears only as a per-user span count; MPM is absent).
+* **MPM needs `--mpm`.** Chargeback has no MPM data; `--mpm` collects monitored models per workspace from the MPM and model-registry APIs (see README-ADMIN.md).
 
 **Examples:**
 ```

cometx/cli/admin.py

@@ -496,6 +496,7 @@ def get_parser_arguments(parser):
     cometx admin growth-report --csv-dir ./out
     cometx admin growth-report --csv-dir ./out --no-html
     cometx admin growth-report --chargeback-report report.json --csv-dir ./out
+    cometx admin growth-report --mpm --csv-dir ./out
 """
     growth_parser = subparsers.add_parser(
         "growth-report",
@@ -583,6 +584,18 @@ def get_parser_arguments(parser):
         default=False,
         action="store_true",
     )
+    growth_parser.add_argument(
+        "--mpm",
+        default=False,
+        action="store_true",
+        help=(
+            "Include MPM presence (monitored models per workspace), collected "
+            "from the MPM and model-registry APIs. The key's user must be an "
+            "organization admin (or a member of every workspace); otherwise "
+            "private models in other workspaces are silently missed. Makes one "
+            "request per registry model in workspaces the user is not a member of"
+        ),
+    )
     growth_parser.add_argument(
         "--chargeback-report",
         default=None,
@@ -916,6 +929,7 @@ def admin(parsed_args, remaining=None):
                     csv_dir=parsed_args.csv_dir,
                     no_html=parsed_args.no_html,
                     chargeback=preloaded,
+                    mpm=parsed_args.mpm,
                 )
             except GrowthReportError as e:
                 # `_exception_text`, not `str(e)`: a broken `__str__` would

cometx/cli/admin_gpu_app.py

@@ -478,7 +478,10 @@ def get_available_workspaces_and_projects(data: Dict) -> Dict[str, List[str]]:
         ws = exp_info.get("workspace", "Unknown")
         proj = exp_info.get("project_name", "Unknown")
         workspace_projects[ws].add(proj)
-    return {ws: sorted(list(projects)) for ws, projects in sorted(workspace_projects.items())}
+    return {
+        ws: sorted(list(projects))
+        for ws, projects in sorted(workspace_projects.items())
+    }
 
 
 def get_available_dates(data: Dict) -> Tuple[List[str], str, str]:
@@ -532,6 +535,7 @@ def main():
     # This must be done before any Streamlit elements are rendered
     try:
         import streamlit.config as _config
+
         _config.set_option("theme.primaryColor", "#1f77b4")
     except Exception:
         pass  # Fallback gracefully if internal API changes
@@ -602,7 +606,8 @@ def main():
     all_workspaces = list(workspace_projects_map.keys())
 
     # Custom CSS for professional styling (override Streamlit's default red/pink accent)
-    st.markdown("""
+    st.markdown(
+        """
         <style>
         /* Override checkbox checked color - target the role=checkbox div */
         div[role="checkbox"][aria-checked="true"] {
@@ -622,7 +627,9 @@ def main():
             color: white !important;
         }
         </style>
-    """, unsafe_allow_html=True)
+    """,
+        unsafe_allow_html=True,
+    )
 
     # Sidebar configuration
     with st.sidebar:
@@ -666,7 +673,9 @@ def main():
                 st.rerun()
 
         # Expandable project selector for fine-grained control
-        with st.expander("Select specific projects", expanded=st.session_state.projects_expander_open):
+        with st.expander(
+            "Select specific projects", expanded=st.session_state.projects_expander_open
+        ):
             # Group projects by workspace for cleaner display
             for ws in all_workspaces:
                 ws_projects = [f"{ws}/{p}" for p in workspace_projects_map.get(ws, [])]

cometx/cli/admin_gpu_report.py

@@ -21,8 +21,6 @@
 
 import matplotlib.pyplot as plt
 from comet_ml import API
-
-from .admin_utils import get_distinct_colors, save_chart
 from comet_ml.query import Metadata
 from PIL import Image
 from reportlab.lib import colors
@@ -44,6 +42,8 @@
 )
 from rich.table import Table
 
+from .admin_utils import get_distinct_colors, save_chart
+
 # Suppress matplotlib warnings about non-GUI backend
 warnings.filterwarnings("ignore", category=UserWarning, module="matplotlib")
 

cometx/cli/admin_growth_csv.py

@@ -19,7 +19,10 @@
 import os
 import tempfile
 
-from cometx.cli.admin_growth_users import _looks_like_service_account
+from cometx.cli.admin_growth_users import (
+    _looks_like_service_account,
+    workspace_org_totals,
+)
 
 USERS_HEADER = [
     "report_date",
@@ -48,6 +51,12 @@
     "num_projects",
     "num_experiments",
     "data_mb",
+    # MPM presence (OPIK-8411), appended per the stable-column-order rule.
+    # `mpm_enabled` is 1/0 (not true/false) so SUM(mpm_enabled) counts MPM
+    # workspaces directly; both are empty when MPM was not collected
+    # (no --mpm) or could not be determined for that workspace.
+    "mpm_enabled",
+    "num_monitored_models",
 ]
 
 ORG_KPIS_HEADER = [
@@ -191,6 +200,8 @@ def build_workspaces_rows(ws_records, report_date):
             _num_or_empty(w.num_projects),
             _num_or_empty(w.num_experiments),
             _num_or_empty(w.data_mb),
+            "" if w.mpm_enabled is None else int(w.mpm_enabled),
+            _num_or_empty(w.num_monitored_models),
         ]
         for w in ws_records
     ]
@@ -236,6 +247,7 @@ def collect_org_kpis(
     active_window_days,
     scope=None,
     excluded_personal_count=0,
+    mpm_member_lookup=None,
 ):
     """Flatten the report's org-level numbers into (name, value, unit) triples.
 
@@ -338,6 +350,26 @@ def collect_org_kpis(
         ("total_experiments", sum(w.num_experiments for w in ws_records), "count")
     )
     kpis.append(("total_data_mb", sum(w.data_mb for w in ws_records), "megabytes"))
+    # MPM presence (--mpm). Omitted (not zero) when not collected. The totals
+    # cover the known workspaces only; `mpm_workspaces_unchecked` (workspaces
+    # whose model count is unknown) says how many they miss, so a partial run
+    # is never mistaken for a complete one -- the totals are exact only when
+    # it is 0. Emitted whenever MPM was requested or reported, including a
+    # run where every lookup failed.
+    #
+    # `mpm_member_lookup` (a label: ok / refused / not_found / error) records
+    # how the mpm/v3/workspaces call went, so a dashboard can tell "MPM not
+    # installed" from "the key was refused" -- in the latter case the registry
+    # fallback may have missed private models. `None` when --mpm was not given.
+    org = workspace_org_totals(ws_records)
+    requested = mpm_member_lookup is not None
+    if org["mpm_workspaces"] is not None:
+        kpis.append(("mpm_workspaces", org["mpm_workspaces"], "count"))
+        kpis.append(("total_monitored_models", org["monitored_models"], "count"))
+    if requested or org["mpm_workspaces"] is not None:
+        kpis.append(("mpm_workspaces_unchecked", org["mpm_unchecked"], "count"))
+    if requested:
+        kpis.append(("mpm_member_lookup", None, "label", mpm_member_lookup))
 
     if split is not None:
         for bucket in ("personal", "service"):