release: wire the marketplace ingest pipeline

.github/workflows/marketplace-ingest.yml

@@ -0,0 +1,44 @@
+# AI Artefact Marketplace — push your catalog from GitHub Actions (spec 077)
+# Drop this into a repo as .github/workflows/marketplace-ingest.yml.
+#
+# Prereqs (only ONE thing is mandatory):
+#   1. Copy docs/marketplace/templates/map-to-envelope.mjs into your repo at ci/map-to-envelope.mjs
+#      (zero-dependency Node mapper; set the `kind` arg below for your repo type).
+#   2. Generate an ingest token in the Marketplace UI (your marketplace detail → Ingest token) and store it
+#      as the repo secret MARKETPLACE_INGEST_TOKEN.   ← the only required setup.
+#   (optional) Override the target by setting a repo variable MARKETPLACE_INGEST_URL; otherwise it defaults
+#      to the public ingest sidecar below.
+#
+# NETWORK: the default URL is the PUBLIC ingest sidecar (infaz-prod-0-0-func-mktingest) — reachable from
+# GitHub-hosted runners. (The hub vp.unic.com is IP-allowlisted and would block hosted runners; only point
+# MARKETPLACE_INGEST_URL there if you run on a self-hosted Unic-network runner.)
+
+name: Marketplace ingest
+on:
+  push:
+    branches: [main]
+    # Tighten to release tags if you only want to publish on release:  tags: ['*@*']
+  workflow_dispatch: {}
+
+jobs:
+  ingest:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with: { node-version: '20' }
+
+      - name: Map repo → artefact envelope
+        run: node ci/map-to-envelope.mjs unic-agents . > body.json
+
+      - name: Push full set to the marketplace
+        env:
+          # Defaults to the public ingest sidecar; override via repo variable if needed.
+          INGEST_URL: ${{ vars.MARKETPLACE_INGEST_URL || 'https://infaz-prod-0-0-func-mktingest.azurewebsites.net' }}
+          VP_TOKEN: ${{ secrets.MARKETPLACE_INGEST_TOKEN }}
+        run: |
+          echo "Pushing $(node -e 'console.log(require("./body.json").artefacts.length)') artefacts to $INGEST_URL …"
+          curl -sS -f -X POST "$INGEST_URL/api/marketplace/ingest" \
+            -H "X-VP-Token: $VP_TOKEN" \
+            -H "Content-Type: application/json" \
+            --data-binary @body.json

AGENTS.md

@@ -30,6 +30,7 @@ docs/
 ├── issues/                   # Grilled and scoped feature issues
 ├── process/                  # Process and workflow guides
 └── research/                 # Research notes and explorations
+ci/                           # Vendored marketplace mapper — copied verbatim, never edited
 ```
 
 ## Navigation
@@ -38,6 +39,7 @@ docs/
 - Shared release scripts: `packages/release-tools/scripts/`
 - Architectural decisions: `docs/adr/`
 - Process templates: `docs/process/`
+- Marketplace mapper: `ci/map-to-envelope.mjs` (vendored — see [Marketplace ingest](#marketplace-ingest))
 
 ## Commands
 
@@ -134,6 +136,26 @@ To ship a new plugin version:
 | Push to `develop` | ✓           | ✓ (changed packages) | —                  |
 | Push to `main`    | ✓           | ✓ (changed packages) | —                  |
 
+## Marketplace ingest
+
+Every push to `main` publishes this repository's six plugin cards to the Unic AI Artefact Marketplace, source `unic-agents-plugins`. `.github/workflows/marketplace-ingest.yml` maps the catalogue with `ci/map-to-envelope.mjs` and posts it to the public ingest sidecar. The mapper kind is `unic-agents`, because this is a Claude-plugin monorepo. The full documentation is [Push your catalog](https://vp.unic.com/docs/#/marketplace/), which needs a VP login. Run it by hand to see the envelope:
+
+```sh
+node ci/map-to-envelope.mjs unic-agents . > body.json
+```
+
+**Read that output, never POST it.** Off CI the mapper resolves `commit`, `ref` and `author` to empty strings, because it reads them from the CI environment. Posting such an envelope is how this repository's six cards got a version history with a blank author in the first place, and VP cannot repair one afterwards. The local Python tool now refuses a push it cannot attribute ([UNICGRAPH-572](https://uniccom.atlassian.net/browse/UNICGRAPH-572)), but that guard lives in that tool: neither this mapper nor the ingest endpoint will stop you. Only the workflow may push.
+
+A push sends the **full set**. VP diffs it and soft-removes any artefact the push omits, so never narrow the set to the plugins that changed.
+
+**Both files are upstream templates, copied verbatim.** Never hand-edit them and never reformat them. To update either one, re-copy it from its VP URL — [`map-to-envelope.mjs`](https://vp.unic.com/docs/marketplace/templates/map-to-envelope.mjs) and [`pipeline-github-action.yml`](https://vp.unic.com/docs/marketplace/templates/pipeline-github-action.yml). Both answer `401` without a signed-in VP session, so a person must fetch them; an agent cannot. The documentation warns twice that a hand-built envelope is how `provenance.author` goes missing, and this repository's six cards already carry an empty author from one such push on 10 June 2026. The same warning covers the artefact `content` block, which does not apply here: `mapUnicAgents` emits no `content` for any plugin, because plugins are `pointer` artefacts obtainable via `/plugin install`.
+
+`biome.json` excludes `ci/map-to-envelope.mjs` so that rule can hold: Biome reports two errors on the vendored mapper and reformats it. Keep the exclusion, and keep it scoped to that one path. A mapper reformatted to repo style makes the next re-copy read as a diff, and a directory-wide `!ci` would silently un-check any file added to `ci/` later.
+
+The mapper resolves `provenance.author` to `GITHUB_ACTOR`, the person who triggered the run, not the commit author. That is wrong on a merge-commit repository like this one, and it is filed upstream as [UNICGRAPH-575](https://uniccom.atlassian.net/browse/UNICGRAPH-575). **Do not patch it here.** A local fix would hide the defect instead of surfacing it.
+
+The ingest token lives only as the repository secret `MARKETPLACE_INGEST_TOKEN` (Settings → Secrets and variables → Actions). It never goes into a file, a commit or an agent conversation. The token also decides which marketplace the push lands in — the request body cannot name another source.
+
 ## Feature-driven development
 
 New work enters through the issue tracker as Features. Plan with `/wayfinder` when the work is too big for one agent session, or `/grill-with-docs` when it fits in one; then `/to-spec` → `/to-tickets` → `/archon-rollout`. `/to-tickets` iterates the breakdown with you and applies `ready-for-agent` once you approve it — that in-session approval is the checkpoint, and nothing re-checks it before dispatch. So keep the ready queue short: grill late, dispatch soon, and re-grill a ticket that has sat for more than a few days rather than trusting it. Use `/tdd` and `/implement` for individual issues. See [`docs/process/ai-development.md`](docs/process/ai-development.md) for the mental model.

biome.json

@@ -10,7 +10,8 @@
       "!**/.claude",
       "!**/.ralph",
       "!**/.remember",
-      "!**/*.min.js"
+      "!**/*.min.js",
+      "!ci/map-to-envelope.mjs"
     ]
   }
 }

ci/map-to-envelope.mjs

@@ -0,0 +1,230 @@
+#!/usr/bin/env node
+// 077 — repo → artefact-envelope mapper for CI pipelines. Zero dependencies (Node 18+).
+// Emits {provenance, artefacts:[...]} JSON to stdout; pipe it to the ingest endpoint.
+//
+//   node map-to-envelope.mjs skillrepo   .   > body.json
+//   node map-to-envelope.mjs unic-agents .   > body.json
+//
+// This is the JS port of specs/077-artefact-registry/tools/ingest_local.py — same envelope, so the
+// catalog accepts pipeline pushes identically to today's local push. The transport (curl to the
+// ingest sidecar / blob+SAS) is the pipeline's job; this only does the mapping.
+
+import { readFileSync, readdirSync, existsSync } from 'node:fs';
+import { join, relative, basename, dirname } from 'node:path';
+
+// ── tiny YAML front-matter parser (handles the SKILL.md shape: quoted multi-line scalars +
+//    one nested `metadata:` map). Avoids a js-yaml dependency so this drops into any pipeline. ──
+function parseFrontMatter(text) {
+  const m = text.match(/^---\s*\r?\n([\s\S]*?)\r?\n---/);
+  if (!m) return {};
+  const lines = m[1].split(/\r?\n/);
+  const data = {};
+  let parent = null;
+  for (let i = 0; i < lines.length; i++) {
+    const raw = lines[i];
+    if (!raw.trim()) continue;
+    const km = raw.match(/^(\s*)([\w-]+):\s?(.*)$/);
+    if (!km) continue;
+    const indent = km[1].length, key = km[2];
+    let val = km[3];
+    if (indent === 0) parent = null;
+    if (val === '' && key === 'metadata') { data.metadata = {}; parent = 'metadata'; continue; }
+    if (/^[>|][+-]?$/.test(val)) {             // YAML block scalar (folded > / literal |)
+      const literal = val[0] === '|';
+      const block = [];
+      while (i + 1 < lines.length) {
+        const next = lines[i + 1];
+        if (next.trim() === '') { block.push(''); i++; continue; }
+        if ((next.length - next.trimStart().length) <= indent) break;   // dedent → block ends
+        block.push(next.trimStart()); i++;
+      }
+      while (block.length && block[block.length - 1] === '') block.pop();
+      if (literal) { val = block.join('\n'); }
+      else {
+        let out = ''; let prevBlank = true;
+        for (const bl of block) {
+          if (bl === '') { out += '\n\n'; prevBlank = true; }
+          else { out += (prevBlank ? '' : ' ') + bl; prevBlank = false; }
+        }
+        val = out.trim();
+      }
+    } else if (val.startsWith('"')) {          // quoted, maybe multi-line
+      let content = val;
+      while (!(content.trimEnd().endsWith('"') && content.length > 1) && i + 1 < lines.length) {
+        i++; content += ' ' + lines[i].trim();
+      }
+      val = content.trim().replace(/^"/, '').replace(/"$/, '').trim();
+    } else {
+      val = val.trim().replace(/^"|"$/g, '');
+    }
+    if (parent === 'metadata' && indent >= 2) data.metadata[key] = val;
+    else data[key] = val;
+  }
+  return data;
+}
+
+function walk(dir, name, out = []) {
+  for (const e of readdirSync(dir, { withFileTypes: true })) {
+    if (e.isDirectory()) {
+      if (e.name === '.git' || e.name === 'node_modules') continue;   // skip noise
+      walk(join(dir, e.name), name, out);
+    } else if (e.name === name) out.push(join(dir, e.name));
+  }
+  return out;
+}
+
+// ── content (spec 115) ────────────────────────────────────────────────────────────────────────
+// The pushing pipeline is the ONLY place with guaranteed read access to its own repo, so it ships
+// the artefact BODY — not just a link to it. Consumers then need nothing but their VP login.
+// Caps mirror the server (which truncates + counts rather than rejecting, but declaring truncation
+// here keeps the numbers honest).
+const MAX_BODY = 256 * 1024;
+const MAX_TOTAL = 1024 * 1024;
+const MAX_FILES = 25;
+
+// Never read these into a payload that becomes company-wide readable — a checked-in secret must not
+// be published by accident. The mapper is the right place for this check: it is the only step with
+// filesystem context.
+const SECRET_RE = /(^|\/)(\.env(\..*)?|id_(rsa|dsa|ecdsa|ed25519)(\.pub)?)$|\.(key|pem|pfx|p12|keystore|jks)$/i;
+const TEXT_RE = /\.(md|txt|json|ya?ml|toml|py|sh|bash|ps1|js|mjs|cjs|ts|sql|csv|html|css|xml|ini|cfg|conf)$/i;
+
+function listFiles(dir, out = [], depth = 0) {
+  if (depth > 4) return out;
+  for (const e of readdirSync(dir, { withFileTypes: true })) {
+    const p = join(dir, e.name);
+    if (e.isDirectory()) {
+      if (['.git', 'node_modules', '__pycache__', '.venv'].includes(e.name)) continue;
+      listFiles(p, out, depth + 1);
+    } else out.push(p);
+  }
+  return out;
+}
+
+// Everything in the artefact's own folder except the primary file: scripts/, references/, templates.
+function collectCompanions(primaryPath) {
+  const dir = dirname(primaryPath);
+  const files = [];
+  let total = 0;
+  let truncated = false;
+  for (const p of listFiles(dir).sort()) {
+    if (p === primaryPath) continue;
+    const rel = relative(dir, p).replace(/\\/g, '/');
+    if (SECRET_RE.test(rel)) { console.error(`! skipped (secret-shaped): ${rel}`); truncated = true; continue; }
+    if (files.length >= MAX_FILES) { truncated = true; break; }
+    const buf = readFileSync(p);
+    if (buf.length > MAX_BODY || total + buf.length > MAX_TOTAL) { truncated = true; continue; }
+    total += buf.length;
+    const isText = TEXT_RE.test(rel);
+    files.push({
+      path: rel, bytes: buf.length,
+      encoding: isText ? 'utf-8' : 'base64',
+      format: isText ? 'text' : 'binary',
+      body: isText ? buf.toString('utf8') : buf.toString('base64'),
+    });
+  }
+  return { files, truncated };
+}
+
+// One SKILL.md (Agent-Skills standard) → envelope. Tolerates minimal front matter (name + description
+// only — version/keywords/owner default cleanly when there's no `metadata:` block).
+function skillFromFile(repo, path) {
+  const src = readFileSync(path, 'utf8');
+  const fm = parseFrontMatter(src);
+  const meta = fm.metadata || {};
+  const name = fm.name || basename(dirname(path));
+  const keywords = String(meta.tags || '').split(',').map((s) => s.trim()).filter(Boolean);
+  const rel = relative(repo, path).replace(/\\/g, '/');
+  const bytes = Buffer.byteLength(src, 'utf8');
+  const { files, truncated } = collectCompanions(path);
+  return {
+    artefact_id: name, type: 'skill', name,
+    // An explicit `version` WINS over `last_verified`, and a top-level one over a nested one.
+    // This read `metadata.last_verified` and nothing else, so an author who wrote
+    // `version: "1.0.0"` saw the skill published as a date — or, with no `last_verified` at all,
+    // as "unversioned" while the version sat right there in the file (six skills in UNICIS-skills
+    // alone). The two are not the same statement: `version` is what the artefact IS,
+    // `last_verified` is when somebody last checked it. Same precedence as the hand-upload path in
+    // marketplace_contrib.py — two doors into one registry must not disagree about what a file
+    // says (UNICGRAPH-438).
+    version: String(fm.version || meta.version || meta.last_verified || 'unversioned'),
+    description: fm.description || '',
+    keywords, owner: meta.owner || '', homepage: '',
+    entrypoints: [{ kind: 'skill', ref: rel, label: 'SKILL.md' }],
+    links: [], spec: fm,
+    // The skill itself — this is what makes it obtainable without repo access.
+    content: {
+      format: 'markdown', path: rel, encoding: 'utf-8',
+      body: bytes <= MAX_BODY ? src : undefined,
+      bytes, files, truncated: truncated || bytes > MAX_BODY,
+    },
+  };
+}
+
+// Agent-Skills repo, skills under `skills/` (e.g. skillrepo-monorepo).
+function mapSkillrepo(repo) {
+  return walk(join(repo, 'skills'), 'SKILL.md').sort().map((p) => skillFromFile(repo, p));
+}
+
+// Agent-Skills repo, ANY layout — every SKILL.md anywhere (e.g. UNICIS-skills: <skill>/SKILL.md at root).
+function mapSkills(repo) {
+  return walk(repo, 'SKILL.md').sort().map((p) => skillFromFile(repo, p));
+}
+
+function mapUnicAgents(repo) {
+  const mk = JSON.parse(readFileSync(join(repo, '.claude-plugin', 'marketplace.json'), 'utf8'));
+  const market = mk.name || 'unic-agent-plugins';
+  const out = [];
+  for (const p of mk.plugins || []) {
+    const pj = join(repo, p.source || '', '.claude-plugin', 'plugin.json');
+    if (!existsSync(pj)) { console.error(`! missing ${pj}`); continue; }
+    const j = JSON.parse(readFileSync(pj, 'utf8'));
+    const name = j.name || p.name;
+    const eps = [{ kind: 'install', ref: `/plugin install ${name}@${market}`, label: 'Install' }];
+    for (const c of j.commands || []) eps.push({ kind: 'command', ref: c });
+    for (const a of j.agents || []) eps.push({ kind: 'agent', ref: a });
+    out.push({
+      artefact_id: name, type: 'claude-plugin', name, version: j.version || '0.0.0',
+      description: j.description || '', keywords: j.keywords || [],
+      owner: (j.author || {}).name || '', homepage: j.homepage || '',
+      entrypoints: eps, links: j.homepage ? [{ label: 'Homepage', url: j.homepage }] : [], spec: j,
+    });
+  }
+  return out;
+}
+
+const MAPPERS = { skills: mapSkills, skillrepo: mapSkillrepo, 'unic-agents': mapUnicAgents };
+const REPO_URLS = {
+  skillrepo: 'https://dev.azure.com/unicag/CloudProjectHub/_git/skillrepo-monorepo',
+  'unic-agents': 'https://github.com/unic/unic-agents-plugins',
+};
+
+// ADO's Build.Repository.Uri carries the org as basic-auth userinfo
+// (https://unicag@dev.azure.com/...). Strip it so the published link is clean & shareable.
+function cleanRepoUrl(u) {
+  return String(u || '').replace(/^(https?:\/\/)[^/@]+@/, '$1');
+}
+
+const [kind, repo = '.'] = process.argv.slice(2);
+if (!MAPPERS[kind]) { console.error(`usage: map-to-envelope.mjs <${Object.keys(MAPPERS).join('|')}> <repoDir>`); process.exit(2); }
+const artefacts = MAPPERS[kind](repo);
+console.error(`mapped ${artefacts.length} artefacts from ${kind}`);
+process.stdout.write(JSON.stringify({
+  provenance: {
+    // Prefer the CI-provided repo URL (works for any repo); fall back to the known map.
+    repo_url: cleanRepoUrl(process.env.BUILD_REPOSITORY_URI)
+      || (process.env.GITHUB_REPOSITORY ? `https://github.com/${process.env.GITHUB_REPOSITORY}` : '')
+      || REPO_URLS[kind] || '',
+    commit: process.env.GIT_COMMIT || process.env.BUILD_SOURCEVERSION || process.env.GITHUB_SHA || '',
+    ref: process.env.BUILD_SOURCEBRANCH || process.env.GITHUB_REF || '',
+    // WHO changed it. Without this the marketplace can show a new version and a commit hash but no
+    // person, and the only name on the artefact page is its OWNER — which reads as an attribution
+    // and is not one (UNICGRAPH-438). The commit AUTHOR is preferred over whoever started the
+    // build: a queued or re-run pipeline says nothing about who wrote the change. VP cannot look
+    // this up afterwards — commit search is unavailable on the unicag organisation — so the push
+    // is the only moment the name exists.
+    author: process.env.BUILD_SOURCEVERSIONAUTHOR || process.env.GITHUB_ACTOR
+      || process.env.BUILD_REQUESTEDFOR || '',
+    pipeline: 'ci:map-to-envelope.mjs',
+  },
+  artefacts,
+}));