Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion plugins/knowledge/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "knowledge",
"version": "0.19.0",
"version": "0.19.1",
"description": "Ingests external knowledge into synthesized artifacts: book-distill (PDF or EPUB into author-attributed reference files), video-digest (one YouTube or X video: transcript, links, repo applicability), course-digest (Dometrain and Teachable courses into recommendations), docpage-digest (one documentation page into a verified knowledge slice), and map-corpus (a multi-resource corpus into a classified link map and an approved docpage-digest queue). setup sets where artifacts land.",
"author": {
"name": "Melodic Software",
Expand Down
15 changes: 15 additions & 0 deletions plugins/knowledge/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,21 @@ All notable changes to the `knowledge` plugin are recorded here. The `version` i
`.claude-plugin/plugin.json` is the delivery vehicle. A consumer receives a change
only after that version increases.

## [0.19.1] - 2026-10-03

### Fixed

- **`video-digest` and `course-digest` install into and load from the knowledge plugin's own data
directory, whatever `CLAUDE_PLUGIN_DATA` the Bash tool's shell holds.** Another plugin's
SessionStart hook can export its own data directory under that name for every Bash call, and
`setup-deps.mjs` installed the extraction dependencies there. `run.mjs` and `setup-deps.mjs`
now take a leading `--data-dir`, which every documented command passes as
`"${CLAUDE_PLUGIN_DATA}"` (or `"<plugin-data>"` in the spoke files). Without the flag they
accept an inherited value only when it names this plugin, and `setup-deps.mjs` stops before
writing anything when no directory resolves. The pre-computed dependency checks read the
substituted path, and the bootstrap recovery command names the launcher by absolute path with
the resolved `--data-dir`.

## [0.19.0] - 2026-10-02

### Added
Expand Down
38 changes: 23 additions & 15 deletions plugins/knowledge/skills/course-digest/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ shell: bash
## Pre-computed context

```!
{ printf 'course-extraction deps: '; node -e "const fs=require('fs'),path=require('path'),p=process.env.CLAUDE_PLUGIN_DATA;console.log(p&&fs.existsSync(path.join(p,'node_modules','@melodic','video-digestion'))?'installed':'MISSING - run setup-deps.mjs (see Prerequisites)')" 2>/dev/null || echo "MISSING - node not found (see Prerequisites)"; }
{ printf 'Playwright Chromium: '; node -e "const fs=require('fs'),path=require('path');const b=process.env.PLAYWRIGHT_BROWSERS_PATH||(process.env.CLAUDE_PLUGIN_DATA&&path.join(process.env.CLAUDE_PLUGIN_DATA,'ms-playwright'));const ok=b&&fs.existsSync(b)&&fs.readdirSync(b).some(n=>n.startsWith('chromium'));console.log(ok?'installed':'MISSING - run setup-deps.mjs (see Prerequisites)')" 2>/dev/null || echo "MISSING - node not found (see Prerequisites)"; }
{ printf 'course-extraction deps: '; node -e "const fs=require('fs'),path=require('path'),p=process.argv[1];console.log(p&&fs.existsSync(path.join(p,'node_modules','@melodic','video-digestion'))?'installed':'MISSING - run setup-deps.mjs (see Prerequisites)')" "${CLAUDE_PLUGIN_DATA}" 2>/dev/null || echo "MISSING - node not found (see Prerequisites)"; }
{ printf 'Playwright Chromium: '; node -e "const fs=require('fs'),path=require('path');const b=process.env.PLAYWRIGHT_BROWSERS_PATH||(process.argv[1]&&path.join(process.argv[1],'ms-playwright'));const ok=b&&fs.existsSync(b)&&fs.readdirSync(b).some(n=>n.startsWith('chromium'));console.log(ok?'installed':'MISSING - run setup-deps.mjs (see Prerequisites)')" "${CLAUDE_PLUGIN_DATA}" 2>/dev/null || echo "MISSING - node not found (see Prerequisites)"; }
{ printf 'ffmpeg: '; command -v ffmpeg >/dev/null 2>&1 && { ffmpeg -version 2>/dev/null | head -1; :; } || echo "MISSING — install ffmpeg (see Prerequisites)"; }
{ printf 'ImageMagick: '; command -v magick >/dev/null 2>&1 && { magick -version 2>/dev/null | head -1; :; } || echo "MISSING — install ImageMagick 7 (see Prerequisites)"; }
```
Expand All @@ -35,7 +35,7 @@ This skill's `.work/` root resolves through the knowledge plugin's own `library_

## Prerequisites (verify before starting)

1. **course-extraction deps**. `node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/setup-deps.mjs"`. Installs the pipeline's node dependencies into `${CLAUDE_PLUGIN_DATA}` (persists across plugin updates) and provisions Playwright's Chromium into `${CLAUDE_PLUGIN_DATA}/ms-playwright`. Idempotent. Safe to re-run, and re-run after a plugin update.
1. **course-extraction deps**. `node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/setup-deps.mjs" --data-dir "${CLAUDE_PLUGIN_DATA}"`. Installs the pipeline's node dependencies into `${CLAUDE_PLUGIN_DATA}` (persists across plugin updates) and provisions Playwright's Chromium into `${CLAUDE_PLUGIN_DATA}/ms-playwright`. Idempotent. Safe to re-run, and re-run after a plugin update.
2. **Platform auth**. Set `COURSE_EMAIL`/`COURSE_PASSWORD` (Dometrain → Clerk) or `TEACHABLE_EMAIL`/`TEACHABLE_PASSWORD` (Teachable) in your shell before invoking; they inherit into the pipeline's node subprocess. The env-var prefix is course-config-driven via `platformConfig.authEnvPrefix`. Session cookies persist under `${CLAUDE_PLUGIN_DATA}/auth/<platform>.auth-state.json` and are reused across runs. **Interactive manual login is the fallback** when no credentials are set. It opens a browser window for you to log in. NOTE: the manual-login prompt (`node:readline` + headed browser) may not function under headless plugin execution; env-var + cookie-reuse carry the skill regardless, and manual login is a known limitation there, not a blocker. These credentials live in shell environment variables, not in plugin `userConfig`: this plugin's `userConfig` options are non-secret scalars, and a stored option value is not a secret store.
3. **ffmpeg**, required for video frame extraction (scene detection, interval capture). Check: `ffmpeg -version`. Install: `winget install Gyan.FFmpeg` (Windows), `brew install ffmpeg` (macOS), `sudo apt install ffmpeg` (Linux). Floor 7.1+ (newer codecs, AV1, Opus, degrade or fail below this).
4. **ImageMagick 7**, required by `classify-frames.js` for contact sheet generation (`magick montage`). Check: `magick -version`. Install: `winget install ImageMagick.ImageMagick` (Windows), `brew install imagemagick` (macOS), `sudo apt install imagemagick` (Linux). Ubuntu <26.04 ships v6. V7 may require building from source.
Expand All @@ -45,10 +45,10 @@ If any prerequisite fails, stop and inform the user. Re-run `setup-deps.mjs` for

## Running the pipeline scripts

Every extraction script runs through the launcher, which resolves the vendored node dependencies from `${CLAUDE_PLUGIN_DATA}` and pins Playwright's browser path:
Every extraction script runs through the launcher, which resolves the vendored node dependencies from the data directory its leading `--data-dir` names and pins Playwright's browser path:

```bash
node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" <script.js> [args…]
node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" --data-dir "${CLAUDE_PLUGIN_DATA}" <script.js> [args…]
```

Gate on `setup-deps.mjs` first (Prerequisites above).
Expand All @@ -59,16 +59,16 @@ The Playwright batch script handles transcripts, video frame extraction, and cou

```bash
# Full extraction: transcripts + frames + metadata
node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" extract-course.js --course-dir <path-to-course-data> --extract-frames
node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" --data-dir "${CLAUDE_PLUGIN_DATA}" extract-course.js --course-dir <path-to-course-data> --extract-frames

# Transcripts only (faster, no ffmpeg needed)
node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" extract-course.js --course-dir <path-to-course-data>
node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" --data-dir "${CLAUDE_PLUGIN_DATA}" extract-course.js --course-dir <path-to-course-data>

# Frames only (skip transcripts already extracted)
node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" extract-course.js --course-dir <path-to-course-data> --extract-frames --skip-transcripts
node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" --data-dir "${CLAUDE_PLUGIN_DATA}" extract-course.js --course-dir <path-to-course-data> --extract-frames --skip-transcripts

# Course metadata only
node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" extract-course.js --course-dir <path-to-course-data> --metadata-only
node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" --data-dir "${CLAUDE_PLUGIN_DATA}" extract-course.js --course-dir <path-to-course-data> --metadata-only
```

Script uses Playwright's bundled Chromium with a fresh temp context (not Chrome itself. Chrome 136+ blocks CDP on default profiles). Auth handled via `addCookies()` after context launch: with credentials set it logs in and saves state; subsequent runs inject cached cookies automatically. Skips already-extracted lessons (crash-safe, resumable). Runs headless by default (`--show-browser` to show the browser).
Expand All @@ -88,7 +88,7 @@ Script uses Playwright's bundled Chromium with a fresh temp context (not Chrome

```bash
# Run in background with nohup — no timeout limit
nohup node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" extract-course.js --course-dir <path> > extraction.log 2>&1 &
nohup node "${CLAUDE_PLUGIN_ROOT}/skills/course-digest/extraction/run.mjs" --data-dir "${CLAUDE_PLUGIN_DATA}" extract-course.js --course-dir <path> > extraction.log 2>&1 &
echo $! # save PID

# Monitor progress periodically
Expand Down Expand Up @@ -237,12 +237,20 @@ Repo-applicability analysis follows the template in [reference/analysis-template

## Spoke paths

The `context/` files write this skill's directory as `<skill-dir>`, which is `${CLAUDE_SKILL_DIR}`.
Put that path in place of the placeholder before running a command. Those files arrive through the
Read tool as plain bytes, so a `${…}` token in them would reach the Bash tool unsubstituted, and the
Bash tool's environment has no `CLAUDE_SKILL_DIR` to expand it from. Basis: the plugins reference,
The `context/` files write this skill's directory as `<skill-dir>`, which is `${CLAUDE_SKILL_DIR}`,
and the plugin data directory as `<plugin-data>`, which is `${CLAUDE_PLUGIN_DATA}`. Put those paths
in place of the placeholders before running a command. Those files arrive through the Read tool as
plain bytes, so a `${…}` token in them would reach the Bash tool unsubstituted, and the Bash tool's
environment has no `CLAUDE_SKILL_DIR` to expand it from.

Every `run.mjs` and `setup-deps.mjs` command takes the leading `--data-dir` flag shown above.
The Bash tool's environment does not carry this plugin's `CLAUDE_PLUGIN_DATA`, and another
plugin's SessionStart hook can put its own data directory there under that name. The scripts
therefore take the directory from the flag, and accept an inherited value only when it names this
plugin. Basis: the plugins reference,
<https://code.claude.com/docs/en/plugins-reference#where-each-variable-resolves>, verified
2026-09-30; recheck when that table adds supporting files to where a `${…}` reference resolves.
2026-10-02; recheck when that table adds supporting files to where a `${…}` reference resolves, or
lists the Bash tool among the processes that receive the variables.

## Storage

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Storage Schema

All course data lives under the invoking project's `library_dir` setting (or `${CLAUDE_PLUGIN_DATA}` when no library dir is configured), as `courses/<platform>/<slug>/`.
All course data lives under the invoking project's `library_dir` setting (or `<plugin-data>` when no library dir is configured), as `courses/<platform>/<slug>/`.

## Platform naming

Expand Down
12 changes: 6 additions & 6 deletions plugins/knowledge/skills/course-digest/context/workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,10 @@ Repo → Validate → THEN Synthesize.

**Steps (sequential):**

1. `node "<skill-dir>/extraction/run.mjs" classify-frames.js --course-dir <path> --phase contact-sheets`: generate labeled thumbnail grids
2. `node "<skill-dir>/extraction/run.mjs" classify-frames.js --course-dir <path> --phase dedup`: near-duplicate detection
3. `node "<skill-dir>/extraction/run.mjs" generate-manifests.js --course-dir <path>`: curate frame sets per lesson
4. `node "<skill-dir>/extraction/run.mjs" classify-frames.js --course-dir <path> --phase summary`: print frame inventory
1. `node "<skill-dir>/extraction/run.mjs" --data-dir "<plugin-data>" classify-frames.js --course-dir <path> --phase contact-sheets`: generate labeled thumbnail grids
2. `node "<skill-dir>/extraction/run.mjs" --data-dir "<plugin-data>" classify-frames.js --course-dir <path> --phase dedup`: near-duplicate detection
3. `node "<skill-dir>/extraction/run.mjs" --data-dir "<plugin-data>" generate-manifests.js --course-dir <path>`: curate frame sets per lesson
4. `node "<skill-dir>/extraction/run.mjs" --data-dir "<plugin-data>" classify-frames.js --course-dir <path> --phase summary`: print frame inventory

**Output:** Contact sheets, dedup report, manifests per lesson.

Expand Down Expand Up @@ -129,7 +129,7 @@ downloaded source code ZIPs. If neither exists, skip.

**Steps (GitHub repo path):**

1. `node "<skill-dir>/extraction/run.mjs" analyze-code-repo.js --course-dir <path>`: clone to temp, detect structure, write metadata
1. `node "<skill-dir>/extraction/run.mjs" --data-dir "<plugin-data>" analyze-code-repo.js --course-dir <path>`: clone to temp, detect structure, write metadata
2. Clone again to `code/repo/` for Phase 3 access: `git clone --depth 1 --single-branch <url> code/repo/`
3. Review `code/analysis.json` for repo structure (per-section vs single-state)
4. Build section-to-module mapping table: which repo sections correspond to which course modules
Expand Down Expand Up @@ -170,7 +170,7 @@ only, then discard.

**Steps:**

1. `node "<skill-dir>/extraction/run.mjs" validate-extraction.js --course-dir <path>`: run all quality checks
1. `node "<skill-dir>/extraction/run.mjs" --data-dir "<plugin-data>" validate-extraction.js --course-dir <path>`: run all quality checks
2. Review `validation-report.json` and fix any FAIL items before proceeding
3. On re-runs: compare against previous `validation-report.json` for regressions

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
import { spawnSync } from "node:child_process";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";

import { afterAll, describe, expect, it } from "vitest";

const dir = path.dirname(fileURLToPath(import.meta.url));
const sandboxes = [];
afterAll(() => {
for (const sandbox of sandboxes) fs.rmSync(sandbox, { recursive: true, force: true });
});

/** Another plugin's data directory, empty, the way a leaked CLAUDE_PLUGIN_DATA names it. */
function foreignDataDir() {
const root = fs.mkdtempSync(path.join(os.tmpdir(), "course-setup-deps-"));
sandboxes.push(root);
const foreign = path.join(root, "codex-openai-codex");
fs.mkdirSync(foreign);
return foreign;
}

function run(script, args, env) {
return spawnSync(process.execPath, [path.join(dir, script), ...args], {
encoding: "utf8",
timeout: 20000,
env: { ...process.env, ...env },
});
}

describe("data directory resolution in the Bash-run entry points", () => {
it("setup-deps refuses an inherited CLAUDE_PLUGIN_DATA naming another plugin and writes nothing there", () => {
const foreign = foreignDataDir();
const result = run("setup-deps.mjs", [], { CLAUDE_PLUGIN_DATA: foreign });
expect(result.status).toBe(1);
expect(result.stderr).toContain('--data-dir "${CLAUDE_PLUGIN_DATA}"');
expect(fs.readdirSync(foreign)).toEqual([]);
});

it("setup-deps rejects an unsubstituted placeholder before creating anything", () => {
const foreign = foreignDataDir();
const result = run("setup-deps.mjs", ["--data-dir", "<plugin-data>"], { CLAUDE_PLUGIN_DATA: foreign });
expect(result.status).toBe(2);
expect(result.stderr).toContain("unsubstituted placeholder");
expect(fs.readdirSync(foreign)).toEqual([]);
});

it("run.mjs hands the child the --data-dir value, not an inherited one naming another plugin", () => {
const foreign = foreignDataDir();
const knowledge = path.join(path.dirname(foreign), "knowledge-melodic-software");
const dest = path.join(path.dirname(foreign), "course");
const stub = pathToFileURL(path.join(dir, "test-support", "register-playwright-stub.mjs")).href;
const result = run(
"run.mjs",
["--data-dir", knowledge, "build-course-json.js", "--course-url", "https://example.test/courses/enrolled/1", "--output-dir", dest],
{ CLAUDE_PLUGIN_DATA: foreign, NODE_OPTIONS: `--import ${stub}` },
);
expect(result.stderr).toContain("playwright stub: chromium.launch blocked in tests");
expect(fs.existsSync(path.join(knowledge, "auth"))).toBe(true);
expect(fs.readdirSync(foreign)).toEqual([]);
});

it("run.mjs drops an inherited value naming another plugin when no flag is given", () => {
const foreign = foreignDataDir();
const home = path.dirname(foreign);
const dest = path.join(home, "course");
const stub = pathToFileURL(path.join(dir, "test-support", "register-playwright-stub.mjs")).href;
const result = run(
"run.mjs",
["build-course-json.js", "--course-url", "https://example.test/courses/enrolled/1", "--output-dir", dest],
{ CLAUDE_PLUGIN_DATA: foreign, HOME: home, USERPROFILE: home, NODE_OPTIONS: `--import ${stub}` },
);
expect(result.stderr).toContain("playwright stub: chromium.launch blocked in tests");
expect(fs.readdirSync(foreign)).toEqual([]);
expect(fs.existsSync(path.join(home, ".claude", "course-digest", "auth"))).toBe(true);
});

it("run.mjs rejects an unsubstituted placeholder instead of launching the script", () => {
const result = run("run.mjs", ["--data-dir", "${CLAUDE_PLUGIN_DATA}", "validate-extraction.js"], {});
expect(result.status).toBe(2);
expect(result.stderr).toContain("unsubstituted placeholder");
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
/**
* The knowledge plugin's data directory for `run.mjs` and `setup-deps.mjs`, which
* Claude runs through the Bash tool. That tool's environment does not carry
* `CLAUDE_PLUGIN_DATA`, and another plugin's SessionStart hook can persist its own
* data directory there under that name, so the skill passes a leading
* `--data-dir "${CLAUDE_PLUGIN_DATA}"`, substituted when the skill loads, and that
* value wins. Without the flag an inherited value is used only when its last path
* segment names this plugin (`knowledge-<marketplace>`), the order the marketplace's
* on-demand-dependencies convention sets. Basis: plugins reference, "Where each
* variable resolves", as of 2026-10-02.
*
* Pure, so the launcher's contract is unit-testable without spawning a process.
*/

/**
* Split a leading `--data-dir <dir>` off the launcher's argv.
*
* @param {string[]} argv
* @returns {{ dataDir: string|undefined, rest: string[] }}
*/
export function takeDataDirFlag(argv) {
if (argv[0] !== "--data-dir") {
return { dataDir: undefined, rest: [...argv] };
}
if (argv.length < 2) {
throw new Error("`--data-dir` requires a directory value");
}
return { dataDir: argv[1], rest: argv.slice(2) };
}

/**
* @param {string|undefined} flagValue
* @param {NodeJS.ProcessEnv} env
* @returns {string|undefined} undefined when neither source names this plugin
*/
export function resolvePluginData(flagValue, env) {
if (flagValue) {
if (flagValue.includes("${") || flagValue.includes("<plugin-data>")) {
throw new Error(`\`--data-dir\` got an unsubstituted placeholder: \`${flagValue}\``);
}
return flagValue;
}
const inherited = env.CLAUDE_PLUGIN_DATA;
const lastSegment = inherited?.replace(/[\\/]+$/, "").split(/[\\/]/).pop() ?? "";
return /^knowledge(-|$)/.test(lastSegment) ? inherited : undefined;
}
Loading
Loading