Disclose repository guidance progressively

CLAUDE.md

@@ -6,50 +6,34 @@ This is the canonical project guide for agents working in this repository. Claud
 
 sci-brain is a skill-based plugin for AI coding assistants (Claude Code, Codex, OpenCode, pi) that provides structured literature, ideation, writing, review, and autonomous-research workflows. It is not a traditional application — its main product is the set of `SKILL.md` interaction protocols and their supporting scripts and references.
 
-## Skills
+## Working on skills
 
-The 16 skills in `skills/` are each defined by a `SKILL.md` with YAML frontmatter and instructions. Each description is one sentence starting with its trigger kind, mirroring the `qude-software-skills` convention:
+The 16 skills in `skills/` are listed with their trigger descriptions in
+[README.md](README.md). Load only the entry point and supporting resources needed
+for the current task. Public names match their containing directory.
 
-- `Agentic trigger. Use when …` — `how-to-*` skills the agent invokes automatically while serving a need (a user may still type them).
-- `User trigger. Use when …` — skills a user invokes by need (everything else).
+- User entry points: **brainstorm-ideas**, **survey**, **write-paper**,
+  **review-paper**, **write-slides**, **autoresearch**, **know-me-better**,
+  **create-advisor**, **dump-chat-history**.
+- Supporting workflows: **how-to-build-kb**, **how-to-download-ref**,
+  **how-to-analyze-dialog**, **how-to-flow**, **how-to-review-figure**,
+  **how-to-technical-writing**, **how-to-write-ideas-report**.
 
-`scripts/validate_skills.py` enforces the prefix; `tests/test_repository_consistency.py` enforces `how-to-*` ⇔ agentic and keeps the README tables identical to the descriptions.
+Each description is one sentence starting with `User trigger. Use when …` or,
+for `how-to-*` skills, `Agentic trigger. Use when …`. `scripts/validate_skills.py`
+enforces the prefix; `tests/test_repository_consistency.py` enforces `how-to-*` ⇔
+agentic and keeps the README tables identical to the descriptions. When moving
+resources, keep public names stable and each skill installable on its own.
 
-**User trigger:**
+**autoresearch** routes topics → db → validator → run from `research/STATE.md`.
+Its user-confirmed acceptance gates, attempt budgets, and sealed holdout are
+research invariants. Status questions are read-only.
 
-- **dump-chat-history** — Selects harnesses and a start date before reading history, preserves original prompts and answers with provenance, and exports JSON/Markdown plus an optional topic-titled Typst/PDF field note; research classification belongs to `how-to-analyze-dialog`.
+**brainstorm-ideas** keeps the main mentor and a selected advisor as separate
+roles. Advisor profiles and literature live in `advisors/<slug>/`; the selected
+advisor uses a subagent. No advisor is required for ordinary brainstorming.
 
-- **brainstorm-ideas** — The main ideation entry point. Socratic research mentor that understands user background, finds attackable problems, and encourages deeper thinking. When an advisor is selected, it launches that advisor as a subagent and loads literature from `advisors/<slug>/.knowledge/`. At Phase 3 wrap-up (or on a past session log) it hands off to `how-to-write-ideas-report`.
-- **survey** — Parallel literature search via 7 strategies; the user picks directions, then `how-to-build-kb` populates `<project>/.knowledge/`. It also owns the report mode that produces a grounded technology/field assessment from a populated KB; `how-to-download-ref` fetches and renders full text between discovery and writing.
-- **write-paper** — Use when drafting or revising an actual scientific manuscript. Encodes the von Delft / Martinis workflow: figures first → telegram outline → body → polish abstract+intro+conclusions last. Distinct from the upstream ideas report in `brainstorm-ideas` — this skill requires real results.
-- **review-paper** — The *review/enhance an existing manuscript* counterpart to `write-paper`'s *drafting*. Reads the whole paper, emits location-anchored comments against seven writing guidelines (one-concept sentences, define-before-use, one-job paragraphs, DRY, display-math discipline, figure integration) plus reference & fact verification (CrossRef → Semantic Scholar → MCP → web fetch, repairs via `how-to-download-ref`) and a journal-fit pass (target venue decided or recommended, official author guidelines fetched, limits and required statements measured, writing reviewed against the journal's own guidance). Comment-first and non-destructive: applies only approved edits, then re-runs the compile-check. Distinct from `survey` report mode, which assesses a field rather than a manuscript.
-- **write-slides** — Builds PDF decks for scientific talks, lectures, and briefings using [GiggleLiu/sci-brain-slides](https://github.com/GiggleLiu/sci-brain-slides), pinned to v0.1.0. The upstream repository owns templates, layouts, themes, and style documentation; this skill guides outline approval, package setup, composition, compilation, and figure review. It uses the published Typst package `@preview/sci-brain-slides:0.1.0` directly, without a local template checkout.
-- **autoresearch** — The autoresearch pipeline, one skill with four stage files under `references/stages/`. Reads `research/STATE.md`, verifies stage gate artifacts, and follows the current stage: **topics** (brainstorms topics scored on Checkable/Cheap/Headroom/Publishable; user picks; primary/guard score metrics with gaming risks; red-teamed, user-confirmed acceptance gate per topic → `topics.md`), **db** (insight-coverage-driven reference downloads via `how-to-download-ref`, distillation into user-selected `research/INSIGHTS.md`, domain database, pinned reference implementations, `research/CATALOG.md`; owns the survey gate), **validator** (publishable bar in `GOAL.md`, user-confirmed validation method, sealed gitignored holdout, Docker-canonical `validate` CLI with rich JSON errors, negative-control strictness self-test; owns the validator gate), and **run** (the loop: attempts in worktrees with `LOG.md`, validator-scored under a hard time limit; the user chooses a recommended cycle size during initial setup, while the agent may adjust each actual cycle by need within the authorized attempt budget; every draft hypothesis must state a *mechanism* against the gap to the bar and its *prior art*, ranked on expected gap closure with cost as a constraint, filtered for novelty and triviality; when stuck it refreshes insights via `survey` into `## Candidate`; each cycle report plots every scored attempt's raw primary score with no cumulative headline KPIs, the index and campaign retain cross-cycle summaries, and each reflection thinks through 4–6 candidates before ranking the best 2–4 evidence-grounded next directions with explicit reasons and a top recommendation; the first plan of each authorization is user-confirmed; each soft gate asks which direction and how many attempts to authorize). Each attempt commits code + `LOG.md` + `report.json` on its `attempt-NNN` branch; a cycle-end sync pushes those branches plus main.
-- **know-me-better** — Lets the agent learn the user's research style so it speaks their language; the mechanism is indexing a paper collection (Zotero / PDF folder / Google Scholar) into the active KB. Default target is `<project>/.knowledge/`; when invoked from `/create-advisor` targets `advisors/<slug>/.knowledge/`. Writes `.raw/` JSON, delegates `references.bib` writes via `how-to-download-ref` helpers.
-- **create-advisor** — Creates or updates a named advisor from JSONL histories or imported Markdown dialogs. It classifies conversations, extracts recurring trigger→reaction patterns, confirms logic jumps with the user, and synthesizes `advisors/<slug>/profile.md`; it can also stop after analysis-only artifacts. The advisor's literature cache lives at `advisors/<slug>/.knowledge/`.
-
-**Agentic trigger (`how-to-*`):**
-
-- **how-to-build-kb** — Turns a list of picked papers (from `survey`, `know-me-better`, or `autoresearch`) into verified KB entries: Semantic Scholar / CrossRef lookup, `.raw/` JSON, `references.bib` append via `append_bibtex.py`, `INDEX.md` regeneration, and `NOTES.md` (landscape, open problems, bottlenecks). Non-interactive; never generates BibTeX from memory.
-- **how-to-write-ideas-report** — Writes the proposal-style ideas report (research question, novelty, MVE, success/hope/pivot signals, risks, venue, verified references) from a finished `brainstorm-ideas` log. Follows `skills/how-to-write-ideas-report/references/writing-workflow.md`.
-- **how-to-technical-writing** — The shared writing style guide: sentence- and paragraph-level rules (one concept per sentence, direct to the point with key information early, simple words with every technical word kept, no undrawn metaphors, locality, say it once), a hunt-for/fix table for language passes, a `checklist.md` with the rules as checkable items, and guardrails naming which fixes are comment-only because they change content. `write-paper` drafts by it, `review-paper` reviews by it, and `how-to-write-ideas-report` / `survey` report mode follow it for report prose. Notation and figure rules stay in `write-paper`.
-- **how-to-review-figure** — Reviews the *visual design quality* of a figure, plot, or diagram and prints a scorecard. Source-aware (renders the figure to a raster to look at it via `helpers/render.py`, reads matplotlib/Typst/SVG source so fixes can cite a line), report-only, terminal-first. Scores against an 18-rule rubric (11 general — alignment, proximity, color, hierarchy, contrast, colorblind-safety, …; plus 7 scientific-plot rules — text size, line weight, space use, chartjunk, legend, cross-panel consistency, resolution). Distinct from `review-paper` (which checks whether a figure is cited/discussed in the text, not how it looks) and `write-paper` (which authors figures). Full rubric in `skills/how-to-review-figure/checklist.md`.
-- **how-to-flow** — Autonomous deep-thinker that conquers one hard goal via a CDCL/DPLL-style search loop: a **preflight gate** (is the goal testable? are all context/KB facts loaded?), then iterate *decide* (**what-if**: assume a condition, test "closer to goal?" + "easier to achieve?") → *propagate* (**simulate**: run consequences forward, reflect; may fan out 2–3 subagents on wide forks) → *learn* (note a reusable clause after **every** trial) → *backjump* (non-chronological, to the real cause) → *pivot* (meta-restart: re-aim to an equally-valuable easier goal when stuck, keeping all notes). Domain-agnostic and KB-optional. Writes a per-trial journal to `docs/flow/<goal-slug>.md` (template in `skills/how-to-flow/journal-template.md`). Terminates SOLVED / PIVOTED-SOLVED / EXHAUSTED (≤3 pivots). Distinct from `brainstorm-ideas` (open-ended, collaborative) — `how-to-flow` is goal-locked and autonomous.
-- **how-to-download-ref** — Adds one or many new arXiv IDs / DOIs to a knowledge base (`<project>/.knowledge/` by default; `advisors/<slug>/.knowledge/` when invoked from advisor flows). Fetches Semantic Scholar metadata, downloads PDFs (with SciHub fallback); when the user opts in, also fetches arXiv LaTeX sources and renders those refs (incl. DOI entries with an arXiv preprint) from flattened LaTeX (`full_text: latex`) via `--tex-source`, otherwise all refs render via `pymupdf4llm`. Regenerates `INDEX.md`, appends to the KB's `references.bib`. Supports `--from-bib` for bulk operations on an existing BibTeX.
-- **how-to-analyze-dialog** — Consumes exported dialog from `dump-chat-history`, classifies topics and user messages across 6 academic dimensions, and writes derived tagged reports to `docs/dialog/analysis/` for `create-advisor`; original exports remain unchanged.
-
-The directory name must match the skill's frontmatter `name`. In particular, the public `know-me-better` skill lives at `skills/know-me-better/`.
-
-## Architecture
-
-**Entry point:** `/brainstorm-ideas` — most users only need this. Other skills are auto-called or can run independently.
-
-**brainstorm-ideas skill uses a primary Socratic mentor plus an optional advisor subagent:**
-- Understands user background (self-intro, Zotero, or Google Scholar)
-- Loads project literature from `<project>/.knowledge/INDEX.md` + `NOTES.md`
-- When an advisor is selected, also loads `advisors/<slug>/.knowledge/INDEX.md` + `NOTES.md` and pre-fetches representative papers into the advisor subagent context. The advisor subagent is launched with file search/read over its `.knowledge/` KB plus web search/fetch, and is instructed to consult its KB and the web before making comments (grounding each comment in a cited source or marking it as opinion)
-- Six principles: clarify motivation, encourage thinking (humbly), flag uncertainty, surface related facts, empower based on skills, inspire with deep theory
-- Phases: Get to Know You → Find Good Problems → Dive Into the Topic → Wrap Up
+## Shared data contracts
 
 **Knowledge base layout** (used by every skill that touches papers):
 
@@ -86,36 +70,11 @@ Each SKILL.md defines installed-resource resolution. Run helpers by their absolu
 
 **BibTeX lookup chain** (never from memory): CrossRef API → Semantic Scholar API → MCP servers → web fetch fallback
 
-## Migrating from the pre-0.3 `<registry-root>/<slug>/` layout
-
-Old sci-brain (≤ 0.2.x) stored surveys under `~/.claude/survey/<topic>/` (or `.codex/survey/`, `.config/opencode/survey/`, `.claude/survey/`) with `summary.md` + `references.bib` per topic. 0.3 moves to one `<project>/.knowledge/` per project (plus per-advisor caches). Migrate by hand:
-
-```sh
-# Pick your project root (where you want .knowledge/ to live):
-PROJ=/path/to/your/project
-mkdir -p "$PROJ/.knowledge"
-
-# Move a single old registry into the project KB:
-OLD=~/.claude/survey/topological-orders     # adapt path
-mv "$OLD/references.bib" "$PROJ/.knowledge/references.bib"   # or merge into existing references.bib
-mv "$OLD/summary.md"     "$PROJ/.knowledge/NOTES.md"
-mv "$OLD"/*.md           "$PROJ/.knowledge/"   2>/dev/null  # rendered papers
-mv "$OLD/.raw"           "$PROJ/.knowledge/.raw"
-mv "$OLD/.figures"       "$PROJ/.knowledge/.figures"
-
-# Regenerate INDEX.md (use a stable title — re-runs must use the same string):
-python3 skills/how-to-download-ref/helpers/index.py \
-  --kb "$PROJ/.knowledge" \
-  --title "topological-orders — references" \
-  --source-note "Migrated from ~/.claude/survey/topological-orders on $(date -u +%Y-%m-%d)."
-
-# Remove the old registry:
-rmdir "$OLD"
-```
-
-For advisor caches built by the abandoned 0.2-era `publications.yml` flow: that layout was never populated; nothing to migrate. The new flow builds `advisors/<slug>/.knowledge/` via `/know-me-better` or `/how-to-download-ref` invoked from `/create-advisor`.
+## Legacy KB migration
 
-Multiple old registries can be merged into one project KB (run the `mv` block per topic; `references.bib` accepts appends; `NOTES.md` accepts merges as separate top-level headings).
+For a requested migration from the pre-0.3 registry layout, read
+[docs/kb-migration.md](docs/kb-migration.md). Ordinary skill use does not require
+that migration guide.
 
 ## Installation
 

README.md

@@ -118,7 +118,7 @@ The whole process is interactive — you review everything before it's published
 | `/survey` (KB-building step) | `/how-to-build-kb`, invoked by `/survey` |
 | `/brainstorm-ideas` (report mode) | `/how-to-write-ideas-report`, invoked by `/brainstorm-ideas` |
 
-> ⚠️ **Breaking change in v0.3.** Knowledge bases moved from per-topic registries (`~/.claude/survey/<topic>/` with `summary.md` + `references.bib`) to one `<project>/.knowledge/` per project, with `references.bib` living *inside* the KB. The `fetch-papers` skill was folded into `download-ref --from-bib`. See [`CLAUDE.md`](./CLAUDE.md) § "Migrating from the pre-0.3 layout" for `mv` commands.
+> ⚠️ **Breaking change in v0.3.** Knowledge bases moved from per-topic registries (`~/.claude/survey/<topic>/` with `summary.md` + `references.bib`) to one `<project>/.knowledge/` per project, with `references.bib` living *inside* the KB. The `fetch-papers` skill was folded into `download-ref --from-bib`. See [`docs/kb-migration.md`](./docs/kb-migration.md) for `mv` commands.
 
 Four internal stages became modes of their goal-level skill in v0.3:
 

docs/kb-migration.md

@@ -0,0 +1,30 @@
+# Migrating from the pre-0.3 `<registry-root>/<slug>/` layout
+
+Old sci-brain (≤ 0.2.x) stored surveys under `~/.claude/survey/<topic>/` (or `.codex/survey/`, `.config/opencode/survey/`, `.claude/survey/`) with `summary.md` + `references.bib` per topic. 0.3 moves to one `<project>/.knowledge/` per project (plus per-advisor caches). Migrate by hand:
+
+```sh
+# Pick your project root (where you want .knowledge/ to live):
+PROJ=/path/to/your/project
+mkdir -p "$PROJ/.knowledge"
+
+# Move a single old registry into the project KB:
+OLD=~/.claude/survey/topological-orders     # adapt path
+mv "$OLD/references.bib" "$PROJ/.knowledge/references.bib"   # or merge into existing references.bib
+mv "$OLD/summary.md"     "$PROJ/.knowledge/NOTES.md"
+mv "$OLD"/*.md           "$PROJ/.knowledge/"   2>/dev/null  # rendered papers
+mv "$OLD/.raw"           "$PROJ/.knowledge/.raw"
+mv "$OLD/.figures"       "$PROJ/.knowledge/.figures"
+
+# Regenerate INDEX.md (use a stable title — re-runs must use the same string):
+python3 skills/how-to-download-ref/helpers/index.py \
+  --kb "$PROJ/.knowledge" \
+  --title "topological-orders — references" \
+  --source-note "Migrated from ~/.claude/survey/topological-orders on $(date -u +%Y-%m-%d)."
+
+# Remove the old registry:
+rmdir "$OLD"
+```
+
+For advisor caches built by the abandoned 0.2-era `publications.yml` flow: that layout was never populated; nothing to migrate. The new flow builds `advisors/<slug>/.knowledge/` via `/know-me-better` or `/how-to-download-ref` invoked from `/create-advisor`.
+
+Multiple old registries can be merged into one project KB (run the `mv` block per topic; `references.bib` accepts appends; `NOTES.md` accepts merges as separate top-level headings).