From 7990661bcc82b270469dd3d370646517a1f75657 Mon Sep 17 00:00:00 2001 From: nzy1997 Date: Mon, 21 Sep 2026 09:30:03 +0800 Subject: [PATCH 1/2] Disclose repository guidance progressively --- CLAUDE.md | 92 ++++++++++++-------------------------------- docs/kb-migration.md | 30 +++++++++++++++ 2 files changed, 55 insertions(+), 67 deletions(-) create mode 100644 docs/kb-migration.md diff --git a/CLAUDE.md b/CLAUDE.md index c3308be..45a4843 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,50 +6,33 @@ This is the canonical project guide for agents working in this repository. Claud sci-brain is a skill-based plugin for AI coding assistants (Claude Code, Codex, OpenCode, pi) that provides structured literature, ideation, writing, review, and autonomous-research workflows. It is not a traditional application — its main product is the set of `SKILL.md` interaction protocols and their supporting scripts and references. -## Skills +## Working on skills -The 16 skills in `skills/` are each defined by a `SKILL.md` with YAML frontmatter and instructions. Each description is one sentence starting with its trigger kind, mirroring the `qude-software-skills` convention: +The 16 skills in `skills/` are listed with their trigger descriptions in +[README.md](README.md). Load only the entry point and supporting resources needed +for the current task. Public names match their containing directory. -- `Agentic trigger. Use when …` — `how-to-*` skills the agent invokes automatically while serving a need (a user may still type them). -- `User trigger. Use when …` — skills a user invokes by need (everything else). +- User entry points: **brainstorm-ideas**, **survey**, **write-paper**, + **review-paper**, **write-slides**, **autoresearch**, **know-me-better**, + **create-advisor**, **dump-chat-history**. +- Supporting workflows: **how-to-build-kb**, **how-to-download-ref**, + **how-to-analyze-dialog**, **how-to-flow**, **how-to-review-figure**, + **how-to-technical-writing**, **how-to-write-ideas-report**. -`scripts/validate_skills.py` enforces the prefix; `tests/test_repository_consistency.py` enforces `how-to-*` ⇔ agentic and keeps the README tables identical to the descriptions. +Descriptions start with `User trigger. Use when …` or, for `how-to-*`, +`Agentic trigger. Use when …`. The validator enforces this prefix and tests keep +the README descriptions aligned. Preserve public names and independent skill +installation when reorganizing resources. -**User trigger:** +**autoresearch** routes topics → db → validator → run from `research/STATE.md`. +Its user-confirmed acceptance gates, attempt budgets, and sealed holdout are +research invariants. Status questions are read-only. -- **dump-chat-history** — Selects harnesses and a start date before reading history, preserves original prompts and answers with provenance, and exports JSON/Markdown plus an optional topic-titled Typst/PDF field note; research classification belongs to `how-to-analyze-dialog`. +**brainstorm-ideas** keeps the main mentor and a selected advisor as separate +roles. Advisor profiles and literature live in `advisors//`; the selected +advisor uses a subagent. No advisor is required for ordinary brainstorming. -- **brainstorm-ideas** — The main ideation entry point. Socratic research mentor that understands user background, finds attackable problems, and encourages deeper thinking. When an advisor is selected, it launches that advisor as a subagent and loads literature from `advisors//.knowledge/`. At Phase 3 wrap-up (or on a past session log) it hands off to `how-to-write-ideas-report`. -- **survey** — Parallel literature search via 7 strategies; the user picks directions, then `how-to-build-kb` populates `/.knowledge/`. It also owns the report mode that produces a grounded technology/field assessment from a populated KB; `how-to-download-ref` fetches and renders full text between discovery and writing. -- **write-paper** — Use when drafting or revising an actual scientific manuscript. Encodes the von Delft / Martinis workflow: figures first → telegram outline → body → polish abstract+intro+conclusions last. Distinct from the upstream ideas report in `brainstorm-ideas` — this skill requires real results. -- **review-paper** — The *review/enhance an existing manuscript* counterpart to `write-paper`'s *drafting*. Reads the whole paper, emits location-anchored comments against seven writing guidelines (one-concept sentences, define-before-use, one-job paragraphs, DRY, display-math discipline, figure integration) plus reference & fact verification (CrossRef → Semantic Scholar → MCP → web fetch, repairs via `how-to-download-ref`) and a journal-fit pass (target venue decided or recommended, official author guidelines fetched, limits and required statements measured, writing reviewed against the journal's own guidance). Comment-first and non-destructive: applies only approved edits, then re-runs the compile-check. Distinct from `survey` report mode, which assesses a field rather than a manuscript. -- **write-slides** — Builds PDF decks for scientific talks, lectures, and briefings using [GiggleLiu/sci-brain-slides](https://github.com/GiggleLiu/sci-brain-slides), pinned to v0.1.0. The upstream repository owns templates, layouts, themes, and style documentation; this skill guides outline approval, package setup, composition, compilation, and figure review. It uses the published Typst package `@preview/sci-brain-slides:0.1.0` directly, without a local template checkout. -- **autoresearch** — The autoresearch pipeline, one skill with four stage files under `references/stages/`. Reads `research/STATE.md`, verifies stage gate artifacts, and follows the current stage: **topics** (brainstorms topics scored on Checkable/Cheap/Headroom/Publishable; user picks; primary/guard score metrics with gaming risks; red-teamed, user-confirmed acceptance gate per topic → `topics.md`), **db** (insight-coverage-driven reference downloads via `how-to-download-ref`, distillation into user-selected `research/INSIGHTS.md`, domain database, pinned reference implementations, `research/CATALOG.md`; owns the survey gate), **validator** (publishable bar in `GOAL.md`, user-confirmed validation method, sealed gitignored holdout, Docker-canonical `validate` CLI with rich JSON errors, negative-control strictness self-test; owns the validator gate), and **run** (the loop: attempts in worktrees with `LOG.md`, validator-scored under a hard time limit; the user chooses a recommended cycle size during initial setup, while the agent may adjust each actual cycle by need within the authorized attempt budget; every draft hypothesis must state a *mechanism* against the gap to the bar and its *prior art*, ranked on expected gap closure with cost as a constraint, filtered for novelty and triviality; when stuck it refreshes insights via `survey` into `## Candidate`; each cycle report plots every scored attempt's raw primary score with no cumulative headline KPIs, the index and campaign retain cross-cycle summaries, and each reflection thinks through 4–6 candidates before ranking the best 2–4 evidence-grounded next directions with explicit reasons and a top recommendation; the first plan of each authorization is user-confirmed; each soft gate asks which direction and how many attempts to authorize). Each attempt commits code + `LOG.md` + `report.json` on its `attempt-NNN` branch; a cycle-end sync pushes those branches plus main. -- **know-me-better** — Lets the agent learn the user's research style so it speaks their language; the mechanism is indexing a paper collection (Zotero / PDF folder / Google Scholar) into the active KB. Default target is `/.knowledge/`; when invoked from `/create-advisor` targets `advisors//.knowledge/`. Writes `.raw/` JSON, delegates `references.bib` writes via `how-to-download-ref` helpers. -- **create-advisor** — Creates or updates a named advisor from JSONL histories or imported Markdown dialogs. It classifies conversations, extracts recurring trigger→reaction patterns, confirms logic jumps with the user, and synthesizes `advisors//profile.md`; it can also stop after analysis-only artifacts. The advisor's literature cache lives at `advisors//.knowledge/`. - -**Agentic trigger (`how-to-*`):** - -- **how-to-build-kb** — Turns a list of picked papers (from `survey`, `know-me-better`, or `autoresearch`) into verified KB entries: Semantic Scholar / CrossRef lookup, `.raw/` JSON, `references.bib` append via `append_bibtex.py`, `INDEX.md` regeneration, and `NOTES.md` (landscape, open problems, bottlenecks). Non-interactive; never generates BibTeX from memory. -- **how-to-write-ideas-report** — Writes the proposal-style ideas report (research question, novelty, MVE, success/hope/pivot signals, risks, venue, verified references) from a finished `brainstorm-ideas` log. Follows `skills/how-to-write-ideas-report/references/writing-workflow.md`. -- **how-to-technical-writing** — The shared writing style guide: sentence- and paragraph-level rules (one concept per sentence, direct to the point with key information early, simple words with every technical word kept, no undrawn metaphors, locality, say it once), a hunt-for/fix table for language passes, a `checklist.md` with the rules as checkable items, and guardrails naming which fixes are comment-only because they change content. `write-paper` drafts by it, `review-paper` reviews by it, and `how-to-write-ideas-report` / `survey` report mode follow it for report prose. Notation and figure rules stay in `write-paper`. -- **how-to-review-figure** — Reviews the *visual design quality* of a figure, plot, or diagram and prints a scorecard. Source-aware (renders the figure to a raster to look at it via `helpers/render.py`, reads matplotlib/Typst/SVG source so fixes can cite a line), report-only, terminal-first. Scores against an 18-rule rubric (11 general — alignment, proximity, color, hierarchy, contrast, colorblind-safety, …; plus 7 scientific-plot rules — text size, line weight, space use, chartjunk, legend, cross-panel consistency, resolution). Distinct from `review-paper` (which checks whether a figure is cited/discussed in the text, not how it looks) and `write-paper` (which authors figures). Full rubric in `skills/how-to-review-figure/checklist.md`. -- **how-to-flow** — Autonomous deep-thinker that conquers one hard goal via a CDCL/DPLL-style search loop: a **preflight gate** (is the goal testable? are all context/KB facts loaded?), then iterate *decide* (**what-if**: assume a condition, test "closer to goal?" + "easier to achieve?") → *propagate* (**simulate**: run consequences forward, reflect; may fan out 2–3 subagents on wide forks) → *learn* (note a reusable clause after **every** trial) → *backjump* (non-chronological, to the real cause) → *pivot* (meta-restart: re-aim to an equally-valuable easier goal when stuck, keeping all notes). Domain-agnostic and KB-optional. Writes a per-trial journal to `docs/flow/.md` (template in `skills/how-to-flow/journal-template.md`). Terminates SOLVED / PIVOTED-SOLVED / EXHAUSTED (≤3 pivots). Distinct from `brainstorm-ideas` (open-ended, collaborative) — `how-to-flow` is goal-locked and autonomous. -- **how-to-download-ref** — Adds one or many new arXiv IDs / DOIs to a knowledge base (`/.knowledge/` by default; `advisors//.knowledge/` when invoked from advisor flows). Fetches Semantic Scholar metadata, downloads PDFs (with SciHub fallback); when the user opts in, also fetches arXiv LaTeX sources and renders those refs (incl. DOI entries with an arXiv preprint) from flattened LaTeX (`full_text: latex`) via `--tex-source`, otherwise all refs render via `pymupdf4llm`. Regenerates `INDEX.md`, appends to the KB's `references.bib`. Supports `--from-bib` for bulk operations on an existing BibTeX. -- **how-to-analyze-dialog** — Consumes exported dialog from `dump-chat-history`, classifies topics and user messages across 6 academic dimensions, and writes derived tagged reports to `docs/dialog/analysis/` for `create-advisor`; original exports remain unchanged. - -The directory name must match the skill's frontmatter `name`. In particular, the public `know-me-better` skill lives at `skills/know-me-better/`. - -## Architecture - -**Entry point:** `/brainstorm-ideas` — most users only need this. Other skills are auto-called or can run independently. - -**brainstorm-ideas skill uses a primary Socratic mentor plus an optional advisor subagent:** -- Understands user background (self-intro, Zotero, or Google Scholar) -- Loads project literature from `/.knowledge/INDEX.md` + `NOTES.md` -- When an advisor is selected, also loads `advisors//.knowledge/INDEX.md` + `NOTES.md` and pre-fetches representative papers into the advisor subagent context. The advisor subagent is launched with file search/read over its `.knowledge/` KB plus web search/fetch, and is instructed to consult its KB and the web before making comments (grounding each comment in a cited source or marking it as opinion) -- Six principles: clarify motivation, encourage thinking (humbly), flag uncertainty, surface related facts, empower based on skills, inspire with deep theory -- Phases: Get to Know You → Find Good Problems → Dive Into the Topic → Wrap Up +## Shared data contracts **Knowledge base layout** (used by every skill that touches papers): @@ -86,36 +69,11 @@ Each SKILL.md defines installed-resource resolution. Run helpers by their absolu **BibTeX lookup chain** (never from memory): CrossRef API → Semantic Scholar API → MCP servers → web fetch fallback -## Migrating from the pre-0.3 `//` layout - -Old sci-brain (≤ 0.2.x) stored surveys under `~/.claude/survey//` (or `.codex/survey/`, `.config/opencode/survey/`, `.claude/survey/`) with `summary.md` + `references.bib` per topic. 0.3 moves to one `/.knowledge/` per project (plus per-advisor caches). Migrate by hand: - -```sh -# Pick your project root (where you want .knowledge/ to live): -PROJ=/path/to/your/project -mkdir -p "$PROJ/.knowledge" - -# Move a single old registry into the project KB: -OLD=~/.claude/survey/topological-orders # adapt path -mv "$OLD/references.bib" "$PROJ/.knowledge/references.bib" # or merge into existing references.bib -mv "$OLD/summary.md" "$PROJ/.knowledge/NOTES.md" -mv "$OLD"/*.md "$PROJ/.knowledge/" 2>/dev/null # rendered papers -mv "$OLD/.raw" "$PROJ/.knowledge/.raw" -mv "$OLD/.figures" "$PROJ/.knowledge/.figures" - -# Regenerate INDEX.md (use a stable title — re-runs must use the same string): -python3 skills/how-to-download-ref/helpers/index.py \ - --kb "$PROJ/.knowledge" \ - --title "topological-orders — references" \ - --source-note "Migrated from ~/.claude/survey/topological-orders on $(date -u +%Y-%m-%d)." - -# Remove the old registry: -rmdir "$OLD" -``` - -For advisor caches built by the abandoned 0.2-era `publications.yml` flow: that layout was never populated; nothing to migrate. The new flow builds `advisors//.knowledge/` via `/know-me-better` or `/how-to-download-ref` invoked from `/create-advisor`. +## Legacy KB migration -Multiple old registries can be merged into one project KB (run the `mv` block per topic; `references.bib` accepts appends; `NOTES.md` accepts merges as separate top-level headings). +For a requested migration from the pre-0.3 registry layout, read +[docs/kb-migration.md](docs/kb-migration.md). Ordinary skill use does not require +that migration guide. ## Installation diff --git a/docs/kb-migration.md b/docs/kb-migration.md new file mode 100644 index 0000000..f323275 --- /dev/null +++ b/docs/kb-migration.md @@ -0,0 +1,30 @@ +# Migrating from the pre-0.3 `//` layout + +Old sci-brain (≤ 0.2.x) stored surveys under `~/.claude/survey//` (or `.codex/survey/`, `.config/opencode/survey/`, `.claude/survey/`) with `summary.md` + `references.bib` per topic. 0.3 moves to one `/.knowledge/` per project (plus per-advisor caches). Migrate by hand: + +```sh +# Pick your project root (where you want .knowledge/ to live): +PROJ=/path/to/your/project +mkdir -p "$PROJ/.knowledge" + +# Move a single old registry into the project KB: +OLD=~/.claude/survey/topological-orders # adapt path +mv "$OLD/references.bib" "$PROJ/.knowledge/references.bib" # or merge into existing references.bib +mv "$OLD/summary.md" "$PROJ/.knowledge/NOTES.md" +mv "$OLD"/*.md "$PROJ/.knowledge/" 2>/dev/null # rendered papers +mv "$OLD/.raw" "$PROJ/.knowledge/.raw" +mv "$OLD/.figures" "$PROJ/.knowledge/.figures" + +# Regenerate INDEX.md (use a stable title — re-runs must use the same string): +python3 skills/how-to-download-ref/helpers/index.py \ + --kb "$PROJ/.knowledge" \ + --title "topological-orders — references" \ + --source-note "Migrated from ~/.claude/survey/topological-orders on $(date -u +%Y-%m-%d)." + +# Remove the old registry: +rmdir "$OLD" +``` + +For advisor caches built by the abandoned 0.2-era `publications.yml` flow: that layout was never populated; nothing to migrate. The new flow builds `advisors//.knowledge/` via `/know-me-better` or `/how-to-download-ref` invoked from `/create-advisor`. + +Multiple old registries can be merged into one project KB (run the `mv` block per topic; `references.bib` accepts appends; `NOTES.md` accepts merges as separate top-level headings). From 8bdadc044b7fd77fee17f56513e6d331ed1fcf93 Mon Sep 17 00:00:00 2001 From: GiggleLiu Date: Wed, 23 Sep 2026 14:10:28 +0800 Subject: [PATCH 2/2] Polish repository guidance PR Keep the concrete validator/test file names for the description convention, and repoint the README migration link at docs/kb-migration.md. Co-Authored-By: Claude Fable 5.1 --- CLAUDE.md | 9 +++++---- README.md | 2 +- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 45a4843..26ee808 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -19,10 +19,11 @@ for the current task. Public names match their containing directory. **how-to-analyze-dialog**, **how-to-flow**, **how-to-review-figure**, **how-to-technical-writing**, **how-to-write-ideas-report**. -Descriptions start with `User trigger. Use when …` or, for `how-to-*`, -`Agentic trigger. Use when …`. The validator enforces this prefix and tests keep -the README descriptions aligned. Preserve public names and independent skill -installation when reorganizing resources. +Each description is one sentence starting with `User trigger. Use when …` or, +for `how-to-*` skills, `Agentic trigger. Use when …`. `scripts/validate_skills.py` +enforces the prefix; `tests/test_repository_consistency.py` enforces `how-to-*` ⇔ +agentic and keeps the README tables identical to the descriptions. When moving +resources, keep public names stable and each skill installable on its own. **autoresearch** routes topics → db → validator → run from `research/STATE.md`. Its user-confirmed acceptance gates, attempt budgets, and sealed holdout are diff --git a/README.md b/README.md index 87f5eb6..a970eeb 100644 --- a/README.md +++ b/README.md @@ -118,7 +118,7 @@ The whole process is interactive — you review everything before it's published | `/survey` (KB-building step) | `/how-to-build-kb`, invoked by `/survey` | | `/brainstorm-ideas` (report mode) | `/how-to-write-ideas-report`, invoked by `/brainstorm-ideas` | -> ⚠️ **Breaking change in v0.3.** Knowledge bases moved from per-topic registries (`~/.claude/survey//` with `summary.md` + `references.bib`) to one `/.knowledge/` per project, with `references.bib` living *inside* the KB. The `fetch-papers` skill was folded into `download-ref --from-bib`. See [`CLAUDE.md`](./CLAUDE.md) § "Migrating from the pre-0.3 layout" for `mv` commands. +> ⚠️ **Breaking change in v0.3.** Knowledge bases moved from per-topic registries (`~/.claude/survey//` with `summary.md` + `references.bib`) to one `/.knowledge/` per project, with `references.bib` living *inside* the KB. The `fetch-papers` skill was folded into `download-ref --from-bib`. See [`docs/kb-migration.md`](./docs/kb-migration.md) for `mv` commands. Four internal stages became modes of their goal-level skill in v0.3: