Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,11 @@ Module `github.com/mozilla/markfluence` (`go 1.25`). `main.go` is a shim to `cmd

### Layout

- `cmd/root.go` — the cobra root: `--url`/`--username`/`--debug`/`--no-color` persistent flags, version from `internal/buildinfo`, and registration of the four subcommands. `Execute()` prints cobra-generated errors (bad args/flags) but not `ui.ErrSilent`, which marks a failure a command already reported.
- `cmd/{update,create,fix,info}/` — one package per command (each exports `Cmd`), orchestrating the `internal` packages and `internal/ui` output. `create` is two-phase and transactional (validate all, then create parents-first in topological order); `fix` is read-only on the server.
- `internal/client` — `ConfluenceClient` over `net/http` with basic auth. Built from a `Config` (site URL, cloud ID, username, token) via `New`; it carries **two bases**: `BaseURL()` is where requests go (the gateway when a cloud ID is set) and `SiteURL()` is always the site. Anything a reader sees uses `SiteURL()` — printed page URLs and, critically, the `baseURL` handed to `convert.MdToConfluence`, since rewritten links are published *into* the page. Pages are Confluence **v2**; attachment writes and the user lookup are **v1** (`/wiki/rest/api/...`). Typed `HTTPError`, per-attempt context timeouts, centralized retry/backoff in `send` (429 for any method honoring `Retry-After`, plus 502/503/504 and network errors for idempotent methods only; exponential backoff capped), `SetContentProperty` retry-once on top (recovers a lost create-POST response), `SyncAttachments` (skip/update by a SHA-256 recorded in the attachment's comment, alongside the source path so `read` recovers image paths exactly; the comment is `markfluence: sha256=… path=…`, and the legacy `mzcld:checksum: …` form is still parsed — the skip test compares the *parsed* checksum, so the format change doesn't force a re-upload), `_links.next` pagination. `config.go` holds `Resolve` and the `.env` reader.
- `cmd/root.go` — the cobra root: `--url`/`--username`/`--debug`/`--no-color` persistent flags, version from `internal/buildinfo`, and registration of every subcommand. `Execute()` prints cobra-generated errors (bad args/flags) but not `ui.ErrSilent`, which marks a failure a command already reported.
- `cmd/{update,create,fix,info,read}/` — one package per command (each exports `Cmd`), orchestrating the `internal` packages and `internal/ui` output. `create` is two-phase and transactional (validate all, then create parents-first in topological order); `fix` is read-only on the server.
- `cmd/attachment{list,upload,download}/` — the flat `attachment-list`/`attachment-upload`/`attachment-download` commands (noun-first so cobra's alphabetized help keeps them together and `attachment-<TAB>` completes as a group). `upload` reuses the checksum skip/update logic, with `--force` (`client.ForceUploadAttachments`) and `--dry-run` (`PlanAttachments`); its `--name` takes a *path* and encodes it, and the recorded `path=` is always the decode of the stored name, so a later publish can't create a duplicate under a different name. `download` restores an attachment to its recorded `path=` (never a decode of the stored name — a hand-uploaded `a%2Fb.png` is indistinguishable from a published one), with `--flat` to opt out; `destPath` is the only place server data becomes a filesystem path and clamps to `--dest`, refusing rather than clipping an escape, since `..` is legitimate in a source path.
- `internal/pageref` — `Resolve`, the single page-argument resolver: a numeric id, a Confluence page URL, or a `.md` file whose frontmatter has a `page_id` (stat'd first, so `123.md` is a file). Every command taking a page uses it.
- `internal/client` — `ConfluenceClient` over `net/http` with basic auth. Built from a `Config` (site URL, cloud ID, username, token) via `New`; it carries **two bases**: `BaseURL()` is where requests go (the gateway when a cloud ID is set) and `SiteURL()` is always the site. Anything a reader sees uses `SiteURL()` — printed page URLs and, critically, the `baseURL` handed to `convert.MdToConfluence`, since rewritten links are published *into* the page. Pages are Confluence **v2**; attachment writes and the user lookup are **v1** (`/wiki/rest/api/...`). Typed `HTTPError`, per-attempt context timeouts, centralized retry/backoff in `send` (429 for any method honoring `Retry-After`, plus 502/503/504 and network errors for idempotent methods only; exponential backoff capped), `SetContentProperty` retry-once on top (recovers a lost create-POST response), `SyncAttachments` (skip/update by a SHA-256 recorded in the attachment's comment, alongside the source path so `read` recovers image paths exactly; the comment is `markfluence: sha256=… path=…`, and the legacy `mzcld:checksum: …` form is still parsed — the skip test compares the *parsed* checksum, so the format change doesn't force a re-upload), `_links.next` pagination. `ListAttachments` expands `metadata.comment,version,extensions` (`fileSize`/`mediaType` live under `extensions`) and pages by `start`/`limit` offset instead — a v1 collection omits `_links.next` when the results fit one page, and its `next` is `/wiki`-context-relative rather than the v2 paths `resolveNext` handles. `DownloadAttachment` goes through `send` (so it inherits retry/backoff) against `_links.download`, which is an API path — `/rest/api/content/{page}/child/attachment/{id}/download`, not the `/download/attachments/...` UI path — and therefore works through the gateway; it 302s to Atlassian's media host, which carries its own token, and Go drops `Authorization` cross-host, so **never** add a `CheckRedirect` that forwards headers. `config.go` holds `Resolve` and the `.env` reader.
- `internal/convert` — the converter (the crux). `MdToConfluence(md *frontmatter.MarkdownFile, baseURL, spaceKey, version string) (*ConfluencePage, error)`. It parses with goldmark (GFM) and renders through a custom `storageRenderer` registered at priority 100 (below the default HTML=1000 and table=500 renderers) that emits Confluence storage format. `shield.go` renames raw `ac:`/`ri:` tags to colon-free sentinels around the goldmark step so pasted storage passes through; `callouts.go` is an AST transformer + blockquote renderer for GitHub alerts; `attachname.go` owns the source-path↔attachment-name mapping (percent-encoding `%`→`%25` then `/`→`%2F`, which is **bijective** — that is what makes the dedupe collision-free and lets `read` recover an image's original path; decode refuses an absolute result, which markfluence never produces); `images.go` (resolution stays page-relative like GitHub; the documentation root — cwd — bounds what may be published, and an image above it is `IMAGE BROKEN`), `links.go` (sibling-file scans, GitHub/Confluence slugs, doc-link + anchor rewriting), `tables.go` (the `<table>` tag, stamped with Confluence's `data-layout="align-start"` so tables auto-size to their content and left-align, plus cells: an AST transformer consumes a leading `<!-- bg:COLOR -->` comment in a cell and `renderTableCell` emits it as `data-highlight-colour` — rows still fall through to the GFM renderer), and `renderer.go` (code macros, text soft-break→space, images, links) do the rest. The `<!-- confluence-toc -->` and `<!-- markfluence-version -->` token substitutions happen **inside** `MdToConfluence`.
- `internal/frontmatter` — flat YAML frontmatter parse/quote/`UpdateField`, and the `MarkdownFile` type (`Parse`/`ParseFile`, exported `Filename`/`Content`/`Frontmatter`/`Body`, and `Title`/`PageID`/`Space`/`Parent` accessors that normalize missing/blank/`"null"`).
- `internal/pagewidth` — the `page_width` `Width` enum (`narrow`/`wide`/`max`, default `max`), `Declared`, the vocab↔content-property maps, `WidthFromProperties`, and `Apply`/`Read` against the client.
Expand All @@ -69,4 +71,4 @@ The design target is **semantic** (not byte-for-byte) equivalence to valid Confl

### Frontmatter-driven publishing

Each markdown file is one page. Frontmatter carries `title`, `page_id`, `space` (a key), `parent` (`null` / a `.md` path / a page id), and `page_width`. `update` requires a `page_id` (from frontmatter or `--page-id`) and errors without one; `--title`/`--page-id` override the frontmatter (single FILE only), `--page-width` overrides too (batch allowed), and `update` never writes back to files. It asserts `page_width` only when set via flag or frontmatter (otherwise the live width is left alone), and skips a file whose mtime predates the page's last version unless `--force`. `create` takes `--title`/`--page-width` overrides (`--title` single-FILE only; `--page-width` batch-ok, default `max`) and, unless `--no-persist` is given, writes `title`/`space`/`parent`/`page_id`/`page_width` back after creating; `fix` reconciles all of these (plus `page_width`) from the live page. `update`, `create`, and `fix` all take `--dry-run`, which previews the actions (pages, attachment uploads, width changes, and — for `create` — frontmatter write-backs) without writing to Confluence or files; per-file human output is identical to a real run, distinguished only by a leading `DRY RUN` banner and a `dry_run: true` field in `--json`. In a `create` dry-run a previewed page has no id/URL yet, and an in-set child's `parent` is null; the parent's source `.md` is always reported in the `parent_file` output field. Commands process multiple files (except `info`, single-arg) and exit non-zero if any fail.
Each markdown file is one page. Frontmatter carries `title`, `page_id`, `space` (a key), `parent` (`null` / a `.md` path / a page id), and `page_width`. `update` requires a `page_id` (from frontmatter or `--page-id`) and errors without one; `--title`/`--page-id` override the frontmatter (single FILE only), `--page-width` overrides too (batch allowed), and `update` never writes back to files. It asserts `page_width` only when set via flag or frontmatter (otherwise the live width is left alone), and skips a file whose mtime predates the page's last version unless `--force`. `create` takes `--title`/`--page-width` overrides (`--title` single-FILE only; `--page-width` batch-ok, default `max`) and, unless `--no-persist` is given, writes `title`/`space`/`parent`/`page_id`/`page_width` back after creating; `fix` reconciles all of these (plus `page_width`) from the live page. `update`, `create`, and `fix` all take `--dry-run`, which previews the actions (pages, attachment uploads, width changes, and — for `create` — frontmatter write-backs) without writing to Confluence or files; per-file human output is identical to a real run, distinguished only by a leading `DRY RUN` banner and a `dry_run: true` field in `--json`. In a `create` dry-run a previewed page has no id/URL yet, and an in-set child's `parent` is null; the parent's source `.md` is always reported in the `parent_file` output field. Commands process multiple files (except `info`/`read`/`attachment-list`, single-arg) and exit non-zero if any fail. A command that names a page takes it three ways via `internal/pageref` (id / URL / `.md` with a `page_id`).
129 changes: 119 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,15 +85,33 @@ delete scopes. This might change in the future.

## Usage

General:

```sh
markfluence --help
```

Manipulating Confluence pages:

```sh
markfluence create --help
markfluence update --help
markfluence fix --help
markfluence info --help
markfluence read --help
```

Manipulating Confluence page attachments:

```sh
markfluence attachment-list --help
markfluence attachment-upload --help
markfluence attachment-download --help
```

Every command that takes a page accepts it three ways: a numeric page id, a
Confluence page URL, or a Markdown file whose frontmatter has a `page_id`.

### `create`

```
Expand Down Expand Up @@ -198,13 +216,13 @@ markfluence fix docs/foo.md --dry-run
### `info`

```
Usage: markfluence info ARG [flags]
Usage: markfluence info PAGE [flags]
```

Print a page's metadata (id, title, status, space, parent, version, page width,
authors, dates, url). `ARG` is a numeric page id or a Markdown file whose
frontmatter has a `page_id`. `--properties` also lists all of the page's content
properties.
authors, dates, url). `PAGE` is a numeric page id, a Confluence page URL, or a
Markdown file whose frontmatter has a `page_id`. `--properties` also lists all of
the page's content properties.

```sh
markfluence info 1234567890
Expand All @@ -214,12 +232,13 @@ markfluence info docs/foo.md --properties
### `read`

```
Usage: markfluence read ARG [flags]
Usage: markfluence read PAGE [flags]
```

Fetch a Confluence page and print its body to stdout. `ARG` is a numeric page id
or a Confluence page URL (the modern `/wiki/.../pages/<id>/...` form or a legacy
`?pageId=<id>` URL). It composes with shell redirection.
Fetch a Confluence page and print its body to stdout. `PAGE` is a numeric page id,
a Confluence page URL (the modern `/wiki/.../pages/<id>/...` form or a legacy
`?pageId=<id>` URL), or a Markdown file whose frontmatter has a `page_id`. It
composes with shell redirection.

`--format` selects the output:

Expand All @@ -244,6 +263,90 @@ markfluence read 1234567890 --format storage > page.storage.xml
markfluence read "https://org.atlassian.net/wiki/spaces/ENG/pages/1234567890/Title"
```

### `attachment-list`

```
Usage: markfluence attachment-list PAGE [flags]
```

List a page's attachments.

```console
$ markfluence attachment-list 1234567890
NAME SIZE VER TYPE SOURCE
assets%2Fdiagram.png 24.1 KB 3 image/png assets/diagram.png
notes.pdf 1.2 MB 1 application/pdf -
```

`NAME` is the name Confluence stores. For an image markfluence published that is
the percent-encoded source path (see [Body](#body)), and `SOURCE` is the Markdown
image path it came from — so the table shows at a glance which attachments a
publish manages and which it will leave alone.

`SOURCE` is a dash when no source path is recorded: either the attachment was
uploaded by hand, or it was published before markfluence recorded source paths.
Those two look the same here; `--json` has a `managed` field that tells them
apart. Attachments left behind by the encoding change show up this way, which is
how you find them.

### `attachment-upload`

```
Usage: markfluence attachment-upload PAGE FILE... [flags]
```

Upload or replace attachments on a page, complementing the automatic sync that
`create` and `update` perform for a page's images.

Each file is attached under its base name. A file whose contents already match
what's on the page is skipped, using the same checksum bookkeeping
`create`/`update` use, so uploading by hand and publishing agree on what's
current. `--force` uploads anyway (bumping the attachment's version), which is
how you repair an attachment whose stored bytes drifted while its checksum still
matches. `--dry-run` previews without writing.

`--name` sets the attachment name for a single file and takes a **path**, which
markfluence encodes for you — so `--name assets/x.png` produces the attachment
that an image written as `![](assets/x.png)` resolves to. The recorded source
path always matches the stored name, so a later publish won't create a duplicate
under a different one.

```sh
markfluence attachment-upload 1234567890 diagram.png
markfluence attachment-upload 1234567890 report.pdf notes.txt
markfluence attachment-upload 1234567890 img.png --name assets/diagram.png
markfluence attachment-upload 1234567890 diagram.png --force
```

### `attachment-download`

```
Usage: markfluence attachment-download PAGE [NAME...] [flags]
```

Download a page's attachments. Each `NAME` is an attachment name as
`attachment-list` reports it; with no `NAME`, every attachment is downloaded.

An attachment markfluence published is written back to the Markdown image path
recorded in its comment, so the downloaded tree matches what the page's Markdown
references and previews locally:

```console
$ markfluence attachment-download 1234567890 --dest ./out
downloaded /out/assets/diagram.png
downloaded /out/notes.pdf
```

An attachment with no recorded path — hand-uploaded, or published before
markfluence recorded them — is written under its stored name. `--flat` writes
everything under stored names. `--dest` defaults to the current directory and is
created if missing. An existing file is skipped unless `--force`, and
`--dry-run` previews without writing.

A recorded path that would resolve outside `--dest` is refused for that
attachment: the path comes from an attachment comment, which anyone who can edit
the page controls.

### `--json` output

The persistent `--json` flag makes any command emit a single machine-readable
Expand Down Expand Up @@ -297,8 +400,14 @@ Notes on the schema:
shapes (empty values are `null` or `[]`); the key *set* differs per command.
`schema_version` is bumped on any breaking change.
- **Status verbs** are per-command: `published`/`skipped` (`update`),
`created`/`not_created` (`create`), `changed`/`consistent` (`fix`), plus
`failed`. `info`/`read` results carry data only (no status verb).
`created`/`not_created` (`create`), `changed`/`consistent` (`fix`),
`created`/`updated`/`skipped` (`attachment-upload`),
`downloaded`/`skipped` (`attachment-download`), plus `failed`. `info`, `read`,
and `attachment-list` results carry data only (no status verb).
- **One result per target**, and the target is per-command: the page for
`info`/`read` (always one), the file for `update`/`create`/`fix`, and the
attachment for the three `attachment-*` commands — so
`.results[] | .filename` works and `summary.total` is the attachment count.
- **Compound values are objects**, never display strings — `version`,
`page_width`, and the `created`/`updated` author stamps on `info`.
- **`create`'s two-phase abort** (a validation failure means nothing is created)
Expand Down
Loading