Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,8 @@ GIT_STATS_TOKEN=

# Data directory holding raw/ and stats.db. Defaults to ./data.
#GIT_STATS_DIR=data

# API host. Defaults to https://api.github.com. GitHub Enterprise Server puts
# the REST API under https://HOST/api/v3 — point this there to track a
# repository hosted on it. Every endpoint this tool uses exists there too.
#GIT_STATS_API_BASE=
33 changes: 21 additions & 12 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,23 +1,32 @@
# Holds the API token and the repository to track — never commit.
.env

# Derived from data/raw/ — always reproducible with `git-stats rebuild`.
data/stats.db
data/stats.db-wal
data/stats.db-shm

# data/raw/ is deliberately NOT ignored. GitHub deletes traffic data after 14
# days, so those files are the only copy that will ever exist — committing them
# is the backup. They compress ~95% (consecutive releases.json are near
# identical), which projects to roughly 15 MB per year of daily collection.
# This repository's own collection output. Views, clones, referrers and paths
# are owner-only analytics of whatever repository is being tracked, and this
# repository is public — committing an archive here would publish them.
#
# The leading slash is deliberate: an unanchored `data/` would also match
# docs/demo/data/, which holds the synthetic dataset the README gifs are
# recorded from and IS committed.
#
# WARNING before committing an archive to a PUBLIC repository: views, clones,
# referrers and paths are owner-only analytics, and committing them publishes
# them. Add `data/` here if you would rather keep them private.
# Committing data/raw/ is still the right default for a PRIVATE tracking repo:
# GitHub deletes traffic data after 14 days, so the archive becomes the only
# copy that will ever exist, and version control is the backup.
/data/

# Generated dashboards.
*.html

# docs/ ships the two recordings the README embeds and nothing else. The
# fixtures they are made from, the VHS tape, the capture scripts and the build
# scratch all stay local — they are how the gifs get made, not what the project
# is. Ignoring the directory and re-including the gifs is deliberate: git does
# not descend into an ignored directory, so a negation can only rescue paths
# whose parents are still visible, which is why this ignores docs/* rather
# than docs/.
/docs/*
!/docs/*.gif

# Build artifacts. Leading slash on /git-stats is deliberate: an unanchored
# `git-stats` would also match the cmd/git-stats/ DIRECTORY, silently hiding
# every source file added there.
Expand Down
37 changes: 28 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,22 @@ So this tool snapshots those endpoints, archives every raw response, and derives
trends from the archive. Everything stays on your machine — there is no service,
no account and no telemetry.

## What it looks like

One command takes a snapshot, the other says where things stand:

![git-stats collect, then git-stats report](docs/demo.gif)

`report -html` writes the same numbers as a self-contained dashboard — one file,
no external requests, opens straight from disk:

![the HTML dashboard, scrolled top to bottom](docs/dashboard.gif)

Both recordings run against a synthetic dataset: `acme/widget` is not a real
repository, and none of those figures are anybody's real traffic. Views, clones,
referrers and paths are owner-only analytics, so a README is the wrong place for
a real one.

## Install

```sh
Expand Down Expand Up @@ -63,6 +79,7 @@ variables always win** — `.env` supplies defaults only, so a one-off
| `GIT_STATS_TOKEN` | API token. Falls back to `GH_TOKEN`, then `GITHUB_TOKEN`. |
| `GIT_STATS_ASSET_PREFIX` | Release asset filename prefix. Defaults to the repository name. |
| `GIT_STATS_DIR` | Data directory holding `raw/` and `stats.db`. Defaults to `./data`. |
| `GIT_STATS_API_BASE` | API host. Defaults to `https://api.github.com`; set it to `https://HOST/api/v3` for GitHub Enterprise Server. |

There is deliberately no default repository: an unset `GIT_STATS_REPO` is an error
rather than a silent fallback, so a misconfigured run cannot quietly archive
Expand Down Expand Up @@ -154,15 +171,17 @@ Both `collect` and `rebuild` write to the database through the same ingest path,
rebuild reproduces exactly what collection produced. The database is safe to delete;
the archive is not. A file is simply absent when that endpoint was unavailable.

**`data/raw/` is not gitignored, on purpose; `data/stats.db` is.** GitHub deletes
traffic data after 14 days, so those files become the only copy that will ever exist —
version control is the backup. The archive compresses about 95% (consecutive
`releases.json` are nearly identical), which works out to roughly 15 MB per year of
daily collection.

> **Before pushing an archive to a public repository:** views, clones, referrers and
> paths are owner-only analytics, and committing them publishes them. Add `data/` to
> `.gitignore` if you would rather keep them private.
**Committing `data/raw/` is worth considering.** GitHub deletes traffic data after 14
days, so those files become the only copy that will ever exist — version control is the
backup. The archive compresses about 95% (consecutive `releases.json` are nearly
identical), which works out to roughly 15 MB per year of daily collection. `stats.db` is
derived and never needs committing.

> **Only in a private repository, though:** views, clones, referrers and paths are
> owner-only analytics of whatever you are tracking, and committing them publishes them.
> This repository ignores its own `/data/` for exactly that reason — it is public, so
> the numbers in the gifs above come from a synthetic dataset rather than a real
> archive.

## Querying it directly

Expand Down
11 changes: 11 additions & 0 deletions cmd/git-stats/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@ const (
envAssetPrefix = "GIT_STATS_ASSET_PREFIX"
// envDir overrides the default data directory.
envDir = "GIT_STATS_DIR"
// envAPIBase overrides the API host. It exists for GitHub Enterprise
// Server, whose REST API lives under https://HOST/api/v3, and for
// pointing a run at a local fixture server, which is how the recordings
// in the README are made.
envAPIBase = "GIT_STATS_API_BASE"
)

func main() {
Expand Down Expand Up @@ -72,6 +77,8 @@ environment:
breakdown of <prefix>-<version>-<os>-<arch>.<ext>.
Defaults to the repository name.
GIT_STATS_DIR default data directory (otherwise ./data)
GIT_STATS_API_BASE API host (otherwise https://api.github.com). Point it at
https://HOST/api/v3 for GitHub Enterprise Server.

All of these may instead be set in a .env file, read from the working directory or
from the directory holding the binary. Exported variables take precedence over it.
Expand Down Expand Up @@ -175,9 +182,13 @@ func runCollect(ctx context.Context, args []string, forceStars bool, env dotenv.
Token: token,
DataDir: *data,
Assets: assetNamer(repo),
APIBase: strings.TrimRight(strings.TrimSpace(os.Getenv(envAPIBase)), "/"),
Stars: *stars || forceStars,
Log: os.Stdout,
}
if cfg.APIBase != "" {
fmt.Printf("api: %s\n", cfg.APIBase)
}

// Name the credential's origin: a 403 on the traffic endpoints is almost
// always a broadly-scoped fallback token being used instead of the
Expand Down
Binary file added docs/dashboard.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/demo.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading