[MAR-3084] Add fetch-with-retries package

[leafrogers]

Following on from the prep PR, this one adds the first real shared package using the new workflow. This package can then be imported into our projects as @financial-times/martech-fetch-with-retries.

Note

All thoughts and ideas are very welcome for this PR too!

Sum’ry

This PR is split out into two commits:

1. adding the fetch-with-retries package
2. adding the release/publish workflow needed to get that package into Cloudsmith

The package is intended to replace local fetch-retry helpers that we currently maintain in a few different repos, so we can stop solving the same problem slightly differently in several places.

Why this is happening

Across the Martech estate, we already have a few variants of “fetch, but retry when upstream is flaky”. They’re not all identical, but they’re close enough that it seems worth consolidating. This package caters to all of those variants.

What changed

Shared package

Added @financial-times/martech-fetch-with-retries.

Usage:

const import { createFetchWithRetries } from '@financial-times/martech-fetch-with-retries';
const fetchWithRetries = createFetchWithRetries(logger);

const response = await fetchWithRetries('https://ft.com'); // tries up to 3 times if needed
// …

Default retry behaviour

By default, the package retries:

• thrown fetch errors
• HTTP statuses 408, 429, 500, 502, 503, 504

And unless configured otherwise, only retries these idempotent methods:

• GET, HEAD, OPTIONS, PUT, DELETE

Retries for POST and PATCH are explicit opt-ins via:

• retryPostRequests
• retryPatchRequests

Configuration

The wrapper supports:

Configurable prop	Type
maximumAttempts	number
backoffStrategy	function
retryPostRequests	boolean
retryPatchRequests	booleans

Logging

The package emits structured debug and warn retry events, and is intended to work with @dotcom-reliability-kit/logger.

Tests/docs

Included:

• unit tests, end-to-end tests, a TypeScript smoke test, a package README and changelog

Release/publish plumbing

This PR also adds the minimum publishing setup so the package is actually consumable:

• CircleCI publish workflow
• Cloudsmith npm configuration
• tag-based release trigger for this package

🌸 🐄 🐖 🐓 🐈 🐑 🌸 🐄 🐖 🐓 🐈 🐑 🌸 🐄 🐖 🐓 🐈 🐑 🌸 🐄 🐖 🐓 🐈 🐑 🌸

A bit o’ context

Key decisions made

Native Node fetch only

This package assumes globalThis.fetch and does not support custom fetch injection. That keeps the runtime and type surface smaller, and matches current usage in our repos.

Transport-level only

This package retries requests and logs retry events, but it doesn’t parse JSON or normalize responses. That felt like the right split. Retry logic is the shared concern here; response/body handling can stay separate.

Sensible defaults

Retrying POST and PATCH by default felt too risky for a shared package, and looking around our repos, not all of them need it.

No Retry-After support in v1

This was considered, and then removed as none of our existing repos use it.

It felt like edge-case handling that we don’t need at the moment. If we need more sophisticated delay behaviour later, we can revisit it or use backoffStrategy.

Publishing approach

This PR adds package publishing, but doesn’t introduce fully automated releasing, unlike @dotcom-reliability-kit. Chat with me about why I decided against this, if you’re intrigued!

So for now, the release model is intentionally simple:

1. bump the package version
2. update the changelog
3. create a GitHub release with a matching tag like fetch-with-retries-v1.0.1
4. let CircleCI publish that version to Cloudsmith

🌸 🐄 🐖 🐓 🐈 🐑 🌸 🐄 🐖 🐓 🐈 🐑 🌸 🐄 🐖 🐓 🐈 🐑 🌸 🐄 🐖 🐓 🐈 🐑 🌸

Next steps 🚀

When/if we’re happy with this package, we can start replacing retry helpers in a couple of repos. I’ve already tried it locally with ip-martech-scs-to-marketo and it seems good.

A moving depiction of this package

[saldixon]

review in progress, but thought I"d submit some questions so far.

[saldixon]

lovely stuff!

[leafrogers]

Can’t believe I left this unalphabetised