Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .okf/build/test-gates.md
Original file line number Diff line number Diff line change
Expand Up @@ -789,3 +789,36 @@ permanently on pages that have no defect.
the platform that will judge it, by recording it twice on the same commit. The
floor for a page is its measured noise, not the repo default. A tolerance that
looks like slack may be the only thing absorbing a rasteriser.

# A new gate is not trusted until you have broken it

Adding a test and watching it pass proves the test RUNS. It does not prove the
test WORKS - a gate that asserts nothing passes forever, and looks identical in
CI to one that guards something.

**Procedure, both halves required:**
1. Inject the exact defect the gate exists to catch.
2. Run it. Quote the failure message - that string is the evidence.
3. Revert. Re-run. Green.

Skipping step 1 has cost this repo repeatedly:

- The testimonial gate asserted `assert_includes canon, rendered`, which passes
when `rendered` is `""` - every string contains the empty string. A vanished
blockquote would have reported green. Caught by a reviewer breaking it, not
by writing it.
- The rendered banned-phrase ratchet sat at 14 against an actual 11. Those 3
spare hits swallowed a planted banned adjective whole. A ratchet with slack
is a gate that has already been disarmed.
- `rake test:links` excluded 133,874 of 149,516 links (production renders
absolute URLs; `--offline` drops every http(s) URI) and was green for a year
on a site with five real broken links, one of them a conversion path and one
a post's own canonical pointing at a 404.

**When a gate cannot discriminate yet, write that in the test.** The derived
tenure assertion cannot tell `derived` from `frozen` while both read "18+" in
2026 - it starts biting on 2027-01-01. That is stated in the test body, so the
next reader does not mistake a passing run for proof.

Full fault-injection matrix, including what nothing guards:
`docs/20-29-testing-qa/20.11-gate-fault-injection-2026-08-22-reference.md`.
32 changes: 32 additions & 0 deletions .okf/log.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,38 @@ make it green: restructure same-day entries under one heading, and add
`timestamp` to the 23 concepts missing it (anchored to each file's last commit
time, which is verifiable - never invented).

## 2026-08-22 - fault injection: the suite caught 3 of 8, and the misses were the point

Eight realistic defects planted one at a time, predictions written down BEFORE
measuring. Three caught. Two of the five misses were gates reporting green
while inspecting almost nothing:

`rake test:links` excluded **133,874 of 149,516 links** - production renders
internal links absolute and `lychee --offline` drops every http(s) URI, so the
homepage's one "OK" was its own skip-link anchor. A `--remap` onto the built
tree took it to 114,050 checked, and it immediately found five real defects
that had been invisible: two wrong blog slugs (5 links), a post whose own
`canonical_url` pointed at a 404, `/contact/` on a conversion page, and a
closing section promising an "Internal Product ROI Calculator" spreadsheet -
itemising five things inside it, "no email required, instant download" - for a
resource that never existed. That last one is a fabricated deliverable, the
same class as an unsourced number, and was removed rather than redirected.

The rendered banned-phrase ratchet carried 3 hits of slack (baseline 14, actual
11) which swallowed a planted phrase whole; `SURFACES` never globbed
`content/next/**`, so the entire v2 rail had no source-side cover.

**The rule this produced, now in CLAUDE.md and in test-gates.md:** a new test is
not done until you have broken the code and watched it fail. Green proves the
test runs, not that it works. Where a gate genuinely cannot discriminate yet -
the derived-tenure assertion, blind until 2027-01-01 - say so in the test body
instead of letting a passing run read as proof.

Two misses were left unguarded on purpose: the `<th>` PurgeCSS trap cannot be
caught by the visual gate even in principle (it builds production, so it purges
exactly as production does and matches its own baseline), and no bespoke gate
is proportionate for it yet.

## 2026-08-22 - the noise floor is per-page, and the pinned tolerances are load-bearing

Chasing the last red Linux key produced a better finding than the fix. Four
Expand Down
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ Operational knowledge lives in `.okf/` (`build/`, `content/`, `design/`, `workfl

- **TDD**: RED → GREEN → REFACTOR. Doctrine: `docs/20-29-testing-qa/` + `docs/incidents/25.0x`.
- **Behavior-focused tests ONLY** — reject implementation/existence/config tests. Don't hardcode tunables (`q=90`, exact sizes); assert the shape (`q=\d+`, has `<picture>`). A test that breaks on a knob change is testing config — relax it.
- **A new test isn't done until you've BROKEN THE CODE and watched it fail (BLOCKING).** Green proves nothing on its own — a test that passes because it asserts nothing passes forever. So: inject the exact defect the test exists to catch, quote the failure message in your handback, then revert and re-run green. Both halves are the evidence. Where a gate can't discriminate yet, say so in the test's own comment rather than pretending. Measured 2026-08-22 (`docs/20-29-testing-qa/20.11`): fault-injecting 8 realistic defects caught only 3 — the testimonial gate passed a *vanished* blockquote (`""` is a substring of everything), the rendered ratchet carried 3 hits of slack that swallowed a planted banned phrase, and the link job excluded 90% of links while reporting green for a year.
- **Framework**: Minitest (`test/system/`, `test/unit/`). Test runner matrix: the header **Test** line — don't restate it.
- **Visual regression + gates**: full matrix, tolerances, baseline procedure → `.okf/build/test-gates.md` (canonical). Tolerance 0.0 for refactoring, ≤0.03 for new features.
- **After HTML/CSS/JS changes**: chrome-devtools pass — zero console errors, zero asset 404s, desktop + mobile screenshots, verify Core Web Vitals. All gates pass → commit; any fail → fix first.
Expand Down
22 changes: 17 additions & 5 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -88,10 +88,19 @@ namespace :test do
end
end

# Broken-link checks (2026-07-21: GA 404 spike audit). Coverage is total by
# construction - every *.html the build emits is globbed from disk and
# passed to lychee as an explicit input, so nothing is skipped the way a
# single-URL or sitemap-only scan would skip.
# Broken-link checks (2026-07-21: GA 404 spike audit). Every *.html the build
# emits is globbed from disk and passed as an explicit input, so no PAGE is
# skipped.
#
# That is not the same as no LINK being skipped, and for a year it wasn't:
# the production build renders internal links absolute
# (https://jetthoughts.com/...) and `--offline` excludes every http(s) URI by
# design, so 133,874 of 149,516 links were excluded and the job was green
# because it inspected almost nothing (measured 2026-08-22 by planting a
# broken link and watching it pass - docs/20-29-testing-qa/20.11). The
# --remap below rewrites those absolute URLs back onto the built tree so
# they are actually resolved; without it this task checks little more than
# each page's own #main-content skip-link anchor.
#
# These build with ENVIRONMENT=production (not the top-level :build task's
# dev default) rather than depending on :build - the dev config mounts
Expand All @@ -118,7 +127,10 @@ namespace :test do
html_files = Dir.glob("#{dir}/**/*.html")
abort "No HTML files found in #{dir} - did the build fail?" if html_files.empty?
puts "lychee: scanning #{html_files.size} pages for broken internal links..."
sh("lychee", "--offline", "--no-progress", "--root-dir", File.expand_path(dir), "#{dir}/**/*.html")
root = File.expand_path(dir)
sh("lychee", "--offline", "--no-progress",
"--remap", "https://jetthoughts.com/(.*) file://#{root}/$1",
"--root-dir", root, "#{dir}/**/*.html")
end

# Same page set including external links. Non-blocking: third-party
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1272,7 +1272,7 @@ Want to go deeper? Explore these advanced patterns:
**Building RAG (Retrieval-Augmented Generation) Systems with pgvector**:
- Combine semantic search with OpenAI GPT-4 for context-aware AI responses
- Store documentation embeddings, retrieve relevant chunks, generate answers
- [Read our complete Ruby AI Integration guide](/blog/2025/complete-guide-ruby-rails-ai-integration-2025/) for RAG implementation patterns
- [Read our complete Ruby AI Integration guide](/blog/complete-guide-ruby-rails-ai-integration-2025/) for RAG implementation patterns

**Hybrid Search: Combining Full-Text and Vector Search**:
- Use PostgreSQL `tsvector` for keyword matching + pgvector for semantic similarity
Expand Down Expand Up @@ -1354,7 +1354,7 @@ For 90% of Rails applications, **pgvector is the right choice**. Start here, upg
- [OpenAI Embeddings API](https://platform.openai.com/docs/guides/embeddings) - Embedding models comparison

**JetThoughts Guides**:
- [Complete Guide to Ruby on Rails AI Integration 2025](/blog/2025/complete-guide-ruby-rails-ai-integration-2025/) - OpenAI/Anthropic integration patterns
- [Complete Guide to Ruby on Rails AI Integration 2025](/blog/complete-guide-ruby-rails-ai-integration-2025/) - OpenAI/Anthropic integration patterns
- [Building RAG Applications with Rails and pgvector](/blog/building-rag-applications-rails-pgvector/) - Retrieval-Augmented Generation tutorial
- [Rails Performance Monitoring](/blog/rails-performance-monitoring-complete-guide/) - APM setup for production apps

Expand Down
2 changes: 1 addition & 1 deletion content/blog/ai-code-ownership-accountability/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ While you wait, open your hosting dashboard. Find out who can deploy to producti

Two of the three rescues we ran last quarter started with a 48-hour silence on exactly this email.

> **If you'd rather have a second pair of eyes do the audit:** [send us](https://jetthoughts.com/contact/) a read-only repo invite. We send back a one-page report in 48 hours - AI-vs-human commit ratio, any unscoped tokens or god-mode keys, and any AI-introduced dependencies nobody vetted. No force-push, no deploy keys, no rebuild pitch. Revoke the invite the moment the report ships. Not ready to share code yet? Send your three biggest concerns and we'll send back a one-page checklist.
> **If you'd rather have a second pair of eyes do the audit:** [send us](/contact-us/) a read-only repo invite. We send back a one-page report in 48 hours - AI-vs-human commit ratio, any unscoped tokens or god-mode keys, and any AI-introduced dependencies nobody vetted. No force-push, no deploy keys, no rebuild pitch. Revoke the invite the moment the report ships. Not ready to share code yet? Send your three biggest concerns and we'll send back a one-page checklist.

## When Not To Bother

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -393,25 +393,6 @@ Your transformation is possible. It just requires measuring and communicating th

---

## Ready to prove your team's value?

Download our **Internal Product ROI Calculator** to start quantifying your team's business impact today. This spreadsheet template includes:

- Four-pillar value calculation framework
- Executive dashboard templates
- Stakeholder communication guides
- 90-day implementation timeline
- Real-world calculation examples

{{< cta title="Get the ROI Calculator"
description="Transform your internal team from cost center to profit driver with our proven framework and templates."
button-text="Download Free Calculator"
button-url="/resources/internal-product-roi-calculator" >}}

*No email required. Instant download.*

---

*Need help implementing value measurement for your internal team? Our engineering management consultants have helped dozens of internal product leaders prove ROI and secure budget increases. [Schedule a consultation](/contact-us) to discuss your specific situation.*

---
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ cover_image: cover.png
metatags:
image: cover.png
cover_image_alt: "JetThoughts blog cover for Active Job Continuations in Rails 8.1 - minimalist dark design with Ruby-to-purple gradient headline"
canonical_url: https://jetthoughts.com/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/
canonical_url: https://jetthoughts.com/blog/rails-8-1-active-job-continuations-background-jobs/
related_posts: false
---

Expand Down
2 changes: 1 addition & 1 deletion content/blog/rails-event-structured-logging-8-1/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ For apps still on Rails 7.x, the monotonic subscribe API exists but the payload

We've moved four production Rails apps from regex-based log monitoring to event subscriptions over the last eight months. Three were on Datadog, one on New Relic. The pattern was consistent: the setup ran between half a day and two days per app depending on how many custom parsers we had to untangle, and false-positive alerts dropped noticeably in the weeks following - **we measured roughly 60% fewer across the four apps**, though the improvement varied by how noisy the original regex parsers were. The apps weren't more reliable. The monitoring just stopped misreading log output from normal Rails behavior.

The [Active Job Continuations work in Rails 8.1](/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/) uses the same instrumentation layer for job lifecycle events. If you're already subscribing to `perform.active_job`, the event shape improvements in 8.1 apply there too.
The [Active Job Continuations work in Rails 8.1](/blog/rails-8-1-active-job-continuations-background-jobs/) uses the same instrumentation layer for job lifecycle events. If you're already subscribing to `perform.active_job`, the event shape improvements in 8.1 apply there too.

For teams using the [Solid Trifecta instead of Redis](/blog/solid-trifecta-hybrid-redis-rails-8/), Solid Queue emits its own `ActiveSupport::Notifications` events for job enqueuing, execution, and failure - subscribable through the same API, no separate polling loop required.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ We split them. Dedicated worker dynos for Sidekiq. A separate Redis instance for

The bigger change: we moved from Sidekiq to [Solid Queue](/blog/solid-queue-vs-sidekiq-complete-comparison/). The client's job patterns were simple - email delivery, webhook processing, report generation - and didn't need Sidekiq Pro's batching or rate limiting features. Solid Queue runs on the database. One fewer infrastructure dependency to manage and monitor.

For jobs that can't afford to be lost, Rails 8.1's [Active Job Continuations](/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/) are worth understanding. They solve the problem where a long-running job crashes halfway through and restarts from zero.
For jobs that can't afford to be lost, Rails 8.1's [Active Job Continuations](/blog/rails-8-1-active-job-continuations-background-jobs/) are worth understanding. They solve the problem where a long-running job crashes halfway through and restarts from zero.

## When NOT to do any of this

Expand Down
2 changes: 1 addition & 1 deletion content/blog/solid-trifecta-hybrid-redis-rails-8/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ Across our last few migrations, the typical post-migration shape looks like this

We moved fragment and API response caching to Solid Cache. Cache size went from 2GB (limited by Redis memory) to roughly 50GB on disk. Average read latency increased from 0.3ms to 1.1ms; nobody noticed. P95 page load actually improved because the cache hit rate climbed from 68% to 91% with the larger cache footprint.

Solid Queue took over email delivery, report generation, and scheduled cleanup at 200 jobs/minute steady. [Active Job Continuations](/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/) made the long-running nightly import deploy-safe for the first time.
Solid Queue took over email delivery, report generation, and scheduled cleanup at 200 jobs/minute steady. [Active Job Continuations](/blog/rails-8-1-active-job-continuations-background-jobs/) made the long-running nightly import deploy-safe for the first time.

Sidekiq stayed for the payment webhook queue (needs sub-second latency) and the real-time inventory sync (2,000 jobs/minute bursts during peak hours). Redis also stayed for Action Cable, where 1,200 concurrent WebSocket connections power the admin dashboard.

Expand Down
2 changes: 1 addition & 1 deletion content/services/fractional-cto-cost/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,4 +226,4 @@ Our fractional CTO cost-effective solutions support businesses across multiple i

Understanding fractional CTO cost is the first step toward getting senior technology leadership for your business. Our transparent pricing and flexible engagement models make strategic CTO expertise accessible to companies of all sizes.

[Contact us today](/contact/) for a free consultation and customized fractional CTO cost proposal tailored to your specific needs and growth objectives.
[Contact us today](/contact-us/) for a free consultation and customized fractional CTO cost proposal tailored to your specific needs and growth objectives.
14 changes: 11 additions & 3 deletions test/unit/marketing_copy_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ class MarketingCopyTest < Minitest::Test
"content/services/**/*.md",
"content/use-cases/**/*.md",
"layouts/next/**/*.html", # the v2 rail is template-authored marketing copy on root layouts
# ...and its CONTENT side. Without this the pilots' copy had no source-side
# cover at all: a planted "world-class" in a pilot stub passed the whole
# suite on 2026-08-22 (docs/20-29-testing-qa/20.11).
"content/next/**/*.md",
"themes/beaver/layouts/home.html",
"themes/beaver/layouts/page/*.html",
# Blog CHROME (list hero, section furniture) is a marketing surface even
Expand Down Expand Up @@ -141,9 +145,13 @@ def test_every_declared_surface_matches_at_least_one_file
# pages - which is the whole argument for reading RENDERED output: source
# matching sees two files, the reader sees twelve.
#
# The remaining 14 live in individual post bodies. Tighten this number every
# time a batch is cleared; a ratchet left slack lets the win regress silently.
RENDERED_BASELINE = 14
# The remaining hits live in individual post bodies. Tighten this number every
# time a batch is cleared; a ratchet left slack lets the win regress silently
# - which is not hypothetical: this sat at 14 against an actual 11, and those
# three spare hits swallowed a planted "world-class" whole (2026-08-22,
# docs/20-29-testing-qa/20.11). Set it to the measured count, then prove the
# ratchet is exact by dropping it one lower and watching it fail.
RENDERED_BASELINE = 11

def test_rendered_pages_do_not_regress_on_banned_phrases
violations = rendered_files.flat_map { |path| rendered_hits(path) }.sort
Expand Down
Loading