diff --git a/.okf/build/test-gates.md b/.okf/build/test-gates.md index ac86e6d5d..f26522931 100644 --- a/.okf/build/test-gates.md +++ b/.okf/build/test-gates.md @@ -789,3 +789,36 @@ permanently on pages that have no defect. the platform that will judge it, by recording it twice on the same commit. The floor for a page is its measured noise, not the repo default. A tolerance that looks like slack may be the only thing absorbing a rasteriser. + +# A new gate is not trusted until you have broken it + +Adding a test and watching it pass proves the test RUNS. It does not prove the +test WORKS - a gate that asserts nothing passes forever, and looks identical in +CI to one that guards something. + +**Procedure, both halves required:** +1. Inject the exact defect the gate exists to catch. +2. Run it. Quote the failure message - that string is the evidence. +3. Revert. Re-run. Green. + +Skipping step 1 has cost this repo repeatedly: + +- The testimonial gate asserted `assert_includes canon, rendered`, which passes + when `rendered` is `""` - every string contains the empty string. A vanished + blockquote would have reported green. Caught by a reviewer breaking it, not + by writing it. +- The rendered banned-phrase ratchet sat at 14 against an actual 11. Those 3 + spare hits swallowed a planted banned adjective whole. A ratchet with slack + is a gate that has already been disarmed. +- `rake test:links` excluded 133,874 of 149,516 links (production renders + absolute URLs; `--offline` drops every http(s) URI) and was green for a year + on a site with five real broken links, one of them a conversion path and one + a post's own canonical pointing at a 404. + +**When a gate cannot discriminate yet, write that in the test.** The derived +tenure assertion cannot tell `derived` from `frozen` while both read "18+" in +2026 - it starts biting on 2027-01-01. That is stated in the test body, so the +next reader does not mistake a passing run for proof. + +Full fault-injection matrix, including what nothing guards: +`docs/20-29-testing-qa/20.11-gate-fault-injection-2026-08-22-reference.md`. diff --git a/.okf/log.md b/.okf/log.md index f6b94b96c..3e952ec20 100644 --- a/.okf/log.md +++ b/.okf/log.md @@ -51,6 +51,38 @@ make it green: restructure same-day entries under one heading, and add `timestamp` to the 23 concepts missing it (anchored to each file's last commit time, which is verifiable - never invented). +## 2026-08-22 - fault injection: the suite caught 3 of 8, and the misses were the point + +Eight realistic defects planted one at a time, predictions written down BEFORE +measuring. Three caught. Two of the five misses were gates reporting green +while inspecting almost nothing: + +`rake test:links` excluded **133,874 of 149,516 links** - production renders +internal links absolute and `lychee --offline` drops every http(s) URI, so the +homepage's one "OK" was its own skip-link anchor. A `--remap` onto the built +tree took it to 114,050 checked, and it immediately found five real defects +that had been invisible: two wrong blog slugs (5 links), a post whose own +`canonical_url` pointed at a 404, `/contact/` on a conversion page, and a +closing section promising an "Internal Product ROI Calculator" spreadsheet - +itemising five things inside it, "no email required, instant download" - for a +resource that never existed. That last one is a fabricated deliverable, the +same class as an unsourced number, and was removed rather than redirected. + +The rendered banned-phrase ratchet carried 3 hits of slack (baseline 14, actual +11) which swallowed a planted phrase whole; `SURFACES` never globbed +`content/next/**`, so the entire v2 rail had no source-side cover. + +**The rule this produced, now in CLAUDE.md and in test-gates.md:** a new test is +not done until you have broken the code and watched it fail. Green proves the +test runs, not that it works. Where a gate genuinely cannot discriminate yet - +the derived-tenure assertion, blind until 2027-01-01 - say so in the test body +instead of letting a passing run read as proof. + +Two misses were left unguarded on purpose: the `` PurgeCSS trap cannot be +caught by the visual gate even in principle (it builds production, so it purges +exactly as production does and matches its own baseline), and no bespoke gate +is proportionate for it yet. + ## 2026-08-22 - the noise floor is per-page, and the pinned tolerances are load-bearing Chasing the last red Linux key produced a better finding than the fix. Four diff --git a/CLAUDE.md b/CLAUDE.md index c6eca905b..0e296fc01 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -60,6 +60,7 @@ Operational knowledge lives in `.okf/` (`build/`, `content/`, `design/`, `workfl - **TDD**: RED → GREEN → REFACTOR. Doctrine: `docs/20-29-testing-qa/` + `docs/incidents/25.0x`. - **Behavior-focused tests ONLY** — reject implementation/existence/config tests. Don't hardcode tunables (`q=90`, exact sizes); assert the shape (`q=\d+`, has ``). A test that breaks on a knob change is testing config — relax it. +- **A new test isn't done until you've BROKEN THE CODE and watched it fail (BLOCKING).** Green proves nothing on its own — a test that passes because it asserts nothing passes forever. So: inject the exact defect the test exists to catch, quote the failure message in your handback, then revert and re-run green. Both halves are the evidence. Where a gate can't discriminate yet, say so in the test's own comment rather than pretending. Measured 2026-08-22 (`docs/20-29-testing-qa/20.11`): fault-injecting 8 realistic defects caught only 3 — the testimonial gate passed a *vanished* blockquote (`""` is a substring of everything), the rendered ratchet carried 3 hits of slack that swallowed a planted banned phrase, and the link job excluded 90% of links while reporting green for a year. - **Framework**: Minitest (`test/system/`, `test/unit/`). Test runner matrix: the header **Test** line — don't restate it. - **Visual regression + gates**: full matrix, tolerances, baseline procedure → `.okf/build/test-gates.md` (canonical). Tolerance 0.0 for refactoring, ≤0.03 for new features. - **After HTML/CSS/JS changes**: chrome-devtools pass — zero console errors, zero asset 404s, desktop + mobile screenshots, verify Core Web Vitals. All gates pass → commit; any fail → fix first. diff --git a/Rakefile b/Rakefile index e7363b3ee..b97575f39 100644 --- a/Rakefile +++ b/Rakefile @@ -88,10 +88,19 @@ namespace :test do end end - # Broken-link checks (2026-07-21: GA 404 spike audit). Coverage is total by - # construction - every *.html the build emits is globbed from disk and - # passed to lychee as an explicit input, so nothing is skipped the way a - # single-URL or sitemap-only scan would skip. + # Broken-link checks (2026-07-21: GA 404 spike audit). Every *.html the build + # emits is globbed from disk and passed as an explicit input, so no PAGE is + # skipped. + # + # That is not the same as no LINK being skipped, and for a year it wasn't: + # the production build renders internal links absolute + # (https://jetthoughts.com/...) and `--offline` excludes every http(s) URI by + # design, so 133,874 of 149,516 links were excluded and the job was green + # because it inspected almost nothing (measured 2026-08-22 by planting a + # broken link and watching it pass - docs/20-29-testing-qa/20.11). The + # --remap below rewrites those absolute URLs back onto the built tree so + # they are actually resolved; without it this task checks little more than + # each page's own #main-content skip-link anchor. # # These build with ENVIRONMENT=production (not the top-level :build task's # dev default) rather than depending on :build - the dev config mounts @@ -118,7 +127,10 @@ namespace :test do html_files = Dir.glob("#{dir}/**/*.html") abort "No HTML files found in #{dir} - did the build fail?" if html_files.empty? puts "lychee: scanning #{html_files.size} pages for broken internal links..." - sh("lychee", "--offline", "--no-progress", "--root-dir", File.expand_path(dir), "#{dir}/**/*.html") + root = File.expand_path(dir) + sh("lychee", "--offline", "--no-progress", + "--remap", "https://jetthoughts.com/(.*) file://#{root}/$1", + "--root-dir", root, "#{dir}/**/*.html") end # Same page set including external links. Non-blocking: third-party diff --git a/content/blog/2025/pgvector-rails-tutorial-production-semantic-search.md b/content/blog/2025/pgvector-rails-tutorial-production-semantic-search.md index fc85f7007..f786d5783 100644 --- a/content/blog/2025/pgvector-rails-tutorial-production-semantic-search.md +++ b/content/blog/2025/pgvector-rails-tutorial-production-semantic-search.md @@ -1272,7 +1272,7 @@ Want to go deeper? Explore these advanced patterns: **Building RAG (Retrieval-Augmented Generation) Systems with pgvector**: - Combine semantic search with OpenAI GPT-4 for context-aware AI responses - Store documentation embeddings, retrieve relevant chunks, generate answers -- [Read our complete Ruby AI Integration guide](/blog/2025/complete-guide-ruby-rails-ai-integration-2025/) for RAG implementation patterns +- [Read our complete Ruby AI Integration guide](/blog/complete-guide-ruby-rails-ai-integration-2025/) for RAG implementation patterns **Hybrid Search: Combining Full-Text and Vector Search**: - Use PostgreSQL `tsvector` for keyword matching + pgvector for semantic similarity @@ -1354,7 +1354,7 @@ For 90% of Rails applications, **pgvector is the right choice**. Start here, upg - [OpenAI Embeddings API](https://platform.openai.com/docs/guides/embeddings) - Embedding models comparison **JetThoughts Guides**: -- [Complete Guide to Ruby on Rails AI Integration 2025](/blog/2025/complete-guide-ruby-rails-ai-integration-2025/) - OpenAI/Anthropic integration patterns +- [Complete Guide to Ruby on Rails AI Integration 2025](/blog/complete-guide-ruby-rails-ai-integration-2025/) - OpenAI/Anthropic integration patterns - [Building RAG Applications with Rails and pgvector](/blog/building-rag-applications-rails-pgvector/) - Retrieval-Augmented Generation tutorial - [Rails Performance Monitoring](/blog/rails-performance-monitoring-complete-guide/) - APM setup for production apps diff --git a/content/blog/ai-code-ownership-accountability/index.md b/content/blog/ai-code-ownership-accountability/index.md index c5315d2dd..b46a70dcf 100644 --- a/content/blog/ai-code-ownership-accountability/index.md +++ b/content/blog/ai-code-ownership-accountability/index.md @@ -96,7 +96,7 @@ While you wait, open your hosting dashboard. Find out who can deploy to producti Two of the three rescues we ran last quarter started with a 48-hour silence on exactly this email. -> **If you'd rather have a second pair of eyes do the audit:** [send us](https://jetthoughts.com/contact/) a read-only repo invite. We send back a one-page report in 48 hours - AI-vs-human commit ratio, any unscoped tokens or god-mode keys, and any AI-introduced dependencies nobody vetted. No force-push, no deploy keys, no rebuild pitch. Revoke the invite the moment the report ships. Not ready to share code yet? Send your three biggest concerns and we'll send back a one-page checklist. +> **If you'd rather have a second pair of eyes do the audit:** [send us](/contact-us/) a read-only repo invite. We send back a one-page report in 48 hours - AI-vs-human commit ratio, any unscoped tokens or god-mode keys, and any AI-introduced dependencies nobody vetted. No force-push, no deploy keys, no rebuild pitch. Revoke the invite the moment the report ships. Not ready to share code yet? Send your three biggest concerns and we'll send back a one-page checklist. ## When Not To Bother diff --git a/content/blog/internal-product-teams-cost-center-to-profit-driver.md b/content/blog/internal-product-teams-cost-center-to-profit-driver.md index c8cb14a50..fdfd7abc5 100644 --- a/content/blog/internal-product-teams-cost-center-to-profit-driver.md +++ b/content/blog/internal-product-teams-cost-center-to-profit-driver.md @@ -393,25 +393,6 @@ Your transformation is possible. It just requires measuring and communicating th --- -## Ready to prove your team's value? - -Download our **Internal Product ROI Calculator** to start quantifying your team's business impact today. This spreadsheet template includes: - -- Four-pillar value calculation framework -- Executive dashboard templates -- Stakeholder communication guides -- 90-day implementation timeline -- Real-world calculation examples - -{{< cta title="Get the ROI Calculator" - description="Transform your internal team from cost center to profit driver with our proven framework and templates." - button-text="Download Free Calculator" - button-url="/resources/internal-product-roi-calculator" >}} - -*No email required. Instant download.* - ---- - *Need help implementing value measurement for your internal team? Our engineering management consultants have helped dozens of internal product leaders prove ROI and secure budget increases. [Schedule a consultation](/contact-us) to discuss your specific situation.* --- diff --git a/content/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/index.md b/content/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/index.md index 56c650fec..831c02951 100644 --- a/content/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/index.md +++ b/content/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/index.md @@ -12,7 +12,7 @@ cover_image: cover.png metatags: image: cover.png cover_image_alt: "JetThoughts blog cover for Active Job Continuations in Rails 8.1 - minimalist dark design with Ruby-to-purple gradient headline" -canonical_url: https://jetthoughts.com/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/ +canonical_url: https://jetthoughts.com/blog/rails-8-1-active-job-continuations-background-jobs/ related_posts: false --- diff --git a/content/blog/rails-event-structured-logging-8-1/index.md b/content/blog/rails-event-structured-logging-8-1/index.md index c8849be95..5493adc92 100644 --- a/content/blog/rails-event-structured-logging-8-1/index.md +++ b/content/blog/rails-event-structured-logging-8-1/index.md @@ -229,7 +229,7 @@ For apps still on Rails 7.x, the monotonic subscribe API exists but the payload We've moved four production Rails apps from regex-based log monitoring to event subscriptions over the last eight months. Three were on Datadog, one on New Relic. The pattern was consistent: the setup ran between half a day and two days per app depending on how many custom parsers we had to untangle, and false-positive alerts dropped noticeably in the weeks following - **we measured roughly 60% fewer across the four apps**, though the improvement varied by how noisy the original regex parsers were. The apps weren't more reliable. The monitoring just stopped misreading log output from normal Rails behavior. -The [Active Job Continuations work in Rails 8.1](/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/) uses the same instrumentation layer for job lifecycle events. If you're already subscribing to `perform.active_job`, the event shape improvements in 8.1 apply there too. +The [Active Job Continuations work in Rails 8.1](/blog/rails-8-1-active-job-continuations-background-jobs/) uses the same instrumentation layer for job lifecycle events. If you're already subscribing to `perform.active_job`, the event shape improvements in 8.1 apply there too. For teams using the [Solid Trifecta instead of Redis](/blog/solid-trifecta-hybrid-redis-rails-8/), Solid Queue emits its own `ActiveSupport::Notifications` events for job enqueuing, execution, and failure - subscribable through the same API, no separate polling loop required. diff --git a/content/blog/ruby-on-rails-performance-optimization-patterns-2026/index.md b/content/blog/ruby-on-rails-performance-optimization-patterns-2026/index.md index 3e23be2c3..f327560a8 100644 --- a/content/blog/ruby-on-rails-performance-optimization-patterns-2026/index.md +++ b/content/blog/ruby-on-rails-performance-optimization-patterns-2026/index.md @@ -117,7 +117,7 @@ We split them. Dedicated worker dynos for Sidekiq. A separate Redis instance for The bigger change: we moved from Sidekiq to [Solid Queue](/blog/solid-queue-vs-sidekiq-complete-comparison/). The client's job patterns were simple - email delivery, webhook processing, report generation - and didn't need Sidekiq Pro's batching or rate limiting features. Solid Queue runs on the database. One fewer infrastructure dependency to manage and monitor. -For jobs that can't afford to be lost, Rails 8.1's [Active Job Continuations](/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/) are worth understanding. They solve the problem where a long-running job crashes halfway through and restarts from zero. +For jobs that can't afford to be lost, Rails 8.1's [Active Job Continuations](/blog/rails-8-1-active-job-continuations-background-jobs/) are worth understanding. They solve the problem where a long-running job crashes halfway through and restarts from zero. ## When NOT to do any of this diff --git a/content/blog/solid-trifecta-hybrid-redis-rails-8/index.md b/content/blog/solid-trifecta-hybrid-redis-rails-8/index.md index 00620a365..63fd116b0 100644 --- a/content/blog/solid-trifecta-hybrid-redis-rails-8/index.md +++ b/content/blog/solid-trifecta-hybrid-redis-rails-8/index.md @@ -137,7 +137,7 @@ Across our last few migrations, the typical post-migration shape looks like this We moved fragment and API response caching to Solid Cache. Cache size went from 2GB (limited by Redis memory) to roughly 50GB on disk. Average read latency increased from 0.3ms to 1.1ms; nobody noticed. P95 page load actually improved because the cache hit rate climbed from 68% to 91% with the larger cache footprint. -Solid Queue took over email delivery, report generation, and scheduled cleanup at 200 jobs/minute steady. [Active Job Continuations](/blog/rails-8-1-active-job-continuations-end-lost-background-jobs/) made the long-running nightly import deploy-safe for the first time. +Solid Queue took over email delivery, report generation, and scheduled cleanup at 200 jobs/minute steady. [Active Job Continuations](/blog/rails-8-1-active-job-continuations-background-jobs/) made the long-running nightly import deploy-safe for the first time. Sidekiq stayed for the payment webhook queue (needs sub-second latency) and the real-time inventory sync (2,000 jobs/minute bursts during peak hours). Redis also stayed for Action Cable, where 1,200 concurrent WebSocket connections power the admin dashboard. diff --git a/content/services/fractional-cto-cost/index.md b/content/services/fractional-cto-cost/index.md index 315ffa75e..9d1266784 100644 --- a/content/services/fractional-cto-cost/index.md +++ b/content/services/fractional-cto-cost/index.md @@ -226,4 +226,4 @@ Our fractional CTO cost-effective solutions support businesses across multiple i Understanding fractional CTO cost is the first step toward getting senior technology leadership for your business. Our transparent pricing and flexible engagement models make strategic CTO expertise accessible to companies of all sizes. -[Contact us today](/contact/) for a free consultation and customized fractional CTO cost proposal tailored to your specific needs and growth objectives. +[Contact us today](/contact-us/) for a free consultation and customized fractional CTO cost proposal tailored to your specific needs and growth objectives. diff --git a/test/unit/marketing_copy_test.rb b/test/unit/marketing_copy_test.rb index 9d4ba1609..b7d2303d2 100644 --- a/test/unit/marketing_copy_test.rb +++ b/test/unit/marketing_copy_test.rb @@ -36,6 +36,10 @@ class MarketingCopyTest < Minitest::Test "content/services/**/*.md", "content/use-cases/**/*.md", "layouts/next/**/*.html", # the v2 rail is template-authored marketing copy on root layouts + # ...and its CONTENT side. Without this the pilots' copy had no source-side + # cover at all: a planted "world-class" in a pilot stub passed the whole + # suite on 2026-08-22 (docs/20-29-testing-qa/20.11). + "content/next/**/*.md", "themes/beaver/layouts/home.html", "themes/beaver/layouts/page/*.html", # Blog CHROME (list hero, section furniture) is a marketing surface even @@ -141,9 +145,13 @@ def test_every_declared_surface_matches_at_least_one_file # pages - which is the whole argument for reading RENDERED output: source # matching sees two files, the reader sees twelve. # - # The remaining 14 live in individual post bodies. Tighten this number every - # time a batch is cleared; a ratchet left slack lets the win regress silently. - RENDERED_BASELINE = 14 + # The remaining hits live in individual post bodies. Tighten this number every + # time a batch is cleared; a ratchet left slack lets the win regress silently + # - which is not hypothetical: this sat at 14 against an actual 11, and those + # three spare hits swallowed a planted "world-class" whole (2026-08-22, + # docs/20-29-testing-qa/20.11). Set it to the measured count, then prove the + # ratchet is exact by dropping it one lower and watching it fail. + RENDERED_BASELINE = 11 def test_rendered_pages_do_not_regress_on_banned_phrases violations = rendered_files.flat_map { |path| rendered_hits(path) }.sort