Skip to content

fix(seo): stop marketing pages serving every deep path with a 200 - #3723

Merged
jannikmaierhoefer merged 1 commit into
mainfrom
seo/marketing-catchall-soft-404
Sep 4, 2026
Merged

jannikmaierhoefer merged 1 commit into
mainfrom
seo/marketing-catchall-soft-404

Conversation

@jannikmaierhoefer

@jannikmaierhoefer jannikmaierhoefer commented Sep 4, 2026 •

Copy link
Copy Markdown
Member

Marketing entries are single pages, and the [section] catch-all ignores the slug when resolving them:

const effectiveSlug = isMarketing ? [section] : slug;

So /{marketing}/{anything} rendered the parent page with a 200 instead of 404ing. That is an unlimited surface of indexable duplicates across all 24 marketing pages, and it is live in production today:

$ for p in /kr/total-garbage /cn/xyz /oss-friends/oss-friends; do
    curl -so /dev/null -w "%{http_code}  $p\n" "https://langfuse.com$p"; done
200  /kr/total-garbage
200  /cn/xyz
200  /oss-friends/oss-friends

Google found two of them and ranks the duplicates above the real pages (last 90 days, Search Console):

URL Clicks
/kr/kr 2,321
/kr 910
/cn/cn 2,032
/cn 1,162

Both duplicates already carry a self-referencing canonical pointing at /kr and /cn; it is not being honoured. /japan/japan correctly 404s because /japan is a DEDICATED_MARKETING_SLUG with its own route, which is why this only surfaced on the catch-all pages.

The fix

  1. The page component and generateMetadata now 404 a marketing section that carries a slug. That closes the surface for all 24 pages. The metadata guard matters too — without it a deep path would still advertise a canonical URL for a route that 404s.
  2. /kr/kr and /cn/cn get permanent redirects (plus their .md variants) rather than 404s, so the traffic those two have accumulated moves to the real pages instead of being dropped. Everything else 404s, which is the correct answer for a URL that never existed — redirecting arbitrary paths to a parent is a soft 404 in its own right.

Verification

Against the dev server.

All 24 marketing pages still serve:

/kr 200   /cn 200   /oss-friends 200   /wrapped 200   /about 200   /enterprise 200
/careers 200   /pricing 200   /pricing-self-host 200   /japan 200   /talk-to-us 200
/research 200   /imprint 200   /privacy 200

The two indexed duplicates redirect:

/kr/kr     308 -> /kr
/cn/cn     308 -> /cn
/kr/kr.md  308 -> /kr.md
/cn/cn.md  308 -> /cn.md

Arbitrary deep paths now 404 (all were 200):

/kr/total-garbage 404   /cn/xyz 404   /oss-friends/oss-friends 404
/about/team 404         /careers/anything 404   /wrapped/2024 404

No regression on non-marketing sections, which legitimately have subpages:

/docs/observability/overview 200            /self-hosting/deployment/docker-compose 200
/faq/all/llm-observability 200              /integrations/frameworks/langchain 200
/users/slite 200                            /changelog/2026-08-17-langfuse-v4 200
/handbook/how-we-hire/hiring-process 200    /security/auth 200

Build: npx next build compiled successfully, 966 static pages generated, no route conflicts. node -e 'require("./lib/redirects.js")' loads clean at 464 permanent redirects, no duplicate sources introduced. prettier --write no-ops on both files.

Found while triaging the September Ahrefs crawl; companion to #3716.

🤖 Generated with Claude Code


Note

Low Risk
Routing and redirect changes are scoped to marketing single-page sections; nested docs and other sections are unchanged, with a small SEO-sensitive surface.

Overview
Fixes an SEO duplicate-content bug where any /{marketing-section}/{extra-segments} URL returned 200 with the parent marketing page, because marketing routes ignore the catch-all slug when resolving content.

The catch-all page and generateMetadata now 404 when a marketing section has a non-empty slug, so bogus deep URLs are not indexable and metadata does not claim a canonical for routes that 404.

Permanent redirects send indexed doubled locale paths (/kr/kr, /cn/cn, plus .md variants) to /kr and /cn so existing search traffic is preserved; other invalid marketing deep paths 404.

Reviewed by Cursor Bugbot for commit e175b76. Bugbot is set up for automated code reviews on this repo. Configure here.

Greptile Summary

This PR closes an SEO duplicate-content surface by rejecting nested paths beneath single-page marketing sections and preserving accumulated traffic for the known Korean and Chinese doubled paths.

  • Adds matching page and metadata guards for nested marketing URLs.
  • Adds permanent redirects for /kr/kr, /cn/cn, and their markdown variants.
  • Leaves legitimate nested routes in non-marketing sections unchanged.

Confidence Score: 4/5

The implementation appears safe to merge, with only non-blocking automated regression coverage recommended for the new routing and redirect behavior.

The guards are aligned for marketing deep paths, optional slugs are handled safely, and all redirect destinations resolve through existing markdown and page routing; the remaining concern is that these SEO-sensitive contracts are protected only by manual verification.

Files Needing Attention: app/[section]/[[...slug]]/page.tsx

Prompt To Fix All With AI
### Issue 1
app/[section]/[[...slug]]/page.tsx:43
**Routing Regression Coverage Missing**

This SEO-sensitive routing change is covered only by manual verification. The page and metadata guards must remain aligned, while the four redirect exceptions must continue to work before nested marketing paths return 404. Add route-level tests for a valid marketing root, an arbitrary nested marketing path, both doubled-locale redirects and their `.md` variants, and a valid nested non-marketing page; otherwise a future routing change could silently reopen the duplicate-content surface.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "fix(seo): stop marketing pages serving e..." | Re-trigger Greptile

Context used:

@vercel

vercel Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
langfuse-docs Ready Ready Preview Sep 4, 2026 1:06pm UTC

Request Review

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

@claude review

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

No review was started: this request came from a bot account. Manual reviews can only be requested by someone with write access to this repository. Ask a maintainer to comment @claude review, or have your automation post the comment from a user account with write access.

Tip: disable this comment in your organization's Code Review settings.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good — small, well-scoped SEO fix that mirrors existing patterns in this codebase.

What was reviewed: the new isMarketing && slug.length > 0 guard in both page.tsx and generateMetadata (consistent with MARKETING_SECTIONS in lib/section-registry.ts, no hardcoded slugs); the new localeDoubledPaths202609 redirect array follows the same append-to-permanentRedirects pattern as prior entries like alertsMoveToObservability202608; confirmed next.config.mjs wires permanentRedirects into redirects() so the /kr/kr and /cn/cn (and .md variant) redirects take effect before the .md rewrite pipeline.

Extended reasoning...

Overview

The diff touches two files: app/[section]/[[...slug]]/page.tsx and lib/redirects.js. It adds a notFound() guard (and matching generateMetadata short-circuit) for marketing sections that receive a non-empty slug, fixing a bug where every deep path under a marketing page (e.g. /kr/anything) rendered the parent page with a 200 instead of 404ing. It also adds four permanent redirects to preserve the two already-indexed duplicate URLs (/kr/kr, /cn/cn, plus .md variants) rather than dropping that traffic to a 404.

Security risks

None. This is routing/SEO logic only — no auth, no data handling, no user input beyond URL path segments that were already reachable and rendered (just incorrectly, with a 200 instead of 404). The redirect destinations are static, hardcoded strings, not derived from user input, so there's no open-redirect concern.

Level of scrutiny

Low-to-moderate is appropriate here given the change is small, self-contained, and additive (a guard clause plus a redirect array), but I still traced through lib/section-registry.ts to confirm MARKETING_SECTIONS/MARKETING_SLUGS are used consistently rather than duplicated/hardcoded, and checked next.config.mjs to confirm permanentRedirects from lib/redirects.js is actually wired into the redirects() config and runs before the .md rewrite logic (Next.js redirects execute before rewrites), so the .md redirect entries will actually fire as intended.

Other factors

The change follows established repository conventions exactly (redirect arrays appended to permanentRedirects, guard logic keyed off the section registry's single source of truth) and the PR author's own verification notes (dev-server curl checks across all 24 marketing pages, confirming no regressions on non-marketing deep paths, and a build/redirect-count sanity check) are consistent with what the code does. No CODEOWNERS or nested CLAUDE.md concerns apply to these two files. No test suite regression risk since this is a routing guard with an existing generateStaticParams that already only generates the single marketing param (no test flakiness expected).

Marketing entries are single pages, and the `[section]` catch-all ignored the
slug when resolving them (`effectiveSlug = [section]`). So every
`/{marketing}/{anything}` path rendered the parent page with a 200 rather than
404ing -- an unlimited surface of indexable duplicates across all 24 marketing
pages, live in production:

    /kr/total-garbage         200
    /cn/xyz                   200
    /oss-friends/oss-friends  200

Google found and indexed two of them, and ranks them above the real pages:
`/kr/kr` takes 2,321 clicks against 910 for `/kr`, and `/cn/cn` 2,032 against
1,162 for `/cn` -- the self-referencing canonical on the duplicates is not
being honoured.

- The page component and `generateMetadata` now 404 a marketing section that
  carries a slug, which closes the surface.
- `/kr/kr` and `/cn/cn` (plus their `.md` variants) get permanent redirects
  instead, so the traffic those two have accumulated moves to the real pages
  rather than dropping into a 404.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jannikmaierhoefer
jannikmaierhoefer force-pushed the seo/marketing-catchall-soft-404 branch from 31254ed to e175b76 Compare September 4, 2026 13:02
@jannikmaierhoefer
jannikmaierhoefer added this pull request to the merge queue Sep 4, 2026
Merged via the queue into main with commit f137318 Sep 4, 2026
17 checks passed
@jannikmaierhoefer
jannikmaierhoefer deleted the seo/marketing-catchall-soft-404 branch September 4, 2026 13:30

This branch was successfully deployed

1 active deployment
Preview — e175b763 Deployed Sep 4, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant