Skip to content

fix(domains): GTFS stop_sequence repeats instead of row order; strict ICPN format - #386

Merged
kevincostner17 merged 1 commit into
mainfrom
fix/domain-rules-transport-icpn
Sep 15, 2026
Merged

kevincostner17 merged 1 commit into
mainfrom
fix/domain-rules-transport-icpn

Conversation

@kevincostner17

Copy link
Copy Markdown
Contributor

Summary

  • GTFS-ST004 (GTFS-ST004 flags valid stop_times whose rows are not sorted by stop_sequence #319): the check compared stop_times rows in file order, so a valid trip listed as stop_sequence 1, 3, 2 raised an error. GTFS only requires stop_sequence to increase along the trip, not the rows to be sorted. The check now flags only rows that repeat an earlier stop_sequence within the same trip. Rows with a missing trip_id or non-numeric stop_sequence are still ignored. The rule name in transport/rules.yaml is updated; the monotonic_sequence func key is kept so existing rule files still resolve.
  • is_valid_icpn (is_valid_icpn accepts free text that contains a barcode (non-digits are stripped) #321): every non-digit was stripped before the length and GS1 mod-10 checks, so free text such as "tel: 036000291452" was accepted. The trimmed value must now be digits optionally grouped by spaces or hyphens (\d[\d -]*\d). Formatted UPC/EAN values such as 0-36000-29145-2 still pass.

Behaviour notes: out-of-order stop_times rows no longer raise GTFS-ST004; numerically equal spellings ("1" vs 1.0) in one trip count as repeats; ICPN values with other separators or surrounding text are now flagged.

Tests

New tests/test_domain_rules_transport_icpn.py:

Existing tests/domains/test_transport.py (duplicate stop_sequence → GTFS-ST004) and tests/domains/test_media.py ICPN tests pass unchanged.

Verification

  • ruff check .: all checks passed
  • mypy src/freshdata: no issues (202 files)
  • Python 3.12: 5063 passed, 6 skipped, 12 deselected
  • Python 3.9: 5059 passed, 10 skipped, 12 deselected

Closes #319
Closes #321

…ict ICPN format

GTFS-ST004 compared consecutive stop_times rows in file order, so a valid
trip listed as stop_sequence 1, 3, 2 raised an error-severity violation.
The GTFS Schedule reference only requires stop_sequence to increase along
the trip; rows need not be sorted. Once a trip is ordered by stop_sequence
it can only fail to increase where a value repeats, so the check now flags
rows that repeat an earlier stop_sequence within the same trip (later
occurrences, in file order). Rows with a missing trip_id or non-numeric
stop_sequence are ignored as before. The rule name is updated to match;
the monotonic_sequence func key is kept so existing rule files resolve.

is_valid_icpn stripped every non-digit before the length and GS1 mod-10
checks, so free text such as "tel: 036000291452" was accepted. The value
(after trimming surrounding whitespace) must now consist of digits
optionally grouped by spaces or hyphens; formatted UPC/EAN values such as
"0-36000-29145-2" still pass.

Closes #319
Closes #321
@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 6fdb8178-a1d9-496c-8b13-6962375f19e5


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

FreshData benchmark report — performance

  • freshdata: ?
  • python: ?
  • platform: ?
fixture n_rows n_cols p50 s p95 s peak MB repair % false-repair % preserve % trust monotonic export %

Authored-code reduction (Metric 6)

@kevincostner17
kevincostner17 merged commit b69b22c into main Sep 15, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant