fix(domains): GTFS stop_sequence repeats instead of row order; strict ICPN format - #386
Merged
Merged
Conversation
…ict ICPN format GTFS-ST004 compared consecutive stop_times rows in file order, so a valid trip listed as stop_sequence 1, 3, 2 raised an error-severity violation. The GTFS Schedule reference only requires stop_sequence to increase along the trip; rows need not be sorted. Once a trip is ordered by stop_sequence it can only fail to increase where a value repeats, so the check now flags rows that repeat an earlier stop_sequence within the same trip (later occurrences, in file order). Rows with a missing trip_id or non-numeric stop_sequence are ignored as before. The rule name is updated to match; the monotonic_sequence func key is kept so existing rule files resolve. is_valid_icpn stripped every non-digit before the length and GS1 mod-10 checks, so free text such as "tel: 036000291452" was accepted. The value (after trimming surrounding whitespace) must now consist of digits optionally grouped by spaces or hyphens; formatted UPC/EAN values such as "0-36000-29145-2" still pass. Closes #319 Closes #321
Contributor
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
FreshData benchmark report —
|
| fixture | n_rows | n_cols | p50 s | p95 s | peak MB | repair % | false-repair % | preserve % | trust | monotonic | export % |
|---|
Authored-code reduction (Metric 6)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
stop_sequence1, 3, 2 raised an error. GTFS only requiresstop_sequenceto increase along the trip, not the rows to be sorted. The check now flags only rows that repeat an earlierstop_sequencewithin the same trip. Rows with a missingtrip_idor non-numericstop_sequenceare still ignored. The rule name intransport/rules.yamlis updated; themonotonic_sequencefunc key is kept so existing rule files still resolve.is_valid_icpn(is_valid_icpn accepts free text that contains a barcode (non-digits are stripped) #321): every non-digit was stripped before the length and GS1 mod-10 checks, so free text such as"tel: 036000291452"was accepted. The trimmed value must now be digits optionally grouped by spaces or hyphens (\d[\d -]*\d). Formatted UPC/EAN values such as0-36000-29145-2still pass.Behaviour notes: out-of-order stop_times rows no longer raise GTFS-ST004; numerically equal spellings (
"1"vs1.0) in one trip count as repeats; ICPN values with other separators or surrounding text are now flagged.Tests
New
tests/test_domain_rules_transport_icpn.py:stop_sequenceflagged (only the later row; other trips unaffected)"1"vs1.0)fd.cleanExisting
tests/domains/test_transport.py(duplicate stop_sequence → GTFS-ST004) andtests/domains/test_media.pyICPN tests pass unchanged.Verification
ruff check .: all checks passedmypy src/freshdata: no issues (202 files)Closes #319
Closes #321