Skip to content

[0.4.0] validate OSW input and output, preserve zero-length edges and co-located nodes, remove warnings - #57 - #58

Merged
sujata-m merged 2 commits into
mainfrom
develop
Aug 6, 2026
Merged

[0.4.0] validate OSW input and output, preserve zero-length edges and co-located nodes, remove warnings - #57#58
sujata-m merged 2 commits into
mainfrom
develop

Conversation

@sujata-m

@sujata-m sujata-m commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

PR summary

Adds validation at both ends of conversion, fixes how degenerate and co-located geometry is carried between formats, and removes the warning subsystem.

Validation

  • OSW input is validated with python-osw-validation before OSW→OSM starts. Rejected input returns status=False with the validator's per-feature issues, each naming the file and feature index; repeated messages are collapsed.
  • OSW output is validated before OSM→OSW reports success, using the same settings that produced it.
  • OSM input is rejected when any node coordinate carries more decimal places than coordinate_precision, for both XML and PBF. No tolerance: one digit over the limit is rejected. Coordinates are read as exact decimal strings, never floats.
  • coordinate_precision and allow_zero_length_lines are passed to the validator, so input, output and conversion are judged by one set of rules.

Error messages

  • A corrupt OSM file returns invalid input file, the OSM file is corrupted and could not be read. The problem is at line N, column M. instead of printing a traceback and surfacing raw parser text. Reads identically whether the failure surfaces during validation or during conversion.
  • Unreadable OSW archives (missing, not a zip, no .geojson inside) are reported in plain language rather than OS error numbers, and echo only the file name — previously the caller's absolute path leaked into the message.

Geometry

  • Zero-length edges survive both directions when allow_zero_length_lines is set: OSW e1 = (n0, n0) becomes OSM w1 = [n0, n0], and back again. With the setting off they collapse to a single point. Required two fixes — restoring the nd ref ogr2osm drops from a single-node way, and a self-loop guard in OSMGraph.simplify() that was splicing the node into its own way and deleting both.
  • Co-located points are no longer merged. Two OSW point features at one coordinate stay two OSM nodes even with identical tags. Way vertices still merge onto node features, so edges stay connected and kerb nodes stay attached.
  • Edge endpoints are restored from _u_id/_v_id. ogr2osm resolves way vertices by coordinate, so an edge between two co-located nodes collapsed onto whichever node was created first. The references are recorded while features are read and applied before renumbering.

Breaking changes

  • Response.warnings and helpers/warnings.py are removed. Callers reading response.warnings will get an AttributeError. Coordinate precision is now enforced at the input gate rather than reported after the fact.
  • allow_zero_length_lines now defaults to True, matching the validator default. Set it to False for the previous behaviour.

New config: validate_input and validate_output, both defaulting to True, each with a Formatter(...) kwarg to opt out.

Testing

311 tests pass. Every OSM fixture in the repo was run through conversion and its output validated before enabling output validation by default — all compliant, so no existing test needed changing.

sujata-m and others added 2 commits August 6, 2026 13:22
… co-located nodes, remove warnings

## PR summary

## What

Adds validation at both ends of conversion, fixes how degenerate and co-located
geometry is carried between formats, and removes the warning subsystem.

### Validation
- **OSW input** is validated with `python-osw-validation` before OSW→OSM starts.
  Rejected input returns `status=False` with the validator's per-feature `issues`,
  each naming the file and feature index; repeated messages are collapsed.
- **OSW output** is validated before OSM→OSW reports success, using the same
  settings that produced it.
- **OSM input** is rejected when any node coordinate carries more decimal places
  than `coordinate_precision`, for both XML and PBF. No tolerance: one digit over
  the limit is rejected. Coordinates are read as exact decimal strings, never floats.
- `coordinate_precision` and `allow_zero_length_lines` are passed to the validator,
  so input, output and conversion are judged by one set of rules.

### Error messages
- A corrupt OSM file returns `invalid input file, the OSM file is corrupted and
  could not be read. The problem is at line N, column M.` instead of printing a
  traceback and surfacing raw parser text. Reads identically whether the failure
  surfaces during validation or during conversion.
- Unreadable OSW archives (missing, not a zip, no `.geojson` inside) are reported
  in plain language rather than OS error numbers, and echo only the file name —
  previously the caller's absolute path leaked into the message.

### Geometry
- **Zero-length edges** survive both directions when `allow_zero_length_lines` is
  set: OSW `e1 = (n0, n0)` becomes OSM `w1 = [n0, n0]`, and back again. With the
  setting off they collapse to a single point. Required two fixes — restoring the
  `nd` ref ogr2osm drops from a single-node way, and a self-loop guard in
  `OSMGraph.simplify()` that was splicing the node into its own way and deleting both.
- **Co-located points are no longer merged.** Two OSW point features at one
  coordinate stay two OSM nodes even with identical tags. Way vertices still merge
  onto node features, so edges stay connected and kerb nodes stay attached.
- **Edge endpoints are restored from `_u_id`/`_v_id`.** ogr2osm resolves way
  vertices by coordinate, so an edge between two co-located nodes collapsed onto
  whichever node was created first. The references are recorded while features are
  read and applied before renumbering.

## Breaking changes

- `Response.warnings` and `helpers/warnings.py` are removed. Callers reading
  `response.warnings` will get an `AttributeError`. Coordinate precision is now
  enforced at the input gate rather than reported after the fact.
- `allow_zero_length_lines` now defaults to `True`, matching the validator default.
  Set it to `False` for the previous behaviour.

New config: `validate_input` and `validate_output`, both defaulting to `True`,
each with a `Formatter(...)` kwarg to opt out.

## Testing

311 tests pass. Every OSM fixture in the repo was run through conversion and its
output validated before enabling output validation by default — all compliant, so
no existing test needed changing.

15 fixtures added, tracked under `tests/unit_tests/test_files/input_validation/`,
covering valid and invalid OSW and OSM input, corrupt XML, non-standard tags,
duplicate points, and six zero-length scenarios across both directions.
[0.4.0] validate OSW input and output, preserve zero-length edges and co-located nodes, remove warnings
@sujata-m
sujata-m requested review from MashB and susrisha August 6, 2026 09:37
@sujata-m
sujata-m merged commit be8e9c7 into main Aug 6, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants