Skip to content

Add more spatial extras (length, area, distance_to_filter and intersection_area) - #187

Open
LionelZoubritzky-IGN wants to merge 37 commits into
mainfrom
136/more-spatial_extras
Open

LionelZoubritzky-IGN wants to merge 37 commits into
mainfrom
136/more-spatial_extras

Conversation

@LionelZoubritzky-IGN

Copy link
Copy Markdown
Contributor

This PR introduces a number of small refactors (each in their own commit) and finally adds four new spatial_extras targets: length, area, distance_to_filter and intersection_area. The latter two can only be used when there is a spatial filter.

Requests error when they are absurd, for example asking for area on a geometry which is stored as format: "geometry-point" in the schema. When they are not absurd but still impossible in practice (the same but with "geometry-any" that actually returns a Point for example), the required field is set to null.

Close #136, although we may want to add even more spatial_extra in the future. For the time being, these new spatial_extras allow answering new questions like:

  • how long is Saône river exactly ? (length) -> 515,08 km
  • give me the 5 largest parks in Tours (area) -> La Cousinerie (24ha), Sainte-Radegonde (16ha), ...
  • what is the swimming pool closest to the "Lux" cinema in Caen? (distance_to_filter) -> Piscine du Sivom des Trois Vallées de Mondeville (1.37 km)
  • what is the surface of forest within 5km of Orléans? (intersection_area) -> 1 039,3 ha

@LionelZoubritzky-IGN
LionelZoubritzky-IGN force-pushed the 136/more-spatial_extras branch 2 times, most recently from 1c76f02 to 67960b8 Compare August 12, 2026 14:50
Base automatically changed from 142-gpfschemastore0.2.0 to main September 21, 2026 12:19
@esgn

esgn commented Sep 22, 2026

Copy link
Copy Markdown
Member

Better to isolate refactors in single PRs than to mix it with a thematic PR. See the new pull_request_template.md

@esgn
esgn self-requested a review September 22, 2026 08:29
Comment thread src/wfs/attributeFilter.ts Outdated
Comment thread src/wfs/spatialExtras.ts Outdated
case "travel_time": {
const spatialFilterGeometry = spatialFilterToGeometry(spatialFilter, resolvedGeometryRef)
const filterPolygons = geometryToPolygons(spatialFilterGeometry);
const inter = filterPolygons ? intersect(featureCollection([feature(filterPolygons), feature(geo)])) : null;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

intersect() is re-run against the full reference geometry for every returned feature, and spatialFilterToGeometry + geometryToPolygons (lines 126-127) are recomputed on every iteration.

Measured with a real reference (département 35, a MultiPolygon with 60,697 vertices) against real communes: 431 ms per feature, i.e. 43 s of blocked event loop at limit=100 — the default — and ~36 min at limit=5000. That is exactly the scenario of the integration test this branch adds (communes × département). Nothing bounds it upstream: the CQL goes out in a POST body, so no URL length limit caps the size of the reference geometry.

Under http-stream transport this freezes every session, and the AbortControllers armed in helpers/http.ts:157 during the freeze all fire at once when it lifts → cascading 504s for unrelated callers.

Measured fix (×217, areas bit-identical — bboxClip, bbox and dropEmptyRings are already available in this file):

case "intersects_feature":
case "travel_time": {
  const spatialFilterGeometry = spatialFilterToGeometry(spatialFilter, resolvedGeometryRef);
  const filterPolygons = geometryToPolygons(spatialFilterGeometry);
  if (!filterPolygons) return null;
  // Whatever lies outside the feature's bbox cannot intersect it, so clipping the
  // reference first leaves the result unchanged while polyclip only ever processes
  // the neighbouring vertices.
  const clippedFilter = dropEmptyRings(bboxClip(filterPolygons, bbox(geo)).geometry);
  if (!clippedFilter) return null;
  const inter = intersect(featureCollection([feature(clippedFilter), feature(geo)]));
  return inter === null ? null : inter.geometry;
}

⚠️ The only sound bbox shortcut is "disjoint bboxes → 0". The symmetric one — "feature bbox contained in the filter bbox → area(geo)" — is wrong: a département is concave.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/wfs/spatialExtras.ts Outdated

if (requires_distance_to_filter) {
try {
const filterCentroid = spatialFilterToCentroid(spatialFilter, resolvedGeometryRef);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

=> The filter geometry and its centroid are recomputed for every feature.

spatialFilterToCentroid / spatialFilterToGeometry depend only on input and resolvedGeometryRef, both constant across the whole collection — yet they are called from the per-feature loop in transformFeatureCollectionResponse (response.ts:94-100).

Measured: centroid(département 35) costs 2.79 ms per call, i.e. ~93% of the 3.0 ms per feature spent on distance_to_filter → 15 s of pure waste at limit=5000 for an identical result. Same pattern for the circle(64) of dwithin_point and the bboxPolygon of bbox_filter.

Suggestion: a prepareSpatialContext(input, resolvedGeometryRef) computed once in transformFeatureCollectionResponse (filter geometry, its polygons, its centroid) and passed down to deriveFromGeometry. It is also the natural home for the intersection_area performance fix.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/wfs/spatialExtras.ts Outdated
return { type: "Point", coordinates: [spatialFilter.lon, spatialFilter.lat] };
}
case "intersects_feature":
case "travel_time":

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

case "travel_time" falls into the centroid(spatialFilterToGeometry(...)) branch, so it measures from the centroid of the isochrone even though travelTimeFilterSchema already carries lon/lat, described as "Longitude / Latitude du point de départ" (schema.ts:95-96). That is exact, free, and it is what a user means by "distance to the travel-time filter".

Suggestion : move case "travel_time" into the dwithin_point / intersects_point branch and return { type: "Point", coordinates: [spatialFilter.lon, spatialFilter.lat] }. Then align the description at schema.ts:180, which currently says "le centroïde du filtre" without distinguishing the cases.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8f8f5c5 (I forgot to commit it separately)

Comment thread src/wfs/spatialExtras.ts Outdated
if (requires_intersection_area) {
try {
const intersection = intersectionAreaWithSpatialFilter(geo, spatialFilter, resolvedGeometryRef);
ret.intersection_area = intersection ? area(intersection) : 0;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

=< intersection_area returns 0 where the published contract promises null.

intersectionAreaWithSpatialFilter returns null for three distinct situations — "nothing areal"
(l. 113-115), "point filter" (l. 119) and "empty intersection" and this line collapses all three
into 0.

The entry point is real: getGeometryTypeDimension("GeometryCollection") returns "?" (properties.ts:81), which short-circuits the validation loop (properties.ts:153), so the 29 geometry-any collections in the catalog (wfs_du, wfs_sup, wfs_scot) pass with no dimension check at all.

Suggestion: have intersectionAreaWithSpatialFilter return number | null, so the distinction is carried by the callee instead of being reconstructed by the caller:

// `null` = not computable (nothing areal in the geometry)
// `0`    = computed, no overlap (including a point filter: a point has an empty interior)

The intersects_point → 0 semantics are preserved: that value is geometrically correct.

@LionelZoubritzky-IGN LionelZoubritzky-IGN Sep 22, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not completely clear on what we actually want in the end. What makes most sense to me is the following:

  1. If, for any reason, we cannot compute what is asked (for example, the centroid of a feature whose WFS did not return a geometry), return null for that specific spatial_extra.
  2. if the LLM already has the information at query-time that should let it know that the query itself is asking for something trivial or absurd (for example, the intersection area with a point filter), error immediately with an explicit message stating why you should not ask for that.

In the case of area intersection here, I see the following possibilities:

  • The filter is not 2D and the LLM should know that: throw an error. This should the behavior for intersects_point_filter only. It's not the case currently.
  • The filter is not 2D but it is not trivial to see (the LLM does not necessarily have this information): return intersection_area=0: simple and correct. return intersection_area=null. This should be the behavior for a non-2D intersects_feature_filter. see next comment
  • The filter is 2D and the intersection with the geometry is empty: return intersection_area=0. This can happen with bbox_filter because of Document how bbox_filter works #196 for example.

@esgn what do you think?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, after writing #187 (comment), I think it's just easier to put intersection_area to 0 in the second case, instead of intersection_area=null. We should reserve null for error cases (for instance, there was an internal error when computing the intersection).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tentative in c972383, let me know

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First in schema.ts it is written

    "Si l'élément à calculer est trivial (bbox d'un point, aire d'une géométrie linéaire), une erreur indiquera comment corriger la requête.\n"+
    "Si une valeur n'est pas calculable pour une autre raison, elle sera remplacée par `null` dans la réponse.

So :

  • case 1 : the mismatch is knowable before the query, from the catalog => error, the whole request fails
  • case 2 : the mismatch only shows up on that one feature => null
  • case 3 : the computation ran => a number, 0 included

Examining the current version of the code :

Extra Returns null should mean State
centroid {lon, lat} geometry missing or unreadable ok
bbox 4-value array geometry missing, unreadable or empty off: an empty geometry yields [null,null,null,null], a third shape that is neither a bbox nor null
length a number non-linear geometry, so 0 only ever means a line of zero length off: returns null for any GeometryCollection, even one made only of LineStrings, where the length is computable and area does walk collections
area a number non-areal geometry, so 0 only ever means a polygon of zero area off: an empty Polygon yields 0
distance_to_filter a number geometry or filter unreadable, so 0 only ever means the feature covers the reference point ok
intersection_area a number feature or filter not areal, so 0 only ever means genuinely no overlap off: five cases out of six yield 0

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does the brief in #136 makes things clearer ?

Comment thread src/wfs/schema.ts Outdated
.describe("Liste ordonnée des critères de tri."),
}))
.merge(gpfGeometryExtraInputSchema)
.merge(gpfGetFeaturesGeometryExtraInputSchema.required()) // must never be optional to ensure queryIsGetFeaturesInput correctness

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.required() is a no-op here: it only unwraps ZodOptional, and this is a ZodEffects(ZodDefault(ZodArray)). Verified on the published schema required === ["typename"].

So the comment credits the wrong construct: .default([]) alone guarantees the field.

Drop both .required() calls (here and l. 405), move the comment, and pin the invariant with expect(gpfGetFeaturesInputSchema.parse({ typename: "X" })).toHaveProperty("spatial_extras").

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/wfs/schema.ts Outdated
return `Éléments calculés depuis la géométrie à renvoyer pour ${target}. Peut inclure ${allowedExtrasDocNames}, aucun par défaut.\n`+
`${SPATIAL_EXTRAS_BASE_DESCRIPTION_LINES.join("\n")}\n`+
optionalFilterLine+
"Si une valeur n'est pas calculable, elle sera remplacée par `null` dans la réponse.";

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This promises "…elle sera remplacée par null dans la réponse", but validateSpatialExtras (properties.ts:134-160) now fails the whole request instead. And the bbox-on-geometry-point rejection (70 collections) is documented nowhere, unlike length/area at l. 154-155.

Restrict the null sentence to what is not computable per feature, and state that a dimension mismatch is a hard error.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's like for #187 (comment): I think we need to distinguish trivial queries (like bbox on a geometry-point, or a dimension mismatch), which should error, and failures to compute, which should be marked with a null in the answer, so the LLM knows that something went wrong.

The edge-case is when we detect a "trivial" case only after running a WFS query, typically for a intersects_feature_filter where the feature ends up not having the correct dimension. I think null is the correct answer here.

If you agree with this, I can document it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See c972383

@esgn

esgn commented Sep 22, 2026

Copy link
Copy Markdown
Member

I'm starting a review commit by commit to try and not miss anything.

Regarding bbc29e7

In src/tools/GpfGetFeatureByIdTool.ts:89

  • Now that executeGetFeatureById takes a GpfGetFeatureByIdInput, and validatedInput is one, this field-by-field projection only existed to feed the intermediate type this commit removes => return executeGetFeatureById(validatedInput);
  • The schema is .strict(), so keeping the projection just adds drift risk a field added to the schema later would be silently dropped here, with no tsc error.

Comment thread src/wfs/byId.ts Outdated
@esgn

esgn commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

Now commit 2 : 026c561

  1. I get the idea. Dealing with // TODO: use a more solid guard that will not break at the first contract change in queryPreparation.ts. But I tend to think limit would be a better choice than spatial_extras. What will happen in gpf_get_features_layer if we use spatial_extras as a discriminant ?
  2. .merge(gpfGeometryExtraInputSchema.required()) Are you sure this required() is doing the job ? Is it not this part that guarantee spatial_extras presence
const gpfGeometryExtraInputSchema = z.object({
  spatial_extras: z
    .array(z.enum(GPF_GET_FEATURES_SPATIAL_EXTRAS))
    .default([]) <= in here

Comment thread src/wfs/catalog.ts
export class FeatureTypeNotFoundError extends Error {
constructor(name: string) {
super(`Le type '${name}' est introuvable`);
super(`Le type '${name}' n'existe pas. Utilise le tool gpf_search_types pour trouver les types existants et pertinents pour ta recherche.`);

@esgn esgn Sep 22, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wrapper is redundant now (and being a catch-all, it also staples the hint onto network failures and overwrites error.name) => suggest dropping it.

Nit: these two new messages are the only Utilise in src/ (2 vs 15 Utiliser).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see what wrapper you mean to drop here?

@esgn

esgn commented Sep 22, 2026

Copy link
Copy Markdown
Member

Commit 3 : bdeadda

Done in previous comments.

Comment thread src/wfs/spatialFilter.ts
travel_time_filter: "travel_time",
} as const satisfies Record<(typeof GPF_GET_FEATURES_SPATIAL_FILTER_KEYS)[number], string>;
const FILTER_KEY_TO_OPERATOR = Object.fromEntries(
GPF_GET_FEATURES_SPATIAL_FILTER_KEYS.map((key) => [key, key.replace(/_filter$/, "")]),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup => I checked the derived SpatialFilter against the hand-written union it replaces
(discriminated field access, literal operator, never exhaustiveness) and it is strictly equivalent,
with key order unchanged.

One thing it makes load-bearing: key.replace(/_filter$/, "") assumes every filter key ends in _filter. If one ever doesn't, the operator silently becomes the whole key, SpatialFilterEntry yields never so that member drops out of the union, and the default-less switch in queryPreparation.ts:198 returns cqlFilter: undefined => a query with no spatial filter at all, and no error.

Cheap guard: default: { const _: never = spatialFilter; } in that switch.

@LionelZoubritzky-IGN LionelZoubritzky-IGN Sep 23, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f3b630b

I think it's a bit redundant with the identical guard already present in wfs/spatialExtras.ts, but it doesn't hurt either.

@esgn

esgn commented Sep 22, 2026

Copy link
Copy Markdown
Member

Commit 4 : 0892147

Done in previous comment

Comment thread src/helpers/geojson.ts Outdated
* @param value Unknown feature geometry value.
* @returns `true` when the value looks like a GeoJSON geometry object.
*/
export function isGeometryLike(value: unknown): value is Geometry {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good deduplication (the two copies were byte-identical).
value is Exclude<Geometry, GeometryCollection> is exactly what the body checks.

But the return type is new, and value is Geometry is not what the body checks. It requires coordinates, so it rejects a valid GeometryCollection (which carries geometries instead), and it never looks at what coordinates holds, so { type: "CustomType", coordinates: null } passes and gets narrowed to Geometry. value is Exclude<Geometry, GeometryCollection> would describe the body exactly.

➡️ Related: test/wfs/geometry.test.ts drops "should throw for a completely unknown type" because "CustomType" no longer typechecks, but that is the very case this guard still lets through from remote data. Worth keeping with a cast.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I narrowed the return type to Exclude<Geometry, GeometryCollection>, which has no downstream effect since callers widen it back to Geometry, and I don't see the point in threading the precise GeometryCollection exclusion everywhere. But the test is relevant, so I added it back in b56ddaf

Comment thread src/gpf/navigation.ts Outdated
@esgn

esgn commented Sep 22, 2026

Copy link
Copy Markdown
Member

Commit 5 : fba3ca3

Done in previous comments

Comment thread src/wfs/spatialCql.ts Outdated
@@ -52,9 +54,9 @@ export function compileDwithinSpatialFilter(geometryName: string, spatialFilter:
* Compiles an `intersects_feature` spatial filter once the reference geometry is already serialized.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The summary line is now wrong: this function is what serializes the geometry, so it is no longer "already serialized" when it arrives. The @param below was updated, this line was not.

 * Compiles an `intersects_feature` spatial filter from the resolved reference geometry.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/wfs/referenceGeometry.ts Outdated
@esgn

esgn commented Sep 22, 2026

Copy link
Copy Markdown
Member

Commit 6 : 8b1d2d5

Done in previous comments

LionelZoubritzky-IGN and others added 11 commits September 29, 2026 17:51
Co-authored-by: Emmanuel S. <5435148+esgn@users.noreply.github.com>
…erlap

- intersection_area: null when the feature or the filter is not areal, or when
  the filter geometry could not be prepared; 0 only when both are areal and
  do not overlap
- every extra is null on a missing or empty geometry
- length sums the linear parts of a GeometryCollection; area of an empty
  Polygon is null
- measures sum the parts of a GeometryCollection, overlaps included (JTS)
- state the contract in deriveFromGeometry and in the LLM-facing descriptions
- regenerate docs/mcp-tools.md
…hema

- move "requires a spatial filter" from compileQueryParts to the zod
  refinement: invalid-tool-params instead of execution-error, raised before
  any catalog or network call
- reject distance_to_filter with intersects_point_filter (always 0)
- list only the compatible filters in the error message
- state the rule in the spatial_extras description, regenerate docs/mcp-tools.md
- move the related tests from compileQueryParts to the tool
buildPropertyNameWithGeometry always passes an empty spatial_extras list, and
the `!spatial_extras` guard let it through: the by-id layer tool and the proxy
by-id resolve then read the geometry format and threw on an unlisted or missing
one. Return early on an empty list, so already minted by-id layer URLs do not
depend on the catalog format values.
…turned features

- tool and spatial_extras descriptions: extras are computed after the query, on
  the returned features only, and cannot be used in where/order_by; a ranking
  or a sum needs numberReturned == numberMatched
- "property does not exist" error: explain when the name is a spatial extra
  (message only, a real catalog column with that name is still accepted)
- centroid: may fall outside a concave geometry, so an intersects_point_filter on it may return neither the feature nor what contains it
- distance_to_filter_center: distance from the filter center to the closest point of the feature, 0 when the feature contains the center; list the center of each filter
- intersection_area: null when the feature or the intersects_feature_filter reference has no areal part, 0 when they do not overlap; name the areal filters (box, disc, isochrone, reference)
- rankings and sums: raise limit or restrict the spatial filter; for the N closest, use dwithin_point_filter with distance_to_filter_center, sort, and widen distance_m when fewer than N come back
- catalog check: report every incompatible extra in a single error, in the order of the request, instead of one per call
- bbox on a Point layer: fix the double period and the missing comma in the error message
- regenerate docs/mcp-tools.md
…al extra hint

The hint is shared by every tool that validates property names. From gpf_get_feature_by_id, which has its own spatial_extras, it pointed to another tool instead of the same call's spatial_extras; from gpf_count_features and the layer tools, it suggested a tool switch that cannot help. Say only that the name is computed via spatial_extras and is not usable in select, where or order_by.
- bbox: [ouest, sud, est, nord] in WGS84 lon/lat, matching the west, south, east and north fields of bbox_filter
- regenerate docs/mcp-tools.md
…atures

gpf_get_features returns several features, so "La géométrie de l'objet" read as if a single one came back. Say "La géométrie renvoyée", as the other catalog errors of the same check already do.
…erences rule out

- intersection_area needs an areal intersects_feature reference: a linear, MultiPoint or Point reference is now an error instead of null for every feature
- distance_to_filter_center is always 0 from a reference reduced to a single position (a Point, or a MultiPoint whose points are all the same): now an error
- check the reference against its catalog format before fetching it, then against its fetched geometry, which alone settles a geometry-any type or a single-position MultiPoint
- run the queried layer check before any network call (reference or isochrone fetch), so its errors now come before select, where and order_by errors
- intersection_area description: the reference must be areal; regenerate docs/mcp-tools.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add geometry metadata in gpf_wfs_get_features response

3 participants