Skip to content

About

Reproducible measurement of Germany's public eForms procurement feed: cache-validator defect, three schema generations, seven namespace prefixes, 38% missing procedure identifiers.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

openeforms-probe

A reproducible measurement of Germany's public procurement notice feed, and the groundwork for OpenEForms — a JavaScript/TypeScript implementation of the eForms standard.

Every working eForms library today is written in Java. This repository establishes, with numbers rather than assertions, what a client for this feed actually has to cope with before it can read a single notice.

Standard library only. No API key. No account. The data is CC0.

python3 probe.py --month 2026-08

Why

Since October 2023, § 10a VgV requires German contracting authorities to transmit above-threshold notices through the Datenservice Öffentlicher Einkauf. The resulting feed is published under CC0, covers all sixteen federal states, and runs at roughly 23,000 notices a month.

In practice almost nobody reads it except commercial tender-alert services, which charge for access to public-domain data. The obstacle is not the licence. It is that consuming eForms means running the EU's Java toolkit, which is not a realistic dependency for a two-person software firm, a newsroom or a municipal IT team working in another ecosystem.

Before writing a library, it is worth measuring precisely how messy the real data is.

Findings

Measured against August 2026: 22,892 notices, 0 parse errors. Full report in findings/2026-08.md; machine-readable in findings/2026-08.json.

1. A cache validator that silently suppresses data

The export returns one ETag per format, identical for every day and every month requested. It does not identify the resource.

The consequence is not cosmetic. Presenting a validator harvested from one day when requesting a completely different day that has never been downloaded:

Request Status Bytes
pubDay=2026-06-03 with If-None-Match from 2026-09-09 304 0
pubDay=2026-06-03, no conditional header 200 5,036,628

Any client that caches by ETag, and any shared HTTP cache or CDN in front of it, will report success and return nothing. An incremental sync built the obvious way loses data without erroring.

2. Three schema generations in parallel

CustomizationID Notices Share
eforms-de-2.1 13,561 59.2%
eforms-sdk-0.1 9,293 40.6%
eforms-sdk-1.0 38 0.2%

A parser targeting only the current German profile silently drops two fifths of the feed.

3. Seven namespace prefixes for the same elements

(no prefix), ns7:, ns8:, ns9:, cn:, can:, pin:.

Prefix-based XPath or string matching, which is how most quick integrations are written, breaks on this feed without raising an error. A parser has to resolve namespaces properly.

4. Version tokens in two incompatible styles

Zero-padded 01–09 and unpadded 1–10 both occur, in filenames and in cbc:VersionID alike. 01 and 1 denote the same version and compare unequal as strings. Any composite key of identifier plus version has to normalise first.

5. No procedure identifier on 38% of notices

8,762 of 22,892 notices carry no ContractFolderID — the field that links a prior information notice to its contract notice and eventual award.

Profile Identifier present Notices
eforms-de full profile yes 13,426
eforms-de full profile no 135
eforms-sdk-0.1 minimal profile yes 666
eforms-sdk-0.1 minimal profile no 8,627

Below-threshold notices, published under the minimal profile, are 93% unlinkable. Any claim to reconstruct procurement lifecycles has to be scoped honestly to the above-threshold half.

Lineage is also not served by the export itself: of 12,999 distinct contract folders in this month, only 863 appear more than once, because a procedure's notices are spread across many monthly files.

6. Legal-basis codes with inconsistent spelling

VOB/A alongside vob-a; UVgO alongside uvgo. Absent entirely on 9,299 notices.

7. No search, and no browser access

The published OpenAPI document declares exactly one path, /api/notice-exports, with three parameters: pubDay, pubMonth, format. There is no query, filter, subscription or webhook facility — the feed is bulk-only.

No Access-Control-* headers are returned, so ingest cannot run in a browser and needs a server-side component.

What the probe does

Check What it establishes
transport Cache validators across six resources, a cross-resource conditional request, CORS, and what the OpenAPI document actually advertises
corpus Schema generations, namespace prefixes, version tokens, procedure identifiers, lineage density and legal-basis codes over one month
python3 probe.py --month 2026-08        # both
python3 probe.py --transport-only       # HTTP behaviour only, seconds
python3 probe.py --month 2026-07 --skip-transport

The monthly export is cached in cache/ after first download (94 MB for August 2026) and is not committed. Reports are written to findings/.

Reproducing

Python 3.9 or newer. No dependencies.

git clone https://github.com/NodrrS/openeforms-probe
cd openeforms-probe
python3 probe.py --month 2026-08

Figures will drift as volumes change and as eForms-DE 2.2 and 3.0 roll out. The transport findings are expected to hold until the service fixes them; if a run shows the conditional request no longer suppressing data, that is good news and worth a pull request against this README.

Next

These measurements define the test corpus and the correctness targets for OpenEForms: a namespace-aware parser that normalises across generations, an EFX interpreter generated from the SDK's openly licensed ANTLR grammar, Schematron validation via SchXslt and SaxonJS checked against the KoSIT validator as an oracle, and a persistent lineage index with its coverage limits stated rather than hidden.

Sources

Licence

MIT. See LICENSE. The measured data is CC0 and belongs to its publisher.

About

Reproducible measurement of Germany's public eForms procurement feed: cache-validator defect, three schema generations, seven namespace prefixes, 38% missing procedure identifiers.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages