Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions packages/e-billing/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,21 @@

### Added

- Manual attribution and explicit rematch ([#27](https://github.com/mooxphp/e-billing/issues/27)): nullable indexed `attribution_source` (`auto`|`manual`) on `ebilling_documents`; `SetInvoiceAttributionAction` and `RematchAttributionAction`; Filament `set_attribution` / `rematch` on the invoice detail (rematch also on the list). Rematch resets `review_status` to `parser_created` and re-runs `InvoiceFieldValidator::validate()`. Manual attributions are never overwritten; the automatic path still refuses confirmed/validated documents. Changing attribution after `HumanConfirmed`/`Validated` clears attestation back to `DbValidated`. No scheduled rematch.
- Attribution corroboration via internal `Support\AttributionCorroborator`: after a customer match, name (significant token overlap ≥ `corroboration.name_min_token_length`, legal-form stop words ignored), VAT, country, and buyer address (existence among company billing addresses via `AddressFingerprint` and `corroboration.address_roles`) are checked against master data. Divergences only flag `needs_review`; corroboration never clears or rewrites attribution. Package now requires `moox/address`. Config keys under `e-billing.corroboration` ([#25](https://github.com/mooxphp/e-billing/issues/25)).
- Customer attribution on `EbillingDocument`: nullable `customer_id` FK → `customers` (`nullOnDelete`), `customer()` BelongsTo, and internal `Support\CustomerMatcher` (normalises buyer identifier; looks up `Customer::withTrashed()` by `customer_number`; derives `company_id` from exactly one Company morph assignment). Package now requires `moox/customer` ([#24](https://github.com/mooxphp/e-billing/issues/24)).
- Buyer identifier (`customer_number`, EN 16931 BT-46) now flows from the parser DTO onto the persisted invoice via `ParsedInvoiceMapper` / `InvoiceFactory`. Empty DTO values become `null`; non-empty values are stored unchanged. The field is validated from the invoice like any other field (present ⇒ `parsed`, absent ⇒ configured MoSCoW priority) and is no longer listed under `INVOICE_FIELDS_WITHOUT_PERSISTED_SOURCE` ([#23](https://github.com/mooxphp/e-billing/issues/23)).
- Three-format registry: XRechnung (pure CII XML), ZUGFeRD (hybrid PDF), Factur-X (hybrid PDF). All share one CII generator; XRechnung uses `XRECHNUNG` profile, hybrids use `EN16931`.
- Per-customer format resolution via `EbillingFormatResolver`: reads `companies.data.preferred_ebilling_format`, falls back to `default_format` config (default `zugferd`). Format is frozen on the document at generation time; preference changes affect only future documents.
- XRechnung documents are validated by KOSIT only (no PDF, veraPDF not invoked) and reach `Validated` on pass.
- Hybrid artifact validation runs veraPDF PDF/A-3 checks in `ValidateArtifactJob` when `moox/verapdf` is installed; verdicts persist via `veraPdfValidations()` alongside KoSIT results. When veraPDF is not configured, hybrid validation falls back to KOSIT-only (degraded mode).

### Changed

- `InvoiceFieldValidator` wires attribution corroboration when a customer was matched: name/VAT/country/address field statuses reflect master-data agreement or `needs_review` on divergence; VAT and country compare only when both sides are present; address/country corroboration runs only after a customer match. Name-fallback (no customer match) keeps exact company name/VAT behaviour. Fields without a persisted source (`payment_terms`, `shipping_method`) return `not_applicable`. `parsed` is included in clean validation statuses so auto-Validated remains reachable ([#25](https://github.com/mooxphp/e-billing/issues/25)).
- `CustomerMatcher::isReviewableMatch` also returns true (⇒ `needs_review` on `customer_number`) when the derived `company_id` is null (no company or multi-company assignment). Soft-deleted/inactive customers remain reviewable; attribution is still kept ([#25](https://github.com/mooxphp/e-billing/issues/25)).
- `InvoiceFieldValidator` attributes documents by buyer identifier first: unique `CustomerMatcher` hit sets `customer_id` and derived `company_id` (`db_validated` when active; soft-deleted/inactive still attributed with `customer_number` = `needs_review`). No match or missing identifier leaves `customer_id` null; name fallback via `CompanyNameMatcher` may set `company_id` only. `customer_id` is the identity / visibility gate; `company_id` is reporting-only. Name/VAT/address corroboration followed in [#25](https://github.com/mooxphp/e-billing/issues/25) ([#24](https://github.com/mooxphp/e-billing/issues/24)).
- Company name matching for field validation and format resolution is unified in `CompanyNameMatcher`; both consumers match against the persisted invoice buyer name (whitespace-collapsed, case-insensitive exact match, unique hit only). No longer filters on removed `company_type` / `is_active` columns. `EBillingFormatResolver` no longer reads `bill_data['customer_name']` ([#22](https://github.com/mooxphp/e-billing/issues/22)).
- Reduced cyclomatic/NPath complexity in `ValidateArtifactJob::handle` (named stages: resolve document/inputs, run validations, persist success/failure), `UnitCodeResolver::lookupMaps`, and `DocumentTypeCodeResolver::resolveLabel`. Behaviour unchanged.
- Deduplicated KOSIT/veraPDF validation persistence in `ValidateArtifactJob` via `ArtifactValidationPersister`; supplemental verdicts (veraPDF) stay as closures so the shared seam does not type-hint optional validator packages. No behaviour change.
- Replaced `GenerateXmlJob` / `ValidateXmlJob` / `MergeZugferdPdfJob` with `GenerateArtifactJob` → `ValidateArtifactJob` (generate-first pipeline).
Expand Down
11 changes: 11 additions & 0 deletions packages/e-billing/CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,15 @@ Glossary for the generic e-billing conversion pipeline (`packages/e-billing`). K
- **Validation** — KOSIT/EN16931 conformance check. For pure XML it runs on the `.xml`; for a hybrid PDF it must run on the XML **as embedded in the PDF/A-3** plus PDF-conformance/XMP checks that only exist once the PDF is built.
- **Format choice** — customer-selected (planned: customer portal). Makes format a per-customer/per-invoice input rather than a global setting. Motivates generating the chosen artifact *first*, then validating that artifact.

### Review and correction

- **Value correction** — a reviewer restoring what the **document actually says** after the bound parser mis-read it. The document is the truth; the correction records field, originally parsed value, corrected value, actor and time, and the originally parsed value is never destroyed. It **never** aligns a parsed value to master data: a parsed value is persisted unchanged precisely so a divergence from master data stays visible, and that divergence is the evidence. Permitted only before approval. *Avoid:* edit, fix, override (that is the severity release); "correction" in the accounting sense — there is no cancellation flow.
- **Master-data divergence** — this field on the document and the matched master-data record disagree, or the master record is empty. The queryable, per-field form of a corroboration result — a secondary check of a match that may flag it for review but can never overturn it (ADR 0003). Arises **automatically for every document**, changes nothing, and is evidence that master data is stale, not that the parser erred. **Derived, never stored**: both sides already persist, so it is a query — and it disappears by itself once the master data is fixed. Never the same record as a *Value correction*; they differ in producer, volume, consumer and lifetime. *Avoid:* correction, error, mismatch (reserved for matching).
- **Divergence status** — an annotation on the pair `{ master-data record, field }` — never on a document, because one stale field shows up on every document of that customer. Sparse: only exceptions get one. Three kinds, with deliberately different lifetimes: *seen but unclear*; *reported as fixed upstream* (transient — it resolves itself when the corrected master data arrives, and returns if it never does); *permanently ignored*, which **requires a reason**. *Avoid:* resolving a divergence — you resolve the master data, not the finding.
- **Action type** — what a reviewer's recorded act *means*: **value correction**, **approval**, **rejection**, or **severity release** (dispatching despite a missing recommended field). Load-bearing, not descriptive: only value corrections carry "the parser got this wrong", so only they feed the parser-feedback report. *Avoid:* treating any reviewer edit as a correction.

Reports over these — the parser-feedback report and the master-data divergence report — live in this package rather than in a reporting package or an integrator's private one. See ADR `docs/adr/0002-reports-live-in-the-package-that-owns-their-data.md`.

## Current pipeline (generate-first, dual validation for hybrids)

`GenerateArtifactJob` (build chosen artifact — for ZUGFeRD: decrypt input PDF, merge unencrypted PDF/A-3 with embedded XML) → `ValidateArtifactJob` (KOSIT on the XML that will be delivered; + veraPDF on the PDF for hybrids when installed; SHA-256 hash on pass). When veraPDF is not installed, hybrid validation runs KOSIT-only (degraded mode). See ADR `docs/adr/0001-generate-then-validate-per-format-artifacts.md`.
Expand All @@ -24,6 +33,8 @@ Generate the customer-chosen artifact first (XRechnung XML / ZUGFeRD PDF / Factu
- **Scope now:** the three **CII-based** formats are live. UBL and Peppol are deferred future strategies.
- **Validator stays single.** KOSIT is the only XML validator. XRechnung = KOSIT only. Hybrids = KOSIT + veraPDF.
- **Format binding.** ✅ `EbillingFormatResolver` reads `companies.data.preferred_ebilling_format` → falls back to `default_format` config (default `zugferd`). Format frozen on `ebilling_documents.format` at generation time (freeze = `xml_storage_path` is set). Preference changes affect future documents only.
- **Document identity = Customer.** ✅ `EbillingDocument.customer_id` is the matched buyer (identity / visibility gate), set via buyer identifier through internal `CustomerMatcher`. `company_id` is derived from the customer's Company assignment when unique, or from `CompanyNameMatcher` name fallback when there is no identifier match — reporting only, never an access boundary. Visibility must require `customer_id` **and** gateway `Validated` (do not expose on attribution alone). Soft-deleted/inactive customers, or a null derived company id (no / multi-company), still attribute but mark `customer_number` as `needs_review`. Attribution (#24) then name/VAT/country/address corroboration (#25) are both in scope and done: after a customer match, `AttributionCorroborator` checks significant name-token overlap, VAT/country only when both sides are present, and buyer-address existence among company billing addresses (`AddressFingerprint`); divergences only flag `needs_review` and never clear attribution ([#24](https://github.com/mooxphp/e-billing/issues/24), [#25](https://github.com/mooxphp/e-billing/issues/25)). Manual attribution + explicit rematch (#27) are done: `attribution_source` records `auto` vs `manual`; rematch resets to `parser_created` and re-validates without overwriting a manual source ([#27](https://github.com/mooxphp/e-billing/issues/27)).
- **Company name matching.** ✅ One `CompanyNameMatcher` serves field validation and format resolution (and the no-identifier / no-customer-match company fallback). Both match on the persisted invoice buyer name (whitespace-collapsed, case-insensitive exact equality, unique hit only); `bill_data['customer_name']` is not used. After a customer match, name corroboration uses token overlap (not exact equality); the exact-match path remains for the no-customer-match name fallback.
- **Validation stack (validate the real artifact).** XML conformance → **KOSIT** for *every* format (pure XRechnung XML, or the XML **extracted from the hybrid PDF** via horstoeko `ZugferdDocumentPdfReader`). PDF/A-3 conformance → **veraPDF** (licensed MPL-2.0; commercial-safe as a CLI process). A hybrid passes iff **KOSIT(xml) AND veraPDF(pdf)**; a pure XRechnung passes iff **KOSIT(xml)**.
- **`moox/verapdf` = own package** (mirrors `kosit-validator`/`zugferd` boundary): config + `verapdf:install` command + `VeraPdfService::validate()` + persisted `VeraPdfValidation` model + morph pivot. Generic, no e-billing knowledge; e-billing orchestrates KOSIT+veraPDF. Note: veraPDF ships as an **installer zip** (headless IzPack install → `verapdf` launcher script), *not* a single `java -jar` standalone like KOSIT — the install command is heavier.
- **Failure handling.** Artifact is generated before validation; on failure it is **retained on disk + flagged** (never auto-deleted), validator reports persisted (KositValidation + VeraPdfValidation), `gateway_status` → failed, surfaced in "needs review". **Delivery is validation-gated** — an invalid artifact is never sent even though it exists.
Expand Down
13 changes: 10 additions & 3 deletions packages/e-billing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Moox e-billing orchestrates the Moox e-invoice pipeline: PDF ingestion through a
- Foreign-invoice filtering (non-domestic invoices moved to an ignored mailbox folder)
- MoSCoW field validation and validation scoring on `EbillingDocument`
- Filament `InvoiceResource` for list, filter, and manual review workflows
- Manual customer attribution and explicit re-match from the invoice detail (and rematch from the list)
- Host-bound invoice parser via `InvoiceParserInterface` (no parser ships with this package)

<!--/features-->
Expand Down Expand Up @@ -45,8 +46,10 @@ This package composes the other Moox e-billing packages. Composer requires:

| Package | Role |
| --- | --- |
| `moox/company` | Company FK on `EbillingDocument` |
| `moox/address` | Address fingerprints / company billing addresses for attribution corroboration |
| `moox/company` | Company FK on `EbillingDocument` (reporting-only; derived from customer) |
| `moox/core` | Base model, Filament resource, Moox installer |
| `moox/customer` | Customer FK on `EbillingDocument` (document identity / visibility gate) |
| `moox/invoice` | Invoice domain models (`Invoice`, lines, parties) |
| `moox/jobs` | Job progress traits |
| `moox/kosit-validator` | KoSIT XML validation and audit persistence |
Expand Down Expand Up @@ -83,6 +86,7 @@ Published as `config/e-billing.php`.
| `foreign_invoice` | Foreign-invoice handling (`ignored_folder_name`) |
| `default_customer_country` | Transitional fallback buyer country when the parser derives none (default `DE`); removed in a future master-data phase |
| `supplier` | Central supplier master data copied onto invoices as a snapshot at creation time |
| `corroboration` | Post-attribution master-data checks (never clears `customer_id`): `name_min_token_length`, `name_legal_form_stop_words`, `address_roles` |
| `field_validation` | MoSCoW priority rules for invoice and line fields |
| `morph_relations` | Morph pivot config for KoSIT and veraPDF validations (`kosit_validatables`, `verapdf_validatables`) |

Expand Down Expand Up @@ -159,15 +163,18 @@ Queries `EbillingDocument` rows where `field_validations` is not null and `valid
| `error_message` | `text` | nullable | Last pipeline error |
| `created_at` | `timestamp` | NOT NULL | |
| `updated_at` | `timestamp` | NOT NULL | |
| `company_id` | `uuid` FK | nullable | References `companies.id` (`nullOnDelete`) |
| `company_id` | `uuid` FK | nullable | References `companies.id` (`nullOnDelete`). Reporting only — derived from the matched customer; never an access boundary |
| `customer_id` | `uuid` FK | nullable | References `customers.id` (`nullOnDelete`). Document identity (matched customer); gate visibility on this |
| `attribution_source` | `string` | nullable | `auto` (matcher) or `manual` (operator). Indexed. Manual attributions survive rematch |
| `invoice_id` | `uuid` FK | nullable | References `invoices.id` (`nullOnDelete`) |
| `scope` | `string` | nullable | Tenant / mailbox scope (indexed) |

### Relationships

- `source()` — `MorphTo` (typically `InboxAttachment`)
- `invoice()` — `BelongsTo` `Moox\Invoice\Models\Invoice`
- `company()` — `BelongsTo` `Moox\Company\Models\Company`
- `customer()` — `BelongsTo` `Moox\Customer\Models\Customer`
- `company()` — `BelongsTo` `Moox\Company\Models\Company` (reporting only)
- `kositValidations()` — `MorphToMany` via `kosit_validatables`
- `veraPdfValidations()` — `MorphToMany` via `verapdf_validatables` (hybrid formats when veraPDF is configured)

Expand Down
2 changes: 2 additions & 0 deletions packages/e-billing/composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@
"php": "^8.4",
"moox/company": "dev-main",
"moox/core": "dev-main",
"moox/customer": "dev-main",
"moox/address": "dev-main",
"moox/data": "dev-main",
"moox/invoice": "dev-main",
"moox/jobs": "dev-main",
Expand Down
66 changes: 64 additions & 2 deletions packages/e-billing/config/e-billing.php
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,68 @@
],
],

/*
|--------------------------------------------------------------------------
| Attribution corroboration
|--------------------------------------------------------------------------
|
| After a customer is attributed via buyer identifier, name / VAT / country /
| address are checked against master data. Divergences only flag needs_review;
| they never clear or rewrite the customer attribution.
|
*/

'corroboration' => [

/*
| Minimum length for a name token to count as significant (after
| lowercase + diacritic fold + punctuation strip).
*/
'name_min_token_length' => 4,

/*
| Legal-form words ignored during name token corroboration.
*/
'name_legal_form_stop_words' => [
'gmbh',
'ag',
'kg',
'ohg',
'ug',
'se',
'eg',
'ev',
'ltd',
'limited',
'inc',
'incorporated',
'corp',
'corporation',
'co',
'plc',
'llc',
'llp',
'sarl',
'sa',
'bv',
'nv',
'ab',
'oy',
'as',
'spa',
'srl',
'sas',
],

/*
| Address-assignment pivot flags used when checking that the parsed
| buyer address exists among the matched company's known addresses.
*/
'address_roles' => [
'billing_address',
],
],

/*
|--------------------------------------------------------------------------
| Field validation (MoSCoW)
Expand Down Expand Up @@ -316,14 +378,14 @@
'currency' => 'must', // BT-5

// Buyer — MUST (core identification)
'customer_number' => 'must',
'customer_number' => 'must', // BT-46
'customer_name' => 'must', // BT-44
'customer_address' => 'must', // BG-8
'country' => 'could', // BT-55
'customer_vat_id' => 'should', // BT-48

// Buyer reference
'customer_reference' => 'should', // BT-10
'customer_reference' => 'could', // BT-10
'order_number' => 'should', // BT-13
'order_date' => 'could', // BT-13 date

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ return new class extends Migration
$table->timestamps();

$table->foreignUuid('company_id')->nullable()->constrained('companies')->nullOnDelete();
$table->foreignUuid('customer_id')->nullable()->index()->constrained('customers')->nullOnDelete();
$table->string('attribution_source')->nullable()->index();
$table->foreignUuid('invoice_id')->nullable()->constrained('invoices')->nullOnDelete();

$table->string('scope')->nullable()->index();
Expand Down
Loading
Loading