Emit generated training data at GROBID's column layout - #698
Merged
Conversation
generate_delft_data appended the label unconditionally, so for the three models whose GROBID corpus writes the label into the feature vector's label slot - citation, name and affiliation_address - it emitted the slot placeholder as well as the label, one column too wide. get_clean_features_mask in trainer-delft drops documents whose width differs from the first seen and logs one line, so a run given both corpora silently kept one and discarded the other. The layout is per-model empirical fact, so it is recorded as data rather than derived: grobid_column_layout.yml lists the expected columns by name and the slot behaviour, which is filled for those three, unfilled for header and reference_segmenter (GROBID leaves the slot at 0 and appends the tag after it), and absent for fulltext and segmentation (printVector has the block commented out). Recording names rather than a count means an inserted or reordered feature fails too, and a model with no entry fails rather than being assumed correct. figure and table share fulltext's entry, since all three use FullTextDataGenerator and GROBID likewise generates their data with FeaturesVectorFulltext; name_header and name_citation share one entry. segmentation's whole_line_text has no GROBID counterpart and sits at the index GROBID uses for the label. It is not a mistake - delft models read it as a text feature - so it stays, behind --include-extra-columns, positioned to keep its index unchanged from what inference emits. The default output matches GROBID and can be mixed with GROBID's corpus. Both feature sources are validated: the emitted width is checked against the recorded layout whether features came from the data generator or from a raw file, and a mismatch is an error rather than a silently wrong corpus. The offline test asserts per model that the generator's feature_names still match the recorded columns and that the emitted line count is what GROBID's corpus has. check_grobid_column_layout re-checks the recorded layout against GROBID's published corpora and is run by hand, since it downloads them; all eight available corpora agree.
The reference corpora were whatever release happened to be at hand per model: 0.9.0 for header, reference-segmenter and citation, 0.8.2 for fulltext, affiliation-address and the two name models, and 0.5.6 for segmentation. The grobid-0.9.0 release has an asset for every one of them, so they all point there now, and the recorded layout is confirmed against a single GROBID version rather than three. Every model agrees, downloaded live rather than by hash against a local cache - which also establishes that the recorded URLs resolve. Notably segmentation is 34 columns at 0.9.0, the same as the 0.5.6 corpus it had been measured against. figure and table still point at the fulltext corpus rather than their own, and their own is still why: at 0.9.0 the figure corpus is 26 feature columns throughout and the table corpus mixes 26 with 27, the 27-column lines being the ones carrying is_superscript. Unchanged from 0.8.2. Models sharing a layout now share one check, so the run downloads the 7MB fulltext corpus once rather than three times.
The field is a labelled .train corpus, which the name did not say. The rest of the entry describes the data generator - the inference vector - and derives the training layout from label_slot, so the one training-specific field is worth naming as such. Notes in the file header that the inference vector has no published counterpart there.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
part of https://github.com/eLifePathways/ScienceBeam2.0/issues/144
generate_delft_data appended the label unconditionally, so for the three models whose GROBID corpus writes the label into the feature vector's label slot - citation, name and affiliation_address - it emitted the slot placeholder as well as the label, one column too wide. get_clean_features_mask in trainer-delft drops documents whose width differs from the first seen and logs one line, so a run given both corpora silently kept one and discarded the other.
The layout is per-model empirical fact, so it is recorded as data rather than derived: grobid_column_layout.yml lists the expected columns by name and the slot behaviour, which is filled for those three, unfilled for header and reference_segmenter (GROBID leaves the slot at 0 and appends the tag after it), and absent for fulltext and segmentation (printVector has the block commented out). Recording names rather than a count means an inserted or reordered feature fails too, and a model with no entry fails rather than being assumed correct. figure and table share fulltext's entry, since all three use FullTextDataGenerator and GROBID likewise generates their data with FeaturesVectorFulltext; name_header and name_citation share one entry.
segmentation's whole_line_text has no GROBID counterpart and sits at the index GROBID uses for the label. It is not a mistake - delft models read it as a text feature - so it stays, behind --include-extra-columns, positioned to keep its index unchanged from what inference emits. The default output matches GROBID and can be mixed with GROBID's corpus.
Both feature sources are validated: the emitted width is checked against the recorded layout whether features came from the data generator or from a raw file, and a mismatch is an error rather than a silently wrong corpus.
The offline test asserts per model that the generator's feature_names still match the recorded columns and that the emitted line count is what GROBID's corpus has. check_grobid_column_layout re-checks the recorded layout against GROBID's published corpora and is run by hand, since it downloads them; all eight available corpora agree.