Repository navigation
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #655
What changed and why
TorchImageSegmentationModel.eval()now sizes the confusion matrix with the largest class index plus one instead of the number of classes, the same expressionexport()already uses. Withtranslation_direction="model_to_dataset"the evaluation ontology is the dataset ontology, and for non-contiguous indices such as RELLIS-3D (20 classes, indices up to 34) the old size dropped GT pixels with high indices and then raisedIndexErrorwhen building the metrics table. Unused indices become NaN rows, which the macro average already skips. For contiguous ontologies the value does not change.tests/models/test_torch_segmentation.py: one 4x4 image with dataset indices 0, 3 and 4 and a model that returns the GT, run in both directions and expecting IoU 1.0 for every class and the macro average. Liketests/utils/test_torch.py, it is skipped when real torch is not installed, so the current CI job skips it.This PR only touches the image segmentation path. I'll extend it to the LiDAR, TF and
eval_predspaths in this PR or in a follow-up, whichever you prefer.How I tested
tutorial_image_segmentation_sample.ipynb, evaluated in both directions: per-class IoU identical, mIoU 0.4617 and micro IoU 0.7737 in both. On mastermodel_to_datasetfails withIndexError: index 23 is out of bounds for axis 0 with size 20.tutorial_image_segmentation_sample.ipynbend to end: no errors.model_to_dataset(one-image Cityscapes-style folder, tiny TorchScript model): the evaluation now completes.Screenshots
Before:
After: