Record reports after the upload, not before - #347
Merged
Conversation
all-reports.py wrote a report's row into [history] as soon as it had written the html to the workspace, and mailed everyone about it in the same run. The files only reach the web server at the end of the stage, after all-plots.py and two dozen report.py invocations, so anything failing in between left a report that the database says exists, that nobody can read, and that the rerun skips because it is recorded. The mail linking to it had already gone out. Before the table existed the record was the published 00_history.html, which only existed if the upload had happened, so a lost upload healed itself on the next run; moving the record into the database made it permanent. The reports and the queued emails now go into pending-reports.json, and publish-reports.py records and sends them once the files are up. The stage uploads history/ right after all-plots.py and keeps a second transfer for overview*.html at the end. | scenario | now | | ----------------------------------- | ---------------------------- | | generate, publish, rerun | skipped | | stage fails before the upload | regenerated next run | | uploaded but not recorded | copied back from the index | | branch named twice in one run | one report | Assisted-by: Claude Opus 5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
all-reports.py wrote a report's row into [history] as soon as it had written the html to the workspace, and mailed everyone about it in the same run. The files only reach the web server at the end of the stage, after all-plots.py and two dozen report.py invocations, so anything failing in between left a report that the database says exists, that nobody can read, and that the rerun skips because it is recorded. The mail linking to it had already gone out.
Before the table existed the record was the published 00_history.html, which only existed if the upload had happened, so a lost upload healed itself on the next run; moving the record into the database made it permanent.
The reports and the queued emails now go into pending-reports.json, and publish-reports.py records and sends them once the files are up. The stage uploads history/ right after all-plots.py and keeps a second transfer for overview*.html at the end.
Assisted-by: Claude Opus 5