Write an hourly running jobs status into a separate YAML#886
Open
Write an hourly running jobs status into a separate YAML#886
Conversation
Append per-hour snapshot only when changed; otherwise emit a one-line heartbeat in ARC.log.
There was a problem hiding this comment.
Pull request overview
This PR reduces repetitive “currently running jobs” log spam by moving periodic status reporting out of ARC.log and into a dedicated YAML file, while logging only short heartbeat lines when the status hasn’t changed.
Changes:
- Added
Scheduler.report_running_jobs_snapshot()and wired it into the hourly scheduler reporting path. - Introduced
arc.common.append_yaml_document()to append multi-document YAML snapshots. - Added unit tests for both the scheduler snapshot reporting and the YAML append helper.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
arc/scheduler.py |
Writes periodic running-jobs/active-pipes snapshots to running_jobs.yml and logs a heartbeat when unchanged. |
arc/scheduler_test.py |
Adds a test ensuring snapshots append only when the status changes. |
arc/common.py |
Adds append_yaml_document() helper for multi-document YAML snapshot files. |
arc/common_test.py |
Adds tests for append_yaml_document(). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+858
to
+865
| payload = {'running_jobs': dict(self.running_jobs), | ||
| 'active_pipes': list(self.active_pipes.keys())} | ||
| n_species = len(self.running_jobs) | ||
| n_jobs = sum(len(v) for v in self.running_jobs.values()) | ||
| n_pipes = len(self.active_pipes) | ||
| summary = f'{n_species} species / {n_jobs} jobs / {n_pipes} active pipes' | ||
| if payload == self._last_status_payload: | ||
| logger.info(f'Status unchanged: {summary}.') |
Comment on lines
+851
to
+870
| def report_running_jobs_snapshot(self) -> None: | ||
| """ | ||
| Append a snapshot of the currently running jobs and active pipes to | ||
| ``<project>/running_jobs.yml`` (each snapshot is a separate YAML | ||
| document prefixed with ``---``). If the payload is identical to the | ||
| previous snapshot, only a one-line heartbeat is logged to ARC.log. | ||
| """ | ||
| payload = {'running_jobs': dict(self.running_jobs), | ||
| 'active_pipes': list(self.active_pipes.keys())} | ||
| n_species = len(self.running_jobs) | ||
| n_jobs = sum(len(v) for v in self.running_jobs.values()) | ||
| n_pipes = len(self.active_pipes) | ||
| summary = f'{n_species} species / {n_jobs} jobs / {n_pipes} active pipes' | ||
| if payload == self._last_status_payload: | ||
| logger.info(f'Status unchanged: {summary}.') | ||
| return | ||
| snapshot = {'timestamp': datetime.datetime.now().isoformat(timespec='seconds'), | ||
| **payload} | ||
| append_yaml_document(self.running_jobs_snapshot_path, snapshot) | ||
| logger.info(f'Status changed: {summary}; snapshot appended to running_jobs.yml.') |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #886 +/- ##
==========================================
- Coverage 60.40% 60.37% -0.04%
==========================================
Files 103 103
Lines 31156 31177 +21
Branches 8121 8122 +1
==========================================
+ Hits 18821 18824 +3
- Misses 9989 10005 +16
- Partials 2346 2348 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
don't spam the ARC.log if the report repeats it self and ARC is idle. outsource the running jobs report into a separate YAML, only update it when needed, and drop a heartbeat line into ARC's log file.
addresses #694