Skip to content

Fix the outputs to stdout/stderr - #321

Merged
max-models merged 6 commits into
develfrom
270-too-much-logs-in-err-files-on-the-clusters
Aug 4, 2026
Merged

Fix the outputs to stdout/stderr#321
max-models merged 6 commits into
develfrom
270-too-much-logs-in-err-files-on-the-clusters

Conversation

@max-models

@max-models max-models commented Aug 3, 2026

Copy link
Copy Markdown
Member

DEBUG/INFO go to stdout, WARNING and above to stderr; records that pass the logger level are also written to the log file.

This means the levels for the different loggers are no longer the same, and these lines were therefore removed:

for handler in logger.handlers:
        handler.setLevel(level)

@max-models max-models linked an issue Aug 3, 2026 that may be closed by this pull request
@max-models
max-models marked this pull request as draft August 3, 2026 14:06
@max-models
max-models requested a review from Copilot August 3, 2026 14:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts runtime output streams to better separate “normal” output from errors in batch/cluster environments by moving tqdm progress bars (and low-severity logs) away from stderr.

Changes:

  • Added a struphy.utils.progress shim that configures tqdm to write to sys.stdout by default.
  • Updated multiple modules to import tqdm from struphy.utils.progress instead of directly from tqdm.
  • Updated struphy logger configuration to route DEBUG/INFO to stdout and WARNING+ to stderr, and to log more verbosely to struphy.log.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/struphy/utils/progress.py Introduces a stdout-targeting tqdm wrapper for consistent progress bar output.
src/struphy/simulation/sim.py Switches to the new tqdm wrapper and reformats progress bar construction.
src/struphy/post_processing/post_processing_tools.py Uses the new stdout tqdm import path.
src/struphy/post_processing/orbits/orbits_tools.py Uses the new stdout tqdm import path.
src/struphy/diagnostics/diagn_tools.py Uses the new stdout tqdm import path.
src/struphy/init.py Adds stdout/stderr split logging handlers and adjusts file handler verbosity.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/struphy/simulation/sim.py
Comment thread src/struphy/__init__.py Outdated
Comment thread src/struphy/__init__.py
@max-models

max-models commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

There are still a few messages that appear in the .err files, such as

Starting run for model Maxwell on 1 ranks ...
WARNING: Class "BasisProjectionOperators" called with degree=(1, 1, 1) (interpolation of piece-wise constants should be avoided).
Struphy run finished.

edit: moved these to logger.info in bd67126

@max-models
max-models marked this pull request as ready for review August 4, 2026 09:41
@max-models
max-models requested a review from spossann August 4, 2026 11:46
@max-models
max-models merged commit 1914636 into devel Aug 4, 2026
27 checks passed
@max-models
max-models deleted the 270-too-much-logs-in-err-files-on-the-clusters branch August 4, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Too much logs in .err files on the clusters

3 participants