Exposing the stream in the logging module

[jauy123]

The logging module should have an option to only stream to standard out/error rather than forcing a log file to disk.

In many HPC environments, the job controller capture both standard out and error to a text file. For many people, this is enough to debug their scripts if logging is redirected to those streams instead. Thus, logging to a file should be optional.

Describe the solution you'd like

Replace the logfile parameter in MDAnalysis.start_logging(logfile='MDAnalysis.log', version='2.1.0') with a stream parameter that acts like stream in logging.basicConfig

import logging
import sys

logging.basicConfig(
    level=logging.INFO,
    stream=sys.stdout,  
    format="%(asctime)s - %(levelname)s - %(message)s"
)

logging.info("This goes to stdout")
logging.error("This also goes to stdout")

Describe alternatives you've considered

You could wrap something elaborate with file steam redirection like in bash, but that is too complicated compared to this fix.

Additional context

[jauy123]

I don't mind working on this if the core devs are ok with this change.

[orbeckst]

How would you make any stream work in the current code? Would it make the code much more complicated?

FYI: start_logging() was only ever meant as a quick way to set up a reasonable logger. At the end of the day, anyone who wants to use logging, can set up their own, as long as they know that the logger is using "MDAnalysis" as its name.

[jauy123]

I don't think so. I need to sit down and code it, but it look fairly straight forward to modify it to aaccept multiple streams.

From package/MDAnalysis/lib/log.py

def start_logging(logfile="MDAnalysis.log", version=version.__version__):
    """Start logging of messages to file and console.

    The default logfile is named `MDAnalysis.log` and messages are
    logged with the tag *MDAnalysis*.
    """
    create("MDAnalysis", logfile=logfile)
    logging.getLogger("MDAnalysis").info(
        "MDAnalysis %s STARTED logging to %r", version, logfile
    )

You would just need to modify create()

def create(logger_name="MDAnalysis", logfile="MDAnalysis.log"):
    """Create a top level logger.

    - The file logger logs everything (including DEBUG).
    - The console logger only logs INFO and above.

    Logging to a file and the console as described under `logging to
    multiple destinations`_.

    The top level logger of MDAnalysis is named *MDAnalysis*.  Note
    that we are configuring this logger with console output. If a root
    logger also does this then we will get two output lines to the
    console.

    .. _logging to multiple destinations:
       http://docs.python.org/library/logging.html?#logging-to-multiple-destinations
    """

    logger = logging.getLogger(logger_name)

    logger.setLevel(logging.DEBUG)

    # handler that writes to logfile
    logfile_handler = logging.FileHandler(logfile)
    logfile_formatter = logging.Formatter(
        "%(asctime)s %(name)-12s %(levelname)-8s %(message)s"
    )
    logfile_handler.setFormatter(logfile_formatter)
    logger.addHandler(logfile_handler)

    # define a Handler which writes INFO messages or higher to the sys.stderr
    console_handler = logging.StreamHandler()
    console_handler.setLevel(logging.INFO)
    # set a format which is simpler for console use
    formatter = logging.Formatter("%(name)-12s: %(levelname)-8s %(message)s")
    console_handler.setFormatter(formatter)
    logger.addHandler(console_handler)

    return logger

You would just need add logic in create() that dynamically create different handlers depending on the input argument(s). Like if it sys.out create a handler that write out to sys.out

[jauy123]

Would it be possible to get some feedback on my PR?